Keywords:

In optimization problems involving uncertainty, probabilistic constraints are an important tool for defining safety of decisions. In Energy management, many optimization problems have some underlying uncertainty. In particular this is the case of unit commitment problems. In this Thesis, we will investigate probabilistic constraints from a theoretical, algorithmic and applicative point of view. We provide new insights on differentiability of probabilistic constraints and on convexity results of feasible sets. New variants of bundle methods, both of proximal and level type, specially tailored for convex optimization under probabilistic constraints, are given and convergence shown. Both methods explicitly deal with evaluation errors in both the gradient and value of the probabilistic constraint. We also look at two applications from energy management: cascaded reservoir management with uncertainty on inflows and unit commitment with uncertainty on customer load. In both applications uncertainty is dealt with through the use of probabilistic constraints. The presented numerical results seem to indicate the feasibility of solving an optimization problem with a joint probabilistic constraint on a system having up to 200 constraints. This is roughly the order of magnitude needed in the applications.

The differentiability results involve probabilistic constraints on uncertain linear and nonlinear inequality systems. In the latter case a convexity structure in the underlying uncertainty vector is required. The uncertainty vector is assumed to have a multivariate Gaussian or Student law. The provided gradient formulae allow for efficient numerical sampling schemes. For probabilistic constraints that can be rewritten through the use of Copulae, we provide new insights on convexity of the feasible set. These results require a generalized concavity structure of the Copulae, the marginal distribution functions of the underlying random vector and of the underlying inequality system. These generalized concavity properties may hold only on specific sets. i 4.1 Numerical Instance Data . . . . .

In a way the basis for this thesis was laid in 2005 when together with some colleagues we were quantifying the impact of uncertainty on weekly unit-commitment. A fruitful collaboration with Pierre Thomas and Riadh Zorgati led to the birth of a software package called LEA ("Laboratoire des Expérimentations des Aléas") in the same year. From a technical perspective this software package allowed the user to execute a simple multiscenario approach around a deterministic optimization kernel. The innovating part was set up with Bernard Beauzamy. Instead of drawing scenarios as iid samples from some law (or black box software), the idea was to build artificial englobing scenarios allowing for bounding of output with only little computational effort. These experiments led, only much later, to an industrial software package. In any case this is how together with Riadh Zorgati, I got interested in dealing with uncertainty in an appropriate way. We had many discussions on Scenario generation, Robust Optimization etc These discussions were also greatly nourished by discussions with Michel Minoux. Around the same time, we had the chance to meet René Henrion and attend a seminar he gave. Chance constrained programming looked like a very powerful and intuitive way to deal with uncertainty. With good hopes and enthusiasm we set out to look where uncertainty in Energy Management plays a key role and should be dealt with appropriately. Once these structures identified we started experimenting with safe tractable approximations of chance constraints, scenario type of approaches and also Robust Optimization. Somewhere in 2009 we had to chance to work closely together with René Henrion and look to what extent Chance Constrained programming could be applied in Energy Management. This led me to be greatly intrigued by this field, and ultimately, gave me a desire to deeply understand this domain. Thanks to EDF I have had the opportunity to do this Thesis in part time. I hope to have contributed to the domain and continue to do so in the future. I do firmly believe that Chance Constrained Programming will be important in Energy Management in the near future.

Thanking people after 10 years of work at EDF is almost a dangerous undertaking since we are bound to forget someone with whom we have had interesting discussions. Nonetheless I will make this attempt at my own risks.

First of all I would like to thank Riadh Zorgati, who is a good friend and colleague. We share many views and in particular that (fundamental) research should have its place in an(y) industrial company. I would also like to thank Grace Doukopoulos, Olivier Feron, Nadia Oudjane, Cyrille Strugarek and Xavier Warin for the many interesting discussions (and years of joint work). Thanks to Thomas Triboulet, Laurent Mulot and their pragmatic viewpoint of what at EDF we commonly call "le métier" it is possible to obtain a partial view of the underlying structures of the optimization problems at hand. This allows for important progress in identifying the key features of Energy Management.

Clearly there is not only Chance Constrained Programming in the world, but also Aikido. Thanks go out to my friend Damien Jacomy, senseis Christian Mouza, Nicolas Paillat and François Pichereau. Aikido provides many physical (and intellectual) challenges in the (eternal) quest for perfected techniques. In a way it is pretty much like Mathematics, where elegancy is also looked for.

I also naturally would like to thank René Henrion, Michel Minoux, Claudia Sagastizábal, Jérôme Malick, Antonio Frangioni, Welington de Oliveira for the joint work and/or important discussions. I am also grateful that Claude Lemaréchal and Werner Römisch have accepted to referee this Thesis.

Finally, I would like to thank my wife Séverine (and children Elsa, Charlotte and Tom) for all their love and support. Since this is a work on "robust" optimization, I should maybe thank all my future children as well. One never knows! Les contraintes en probabilité constituent un modèle pertinent pour gérer les incertitudes dans les problèmes de décision. En management d'énergie de nombreux problèmes d'optimisation ont des incertitudes sous-jacentes. En particulier c'est le cas des problèmes de gestion de la production au court-terme. Dans cette Thèse, nous investiguons les contraintes probabilistes sous l'angle théorique, algorithmique et applicative. Nous donnons quelques nouveaux résultats de différentiabilité des contraintes en probabilité et de convexité des ensembles admissibles. Des nouvelles variantes des méthodes de faisceaux " proximales " et " de niveaux " sont spécialement mises au point pour traiter des problèmes d'optimisation convexe sous contrainte en probabilité. Ces algorithmes gèrent en particulier, les erreurs d'évaluation de la contrainte en probabilité, ainsi que son gradient. La convergence vers une solution du problème est montrée. Enfin, nous examinons deux applications : l'optimisation d'une vallée hydraulique sous incertitude sur les apports et l'optimisation d'un planning de production sous incertitude sur la demande. Dans les deux cas nous utilisons une contrainte en probabilité pour gérer les incertitudes. Les résultats numériques présentés semblent montrer la faisabilité de résoudre des problèmes d'optimisation avec une contrainte en probabilité jointe portant sur un système de environ 200 contraintes. Il s'agit de l'ordre de grandeur nécessaire pour les applications.

Les nouveaux résultats de différentiabilité concernent à la fois des contraintes en probabilité portant sur des systèmes linéaires et non-linéaires. Dans le deuxième cas, la convexité dans l'argument représentant le vecteur incertain est requise. Ce vecteur est supposé suivre une loi Gaussienne ou Student multi-variée. Les formules de gradient permettent l'application directe d'un schéma d'évaluation numérique efficient. Pour les contraintes en probabilité qui peuvent se réécrire à l'aide d'une Copule, nous donnons de nouveau résultats de convexité pour l'ensemble admissibles. Ces résultats requirent la concavité généralisée de la Copule, les distributions marginales sous-jacents et du système d'incertitude. Il est suffisant que ces propriétés de concavité généralisée tiennent sur un ensemble spécifique.

Chapter 1 Introduction 1.1 Introduction

Energy management optimization problems deal with decision making problems with time spans ranging from intra-daily to several decades. Problems are as varied as:

1. computing day-ahead, the production schedule in a hydro-thermal generation system 2. obtaining a strategy of use of water and opportunity costs for the coming year 3. computing the maintenance schedule of thermal plants

obtaining investment decisions

It is clear that uncertainty intervenes in each of these problems. For instance in the first problem, the schedule is determined before observing uncertainty. Random deviations of load, renewable generation and (partial) outages then lead to offer-demand mismatches. These have to be dealt with appropriately. For instance, by setting spinning reserve requirements. Uncertainty on inflows also impacts the production schedule, but may render it infeasible. Indeed, for a fixed production schedule, excessive inflows may bring forth violations of the upper reservoir bound. Similarly, a lack of inflows may bring forth violations of the lower bound. As a matter of fact the earlier mentioned uncertainty factors also make the production schedule infeasible since the offer-demand balance is no longer satisfied. In the second and third problem, the decision taken in a specific time stage of the problem is assumed to have knowledge of all uncertainty that has preceded (dynamic decision making). In this aspect it differs from the first and fourth problem wherein the decisions are taken prior to observing uncertainty (static decision making). Dynamic decision making problems are frequently formulated as (Stochastic) Dynamic Programming problems or as optimization problems on a scenario tree (e.g., [START_REF] Kall | Stochastic Linear Programming: Models, Theory and Computation[END_REF][START_REF] Heitsch | Scenario reduction algorithms in stochastic programming[END_REF][START_REF] Gröwe-Kuska | Scenario reduction and scenario tree construction for power management problems[END_REF]). This essentially renders the problem deterministic (but large scale). Formulating constraints involving both the decision vector and a random variable is then nearly as easy as in the deterministic setting. This becomes more complicated when we assume that decisions are taken prior to observing uncertainty. For example, what is meant with x being feasible in a linear inequality system Ax ≤ b when both A and b are random variables ? Yet, such a question has to be answered if we are to solve problem 1 highlighted above. Two fields of optimization provide a framework for giving a sensible meaning to such a "random inequality system". These are Robust Optimization ( [START_REF] Ben-Tal | Robust Optimization[END_REF]) and Probabilistic Programming ( [START_REF] Prékopa | Stochastic Programming[END_REF]). We shall (nearly) exclusively focus on the latter approach. As an application we will investigate to what extent a production schedule can be computed while integrating uncertainty on both inflows and load.

Probabilistic constraints are encountered in many engineering problems involving uncertain data. We can find applications in water management, telecommunications, electricity network expansion, mineral blending, chemical engineering etc. (e.g., [START_REF] Henrion | Optimization of a continuous distillation process under random inflow rate[END_REF][START_REF] Morgan | Aquifer remediation design under uncertainty using a new chance constraint programming technique[END_REF][START_REF] Van De Panne | Minimum-cost cattle feed under probabilistic protein constraints[END_REF][START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Prékopa | On optimal regulation of a storage level with application to the water level regulation of a lake[END_REF][START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF][START_REF] Van Ackooij | Decomposition approaches for block-structured chance-constrained programs with application to hydro-thermal unit-commitment[END_REF]) For an overview of theory, numerics and applications of chance constraints we refer to [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF][START_REF] Prékopa | Stochastic Programming[END_REF][START_REF] Prékopa | Probabilistic programming[END_REF] and references therein.

Initiated by Charnes and Cooper [START_REF] Charnes | Chance-constrained programming[END_REF] and pioneered by Prékopa (e.g., by his celebrated log-concavity-Theorem [START_REF] Prékopa | On logarithmic concave measures and functions[END_REF]) the analysis of probabilistic constraints has attracted much attention in recent years with a focus on algorithmic approaches. Without providing an exhaustive list, we refer here to models like robust optimization [START_REF] Ben-Tal | Robust solutions of linear programming problems contaminated with uncertain data[END_REF], penalty approach [START_REF] Ermoliev | Stochastic optimization of insurance portfolios for managing exposure to catastrophic risk[END_REF], p-efficient points [START_REF] Dentcheva | Regularization methods for optimization problems with probabilistic constraints[END_REF][START_REF] Dentcheva | Concavity and efficient points for discrete distributions in stochastic programming[END_REF], scenario approximation [START_REF] Calafiore | The scenario approach to robust control design[END_REF], sample average approximation [START_REF] Pagnoncelli | Sample average approximation method for chance constrained programming: Theory and applications[END_REF] or convex approximation [START_REF] Nemirovski | Convex approximations of chance constrained programs[END_REF].

In this Thesis we will investigate probabilistic constraints following the traditional (as initiated by Prékopa) approach. We will investigate structural properties such as differentiability and convexity. We will also provide two specially designed Algorithms for dealing with (convex) optimization problems under probabilistic constraints. Globally the work can be seen as a "quest for tractability". Unlike commonly stated, we firmly believe that probabilistic programming is perfectly tractable in most/many situations of interest. To illustrate some advances it is of interest to recall that J. Mayer (in [START_REF] Mayer | On the Numerical solution of jointly chance constrained problems[END_REF]) stated that the largest probabilistic program solved before 2000 and published had a random vector in dimension 7. In the same work a problem in dimension 30 is solved, requiring up to 800 minutes of CPU time. In this Thesis, we illustrate one of the algorithms on an instance having a random vector in dimension 168. Computation time is "only" around 350 minutes. It is likely that such large scale probabilistically constrained problems only allow for feasible solutions because of a strong underlying correlation structure. It might also be required that some constraints are inactive in a to be defined probabilistic sense.

This document is organized around four chapters. Chapter 2 gives an introduction to the known theory and structural properties of probabilistic constraints. These properties will be used frequently in the document. This chapter also helps as a guide to show how newly derived results fit in. Extensions of theory and algorithms can be found in Chapters 3 and 4 respectively. Chapter 5 is devoted to applications in energy management. In particular, we investigate cascaded reservoir management, robust against uncertainty on inflows and unit-commitment, robust against uncertainty on both inflows and load.

Main Contribution of this Work

In section 3.1 we investigate to what extent a characterization of the feasible set of a chance constraint can be derived involving simpler chance constraints. An example shows the interest of the suggested approach. We then provide several sufficient (but not necessary) conditions with negative results. These results hold in particular for the specially structured probabilistic constraints appearing in the applications of Chapter 5.

To the best of our knowledge these ideas have not been investigated yet.

Many algorithms of non-linear programming require the knowledge of (sub-)gradients of the constraints in order to numerically solve a problem. If these constraints are probabilistic constraints, we need to be able to evaluate such a gradient as efficiently as possible. Very general differentiability statements exist ([237]) that represent the gradient of a probabilistic constraint as the sum of an integral over a volume and integral over a surface. Numerical use of these formulae appears to be quite difficult. Many other approaches investigate special cases. These results are of interest because they link a component of the partial derivative of a probabilistic constraint and the evaluation of another probabilistic constraint. Frequently the latter constraint is of similar nature as the one we took the derivative from. This means that if one is able to evaluate a probabilistic constraint (clearly a prerequisite), one can also compute its derivative. These differentiability results exist for the derivative of non-degenerate Gaussian distribution functions [START_REF] Prékopa | On probabilistic constrained programming[END_REF], multi-variate Gamma distribution functions [START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF] and multi-variate Dirichlet distributions [START_REF] Szántai | Numerical evaluation of probabilities concerning multi-dimensional probability distributions[END_REF]. The results of section 3.2 (published in [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF][START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF]) provide efficient gradient (and Hessian) formulae of the above type for specially structured probabilistic constraint appearing in Energy Management Applications. These results extend [START_REF] Prékopa | On probabilistic constrained programming[END_REF] to bilateral constraints involving a non-degenerate Gaussian random variable. We also provide a result ( [START_REF] Van Ackooij | On joint probabilistic constriants with Gaussian Coefficient Matrix[END_REF]) involving the computation of the gradient of a probabilistic constraint with "Gaussian Matrix Uncertainty". The results appearing in section 3.2 were derived with R. Henrion, A. Möller and R. Zorgati.

Differentiability of probabilistic constraints is further examined in Section 3.3 where efficient gradient formulae are derived for probabilistic constraints involving a Gaussian (or Student) random variable linked to the decision vector through a rather general mapping. Special cases involve log-normal and chi-squared random variables. These results are taken from a draft submitted with R. Henrion ([242]).

Convexity of the feasible set is an important feature in probabilistic programming. Occasionally such convexity depends on the requested safety level. This is known as eventual convexity and clearly suffices in practical applications. Early results on eventual convexity were derived for specific individual probabilistic constraints [START_REF] Kataoka | A stochastic programming model[END_REF]. Recently [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] provided new insights in eventual convexity of probabilistically constrained feasible sets. The involved multi-variate random variable has components "correlated" through a specially structured Copulae. The results ( [START_REF] Van Ackooij | Eventual convexity of chance constrained feasible sets[END_REF]) in Section 3.4 provide an extension of the latter results on eventual convexity, by allowing for more Copulae and providing lower thresholds. Such lower thresholds allow us to prove convexity for probabilistically constrained feasible set with lower safety-levels.

In sections 4.1 and 4.2 we provide two Bundle-Algorithms for dealing with convex constrained optimization problems. The constraint is an appropriate transform of the probabilistic constraint making it convex. We thus (implicitly) assume that the elements appearing in the constraint satisfy appropriate hypothesis for this to hold. We also assume that the value of the constraint and a (sub-)gradient can be computed. It is not reasonable to assume that this can be done up to arbitrary precision. The suggested algorithms are designed in such a way that this imprecision is dealt with. The specific case of probabilistic constraints led to the definition of an "upper"-oracle and considerably complicates convergence analysis. The results in Section 4.1 are taken from a joint work [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF] with C. Sagastizábal, whereas the results of Section 4.2 are taken from a joint work [START_REF] Van Ackooij | Level bundle methods for constrained convex optimization with various oracles[END_REF] with W. de Oliveira.

Chapter 5 contains applications involving some of the largest problems featuring probabilistic constraints seen in the literature so far ( [START_REF] Mayer | On the Numerical solution of jointly chance constrained problems[END_REF]). Especially the global unitcommitment problem of Section 5.2 requires iteratively solving many probabilistic programs. It is intuitively felt that setting up a robust unit-commitment problem with a probabilistic constraint on the offer-demand equilibrium leads to a tractable model. Several decomposition approaches are suggested and experimented with on a typical example. Making cascaded reservoir management robust against uncertainty on inflows is investigated in Section 5.1. Those results are partially taken from [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF], which is a joint work with R. Henrion, A. Möller and R. Zorgati. Most papers on cascaded reservoir management with chance constraints consider the easier case of individual chance constraints (e.g., [START_REF] Loucks | Water Resource Systems Planning and Analysis[END_REF][START_REF] Duranyildiz | A chance-constrained LP model for short term reservoir operation optimization[END_REF][START_REF] Edirisinghe | Capacity planning model for a multipurpose water reservoir with target-priority operation[END_REF][START_REF] Loiaciga | On the use of chance constraints in reservoir design and operation modeling[END_REF][START_REF] Morgan | Aquifer remediation design under uncertainty using a new chance constraint programming technique[END_REF]). Although we should consider joint chance constraints from a robustness perspective, this only rarely done [START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF]. Also most of these papers only consider very simple models without serially linked reservoirs, flow delay, time series modelling of statistical data and have random vectors in small dimension.

Chapter 2 Preliminaries

Chance constrained programming is the branch of Stochastic Programming dealing with constraints of the form P[g(x, ξ) ≥ 0] ≥ p, (2.0.1) where x ∈ R n is the decision vector, ξ ∈ R m a random variable and g : R n × R m → R k a constraint mapping. The level p ∈ (0, 1) is user given and defines the preference for safety of the decision x. The constraint (2.0.1) means that we wish to take a decision x that satisfies the k-dimensional random inequality system g(x, ξ) ≥ 0 with high enough probability. Such chance constraints naturally arise in engineering problems when in usual constraints h(x) ≥ 0, h : R n → R k one identifies unknown parameters or random vectors. The typical situation is one wherein h(x) := g(x, E (ξ)), e.g., E (ξ) can be identified with a forecast of ξ. Constraints of the form (2.0.1) can then be formed upon realizing that the forecast was not accurate enough or additional engineering securities had to be built into h in order to account for variations in ξ.

Now two situations may arise, one wherein we consider that the distribution of ξ is unknown, insufficiently characterized or simply we forbid ourselves to use it. In such a situation constraints of the form (2.0.1) can be replaced by constraints of the form g(x, z) ≥ 0, ∀z ∈ Θ, (2.0.2)

where Θ is an appropriately chosen, explicitly defined set called uncertainty set. This is a version of "Robust Optimization". We refer to the book [START_REF] Ben-Tal | Robust Optimization[END_REF] and references therein for a full treatment of this topic. In the second situation we assume that ξ is well-characterized and that knowledge of the distribution is available. Then constraints of the form (2.0.1) can be investigated and this is the hypothesis that we make in this Thesis. Such a situation naturally arises when ξ has been investigated by Statisticiens or Econometrists. Such is frequently the situation when in decision making problems sensitivity or risk analyses are conducted. Often such analyses are based on applying an optimization tool on a set of generated scenarios for ξ. The scenario generator is frequently based on a time-series decomposition of ξ, at least when a temporal effect is clearly identified or otherwise on a finely characterized distribution of ξ.

Two conceptually different versions of (2.0.1) exist and are referred to as Individual Chance Constraints (ICC) or Joint Chance Constraints (JCC). The equation appearing in (2.0.1) is a version of a Joint Chance Constraint. A deduced set of Individual Chance Constraints would be P[g i (x, ξ) ≥ 0] ≥ p i , i = 1, ..., k, (2.0.3)

where g i refers to the i-th component of the mapping g and p i ≥ p are arbitrary choices. The situation of (2.0.3) refers to a situation wherein we wish to satisfy each individual equation in the random k-dimensional inequality system g(x, ξ) ≥ 0 with high enough probability, but we make no request on the system as a whole. This loss of robustness is compensated, in many situations, by an easier numerical and theoretical treatment.

From an engineering perspective one can also argue that (2.0.3) offers already increased robustness with respect to solving deterministic problems wherein ξ was replaced with a forecast, e.g., E (ξ). One readily observes that a feasible solution for (2.0.1) is feasible for (2.0.3) and that the inverse holds whenever k i=1 p i ≥ p + (k -1). Despite this inner and outer approximation of the feasible set of (2.0.1), we believe that it is worthwhile to consider the JCC of (2.0.1). This is exactly what we will look at in this Thesis.

An important special case of (2.0.1) is one wherein x and ξ are not coupled through the mapping g but appear separate. The mapping g is then of the form g(x, ξ) = h(x)g(ξ). The constraint (2.0.1) then becomes:

P[h(x) -g(ξ) ≥ 0] ≥ p (2.0.4)
and is referred to as a separable (joint) chance constraint.

Chance constraints were first formulated as ICCs by [START_REF] Charnes | Cost horizons and certainty equivalents: an approach to stochastic programming of heating oil[END_REF] and further developed in [START_REF] Charnes | Chance-constrained programming[END_REF][START_REF] Charnes | Chance constraints and normal deviates[END_REF][START_REF] Charnes | Deterministic equivalents for optimizing and satisficing under chance constraints[END_REF]. Joint Chance Constraints were first formulated by [START_REF] Miller | Chance constrained programming with joint constraints[END_REF]. The field has then received major contributions by Prékopa in the early 70s [START_REF] Prékopa | On probabilistic constrained programming[END_REF][START_REF] Prékopa | On logarithmic concave measures and functions[END_REF] with in particular very general convexity results for the feasible set of (2.0.1). Other key contributions are from Szántai (e.g., [START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF][START_REF] Szántai | Numerical evaluation of probabilities concerning multi-dimensional probability distributions[END_REF][START_REF] Szántai | A computer code for solution of probabilistic-constrained stochastic programming problems[END_REF]), Uryasev (e.g., [START_REF] Uryas'ev | Derivatives of probability functions and some applications[END_REF]), Henrion (e.g., [START_REF] Henrion | On the connectedness of probabilistic constraint sets[END_REF][START_REF] Henrion | Structural properties of linear probabilistic constraints[END_REF][START_REF] Henrion | Hölder and lipschitz stability of solution sets in programs with probabilistic constraints[END_REF][START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF]), Römisch (e.g., [START_REF] Henrion | Lipschitz and differentiability properties of quasiconcave and singular normal distribution functions[END_REF][START_REF] Römisch | Stability of solutions for stochastic programs with complete recourse[END_REF]), Dentcheva and Ruszczyński (e.g., [START_REF] Dentcheva | Concavity and efficient points for discrete distributions in stochastic programming[END_REF]).

In this chapter we will sketch an overview of known results on structural properties of probabilistic constraints of the form (2.0.1). It will serve as a guideline to show where our results fit in.

Measurability

Even though one easily writes a constraint of the type (2.0.1) a first question consists of its well-foundedness. In particular, for a fixed x ∈ R n is the set {ω : g(x, ξ(ω)) ≥ 0} measurable so that P[g(x, ξ) ≥ 0] is a meaningful composition ?

This can be answered positively in quite a general setting. For this analysis the dependence on x will be neglected. Let (Ω, F, P) be a probability space. Consider a random vector ξ : Ω → E, where (E, B(E)) is a topological vector space equipped with its Borelsigma algebra B(E). Now consider a mapping g : E → R k . It is then clear that Borel measurability of g suffices for the set {ω ∈ Ω : g(ξ(ω)) ≥ 0} , to be F-measurable. We can thus write

P[g(ξ) ≥ 0],
as this is now a well defined expression. In particular every upper-semi-continuous mapping g is Borel-Measurable.

This obvious statement hides some difficulties. In particular, it is not sufficient for g to be concave, since not all convex sets are Borel measurable. If ξ admits a density with respect to the Lebesgue measure, the above difficulty disappears since convex sets are Lebesgue measurable ( [START_REF] Lang | A note on the measurability of convex sets[END_REF][START_REF] Elstrodt | Maßund Integrationstheorie[END_REF]).

We can thus conclude that as long as ξ ∈ R m appearing in (2.0.1) admits a density and z → g(x, z) is concave (or upper-semi-continuous), the constraint (2.0.1) is well-defined.

The most common forms of inequality systems g(x, ξ) ≥ 0 are those wherein g represents a linear random inequality system of the form Ax ≥ b. Either components of A or b are elements of the vector ξ. Measurability in this case is thus assured.

Closedness of the Feasible Set

A second question of importance is whether the feasible set

M (p) := {x ∈ R n : P[g(x, ξ) ≥ 0] ≥ p} (2.2.1)
is actually closed. This is of-course a necessary property for any optimization problem with a constraint of the type (2.0.1) to actually attain an optimum. This can be shown under fairly general assumptions on the mapping g. In what follows we (trivially) generalize the known results to metric spaces and an arbitrary set of constraints. In this we follow the work [START_REF] Henrion | Optimierungsprobleme mit wahrscheinlichkeitsrestriktionen: Modelle, struktur, numerik[END_REF], but the original results are published in [START_REF] Raik | Qualitative research into the stochastic nonlinear programming problems (russian)[END_REF][START_REF] Raik | The quantile function in stochastic nonlinear programming (russian)[END_REF][START_REF] Raik | On the stochastic programming problem with the probability and quantile functionals (russian)[END_REF]. We can also cite [START_REF] Römisch | Stability analysis for stochastic programs[END_REF] for a result similar to Lemma 2.2.2 below.

Lemma 2.2.1. Let (E, d) be a metric space, with metric d : E × E → R + . Consider an arbitrary mapping h : E → R. The level sets L t = {x ∈ E : h(x) ≥ t} are closed for all t ∈ R if and only if h is upper semi-continuous.

Proof. Fix an arbitrary t and pick any sequence x n ∈ L t , converging to limit x ∈ E. Since E is a metric space, upper semi continuity of h gives h(x) ≥ lim sup y→x h(y) = lim sup n→∞ h(x n ) ≥ t, so x ∈ L t and L t is therefore closed. Conversely let x ∈ E be arbitrary and pick a sequence x n → x such that

t := h(x) = lim sup y→x h(y) = lim n→∞ h(x n ),
Then for arbitrary ε > 0, there exists some N with h(x n ) ≥ tε for all n ≥ N , so x n ∈ L t-ε for those n. From closedness of L t for all t this gives x ∈ L t-ε , therefore h(x) ≥ lim sup y→x h(y)ε.

since ε > 0 is arbitrary, h is upper semi continuous.

Lemma 2.2.2. Let E, F be arbitrary metric spaces, T be an arbitrary index set and let ξ ∈ F be a random variable. Assume furthermore that all members g t : E × F → R of the family of mappings {g t } t∈T are (jointly) upper semi continuous. The mapping ϕ : E → [0, 1] defined as ϕ(x) = P[g t (x, ξ) ≥ 0 ∀t ∈ T ] is then upper semi-continuous. As a consequence, the set M (p) := {x : ϕ(x) ≥ p} is closed for all p ∈ [0, 1].

Proof. We begin by observing that M (p) = ∅ if p > 1 and M (p) = E if p < 0 and that these sets are closed. The asserted upper semi-continuity of the mapping ϕ is therefore equivalent with M (p) being closed as a consequence of Lemma 2.2.1.

Let p ∈ [0, 1] be arbitrary. We will show that M (p) is a closed set. To this end, pick an arbitrary converging sequence x n ∈ M (p) with limit point x, i.e., lim n→∞ x n = x, where convergence is in E. We will show that x ∈ M (p).

To this end, define

H(x) = {z ∈ F : g t (x, z) ≥ 0 ∀t ∈ T } A k = ∪ y :d E (y,x)≤k -1 H(y), k ≥ 1.
Then P[ξ ∈ H(x)] = P[g t (x, ξ) ≥ 0 ∀t ∈ T ] and in particular P[ξ ∈ H(x n )] ≥ p. We now claim that H(x) = ∩ k≥1 A k .

(2.2.2)

Clearly z ∈ H(x) implies z ∈ A k for all k ≥ 1. To show the opposite, let z ∈ ∩ k≥1 A k , be arbitrary. Then one can find sequences z k → z, y k → x, such that z k ∈ H(y k ) and from upper semi-continuity of g t we get for each t ∈ T :

g t (x, z) ≥ lim sup k→∞ g t (y k , z k ) ≥ 0, (2.2.3)
yielding z ∈ H(x). We have thus shown (2.2.2). Since clearly A k+1 ⊆ A k for all k ≥ 1 it follows from standard measure theory [START_REF] Billingsley | Probability and measure[END_REF] that

P[ξ ∈ A k ] ↓ P[ξ ∈ H(x)
]. Now pick an arbitrary ε > 0 and k ′ ≥ 0 such that P[ξ ∈ H(x)]-P[ξ ∈ A k ′ ] ≥ -ε. Such k ′ can be found according to the statements above. Since x n converges to x in E, one can moreover find N such d E (x n , x) ≤ k ′-1 for all n ≥ N . According to the definition of A k ′ this implies that H(x n ) ⊆ A ′ k for n ≥ N . Altogether we obtain

P[ξ ∈ H(x)] -P[ξ ∈ H(x n )] ≥ P[ξ ∈ H(x)] -P[ξ ∈ A k ] ≥ -ε, (2.2.4)
i.e., P[ξ ∈ H(x)] ≥ pε. Since ε > 0 is arbitrary, P[ξ ∈ H(x)] ≥ p and x ∈ M (p) as was to be shown.

Continuity of Probabilistic Constraints

In order to show continuity of the probabilistic constraint we require a situation wherein no probabilistic mass is assigned to the boundary of the set {z ∈ F : g(x, z) = 0}, where the situation is as in Lemma 2.2.2. An example of a discontinuous probability constraint can be found in [START_REF] Henrion | Optimierungsprobleme mit wahrscheinlichkeitsrestriktionen: Modelle, struktur, numerik[END_REF]Beispiel 2.1]. The following result is a straightforward extension of [START_REF] Henrion | Optimierungsprobleme mit wahrscheinlichkeitsrestriktionen: Modelle, struktur, numerik[END_REF][START_REF] Raik | Qualitative research into the stochastic nonlinear programming problems (russian)[END_REF].

Lemma 2.3.1. Let E, F be arbitrary metric spaces, T be an arbitrary index set and let ξ ∈ F be a random variable. Assume furthermore that all members g t : E × F → R of the family of mappings {g t } t∈T are (jointly) lower semi-continuous. Assume furthermore that the sets N x := {z ∈ F : inf t∈T g t (x, z) = 0} are P-null-sets for all x ∈ E, i.e., P[ξ ∈ N x ] = 0. The mapping ϕ : E → [0, 1] defined as ϕ(x) = P[g t (x, ξ) ≥ 0 ∀t ∈ T ] is then also lower semi-continuous.

Proof. Fix x ∈ E arbitrarily and define the mapping g inf : E × F → R as g inf (x, z) := inf t∈T g t (x, z). The mapping g inf is then also lower semi-continuous and -g inf upper semi-continuous. Upon applying Lemma 2.2.2 we derive lim sup

x→x P[-g inf (x, ξ) ≥ 0] ≤ P[-g inf (x, ξ) ≥ 0]. (2.3.1)
According to our assumption we have P[g inf (x, ξ) = 0] = 0. So combining this with (2. We have thus shown that ϕ is indeed lower semi-continuous.

Remark 2.3.2. If T is a finite set, it is sufficient to require that N x := {z ∈ F : g t (x, z) = 0} is a P-null set for each t ∈ T .

Upon combining Lemma 2.3.1 and 2.2.2 we thus obtain the following Theorem providing conditions for continuity of the probabilistic constraint:

Theorem 2.3.3. Let E, F be arbitrary metric spaces, T be an arbitrary index set and let ξ ∈ F be a random variable. Let all members g t : E × F → R of the family of mappings {g t } t∈T be (jointly) continuous. Assume furthermore that the sets N x := {z ∈ F : inf t∈T g t (x, z) = 0} are P-null-sets for all x ∈ E, i.e., P[ξ ∈ N x ] = 0. The mapping ϕ : E → [0, 1] defined as ϕ(x) = P[g t (x, ξ) ≥ 0 ∀t ∈ T ] is then also continuous.

An important specification is obtained when F = R m : Lemma 2.3.4. Let E be an arbitrary metric space, T be an arbitrary index set and let ξ ∈ R m be a random variable having a density with respect to the Lebesgue measure. Let all members g t : E × R m → R of the family of mappings {g t } t∈T be (jointly) continuous. Assume furthermore that the sets N x := {z ∈ R m : inf t∈T g t (x, z) = 0} are Lebesgue-nullsets for all x ∈ E. The mapping ϕ : E → [0, 1] defined as ϕ(x) = P[g t (x, ξ) ≥ 0 ∀t ∈ T ] is then continuous.

Proof. Since ξ admits a density with respect to the Lebesgue measure it is clear that P[ξ ∈ N x ] = 0 for all x ∈ E. We can now use Theorem 2.3.3 to conclude the proof.

Connectedness of Feasible Set

One of the last most general results we can derive is that of connectedness of the feasible sets of constraints of the type (2.0.1).

Theorem 2.4.1. Let E, F be arbitrary metric spaces, T be an arbitrary index set and let ξ ∈ F be a random variable. Let all members g t : E × F → R of the family of mappings {g t } t∈T be (jointly) quasi-concave. Assume furthermore that g is max-stable, i.e., for each x 1 , x 2 ∈ E, one can find x 3 ∈ E such that g t (x 3 , z) ≥ max {g t (x 1 , z), g t (x 2 , z)} for each z ∈ F, t ∈ T . The set M (p) := {x ∈ E : ϕ(x) ≥ p} is then path-connected for each p ∈ [0, 1], where ϕ : E → [0, 1] is defined as ϕ(x) = P[g t (x, ξ) ≥ 0 ∀t ∈ T ].

Proof. This follows directly from [97, Theorem 2.1] upon inverting the inequalities figuring therein.

Convexity of the Feasible Set

For a (highly) efficient numerical treatment it is clearly of interest to know under which conditions convexity of the feasible set for constraints of the type (2.0.1) is to be expected. Upon examining the special case wherein g(x, z) = xz, in which the constraint (2.0.1) directly relates to the distribution function of ξ, it is clear that concavity of x → P[g(x, ξ) ≥ 0] can't be expected. Indeed, already one dimensional distribution functions are not concave. However it is sufficient that the mapping x → P[g(x, ξ) ≥ 0] is quasi-concave for the feasible set M (p) := {x ∈ R n : P[g(x, ξ) ≥ 0] ≥ p} to be convex. Then the feasible set M (p) is convex for all p ∈ [0, 1] if and only if ϕ is quasi-concave.

Proof. Let p ∈ [0, 1], x, y ∈ M (p), λ ∈ [0, 1] all be arbitrary and form x λ = λx + (1λ)y.

Quasi-concavity of ϕ implies ϕ(x λ ) ≥ min {ϕ(x), ϕ(y)} ≥ p, showing x λ ∈ M (p). To show the other implication let x, y ∈ R n , λ ∈ [0, 1] all be arbitrary. Upon defining p := min {ϕ(x), ϕ(y)} it follows that x, y ∈ M (p). Hence from convexity of M (p) we obtain x λ ∈ M (p), i.e., ϕ(x λ ) ≥ p as was to be shown.

Quasi-concavity (and further generalized) concavity has a good chance of arising in many situations. To this end we introduce the mapping: Definition 2.5.2. Let α ∈ [-∞, ∞] and m α : R + × R + × [0, 1] → R be defined as follows m α (a, b, λ) = 0 if ab = 0, (2.5.1)

for a > 0, b > 0, λ ∈ [0, 1]:

m α (a, b, λ) =        a λ b 1-λ if α = 0 max {a, b} if α = ∞ min {a, b} if α = -∞ (λa α + (1 -λ)b α ) 1 α else (2.5.2)
We can now provide the definition of generalized concavity: Definition 2.5.3. A non-negative function f defined on some convex set C ⊆ R n is called α-concave (α ∈ [-∞, ∞]) if and only if for all x, y ∈ C, λ ∈ [0, 1]:

f (λx + (1 -λ)y) ≥ m α (f (x), f (y), λ), (2.5.3)
where m α is as in Definition 2.5.2.

Remark 2.5.4. A function f is 0-concave if its logarithm is concave. For α = 0, α ∈ R, the function f is α-concave if either f α is concave for α > 0 or f α is convex for α < 0.

Quasi-concavity refers to -∞-concavity in Definition 2.5.3.

The following lemma can be found in [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF] and shows that α-concavity implies quasiconcavity, i.e., quasi-concavity is the weakest form of "generalized" concavity that exists.

In particular concave mappings are log-concave etc... Lemma 2.5.5. Let m α be the mapping as defined in Definition 2.5.2. The mapping α → m α is nondecreasing and continuous.

For calculus rules with α-concavity we refer to Theorems 4. 19-4.23 of [51].

Definition 2.5.6. A probability measure P defined on the Lebesgue measurable subsets of a convex set C ⊆ E is said to be α-concave if for all Borel measurable subsets A, B ⊆ C, λ ∈ [0, 1] we have

P[λA + (1 -λ)B] ≥ m α (P[A], P[B], λ), (2.5.4)
where λA + (1λ)B = {λx + (1λ)y : x ∈ A, y ∈ B} is the Minkowski sum of A and B.

Remark 2.5.7. Prékopa introduced the notion of log-concave measures in [START_REF] Prékopa | Logarithmic concave measures with applications to stochastic programming[END_REF] but the above generalization was suggested by [START_REF] Borell | Convex set functions in d-space[END_REF][START_REF] Brascamp | On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log-concave functions and with an application to the diffusion equations[END_REF].

In fact it turns out that quasi-concavity of measures relates to densities. In fact "all" non-degenerate quasi-concave measures must have a density: Lemma 2.5.8 ( [START_REF] Borell | Convex set functions in d-space[END_REF]). If P is a quasi-concave measure on R m and the dimension of its support is also m, then P has a density with respect to the Lebesgue measure.

Now that we know that quasi-concave measures have densities, it is natural to wonder if generalized concavity of such densities carries over to the measure. This turns out to be the case as the following result shows. We also refer to [START_REF] Prékopa | Stochastic Programming[END_REF]Chapter 4] and references therein.

Theorem 2.5.9 ([176, 178, 22, 196]). Let C be a convex subset of R m and let s > 0 be the dimension of the smallest affine subspace L containing C. The probability measure P on C is γ-concave with γ ∈ [-∞, 1 s ] if and only if its probability density function with respect to the Lebesgue measure on L is α-concave with

α =    γ 1-sγ if γ ∈ (-∞, 1 s ) -1 s if γ = -∞ ∞ if γ = 1 s (2.5.5)
Corollary 2.5.10. Let f : R m → R be integrable and positive on a convex set C ⊆ R m with non-zero Lebesgue Measure. If f is α-concave on C with α ∈ [-1 m , ∞] and positive on the interior of C, then the measure

P[A] = 1 C θdλ A f dλ is γ-concave on C with γ =    α 1+mα if α ∈ ( 1 m , ∞) 1 m if α = ∞ -∞ if α = -1 m (2.5.6)
Lemma 2.5.11. Let ξ ∈ R m be a random variable with α-concave probability distribution P ξ , where α ∈ [-∞, ∞]. Let T be an s × m constant matrix and let η ∈ R s be the random variable defined as η := T ξ. Then the probability distribution of η is α-concave too.

Remark 2.5.12. For the application of Theorem 2.5.9 it is important to distinguish probability distribution and probability distribution function. The former concept relates to the measure P defined on an appropriate σ-algebra. The latter concept is a restriction of the measure to sets of the form (-∞, x]. In order to illustrate the difference, the lognormal law has a probability distribution without any generalized concavity property, but probability distribution function that is log-concave.

In applications the following theorem (proved for multivariate normal distributions in [START_REF] Zalgaller | Mixed volumes and the probability of falling into convex sets in case of multivariate normal distributions[END_REF]) provides a strong tool for deriving convexity of feasible sets: Theorem 2.5. 13 ([176]). Let C ⊆ R m be a convex set and let P be a α-concave measure defined on Lebesgue measurable subsets of C, where α ∈ [-∞, ∞]. Let A be a convex (i.e., Lebesgue measurable) subset of C and x ∈ R m arbitrary, then x → f (x) := P[A + x] is an α-concave function.

Proof. Let x, y ∈ R m , λ ∈ [0, 1] all be arbitrary and remark that convexity of A provides the following invariance under the Minkowski sum A = λA + (1λ)A. It thus follows that λ(A + x) + (1λ)(A + y) = A + λx + (1λ)y. Upon applying the α-concavity of the measure P we obtain:

f (λx + (1 -λ)y) = P[A + λx + (1 -λ)y] = P[λ(A + x) + (1 -λ)(A + y)]
≥ m α (f (x), f (y), λ).

Theorem 2.5.14 ([176]). Let ξ ∈ R m be a random variable that induces an α-concave probability distribution P, where α ∈ [-∞, ∞]. Then its associated probability distribution function is also α-concave.

Proof. Define the convex set A = {z ∈ R m : z ≤ 0}. Then x, y ∈ R m with y ≤ x ⇔ y ∈ A + x. As a consequence F ξ (x) := P[ξ ≤ x] = P[ξ ∈ (A + x). The result now follows from Theorem 2.5.13.

One of the most general concavity results is the following. It can be found in this form in [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF], but its original form is found in [START_REF] Prékopa | A class of stochastic programming decision problems[END_REF][START_REF] Prékopa | On logarithmic concave measures and functions[END_REF] where it is specified to log-concave (i.e., 0-concave) measures. Tamm [START_REF] Tamm | On g-concave functions and probability measures (russian)[END_REF] observed that quasi-concavity of the mapping g was sufficient and concavity of g an excessive requirement. The generalization to α-concave measures was provided by [START_REF] Borell | Convex set functions in d-space[END_REF][START_REF] Brascamp | On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log-concave functions and with an application to the diffusion equations[END_REF].

Theorem 2.5.15. Let g : R n × R m → R k be a (jointly) quasi-concave function and let ξ ∈ R m be a random variable inducing an α-concave probability distribution P. Then the mapping x ∈ R n → G(x) := P[g(x, ξ) ≥ 0] is an α-concave function on the set

D = {x ∈ R n : ∃z ∈ R m with g(x, z) ≥ 0}.
Remark 2.5.16. Consider separable constraints in (2.0.1), i.e., g i (x, z) = h i (x)z, i = 1, ..., k where h : R n → R m . Then G(x)

:= P[g(x, ξ) ≥ 0] = P[ξ ≤ h(x)] = F ξ (h(x))
and convexity of the feasible set directly relates to the generalized concavity properties of distribution function and (generalized) concavity properties of h.

Remark 2.5.17. Consider an individual separable constraint of the form 2.0.3 (i.e., set k = 1 in (2.0.1)). One then easily derives that

P[ξ ≤ h(x)] ≥ p is equivalent with h(x) ≥ F (-1) (p)
, where the latter is the generalized inverse of the one-dimensional distribution function F . This is an explicit constraint and convexity can be deduced from properties of h directly.

Remark 2.5.18. The strength of Theorems 2.5.9 and 2.5.15 is that many random variables actually have generalized concavity properties. Indeed we can provide the following list of random variables having α-concave densities (see [START_REF] Prékopa | Probabilistic programming[END_REF]):

1. Gaussian random variables have log-concave densities 2. Dirichlet distributions with parameters ϑ j ≥ 1 are log-concave, whereas for ϑ j ≤ 1, they are log-convex. m -concave density. Its distribution is therefore quasi-concave. 9. The multi-variate Student t-distribution has a -1 m -concave density. Its distribution is therefore quasi-concave.

Special Cases with Convexity

Joint quasi-concavity of the mapping g appearing in (2.0.1) does not hold in all cases of interest. This does not imply that convexity of the feasible set can't be derived. A particular case of interest is one wherein g(x, ξ) ≥ 0 represents a random linear inequality system Ax ≤ b, where A is random.

The following easily derived result provides a convexity statement in such a setting Lemma 2.5. 19 ([121]). Consider the constraint of the form (2.0.1) where k = 1, g(x, z) = z T xb and ξ ∈ R m is a multivariate Gaussian random variable. Then the feasible set M (p) is convex for all p > 1 2 .

Generalizations of this result to k > 1 require special correlation structures for the Gaussian random vector. The column version is due to [START_REF] Prékopa | Programming under probabilistic constraints with a random technology matrix[END_REF], whereas the row version comes from [START_REF] Burkauskas | On the convexity problem of probabilistic constrained stochastic programming problems (hungarian)[END_REF]:

Theorem 2.5.20. Let A be a m × n matrix having a non degenerate multivariate Gaussian density in R nm . Assume that there is a fixed covariance matrix C and a n × n (or m × m) symmetric matrix S, such that either:

1. The columns A .j of A, satisfy

Cov (A .j , A .k ) = s jk C, j, k = 1, ..., n,

The rows

A j. of A, satisfy Cov (A j. , A k. ) = s jk C, j, k = 1, ..., m,
Then the set {x :

P[Ax ≤ 0] ≥ p} is convex for all p ≥ 1 2 .
Proof. A very comprehensive proof can be found in [119, Theorem 2.12].

Eventual Convexity of the Feasible set

As in the result by Kataoka ([121]), convexity of the feasible set M (p) cannot always be obtained for any probability level p. From a practical perspective this is not necessarily a problem since we are naturally looking for high p levels when formulating constraints of the type (2.0.1). In some cases, we can show that the feasible set is convex if p is large enough. Convexity of feasible sets with high enough p is known as eventual convexity.

Some key results on eventual convexity have been derived by [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. The following concept plays a key role in their results.

Definition 2.5.21. The function f : R → R is called r-decreasing (r ∈ R) if it is continuous on (0, ∞) and there exist some t * > 0 such that t → t r f (t) decreases strictly for t > t * .

Lemma 2.5.22 ([110]). Let F : R → [0, 1] be a probability distribution function with a γ + 1-decreasing density for some γ > 0. Then the function z → F (z -1 γ ) is concave on (0, (t * ) -γ ), with t * as in Definition 2.5.21. Moreover F (t) < 1 for all t ∈ R. Definition 2.5.23. Let q ∈ (0, 1) m be arbitrary. A copula

C : [0, 1] m → [0, 1] is called log-exp concave on [q, 1] if the mapping u ∈ R m → log C(e u 1 , ..., e u m ) is concave on [log(q), 0].
Theorem 2.5.24 ([110, 111]). Let ξ ∈ R m be a random vector, C : [0, 1] m → [0, 1] be a copula, and let g i : R n → R be mappings such that

P[ξ ≤ g(x)] = C(F 1 (g 1 (x)), ..., F m (g m (x))), (2.5.7)
where F i are the marginal distribution functions of component i of ξ, i = 1, ..., m. If we can find r i > 0, i = 1, ..., m such that the following three conditions hold:

1. The functions g i are (-r i )-concave 2. The marginal distribution Functions F i are generated by (r i +1)-decreasing densities f i with associated parameter t * i 3. The copula C is log-exp concave on [q, 1], where q ∈ (0, 1) m is defined as

q i = F i (t * i ), i = 1, ..., m.
Then the set {x ∈ R n :

P[ξ ≤ g(x)] ≥ p} is convex for all p > p * := max i=1,...,m F (t * i ).
Upon exploiting the independence of the rows of a Gaussian Technology Matrix, one can derive a formulation that allows for an indirect application of the previous result. This, in turn, allows us to assert eventual convexity of feasible sets with Gaussian Technology Matrices as the following result shows:

4λ i max [λ min ] -3 2 µ i .
Moreover, µ i is the average vector of the i-th row and λ i min , λ i max refer to the smallest and largest eigenvalues of its covariance matrix Σ i .

Eventual convexity will receive further attention in Section 3.4.

(Local) Lipschitz Continuity

Consider the special case of (2.0.1) wherein g(x, z) = h(x)z, for h : R n → R m . In this case the probability constraint (2.0.1) becomes

P[h(x) ≥ ξ] ≥ p.
(2.6.

Differentiability of Probabilistic Constraints

An important question for numerical treatment of constraints of type (2.0.1) is differentiability. Indeed many optimization algorithms require a gradient. Differentiability statements of chance constraints of the type (2.0.1) come mainly in two forms. On one hand we have very general differentiability statements providing rather complex characterizations. On the other hand we have practical characterizations for special cases.

In the works dealing with such special cases one tries to reduce the computation of any component of a gradient to the computation of integrals for which efficient numerical sampling approaches are available. Such approaches will also receive a thorough discussion in Sections 3.2 and 3.3.

General Statements

Theorem 2.7.1 ([234, 235, 237], for similar work see also [START_REF] Marti | Differentiation of probability functions : The transformation method[END_REF][START_REF] Marti | Differentiation of probability functions : The transformation method[END_REF]). Let g : R n × R m → R k be a continuously differentiable function and let θ : R n × R m → R be a continuously differentiable density. Pick moreover 1 ≤ l < k arbitrarily. Assume moreover that 1. The set µ(x) := {y ∈ R m : g(x, y) ≤ 0} is bounded in a neighbourhood U of some point x.

2. At x all constraints g i (x, y) ≤ 0, i = 1, ..., k are active 3. One can a find continuous matrix function

H l : R n × R m → R n×m satisfying H(x, y)∇ y g l (x, y) + ∇ x g l (x, y) = 0,
where g l (x, y) = (g 1 (x, y), ..., g l (x, y)) ∈ R l .

4. The matrix function H l has a continuous partial derivative with respect to y.

The gradient

∇ y g i (x, y) = 0 on ∂ i µ(x) := µ(x) ∩ {y ∈ R m : g i (x, y) = 0}.
6. For each y ∈ µ(x), the vectors ∇ y g i (x, y), i ∈ I(x, y) := {j : g j (x, y) = 0} are linearly independent

Then the mapping ϕ(x) := µ(x) θ(x, y)dλ(y) = P[µ(x)] is differentiable at x and

∇ x ϕ(x) = µ(x) ∇ x θ(x, y) + div y (θ(x, y)H l (x, y))dλ(y) - k i=l+1 ∂ i µ(x) θ(x, y) ∇ y g i (x, y) [∇ x g i (x, y) + H l (x, y)∇ y g i (x, y)]dS,
where λ is the Lebesgue measure on R m .

In Theorem 2.7.1 above one can select 1 ≤ l < k in a way that is convenient for the application at hand. The special choices l = 0 and l = k can also be made and lead to the following special cases (we refer to [START_REF] Uryas'ev | Introduction to the Theory of Probabilistic Functions and Percentiles (Value-at-Risk) Chapter 1 in[END_REF] for a modern version):

Theorem 2.7.2 ([193, 200, 232, 231, 233]). With notation and conditions as in theorem 2.7.1. Let l = 0, then we have:

∇ x ϕ(x) = µ(x) ∇ x θ(x, y) - k i=1 ∂ i µ(x) θ(x, y) ∇ y g i (x, y) ∇ x g i (x, y)dS.
If l = k we have:

∇ x ϕ(x) = µ(x) ∇ x θ(x, y) + div y (θ(x, y)H k (x, y))dλ(y)

Special Cases

In some special cases very efficient formulae for gradients can be established. These formulae can be derived upon considering specific densities for the random vector ξ of (2.0.1) and/or specifically structured chance constraints. When considering chance constraints with a separable structure as in (2.6.1), differentiability depends completely on that of the distribution function and the mapping h.

The following general result on differentiability of distribution functions can be derived.

(For a comprehensive proof we refer to [START_REF] Henrion | Optimierungsprobleme mit wahrscheinlichkeitsrestriktionen: Modelle, struktur, numerik[END_REF]).

Theorem 2.7.3 (e.g., [START_REF] Prékopa | Stochastic Programming[END_REF]). Let ξ ∈ R m be a random vector with density f ξ : R m → R.

Fix any z ∈ R m and consider F ξ (z

) := P[ξ ≤ z]. If ϕ (i) (t) := z1 -∞ . . . zi-1 -∞ zi+1 -∞ . . . zs -∞ f ξ (u 1 , ..., u i-1 , t, u i+1 , ..., u s )du 1 . . . du i-1 du i+1 . . . du s ,
is continuous for all i = 1, ..., s, then F ξ (z) is partially differentiable at z and

∂F ξ ∂z i (z) = ϕ (i) (z i ). (2.7.1)
Remark 2.7.4. Stochastic Gradient Approaches for distribution functions of the above form can be found in [START_REF] Kibzun | Stochastic Programming Problems with Probability and Quantile Functions[END_REF][START_REF] Kibzun | Guaranteeing approach to solving quantile optimization problems[END_REF]. In energy applications, stochastic gradient approaches have been investigated for minimizing expected value objective functions (e.g., [START_REF] Dodu | Méthodes de gradient stochastique pour l'optimisation des investissements dans un réseau électrique[END_REF][START_REF] Goursat | Stochastic gradient methods for optimizing electrical transportation networks[END_REF]).

An important specialization Theorem 2.7.3 can be obtained if we consider Gaussian random variables.

Lemma 2.7.5 ([175, 181]). Let ξ be an m-dimensional Gaussian random vector with mean µ ∈ R m and positive definite variance-covariance matrix Σ. Then the distribution function F ξ (z) := P[ξ ≤ z] is continuously differentiable and in any fixed z ∈ R m the following holds:

∂F ξ ∂z i (z) = f ξ i (z i )F ξ(z i ) (z 1 , ..., z i-1 , z i+1 , ..., z m ), i = 1, ..., m. (2.7.2)
Here ξ(z i ) is a Gaussian random variable with mean μ ∈ R m-1 and (m -1) × (m -1) positive definite covariance matrix Σ. Let D i m denote the m-th order identity matrix from which the ith row has been deleted. Then

μ = D i m (µ + Σ -1 ii (z i -µ i )Σ i ) and Σ = D i m (Σ -Σ -1 ii Σ i Σ T i )(D i m ) T
, where Σ i is the i-th column of Σ.

The above result requires the Gaussian random variable to be non-degenerate, i.e., have positive definite covariance matrix. This is an important restriction in many situations, in particular in network design when randomness on nodes is considered. In such applications the degeneracy occurs because we multiply a non-degenerate Gaussian random variable with a matrix having more lines than columns. The following result therefore allows for an important generalization of Lemma 2.7.5.

Theorem 2.7.6 ( [START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF]). Let A be a k × m matrix. Consider a linear inequality system Ax ≤ z and define

I(A, z) = I ⊆ {1, ..., k} : ∃x ∈ R m a T i x = z i , i ∈ I, a T i x < z i , i / ∈ I . Assume that z ∈ R m is such that Ax ≤ z is nondegenerate (i.e., rank {a i } i∈I = |I| ∀I ∈ I(A, z)).
Let ξ be an m-dimensional Gaussian random vector with mean µ and positive definite variance-covariance matrix Σ. Then the mapping ϕ(z

) = P[Aξ ≤ z] is differen- tiable at z and ∂ϕ ∂z j (z) = 0 if {j} / ∈ I(A, z) f j (z j )P[A (j) L (j) ξ (j) ≤ z (j) -A (j) w (j) ] if {j} ∈ I(A, z) (2.7.3)
Here ξ (j) is a centered m -1 dimensional Gaussian random variable with independent components, A (j) is obtained from A by deleting row j, z (j) is defined similarly. Moreover, L (j) is the Choleski matrix of S (j) := Σ -1 a T j Σa j Σa j a T j Σ (i.e., S (j) = L (j) (L (j) ) T ), w (j) = µ+ z j -a T j µ a T j Σa j Σa j and f j the one-dimensional Gaussian density with mean µ T a j and variance a T j Σa j . Finally the inequality system A (j) L (j) y ≤ z (j) -A (j) w (j) is nondegenerate.

Other important special cases involve the computation of gradients of multi-variate Gamma [START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF] and Dirichlet Distributions [START_REF] Szántai | Numerical evaluation of probabilities concerning multi-dimensional probability distributions[END_REF][START_REF] Prékopa | Stochastic Programming[END_REF] Theorem 2.7.7 ( [START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF][START_REF] Prékopa | Stochastic Programming[END_REF]).

A multivariate Gamma distribution ζ ∈ R m is defined as ζ = Aη, where η ∈ R 2 m -1 contains independent standard Gamma (with parameters ϑ j ) distributed components and A is a m × 2 m -1 matrix with non-zero columns, A ij ∈ {0, 1} for i = 1, ..., m, j = 1, ..., 2 m -1. Define for each i = 1, ..., m, I i ⊆ {1, ..., 2 m -1} as I i = {j : A ij = 1}. Then δ i ∈ R m-1 , δ i k = j∈I k ∩I i η j j∈I i η j , k = 1, ..., i -1, i + 1, ..., m, (2.7.4
)

is an m -1 dimensional Dirichlet Distribution with parameters Θ k = j∈I k ∩I i ϑ j , k = 1, ..., i -1, i + 1, ..., m Θ m+1 = j∈∪ k =i I k \I i ϑ j , for each i = 1, ..., m. Now F (z) := P[ζ ≤ z] is partially differentiable and ∂F ξ ∂z i (z) = P[z i δ i k + γ k ≤ z k ∀k = i] z ϑ i -1 i e -z i Γ(ϑ i ) , (2.7.5)
where

γ k = j∈I k ∩ Īi η j , k = 1, ..., i -1, i + 1, ..., m, is an m -1 dimensional multivariate gamma distribution independent of δ i .
Theorem 2.7.8 ([258, 181, 88]). Let ξ ∈ R m have a multivariate Dirichlet distribution, i.e., have the density:

f (z 1 , ..., z m ) = Γ(ϑ 1 + ... + ϑ m+1 ) Γ(ϑ 1 )...Γ(ϑ m+1 ) z ϑ 1 -1 1 ...z ϑ m -1 m (1 - m j=1 z j ) ϑ m+1 -1 ,
on the unit simplex z ∈ ∆ m in dimension m (zero elsewhere). If

y i = ( z 1 1 -z i , ..., z i-1 1 -z i , z i+1 1 -z i , ..., z m 1 -z i ) ∈ R m-1
satisfies y (1) + y (2) > 1 or y (1) + y (2) + y (3) > 1 (but y (1) + y (2) ≤ 1) for the order-statistics y (.) , then F (z) := P[ξ ≤ z] is partially differentiable at z and

∂F ξ ∂z i (z) = P[ ξi k ≤ y i k , ∀k = i] Γ(ϑ 1 + ... + ϑ m+1 ) Γ(ϑ i )Γ( j =i ϑ j ) z ϑ i -1 i (1 -z i ) j =i ϑ j -1 , (2.7.6)
where ξi has an m -1 dimensional Dirichlet distribution with parameters ϑ 1 , ..., ϑ i-1 , ϑ i+1 , ..., ϑ m+1 .

Algorithms

In this section we provide a brief overview of known algorithms for solving optimization problems with chance constraints of type (2.0.1). Throughout this section we will assume that the resulting optimization problem is convex. For instance in a case where Theorem 2.5.15 can be applied. We refer to [START_REF] Prékopa | Probabilistic programming[END_REF] for schemes for each of these algorithms. Algorithms will be discussed in details in sections 4.1 and 4.2.

For the purpose of discussion, let c : R n → R be a convex function, providing an alternative equivalent representation of constraint (2.0.1), i.e., c(x) ≤ 0 if and only if (2.0.1) holds. For instance if g is (jointly)-quasi concave and ξ has a log-normal density, we may define c(x) := log(p)log(P[g(x, ξ) ≥ 0]). The sub-gradient inequality then gives:

c(y) ≥ c(x) + g, y -x ∀y ∈ R n , (2.8.1)
where g ∈ ∂c(x) is an arbitrary sub-gradient of the convex function c at point x ∈ R n . When y ∈ R n is feasible for (2.0.1) this implies that 0 ≥ c(y) ≥ c(x) + g, yx ∀y s.t. (2.0.1) holds .

Now if

x ∈ R n is moreover chosen such that c(x) = 0, this reduces to

g, y -x ≤ 0, (2.8.2)
which is a valid inequality for all feasible solutions y ∈ R n .

Feasible direction Methods

In this method the mapping c is assumed to be continuously differentiable. In a first stage a linear programming problem is solved in which constraint (2.8.2) appears with an additional auxiliary variable z ∈ R also appearing in a similarly formed equation involving the objective function:

g, y -x -Θz ≤ 0,
where Θ is an appropriately chosen constant and x our current iterate. Let (y * , z * ) be the optimal solution of that linear program. In a second step we compute λ such that x + λ(y *x) is feasible. The algorithm is stopped with optimal solution y * whenever z * = 0.

This approach is originally due to [START_REF] Zoutendijk | Methods of Feasible Directions : a Study in Linear and Non-Linear Programming[END_REF]. It should be noted however that Zoutendijk's method lacks the global convergence property as shown in [START_REF] Topkis | On the convergence of some feasible direction algorithms for nonlinear programming[END_REF]. We refer to the discussion in [START_REF] Minoux | Programmation Mathématique: Théorie et Algorithmes[END_REF] for further information. A method of feasible directions was adapted to the chance constrained programming setting by [START_REF] Prékopa | On probabilistic constrained programming[END_REF], further tested in [START_REF] Deák | Computer evaluation of a stochastic programming model[END_REF] and applied to a problem from the industry in [183].

SUMT or "penalty function methods"

Let ϕ : R n → [0, 1] be defined as ϕ(x) = P[g(x, ξ) ≥ 0], where ξ and g are as in (2.0.1). In this approach we assume that ϕ is log-concave. It then follows that x → ϕ(x)p is also log-concave on M (p), i.e., the feasible set for (2.0.1). The idea of the log-barrier approach is to add -s k log(ϕ(x)p) to the objective function. The sequence s k k≥0 of scalar multipliers is chosen strictly decreasing to zero. We then solve the (probabilistically) unconstrained optimization problem and update the multipliers iteratively. Details on SUMT approaches can be found in [START_REF] Fiacco | Nonlinear Programming: Sequential Unconstrained Minimization Techniques[END_REF]. The application of this approach to chance constrained programming was suggested by [START_REF] Prékopa | A class of stochastic programming decision problems[END_REF]. An application and implementation can be found [START_REF] Rapcsák | On the Numerical Solution of a Reservoir Model[END_REF][START_REF] Prékopa | Serially linked reservoir system design using stochastic programming[END_REF].

Interior Point Methods

In this approach we add an additional slack variable w and the constraint ϕ(x)p = w. We then form the classic logarithmic barrier function, wherein -µ log(w) appears. The above slack constraints are dualized in the Langrangian. We then apply the basic Interior Point approach: we establish the KKT conditions and use Newton's method to come up with search directions. This method has been investigated by many authors (e.g., [START_REF] Vanderbei | Interior-point algorithms for nonconvex nonlinear programming: orderings and higher-order methods[END_REF], [START_REF] Roos | Interior Point Methods for Linear Optimization[END_REF], [START_REF] Nocedal | Numerical Optimization[END_REF], [START_REF] Wright | Primal-Dual Interior-Point Methods[END_REF], etc...). One problem is that the Hessian often tends to become illconditioned. This might especially be true for probabilistic constraints. It is suggested in [START_REF] Prékopa | Probabilistic programming[END_REF] to use the approaches of [START_REF] Nash | A barrier-method for large-scale constrained optimization[END_REF][START_REF] Nash | Why extrapolation helps barrier methods[END_REF] in order to counterbalance this effect.

Supporting Hyperplane Method

In this approach we require the explicit knowledge of a Slater point x s ∈ R n . In particular we require ϕ(x s ) > p, where notation is as in section 2.8.2. The feasible set induced by constraint (2.0.1) needs to be convex as well. If x k is the current iterate, we use the Slater point to form x c = λx k + (1λ)x s in such a way that c(x c ) = 0, i.e., ϕ(x c ) = p. We then add (2.8.2) to the otherwise probabilistically unconstrained-optimization problem. This relaxation is then solved to produce the new iterate. More details on this approach are provided in sections 4.1 and section 5.1.

The idea of using cutting planes approaches for convex optimization dates back to the early 1960s ( [START_REF] Kelley | The cutting-plane method for solving convex programs[END_REF][START_REF] Gomory | Outline of an algorithm for integer solutions to linear programs[END_REF][START_REF] Cheney | Newton's method for convex programming and tchebycheff approximation[END_REF]). Cheney and Goldstein state in [START_REF] Cheney | Newton's method for convex programming and tchebycheff approximation[END_REF] that the key idea of their approach can be traced back to E. Remez (e.g., [START_REF] Ya | Sur un procédé convergent d'approximations successives pour determiner les polynomes d'approximation[END_REF]). We can also refer to [START_REF] Veinott | The supporting hyperplane method for unimodal programming[END_REF]. The cutting plane approach was adapted to the context of joint chance constrained programming by [START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Szántai | A computer code for solution of probabilistic-constrained stochastic programming problems[END_REF].

Related Concepts

In this section we provide a brief overview of topics related to chance constrained programming but not directly used in this Thesis. They figure here for the sake of providing a global overview of the field.

Stability

A frequently formulated objection against using chance constraints of the type (2.0.1) concerns the availability of knowledge of ξ. It is then argued that either the ξ is unknown as a whole or at best its parameters are known up to some estimation error. It is then natural to investigate to what extent the solutions of optimization problems with constraints of type (2.0.1) are impacted by substituting a wrong law for ξ or picking wrong parameters. Stability results address this question.

As an application one can consider the reduction of the sample size when ξ has a discrete distribution and the sample would be too large for numerical treatment. Such a case is considered in [START_REF] Henrion | Discrepancy distances and scenario reduction in two-stage stochastic integer programming[END_REF][START_REF] Henrion | Scenario reduction in stochastic programming with respect to discrepancy distances[END_REF].

For a general treatment of stability in stochastic programming problems we refer to [START_REF] Römisch | Stability of Stochastic Programming Problems[END_REF]. Specific stability results for optimization problems with chance constraints are derived in [START_REF] Dupačová | Stability in stochastic programming -probabilistic constraints[END_REF][START_REF] Gröwe | Estimated stochastic programs with chance constraints[END_REF][START_REF] Henrion | Metric regularity and quantitative stability in stochastic programs with probabilistic constraints[END_REF][START_REF] Henrion | Qualitative stability of convex programs with probabilistic constraints[END_REF][START_REF] Henrion | Perturbation Analysis of Chance-constrained Programs under Variation of all Constraint Data[END_REF][START_REF] Henrion | Hölder and lipschitz stability of solution sets in programs with probabilistic constraints[END_REF]118,[START_REF] Kaňková | A note on estimates in stochastic programming[END_REF][START_REF] Römisch | Stability analysis for stochastic programs[END_REF][START_REF] Römisch | Distribution sensitivity for certain classes of chanceconstrained models with application to power dispatch[END_REF][START_REF] Salinetti | Approximations for chance-constrained programming problems[END_REF][START_REF] Wang | Continuity of feasible solution sets of probabilistic constrained programs[END_REF].

In particular in the case of separable chance constraints of type (2.6.1) under a generalized concavity assumption on the distribution function, concavity of h and a compactness (non-emptiness) assumption on the optimal solution set of the unperturbed problem one can derive a Lipschitz continuity condition on the optimal values under small perturbations of ξ ( [START_REF] Henrion | Qualitative stability of convex programs with probabilistic constraints[END_REF][Theorem 1]). Under additional, somewhat restrictive conditions, one can also derive a Hausdorff-Hölder condition on the set of optimal solutions ([108]).

p-Efficient points

The concept of p-Efficient points generalizes the notion of quantile for 1-dimensional distribution functions. It is defined as follows: Definition 2.9.1. Let ξ ∈ R m be a random vector with distribution function F ξ . Let p ∈ (0, 1) be given.

A point v ∈ R m is called a p-efficient point of the distribution function F ξ if F ξ (v) ≥ p and no z ∈ R m , z = v can be found s.t. z ≤ v and F ξ (z) ≥ p.
It is then clear that a chance constraint of the type (2.6.1) can be replaced with h(x) ∈ E p , where E p denotes the set of p-efficient points for the distribution function F ξ . We refer to [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF][pages [START_REF] Hoeffding | Probability inequalities for sums of bounded random variables[END_REF][START_REF]System modeling and Optimization[END_REF][START_REF]Computational Combinatorial Optimization: Optimal or Provably Near-Optimal Solutions[END_REF][118][START_REF] Kall | Stochastic Linear Programming: Models, Theory and Computation[END_REF][START_REF] Karas | A bundle-filter method for nonsmooth convex constrained optimization[END_REF][START_REF] Kataoka | A stochastic programming model[END_REF][START_REF] Kaňková | A note on estimates in stochastic programming[END_REF] for some properties of the set E p .

The concept of p-efficient points was originally introduced for discrete distributions by [START_REF] Prékopa | Sharp bound on probabilities using linear programming[END_REF]. An enumeration method and solution approach based on a cutting plane method was derived in [START_REF] Prékopa | Programming under probabilistic constraints with discrete random variable[END_REF]. The cone generation method wherein only a single p-efficient point is generated at each iteration was developed in [START_REF] Dentcheva | Concavity and efficient points for discrete distributions in stochastic programming[END_REF] and further considered in [START_REF] Vízvári | The integer programming background of a stochastic integer programming algorithm of Dentcheva-Prékopa-Ruszczyński[END_REF]. A branch and bound algorithm was derived in [START_REF] Beraldi | A branch and bound method for stochastic integer problems under probabilistic constraints[END_REF] for the case wherein both x and ξ are integer valued. We also refer to [START_REF] Dentcheva | Regularization methods for optimization problems with probabilistic constraints[END_REF] (and references therein) for a modern algorithm based on augmented Lagrangian methods using p-efficient points.

Safe Tractable Approximations

As highlighted throughout this section, theoretical and numerical treatment of constraints of type (2.0.1) might be involved. A very natural idea is then to replace the constraint (2.0.1) by constraints in a potentially higher dimensional space that imply feasibility for (2.0.1). More specifically, following [START_REF] Ben-Tal | Robust Optimization[END_REF], we define a "safe tractable approximation" as follows:

Definition 2.9.2. Let S ⊆ R n ×R l be a convex set described by explicit convex functions. We call S a safe tractable approximation of constraint (2.0.1) if for every (x, v) ∈ S, x satisfies (2.0.1).

Two important contributions in this field are [START_REF] Nemirovski | Convex approximations of chance constrained programs[END_REF][START_REF] Ben-Tal | On safe tractable approximations of chanceconstrained linear matrix inequalities[END_REF]. In the first work the so called "Bernstein" Approximation is derived for fairly general probability constraints under an independence assumption of the random vector ξ. In the second work the authors consider chance constraints on inequalities arising in Semi-Definite programming1 .

Frequently such an approximation is derived by using bounding techniques such as Hoeffding's bound ( [START_REF] Hoeffding | Probability inequalities for sums of bounded random variables[END_REF]). Such an approach is considered in [START_REF] Zorgati | Optimizing financial and physical assets with chance-constrained programming in the electrical industry[END_REF][START_REF] Zorgati | Optimizing financial and physical assets with chance-constrained programming in the electrical industry[END_REF]. The obvious loss of optimality inherent to such an approach is a serious drawback. It can be attenuated by some ad-hoc parameter tweaking, such as deriving the safe tractable approximation under another security level p (in (2.0.1)). These approaches could be of interest to derive a first estimate of the optimal solution of an optimization problem with a constraint of type (2.0.1) cheaply. One would then consider a hybrid solution approach with a first stage wherein one derives a good estimate x 0 of the optimal solution. A second stage dealing with (2.0.1) in an appropriate way would then be applied to obtain the optimal solution.

Scenario Approximations / Sample Average Approximations

The key idea of these approaches is to replace (2.0.1) by a sampled version. To this end let ξ 1 , ..., ξ N be an (iid) sample of the random vector ξ. We introduce auxiliary binary variables z i ∈ {0, 1} for i = 1, ..., N and look at the following approximation of (2.0.1):

g(x, ξ i ) ≥ (z i -1)M e (2.9.1) 1 N N i=1 z i ≥ q,
where q ≥ p, M > 0 is an appropriately chosen "big"-M constant and e = (1, ..., 1) ∈ R k .

Intuitively each z i corresponds to an active constraint on sample value ξ i when z i = 1 and deactivated constraints whenever z i = 0. Other than inherent difficulties due to "big"-M constraints this approach raises several questions. In particular to what extent can we expect a feasible solution of (2.9.1) to be feasible for (2.0.1). A second question naturally involves the required sample size N .

The Scenario Approximation refers to a situation wherein we pick q = 1, i.e., all binary variables in (2.9.1) are removed. This approach was developed by [START_REF] Calafiore | Uncertain convex programs: Randomized solutions and confidence levels[END_REF] and extended for large p by [START_REF] Nemirovski | Scenario approximations of chance constraints[END_REF][START_REF] Nemirovski | Scenario Approximations of Chance Constraints[END_REF]. Both works provide a result linking a given confidence level, q, and the sample size N . Very similar links are also exhibited in [START_REF] De Farias | On constraint sampling in the linear programming approach to approximate dynamic programming[END_REF] in the context of sampling constraints in approximate dynamic programming. The scenario approximation approach can alternatively be interpreted as a version of Robust Optimization with a discrete uncertainty set. Heuristically discarding some elements of the sample has also been investigated in [START_REF] Campi | A sampling-and-discarding approach to chanceconstrained optimization: Feasibility and optimality[END_REF]. The Sample Average Approximation approach (i.e., picking q < 1 in (2.9.1)) was first investigated for Probabilistic constraints in [START_REF] Luedtke | A sample approximation approach for optimization with probabilistic constraints[END_REF]. The same approach was traditionally used for stochastic programs with expected value objective functions (see [START_REF] Shapiro | Monte Carlo sampling methods[END_REF] and references therein, see also [START_REF] Pagnoncelli | Sample average approximation method for chance constrained programming: Theory and applications[END_REF]). Hence [START_REF] Luedtke | A sample approximation approach for optimization with probabilistic constraints[END_REF] provide an extension of this concept to applications involving probabilistic constraints. In particular, a a result linking a given confidence level, q, and the sample size N is given.

In a recent paper [START_REF] Luedtke | A branch-and-cut decomposition algorithm for solving chanceconstrained mathematical programs with finite support[END_REF] an approach is presented for specially structured problems that allows one to do away with the big M -constraints in (2.9.1). Under additional assumptions on x, even the auxiliary binary variables need not be explicitly generated. This is investigated in [START_REF] Song | Chance constrained binary packing problems[END_REF] Even though, in all these results, we can link the required sample size N , q and a confidence level for satisfying the actual constraint (2.0.1), N can be prohibitively large. We note that the number of constraints in (2.9.1) is equal to N k. Now if k = 100 and N = 10000 this may be particularly troublesome for MIP / LP solvers, even if g(x, z) is linear in x. Moreover, in a recent paper [START_REF] Henrion | A Critical Note on Empirical (Sample Average, Monte Carlo) Approximation of Solutions to Chance Constrained Programs[END_REF] shows that one can design problems already in dimension 2, i.e., n = 2 in (2.0.1) wherein N has to be arbitrarily large in order to have a rough estimate of the optimal solution. The results are illustrated for the Gaussian distribution. This shows that such sample average approximations should be used with care.

Chapter 3 Theoretical Extensions

We begin this chapter by investigating to what extent the feasible set induced by a joint probabilistic constraint can be approximated by a generalization of individual probabilistic constraints. In principle an infinite set of such individual probabilistic constraints should be considered. This approach could be of interest if for a current set of constraints and current candidate solution we can readily identify a violated constraint, which can then be added to the problem. A second condition is that the feasible set is a better approximation of the feasible set induced by the joint probabilistic constraint than that generated by individual probabilistic constraints. An example shows that this second condition could be satisfied in specific applications. We will also derive (negative) results showing that this second point does not hold in general.

Sections 3.2 and 3.3 are devoted to deriving gradient Formulae for probabilistic constraints involving Gaussian random variables in one way or another. The provided formulae allow for a direct application of the same sampling approach used to evaluate the probability constraint we took the derivative of.

Whenever the feasible set for constraints of type (2.0.1) is convex when p is high enough, we will speak of eventual convexity. For separable constraints (2.0.4) and random variables allowing for a special "correlation" structure through the use of a Copula recent results [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] on eventual convexity exist. The results in Section 3.4 provide an extension of the latter results on eventual convexity, by allowing for more Copulae and providing lower thresholds. Such lower thresholds allow us to assert convexity for probabilistically constrained feasible set with lower safety-levels.

Semi-Infinite Individual Chance Constraints

Consider the stochastic inequality system g(x, ξ) ≥ 0 appearing in (2.0.1). It is then clear that for each fixed ξ ∈ R m we have g(x, ξ) ≥ 0 if and only if v T g(x, ξ) ≥ 0 for all v ∈ R k , v ≥ 0. This motivates the following definition: Definition 3.1.1. Let g : R n × R m → R k be a mapping and ξ ∈ R m a random variable.

Consider the probabilistic constraint

P[g(x, ξ) ≥ 0] ≥ p, (3.1.1)
and its feasible set

M (p) := {x ∈ R n : P[g(x, ξ) ≥ 0] ≥ p}. Let v ∈ R k , v ≥ 0 be arbi- trary. We call x ∈ R n feasible for the v-Individual Chance Constraint (v-ICC) if and only if P[v T g(x, ξ) ≥ 0] ≥ p. (3.1.2)
We say that x ∈ R n is feasible for the semi-infinite individual chance constraint system if x is v-ICC feasible for each v ∈ R k , v ≥ 0. We denote the latter feasible set by

M SI (p) := x ∈ R n : (3.1.2) holds for all v ∈ R k , v ≥ 0 . Definition 3.1.
P[g i (x, ξ) ≥ 0] ≥ p, i = 1, ..., k. (3.1.3)
We will denote this with x ∈ M I (p).

The following Lemma shows that Definition 3.1.1 might allow for a tighter approximation of M (p) using v-Individual chance constraints. Proof. The inclusion M SI (p) ⊆ M I (p) follows trivially. Indeed when x ∈ R n is ICCfeasible it is e i -ICC feasible, where e i ∈ R k denotes the i-th standard unit vector and i = 1, ..., k is arbitrary. Now let x ∈ R n be such that x ∈ M (p). It then follows that a (measurable)-set Ξ ⊆ R m can be found with P[ξ ∈ Ξ] ≥ p and g(x, z) ≥ 0 for all z ∈ Ξ. As a consequence, v T g(x, z) ≥ 0 for all z ∈ Ξ and

v ∈ R k , v ≥ 0. Therefore x is v-ICC feasible for each v ∈ R k , v ≥ 0, i.e., x ∈ M SI (p).
The potential interest of Definition 3.1.1 is revealed by the following example:

Example 3.1.4. Let ξ ∈ R 2 be a random variable with a discrete distribution taking values (2, 1),(1, 1) and (1, 2) with probability 1 5 , 3 5 , 1 5 respectively. Consider the constraint

P[ξ ≤ x] ≥ 4 5 . It is then readily observed that M I (p) = (1, 1) + R 2 + and M (p) = (2, 1) + R 2 + ∪ (1, 2) + R 2 + . Taking v = ( 1 2 , 1 2 ), it is clear that P[v T ξ ≤ v T x] ≥ 4 5 if and only if v T x ≥ 3 2 . It readily follows that M SI (p) = (1, 1) + R 2 + ∩ x ∈ R 2 : v T x ≥ 3 2
= conv(M (p)). This example shows that in general the inclusions in Lemma 3.1.3 are strict.

The following Lemma is trivially derived upon noting that x ∈ R n is v-ICC feasible for v ∈ R k , v ≥ 0 if and only if x is λv-ICC feasible for an arbitrary λ > 0.

v ∈ R k , v ≥ 0 with v = 1.
The following results show that if the constraint of (3.1.1) is separable and ξ has a (non-degenerate) symmetric log-concave density, then M SI (p) = M I (p). Lemma 3.1.6. Let A be a m × n deterministic matrix and let g appearing in (3.1.1) be defined as g(x, z) = Axz. Assume moreover that the random variable ξ ∈ R m induces a non-degenerate probability measure P, with symmetric strictly positive log-concave density. Then for all p ∈ ( 1 2 , 1) we have M SI (p) = M I (p).

Proof. Let p ∈ ( 1 2 , 1) be fixed but arbitrary. Under the assumptions on ξ it follows that there exists a so called floating body for P ([51, Theorem 4.33], [START_REF] Meyer | Characterization of affinely-rotation invariant log-concave measures by section-centroid location[END_REF]). This means that there exists a convex compact set C p ⊆ R m such that

P[v T ξ ≥ s C p (v)] = 1 -p, (3.1.5)
where v ∈ R m is arbitrary and s C p (v) = sup h∈C p h T v is the support function of the set C p . Since ξ ∈ R m has a density (with respect to the Lebesgue measure) it does not allow for point masses. Hence we also have

P[v T ξ ≤ s C p (v)] = p. We now claim that x is v-ICC feasible if and only if v T (Ax) ≥ s C p (v). To show this claim, fix v ∈ R m , v ≥ 0 arbitrarily. If x is such that v T (Ax) ≥ s C p (v) then P[v T ξ ≤ v T (Ax)] ≥ P[v T ξ ≤ s C p (v)] = p and so x is v-ICC feasible. In contrast assume that x is v-ICC feasible, but that v T (Ax) < s C p (v).
We have made the assumption that the density of ξ is strictly positive. Hence the density of v T ξ is also strictly positive. In particular on the interval

[v T (Ax), s C p (v)] ⊆ R.
As a consequence of this and (3.1.5):

P[v T ξ ≥ v T (Ax)] > P[v T ξ ≥ s C p (v)] = 1 -p, (3.1.6)
i.e., P[v T ξ ≤ v T (Ax)] < p. This contradicts that x is v-ICC feasible. We have thus shown our claim.

Let v ∈ R m now be such that v 1 = 1 and v ≥ 0. Let x ∈ M I (p) be arbitrary. From the above characterization it follows that e T i (Ax) ≥ s C p (e i ) for i = 1, ..., m, where e i denotes the i-th standard unit vector of R m . Then,

v T Ax = m i=1 v i (e T i Ax) ≥ m i=1 v i (s C p (e i ) ≥ s C p ( i v i e i ),
where we have used the convexity of the mapping s C p . Using once again the above claim, we have shown that x is v-ICC feasible. Since v was arbitrary, upon using Lemma 3.1.5, we can conclude that x ∈ M SI (p). The result then follows upon using Lemma 3.1.3.

Lemma 3.1.7. Let A be a k × m deterministic matrix. Assume moreover that the random variable ξ ∈ R m induces a non-degenerate probability measure P, with symmetric log-concave density. Then for each p ∈ ( 1 2 , 1), there exists a convex compact set Ĉp ⊆ R k such that

P[v T Aξ ≥ s Ĉp (v)] = 1 -p, (3.1.7)
for all v / ∈ ker A T . Here s Ĉp : R k → R is the support function of the set Ĉp and ker A T denotes the null-space of the matrix A T .

Proof. Let p ∈ ( 1 2 , 1) be arbitrary. We can apply the floating body Theorem to the random variable ξ and thus establish equation (3.1.5) for a convex compact set C p ⊆ R m . We now define Ĉp = AC p ⊆ R k . This set is clearly convex and compact. It remains to establish (3.1.7). We begin by noting that for any v ∈ R k ,

s Ĉp (v) = sup k∈ Ĉp k T v = sup h∈C p (Ah) T v = sup h∈C p h T A T v. ( 3
P[v T (Aξ) ≥ s Ĉp (v)] = P[(A T v) T ξ ≥ s C p (A T v)] = 1 -p, since A T v ∈ R m is arbitrary.
We can now establish :

Theorem 3.1.8. Let A and B be deterministic matrices of sizes k × m and k × n respectively. Let g appearing in (3.1.1) be defined as g(x, z) = Bx-Az. Assume moreover that the random variable ξ ∈ R m induces a non-degenerate probability measure P, with symmetric strictly positive log-concave density. Then for all p ∈ ( 1 2 , 1) we have

M SI (p) = M I (p).
Proof. Under the assumptions of the Theorem, we can apply Lemma 3.1.7 to establish the existence of a compact convex set Ĉp ⊆ R k such that (3.1.7) holds. We now claim that x ∈ R n is v-ICC feasible if and only if v T (Bx) ≥ s Ĉp (v) for all v ∈ R k . For any v ∈ ker A T this relation is directly established from the definition of v-ICC feasibility upon noting that s Ĉp (v) = 0 in that case. We can thus restrain ourselves to v / ∈ ker A T . The proof of the claim is then identical to that of Lemma 3.1.6.

The claim having been established we can conclude the proof in a similar way as that of Lemma 3.1.6. Theorem 3.1.8 allows us to fully characterize the set M SI (p) for important constraints appearing in energy management applications: Corollary 3.1.9. Let g appearing in (3.1.1) be defined in such a way that (3.1.1) is equivalent with: Proof. The constraint appearing in (3.1.9) is equivalent with:

P[a + Ax ≤ ξ ≤ b + Bx] ≥ p, ( 3
P[ I -I ξ ≤ B (b -µ) -A (-a + µ) x] ≥ p, (3.1.10)
where x ∈ R n+1 , xn+1 = 1 and ξ ∈ R n is a centered multi-variate Gaussian random variable with positive definite covariance matrix Σ. The random vector ξ and constraint (3.1.10) satisfy the conditions of Theorem 3.1.8. We thus establish that M SI (p) = M I (p) as subsets of R n+1 . In particular for each x ∈ M I (p) with xn+1 = 1, we have x ∈ M SI (p).

We have thus shown that M SI (p) = M I (p) as was claimed.

We end this section with an explicit characterization of bilateral individual chance constraints in the setting of (3.1.9) when B = A.

i ≤ ξ i ≤ b i + y i ] ≥ p.
It then follows from p > 1 2 that a i + y i ≤ 0 and b i + y i ≥ 0. Hence y i ∈ [-b i , -a i ] and the latter set is bounded. Since ξ i is a non-degenerate Gaussian random variable, it follows from Theorem 2.5.13 that the set of e i -ICC feasible points is convex. As a matter of fact, even M I (p) is convex, since i = 1, ..., m is arbitrary. In particular if y 1 i and y 2 i (with y 1 i < y 2 i ) are two e i -ICC feasible points it follows from convexity that any point in the interval [y 1 i , y 2 i ] is also e i -ICC feasible. Since ξ i does not have point-masses a smallest feasible y i can be found, i.e., y(p) i := min {y i : P[a i + y i ≤ ξ i ≤ b i + y i ] ≥ p} is well-defined. We can define y(p) i similarly. The lemma has been shown. The strength of Lemma 3.1.10 is that the box [y(p), y(p)] can be computed numerically with very high precision with for instance a dichotomy procedure. For each i = 1, ..., m, the above proof shows that y(p) i ∈ [-b i , -a i ]. By iteratively dividing this interval in two and evaluating the Gaussian distribution function in dimension 1 we can compute y(p) i up to (nearly) arbitrary precision. Lemma 3.1.10 has the advantage of providing an explicit characterization of M I (p) that does not require yet a weaker characterization involving individual unilateral constraints.

Efficient Gradient Formulae : bilateral separable probabilistic constraints

In energy management, probabilistic constraints of the type (2.0.1) arise naturally, but have a special structure. We refer to [START_REF] Van Ackooij | Early evaluation of chanceconstrained programming for energy management optimization problems[END_REF][START_REF] Van Ackooij | Chance Constrained Programming and Its Applications to Energy Management[END_REF] for an investigation of these structures.

Often, the random inequality system g(x, ξ) ≥ 0 appearing in (2.0.1) is a linear inequality system A(ξ)x ≤ b(ξ), where A(ξ) and/or b(ξ) are random. One even naturally encounters an inequality system of the form

a + Ax ≤ ξ ≤ b + Bx, (3.2.1)
where ξ ∈ R m is a random vector and the vectors a, b ∈ R m and m × n matrices A, B are deterministic. A system of the form (3.2.1) is encountered in cascaded reservoir management. Then, ξ represents random inflows and the constraint the fact that the volume in a hydro-reservoir has to remain between a lower and upper bound. More details can be found in section 5.1.

The probabilistic constraint of the form (2.0.1) associated with system (3.2.1) is:

P[a + Ax ≤ ξ ≤ b + Bx] ≥ p. (3.2.2)
A first question of interest is the differentiability with respect to x of the constraint appearing in (3.2.2). For instance if ξ ∈ R m is assumed to follow a non-degenerate multivariate Gaussian distribution. Of course, such a bilateral system can be mapped to a unilateral one as follows:

I -I ξ ≤ B -A x + b -a . (3.2.3)
According to the reformulation (3.2.3), one might be tempted to apply Lemma 2.7.5 in order to derive a formula for the derivative. However the random vector

η := I -I ξ ∈ R 2m
appearing in (3.2.3) is degenerate. Therefore, Lemma 2.7.5 can not be applied. Clearly the recent Theorem 2.7.6 would directly lead to a formula for the derivative. We report here an earlier derived result ( [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF]) that has the advantage of not requiring the evaluation of probabilities in dimension 2m contrary to the result appearing in Theorem 2.7.6.

Lemma 3.2.1. Let ξ be an m-dimensional Gaussian random vector with mean µ ∈ R m and positive definite variance-covariance matrix Σ. We define the mapping F ξ (a, b) := P[a ≤ ξ ≤ b] for any rectangle, i.e., a ≤ b. Let D i m denote the m-th order identity matrix from which the ith row has been deleted. For each y ∈ R m , 1 ≤ i ≤ m and z ∈ R we define

y c i (z) = D i m (y + Σ -1 i,i (z -y i )Σ i ) ∈ R m-1 , where Σ i is the ith column of Σ. Also define Σ c m (i) = D i m (Σ -Σ -1 i,i Σ i Σ T i )(D i m ) T .
Moreover, let f ν,σ (x) be the one-dimensional Gaussian density with mean ν and variance σ. Then for arbitrary but fixed i = 1, ..., m, we have:

∂ ∂b i F ξ (a, b) = f µ i ,σ ii (b i )F ξ i (b i ) (D i m a, D i m b) ∂ ∂a i F ξ (a, b) = -f µ i ,σ ii (a i )F ξ i (a i ) (D i m a, D i m b),
where

ξ i (z) ∼ N (µ c i 1 (z) , Σ c m (i) ) ∈ R m-1 .
Proof. It is well known (e.g., [START_REF] Billingsley | Probability and measure[END_REF]) that

F ξ (a, b) = j 1 ,...,j m ∈{0,1} (-1) m+ m k=1 j k Φ ξ (y j 1 , ..., y j m ), (3.2.4)
where Φ ξ : R m → [0, 1] is the cumulative distribution function of ξ and

y j k = a k if j k = 0 b k if j k = 1.
Let i = 1, ..., m be arbitrary but fixed and consider the partial derivative of F ξ (a, b) with respect to b i . Then according to (3.2.4) we have:

∂ ∂b i F ξ (a, b) = j 1 ,...,j m ∈{0,1}
(-1) m+ m k=1 j k ∂ ∂b i Φ ξ (y j 1 , ..., y j m ). Now if j i = 0 then b i / ∈ {y j 1 , ..., y j m } and ∂ ∂b i Φ ξ (y j 1 , ..., y j m ) = 0. On the other hand, if j i = 1, we derive by applying Lemma 2.7.5 that:

∂ ∂b i Φ ξ (y j 1 , ..., y j m ) = ∂ ∂b i Φ ξ (y j 1 , ..., y j i-1 , b i , y j i+1 ..., y j m ) = f µ i ,σ ii (b i )Φ ξ i (b i ) (y j 1 , ..., y j i-1 , y j i+1 ..., y j m ), where ξ i (b i ) ∼ N (µ c i 1 (b i ) , Σ c m (i) ) ∈ R m-1 is a Gaussian random variable.
Upon combining this result with (3.2.4) we derive:

∂ ∂b i F ξ (a, b) = = j 1 ,...,j i-1 ,j i+1 ,...,j m ∈{0,1} (-1) m+1+ m k=1,k =i j k f µ i ,σ ii (b i )Φ ξ i (b i ) (y j 1 , ..., y j i-1 , y j i+1 ..., y j m ) = f µ i ,σ ii (b i ) j 1 ,...,j i-1 ,j i+1 ,...,j m ∈{0,1} (-1) m-1+ m k=1,k =i j k Φ ξ i (b i ) (y j 1 , ..., y j i-1 , y j i+1 ..., y j m ) = f µ i ,σ ii (b i )F ξ i (b i ) (D i m a, D i m b).
The asserted formula for the partial derivative with respect to a i follows similarly upon noting that a i / ∈ {y j 1 , ..., y j m } when j i = 1.

This result can be readily extended to derive a Formula for the Hessian ( [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF]):

Lemma 3.2.2. Let ξ be an m-dimensional Gaussian random vector with mean µ ∈ R m and positive definite variance-covariance matrix Σ. We define the mapping F ξ (a, b) := P[a ≤ ξ ≤ b] for any rectangle, i.e., a ≤ b. Let D i m denote the m-th order identity matrix from which the ith row has been deleted. For each y ∈ R m , 1 ≤ i ≤ m and z ∈ R we define

y c i,m (z,Σ i ) = D i m (y + Σ -1 i,i (z -y i )Σ i ) ∈ R m-1
, where Σ i is the ith column of Σ. We will occasionally abbreviate this with y c i 1 (z) . We also define

y c i,j 2 (z,w) = (y c i,m (z,Σ i ) ) c j,m-1 (w,Σ c m (i) j ) ,
where we have defined

Σ c m (i) = D i m (Σ-Σ -1 i,i Σ i Σ T i )(D i m ) T .
We define ξ c i 1 (z) as the Gaussian random variable with mean µ c i 1 (z) and covariance matrix Σ c m (i) . In a similar way, we define ξ c i,j 2 (z,w) as the Gaussian random variable with mean µ c i,j 2 (z,w) and covariance matrix

Σ c i,j 2 := D j m-1 (Σ c m (i) -(Σ c m (i) j,j ) -1 Σ c m (i) j (Σ c m (i) j ) T )(D j m-1 ) T , where Σ c m (i) j
denotes the j-th column of Σ c m (i) . The following holds, for j = ĵ if ĵ < i and j = ĵ -1 if ĵ > i:

∂ 2 ∂a ĵ ∂a i F ξ (a, b) = f µ c i 1 (a i ) j ,Σ c m (i) j,j (a j )f µ i ,Σ i,i (a i )F ξ c i,j ∂ 2 ∂b ĵ ∂a i F ξ (a, b) = -f µ c i 1 (a i ) j ,Σ c m (i) j,j (b j )f µ i ,Σ i,i (a i )F ξ c i,j 2 (a i ,b j ) (D j m-1 D i m a, D j m-1 D i m b) ∀ ĵ = i 0 ĵ = i ∂ 2 ∂b ĵ ∂b i F ξ (a, b) = f µ c i 1 (b i ) j ,Σ c m (i) j,j (b j )f µ i ,Σ i,i (b i )F ξ c i,j 2 (b i ,b j ) (D j m-1 D i m a, D j m-1 D i m b) ∀ ĵ = i,
where f ν,σ (x) is the one-dimensional Gaussian density with mean ν and variance σ. Moreover, whenever j = i and z is a or b we have:

∂ ∂z i (b i -a i ) ∂ 2 ∂z 2 i F ξ (a, b) = - z i -µ i Σ i,i f µ i ,Σ i,i (z i )F ξ c i 1 (z i ) (D i m a, D i m b) -f µ i ,Σ i,i (z i )(D i m Σ -1 i,i Σ i ) T (∇ ãF ξc i 1 (z i ) (ã, b) + ∇ bF ξc i 1 (z i ) (ã, b)), where ã = D i m a -µ c i 1 (z i ) , ξc i 1 (z i ) = ξ c i 1 (z i ) -µ c i 1 (z i )
and b is defined similarly.

Proof. The formula for the cross derivatives follow from a straight-forward second application of Lemma 3.2.1. The diagonal terms are more subtle to derive and require the following reformulation:

F ξ c i 1 (z i ) (D i m a, D i m b) = P D i m a ≤ ξ c i 1 (z i ) ≤ D i m b = P D i m a -µ c i 1 (z i ) ≤ ξ c i 1 (z i ) -µ c i 1 (z i ) ≤ D i m b -µ c i 1 (z i ) = F ξc i 1 (z i ) (ã, b). In particular one obtains for ã(z i ) = ã ã = D i m a -µ c i 1 (z i ) = D i m a -D i m (µ + Σ -1 i,i (z i -µ i )Σ i ) = D i m (a -µ + Σ -1 i,i µ i Σ i ) -D i m Σ -1 i,i z i Σ i ,
which together with the following identity

∂ ∂z i F ξc i 1 (z i ) (ã(z i ), b(z i )) = ∇ ãF ξc i 1 (z i ) (ã, b)D z i ã(z i ) + ∇ bF ξc i 1 (z i ) (ã, b))D z i b(z i ),
an application of the chain-rule and the already established formula for 1st derivatives gives the proposition.

Considering the probability constraint (3.2.2), its derivative can be computed according to the following Corollary:

Corollary 3.2.3. Let ϕ : R n → [0, 1] be defined as ϕ(x) := P[a + Ax ≤ ξ ≤ b + Bx],
where ξ ∈ R m is a Gaussian random variable with mean µ ∈ R m and positive definite variance-covariance matrix Σ. Moreover a, b, A, B are as in (3.2.2). Then the mapping ϕ is twice differentiable and we have:

∇ϕ = ∇ a F ξ (a, b) T A + ∇ b F ξ (a, b) T B ∇ 2 ϕ = A T ∇ 2 aa F ξ (a, b)A + A T ∇ 2 ab F ξ (a, b)B + B T ∇ 2 ba F ξ (a, b)A + B T ∇ 2 bb F ξ (a, b)B,
where F ξ is defined as in Lemma 3.2.1.

One can also compute the derivative of a chance constraint wherein the inequality system g(x, ξ) ≥ 0 of (2.0.1) is a linear inequality system A(ξ)x ≤ b with A(ξ) having a multivariate Gaussian distribution. The following results was derived in [START_REF] Van Ackooij | On joint probabilistic constriants with Gaussian Coefficient Matrix[END_REF] in a (slightly) more general form Theorem 3.2.4. Consider the mapping ϕ : R n → [0, 1] defined as ϕ(x) = P[A(ξ)x ≤ α(x)], where α : R n → R m is a differentiable mapping and A(ξ) a m×n matrix, with components following a multi-variate Gaussian random variable in R m×n with mean (matrix) µ ∈ R m×n and positive definite mn × mn covariance matrix Σ. Let x = 0 be arbitrary and fixed. Then η(x) := A(ξ)x ∈ R m is a non-degenerate multivariate Gaussian random variable with mean µ(x) ∈ R m , covariance matrix Σ(x) and correlation matrix R(x).

Define D(x) ∈ R m as the diagonal of Σ(x) and D(x) as the diagonal matrix with vector D(x) -1 2 on its diagonal. The latter operation is to be understood element by element. Define the mapping β : R n → R m as β(x) = D(x)(α(x)µ(x)). Then ϕ is differentiable at x and we have :

∇ϕ(x) = - m i=1 f 0,R ii (x) (β i (x))P[ ξi (x) ≤ D i m β(x)](α i (x) -µ i (x)) 1 2 Σ -3 2 ii (x)Σ ii x + m i=1 f 0,R ii (x) (β i (x))P[ ξi (x) ≤ D i m β(x)]Σ -1 2 ii (x)(∇α i (x) -µ i ) + 1≤i<j≤m Σ -3 2 ii (x)Σ -3 2 jj (x)[Σ ii (x)Σ jj (x)Σ ii x - 1 2 Σ ij (x)[Σ jj (x)Σ ii x + Σ ii (x)Σ jj x] • • f 0, Ri j-1,j-1 (x) (β j (x) -μi j-1 (x))f 0,R ii (x) (β i (x))P[ ξij (x) ≤ D j-1 m-1 D i m (β(x) -μi (x))]
where for i = 1, ..., m, D i m is defined as in Lemma 2.7.5, µ i denotes the i-th row of matrix µ and ξi (x) ∈ R m-1 is a multi-variate Gaussian random variable with mean μi (x) :

= D i m R -1 ii (x)β i (x)R i (x) and covariance matrix Ri (x) := D i m (R(x) -R -1 ii (x)R i (x)R i (x) T )(D i m ) T ,
where R i (x) denotes the i-th column of the matrix R(x). Moreover f 0,σ (z) denotes the density of a 1-dimensional Gaussian random variable with variance σ evaluated in z ∈ R.

Finally, for i, j = 1, ..., m ξij (x) ∈ R m-2 is a non-degenerate Gaussian random variable with mean D j-1 m-1 (( Ri j-1,j-1 (x)) -1 (β j (x)μi j-1 (x)) Ri j-1 (x) and covariance matrix

D j-1 m-1 ( Ri (x) -Ri j-1,j-1 (x)) -1 Ri j-1 (x)( Ri j-1 (x)) T .
Again Ri j-1 (x) denotes the j -1-st column of matrix Ri (x). The matrix Σ ij appearing in the above expression is an n × n matrix defined as

Σ ij lk = Σ (i-1)n+l,(j-1)n+k , 1 ≤ l, k ≤ n.
Proof. It is convenient to reformulate ϕ. To this end we introduce the mapping T : R n → R m×n :

T (x) =      x T 0 • • • 0 0 x T • • • 0 . . . • • • • • • . . . 0 0 • • • x T      .
We also define the following matrix operation . ⊙ : R m×n → R mn , defined as: T . This means that for any 1

A ⊙ =          A 11 . . . A 1n A 21 . . . A mn          . It then readily follows that ϕ(x) = P[T (x)ξ ⊙ ≤ α(x)]. Now η(x) := A(ξ)x = T (x)ξ ⊙ follows a multi-variate Gaussian distribution with mean µ(x) = T (x)µ ⊙ = µx ∈ R m and m × m covariance matrix Σ(x) = T (x)ΣT (x)
≤ i, j ≤ m one has Σ ij (x) = x T Σ ij x, where Σ ij is the n × n matrix defined as Σ ij lk = Σ (i-1)n+l,(j-1)n+k , 1 ≤ l, k ≤ n.
Let R(x) = D(x)Σ(x)D(x). It is then easily observed that R(x) is a correlation matrix and that ϕ is equivalent with ϕ(x) = Φ R(x) (β(x)), where Φ R (z) denotes the m dimensional multivariate Gaussian distribution function with correlation matrix R evaluated at z ∈ R m . Let w ∈ R m be such that w = 0 but otherwise arbitrary. It is easily observed that T (x) T w = (w 1 x, ..., w m x) ∈ R mn and as a consequence for x = 0, Σ(x) is positive definite. We have thus shown that η(x) is non-degenerate as claimed. Now define the mapping γ : R m×m × R m → [0, 1] as γ(R, z) = Φ R (z). This mapping is continuously differentiable (when R is positive definite) and we establish in this manner

∇ϕ(x) = m i=1 ∂γ ∂z i (R(x), β(x))∇β i (x) + 1≤i<j≤m ∂γ ∂R ij (R(x), β(x))∇R ij (x).
(3.2.5)

Let i = 1, ..., m be arbitrary. Differentiating β i (x) leads to

∇β i (x) = (α i (x) -µ i (x))∇Σ -1 2 ii (x) + Σ -1 2 ii (x)(∇α i (x) -∇µ i (x)). (3.2.6)
We also readily derive

∇µ i (x) = (µ i1 , ..., µ in ) = µ i ∇Σ -1 2 ii (x) = - 1 2 Σ -3 2 ii (x)∇Σ ii (x) = - 1 2 Σ -3 2 ii (x)Σ ii x.
The derivative ∂γ ∂z i (R(x), β(x)) is easily computed by applying Lemma 2.7.5 and we establish that

∂γ ∂z i (R(x), β(x)) = f 0,R ii (x) (β i (x))P[ ξi (x) ≤ D i m β(x)], (3.2.7)
where ξi (x) ∈ R m-1 is a multi-variate Gaussian random variable with mean μi (x) and covariance matrix Ri (x).

One can note that

R ij (x) = Σ -1 2 ii (x)Σ -1 2 jj (x)Σ ij (x)
for i, j ∈ {1, ..., m}. Taking the derivative of this expression leads to

∇R ij (x) = Σ -3 2 ii (x)Σ -3 2 jj (x)[Σ ii (x)Σ jj (x)Σ ii x - 1 2 Σ ij (x)[Σ jj (x)Σ ii x + Σ ii (x)Σ jj x]
Let 1 ≤ i < j ≤ m be given. From Gupta's formula [START_REF] Gupta | Probability integrals of multivariate normal and multivariate t[END_REF] we derive that

∂γ ∂R ij (R(x), β(x)) = ∂ 2 γ ∂z i ∂z j (R(x), β(x)). (3.2.8)
The latter second derivative can be computed by using Lemma 3.2.2 above by setting a = -∞. We then establish that

∂ 2 γ ∂z i ∂z j (R(x), β(x)) = = f 0, Ri j-1,j-1 (x) (β j (x) -μi j-1 (x))f 0,R ii (x) (β i (x))P[ ξij (x) ≤ D j-1 m-1 D i m (β(x) -μi (x))].
Combining the above inequalities then yields the result.

Efficient Gradient Formulae : nonlinear probabilistic constraints with Gaussian and Gaussianlike distributions

A probabilistic constraint is an inequality of the type

P[g(x, ξ) ≥ 0] ≥ p, (3.3.1)
where g is a mapping defining a (random) inequality system and ξ is an s-dimensional random vector defined on some probability space (Ω, A, P). The constraint (3.3.1) expresses the requirement that a decision vector x is feasible if and only if the random inequality system g(x, ξ) ≥ 0 is satisfied at least with probability p ∈ [0, 1].

From a formal viewpoint, (3.3.1) is a conventional inequality constraint α(x) ≥ p with α(x) := P[g(x, ξ) ≥ 0]. On the other hand, a major difficulty arises from the fact that typically no analytical expression is available for α. All one can hope for, in general, are tools for numerically approximating α. Beyond crude Monte Carlo estimation of the probability defining α, there exist a lot of more efficient approaches based, for instance, on graph-theoretical arguments [START_REF] Bukszár | Hypermultitrees and sharp Bonferroni inequalities[END_REF], variance reduction [START_REF] Szántai | Improved bounds and simulation procedures on the value of the multivariate normal probability distribution function[END_REF], Quasi-Monte-Carlo (QMC) techniques or sparse grid numerical integration [START_REF] Geletu | Monotony analysis and sparsegrid integration for nonlinear chance constrained process optimization[END_REF]. It seems, however, that such approaches are most successful when exploiting the special model structure (i.e., the mapping g and the distribution of ξ). For instance, in the special case of separable constraints g(x, ξ) = xξ and of ξ having a regular Gaussian distribution (such that α reduces to a multivariate Gaussian distribution function), Genz [START_REF] Genz | Numerical computation of multivariate normal probabilities[END_REF][START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF] developped a numerical integration scheme combining separation and reordering of variables with randomized QMC.

Using this method, one may compute values of the Gaussian distribution function at fairly good precision in reasonable time even for a few hundred random variables. A similar technique has been proposed for the multivariate Student (or T-) distribution [START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF]. The numerical evaluation of other multivariate distribution functions such as Gamma or exponential distribution has been discussed, e.g., in [START_REF] Szántai | Evaluation of a special multivariate gamma distribution[END_REF][START_REF] Olieman | Estimation method of multivariate exponential probabilities based on a simplex coordinates transform[END_REF].

For an efficient solution of probabilistically constrained problems via numerical nonlinear optimization it is evidently not sufficient to calculate just functional values of α, one also has to have access to gradients of α. The need to calculate gradients of probability functions has been recognized a long time ago and has given rise to many papers on representing such gradients (e.g., [START_REF] Marti | Differentiation of probability functions : The transformation method[END_REF], [START_REF] Uryas'ev | Derivatives of probability functions and integrals over sets given by inequalities[END_REF], [START_REF] Kibzun | Differentiability of probability function[END_REF], [START_REF] Pflug | Probability gradient estimation by set-valued calculus and applications in network design[END_REF], [START_REF] Garnier | Asymptotic formulas for the derivatives of probability functions and their Monte Carlo estimations[END_REF]). In the separable case with Gaussian distribution mentioned above, it is well-known [181, p. 203], that partial derivatives of α can be reduced analytically to function values α of a Gaussian distribution with different parameters. This fact has three important consequences: first it allows one to employ the same efficient method (e.g. by Genz) available for values of Gaussian distribution functions in order to compute gradients simultaneously; second, doing so, the error in calculating ∇a can be controlled by that in caculating α [START_REF] Henrion | Gradient estimates for Gaussian distribution functions: Application to probabilistically constrained optimization problems[END_REF]; third, the mentioned analytic relation can be applied inductively, in order to get similar analytic relations between function values and higher-order derivatives. Fortunately, this very special circumstance can be extended to more general models: it has been demonstrated in [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF][START_REF] Van Ackooij | On joint probabilistic constriants with Gaussian Coefficient Matrix[END_REF][START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF] how for general linear probabilistic constraints α(x) := P[T (x)ξ ≤ a(x)] ≥ p under Gaussian distribution and with possibly nonregular, nonquadratic matrix T (x) not only the computation of α (which is evident) but also of ∇a can be analytically reduced to the computation of Gaussian distribution functions. Combining appropriately these ideas with Genz' code and an SQP solver, it is possible to solve corresponding optimization problems for Gaussian random vectors in dimension of up to a few hundred (where the dimension of the decision vector x is less influential). Applications to various problems of power management can be found, e.g., in [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF][START_REF] Van Ackooij | On joint probabilistic constriants with Gaussian Coefficient Matrix[END_REF][START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF][START_REF] Arnold | A mixed-integer stochastic nonlinear optimization problem with joint probabilistic constraints[END_REF][START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF]. It seems that the same approach can be elaborated also for the multivariate Student distribution, whereas it would work for the Log-normal distrbution only in the special case of α(x

) = P[b(x) ≤ ξ ≤ a(x)].
When considering models which are nonlinear in ξ, a reduction to distribution functions seems not to be possible any more. In that case, another approach, the so-called sphericalradial decomposition of Gaussian random vectors (see, e.g., [START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF]) seems to be promising both for calculating function values and gradients of α. More precisely, let ξ be an m-dimensional random vector normally distributed according to ξ ∼ N (0, R) for some correlation matrix R. Then, ξ = ηLζ, where R = LL T is the Cholesky decomposition of R, η has a chi-distribution with m degrees of freedom and ζ has a uniform distribution over the Euclidean unit sphere

S m-1 := z ∈ R m m i=1 z 2 i = 1 of R m .
As a consequence, for any Lebesgue measurable set M ⊆ R m its probability may be represented as

P[ξ ∈ M ] = v∈S m-1 µ η ({r ≥ 0 : rLv ∩ M = ∅}) dµ ζ , (3.3.2)
where µ η and µ ζ are the laws of η and ζ, respectively. This probability can be numerically computed by employing an efficient sampling scheme on S m-1 proposed by Deák [START_REF] Deák | Computing probabilities of rectangles in case of multinormal distribution[END_REF][START_REF] Deák | Subroutines for computing normal probabilities of sets -computer experiences[END_REF]. More generally, one may approximate the integral

v∈S m-1 h(v)dµ ζ (3.3.3)
for any Lebesgue measurable function h : S m-1 → R. In particular, for

h(v) := µ η ({r ≥ 0 : rLv ∩ M = ∅}) ,
we obtain the probability (3.3.2). In this section, we will show how -with different functions h(v) -the same efficient sampling scheme can be employed in order to simultaneously compute derivatives of this probability with respect to an exterior parameter.

The results may serve as a basis for a numerical treatment of nonlinear convex probabilistic constraints with Gaussian and alternative distributions via nonlinear optimization.

The formula, for the Gaussian case, that we provide in this paper was provided in a somewhat similar form in [58, Section 9.2], but without proof. A formula similar in idea is given in [START_REF] Royset | Implementable algorithm for stochastic optimization using sample average approximations[END_REF] under an implicit assumption that M (x) := {z ∈ R m : g(x, z) ≤ 0}, appearing in (3.3.2) is bounded, 0 ∈ int M (x) and an assumption that the gradients of g with respect to z are "non-degenerate" on the boundary of M (x). That latter means that these gradients are not orthogonal to any ray starting at 0 hitting the boundary of M (x). In the proof, differentiation and integration are interchanged without further comments. In the later work [START_REF] Royset | Extensions of stochastic optimization results to problems with system failure probability functions[END_REF] the same authors provide a formula identical to ours. Again for the Gaussian case only. They moreover explicitly assume that M (x) is bounded and that 0 ∈ int M (x). They also request that the gradients of g with respect to z are non-degenerate in the above way and make an additional µ ζ zero measure assumption.

The last assumption means that the set of directions v ∈ S m-1 hitting the boundary of M (x) in points where more than one component of g is active, has zero measure. This last assumption is needed to derive a multi-variate version of the here provided formula. However, the boundedness assumption is restrictive. For example, it rules out the use of the formula for distribution functions. In this paper most work is devoted to the case when M (x) is not assumed to be bounded. We moreover show that the gradients of g with respect to z are "non-degenerate" in the above way. If we would make the same zero-measure assumption as in [START_REF] Royset | Extensions of stochastic optimization results to problems with system failure probability functions[END_REF] our results would moreover directly extend to the multi-variate case. However such a zero-measure assumption can not hold in general as can be observed in [109, Figure 1]. We will therefore restrain ourselves, for now, to the case wherein g has a single component only.

In Section 3.3.1, a rigorous justification for differentiating under the integral sign will be given. Doing so, we arrive at sufficient conditions for continuous differentiability of probability functions in the concave and Gaussian case as well as at an explicit integrand in (3.3.3). In Section 3.3.2, the obtained results are applied to various examples involving Gaussian and alternative distributions. Particular attention is paid to the multivariate Student distribution.

A gradient formula for parameter-dependent Gaussian probabilities in the convex case

In the following, we assume that g : R n ×R m → R is a continuously differentiable function which is concave with respect to the second argument. We define

ϕ (x) := P[g(x, ξ) ≥ 0], (3.3.4)
where ξ ∼ N (0, R).

Remark 3.3.1. We recall that convex sets are Lebesgue measurable so that the probabilities in (3.3.4) are well-defined by virtue of ξ having a density. where g has the same properties as g (it is continuously differentiable and concave with respect to the second argument). Therefore, in (3.3.4), we may indeed assume without loss of generality, that ξ ∼ N (0, R).

By (3.3.2) and (3.3.4), we have, for all x ∈ R n , that

ϕ (x) = v∈S m-1 µ η ({r ≥ 0 : g(x, rLv) ≥ 0}) dµ ζ = v∈S m-1 e(x, v)dµ ζ (3.3.5) for e(x, v) := µ η ({r ≥ 0 : g(x, rLv) ≥ 0}) ∀x ∈ R n ∀v ∈ S m-1 . (3.3.6)
According to the possibility of evaluating (3.3.3) for instance by Deàk's method, we can obtain a value ϕ (x) for each fixed x. We now address the computation of ∇ϕ. It is convenient to introduce the following two mappings F, I : R n ⇒ S m-1 of directions with finite and infinite intersection length:

F (x) := v ∈ S m-1 |∃r > 0 : g (x, rLv) = 0 I(x) := v ∈ S m-1 |∀r > 0 : g (x, rLv) = 0 .
The following Lemma collects some elementary properties needed later:

Lemma 3.3.3. Let x ∈ R n be such that g(x, 0) > 0. Then, 1. v ∈ I(x) if and only if g (x, rLv) > 0 for all r > 0.

2. F (x) ∪ I(x) = S m-1 .

3. For v ∈ F (x) let r > 0 be such that g (x, rLv) = 0. Then,

∇ z g (x, rLv) , Lv ≤ - g (x, 0) r . 4. If v ∈ I(x) then e(x, v) = 1
, where e is defined in (3.3.6).

Proof. 1. follows from the continuity of g and 2. is evident from the definitions. The convexity of -g with respect to the second argument yields

1 2 r ∇ z g (x, rLv) , Lv = -∇ z g (x, rLv) , 1 2 rLv -rLv ≤ g (x, rLv) -g x, 1 2 rLv = -g x, 1 2 rLv ≤ - 1 2 g (x, 0) - 1 2 g (x, rLv) = - 1 2 g (x, 0) .
This proves 3. If v ∈ I(x) then e(x, v) = µ η (R + ) = 1 because R + is the support of the chi-distribution. Therefore, 4. holds true.

Next, we provide a local representation of the factor r as a function of x and v: Lemma 3.3.4. Let (x, v) be such that g(x, 0) > 0 and v ∈ F (x). Then, there exist neighbourhoods U of x and V of v as well as a continuously differentiable function ρ x,v : U × V → R + with the following properties:

1. For all

(x ′ , v ′ , r ′ ) ∈ U × V × R + the equivalence g(x ′ , r ′ Lv ′ ) = 0 ⇔ r ′ = ρ x,v (x ′ , v ′ ) holds true.
2. For all (x ′ , v ′ ) ∈ U × V one has the gradient formula

∇ x ρ x,v (x ′ , v ′ ) = - 1 ∇ z g(x ′ , ρ x,v (x ′ , v ′ )Lv ′ ), Lv ′ ∇ x g(x ′ , ρ x,v (x ′ , v ′ )Lv ′ ).
Proof. By definition of F (x) we have that g (x, rLv) = 0 for some r > 0. According to 3. in Lemma 3.3.3, we have that

∇ z g (x, rLv) , Lv ≤ - g (x, 0) r < 0.
This allows to apply the Implicit Function Theorem to the equation g (x, rLv) = 0 and to derive the existence of neighbourhoods U of x, V of v and W of r along with a continuously differentiable function ρ x,v : U × V → W , such that the equivalence

g(x ′ , r ′ Lv ′ ) = 0, (x ′ , v ′ , r ′ ) ∈ U × V × W ⇔ r ′ = ρ x,v (x ′ , v ′ ), (x ′ , v ′ ) ∈ U × V (3.3.7)
holds true. By continuity of ρ x,v , we may shrink the neighbourhoods U and V such that ρ x,v maps into R + and we may further shrink U such that g(x ′ , 0) > 0 for all x ′ ∈ U . Now, assume that g(x ′ , r * Lv ′ ) = 0 holds true for some (x

′ , v ′ , r * ) ∈ U × V × (R + \W ). Then, by '⇐' in (3.3.7), g(x ′ , ρ x,v (x ′ , v ′ )Lv ′ ) = 0, where ρ x,v (x ′ , v ′ ) ∈ W . Consequently, r * = ρ x,v (x ′ , v ′ ).
On the other hand, r * , ρ x,v (x ′ , v ′ ) ∈ R + . This contradicts the concavity of g with respect to the second argument and the fact that g(x ′ , 0) > 0. It follows that in (3.3.7) W may be replaced by R + which proves 1. In particular, we have that g(x ′ , ρ x,v (x ′ , v ′ )Lv ′ ) = 0 for all (x ′ , v ′ ) ∈ U × V , which after differentiation gives the formula in 2.

The preceding Lemma allows us to observe the following:

Lemma 3.3.5. Let x ∈ R n be such that g(x, 0) > 0. Then, 1. If v ∈ F (x) then there exist neighbourhoods U of x and V of v such that e(x ′ , v ′ ) = F η (ρ x,v (x ′ , v ′ )) for all (x ′ , v ′ ) ∈ U × V
, where e is defined in (3.3.6), F η is the cumulative distribution function of the chi-distribution with m degrees of freedom and ρ x,v refers to the resolving function introduced in Lemma 3.3.4.

If

v ∈ I(x) then ρ x k ,v k (x k , v k ) → ∞ for any sequence (x k , v k ) → (x, v) with v k ∈ F (x k ).
Proof. By 1. in Lemma 3.3.4, we have for all (x ′ , v ′ ) that g(x ′ , ρ x,v (x ′ , v ′ )Lv ′ ) = 0 and g(x ′ , r ′ Lv ′ ) = 0 for all r ′ ∈ R + with r ′ = ρ x,v (x ′ , v ′ ). Now, (3.3.6) implies that

e(x ′ , v ′ ) = µ η ([0, ρ x,v (x ′ , v ′ )]) = F η (ρ x,v (x ′ , v ′ )) -F η (0) ∀ (x ′ , v ′ ) ∈ U × V.
Now, 1. follows upon observing that the chi-density is zero for negative arguments, whence

F η (0) = 0. Next, let v ∈ I(x) and (x k , v k ) → (x, v) with v k ∈ F (x k ). If not ρ x k ,v k (x k , v k ) → ∞,
then there exists a converging subsequence ρ x k l ,v k l (x k l , v k l ) → r for some r ≥ 0. Since g(x, 0) > 0, we have that g (x k l , 0) > 0 for l sufficiently large. This allows us to apply Lemma 3.3.4 to the points (x k l , v k l ), and so we infer from 1. in this Lemma that g (x k l , ρ x k l ,v k l (x k l , v k l ) Lv k l ) = 0 for all l sufficiently large. By continuity of g we derive the contradiction g (x, rLv) = 0 with our assumption v ∈ I(x). This proves 2.

Corollary 3.3.6. The function e : R n × S m-1 → R defined in (3.3.6) is continuous at any (x, v) ∈ R n × S m-1 such that g(x, 0) > 0.

Proof. Let (x, v) ∈ R n × S m-1 with g(x, 0) > 0 be arbitrarily given. Referring to the sets F (x) and I(x) characterized in Lemma 3.3.3, there are two possibilities: if v ∈ F (x), then the function ρ x,v is defined on a neighbourhood of (x, v) and is continuous there by Lemma 3.3.4. Moreover, in this case, e has the representation given in 1. of Lemma 3.3.5. But with the cumulative distribution function F η of the chi-distribution being continuous, e is continuous too at (x, v) as a composition of continuous mappings. If, in contrast, v / ∈ F (x), then v ∈ I(x) by 2. of Lemma 3.3.3. From 4. of the same Lemma we know that e(x, v) = 1. Consider an arbitrary sequence (x k , v k ) → (x, v) with v k ∈ S m-1 . Since g(x, 0) > 0, we have that g(x k , 0) > 0 for k sufficiently large. Assume that not e (x k , v k ) → 1. Then, there is a subsequence (x k l , v k l ) and some ε > 0 such that for all l

|e (x k l , v k l ) -1| ≥ ε. (3.3.8) By 4. in Lemma 3.3.3, v k l / ∈ I(x k l ), whence v k l ∈ F (x k l
) for all l due to v k l ∈ S m-1 and 2. in Lemma 3.3.3. Then, ρ x k l ,v k l (x k l , v k l ) → ∞ by 2. of Lemma 3.3.5. Since F η is the distribution function of a random variable, it satisfies the relation lim t→∞ F η (t) = 1. Consequently, we may invoke 1. of Lemma 3.3.5 in order to verify that

lim l→∞ e (x k l , v k l ) = lim l→∞ F η (ρ x k l ,v k l (x k l , v k l )) = 1.
This contradicts (3.3.8) and, hence, again by 4. in Lemma 3.3.3,

lim k→∞ e (x k , v k ) = 1 = e(x, v).
This proves continuity of e at (x, v). Corollary 3.3.7. For any x ∈ R n with g(x, 0) > 0 and v ∈ F (x) the partial derivative w.r.t x of the function e : R n × S m-1 → R defined in (3.3.6) exists and is given by

∇ x e(x, v) = - χ (ρ x,v (x, v)) ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv)
where χ is the density of the chi-distribution with m degrees of freedom and ρ x,v refers to the function introduced in Lemma 3.3.4.

Proof. By 1. in Lemma 3.3.5 we have that e(x ′ , v ′ ) = F η (ρ x,v (x ′ , v ′ )) for all x ′ in a neighbourhood of x and v ′ in a neighbourhood of v. Differentiation with respect to x yields

∇ x e(x ′ , v ′ ) = χ (ρ x,v (x ′ , v ′ )) ∇ x ρ x,v (x ′ , v ′ ) (3.3.9)
due to F ′ η (τ ) = χ(τ ) for τ > 0. In particular, ∇ x e(x, v) = χ (ρ x,v (x, v)) ∇ x ρ x,v (x, v). Now, the assertion follows from 2. in Lemma 3.3.4.

Next, we prove a relation which is the key to some desired continuity properties. Definition 3.3.8. Let g : R n × R m → R be a differentiable function. We say that g satisfies the polynomial growth condition at x if there exist constants C, κ > 0 and a neighbourhood U (x) such that

∇ x g (x ′ , z) ≤ z κ ∀x ′ ∈ U (x) ∀z : z ≥ C.
Lemma 3.3.9. Let x be such that g (x, 0) > 0 and that g satisfies the polynomial growth condition at x. Consider any sequence

(x k , v k ) → (x, v) for some v ∈ I(x) such that v k ∈ F (x k ). Then, lim k→∞ ∇ x e(x k , v k ) = 0. Proof. First observe that ρ x k ,v k (x k , v k ) → ∞ by 2. in Lemma 3.3.5.
Referring to the neighbourhood U (x) from Definition 3.3.8, we verify that for k sufficiently large

∇ x g (x k , ρ x k ,v k (x k , v k ) Lv k ) ≤ [ρ x k ,v k (x k , v k )] κ Lv k κ ≤ L κ [ρ x k ,v k (x k , v k )] κ (3.3.10) (recall that v k = 1 due to v k ∈ F (x k ))
. Moreover, by continuity of g, there exists some δ 1 > 0 such that g (x k , 0) ≥ δ 1 > 0 for k sufficiently large. Since g (x k , ρ x k ,v k (x k , v k ) Lv k ) = 0 (see 1. in Lemma 3.3.4), 3. in Lemma 3.3.3 provides that

∇ z g (x k , ρ x k ,v k (x k , v k ) Lv k ) , Lv k ≤ - g (x k , 0) ρ x k ,v k (x k , v k ) .
Therefore,

∇ z g (x k , ρ x k ,v k (x k , v k ) Lv k ) , Lv k ≤ -δ 1 [ρ x k ,v k (x k , v k )] -1 < 0. (3.3.11)
And as a consequence

| ∇ z g (x k , ρ x k ,v k (x k , v k ) Lv k ) , Lv k | ≥ δ 1 [ρ x k ,v k (x k , v k )] -1 > 0. (3.3.12)
Using the definition χ (y) = δ 2 y m-1 e -y 2 /2 of the density of the chi-distribution with m degrees of freedom (where δ 2 > 0 is an appropriate factor), we may combine Corollary 3.3.7 with (3.3.10) and (3.3.12) in order to derive that

∇ x e(x k , v k ) = χ (ρ x k ,v k (x k , v k )) ∇ z g (x k , ρ x k ,v k (x k , v k ) Lv k ) , Lv k ∇ x g (x k , ρ x k ,v k (x k , v k ) Lv k ) ≤ δ -1 1 ρ x k ,v k (x k , v k ) • δ 2 [ρ x k ,v k (x k , v k )] m-1 e -[ρ x k ,v k (x k ,v k )] 2 /2 • L κ [ρ x k ,v k (x k , v k )] κ = (3.3.13) δ -1 1 δ 2 L κ [ρ x k ,v k (x k , v k )] κ+m e -[ρ x k ,v k (x k ,v k )] 2 /2 → k 0,
where the last limit follows from ρ x k ,v k (x k , v k ) → ∞ and the fact that y α e -y 2 /2 → 0 for y → ∞, where α > 0 is an arbitrary constant. This proves our assertion.

Remark 3.3.10. One may observe from the proof of Lemma 3.3.9 that a weaker growth condition than that in Definition 3.3.8 (involving an exponential term) would suffice for proving the same result. One could for instance use the following exponential growth condition:

Let g : R n ×R m → R be a differentiable function. We say that g satisfies the exponential growth condition at x if there exist constants δ 0 , C > 0 and a neighbourhood U (x) such that

∇ x g (x ′ , z) ≤ δ 0 exp( z ) ∀x ′ ∈ U (x) ∀z : z ≥ C.
and observe that the key estimate (3.3.13) of Lemma 3.3.9 becomes

∇ x e(x k , v k ) ≤ δ 0 δ -1 1 δ 2 [ρ x k ,v k (x k , v k )] m e -[ρ x k ,v k (x k ,v k )] 2 /2 e L ρ x k ,v k (x k ,v k ) .
The same conclusion then easily follows.

We do not put the emphasis on the weakest possible form of the growth condition but rather on its simplicity. It should be noted however that each of the following results requiring the polynomial growth condition hold upon requiring the above exponential growth condition instead.

Corollary 3.3.11. Let x be such that g (x, 0) > 0 and that g satisfies the polynomial growth condition at x. Then, for any v ∈ S m-1 the partial derivative w.r.t x of the function e exists at (x, v) and is given by

∇ x e(x, v) = - χ(ρ x,v (x,v)) ∇ z g(x,ρ x,v (x,v)Lv),Lv ∇ x g (x, ρ x,v (x, v) Lv) if v ∈ F (x) 0 else
where χ is the density of the chi-distribution with m degrees of freedom and ρ In particular, v ∈ F (x + t k u i ) for all k because otherwise v ∈ I(x + t k u i ) and so e(x + t k u i , v) = 1 (again by 4. in Lemma 3.3.3), thus contradicting (3.3.15). We may also assume that g(x + t k u i , 0) > 0 for all k. Now, fix an arbitrary k and define (recall that

t k < 0) α := inf {τ ∈ [t k , 0] |e(x + τ u i , v) = 1} .
Due to e(x, v) = 1 we have that α ≤ 0. On the other hand, e(x + t k u i , v) < 1 and the continuity of e (see Corollary 3.3.6) provide that α > t k . We infer that e(x + τ u i , v) < 1 for all τ ∈ [t k , α) and, hence,

v ∈ F (x + τ u i ) ∀τ ∈ [t k , α) (3.3.16)
(once more by 2. and 4. in Lemma 3.3.3). But then, the function

β(τ ) := e(x + τ u i , v)
is differentiable for all τ ∈ (t k , α) by virtue of Corollary 3.3.7 and its derivative is given by

β ′ (τ ) = ∇ x e(x + τ u i , v), u i .
Therefore, the mean value theorem guarantees the existence of some τ * k ∈ (t k , α) such that

β ′ (τ * k ) = β(α) -β(t k ) α -t k or equivalently ∇ x e(x + τ * k u i , v), u i = e(x + αu i , v) -e(x + t k u i , v) α -t k .
By continuity of e and by definition of α, we have that e(x + αu i , v) = 1 = e(x, v), whence, by t k < α ≤ 0,

∇ x e(x + τ * k u i , v), u i = e(x, v) -e(x + t k u i , v) α -t k ≥ e(x, v) -e(x + t k u i , v) -t k ≥ ε,
where the last relation follows from (3.3.15). Now, since k was arbitrarily fixed, we have constructed a sequence τ * k such that t k < τ * k ≤ 0 such that (3.3.16). Due to our assumption that g satisfies the polynomial growth condition at x and due to v ∈ I(x), Lemma 3.3.9 yields that lim k→∞ ∇ x e(x k , v) = 0 which contradicts (3.3.17). This proves our Corollary.

∇ x e(x + τ * k u i , v), u i ≥ ε ∀k. (3.3.17) Since t k ↑ 0, we also have that τ * k ↑ 0. Moreover, v ∈ F (x + τ * k u i ) by
Corollary 3.3.12. Let x be such that g (x, 0) > 0 and that g satisfies the polynomial growth condition at x. Then, for any v ∈ S m-1 the partial derivative ∇ x e is continuous at (x, v).

Proof. Let x ∈ R n with g(x, 0) > 0 and v ∈ S m-1 be arbitrarily given. Let also (x k , v k ) → (x, v) be an arbitrary sequence with v k ∈ S m-1 . If v ∈ F (x), then relation (3.3.9) holds true locally around (x, v). In particular, for k large enough,

∇ x e(x k , v k ) = χ (ρ x,v (x k , v k )) ∇ x ρ x,v (x k , v k ) → χ (ρ x,v (x, v)) ∇ x ρ x,v (x, v) = ∇ x e(x, v),
where the convergence follows from the continuity of the chi-density and of ∇ x ρ x,v as a result of Lemma 3.3.4. Hence, in case of v ∈ F (x), ∇ x e is continuous at (x, v). Now, assume in contrast that v ∈ I(x). Then, ∇ x e(x, v) = 0 by Corollary 3.3.11. Now, assume that ∇ x e(x k , v k ) does not converge to zero. Then, ∇ x e(x k l , v k l ) ≥ ε for some subsequence and some ε > 0. Then, v k l ∈ F (x k l ) for all l because otherwise v k l ∈ I(x k l ) and, thus, ∇ x e(x k l , v k l ) = 0 due to Corollary 3.3.11 (applied to x k l rather than x; observe that the condition g (x, 0) > 0 and the polynomial growth condition at x are open conditions, hence continue to hold true for the x k l ). Now, Lemma 3.3.9 yields the contradiction lim

l→∞ ∇ x e(x k l , v k l ) = 0 with ∇ x e(x k l , v k l ) ≥ ε.
This proves our Corollary.

Now we are in a position to state our main result:

Theorem 3.3.13. Let g : R n ×R m → R be a continuously differentiable function which is concave with respect to the second argument. Consider the probability function ϕ defined in (3.3.4), where ξ ∼ N (0, R) has a standard Gaussian distribution with correlation matrix R. Let the following assumptions be satisfied at some x:

1. g (x, 0) > 0.

2. g satisfies the polynomial growth condition at x (Def. 3.3.8).

Then, ϕ is continuously differentiable on a neighbourhood U of x and it holds that

∇ϕ (x) = - v∈F (x) χ (ρ x,v (x, v)) ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) dµ ζ (v) ∀x ∈ U.
(3.3.18) Here, µ ζ is the law of the uniform distribution over S m-1 , χ is the density of the chidistribution with m degrees of freedom, L is a factor of the Cholesky decomposition R = LL T and ρ x,v is as introduced in Lemma 3.3.4.

Proof. Since ξ ∼ N (0, R), the probability function ϕ gets the representation (3.3.5). With g (x, 0) > 0, let U be a sufficiently small neighbourhood of x such that for all x ∈ U we still have that g(x, 0) > 0 and that the polynomial growth condition is satisfied at x. Then, the partial derivative ∇ x e of the function e defined in (3.3.6) exists on U × S m-1 by Corollary 3.3.11 and is continuous there by Corollary 3.3.12. By compactness of S m-1 , there exists some K > 0 such that

∇ x e(x, v) ≤ K ∀v ∈ S m-1 .
Again, continuity of ∇ x e on U × S m-1 and compactness of S m-1 guarantee that the function α : U → R defined by

α(x) := max v∈S m-1 ∇ x e(x, v)
is continuous. Since α(x) ≤ K, we may assume, after possibly shrinking U , that α(x) ≤ 2K for all x ∈ U , whence

∇ x e(x, v) ≤ 2K ∀x ∈ U ∀v ∈ S m-1 . (3.3.19)
From µ ζ (S m-1 ) = 1 for the law µ ζ of the uniform distribution on S m-1 we infer that the constant 2K is an integrable function on S m-1 uniformly dominating ∇ x e(x, v) on S m-1 for all x ∈ U . Now, Lebesgue's dominated convergence theorem allows us to differentiate (3.3.5) under the integral sign:

∇ϕ (x) = v∈S m-1 ∇ x e(x, v)dµ ζ .
As stated in the beginning of this proof, the assumptions 1. and 2. imposed in the Theorem for the fixed point x keep to hold for all x in the neighbourhood U . Therefore, we may derive that

∇ϕ (x) = v∈S m-1 ∇ x e(x, v)dµ ζ ∀x ∈ U. (3.3.20)
Exploiting once more the dominance argument from (3.3.19), the continuity of ∇ x e on U × S m-1 and the compactness of S m-1 ensure by virtue of Lebesgue's dominated convergence Theorem that ∇ϕ is continuous. Finally, formula (3.3.18) follows directly from Corollary 3.3.11.

Remark 3.3.14. Evidently, formula (3.3.18) is explicit and can be used inside Deák's method in order to calculate ∇ϕ in parallel with ϕ by efficient sampling on S m-1 . For each sampled point v ∈ S m-1 one first has to check whether the equation g(x, rLv) = 0 has a solution r ≥ 0 at all. If not so (v ∈ I(x)), then such v does not contribute to the (approximated) integral in (3.3.18). Otherwise (v ∈ F (x)), one has to evaluate the integrand in (3.3.18) which amounts to finding the unique solution r ≥ 0 of the equation g(x, rLv) = 0. In general, a few Newton-Raphson iterations should do the job.

We now want to focus our attention on the assumptions of Theorem 3.3.13. First, recall that assuming a standard Gaussian distribution ξ ∼ N (0, R) does not mean any loss of generality by virtue of Remark 3.3.2. Also assumption 1. of the Theorem is not restrictive. This will come as a consequence of the following proposition:

Proposition 3.3.15. With g and ϕ as in Theorem 3.3.13, let the following assumptions be satisfied at some x:

1. There exists some z such that g(x, z) > 0.

2. ϕ(x) > 1/2.
Then, g(x, 0) > 0.

Proof. As in the proof of Theorem 3.3.13 we may assume that ξ ∼ N (0, R) so that ϕ gets the representation (3.3.5). Define the set M := {z ∈ R m |g(x, z) ≥ 0}. Clearly, M is convex and nonempty by our assumption 1. This same assumption (Slater point)

guarantees that int M = {z ∈ R m |g(x, z) > 0} .
Assume that g(x, 0) ≤ 0. Then 0 / ∈ int M and, hence, one could separate 0 from M , which would mean that there exists some c ∈ R m \{0} such that

M ⊆ z ∈ R m |c T z ≥ 0 =: M .
With ξ having a centered Gaussian distribution, the one-dimensional random variable c T ξ has a centered Gaussian distribution too and, hence, we arrive with our assumption 3. at the contradiction

1/2 = P c T ξ ≥ 0 = P ξ ∈ M ≥ P [ξ ∈ M ] = ϕ (x) > 1/2.
The proposition means that violation of the first assumption in Theorem 3.3.13 implies that g(x, z) ≤ 0 for all z or that ϕ(x) ≤ 1/2. A typical application of Theorem 3.3.13 is probabilistic programming where one is imposing the chance constraint ϕ(x) ≥ p with some probability level p close to one. Since gradients of ϕ are usually calculated at or close to feasible points (e.g. by cutting planes), the case ϕ(x) ≤ 1/2 is very unlikely to occur. On the other hand, g(x, z) ≤ 0 for all z is a degenerate situation meaning that there exists no Slater point for the concave function g(x, •). In such situation it typically happens that the set {z|g(x, z) ≥ 0} becomes empty for x arbitrarily close to x which would entail a discontinuity of ϕ at x. Then, of course, there is no hope to calculate a gradient at all. Finally, turning to condition 2. of Theorem 3.3.13 (growth condition) it may require some technical effort to check it in concrete applications (see, e.g., the examples discussed in the following section). On the other hand, we shall see in a moment that we may do without this condition in case that the set {z|g(x, z) ≥ 0} is bounded. To formulate a corresponding statement we need the following two auxiliary results: Lemma 3.3.16. Let g : R n × R m → R be continuous. Moreover, let g be convex in the second argument. Then, for any x ∈ R n with g (x, 0) > 0 one has that

I(x) = ∅ if and only if M (x) := {z ∈ R m |g(x, z) ≥ 0} is bounded.
Proof. Let x be arbitrary such that g (x, 0) > 0. Obviously boundedness of M (x) implies that I(x) = ∅, so let us assume that I(x) = ∅ and that M (x) is unbounded. Then, there is a sequence z n with g (x, z n ) ≥ 0 and z n → ∞. Without loss of generality, we may assume that z n -1 z n → z for some z ∈ R m \{0}. Let t ≥ 0 be arbitrary. Then, z n -1 t ≤ 1 for n sufficiently large. From concavity of g(x, •), g (x, 0) > 0 and g (x, z n ) ≥ 0 we infer that g x, z n -1 tz n ≥ 0 for n sufficiently large. Passing to the limit, we get that g (x, tz) ≥ 0. Thus, as t ≥ 0 was arbitrary, g (x, tz) ≥ 0 ∀t ≥ 0.

(3.3.21)

Assume that there was some τ ≥ 0 with g (x, τ z) = 0. Then, again by concavity of g(x, •) and by g (x, 0) > 0, one would arrive at the following contradiction with (3.3.21):

g (x, tz) < g (x, τ z) = 0 ∀t > τ.
Hence, actually g (x, tz) > 0 for all t ≥ 0. Putting v := L -1 z/ L -1 z -where L is the (invertible) matrix appearing in the definition of I(x) -and observing that this definition is correct due to z = 0, we derive that g (x, t L -1 z Lv) > 0 for all t ≥ 0. Since L -1 z > 0, this implies that g (x, rLv) > 0 for all r ≥ 0. Hence the contradiction v ∈ I(x) with our assumption I(x) = ∅. It follows that M (x) is bounded as was to be shown.

Proposition 3.3.17. Let g be as in Lemma 3.3.16 and

x ∈ R n with g (x, 0) > 0. If M (x) is bounded, then there is a neighbourhood U of x such that M (x) remains bounded for all x ∈ U .
Proof. By continuity of g, we may choose U small enough that g (x, 0) > 0 for all x ∈ U . If the assertion was not true, then by virtue of Lemma 3.3.16 there exists a sequence x n → x such that I (x n ) = ∅ for all n ∈ N. By 1. in Lemma 3.3.3 this implies the existence of another sequence v n ∈ S m-1 such that

g (x n , rLv n ) > 0 ∀r ≥ 0 ∀n ∈ N.
Without loss of generality, we may assume that v n → v for some v ∈ S m-1 . For each r ≥ 0 we may pass to the limit in the relation above, in order to derive that g (x, rLv) ≥ 0 for all r ≥ 0. With the same reasoning as below (3.3.21) we may conclude that indeed g (x, rLv) > 0 for all r ≥ 0. This means that v ∈ I (x), whence M (x) is unbounded by Lemma 3.3.16. This is a contradiction with our assumption. Now we are in a position to state an alternative variant of Theorem 3.3.13 which does not require the verification of the growth condition: Theorem 3.3.18. Theorem 3.3.13 remains true if the second condition (growth condition) is replaced by the condition that the set {z|g(x, z) ≥ 0} is bounded. Then, (3.3.18) becomes

∇ϕ (x) = - v∈S m-1 χ (ρ x,v (x, v)) ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) dµ ζ (v) ∀x ∈ U. (3.3.22)
Proof. As in the proof of Theorem 3.3.13, the function e is continuous on U × S m-1 by Corollary 3.3.6 because this result does not require the growth condition to hold. Moreover, ∇ x e exists on U × S m-1 . Indeed, our boundedness assumption ensures via Proposition 3.3.17 that -after possibly shrinking the neighbourhood U of x -the set {z|g(x, z) ≥ 0} remains bounded for all x ∈ U . Lemma 3.3.16 implies that I(x) = ∅ or, equivalently according to 2. in Lemma 3.3.3 -that F (x) = S m-1 for all x ∈ U . Then, Corollary 3.3.7 yields that ∇ x e exists on U × S m-1 and is given by

∇ x e(x, v) = - χ (ρ x,v (x, v)) ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) .
Since all occurring functions are continuous, the same holds true for ∇ x e. Now the same argument as in the proof of Theorem 3.

Selected Examples

In this section we are going to discuss some instances of the probabilistic constraint (3.3.1) to which our gradient formulae obtained in Theorems 3.3.13 and 3.3.18 apply and thus could be used in the numerical solution of corresponding optimization problems.

Gaussian distributions

We assume first, as before, that the random vector has a Gaussian distribution. We shall focus on the particular model

P[ f (ξ), h 1 (x) ≤ h 2 (x)] ≥ p (3.3.23)
with nonlinear mappings f : R m → R l and h 1 : R n → R l and h 2 : R n → R involving a coupling of random and decision vector.

Proposition 3.3.19. In the probabilistic constraint (3.3.23), let f, h 1 , h 2 be continuously differentiable, let the components f i of f be convex and the components h 1,i of h 1 be nonnegative. Furthermore, let ξ ∼ N (0, R) have a standard Gaussian distribution with correlation matrix R and associated Cholesky decomposition R = LL T . Consider any x with f (0), h 1 (x) < h 2 (x). Finally, let f satisfy the following polynomial growth condition:

f (z) ≤ z κ ∀z : z ≥ C for certain κ, C > 0. Then the probability function ϕ (x) := P [ f (ξ), h 1 (x) ≤ h 2 (x)]
defining the constraint (3.3.23) is continuously differentiable on a neighbourhood U of x and its gradient is given by

∇ϕ (x) = v∈F (x) χ (ρ x,v (x, v)) [ h T 1 (x)∇f (ρ x,v (x, v) Lv) , Lv ] -1 ∇h 2 (x)dµ ζ (v) - v∈F (x) χ (ρ x,v (x, v)) [f (ρ x,v (x, v) Lv)] T ∇h 1 (x) h T 1 (x)∇f (ρ x,v (x, v) Lv) , Lv dµ ζ (v) ∀x ∈ U.
Proof. In our setting the general function g in (3.3.4) becomes g (x, z) =f (z), h 1 (x) + h 2 (x). The continuous differentiability and concavity with respect to the second argument of g are evident from our assumptions. Moreover, g (x, 0) > 0. As for the growth condition, let U be a neighbourhood of x on which max{ ∇h 1 , ∇h 2 } ≤ K for some K > 0. Then, taking without loss of generality, the maximum norm, we have that

∇ x g(x, z) = ∇h 2 (x) -[f (z)] T ∇h 1 (x) ≤ K( f (z) + 1) ≤ z 2+κ ∀x ∈ U, z : z ≥ max{C, K, 2}.
Consequently, we may apply Theorem 3.3.13. (3.3.24) follows immediately from (3.3.18) for the given form of the function g.

Gaussian-like distributions

We are now going to apply Theorem 3.3.13 to probabilistic constraints with random vectors having non-Gaussian distributions. In a first case, we consider a linear probabilistic constraint

P[ η, x ≤ b] ≥ p (3.3.24)
with a random vector η whose components η i (i = 1, . . . , l) are independent and have a χ 2 -distribution with n i degrees of freedom. By definition,

η i = n i k=1 ξ 2 i,k
, where the ξ i,k ∼ N (0, 1) are independent for k = 1, . . . , n i . We are interested in the gradient of the probability function ϕ(x) := P[ η, x ≤ b]. Define a Gaussian random vector with independent components ξ := (ξ 1,1 , . . . , ξ 1,n 1 , . . . , ξ l,1 , . . . , ξ l,n l ) ∼ N (0, I) .

Clearly, η ∼ f (ξ), where f i (z) := n i k=1 z 2 i,k for i = 1, . . . , l and z is partitioned in the same way as ξ above. Then, the probability function defining (3.3.24) becomes

ϕ(x) = P[ η, x ≤ b] = P[ f (ξ), x ≤ b].
We derive the following gradient formula which does not need the verification of a polynomila growth condition and which is even fully explicit with respect to the resolving function ρ x,v : Proposition 3.3.20. In (3.3.24), let b > 0. Consider any feasible point x of (3.3.24) satisfying xi > 0 for i = 1, . . . , n. Then the probability function ϕ is continuously differentiable on a neighbourhood U of x and its gradient is given by

∇ϕ (x) = - √ b 2 v∈S m-1 χ b/ f (v), x f (v), x 3/2 [f (v)] T dµ ζ (v) ∀x ∈ U. (3.3.25)
Proof. In our setting the general function g in (3.3.4) becomes g (x, z) =f (z), x + b which is continuously differentiable. Since the components f i are convex, g (x, •) is concave whenever x ≥ 0, which by our assumption holds true in a neighbourhood of x. Evidently, the result of Theorems 3.3.13 and 3.3.18 are of local nature (differentiability around x) so they actually do not need concavity of g (x, •) for all x ∈ R n but only for x in a neighbourhood of x which is satisfied here. Next observe that g (x, 0) = b > 0.

Finally, recalling that xi > 0 for i = 1, . . . , n, we obtain the estimate

{z|g(x, z) ≥ 0} = {z| f (z), x ≤ b} ⊆ z| min i=1,...,n xi n i=1 f i (z) ≤ b = z| z 2 ≤ b min i=1,...,n xi -1
, whence the set on the left-hand side is bounded. Altogether, this allows us to invoke Theorem 3.3.18 and to derive the validity of formula (3.3.22). We now specify this formula in our setting. First observe that given ξ ∼ N (0, I), we have that R = I, hence we have L = I for the Cholesky decomposition R = LL T . Next we calculate explicitly the function ρ x,v (x, v) which is the unique solution in r ≥ 0 of the equation

f (rLv), x = b. Now, by definition of f , f (rLv), x = r 2 f (v), x = b, whence r = b/ f (v), x . (3.3.26)
Next, we calculate

-∇ x g (x, ρ x,v (x, v) Lv) = [f (ρ x,v (x, v) v)] T = [ρ x,v (x, v)] 2 [f (v)] T = (b/ f (v), x ) [f (v)] T -∇ z g (x, ρ x,v (x, v) Lv) , Lv = -∇ z g (x, ρ x,v (x, v) v) , v = n i=1 x i ∇f i (ρ x,v (x, v) v) , v = n i=1 x i ∇f i (ρ x,v (x, v) v) , v = 2ρ x,v (x, v) n i=1 x i n i k=1 v 2 i,k = 2ρ x,v (x, v) f (v), x = 2 b f (v), x .
Combination of these last relations with (3.3.26) provides formula (3.3.25).

As a second instance for a non-Gaussian but Gaussian-like distribution, we consider the multivariate log-normal distribution. Recall, that a random vector η follows a multivariate lognormal distribution if the vector ξ := log η (componentwise logarithm) has a Gaussian distribution. We consider now a probabilistic constraint of type

P[ η, x ≤ h(x)] ≥ p (3.3.27)
where η is an m-dimensional random vector with lognormal distribution and h : R m → R is some function. We are interested in the gradient of the associated probability function ϕ(x) := P[ η, x ≤ h(x)]. We denote by ξ := log η the Gaussian random vector associated with η. Without loss of generality (see Remark 3.3.2) we may assume that ξ ∼ N (0, R) for some correlation matrix R. We denote by L the associated factor in the Cholesky decomposition R = LL T .

Proposition 3.3.21. In the setting above, assume that x satisfies xi > 0 for i = 1, . . . , m.

Assume moreover that h is continuously differentiable and that h(x) > m i=1 xi . Then,

∇ϕ (x) = - {v∈S m-1 |∃i:L i v>0} χ (ρ x,v (x, v)) m i=1 x i e ρ x,v (x,v)L i v i L i v e ρ x,v (x,v)Lv -∇h(x) dµ ζ (v) ∀x ∈ U.
Here, L i refers to the ith row of L and the expression e z has to be understood componentwise.

Proof. In our setting the general function g in (3.3.4) becomes g (x, z) = -e z , x + h(x).

Clearly, g is continuously differentiable and concave with respect to z for all x close to x (as mentioned in the proof of Proposition (3.3.24) this weakened condition is enough in the context of Theorem 3.3.13). Moreover, g (x, 0) = -m i=1 xi + h(x) > 0. In order to apply Theorem 3.3.13, it is sufficient to verify the exponential growth condition of Remark 3.3.10 (note that the originally imposed polynomial growth condition would not hold true here). To this aim, let U be a neighbourhood of x on which ∇h ≤ K for some K > 0. Then, with respect to the maximum norm, we get that

∇ x g (x ′ , z) ≤ e z + ∇h(x ′ ) ≤ e z + K ≤ 2e z ∀x ′ ∈ U (x) ∀z : z ≥ log K.
Hence, the exponential growth condition of Remark 3.3.10 is satisfied. This allows us to apply Theorem 3.3.13. Inserting the corresponding derivative formulae for g, we derive that ϕ is continuously differentiable on a neighbourhood U of x and its gradient is given by

∇ϕ (x) = - v∈F (x) χ (ρ x,v (x, v)) m i=1 x i e ρ x,v (x,v) L i ,v L i , v e ρ x,v (x,v)Lv -∇h(x) dµ ζ (v) ∀x ∈ U.
(3.3.28) Here, L i denotes the ith row of the Cholesky factor L. To complete the proof, we have to verify the representation of the integration domain F (x) asserted in the statement of this proposition. Without loss of generality, we assume the neighbourhood U of x in the formula above to be small enough that g (x, 0) > 0 and x i > 0 for i = 1, . . . , m and for all x ∈ U (recall that g (x, 0) > 0 and xi > 0 for i = 1, . . . , m). We claim that for all x ∈ U the set I(x) introduced below (3.3.6) can be written as

I(x) = v ∈ S m-1 |Lv ≤ 0 . (3.3.29)
Indeed, let x ∈ U and v ∈ S m-1 with Lv ≤ 0 be arbitrary. Then, for all r > 0,

g(x, rLv) = -e rLv , x + h(x) ≥ -e 0 , x + h(x) = g (x, 0) > 0,
whence v ∈ I(x) by 1. of Lemma 3.3.3. Conversely, let x ∈ U and v ∈ I(x) be arbitrary.

Then, e rLv , x < h(x) for all r > 0. Define J := {i|L i v > 0}. It follows from

x i > 0 for i = 1, . . . , m that h(x) > i∈J x i e r L i ,v .
If J = ∅, then the sum on the right-hand side tends to ∞ for r → ∞ which is a contradiction to this sum being bounded from above by h(x) for all r > 0. Consequently, J = ∅, proving Lv ≤ 0 and, thus, the reverse inclusion of (3.3.29). Since, by definition,

F (x) = S m-1 \I(x)
, we may plug the information from (3.3.29) into (3.3.28) in order to derive our asserted formula.

Student (or T-) distribution

As a last application, we are going to consider probabilistic constraints of type (3.3.1), where the random vector ξ follows a so-called multivariate Student or T-distribution. This is an important type of distribution in particular due to its application in the context of copulas. We recall that ξ ∼ T (µ, Σ, ν) -i.e., ξ obeys a multivariate T-distribution with parameters µ, Σ, ν -if ξ = µ+ϑ ν u , where ϑ ∼ N (0, Σ) has a multivariate Gaussian distribution with mean µ and covariance matrix Σ, u ∼ χ 2 (ν) has a chi-squared distribution with ν degrees of freedom and ϑ and u are independent [START_REF] Nadarajah | Mathematical properties of the multivariate t distribution[END_REF]. We are interested in the probability function (3.3.4) but this time for a T-variable rather than for a Gaussian one.

Remark 3.3.22. Using the definition of a T-distribution, we may copy the arguments of Remark 3.3.2 in order to convince ourselves that in the consideration of (3.3.4) we may assume without loss of generality that ξ ∼ T (0, R, ν), where R is a correlation matrix. In particular, this can be arranged without disturbing the assumption of g in (3.3.4) being continuously differentiable and convex with respect to the second argument.

In a first step, we provide an expression for the probability function (3.3.4) in case of a T-distribution:

Theorem 3.3.23. Let g : R n × R m →
R be a continuously differentiable function which is concave with respect to the second argument. Moreover, let ξ ∼ T (0, R, ν) for some correlation matrix R. Consider a point x such that g(x, 0) > 0. Then, there exists a neighbourhood U of x such that the probability function (3.3.4) admits the representation

ϕ (x) = v∈S m-1 ẽ (x, v) dµ ζ ∀x ∈ U, where for all x ∈ U and v ∈ S m-1 ẽ (x, v) := F m,ν (m -1 [ρ x,v (x, v)] 2 ) v ∈ F (x) 1 v ∈ I(x)
and F m,ν refers to the distribution function of the Fisher-Snedecor distribution with m and ν degrees of freedom. Moreover, ρ x,v is as introduced in Lemma 3.3.4 and F (x) and I(x) are defined in Lemma 3.3.3.

Proof. Let U be a neighbourhood of x small enough such that g(x, 0) > 0 for all x ∈ U . Fix an arbitrary x ∈ U . According to the definition of ξ, there exist ϑ ∼ N (0, R) and u ∼ χ 2 (ν) such that ϑ and u are independent and

ϕ (x) = P g(x, ϑ ν u ) ≥ 0 = { (y,t)|t>0,g(x,y √ ν t )≥0 } f ϑ,u (y, t) dydt,
where f ϑ,u denotes the joint density of the vector (ϑ, u). By independence, f ϑ,u (y, t) = f ϑ (y) f u (t) where f ϑ and f u are the densities of ϑ and u, respectively. In particular, with Γ referring to the Gamma function, it holds that

f u (t) = 1 2 ν/2 Γ(ν/2) t ν/2-1 e -t/2 t ≥ 0 0 t < 0 (3.3.30) Therefore, ϕ (x) = ∞ 0 { y|g(x,y √ ν t )≥0 } f ϑ (y) dy f u (t) dt = 1 2 ν/2 Γ (ν/2) ∞ 0 P g(x, ϑ ν t ) ≥ 0 t ν/2-1 e -t/2 dt. (3.3.31)
With M := {z ∈ R m |g(x, z) ≥ 0} one has that, for t > 0,

P g(x, ϑ √ ν t ) ≥ 0 = P ϑ ∈ t √ ν M .
Since ϑ ∼ N (0, R), (3.3.2) yields that for all t > 0

P g(x, ϑ √ ν t ) ≥ 0 = v∈S m-1 µ η {r ≥ 0| √ ν t rLv ∈ M } dµ ζ = v∈S m-1 µ η {r ≥ 0|g(x, √ ν t rLv) ≥ 0} dµ ζ ,
where η has a χ-distribution with m degrees of freedom and ζ has a uniform distribution over S m-1 . Moreover, L is a factor of the Cholesky decomposition R = LL T . Let t > 0 be arbitrary. Assume first that v ∈ F (x). With g(x, 0) > 0, let ρ x,v : Ũ × Ṽ → R + be the function defined on certain neighbourhoods Ũ , Ṽ of x and v, respectively. It follows from 1. in Lemma 3.3.4 that

{r ≥ 0|g(x, √ ν t rLv) ≥ 0} = 0, t √ ν ρ x,v (x, v) .
If in contrast v ∈ I(x) then g(x, rLv) > 0 for all r ≥ 0, whence

{r ≥ 0|g(x, √ ν t rLv) ≥ 0} = R + .
Combining this with (3.3.31), we conclude that

ϕ (x) = 1 2 ν/2 Γ (ν/2) ∞ 0 v∈F (x) µ η 0, t √ ν ρ x,v (x, v) dµ ζ t ν/2-1 e -t/2 dt + 1 2 ν/2 Γ (ν/2) ∞ 0 v∈I(x) µ η (R + ) dµ ζ t ν/2-1 e -t/2 dt (3.3.32) = 1 2 ν/2 Γ (ν/2) ∞ 0 v∈F (x) F η t √ ν ρ x,v (x, v) dµ ζ + µ ζ (I(x)) t ν/2-1 e -t/2 dt = µ ζ (I(x)) + 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 v∈F (x) F η t √ ν ρ x,v (x, v) dµ ζ dt,
where F η denotes the distribution function of η and we exploited that

F η (0) = 0, µ η (R + ) = 1 and 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 dt = R f u (t)dt = 1.
F m,ν (m -1 r 2 ) = P U -1 ν U m ≤ ν -1 r 2 = {(τ,t)|ντ ≤tr 2 } f U m ,U ν (τ, t) dτ dt,
where f U m ,U ν denotes the joint density of the vector (U m , U ν ). By independence,

f U m ,U ν (τ, t) = f U m (τ ) f U ν (t)
where the single χ 2 -densities are defined with appropriate degrees of freedom in (3.3.30). It follows that

F m,ν (m -1 r 2 ) = ∞ 0 tr 2 /ν 0 f U ν (t) f U m (τ ) dτ dt = 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 1 2 m/2 Γ (m/2) tr 2 /ν 0 τ m/2-1 e -τ /2 dτ dt = 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 1 2 m/2-1 Γ (m/2) r √ t/ν 0 s m-1 e -s 2 /2 dsdt = 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 r √ t/ν 0 f η (s)dsdt.
Here, we used that the variable η introduced above has a χ-distribution with m degrees of freedom and so its density is given by

f η (s) = 1 2 m/2-1 Γ(m/2) s m-1 e -s 2 /2 s ≥ 0 0 s < 0 .
Consequently,with F η denoting the distribution function of η,

F m,ν (m -1 r 2 ) = 1 2 ν/2 Γ (ν/2) ∞ 0 t ν/2-1 e -t/2 F η r t/ν dt = 1 2 ν/2-1 Γ (ν/2) ∞ 0 s ν-1 e -s 2 /2 F η sr/ √ ν ds. (3.3.33)
Consequently, exploiting the definition ẽ in the statement of Theorem 3.3.23, putting r := ρ x,v (x, v) in (3.3.33) and applying Fubini's theorem, we end up via (3.3.32) at

v∈S m-1 ẽ (x, v) dµ ζ = µ ζ (I(x)) + v∈F (x) F m,ν (m -1 [ρ x,v (x, v)] 2 )dµ ζ = µ ζ (I(x)) + 1 2 ν/2-1 Γ (ν/2) ∞ 0 v∈F (x) s ν-1 e -s 2 /2 F η sρ x,v (x, v) / √ ν dµ ζ ds = ϕ (x) .
Now, we may copy the proof of Corollary 3.3.6 but with the function e there replaced by the function ẽ introduced above and with the expression

F η (ρ x,v (x ′ , v ′ )) in statement 1. of Lemma 3.3.5 replaced by the expression F m,ν (m -1 [ρ x,v (x ′ , v ′ )] 2
) in order to derive the continuity of ẽ at any x ∈ U , where U is defined in the Theorem above. Next, we may copy the proof of Corollary 3.3.7 (again with the appropriate replacements) and get the following:

Corollary 3.3.24. For any x ∈ R n with g(x, 0) > 0 and v ∈ F (x) the partial derivative w.r.t x of the function ẽ : R n × S m-1 → R defined in Theorem 3.3.23 exists and is given by

∇ x ẽ(x, v) = -2ρ x,v (x, v) f m,ν m -1 [ρ x,v (x, v)] 2 m ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) (3.3.34)
where

f m,ν (t) = Γ(m/2+ν/2) Γ(m/2)Γ(ν/2) m m/2 ν ν/2 t m/2-1 (mt + ν) -(m+ν)/2 t ≥ 0 0 t < 0 (3.3.35)
is the density of the Fisher-Snedecor distribution with m and ν degrees of freedom, ρ x,v refers to the function introduced in Lemma 3.3.4 and L is a factor of the Cholesky decomposition R = LL T .

It is the equivalent of Lemma 3.3.9 that requires some additional conditions and work: Lemma 3.3.25. Let x be such that g (x, 0) > 0 and that g satisfies the polynomial growth condition at x with coefficient κ < ν (Def. 3.3.8). Consider any sequence

(x k , v k ) → (x, v) for some v ∈ I(x) such that v k ∈ F (x k ). Then, lim k→∞ ∇ x ẽ(x k , v k ) = 0. Proof. First observe that ρ x k ,v k (x k , v k ) → ∞
∇ x ẽ(x k , v k ) = = 2ρ x k ,v k (x k , v k )f m,ν (m -1 [ρ x,v (x, v)] 2 ) m ∇ z g (x k , ρ x k ,v k (x k , v k ) Lv k ) , Lv k ∇ x g (x k , ρ x k ,v k (x k , v k ) Lv k ) ≤ 2ν ν/2 Γ (m/2 + ν/2) Γ (m/2)Γ(ν/2) L κ δ -1 1 ρ x k ,v k (x k , v k ) m+κ (1 + ρ x k ,v k (x k , v k ) 2 ν ) -m+ν 2 → k 0,
where the last limit follows from

ρ x k ,v k (x k , v k ) → ∞ and κ < ν.
Upon having established Lemma 3.3.25 the same arguments of Corollary 3.3.11 can be used to show that ẽ is differentiable with respect to x and to derive a similar formula. This can be done since the proof of Corollary 3.3.11 uses only the properties of e and we have established the same properties for ẽ. Accordingly,

∇ x ẽ(x, v) is given by formula (3.3.34) if v ∈ F (x) and ∇ x ẽ(x, v) = 0 if v ∈ I(x).
In the same way as in Corollary 3.3.12 one establishes the continuity of ∇ x ẽ upon noting that f m,ν (t) defined in (3.3.35) is also continuous. We thus arrive at the following key result, of which the proof is a verbatim copy of that of Theorem 3.3.13 (Again e and ẽ have the same properties).

Theorem 3.3.26. Let g : R n ×R m → R be a continuously differentiable function which is concave with respect to the second argument. Consider the probability function ϕ defined in (3.3.4), where ξ ∼ T (0, R, ν). Let the following assumptions be satisfied at some x:

1. g (x, 0) > 0.

2. g satisfies the polynomial growth condition at x (Def. 3.3.8) with coefficient κ < ν.

Then, ϕ is continuously differentiable on a neighbourhood U of x and it holds that

∇ϕ (x) = v∈F (x) - 2ρ x,v (x, v)f m,ν (m -1 [ρ x,v (x, v)] 2 ) m ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) dµ ζ (v) ∀x ∈ U.
(3.3.36) Here, µ ζ is the law of the uniform distribution over S m-1 , f m,ν is the density of the Fisher-Snedecor-distribution with m and ν degrees of freedom and ρ x,v is as introduced in Lemma 3.3.4.

In the above result, the degrees of freedom ν of ξ ∼ T (0, R, ν) imposes an important restriction on the growth condition. Hence, on the mappings g that can be allowed for. In Theorem 3.3.18 we were able to replace the growth condition by a boundedness assumption. This can also be done now. Again the proof of the following result is a verbatim copy of that of Theorem 3.3.18.

Theorem 3.3.27. Theorem 3.3.26 remains true if the second condition (growth condition) is replaced by the condition that the set {z|g(x, z) ≥ 0} is bounded. Then, (3.3.36) becomes:

∇ϕ (x) = v∈S m-1 - 2ρ x,v (x, v)f m,ν (m -1 [ρ x,v (x, v)] 2 ) m ∇ z g (x, ρ x,v (x, v) Lv) , Lv ∇ x g (x, ρ x,v (x, v) Lv) dµ ζ (v) ∀x ∈ U.
(3.3.37) Moreover this result holds for all ν ≥ 1.

Remark 3.3.28. Theorem 3.3.27 in particular covers the case when ξ follows a multivariate Cauchy Distribution, i.e., ξ ∼ T (0, R, 1). This case was excluded in Theorem 3.3.26.

Example 3.3.29. Let us consider the optimization problem

min x∈R 2 c T x s.t. ϕ(x) := P[η x 1 1 η x 2 2 ≤ 2] ≥ 0.8, (3.3.38) Ax ≤ b x ≥ 0
where η = (η 1 , η 2 ) follows a log-normal law with underlying multivariate Gaussian distribution having mean (0, 0) and 2 × 2 correlation matrix R with coefficient R 12 = -0.88. The cost vector c is defined as c = (-4, -1). The matrix A is given by

A =   -3 2 1 1 2 -1 1 1   , b = ( 1 2 , 1 2 , 4
). The unconstrained optimal solution is x * = (3, 1). This solution turns out to be infeasible since it only provides a probability level of 0.63 roughly. Upon defining the mapping

g : R 2 × R 2 → R 2 as g(x, z) = -exp(x 1 z 1 + x 2 z 2 ) + 2, we can observe that the chance constraint ϕ(x) is equivalent with ϕ(x) = P[g(x, ξ) ≥ 0]. Alternatively ϕ(x) can be characterized as ϕ(x) = P[x 1 z 2 + x 2 z 2 ≤ log(2)
] as the logarithm is a strictly increasing mapping. The last characterization allows us to deduce the convexity of the feasible set by Kataoka's result [START_REF] Kataoka | A stochastic programming model[END_REF] whenever p > 1 2 as in the example. The same result allows us to show that Problem (3.3.38) is in fact equivalent with

min x∈R 2 c T x s.t. x T Rx ≤ ( log(r) Φ -1 (0.8) ) 2 , (3.3.39) Ax ≤ b x ≥ 0
where Φ -1 denotes the inverse of the standard normal distribution function. The optimal solution of (3.3.39) is readily found to be x = (1.7250, 1.6010). One easily derives that x s = (0.2, 0.2) is a Slater point for problem (3.3.38).

Since the feasible set is convex, we can apply a Supporting Hyperplane method for solving (3.3.38) directly. Theorem 3.3.13 allows us to compute a gradient if the conditions are satisfied. Returning to our mapping g, it is easily seen that g is concave in z whenever we impose the additional constraint x ≥ 0. Moreover g(x, 0) > 0 regardless of x ≥ 0. One readily derives from the Cauchy-Schwarz inequality that ∇ x g(x, z) ≤ z exp( z x ). Following Remark 3.3.10 this is also a sufficient condition for Lemma 3.3.9 to hold. This in turn applies that the polynomial growth condition of Theorem 3.3.13 can be substituted by the above exponential growth condition. We can thus use the formula (3.3.18) for computing a gradient of ϕ(x). The numerical sampling scheme used for the integral over the sphere S m-1 is the O 2 -estimator of Déak [START_REF] Deák | Subroutines for computing normal probabilities of sets -computer experiences[END_REF] where the uniform orthonormal system is generated according to the ideas of [START_REF] Diaconis | The subgroup algorithm for generating uniform random variables[END_REF]. We have generated a 1000 orthonormal systems.

Let k be the current iteration of the supporting hyperplane algorithm and x k the current candidate solution. We will first compute λ > 0 such that

x c = λx k + (1 -λ)
x s is on the boundary of the feasible set, i.e., ϕ(x c ) = p. We then add the constraint -∇ϕ(x c ) T x ≤ -∇ϕ(x c ) T x c to problem (3.3.38) and solve the newly obtained problem in order to obtain a new iterate. Each iterate x k is, a priori, infeasible and provides a lower bound on the optimal value. Each iterate x c is feasible and provides an upper bound. We will stop the The thus obtained solution is x * = (1.7356, 1.5959) with objective function value -8.5383, whereas the truly optimal solution x provides an objective function value of -8.5010. This difference is explained by the fact that x * is slightly infeasible due to numerical noise.

algorithm if c T x c -c T x k c T x k < 0.01.

Eventual Convexity

Consider a constraint of the type (2.0.1), i.e.,

P[g(x, ξ) ≥ 0] ≥ p, (3.4.1)
where g : R n × R m → R k is a constraint mapping, x ∈ R n the decision vector, ξ ∈ R m a random variable, P its associated probability measure and p ∈ (0, 1) a pre-specified robustness-level.

A key question in Chance Constrained Programming is the convexity of the feasible set, i.e., of the set [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF][START_REF] Prékopa | Stochastic Programming[END_REF][START_REF] Prékopa | Probabilistic programming[END_REF]) that if ξ admits a density with specific generalized concavity properties, and g is a (jointly) quasi-concave mapping, then indeed the feasible set is convex. In many practical applications, we have separable constraint mappings, i.e., g(x, ξ) = h(x)ξ, where h : R n → R m . The requirement of joint quasi concavity of g then can be asserted if h is concave. Note that quasi concavity of h is not sufficient.

M (p) = {x ∈ R n : P[g(x, ξ) ≥ 0] ≥ p}. It is well known ([
The latter requirement on the i-th component h i of the mapping h can be relaxed to -α i -concavity, α i > 0 if ξ has independent components and each component ξ i has a so-called α i + 1-decreasing density ( [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF]). This comes at the cost of only being able to assert convexity of M (p) for p values larger than some threshold. Such convexity is called eventual convexity and is clearly sufficient in many practical applications when we are looking at large p values. Independence of components of ξ is a strong requirement, which was relaxed in a second work of the same authors [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. Indeed, ξ can be allowed to have a dependence structure induced by a specially structured Copula, called a logexp-concave Copula. Some well-known Copulae (Maximum, Independent, Gumbel) are log-exp-concave, as shown in [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. However, it turns out that the Clayton Copula is not log-exp-concave.

When examining very carefully the results of [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF], it appears that the link between generalized concavity of h, individual distribution functions F i of each component ξ i , i = 1, ..., m and the Copula is not clearly exhibited. As a result all log-exp concave Copulae actually provide the same probability threshold. Moreover this level depends in a way on one of the distribution functions F i only. One can thus set up two versions of a problem wherein only one component of ξ has the same distribution and obtain convexity results for the same asserted probability threshold. In particular ξ can have independent components, or components linked through an arbitrary log-exp concave Copula and the same probability threshold is obtained. This is intuitively puzzling.

In this section, we will show that one can derive eventual convexity of the feasible set M (p) for a larger class of Copulae. In particular, we will show that the Clayton Copula is in this extended class. We will moreover exhibit clearly the link between the generalized concavity properties of the mapping h, the individual distribution functions F i , i = 1, ..., m and that of the Copula. We will also show that by adding some additional explicit constraints to the optimization problem, defining a convex feasible set, the probability threshold can be made to depend on the Copula. We will provide several examples showing that one can obtain eventual convexity results for lower p values than in the papers [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. Finally we provide a characterization of the Gaussian Copula in the tail and show that it decomposes as the sum of log-exp concave functions.

This section is organized as follows. We will begin by introducing some useful notation and generalized concavity in section 3.4.1. We will define a class of Copulae containing the class of log-exp-concave Copula and characterize this class in section 3.4.2. In section 3.4.3 we will provide our main Theorems proving eventual convexity for the feasible sets M (p) under specific conditions on the Copula, the individual distribution functions and constraint mappings h i , i = 1, ..., m. Section 3.4.4 provides a series of examples and results showing that sharper bounds on the probability threshold can be obtained and that this new class of Copulae contains strictly more Copulae than just log-exp-concave Copulae. In section 3.4.5 we derive a characterization of the Gaussian Copula, showing that it decomposes as the sum of log-concave functions, at least in its upper-tail. A potential application for modelling probabilistic Constraints with Copula is provided in section 3.4.6.

Notation

Throughout this section, we will apply many algebraic operations on vectors. In order to have short notation, these are understood componentwise. As an example, for any u ∈ R m , e u will be defined as e u = (e u 1 , ..., e u m ). In a very similar way, we will define u

1 γ for γ = 0. For a mapping h : R m → R, u ∈ R m → h(e u
) is thus understood as u → h(e u 1 , ..., e u m ). We will also extend this short notation to one-dimensional mappings applied to a vector. If ϕ : R → R is a mapping, we mean ϕ(u) = (ϕ(u 1 ), ..., ϕ(u m )) when u ∈ R m . Throughout this section, h : R n → R m will be a constraint mapping, ξ ∈ R m an mdimensional random vector. The component ξ i is assumed to have one dimensional

distribution function z ∈ R → F i (z) := P[ξ i ≤ z], i = 1, ..., m. Finally C : [0, 1] m → [0, 1] is a Copula, such that P[ξ ≤ h(x)] = C(F 1 (h 1 (x)), ..., F m (h m (x))). (3.4.2)
We will assume that the mapping (3.4.2) defines a constraint of a Stochastic optimization problem in the following way:

min x∈R n f (x) s.t. P[ξ ≤ h(x)] ≥ p, (3.4.3)
for some probability level p and convex function f . This problem is assumed to be the "Stochastic" variant of the deterministic problem

min x∈R n f (x) s.t. b ≤ h(x), (3.4.4)
for an appropriately chosen vector b ∈ R m , e.g., b = E (ξ). Assuming convexity of f , problem (3.4.4) is a convex optimization problem if and only if the mapping h has some generalized concavity property. It is therefore natural to make such an assumption. Problem (3.4.3) arises, whenever problem (3.4.4) has turned out insufficiently robust and a decision vector x accounting for uncertainty is looked for. Such left-hand side uncertainty arises in many practical applications such as [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF][START_REF] Van Ackooij | Decomposition approaches for block-structured chance-constrained programs with application to hydro-thermal unit-commitment[END_REF].

In this section the set M (p) will be defined as

M (p) := {x ∈ R n : P[ξ ≤ h(x)] ≥ p}, where P[ξ ≤ h(x)] is as in equation (3.4.2).

Copulae and generalized concavity

In [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF], the authors call a Copula C a log-exp concave Copula if and only if u

∈ [0, 1] m → log(C(e u )
) is a concave mapping. It therefore appears natural to provide the following extension of this concept. We will thus speak of a δ-γ-concave Copula.

Definition 3.4.1. Let γ ∈ R be given, and let the set X(γ) be defined as

X(γ) = [0, 1] m for γ > 0, X(0) = (-∞, 0] m and X(γ) = [1, ∞) m for γ < 0. Let δ ∈ [-∞, ∞] be equally given. We call a Copula C : [0, 1] m → [0, 1] δ-γ-concave if the mapping u ∈ X(γ) → C(u 1 γ ) is δ-concave, whenever γ = 0 and u ∈ X(0) → C(e u ) is δ-concave whenever γ = 0.
The presence of the set X(γ) is only to have the arguments mapped in [0, 1] m , so that we can compose with the Copula afterwards.

Remark 3.4.2. This is indeed an extension of the notion of log-exp-concave Copulae as defined in the paper [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. Indeed a log-exp-concave Copula is 0-0-concave in our setting.

Examples of log-exp-concave Copulae are the independent, maximum and Gumbel Copula. The latter is an Archimedean Copula, a family of Copulae generated by a one-dimensional function called the generator. We refer to [START_REF] Mcneil | Multivariate archimedian copulas, d-monotone functions and l 1 norm symmetric distributions[END_REF] for a full characterization of generators of Archimedean Copulae.

Remark 3.4.3. Copulae are such that C(u) = 0 if and only if there is some i = 1, ..., m

with u i = 0. Pick u ∈ [0, 1] m with C(u) = 0, some v ∈ [0, 1] m and λ ∈ [0, 1] and form z = λu+(1-λ)v.
Pick moreover an arbitrary γ > 0, it is then clear from C being a Copula that C(z 1 γ ) ≥ 0 and from the definition of δ-concavity that m δ (C(u

1 γ ), C(v 1 γ ), λ) = 0.
It is therefore sufficient to verify δ-γ-concavity of a Copula with γ > 0 on (0, 1] m only. This avoids the problem of degenerate arguments. Degenerate arguments are naturally avoided when γ ≤ 0. Remark 3.4.4. One could alternatively replace the δ-concavity requirement with a gconcavity requirement as defined in [6] and [START_REF] Tamm | On g-concave functions and probability measures (russian)[END_REF]. We are however interested in eventual convexity, i.e., convexity of all levels sets above some threshold. The notion of g-concavity then implies quasi-concavity on that (sub)-set. Some Copulae might not be δ-γ-concave on the whole domain X(γ) as the following example shows:

Example 3.4.5. Let C : [0, 1] m → [0, 1] be the Gaussian Copula, i.e., C(u) = Φ R (ϕ -1 (u)),
where Φ R is the probability distribution function of a centered Gaussian random variable in dimension m with correlation matrix R and ϕ -1 : (0, 1) → R the inverse of a standard normal probability distribution function in dimension 1. The mapping ϕ -1 : (0, 1) → R is concave on (0, 1 2 ] and Φ R is log-concave ( [START_REF] Prékopa | Stochastic Programming[END_REF]). It then follows that

C is 0-1-concave on [0, 1 2 ] m . Indeed, pick x, y ∈ [0, 1 2 ] m , λ ∈ [0, 1].
From concavity of ϕ -1 and monotonicity of the distribution function we get

C(m 1 (x, y, λ)) ≥ Φ R (m 1 (ϕ -1 (x 1 ), ϕ -1 (y 1 ), λ), ..., m 1 (ϕ -1 (x m ), ϕ -1 (y m ), λ)) ≥ m 0 (C(x), C(y), λ).
Remark 3.4.6. The above example does not exclude a more general δ-γ-concavity property on other sets.

This therefore motivates the following definition: Definition 3.4.7. Let q ∈ (0, 1) m be some point and define the sets X(q, γ) as follows We will introduce one last notion, wherein δ-γ-concavity holds in some asymptotic way. Definition 3.4.9. We call a Copula C : [0, 1] m → [0, 1] asymptotically δ-concave if for each γ > 0 there exists a point q(γ) ∈ (0, 1) m such that C is δ-γ-concave when restricted to the set X(γ) = [0, q(γ)] m and moreover lim γ↓0 q(γ) = 1.

X(q, γ) = m i=1 [q γ i , 1] for γ > 0, X(q, 0) = m i=1 [log(q i ), 0] and X(q, γ) = m i=1 [1, q γ i ] for γ < 0. We call a Copula C : [0, 1] m → [0, 1] δ-γ-q-concave if the mapping u ∈ X(q, γ) → C(u 1 γ ) is δ-concave, whenever γ = 0 and u ∈ X(q, 0) → C(e u ) is δ-concave whenever γ = 0.

Structure of the family of δ-γ-concave Copulae

At first sight it might appear that the family of δ-γ-concave Copulae is rather loose. In particular it may appear that the δ-0-Copulae fit in rather artificially. The following results show that this is not true and that the family is naturally ordered.

It follows from Lemma 2.5.5 that δ-concavity implies some "descending" order, i.e., a δ-concave mapping is also β-concave whenever β ≤ δ. It turns out that the effect of γ is ascending as the following lemma shows.

Lemma 3.4.10. Let C : [0, 1] m → [0, 1] be a δ-β-concave Copula and let α ∈ R be given such that β ≤ α. Then C is also δ-α-concave.
Proof. Pick any x, y ∈ [0, 1] m and λ ∈ [0, 1] arbitrarily. We begin with the special case β = 0, and α > 0. We derive from concavity of the log-function and monotonicity of the exp function that:

m 1 (x, y, λ) 1 α = exp ( 1 α log m 1 (x , y, λ)) ≥ exp ( 1 α m 1 (log x , log y, λ)) = exp (m 1 ( 1 α log x , 1 α log y, λ)) = exp (m 1 (log x 1 α , log y 1 α , λ)),
where with x ∈ (0, 1] m , it follows that log(x 1 α ) ∈ X(0). Now from monotonicity of the Copula we get

C(m 1 (x, y, λ) 1 α )) ≥ C(exp (m 1 (log x 1 α , log y 1 α , λ))) ≥ m δ (C(x 1 α ), C(y 1 α ), λ),
which was to be shown by δ-concavity of u → C(e u ) and Remark 3.4.3. Now picking any β = 0 and α ≥ β, we define z = x 1 α , w = y 1 α , when α = 0 and z = exp (x ), w = exp (y) when α = 0. Together with x, y ∈ X(α), this implies z, w ∈ [0, 1] m . Now with β = 0, one obtains z β , w β ∈ X(β). We then observe that m 1 (x, y, λ) 1 α = m α (z, w, λ), when α = 0 and exp (m 1 (x , y, λ)) = m 0 (z, w, λ) when α = 0. It therefore follows from monotonicity of the Copula combined with Lemma 2.5.5 and δ-β-concavity of the Copula that

C(m 1 (x, y, λ) 1 α ) = C(m α (z, w, λ)) ≥ C(m β (z, w, λ)) ≥ m δ (C(z), C(w), λ) = m δ (C(x 1 α ), C(y 1 α ), λ),
when α = 0 and for α = 0, we get

C(exp (m 1 (x , y, λ))) = C(m 0 (z, w, λ)) ≥ C(m β (z, w, λ)) ≥ m δ (C(z), C(w), λ) = m δ (C(e x ), C(e y ), λ),
as was to be shown.

Remark 3.4.11. This is an extension of Proposition 3 of [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] where it is shown that 0-0-concavity implies 0-1-concavity.

One can also prove this same lemma with the local δ-γ-concavity notion.

Lemma 3.4.12. Let q ∈ (0, 1) m be given and let

C : [0, 1] m → [0, 1] be a δ-β-q-concave
Copula and let α be given such that 0 ≤ β ≤ α. Then C is also δ-α-q-concave.

Proof. The proof is identical to that of Lemma 3.4.10 except that we need to take care of the domains. To this end pick x ∈ X(q, α). We begin by considering the case β = 0, α > 0. It then follows from monotonicity of the log function and of x → x 1 α , that log x 1 α ≥ log(q). In a similar way it follows with α > β > 0 that x β α ≥ q, so that β-q-concavity can be applied.

Since δ-concavity of an arbitrary mapping implies weaker concavity properties for the same mapping, we trivially derive the following corollary: Corollary 3.4.13. Let C : [0, 1] m → [0, 1] be a δ-γ-concave Copula and let α ≥ γ and β ≤ δ be given. Then C is also β-α-concave.

Tools for deriving δ-γ-Concavity of Copulae

Copulae are in particular probability distribution functions, some of them admitting densities. The following result is therefore a trivial consequence of Theorem 4.15 [START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF]:

Lemma 3.4.14. Let C : [0, 1] m → [0, 1] be a Copula, admitting a density c : [0, 1] m → [0, 1]. If the density c is α-concave for some α ≥ -1 m , then C is a δ-1-concave Copula for δ = α 1+mα .
The following Corollary follows immediately from this result

Corollary 3.4.15. Let C : [0, 1] m → [0, 1] be a Copula, admitting a density c : [0, 1] m → [0, 1]. If the density c is such that u ∈ (-∞, 0] m → c(exp (u)) is log-concave then the Copula is 0-0-concave.
Proof. We begin by remarking that h :

(-∞, 0] m → [0, 1], defined as h(u) = C(exp (u))
is also a distribution function. Since C admits a density, the density of h is given by

∂ m ∂u 1 • • • ∂u m h = c(e u ) m i=1 e u i .
Now the logarithm of the latter function is log(c(e u )) + m i=1 u i , i.e., the sum of concave functions, i.e., concave. It follows from Lemma 3.4.14 that h is log-concave, i.e., C is 0-0-concave.

Estimates with Copulae

Before moving to our main result, we first derive some useful auxiliary results.

Lemma 3.4.16. Let C : [0, 1] m → [0, 1] be a δ-γ-concave Copula and ∞ ≥ γ i ≥ γ, i = 1, ..., m any sequence of numbers. For any x, y ∈ [0, 1] m , λ ∈ [0, 1], the following inequality holds C(m γ 1 (x 1 , y 1 , λ), ..., m γ m (x m , y m , λ)) ≥ m δ (C(x 1 , ..., x m ), C(y 1 , ...., y m ), λ).
Proof. Let γ i ≥ γ for all i = 1, ..., m be given and pick x, y ∈ [0, 1] m and λ ∈ [0, 1] in an arbitrary way. We begin by noting that m γ i (x i , y i , λ) ≥ m γ (x i , y i , λ) for any i ∈ {1, ..., m}, since this mapping is non-decreasing by Lemma 2.5.5. Copulae are distribution functions and hence are increasing in increasing arguments, so we get

C(m γ 1 (x 1 , y 1 , λ), ..., m γ m (x m , y m , λ)) ≥ C(m γ (x 1 , y 1 , λ), ..., m γ (x m , y m , λ)). (3.4.5)
Now, if γ = 0, the right-hand side of (3.4.5) is equal to

C(m γ (x 1 , y 1 , λ), ..., m γ (x m , y m , λ)) = C(m 1 (x γ 1 , y γ 1 , λ) 1 γ , ..., m 1 (x γ m , y γ m , λ) 1 γ ) (3.4.6)
and in the case γ = 0, we get

C(m 0 (x 1 , y 1 , λ), ..., m 0 (x m , y m , λ)) = C(exp (m 1 (log x 1 , log y 1 , λ)), ..., exp (m 1 (log x m , log y m , λ))). (3.4.7)
For z, w ∈ (0, 1], we can also derive that

m 1 (z γ , w γ , λ) ∈ [min {z γ , w γ } , max {z γ , w γ }],
when γ = 0 and m 1 (log z, log w, λ)

∈ [min {log(z), log(w)} , max {log(z), log(w)}]. This shows that (m 1 (x γ 1 , y γ 1 , λ), ..., m 1 (x γ m , y γ m , λ)) ∈ X(γ), when γ = 0 and (m 1 (log x 1 , log y 1 , λ), ..., m 1 (log x m , log y m , λ)) ∈ X(0).
Hence, since the mappings on the right-hand side of (3.4.6) and (3.4.7) are δ-concave we obtain the estimates

C(m γ 1 (x 1 , y 1 , λ), ..., m γ m (x m , y m , λ)) ≥ m δ (C(x 1 , ..., x m ), C(y 1 , ...., y m ), λ),
as was to be shown. Lemma 3.4.17. Let q ∈ (0, 1) m be a given point and let C : [0, 1] m → [0, 1] be a δ-γ-qconcave Copula. Assume furthermore that ∞ ≥ γ i ≥ γ, i = 1, ..., m is any sequence of numbers. Then for any x, y ∈ [q, 1] m and λ ∈ [0, 1] the following inequality holds:

C(m γ 1 (x 1 , y 1 , λ), ..., m γ m (x m , y m , λ)) ≥ m δ (C(x 1 , ..., x m ), C(y 1 , ...., y m ), λ).
Proof. Let γ i ≥ γ for all i = 1, ..., m be given and pick x, y ∈ [q, 1] m and λ ∈ [0, 1] in an arbitrary way. Using the arguments of the proof of Lemma 3.4.16 we can derive equations (3.4.6) and (3.4.7). It remains to show that

(m 1 (x γ 1 , y γ 1 , λ), ..., m 1 (x γ m , y γ m , λ)) ∈ X(q, γ),
when γ = 0 and

(m 1 (log x 1 , log y 1 , λ), ..., m 1 (log x m , log y m , λ)) ∈ X(q, 0).
Then we can apply again the δ-concavity inequality to derive the final estimate of the Lemma. To this end, pick z, w ∈ [q, 1] arbitrarily and consider the case γ ≥ 0. From monotonicity of the mapping u ∈ [0, 1] → u γ (or u ∈ (0, 1] → log(u)), we obtain for u ≥ q that u γ ≥ q γ (or log(u) ≥ log(q)). Altogether this implies m 1 (z γ , w γ , λ) ≥ min {z γ , w γ } ≥ q γ , i.e., m 1 (x γ , y γ , λ) ∈ X(q, γ). When γ < 0, the map u ∈ (0, 1] → u γ is decreasing, so u ≥ q yields u γ ≤ q γ . This in turn implies m 1 (z γ , w γ , λ) ≤ max {z γ , w γ } ≤ q γ , i.e., m 1 (x γ , y γ , λ) ∈ X(q, γ).

Eventual Convexity of the Feasible Set

As in the result by Kataoka ([121]), convexity of the feasible set M (p) cannot always be obtained for any probability level p. From a practical perspective this is not necessarily a problem since we are naturally looking for high p levels in problems of type (3.4.3). In some cases, we can show that the feasible set is convex if p is large enough. Convexity of feasible sets with high enough p is known as eventual convexity. In this section we will show that we can derive such eventual convexity if the Copulae, individual probability distribution functions and constraint mappings have some specific generalized concavity properties.

In the following theorem, we will provide conditions on Copulae, individual probability distribution functions and constraint mappings h such that eventual convexity of the feasible set can be asserted.

Theorem 3.4.18. Let ξ ∈ R m be a random vector with associated Copula C, and let h i : R n → R be functions such that

P[ξ ≤ h(x)] = C(F 1 (h 1 (x)), ..., F m (h m (x))), (3.4.8)
where F i are the marginal distribution functions of component i of ξ, i = 1, ..., m. Assume that we can find α i ∈ R, such that the functions h i are α i -concave and a second set of parameters γ i ∈ (-∞, ∞], b i > 0 such that either one of the following conditions holds:

1. α i < 0 and z → F i (z

1 α i ) is γ i -concave on (0, b α i i ] 2. α i = 0 and z → F i (exp z) is γ i -concave on [log b i , ∞) 3. α i > 0 and z → F i (z 1 α i ) is γ i -concave on [b α i i , ∞)
,

where i ∈ {1, ..., m} is arbitrary. If the Copula is either δ-γ-concave or δ-γ-F (b)-concave for γ ≤ γ i ≤ ∞, i = 1, ..., m, then the set M (p) := {x ∈ R n : P[ξ ≤ h(x)] ≥ p} is convex for all p > p * := max i=1,...,m F i (b i ).
Convexity can moreover be derived for all p ≥ p * if each individual distribution function F i , i = 1, ..., m is strictly increasing. In the specific case that α i ≥ 0, γ i -concavity of the distribution functions holding everywhere, for all i ∈ {1, ..., m} and C being a δ-γ-concave Copula, the set M (p) is convex for all p.

Proof. Pick any p > p * , x, y ∈ M (p), λ ∈ [0, 1] and i ∈ {1, ..., m} arbitrarily. Define x λ := m 1 (x, y, λ). Since all Copulae are dominated by the maximum-Copula, we get:

F i (h i (x)) ≥ min j=1,...,m F j (h j (x)) ≥ C(F 1 (h 1 (x)), ..., F m (h m (x))) ≥ p > p * ≥ F i (b i ). (3.4.9)
Now the latter entails

h i (x) ≥ b i . (3.4.10)
Estimate (3.4.10) also holds whenever p ≥ p * and F i is strictly increasing for each i = 1, ..., m. A similar estimate is obtained for y clearly. We make a case distinction 1. α i < 0: In this case λh i (x)

α i + (1 -λ)h i (y) α i ≤ max {h i (x) α i , h i (y) α i } ≤ b α i i 2. α i = 0: In this case λ log h i (x)+(1-λ) log h i (y) ≥ min {log h i (x), log h i (y)} ≥ log b i . 3. α i > 0: In this case λh i (x) α i + (1 -λ)h i (y) α i ≥ min {h i (x) α i , h i (y) α i } ≥ b α i i .
From monotonicity of the probability distribution function F i , and α i -concavity of g i we obtain

F i (h i (x λ )) ≥ F i (m α i (h i (x), h i (y), λ)) = F i ((λh i (x) α i + (1 -λ)h i (y) α i ) 1 α i ), (3.4.11)
whenever α i = 0 and

F i (h i (x λ )) ≥ F i (m 0 (h i (x), h i (y), λ)) = F i (exp (λ log h i (x) + (1 -λ) log h i (y))), (3.4.12)
when α i = 0. The mappings in the right-hand side are γ i concave by assumption on a specific domain. Since we have shown that our arguments map in this domain, we can apply γ i -concavity and obtain:

F i (h i (x λ )) ≥ m γ i (F i (h i (x)), F i (h i (y)), λ). (3.4.13)
Since i was fixed but arbitrary, the above equation holds for all i = 1, ..., m.

A Copula is strictly increasing in its arguments, so we get from (3.4.13):

C(F 1 (h 1 (x λ )), ..., F m (h m (x λ ))) ≥ C(m γ 1 (F 1 (h 1 (x)), F 1 (h 1 (y)), λ), ..., m γ m (F m (h m (x)), F m (h m (y)), λ)). (3.4.14)
If the Copula is δ-γ-concave everywhere then we can apply Lemma 3.4.16 to obtain:

C(m γ 1 (F 1 (h 1 (x)), F 1 (h 1 (y)), λ), ..., m γ m (F m (h m (x)), F m (h m (y)), λ)) ≥ m δ (C(F 1 (h 1 (x)), ..., F m (h m (x))), C(F 1 (h 1 (y)), ..., F m (h m (y))), λ) ≥ p, which together with (3.4.14) gives C(F 1 (h 1 (x λ )), ..., F m (h m (x λ ))) ≥ p, i.e., x λ ∈ M (p). If the Copula is δ-γ-F (b)-concave, we have F i (h i (x)) ≥ F i (b i )
and we can apply Lemma 3.4.17 to obtain the same expression. Altogether we have shown:

C(F 1 (h 1 (x λ )), ..., F m (h m (x λ ))) ≥ m δ (C(F 1 (h 1 (x)), ..., F m (h m (x))), C(F 1 (h 1 (y)), ..., F m (h m (y))), λ) ≥ p, which is equivalent with C(F 1 (h 1 (x λ )), ..., F n (h m (x λ ))) ≥ p, i.e.,
x λ ∈ M (p) as was to be shown.

The asserted convexity of M (p) regardless of p under the additional assumptions regarding the generalized concavity of h i and F i (for each i = 1, ..., m) follows since one can apply the estimates (3.4.11) and (3.4.12) for all x, y, regardless of the domains to which h i (x) and h i (y) belong. All other estimates of this proof also carry through. Therefore, since the request p ≥ p * was only needed to obtain (3.4.10), convexity of M (p) can be shown regardless of p in that case.

An unfortunate effect in this lemma is that p * depends somehow on the "worst" distribution function F i and it is only needed to obtain estimate (3.4.9). We can actually sharpen the bound on p * as the following theorem shows:

Theorem 3.4.19. Define the set D := {x ∈ R n : h i (x) ≥ b i , ∀i = 1, ..., m}
, where b i is as defined in Theorem 3.4.18 and we make the same assumptions on ξ, F i and the Copula.

Then the set D is convex and

D ∩ M (p) is convex for all p ≥ p * = C(F 1 (b 1 ), ..., F n (b n )).
Proof. Since the mappings h i are α i -concave for all i = 1, ..., m, the set D is indeed convex. Now from monotonicity of the Copula and the distribution functions

F i , x ∈ D implies C(F 1 (h 1 (x)), ..., F m (h m (x))) ≥ p * .
Since we dispose of estimate (3.4.10), for any x, y ∈ D, the rest of the proof of Theorem 3.4.18 carries through.

Finally, we provide one last extension of the theorem with asymptotic δ-concave Copulae.

Theorem 3.4.20. Let ξ ∈ R m be a random vector with associated Copula C, and let h i : R n → R be functions such that

P[ξ ≤ g(x)] = C(F 1 (h 1 (x)), ..., F m (h m (x))), (3.4.15)
where F i are the marginal distribution functions of component i of ξ with unbounded support. Assume that we can find α i ∈ R, such that the functions h i are α i -concave and a second set of parameters γ i ∈ [γ, ∞], b i > 0, γ > 0 such that either one of the following conditions holds:

1. α i < 0 and z → F i (z

1 α i ) is γ i -concave on (0, b α i i ] 2. α i = 0 and z → F i (exp z) is γ i -concave on [log b i , ∞) 3. α i > 0 and z → F i (z 1 α i ) is γ i -concave on [b α i i , ∞),
for all i ∈ {1, ..., m}. If the Copula is asymptotically δ-concave, then set

M (p) := {x ∈ R n : P[ξ ≤ h(x)] ≥ p} is convex for all p > p * := max i=1,...,m F i (b i ).
Proof. Pick x, y ∈ M (p), λ ∈ [0, 1] and i ∈ {1, ..., m} arbitrarily. Since the distribution functions F i are assumed to have unbounded support, it follows that

F i (h i (x)) < 1, F i (h i (y)) < 1 and F i (h i (λx + (1 -λ)y)) < 1.
From the definition of asymptotic δconcavity, it follows that one can find a γ > 0 such that F i (h i (z)) ≤ q i (γ) for all γ < γ, z = x, y or z = λx + (1λ)y and for all i = 1, ..., m. One can pick γ moreover such that γ ≤ γ. This shows that all arguments in the proof of Theorem 3.4.18 are in the set X(q, γ) for γ < γ and one can therefore apply that theorem to conclude the proof.

Remark 3.4.21. Our main Theorem 3.4.18 provides a link between generalized concavity requirements on the constraint mapping g, that of the one dimensional distribution functions F i , i = 1, ..., m and that of the Copula. This link was less apparent in the earlier results [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] since it was required to have α i < 0, γ i = 1 together with δ = γ = 0. The results presented here show that already in the setting of [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] an improvement could be obtained by remarking that concavity of z → F (z 1 α ) for one dimensional probability distribution functions F on a set K implies log-concavity on the same set. The result is clearly an extension of those obtained in [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF], since the independent Copula is 0-0-concave. An extension of interest is obtained only if either 1. More constraint mappings h can be allowed for. This is not truly the case, since any α-concave mapping is in particular β-concave for some β < 0. However we might be able to exploit better the true concavity properties of the mappings. This is shown in section 3.4.4.1.

2. The obtained probability level p * is lower. This is shown in the examples of section 3.4.4.2.

The class of δ-γ-concave

Copulae is larger than that of the log-exp-concave Copulae. This is shown in section 3.4.4.3.

These points are illustrated in section 3.4.4 through examples and results.

One key question if probability distribution functions with the required properties of Theorem 3.4.18 exist is answered positively already in the papers [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. It is shown that a specific property of the density known as r-decreasingness is sufficient (see Definition 2.5.21). Indeed this property induces, through Lemma 2.5.22 concavity of z → F (z -1 r ) on a set (0, t * ) -r for some t * > 0.

It turns out that many distribution functions have this property and the dependence of t * on r can be analytically computed for these distributions. We refer to table 1 of [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] for those results.

A generalization of Results

Improved use of Generalized Concavity of the Constraint Mappings

We begin by showing that Theorem 3.4.18 allows us to exploit the true generalized concavity properties of the constraint mappings h better. To this end consider the following example:

Example 3.4.23. Consider log-concave mappings h i and a log-exp-concave Copula together with a random vector having exponentially distributed components. Let λ denote the parameter of these exponential distributions. We can for instance take h 1 (x, y) = log(x + y) + 0.1 and h 2 (x, y) = log(2x + 3y) + 0.2, where h i : R2 + → R, i = 1, 2. In order to be able to apply Theorem 3.4.18 and derive convexity of the set M (p) for chance constraints structured as (3.4.2), we have to show that the mappings z → F (e z ) are logconcave, i.e., z → f (z) := log(1exp(-λe z )) has to be concave. This can be shown to hold on R. Pick x, y ∈ R and µ ∈ [0, 1] arbitrarily. From convexity of the exponential function we get -λ exp(m 1 (x, y, µ)) ≥ -λm 1 (exp(x), exp(y), µ). A second application then yields

-exp(-λm 1 (exp(x), exp(y), µ)) ≥ -m 1 (exp(-λ exp(x)), exp(-λ exp(y)), µ).
Altogether we get from strict increasingness of the log

log(1 + (-exp(-λ exp(m 1 (x, y, µ))))) ≥ log(m 1 (1 -exp(-λ exp(x)), 1 -exp(-λ exp(y)), µ)) ≥ m 1 (f (x), f (y), µ),
as was required. We can therefore apply Theorem 3.4.18 to show that convexity of M (p) holds for all p levels. Since any log-concave mapping is in particular r-concave for all r < 0, we could have also applied the earlier results obtained by [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. Then we would obtain convexity of the feasible set for p ≥ 1e r-1 . And this would hold for all r < 0, yielding convexity for p > 1e -1 = 0.63.

Improved Estimates of p *

Following Remark 3.4.21, Theorem 3.4.18 appears in a somewhat weaker form in [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF][START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. In particular, in those papers the authors show that if a density is r + 1-decreasing, then the constraint mapping z → F (z -1 r ) is concave on some set (0, t * r ). Now this corresponds to picking γ i = 1 in Theorem 3.4.18, but log-exp concavity of Copulae implies γ = 0, leaving room for a gap to be filled. Indeed the following Lemma is a trivial consequence of generalized concavity: Lemma 3.4.24. Let F : R → [0, 1] be a probability distribution function with a γ + 1decreasing density for some γ > 0. Then the function z → F (z -1 γ ) is α-concave on (0, (t # ) -γ ), with t # ≤ t * , where t * is as in Definition 2.5.21. Moreover this holds for all α ≤ 1.

Proof. This follows trivially since α-concavity implies β-concavity for all β ≤ α on the same set, so one can pick t # ≤ t * , potentially degenerate t # = t * .

An example shows that weaker concavity is actually obtained on a larger set, i.e., t # < t * . Example 3.4.25. We can get back to [110, example 4.1]. To this end we pick -1-concave mappings h i . In the above cited example the specific mappings h 1 (x, y) = z → F (1/z) is log-concave on some set (0, (t # ) -1 ). Assume that F is the distribution function of an exponential random variable with parameter λ. Then upon defining the mapping f (z) = log F (1/z) = log (1exp (-λ z )), we have to show that this mapping is concave. To this end we will compute the first and second derivative and we obtain for any z > 0:

f ′ (z) = (1 -(1 -exp (- λ z )) -1 )λz -2 f ′′ (z) = f ′ (z)[-2z -1 + λz -2 (1 -exp (- λ z )) -1 ]. Now z > 0 implies 0 < exp (-λ z ) < 1, yielding (1 -exp (-λ z )) -1 ) > 1 and therefore f ′ (z) < 0. So the sign of f ′′ (z) depends on that of [-2z -1 + λz -2 (1 -exp (-λ z )) -1 ].
From the above estimate we get

-2z -1 + λz -2 (1 -exp (- λ z )) -1 ≥ -2z -1 + λz -2 , (3.4.16)
so indeed f ′′ (z) < 0 for small z (but we knew this already, since the exponential density is 2-decreasing). It turns out that f ′′ (z) = 0 if and only if exp (-λ z ) = 1 -λ 2z . We can compute this z # by using for instance a dichotomy procedure (picking λ = 1, yielding z # = 0.62750048, i.e., t # = 1.59362426). The obtained p * of Theorem 3.4.18 is then equal to p * = 1exp (-λt # ) = 0.7968121, which is significantly better than the earlier obtained p * = 0.864. Empirically varying λ yields the result that the improved p * does not depend on λ, similarly to the results obtained in [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF].

We can also look at Theorem 3.4.19 and look at the set

M (p) ∩ x ∈ R n : h i (x) ≥ t # for p ≥ p * = C(F 1 (t # ), ..., F m (t # ))
. Now this results depends on the Copula. Picking the Maximum Copula we get p * = 0.7968121 as before. Picking the Independent Copula we get p * = 0.7968121 2 = 0.6349 and the Gumbel Copula with θ = 1.1 yields p * = 0.652770. The latter result does not depend on θ, at least as found empirically.

Example 3.4.26. Returning once again to [110, example 4.1] and example 3.4.25 above, we can likewise stipulate that the components of ξ follow a standard normal distribution. These components are linked through a log-exp concave Copula. In the paper [START_REF] Henrion | Convexity of chance constraints with independent random variables[END_REF] the authors have shown that the map z → Φ(1/z) is concave on (0, 1/ √ 2). This then yields a bound of p * = Φ( 1 √ 2 ) = 0.921, where Φ is the probability distribution function of a standard normal random variable in dimension 1. In order to be able to apply Theorem 3.4.18 and derive convexity of the set M (p) for chance constraints structured as (3.4.2), we have to show that z → Φ(1/z) is log-concave only. Calling f (z) = log(Φ(1/z)), we can compute and obtain for z > 0:

f ′ (z) = -z -2 Φ ′ (z -1 )Φ(z -1 ) -1 f ′′ (z) = f ′ (z -1 )(-2z -1 + z -3 + z -2 Φ ′ (z -1 )Φ(z -1 ) -1 ),
where we have used that Φ ′′ (z -1 ) = -z -1 Φ ′ (z -1 ). Indeed, f ′ (z) < 0 for z > 0 and f ′′ (z) < 0 for z > 0 small enough. We can again numerically compute z # such that f ′′ (z # ) = 0, i.e., -2z -1 + z -3 + z -2 Φ ′ (z -1 )Φ(z -1 ) -1 = 0 at z = z # , and obtain z # = 0.754205. Convexity of the feasible set is then obtained for p ≥ p * = Φ( 1 z # ) = 0.90756, from Theorem 3.4.18 which is only marginally better. Applying Theorem 3.4.19 however and distinguishing Copulae allows us to obtain an improved p * level for the Gumbel and Independent Copula. Indeed, then we get p * = 0.8334 and p * = 0.824 respectively.

More Copulae

The last extension that Theorem 3.4.18 allows for is that we can use more Copulae, if the class of δ-γ-concave Copulae contains more Copulae than just log-exp-concave Copulae. This turns out to be the case. Indeed the following Lemma shows that the Clayton Copula is δ-γ-concave for specific δ values. It was however shown in [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] that it was not log-exp-concave. Lemma 3.4.27. Let θ > 0 be the parameter of the strict generator ψ :

[0, 1] → R + , ψ(t) = θ -1 (t -θ -1) of the Clayton Copula. This Copula is δ-γ-concave for all γ > 0 provided that δ ≤ -θ < 0.
Proof. The inverse of the generator is given by ψ -1 (s) = (θs + 1) -1 θ , so the Copula is defined as

C(u) = ψ -1 ( n i=1 ψ(u i ))
, where u ∈ [0, 1] m . We begin by computing the derivatives of the generator

dψ dt (t) = -t -θ-1 dψ -1 ds (s) = (θs + 1) -1 θ -1 = -ψ -1 (s)(θs + 1) -1 .
We will consider the mapping f (u) = C(u 1 γ ) δ and have to show that it is convex. From Remark 3.4.3 it follows that it is sufficient to consider u ∈ (0, 1] m only. This implies that C(u 1 γ ) > 0 and in particular that f (u) and its derivatives are well defined. We therefore fix u ∈ (0, 1] m arbitrarily and compute the first and second order derivatives of f . A computation gives

∂f ∂u k (u) = δC(u 1 γ ) δ-1 ∂C ∂z k (u 1 γ ) 1 γ u 1 γ -1 k , k = 1, ..., m. (3.4.17)
When computing the derivative of the Copula, we obtain:

∂C ∂u k (u) = dψ -1 ds ( m i=1 ψ(u i )) dψ dt (u k ) = C(u)(θ m i=1 ψ(u i ) + 1) -1 u -θ-1 k , k = 1, ..., m (3.4.18)
Substituting altogether we get:

∂f ∂u k (u) = δ γ C(u 1 γ ) δ (θ m i=1 ψ(u 1 γ i ) + 1) -1 u -θ+γ γ k , k = 1, ..., m. (3.4.19)
Deriving a second time, we get

∂ 2 f ∂u 2 k (u) = δ γ (θ m i=1 ψ(u 1 γ i ) + 1) -1 u -θ+γ γ k ∂ ∂u k C(u 1 γ ) δ + δ γ C(u 1 γ ) δ u -θ+γ γ k ∂ ∂u k (θ m i=1 ψ(u 1 γ i ) + 1) -1 + δ γ C(u 1 γ ) δ (θ m i=1 ψ(u 1 γ i ) + 1) -1 ∂ ∂u k u -θ+γ γ k , k = 1, ..., m.
In the first line we can just substitute (3.4.19), in the second line we obtain

∂ ∂u k (θ m i=1 ψ(u 1 γ i ) + 1) -1 = -(θ m i=1 ψ(u 1 γ i ) + 1) -2 θ dψ dt (u 1 γ k ) 1 γ u 1 γ -1 k = θ γ (θ m i=1 ψ(u 1 γ i ) + 1) -2 u -θ+γ γ k , k = 1, ..., m.
and in the third line,

∂ ∂u k u -θ+γ γ k = - θ + γ γ u -θ+γ γ -1 k , k = 1, ..., m. (3.4.20)
For the cross-derivative, j, k = 1, ..., m, the expression becomes:

∂ 2 f ∂u j ∂u k (u) = δ γ (θ m i=1 ψ(u 1 γ i ) + 1) -1 u -θ+γ γ k ∂ ∂u j C(u 1 γ ) δ + δ γ C(u 1 γ ) δ u -θ+γ γ k ∂ ∂u j (θ m i=1 ψ(u 1 γ i ) + 1) -1 , j = k
In the first line one substitutes (3.4.19) with k replaced by j and the second is dealt with similarly as before. Combining these expression we obtain for all u ∈ (0, 1] m , k, j = 1, ..., m:

∂ 2 f ∂u 2 k (u) = δ 2 + δθ γ 2 C(u 1 γ ) δ (θ m i=1 ψ(u 1 γ i ) + 1) -2 (u -θ+γ γ k ) 2 - δ γ θ + γ γ C(u 1 γ ) δ (θ m i=1 ψ(u 1 γ i ) + 1) -1 u -θ+γ γ -1 k ∂ 2 f ∂u j ∂u k (u) = δ 2 + δθ γ 2 C(u 1 γ ) δ (θ m i=1 ψ(u 1 γ i ) + 1) -2 u -θ+γ γ k u -θ+γ γ j , j = k.
We have to show that the Hessian is positive semi-definite for all u ∈ (0, 1] m , we will do this directly, by picking z = 0, z ∈ R m and forming z T ∇ 2 f (u)z. The special structure of the above second-derivatives is as follows

∂ 2 f ∂u 2 k (u) = α(u)x 2 k + β k (u) and ∂ 2 f ∂u j ∂u k (u) = α(u)x j x k . Now z T ∇ 2 f (u)z = m j=1 m k=1 z j ∂ 2 f ∂u j ∂u k (u)z k = j,k,j =k α(u)z j x j x k z k + m k=1 α(u)z 2 k x 2 k + z 2 k β k (u) = α(u) m j=1 m k=1 z j x j x k z k + m k=1 z 2 k β k (u) = α(u)( m j=1 z j x j ) 2 + m k=1 z 2 k β k (u).
Substituting the above expressions, we obtain:

z T ∇ 2 f (u)z = C(u 1 γ ) δ δ 2 + δθ γ 2 (θ m i=1 ψ(u 1 γ i ) + 1) -2 ( m i=1 z i u -θ+γ γ i ) 2 -C(u 1 γ ) δ δ γ θ + γ γ (θ m i=1 ψ(u 1 γ i ) + 1) -1 m i=1 z 2 i u -θ+γ γ -1 i . (3.4.21)
Now with our choice of δ, we obtain δ 2 + δθ ≥ 0 and similarly -δ γ ≥ 0. Together with u ∈ (0, 1] m , one can see that all expressions in (3.4.21) are positive. This therefore yields z T ∇ 2 f (u)z ≥ 0, i.e., f is a convex function, as was to be shown. Remark 3.4.28. Numeric evidence would indicate that the Clayton Copula is also δγ-concave for δ slightly bigger than -θ, but not for all δ < 0. Indeed, in dimension 2, picking δ = -0.03, θ = 0.1, γ = 0.5 and evaluating the above Hessian at the point u = (0.96, 0.985), one obtains a negative eigenvalue. Remark 3.4.29. We can also prove that the Clayton Copula is δ-0 concave for δ ≤ -θ.

Indeed setting f (u) = C(e u ) δ one obtains for u ∈ (-∞, 0] m : ∂f ∂u k (u) = δC(e u ) δ (θ m i=1 ψ(e u i ) + 1) -1 e -θu k , k = 1, ..., m
and for j, k = 1, ..., m:

∂ 2 f ∂u 2 k (u) = (δ 2 + δθ)C(e u ) δ (θ m i=1 ψ(e u i ) + 1) -2 e -2θu k -θδC(e u ) δ (θ m i=1 ψ(e u i ) + 1) -1 e -θu k ∂ 2 f ∂u j ∂u k (u) = (δ 2 + δθ)C(e u ) δ (θ m i=1
ψ(e u i ) + 1) -2 e -θu k e -θu j , j = k.

And the same results follow.

We have shown in lemma 3.4.27 that the Clayton Copula is δ-γ-concave for all γ > 0 and δ ≤ -θ. Since this Copula is not log-exp concave, the results of [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF] could not be applied. We can however use our theorem to derive convexity of feasible sets M (p). As such we get the example:

Example 3.4.30. Consider again the same setting as that of example 3.4.23, except that this time we use the Clayton Copula to link the components of ξ together. Since this Copula is δ-γ-concave for any γ > 0 and δ ≤ -θ, we have to show that the mappings z → F (1/z) are γ-concave on some set (0, (t # ) -1 ). Since results hold in particular for γ < 1, concavity of those maps suffices. That is obtained whenever their densities are for instance 2-decreasing. Assuming that ξ follows an exponential distribution with parameter λ, we obtain the very rough bound p * = 1e -2 = 0.864. But we can do better, to this end we have to show that the mapping

f (z) = F (1/z) γ = (1 -exp (-λ z )) γ , is concave. A computation gives f ′ (z) = -λγz -2 (1 -exp (-λz -1 )) γ-1 exp (-λz -1 ) f ′′ (z) = f ′ (z)[-2z -1 -(γ -1)λz -2 (1 -e -λz -1 ) -1 e -λz -1 + λz -2 ].
Applying Theorem 3.4.18, and picking for instance γ = 1 2 , one obtains z # = 0.54807, giving the estimate p * = 0.8387. With γ = 0.01, one obtains z # = 0.62537 and the estimate p * = 0.7979. Again p * does not depend on λ.

We can also apply Theorem 3.4.19 to obtain p * = 0.638 for θ = 0.1. This time the result depends on θ as can be shown empirically.

A Partial Characterization of the Gaussian Copula

So far we have only provided examples of Archimedean Copulae with the δ-γ-concavity property. In this section we investigate the Gaussian Copula and provide a partial characterization of the δ-γ-concavity properties of this Copula. Lemma 3.4.31. Let R be an m × m correlation matrix. Assume furthermore that the m × m inverse matrix is such that Q := R -1 -I has only positive components. Let δ = min i=1,...,m max j=1,...,m Q ij and define q = max log(Φ( 1 δ )), log(Φ(1)) , then the density of the Gaussian Copula is 0-0-concave for all u ∈ [q, 0] m , where Φ is the probability distribution function of a standard normal random variable.

Proof. Define the mapping f : (-∞, 0] m → R m by setting

f (u) = Φ -1 (e u ) = (Φ -1 (e u 1 ), ..., Φ -1 (e u m )),
where Φ -1 is the inverse of the standard normal distribution function. We wish to show that the density of the Gaussian Copula wherein we substitute e u , is eventually logconcave, i.e., the mapping u → c R (e u ) is concave. Now the density c R of the Gaussian Copula is given by

c R (u) = 1 √ det R exp (-0 .5Φ -1 (u) T (R -1 -I )Φ -1 (u)). (3.4.22)
This boils down to showing that the mapping u → f (u

) := f (u) T (R -1 -I)f (u) is convex.
We will begin by computing some derivatives. To this end, fix any u ∈ (-∞, 0] m completely arbitrarily. For convenience we will note Q := R -1 -I. We begin by computing the first and second derivative of a component of the vector f (u), i.e., we compute the derivatives of the 1-dimensional mapping v ∈ (-∞, 0] → f i (v) for arbitrary i = 1, ..., m:

df i dv (v) = dΦ -1 dv (e v )e v = √ 2π exp ( 1 2 Φ -1 (e v ) 2 )e v , i = 1, ..., m, ∀v ∈ (-∞, 0]
and

d 2 Φ -1 ds 2 (s) = √ 2π exp ( 1 2 Φ -1 (s) 2 ) 1 2 2Φ -1 (s) dΦ -1 ds (s) = ( √ 2π exp ( 1 2 Φ -1 (s) 2 )) 2 Φ -1 (s), ∀s ∈ (0, 1).
Giving

d 2 f i dv 2 (v) = d 2 Φ -1 ds 2 (e v )e v e v + df i dv (v) = f i (v)( df i dv (v)) 2 + df i dv (v), i = 1, ..., m, ∀v ∈ (-∞, 0].
Now we can compute the derivatives of the mapping f ,

∂ f ∂u k (u) = 2 df k du (u k ) m j=1 Q kj f j (u j ), k = 1, ..., m
and the second derivatives are:

∂ 2 f ∂u 2 k (u) = 2 d 2 f k du 2 (u k )Q kk f k (u k ) + 2 df k du (u k )Q kk df k du (u k ) + 2 m j=1,j =k d 2 f k du 2 (u k )Q kj f j (u j ), k = 1, ..., m ∂ 2 f ∂u j ∂u k (u) = 2 df k du (u k )Q kj df j du (u j ), j, k = 1, ..., m, j = k
Substituting in the above expression the previously found identity for the second derivative of f i , we obtain

∂ 2 f ∂u 2 k (u) = 2 df k du (u k )Q kk df k du (u k ) + 2 m j=1 d 2 f k du 2 (u k )Q kj f j (u j ) = 2 df k du (u k )Q kk df k du (u k ) + 2 df k du (u k ) 2 [(f k (u k ) + ( df k du (u k )) -1 ) m j=1 Q kj f j (u j ))], k = 1, ..., m
where we have used that df k du (u k ) > 0 for all u ∈ (-∞, 0] m . Now picking z ∈ R m and forming z T ∇ 2 h(u)z, one obtains the following

z T ∇ 2 f (u)z = 2 m i,j=1 z i df i du (u i )Q ij df j du (u j )z j + 2 m k=1 z 2 k df k du (u k ) 2 [(f k (u k ) + ( df k du (u k )) -1 ) m j=1 Q kj f j (u j ))],
which holds for all u ∈ (-∞, 0] m as u was chosen arbitrarily.

If we define the vector x(u) ∈ R m as x(u) = (z 1

df 1 du (u 1 ), ..., z m df m
du (u m )), and the matrix Q(u) as

Q(u) ij = Q ij if i, j = 1, ..., m, i = j Q ii + (f i (u i ) + ( df i du (u i )) -1 ) m j=1 Q ij f j (u j )) if i = 1, ..., m It is clear that z T ∇ 2 f (u)z = x(u) T Q(u)x(u) and z T ∇ 2 f (u)z ≥ 0 if and only if Q(u) is positive semi-definite. Defining the vector α(u) ∈ R m as the diagonal of Q(u) minus the diagonal of Q, it is clear that Q(u) = Q + diag α(u)
, where diag α(u) is the diagonal matrix with elements of the vector α(u). It therefore follows that the eigenvalues of Q(u) are those of R -1 to which we add α(u) -1. Making sure that α i (u) -1 ≥ 0, ∀i = 1, ..., m is therefore sufficient for Q(u) to be positive semi-definite. From the conditions of the lemma it follows that m j=1 Q ij f j (u j ) ≥ δf j * (u j * ), ∀i = 1, ..., m, where j * is that element of i-th line of Q with Q ij * ≥ δ. Since the mapping f is strictly increasing, it follows m j=1 Q ij f j (u j ) ≥ δf j * (q) ≥ 1, ∀i = 1, ..., m. The choice of q also implies f j (u j ) ≥ 1, ∀j = 1, ..., m. Altogether we obtain α i (u) ≥ 1, i = 1, ..., m for all u ≥ q and the result follows.

Remark 3.4.32. The condition that Q = R -1 -I needs to have only positive components is not very restrictive. For instance if m = 2, this implies that the off-diagonal element needs to be non-positive. When m = 3 such a simple characterization is not possible any longer. To show that cases exist satisfying the condition on Q (m ≥ 3), we can consider as an example the three dimensional matrix with R 12 = -0.9, R 13 = -0.7 and R 23 = 0.5.

Example 3.4.33. Let R be the correlation matrix in dimension R 2 having -0.9 on the off-diagonal. Then δ = 4.74 and q = max {-0.54, -0.17}. In fact for all 2-dimensional matrices with off-diagonal element ρ ≤ 1- √ 5

2 , this same bound holds. Lemma 3.4.34. Let R be an m times m correlation matrix. Assume furthermore that the inverse matrix is such that Q := R -1 -I has only positive components. Let δ = min i=1,...,m min j=1,...,m Q ij > 0 and define q = max log(Φ( 1 δ )), log(Φ(1)) , where Φ is the probability distribution function of a standard normal random variable. Then the Gaussian Copula admits the following decomposition

C(e u ) = C(e q ) + 2 m i=2 ϕ i (u),
where each function ϕ i : [0, 1] m → [0, 1] is a 0-concave probability distribution up to a multiplicative constant, i = 2, ..., 2 m . This decomposition holds for all u ∈ [q, 0] m . Proof. In the proof of Lemma 3.4.31 we have seen that the density of the Gaussian Copula is "eventually" 0-0-concave. As a matter of fact, when looking at the final estimates, one can also derive m j=1 Q ij f j (u j ) ≥ δf j (u j ), ∀i = 1, ..., m, ∀j = 1, ..., m, since this time δ is defined differently. This shows that this density is 0-0-concave, whenever u j ≥ q for a single j ∈ {1, ..., m}. Now let C 1 , ..., C n , with n = 2 m be the partition of [0, 1] m into orthants based at q, i.e.,

C 1 = {x ∈ (-∞, 0] m , x ≤ q}, C n = {x ∈ (-∞, 0] m , x
C(e u ) = n i=1 m j=1 (-∞,u j ] fi (s)ds = m j=1 (-∞,u j ] f1 (s)ds + n i=2 ϕ i (u).
If u ∈ [q, 0] m , the first element is nothing else but C(e q ), since C(e q ) = m j=1 (-∞,q j ] f1 (s)ds = C 1 f1 (s)ds.

Example 3.4.35. Let R be the correlation matrix in dimension R 2 having -0.9 on the off-diagonal. Then δ = 4.263 and q = max {-0.526, -0.17}

A Potential Application

In some engineering problems in energy one easily stumbles across mixed laws in "columns". In particular, offer demand equilibrium constraints in unit commitment require that we commit a production schedule producing enough energy in most situations. However, uncertainty is only discovered later. This uncertainty consists of load uncertainty and uncertainty on renewable generation such as wind power. Wishing to produce enough electricity in most situations for all time steps simultaneously would then result in a constraint of the type

p ≤ P[ξ + η ≤ h(x)], (3.4.23)
where h : R n → R m is the mapping associating with a decision vector x ∈ R n its actual production level and ξ ∈ R m and -η ∈ R m are two random vectors, modeling for instance load uncertainty and wind generation respectively. Now to show that Copulae can be used to obtain a convex model requires an additional result. Theorem 4.2.3. of [START_REF] Prékopa | Stochastic Programming[END_REF] (dating back to [START_REF] Davidovich | A property of logarithmically concave functions[END_REF][START_REF] Borell | Convex set functions in d-space[END_REF][START_REF] Brascamp | On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log-concave functions and with an application to the diffusion equations[END_REF], [213, proof in dimension 1]) indicates that the convolution of logconcave densities is again a log-concave density. As such, picking each ξ i , η i individually following a log-concave density, it follows that their sum follows a log-concave density, i.e., F i (z) = P[ξ i + η i ≤ z] is a log-concave function. If we now use a δ-γ-concave Copula with γ ≤ 0 and concave functions h i (x), the set defined by (3.4.23) will then turn out to be eventually convex. Since in practice h i is often linear, as a sum of production levels, it will be concave. Hence we can come up with a tractable model for such a mixed law setting.

Chapter 4

Algorithms for (convex) Probabilistic Programming

Optimization problems involving probabilistic constraints of type (2.0.1) can be transformed into convex optimization problems under some additional assumptions. For example if g is jointly quasi-concave and ξ has a log-concave density, then Theorem 2.5.15 allows us to assert that x → log(p)log(P[g(x, ξ) ≥ 0]) is a convex mapping. Under such assumptions solving the probabilistically constrained optimization problem amounts to solving a convex constrained problem. However as indicated by J. Mayer ([149]) the probabilistic constraint, even if continuously differentiable, might be stiff. It is therefore preferable to consider the tools from non-smooth optimization. An elementary, but convincing example of this statement is the optimization of the stiff differentiable "maxanal" function ([20, Section 12.1.2]). It turns out that methods from smooth optimization are outperformed by methods from non-smooth analysis (e.g., bundle methods).

Returning to optimization problems with probabilistic constraints, classically only the supporting hyperplane method has been employed. Now the bottle-neck resides in the computation of the (sub-)gradient. This is not only time-consuming but can also only be done up to some precision. The latter precision can be controlled but a trade-off with the aforementioned computational effort has to be found. These gradients can be used to compute a linearization of the convex constraint induced by the probabilistic constraint. However, in general, we can not assert that the derived linearization remains below the convex mapping. The latter feature leads to the definition of a so-called upper oracle in section 4.1. A proximal and level bundle method for dealing with convex constrained optimization involving such upper-oracles are derived in sections 4.1 and 4.2. As a consequence a framework is provided for solving (convex) probabilistically constrained optimization problems efficiently.

A proximal Bundle Method

Real-life problems are often modeled in an uncertain setting, to better reflect unknown phenomena specific to the application. In particular, such is the case in the energy sector; see [START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF]. For the numerical experience of this section1 we focus on a specific energy problem arising in stochastic unit commitment ( [START_REF] Carpentier | Stochastic optimization of unit commitment: a new decomposition framework[END_REF], [START_REF] Takriti | A stochastic model for the unit commitment problem[END_REF], [START_REF] Nowak | A stochastic integer programming model for incorporating day-ahead trading of electricity into hydro-thermal unit commitment[END_REF], [START_REF] Wallace | Stochastic programming models in energy[END_REF], [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF]). This is the problem of optimally managing reservoirs of a hydro valley in the short term.

A hydro valley is a set of power plants cascaded along the same hydrological bassin. For the considered system, part of Electricité de France mix, uncertainty is mostly related to the amount of melted snow arriving as a streamflow to the most uphill reservoirs. The volume of these reservoirs changes with the inflows and determines the amount of water that can be converted into energy. After turbining water to produce power, the upstream reservoirs release a certain volume that fills the reservoirs downstream, and the process continues until the power plant at the bottom of the bassin. In this interconnected context, it is important to jointly manage the generation of the cascaded plants in a manner that not only is economical but also reliable. More precisely, it is crucial to keep the volume of each reservoir in the valley between prescribed minimum and maximum levels (to prevent floods, to ensure touristic activities, etc). Since it is not realistic to ensure such conditions for every possible streamflow, satisfaction of lower and upper bounds for the volumes can be required in a probabilistic manner.

Introduced by [START_REF] Charnes | Chance-constrained programming[END_REF], probability constraints are quite an appealing tool for dealing with uncertainty, because they give a physical interpretation to risk. For hydro valley management, chance constraints have been employed in [START_REF] Loucks | Water Resource Systems Planning and Analysis[END_REF][START_REF] Duranyildiz | A chance-constrained LP model for short term reservoir operation optimization[END_REF][START_REF] Edirisinghe | Capacity planning model for a multipurpose water reservoir with target-priority operation[END_REF][START_REF] Loiaciga | On the use of chance constraints in reservoir design and operation modeling[END_REF][START_REF] Morgan | Aquifer remediation design under uncertainty using a new chance constraint programming technique[END_REF][START_REF] Zorgati | Supply shortage hedging: estimating the electrical power margin for optimizing financial and physical assets with chance-constrained programming[END_REF][START_REF] Zorgati | Optimizing financial and physical assets with chance-constrained programming in the electrical industry[END_REF][START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF].

Most of these works require each component of the uncertain constraint to be satisfied in a probabilistic sense in a separate manner. As explained in [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF], a stochastic model with such individual chance constraints may sometimes result in unreliable optimal decisions, because there is no guarantee that the whole stochastic inequality will be satisfied with a given probability. In this work we build upon the model in [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF], with joint chance constraints, and derive a sound numerical solution procedure, based on bundle methods, [START_REF] Hiriart-Urruty | Convex Analysis and Minimization Algorithms II[END_REF][START_REF] Bonnans | Numerical Optimization: Theoretical and Practical Aspects[END_REF].

In the classical textbook [START_REF] Prékopa | Stochastic Programming[END_REF], convexity of chance constraints is ensured for a variety of distributions. Accordingly, for appropriate choices of the stochastic model for the inflows (cf. Section 5.1 for details), the hydro valley management problem has the abstract form

min x∈X⊆R n {f (x) : c(x) ≤ 0} (4.1.1)
for f and c finite-valued convex continuous functions and X a compact convex polyhedron. Even when c can be shown to be differentiable ( [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF]), it is reported in [START_REF] Mayer | On the Numerical solution of jointly chance constrained problems[END_REF] that joint chance constraints are occasionally sufficiently stiff for smooth optimization methods to encounter convergence issues. Moreover in many applications c is actually non-smooth ( [START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF]). For this reason it is of interest to make no smoothness assumption on c. For algorithmical purposes, given any x ∈ X, joint chance constraints (corresponding to c in (4.1.1)) need to be efficiently computed, together with a gradient. Gradient formulae for multi-variate Gamma, Dirichlet, and Gaussian distributions can be found in [START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF], [START_REF] Prékopa | Stochastic Programming[END_REF][START_REF] Gouda | On numerical calculation of probabilities according to dirichlet distribution[END_REF][START_REF] Szántai | Numerical evaluation of probabilities concerning multi-dimensional probability distributions[END_REF], and [START_REF] Prékopa | Stochastic Programming[END_REF][START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF][START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF], respectively. We also refer to Section 2.7.

Similarly to evaluating the function, these formulae involve computing a probability. In turn, the calculation of a probability amounts to compute, for any given x, an integral in relatively high dimension (for our numerical application in Section 4.1.6, the corresponding dimension is 48). For multi-variate Gaussian distributions, the code developped by A. Genz can be used to efficiently approximate probabilities; [START_REF] Genz | Numerical computation of multivariate normal probabilities[END_REF][START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF]. The numerical method therein outputs values that can be as accurate as required on input, provided enough time can be spent in the calculation. Since the numerical solution of (4.1.1) requires evaluating the constraint c (and a gradient) at many trial points x, the evaluation is not done exactly, but with some error, whose sign is unknown. As a result, and in spite of convexity, a linearization of the form c x + g c x , •x with c x ≈ c(x) and g c x an approximate subgradient, may locally overestimate the function c(•). An oracle providing such linearizations will be referred to as an upper inexact oracle. This naming is somewhat unsatisfactory because the linearizations need not overestimate the function c at all or would only do so locally. A more verbose choice would have been locally upper inexact oracle. This choice is also somewhat unsatisfactory as the provided linearization may overestimate c on a rather large set. We will therefore speak of upper inexact oracles.

To circumvent this difficulty, in this section we present a bundle method specially taylored to solve problems of the form (4.1.1) when computing f and/or c (as well as respective gradients) is computationally heavy. The algorithm is special because it solves a constrained non-smooth problem based on the information provided by an inexact oracle, possibly of upper type. This means that the oracle output provides linearizations for f and c in (4.1.1) that are inexact and may locally overestimate the corresponding function.

The simpler case of lower oracles, yielding linearizations that always remain below the convex function, is also considered as a corollary. The convergence analysis of bundle methods with lower oracles is simpler, because it fits better the usual exact framework, in which the oracle linearizations define cutting planes for the function of interest (f and c in our case).

For unconstrained problems, bundle methods dealing with inexact oracles can be found in [START_REF] Hintermüller | A proximal bundle method based on approximate sub-gradients[END_REF][START_REF] Solodov | On approximations with finite precision in bundle methods for nonsmooth optimization[END_REF][START_REF] Kiwiel | A proximal bundle method with approximate subgradient linearizations[END_REF][START_REF] Emiel | Incremental like bundle methods with applications to energy planning[END_REF][START_REF] Fábián | Bundle-type methods for inexact data[END_REF][START_REF] Oliveira | Inexact bundle methods for two-stage stochastic programming[END_REF]. Most of these works consider only lower oracles; we refer to [START_REF] Oliveira | Inexact bundle methods for two-stage stochastic programming[END_REF] for a discussion on how such a setting considerably simplifies the convergence analysis. For constrained problems like (4.1.1), inexact bundle methods are more rare; see [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF][START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF][START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF]. These works consider oracles that are either lower ones, or asymptotically exact. In this section, we give a method suitable for the more general upper setting, and hence, adapted to the hydro application of interest.

This section is organized as follows. Section 4.

Designing Bundle methods for Constrained Optimization

Following the lead of [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF], the numerical solution of (4.1.1) is addressed by means of an improvement function H τ : R n → R defined by

H τ (y) = max f (y) -τ 1 , c(y) -τ 2 , for suitably chosen scalar targets τ 1 , τ 2 . (4.1.2)
However, unlike the exact setting considered in [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF], the oracles which provide function and subgradient values for f and c make calculations with some error. For this reason, our method minimizes approximations of H τ , built using oracle information computed with some inaccuracy. For clarity, at any point x ∈ R n the symbols f (x) and c(x) refer to exact function values. Following [START_REF] Kiwiel | A proximal bundle method with approximate subgradient linearizations[END_REF], to denote inexact values the argument is put as a subscript, like in f x , c x (for functions) and g f x and g c x (for subgradients). Accordingly, given an iterate x j ∈ X, the oracle provides f x j and g f x j shortened for convenience to f j = f x j and g j f = g f x j , and similarly for the c-values.

Initial setting

We assume that at any x j ∈ X, the oracle provides f j and c j , estimates for the functional values, as well as g j f and g j c , estimates for the respective subgradients.

(4. 1.3) Since in this oracle the signs of the errors, e.g., f (x j )f j , are not specified, the true function values can be either overestimated or underestimated, and similarly for the subgradients. In particular, nothing is known on the linearizations, e.g., f j + g j f , •x j , that may locally overestimate the corresponding function, e.g., f . Further conditions on the (possibly upper) inexact oracle will be required in what follows, as needed (cf. Along the iterations the method keeps aside a reference solution called the stability center. The current stability center is denoted by xk at iteration k and has function values denoted by f k and ĉk . This center corresponds to some past iterate that was singled out because it produced significant progress towards the goal of solving (4.1.1). Progress is measured with respect to the current approximation of the improvement function, in a sense to be made clear below. Specifically, the k-th inexact improvement function is

h k y = max f y -τ k 1 , c y -τ k 2 where τ k 1 = f k + ρ k max(ĉ k , 0) for ρ k ≥ 0 τ k 2 = σ k max(ĉ k , 0) for σ k ≥ 0 . ( 4
.1.4) In the expression above, the targets τ k are more general than those considered in [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF], which correspond to taking null penalties ρ k and σ k . Relations with other improvement functions in the literature that are also covered by the setting (4.1.4) are discussed in § 4.1.4.1.

The oracle output is collected along iterations to form the Bundle of information

B k = {x k , f k , ĉk } ∪ (x j , f j , c j , g j f , g j c ) : j ∈ J k for J k ⊂ {1, . . . , k} .
Having this information, the k-th inexact improvement function is modelled by a convex function M k : R n → R which uses approximate cutting-plane functions fk and čk :

M k (y) = max fk (y)-τ k 1 , čk (y)-τ k 2 where fk (y) = max f j + g j f , y -x j : j ∈ J k čk (y) = max c j + g j c , y -x j : j ∈ J k . (4.1.5)
To generate iterates, the algorithm chooses a prox-parameter µ k > 0 and solves the quadratic program (QP)

x k+1 = arg min y∈X {M k (y) + 1 2 µ k y -xk 2 } . (4.1.6)
As a result, x k+1 ∈ X and

x k+1 = xk - 1 µ k (G k + ν k ) where G k ∈ ∂M k (x k+1 ) and ν k ∈ N X (x k+1 ) , (4.1.7)
where N X (x k+1 ) denotes the normal cone (of convex analysis) of X at the new iterate and ∂M k (x k+1 ) the sub-gradient of M k at x k+1 . After solving the problem, the aggregate linearization

M k (y) = M k (x k+1 ) + G k , y -x k+1 , (4.1.8)
which is an affine function, can be defined. Clearly, because

G k ∈ ∂M k (x k+1 ), M k (y) ≤ M k (y) for all y ∈ R n . (4.1.9)
The last ingredient in the bundle method is given by the aggregate error, defined by

E k = h k xk -M k (x k ) -ν k , xk -x k+1 . (4.1.10)
In view of (4.1.8), for any y it holds that

G k = ∇M k (y). Therefore, because M k (x k ) = M k (y) + G k , xk -y with M k ≤ M k by (4.
1.9), we derive the relation

h k xk -E k ≤ M k (y) + G k + ν k , xk -y for all y ∈ X , (4.1.11)
where we used the fact that the term ν k , x k+1y is nonnegative for all y ∈ X, because ν k ∈ N X (x k+1 ).

Handling inexact oracle information

Usually, the noise introduced by the inexact evaluations is deemed "too large" when the function value at the algorithmic center is below the minimum model value (a situation that is impossible with an exact oracle, by convexity). For our setting, this amounts to checking if the noise measurement quantity defined below is negative:

h k xk -M k (x k+1 ) + 1 2 µ k x k+1 -xk 2 < 0 .
When the relation above holds, the algorithm maintains the model and the center, and reduces the prox-parameter. The new iterate yields a smaller noise measurement quantity, thus attenuating the noise induced by the inexact bundle information. For the sake of numerical versatility, we consider here an alternative mechanism that is more general, and checks asymptotic satisfaction of the inequality above, based on a relative criterion. More precisely, noise is considered too large if

h k xk -M k (x k+1 ) + 1 2 µ k x k+1 -xk 2 1 2 µ k x k+1 -xk 2 < -β k (4.1.12)
for a parameter β k satisfying (4.1.15) below.

To measure progress towards the goal of solving (4.1.1), certain predicted decrease δ k is employed. Usual definitions for the decrease are

δ k = h k xk -M k (x k+1 ), or δ k = h k xk -M k (x k+1 ) -1 2 µ k x k+1 -xk 2 .
We consider a slightly more general variant, and let Since the numerator in (4.1.12) equals δ k -1 2 (1-α k )µ k x k+1xk 2 , we see that detecting the need of a noise attenuation step amounts to checking satisfaction of the inequality

δ k = h k xk -M k (x k+1 ) - 1 2 α k µ k x k+1 -xk 2 , ( 4
δ k < 1 2 1 -(α k + β k ) µ k x k+1 -xk 2 . (4.1.14)
The choice of parameters α k , β k should ensure that the nominal decrease in (4.1.16) is nonnegative when noise is not too large. Accordingly, we suppose that The parameters α k , β k in the criterion (4.1.14) make it possible to control the relation between noise attenuation and descent, a flexibility that can help the numerical performance of the algorithm. More specifically, to progress towards a solution, it is preferable for the algorithm to:

∃b > -1 and B > 0 such that β k ∈ [b, 1 -α k -B] for α k ∈ [0, 2] . ( 4
-make more serious iterations, because serious iterates converge to a solution, and -have few noise attenuation steps. Noise attenuation steps are undesirable to occur often, because they prevent the algorithm from having "true" iterates, for which to check the descent condition.

-However, checking (4.1.14) does not involve any f /c-oracle calculation at x k+1 , and can therefore be considered an inexpensive test.

The flexibility introduced by the additional parameters α k , β k allows the user to seek for a trade-off between the time spent in oracle calculations and the CPU time required for the algorithm to find a solution to (4.1.1). By (4.1.16), more serious iterations are achieved by taking larger α k 's (yielding smaller δ k 's), while a larger β k reduces the left handside term in (4.1.14), making less likely noise attenuation. Our numerical experience in Section 4.1.6 shows how different choices of these parameters impact the numerical performance, both in terms of CPU time and accuracy.

We now list some consequences resulting from the various definitions above, obtained with some simple algebraic manipulations.

First, by (4.1.8) written with y = xk and (4.1.13),

h k xk -M k (x k ) + G k , xk -x k+1 = δ k + 1 2 α k µ k x k+1 -xk 2 .
Together with (4.1.10) and (4.1.7) we see that We now give our bundle algorithm for solving problem (4.1.1).

E k = δ k - 2 -α k 2 µ k x k+1 -
Algorithm 4.1 (Proximal Bundle). We assume given an oracle computing approximate f /c values as in (4.1.3) for any x ∈ X, possibly of upper type.

Step 0 (Input and Initialization) Select a initial starting point x0 a stopping tolerance tol≥ 0, an Armijo-like parameter m ∈ (0, 1). Initialize the iteration counter k = 0, the bundle index set J 0 := {0}, and the first candidate point x 0 := x0 . Call (4.1.3) to compute f 0 , c 0 as well as g f 0 and g c 0 . Choose the starting prox-parameter µ 0 > 0, parameters α 0 , β 0 satifying (4.1.15), and penalties ρ 0 , σ 0 ≥ 0 satisfying (4.1.20) below.

Step 1 (Model Generation and QP Subproblem) Having the current algorithmic center xk , the bundle B k , the prox-parameter µ k and the penalties ρ k , σ k , define the convex piecewise linear model function M k and compute x k+1 = arg min{M k (y) + We now explain how Algorithm 4.1 handles the update of its prox-parameter µ k and penalties ρ k , σ k . For the prox-parameter, the update uses positive constants µ max and ∆, as follows:

µ k+1 ≤ µ max < +∞ if iteration k was declared serious (4.1.19a) µ k+1 ≤ µ k -∆ < µ k if iteration k resulted in noise attenuation (4.1.19b) µ k+1 ∈ [µ k , µ max ] if iteration k was declared null. (4.1.19c)
The rule for the penalty parameters uses a positive constant RS and is given below:

0 ≤ σ k+1 ≤ 1 and ρ k+1 ≥ 0 satisfy 1 -σ k+1 + ρ k+1 ≥ RS > 0 . (4.1.20)
The main purpose of these conditions is to ensure satisfaction of the relations stated in the proposition below. -At every iteration k,

h k xk = 0 if ĉk ≤ 0 and h k xk = ĉk (1 -σ k ) ≥ 0 otherwise .
-At every iteration k declared null, the inequality

h k x k+1 > h k xk -mδ k is satisfied. -Suppose that, in addition, the oracle (4.1.3) is bounded, in the sense that the inexact values {f k , c k } and { g k f , g k c } (4.1.21)
are bounded for every sequence {x k } ⊂ X. Then, there exist positive constants M and M ′ such that for any iteration k that is declared null or needing noise attenuation

M k (x k ) ≤ max(M -f k , M ) and h k xk ≤ M ′ . (4.1.22)
Proof. From (4.1.4), it readily follows that ĉk ≤ 0 implies

τ k 1 = f k , whereas ĉk > 0 implies τ k 1 = f k + ρ k ĉk and τ k 2 = σ k ĉk . As a consequence ĉk ≤ 0 implies h k xk = 0. Inversely when ĉk > 0, we have h k xk = ĉk max(-ρ k , 1 -σ k ) = ĉk (1 -σ k ) ≥ 0 because σ k ∈ [0, 1] and ρ k ≥ 0 by (4.1.20).
To prove the second item, let us first assume ĉk > 0. Then negating (4.1.16) and using the above derived identity for h k xk gives

h k x k+1 = max(f k+1 -f k -ρ k ĉk , c k+1 -σ k ĉk ) ≥ c k+1 -σ k ĉk > ĉk (1 -σ k ) -mδ k = h k xk -mδ k .
When ĉk ≤ 0, negating (4.1.16) implies that either f k+1 > f kmδ k or c k+1 > 0. In the first case we establish:

h k x k+1 = max(f k+1 -f k , c k+1 ) ≥ f k+1 -f k > -mδ k ,
as was to be shown, since h k xk = 0. In the second case, since k is a null step, (4.1.14) does not hold and δ k > 0 as a consequence. We thus establish

h k x k+1 = max(f k+1 -f k , c k+1 ) ≥ c k+1 > 0 ≥ h k
xkmδ k as was to be shown.

Finally, to see (4.1.22), notice that x k ∈ X with X bounded, so (4.1.21) ensures that each linearization f k + g k f , •x k (or its c-counterpart) is bounded over X. In particular, both fk (x k ) and čk (x k ) are bounded by some constant M . In view of the model definition (4.1.5) and ( 4

.1.4), M k (x k ) ≤ max(M -τ k 1 , M -τ k 2 ) = max(M -f k -ρ k max(ĉ k , 0), M - σ k max(ĉ k , 0)).
The bound for M k (x k ) follows, because the penalty terms are nonnegative by (4.1.20). An analogous reasoning gives the bound for h k xk .

Asymptotic Analysis

We now analyse the different cases that can arise when the algorithm in Section 4.1 loops forever, i.e., k → ∞. Then only one of the following mutually exclusive cases can occur:

-either there are infinitely many serious iterates, -or the stability center remains unchanged after a finite number of iterations. In this case, -either there is an infinite number of noise attenuation steps, -or there is a finite number of noise attenuation steps and eventually only null steps are done.

The first case is considered in Lemma 4. noting that δ k ≥ 0, because serious steps can only take place when no noise attenuation occurs. Since the nonincreasing sequence {ĉ k } K s is bounded below by 0, it converges. From (4.1.23) we deduce that 0

≤ δ k ≤ c k -c k+1 m . Since {ĉ k } K s converges this gives that δ k → 0 as K s ∋ k → ∞.
Otherwise, there is some k ∈ K s such that ĉk ≤ 0. In view of (4.1.16), all subsequent serious iterates are feasible: for each serious step k ≤ k ∈ K s we have ĉk+1 = c k+1 ≤ 0 and

f k+1 ≤ f k -mδ k with δ k ≥ 0 . (4.1.24)
Thus, the nonincreasing sequence { f k }k ≤k∈K s is either unbounded below or converges. The first case is ruled out by (4.1.21). As for the second case, it implies that

δ k → 0 since 0 ≤ δ k ≤ f k -f k+1 m .
We now show that both

G k + ν k → 0 and E k → 0. Since (1 -(α k + β k )) ≥ B > 0 by (4.1.15
), the negation of (4.1.14) and (4.1.7) imply that

0 ← δ k ≥ 1 2 1 -(α k + β k ) µ k x k+1 -xk 2 = 1 2µ k 1 -(α k + β k ) G k + ν k 2 ≥ B 2µ k G k + ν k 2 ≥ B 2µ max G k + ν k 2 ≥ 0
where we used (4.1.19a) for the last inequality. As a result, both

G k + ν k 2 /µ k → 0 and G k + ν k → 0 as K s ∋ k → ∞. Since α k ≥ 0 and α k + β k ≤ 1 -B by (4.1.15), we deduce that -(α k + 2β k )/2 = -(α k + β k ) + α k /2 ≥ B -1.
To show that the aggregate error goes to 0, pass to the limit in the negation of (4.1.18) and use the above estimate to see that lim

k∈K s E k ≥ (B -1) lim k∈K s µ k x k+1 -xk 2 .
Since by (4.1.7),

µ k x k+1 -xk 2 = G k + ν k 2 /
µ k and we have just shown this term vanishes asymptotically, the error limit is nonnegative. Then E k → 0, because E k ≤ δ k by (4.1.17) and δ k → 0.

The stated inequality holds follows from (4.1.11) by boundedness of X and the fact that both G k + ν k and E k → 0.

The analysis above shows that Step 5 of Algorithm 4.1 can freely manage the bundle at serious iterations. Of course, a richer bundle yields better cutting-plane models for f and c, so having larger index-sets J k should improve the speed of the method (keeping in mind that large sets J k make the QP subproblem more difficult).

Notice also that in the inequality in Lemma 4.1.2 the remainder o(1/k) corresponds in fact to both G k + ν k and E k going to zero. Similar relations will hold when there is a finite number of serious iterations, as shown below.

In the next two cases, eventually no more serious steps occur and after a finite number of iterations the algorithmic center remains unchanged. As a result, there exists k and x such that xk = x for all k > k. Unlike Lemma 4.1.2, the results below rely on conditions (4.1.20), required for the penalty parameters defining the inexact improvement function (4.1.4). Suppose at iteration k there is a last serious iterate x and let K a denote the set gathering indices of iterations larger than k for which (4.1.14) holds and noise is deemed too large.

If there are infinitely many of such indices and (4. 1.19b) holds then

h k xk = h k x ≤ M k (y) + o(1/k
) for all y ∈ X and k ∈ K a sufficiently large.

Proof. Consider k ∈ K a . By (4.1.17) and (4.1.14) we obtain the following estimate

2E k = 2δ k -(2 -α k )µ k x k+1 -xk 2 < -(1 + β k )µ k x k+1 -xk 2 ≤ 0, (4.1.25) since β k ≥ b > -1 by (4.1.15). Since E k ≤ 0, h k xk -E k ≥ h k xk ,
and with (4.1.11) we see that

h k xk ≤ M k (y) + G k + ν k , x -y for all y ∈ X.
Since the set X is bounded, the stated inequality would hold if G k + ν k → 0. To show this result, first note that (4.1.10) and (4.1.8) imply that

-E k = M k (x k+1 ) + G k + ν k , x -x k+1 -h k xk ≤ M k (x) -h k xk ,
where the inequality comes from (4.1.7). Let f denote the f -value computed by the oracle at x. By the first item in Proposition 4.1.1,

h k xk ≥ 0 because σ k ≤ 1 by (4.1.20), so -E k ≤ M k . Since M k (x) ≤ max( f -M, f )
µ k E k < -G k + ν k 2
or, equivalently, that

0 ≤ G k + ν k ≤ - 2 1 + b µ k E k ≤ 2 1 + b µ k M .
Since the update in (4. 1.19b) ensures that the sequence {µ k } k∈K a is strictly decreasing, when K a ∋ k → ∞ we obtain that µ k → 0 and G k + ν k → 0, as desired.

For the result above to hold, Step 2 in Algorithm 4.1 only needs the sequence {µ k } K a to be strictly decreasing, and the left bound in (4.1.22) to hold. So, as for the case of infinitely many serious iterations, there is freedom in how the bundle is managed when noise is deemed too large. Since in this case there is no new oracle information, it seems reasonable to maintain the current bundle.

In a manner similar to Lemma 4.1.2, the remainder term in Lemma 4.1.3 corresponds in fact to

G k + ν k → 0 with E k ≤ 0.
The final result considers that there are finitely many serious and noise attenuation steps, which implies that the algorithm makes infinitely many consecutive null steps. For this QP subproblem (4.1.6) defining x k+1 ). Since ν k ∈ N X (x k+1 ) and y ∈ X, the term ν k , yx k+1 ≥ 0 and, hence, we derive from (4.1.28) that

∀y ∈ X M k (y) + 1 2 µ k y -x 2 ≥ OV k + 1 2 µ k y -x k+1 2 . (4.1.29)
By evaluating at y = x ∈ X, using (4.1.9) and the third item in Proposition 4.1.1, there exists some constant M such that

OV k + 1 2 µ k x -x k+1 2 ≤ M k (x) ≤ M k (x) ≤ M .
In particular, the sequence {OV k } is bounded from above.

Assumption (4.1.26a) in (4.1.29) yields that

M k+1 (y) + 1 2 µ k y -x 2 ≥ OV k + 1 2 µ k y -x k+1 2
for all y ∈ X. Since µ k+1 ≥ µ k by (4.1.19c), by evaluating the inequality above at y = x k+2 we see that

OV k+1 ≥ OV k + 1 2 µ k x k+2 -x k+1 2 ,
and, being bounded above, the non-decreasing sequence {OV k } converges. Since the righthand side is larger than OV k , we conclude that Recall that condition (4.1.15) implies that δ k ≥ 0 when (4.1.14) fails, i.e., no noise is detected. The descent test also fails, so by the second item in Proposition 4.1.1,

OV k+1 -OV k + 1 2 µ k x k+2 -x k+1 2 →
h k x k+1 > h k xk -mδ k = h k x -mδ k .
Adding δ k to both terms and using the definition in (4.1.13) we obtain that

0 ≤ (1 -m)δ k < δ k + h k x k+1 -h k xk = h k x k+1 -M k (x k+1 ) - 1 2 α k µ k x k+1 -x 2 ≤ h k x k+1 -M k (x k+1 ) = h k x k+1 -M k+1 (x k+2 ) + M k+1 (x k+2 ) -M k (x k+1 ) .
Writing (4.1.26b) for y = x k+2 and using the Cauchy-Schwarz inequality yields that

M k+1 (x k+2 ) ≥ max(f k+1 -τ k 1 -g k+1 f x k+2 -x k+1 , c k+1 -τ k 2 -g k+1 c x k+2 -x k+1 ) .
By (4.1.21), there exists a constant Γ such that g k+1 f , g k+1 c ≤ Γ and, hence,

M k+1 (x k+2 ) ≥ max(f k+1 -τ k 1 , c k+1 -τ k 2 ) -Γ x k+2 -x k+1 .
The first righthand side term above equals h k x k+1 , by (4.1.4). Continuing and using the definition of OV k and (4.1.19c),

0 ≤ (1 -m)δ k ≤ Γ x k+2 -x k+1 + M k+1 (x k+2 ) -M k (x k+1 ) = Γ x k+2 -x k+1 + OV k+1 - 1 2 µ k+1 x k+2 -x 2 -OV k + 1 2 µ k x k+1 -x 2 = Γ x k+2 -x k+1 + OV k+1 -OV k - 1 2 µ k x k+2 -x 2 + 1 2 µ k x k+1 -x 2 = Γ x k+2 -x k+1 + OV k+1 -(OV k + 1 2 µ k x k+2 -x k+1 2 ) + 1 2 µ k x k+2 -x k+1 2 -x k+2 -x 2 + x k+1 -x 2 .
Following (4.1.27), we can observe that the last three terms equal

µ k x k+2 -x k+1 , x -x k+1 . By (4.1.7), µ k (x -x k+1 ) = G k + ν k and, since ν k ∈ N X (x k+1 ) and x k+2 ∈ X, µ k x k+2 -x k+1 , x -x k+1 = x k+2 -x k+1 , G k + ν k ≤ x k+2 -x k+1 , G k ≤ G k x k+2 -x k+1 Since G k ∈ ∂M k (x k+1 ) ⊂ conv{g j f , g j c : j ∈ J k }, assumption (4.1.21) implies that G k ≤ Γ and, hence, 0 ≤ (1 -m)δ k ≤ 2Γ x k+2 -x k+1 + OV k+1 -(OV k + 1 2 µ k x k+2 -x k+1 2 ) .
In view of (4.1.30), the right handside above tends to zero. Since m ∈ (0, 1), δ k → 0, as desired.

The negation of (4.1.14), condition 1 -(α k + β k ) ≥ B > 0 from (4.1.15), and the fact that µ k ≥ µk +1 from (4.1.19c) imply that

0 ← δ k ≥ 1 2 µk +1 B x k+1 -x 2 ≥ 0 , so x k+1 → x, as stated. Since by (4.1.7) G k + ν k = µ k (x -x k+1 ) and by (4.1.19c) µ k ≤ µ max , we obtain that 0 ≤ 1 µ max G k + ν k ≤ 1 µ k G k + ν k = x k+1 -x → 0 and, hence, G k + ν k → 0 .
Finally, using in (4.1.17) that α k ≥ 0 by (4.1.15) gives

0 ← δ k ≥ E k = δ k - 2 -α k 2 µ k x k+1 -x 2 ≥ δ k -µ k x k+1 -x 2 ≥ δ k -µ max x k+1 -x 2 .
Since both righthand side terms go to zero, so does E k and the proof is finished.

For the result above to hold, at null steps the bundle needs to be managed in a manner ensuring the relations (4.1.26). Condition (4.1.26b) holds if the last generated information enters the bundle, that is,

k + 1 ∈ J k+1 which is equivalent to having x k+1 , f k+1 , c k+1 , g f k+1 , g c k+1 ∈ B k+1 .
As for (4.1.26a), the inequality is typically ensured by introducing aggregate information in the bundle. In our improvement function setting, this can be done by splitting the aggregate information into their f and c parts. Specifically, in view of the model definition (4.1.5), there is a multiplier

λ k ∈ [0, 1] such that M k (x k+1 ) = λ k ( fk (x k+1 ) -τ k 1 ) + (1 -λ k )(č k (x k+1 ) -τ k 2 ) , and
G k = λ k G k f + (1 -λ k )G k c with G k f ∈ ∂ fk (x k+1 ) , G k c ∈ ∂č k (x k+1
) . For (4.1.26a) to hold it is then enough to take

x k+1 , fk (x k+1 ), čk (x k+1 ), G k f , G k c ∈ B k+1 .
Similar calculations can be derived for economic bundles which, in the information (x j , f j , c j , g j f , g j c ), replace the knowledge of the vector x j by two scalars, referred to as linearization errors for f and c at xk . We refer to [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF] for more details.

Finally, like in the case of infinitely many serious steps, the remainder term corresponds to G k + ν k → 0 with E k → 0. For this reason, instead of the stopping criteria in Step 3 of Algorithm 4.1, one could stop when G k + ν k is sufficiently small, as long as E k is also small (serious and null cases) or nonpositive (noisy case).

Link with the Original Problem

In Section 4.1.2 we established the limiting behaviour of Algorithm 4.1. We still need to analyze in which sense the method converges to an approximate solution to problem (4.1.1). It is at this stage that the oracle errors play a major role. Our results below show what can be expected in terms of solving (4.1.1) for oracles that are of upper or lower type. Partly assymptotically exact and exact oracles are also considered.

Convergence Results

We start with a general result, suitable for oracles yielding inexact linearizations that may lie above the function. Specifically, we suppose that for any x k ∈ X the oracle output (4.1.3) is of upper type, in the sense that errors ε k at iteration k and asymptotic error ε ≥ 0 satisfy: When the algorithm loops forever, for any accumulation point x of the (bounded) sequence {x k } and its limiting inexact f /c-values and parameters ( f , c, ρ, σ) it holds that

∀y ∈ X f k + g k f , y -x k ≤ f (y) + ε k c k + g k c , y -x k ≤ c(y) + ε k with ε := lim sup ε k . ( 4
∀y ∈ X max(c, 0)(1-σ) ≤ max f (y)-f -ρ max(c, 0), c(y)-σ max(c, 0) +ε. (4.1.32)
Moreover, :

(i) If c > 0 then ∃R : ρ ≥ R =⇒ c ≤ c(y) + ε for all y ∈ X.
(ii) If c ≤ 0 and the set X ε = {y ∈ X : c(y) ≤ -2ε} is not empty, then

f ≤ f (y) + ε for all y ∈ X ε .
Proof. When the algorithm loops forever, one of the index sets K s , K a , K n , defined respectively in Lemmas 4.1.2, 4.1.3, 4.1.4, is infinite. Since xk ∈ X and X is compact, for any of such sets there exists a subset K ′ such that {x k } k∈K ′ → x, recalling that when Consider first the case c > 0. Since X is bounded and both f and c are real-valued, the constant R = 1 c max y∈X (f (y)c(y)) -f + σ is well defined and any ρ > R satisfies (ρσ)c > f (y)c(y) -f for all y ∈ X. Since the inequality is equivalent to having f (y) -fρ max(c, 0) < c(y)σ max(c, 0), the stated results follows from (4.1.32).

K ′ = K a and K ′ = K n eventually xk = x =
∀y ∈ X max(ĉ k , 0)(1 -σ k ) ≤ max(f (y) -τ k 1 , c(y) -τ k 2 ) + o(1/k) + ε k . ( 4
When c ≤ 0, (4.1.32) becomes 0 ≤ max f (y) -f , c(y) + ε and evaluating at any y ∈ X ε gives the final result. Theorem 4.1.5 states that, as long as ρ k is managed as a penalty parameter (for instance ρ k+1 = 2ρ k if c xk+1 > 0, and ρ k+1 = ρ k otherwise), Algorithm 4.1 will eventually detect problem (4.1.1) as infeasible up to the accuracy ε, or it will find an approximate minimizer in the sense stated by the second item in the theorem. For the latter to happen, the accuracy should be small enough to ensure nonemptiness of the set X ε (noting that in this set the factor -2 could be replaced by any value strictly smaller than -1).

We now state a refinement of this result, for the case when inaccurate linearizations eventually stay below the functions f and c. We refer to this situation as having "lower" oracles. Proof. The case c ≤ 0 is Theorem 4.1.5(ii) with ε = 0. As for the case c > 0, it is excluded by observing that for ρ sufficiently large (4.1.32) becomes c(1σ) ≤ max f (y) -fρc, c(y)σc = c(y)σc , so 0 < c ≤ c(y) for all y ∈ X, an inequality that cannot hold when y is the Slater point.

A further refinement is possible for lower oracles that are partly asymptotically exact, see [START_REF] Gaudioso | An incremental method for solving convex finite min-max problems[END_REF], [START_REF] Kiwiel | Bundle methods for convex minimization with partially inexact ora-cles[END_REF], [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF]. Specifically, these are oracles that become progressively more and more accurate at serious steps. We now show that, unlike the previous cases, in this situation the penalty ρ k does not need to increase to infinity when the center is deemed infeasible (that is, when c xk > 0). Proof. When c ≤ 0, the first result is Theorem 4.1.5(ii), recalling that f = f (x) and ε = 0.

Similarly for c > 0 and ρ = +∞, applying the first item in Theorem 4.1.5 with c = c(x) and ε = 0. When c > 0 and ρ is finite (not necessarily larger than R in the theorem), adding σc to both sides of (4.1.32) and using that ε = 0 together with the assumption that c(x) = c and f (x) = f gives that c(x) ≤ H(y) := max f (y)f (x) -(ρσ)c(x) , c(y) for all y ∈ X.

(4.1.34)

In the right handside above we use the notation H(•) for the (convex) exact improvement function (4.1.2), written with target τ = (f (x)+(ρ-σ)c(x), 0). In particular, when y = x the inequality (4.1.34) becomes

0 < c(x) ≤ H(x) = max -(ρ -σ)c(x) , c(x) .
Since by (4.1.20) the penalties satisfy ρ k ≥ RS + σ k -1 > σ k -1, this means that -(ρσ) < 1 and the maximum above is attained at c(x) and, hence, H(x) = c(x). As a result, (4.1.34) states that 0 ∈ ∂(H + i X )(x), where i X denotes the indicator function of the set X. Being a max-function, the subgradients of H at x are of the form

λg f + (1 -λ)g c for λ ∈ [0, 1] with g f ∈ ∂f (x) and g c ∈ ∂c(x) with λ ∈ (0, 1] ⇐⇒ -(ρ -σ)c(x) ≥ c(x) .
Since c(x) > 0 and -(ρσ) ≤ 1 -RS < 1 by (4.1.20), the only possibility is λ = 0, i.e., 0 ∈ ∂c(x) and, therefore, 0 < c(x) ≤ c(y) for all y ∈ X, as stated.

For completeness, and to relate our method with previous algorithms in the literature, we finish with a result for oracles that make exact calculations. In this case, there is no noise attenuation step and the nominal decrease (4.1.13) is defined with

α k ∈ [0, 1].
Corollary 4.1.8 (Convergence for exact oracles). Consider an exact oracle: for any x j ∈ X (4.1.3) returns f j = f (x j ), c j = c(x j ) and subgradients g j f ∈ ∂f (x j ) and g j c ∈ ∂c(x j ). Then Algorithm 4.1 with β k ≡ 0 never executes Step 2 (noise attenuation). Furthermore, suppose α k ∈ [0, 1], the penalties satisfy (4.1.20), the prox-parameter is updated according to (4. 1.19a) and (4. 1.19c), and the model satisfies (4.1.26). Then the statements in Corollary 4.1.7 apply.

Proof. By convexity, an exact oracle is of the lower type, so (4.1.31) holds with ε = 0. In particular, writing (4.1.4) and (4.1.5) with y = xk we see that

h k xk ≥ M k (x k ). Since, by (4.1.6), M k (x k ) ≥ M k (x k+1 ) + 1 2 µ k x k+1 -xk 2
, the left handside in (4.1.12) is always nonnegative and, as long as β k ≥ 0, the algorithm will never consider noise too large (naturally so, since for exact oracles there is no noise to be detected).

The set X is bounded and the subdifferential mapping of a convex function is locally bounded, so (4.1.21) holds for exact oracles. Since in addition, (4.1.15) holds by taking for example B = 1 2 = -b, Corollary 4.1.7 completes the proof.

Relation with previous work

Corollary 4.1.8 covers methods based on improvement functions already considered in the literature for solving constrained non-smooth problems using exact or lower oracles. More precisely, conditions (4.1.20) are satisfied by at least the following three choices:

ρ k = σ k ≡ 0, or ρ k ≡ ρ < +∞ and σ k ≡ 1, or ρ = +∞ and σ k ≡ 0,
corresponding to the improvement functions in [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF], [2], and [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF], respectively. The first two methods were developed for exact oracles ([2] deals with nonconvex functions). The approach suggested in [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] considers a setting corresponding to a lower oracle, and is addressed by Corollary 4.1.7.

To decide if the iterate gives a serious step, the three methods ( [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF]2,[START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF]) use in Step 4 a criterion based on descent of the improvement function. Namely, It should also be mentioned that [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] uses an alternative stopping test, suitable for unbounded feasible sets (which is not the case in (4.1.1)). More precisely, instead of checking if δ k ≤ tol, Kiwiel in [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] uses the conditions

if h k x k+1 = max(f k+1 -τ k 1 , c k+1 -τ k 2 ) ≤ h k xk -
max G k + ν k , E k + G k + ν k , xk ≤ tol [129] and c xk ≤ 0. ( 4
E k + G k + ν k , xk = δ k + G k + ν k , xk+1 .
Using the fact that X is compact and in particular bounded together with G k + ν k ≤ tol [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] , the stopping criteria (4. 1.36) implies that δ k ≤ tol for certain modified tolerance.

Another stopping criterion is the one used in [START_REF] Emiel | Incremental like bundle methods with applications to energy planning[END_REF]:

max µ k x k+1 -xk 2 , E k ≤ tol [68] . ( 4
δ k = E k + 2 -α k 2 µ k x k+1 -xk 2 ≤ 2 max E k , µ k x k+1 -xk 2 , (4.1.38)
showing that (4.1.37) also implies δ k ≤ tol for an appropriately chosen tolerance tol.

Therefore, for convex problems as (4.1.1), our approach includes previous work, significantly extending the applicability of algorithms in the literature:

-not only exact and lower oracles can be used, but also upper ones, satisfying (4.1.31); -the criterion for serious steps can be (4. 1.16) or based on the improvement function; -the descent and noise parameters α k , β k can be chosen in any manner satisfying (4.1.15). This versatility has a positive impact on the numerical results as shown in Table 4.4.

-any choice for the penalty parameters ρ k and σ k in (4.1.4) satisfying (4.1.20) is possible.

Regarding the last item, the role/utility of the penalty in second target (in τ k 2 ) is not clear. At least in the convex setting, taking σ k = 0 and forgetting about this additional parameter seems sufficient. It is argued in [2], however, that a positive σ k can be beneficial in the numerical performance of the algorithm, so the situation may be different for nonconvex problems.

Energy Application: Hydro Reservoir Management

We consider a cascaded reservoir management problem with uncertainty on inflows in the numerical experience. Figures 4.1(a) and 4.1(b) provide two typical instances of such a problem. Specific modelling details can be found in section 5.1 and paper [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF].

Such a problem fits the following abstract structure:

min x≥0,x∈R n f, x s.t. Ãx ≤ b (4.1.39) p ≤ P[a r + A r x ≤ ξ ≤ b r + A r x],
where ξ ∈ R m is a Gaussian random vector with variance-covariance matrix Σ and zero mean (we have explicitly extracted the non-zero average in the vector a r , b r ∈ R m ). This random vector represents uncertainty on inflows and the probabilistic constraint the fact that we are looking for a turbining schedule that keeps volume between a lower and upper bound with high probability.

By a classical result (Theorem 2.5.13), the feasible set induced by the joint chance constraint (5.1.4) is convex. Moreover from the same result it follows that the function

log(p) -log(P[a r + A r x ≤ ξ ≤ b r + A r x]) (4.1.40)
is a convex function.

So (4.1.39) corresponds to (4.1.1) with

f (x) := f, x , X := {x ∈ R n : x ≥ 0, Ãx ≤ b} and c(x) := log(p) -log(P[a r + A r x ≤ ξ ≤ b r + A r x]).
It is moreover clear that X is bounded according to (5.1.1). In this setting, the f -oracle is exact. As for the c-oracle, it falls into the framework (4.1.31) with ε > 0, as explained below.

Devising an inexact upper oracle for the constraint

It is shown in Lemma 3.2.1 (see also Corollary 3.2.3) that the mapping . These results can be applied since ξ is a non-degenerate (i.e., with positive definite covariance matrix) Gaussian random variable. In the potentially degenerate case we can refer to [START_REF] Henrion | A gradient formula for linear chance constraints under Gaussian distribution[END_REF].

ϕ : R n → [0, 1] defined by ϕ(x) = P[a r + A r x ≤ ξ ≤ b r + A r x]
The analysis of what follows will be completely similar as again the computation of the gradient is reduced to evaluation mappings with a structure close to the one of ϕ above.

Accordingly, the i-th component of

∇ âF ξ (â, b) (-∇ bF ξ (â, b) respectively) is equal to -ψ(z)P[ã r + Ãr x ≤ ξ ≤ br + Ãr x].
In this expression, ψ is the density of a standard Gaussian random variable in 1 dimension; z a specific point depending on x, and ãr , br , Ãr , ξ of a r , b r , A r are appropriate modifications of the respective objects. These modifications depend on whether the derivative is taken with respect to â or b.

We see that, in order to compute one component of the gradient, one needs to evaluate a mapping of the form F ζ (â, b) at specific points â ≤ b ∈ R m , with ζ ∈ R m a nondegenerate multi-variate Gaussian random variable. Since the evaluation of the c-function also requires to compute a similar probability, the core of the c-oracle is to make the involved multidimensional calculations in a fast and efficient manner. This is achieved by using the code [START_REF] Genz | Numerical computation of multivariate normal probabilities[END_REF], developped by A. Genz for multivariate normal probabilities.

Having as input a requested accuracy ε g > 0, the code either returns a value F such that F -F ζ (â, b) < ε g , or issues an error message, stating the impossibility of making the calculation with the requested precision. In the latter case, it is possible to increase the number of quasi Monte Carlo particles used in the numerical integration and make another attempt to obtain the desired accuracy.

Since the integral approximation estimate can be larger or smaller than the exact value, the c-linearizations may lie below or above the exact function c, noting that asymptotically exact (up to say floating point precision) calculations would be possible (although they might considerably increase the time spent in the oracle).

Convergence results for the application

According to the model set up in this section, the objective function f of problem (4.1.1) is linear and therefore has an exact oracle. Inexactness arises from evaluating the probabilistic constraint c. The corresponding oracle may be of the upper type, because no information is available on the sign of the incurred error, neither for the function values nor for the subgradients. However, as explained in [START_REF] Henrion | Gradient estimates for Gaussian distribution functions: Application to probabilistically constrained optimization problems[END_REF], the user can control such error, keeping it sufficiently small if desired (provided enough CPU time is spent in the calculations).

We now derive an explicit expression for ε > 0 in (4.1.31). Consider a constant Φ > 0, for which ϕ(x k ) > Φ > 0 for all iterations k. For example, one can take as initial point x0 an appropriate convex combination with the Slater point x s ∈ R n ensuring that ϕ(x 0 ) > Φ for any given p > Φ > 0. Then, as far as serious iterates are concerned, (4.1.16) ensures that ϕ(x k ) > Φ. Since ε in (4.1.31) depends on the limiting behaviour, clearly one can choose µ k in (4.1.6) according to rule (4. 1.19c) in such a way that ϕ(x k ) > Φ also in

The various compared algorithms

We first compare the performance of several methods, including variants of Algorithm 4.1 using different choices for the parameters. More precisely, we consider:

-A configuration of Alg.4.1 with a strong noise test:

Alg.PB.SEV :

σ k ≡ 0 , ρ k ≡ 0 , α k ≡ 1 , β k ≡ -1 + ε m
with stopping test (4.1.37) and descent test (4. 1.35). Here ε m is the machine precision. -A configuration of Alg.4.1 with null parameters: Alg.PB.NUL : σ k ≡ 0 , ρ k ≡ 0 , α k ≡ 0 , β k ≡ 0 with stopping test (4.1.36) and descent test (4.1.35) -The method of centers from [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF], bearing some similarities with Alg. 4.1 provided parameters are properly set:

Alg. [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] :

σ k ≡ 0 , ρ k → ∞ if {x k+1 } infeasible , α k ≡ 0 , β k = ad-hoc
with stopping test (4.1.36) and descent test (4.1.35) -The conic bundle method from [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF]:

Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] : No σ k , ρ k , α k ≡ 0 , ad-hoc QP in Step 1, β k , Stop and Descent Test -The supporting hyperplanes method from [START_REF] Prékopa | Probabilistic programming[END_REF] (see also [START_REF] Veinott | The supporting hyperplane method for unimodal programming[END_REF][START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Szántai | A computer code for solution of probabilistic-constrained stochastic programming problems[END_REF]):

Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] : No σ k , ρ k , α k , β k , with a Linear Program in Step 1 and ad-hoc stopping and descent tests.

In both configurations of Alg.4.1 above taking RS = 1 ensures satisfaction of (4.1.20) for any choice of ρ k+1 ; as for (4.1.15), it suffices to take any positive B and b = β k (constant in k). Since the conditions in Section 4.1.3 are satisfied, eventual convergence to an approximate solution is assured with these methods.

We now describe the specific ad-hoc updates of the last three methods. To help the comparison, the algorithms steps are numbered as in Alg. 4.1.

Alg.[129] Specifics

The penalty ρ k+1 is halved at serious steps and doubled if c xk+1 > 0. Once a feasible stability center is found, the numerical value of this penalty becomes irrelevant. For an additional Armijo-like parameter m ′ ∈ (0, 1], the method of centers modifies the noise management Step 2 in Alg. 4.1 as follows.

Step 2'a (Infeasible serious point) If ĉk > 0 and δ k < m ′ ĉk , noise is too large. Decrease the prox-parameter as in (4. 1.19b), take ρ k+1 = 2ρ k , maintain the bundle, and loop to

Step 1 (solve a new QP subproblem).

Step 2'b (Noise attenuation). In the other cases, that is, if ĉk ≤ 0 or δ k ≥ m ′ ĉk , check if condition (4.1.14) holds. If this condition holds, noise is too large. Decrease the prox-parameter as in (4. 1.19b), take ρ k+1 > ρ k if ĉk > 0, maintain the bundle, and loop to Step 1 (solve a new QP subproblem).

The difference with our noise management step is in the first item: when the current serious point is infeasible, instead of (4.1.14) the condition δ k < m ′ ĉk is checked. Such condition amounts to verifying if (4.1.14) holds with β k = 1-2m ′ ĉk µ k x k+1 -x k 2 . Suppose such value of β k also satisfies (4.1.15) (for example suppose m ′ is sufficiently small). Then this alternative can also be considered a special case of Algorithm 4.1, with a specific updating of β k for infeasible serious points.

Alg.[132] Specifics

This method is applicable when, like in our application, the objective function in (4.1.1) is linear or quadratic: the QP subproblem approximates (4.1.1) with a cutting-plane model for the constraint. Furthermore, the method needs the knowledge of a Slater point x s for (4.1.1), which is also its starting point: x0 = x s . The algorithm performs the following steps.

Step 1' (Alternative subproblem, δ k , E k ) Let (x k+1 , η k+1 ) be a primal-dual solution to the QP

min y∈X⊆R n f, y + 1 2 µ k y -xk 2 s.t. čk (y) ≤ 0 .
This amounts to taking in (4.1.7), instead of subgradient G k ∈ ∂M k (x k+1 ), a vector G k ∈ f + η k+1 ∂č k (x k+1 ) given by the QP optimality conditions. Taking δ k := f, xkx k+1 and E k = δ kµ k x k+1xk2 ensures satisfaction of the left handside identity in (4.1.17) with α k = 0.

Step 3'a

(Stopping test) Stop if G k + ν k ≤ tol and E k + G k + ν k , xk ≤ 0, Step 3'b (Interpolation Step) If c x k+1 ≤ 0, set γ k := 1, otherwise γ k := -c x s c x k+1 -c x s . Define xk := γ k x k+1 + (1 -γ k )x s .
Step 4' (Serious step Test) If f, xk f, xkmδ k then declare a serious step, taking as next center xk+1 = xk . Otherwise, declare a null step.

When the c-oracle is exact or lower 2 , each stability center will be feasible, by convexity. When the c-oracle is of the upper type, like in our case, by the discussion in Section 4.1.5.1 it is possible to take a slightly larger γ k so that ϕ(x k ) ≥ p+ε g . This is a potentially costly fix that may require several evaluations (our implementation uses the original definition for γ k .) The analysis in [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] does not cover upper oracles, so convergence of this method is unclear, although its numerical behaviour was reasonable for our tests.

4.1.6.1.3 Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] Specifics Like Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF], the cutting-planes model for the constraint is built using for the c-evaluation points, points obtained from interpolating with the Slater point. The difference is that now the constraint is required to be active at the interpolation point (x 0 c = (1γ 0 )x 0 + γ 0 x s satisfies c x 0 c = 0 for some γ 0 ∈ [0, 1].) For a linear objective function the algorithm performs the following steps.

Step 1' (Linear Programming subproblem) Let x k+1 be a solution to min y∈X⊆R n f, y s.t. čk (y) ≤ 0 .

Step For exact c-oracles, the method is a specialization of the cutting-plane algorithm in nonsmooth optimization, hence it converges. For inexact c-oracles, convergence results are not known.

Computational results

We now comment some of the numerical results. The reported CPU times are to be taken as a measure for comparing different algorithms, rather than as a measure of performance. Our implementation does not contain several improvements and tweaks that could optimize the code (oracle parallelization or multi-threading, for instance). Another important issue is that computing a Slater point is very consuming in terms of CPU time: this amounts to solving a problem as difficult as the original one (4.1.39) (i.e., solving (4.1.39) with objective function replaced by maximizing the probability level).

Results reported in Tables 4.1,4.2,4.3 were obtained on a HP xw6200 workstation with 8 Gb memory, whereas results of Tables 4.4, 4.6, 4.5 were obtained on a HP z600 workstation.

The key findings are that Alg.PB.SEV 1. Does not need the explicit knowledge of a Slater point. Moreover the Bundle Algorithms perform better if started in an infeasible point. Such an infeasible start can not be performed with Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] and Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] which are definitely outperformed.

2. Provides the optimal solution and does not converge early as do Alg. [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] and Alg.PB.NUL.

3. Is quite robust to the accuracy with which the probabilistic constraints and gradients are evaluated.

Feasible Start using a Slater Point and Infeasible Start

In order to put all algorithms on equal foot, a first comparison supposes that a Slater point is available to all of the methods in the benchmark. In this case, for a suitable convex combination of x d and x s , the starting point x0 is always feasible (Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] starts at x s ). Computing this convex multiplier requires executing step 3'a of Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. This involves calling the c-oracle several times and might be costly. Table 4.1 contains the output of all the runs, with various parameter settings for each algorithm (to decide on the best choice of parameters for the benchmark).

The , µ 0 = 1e -6 , µ s = 4.0, κ = 0.5 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 525 1933.29 K = 1, µ 0 = 1e -5 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 481 1691.07 K = 1, µ 0 = 1e -6 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104027 447 1749.57 K = 1, µ 0 = 1e -7 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104027 454 1852.24 K = 1, µ 0 = 1e -8 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104025 339 1311.24 K = 1, µ 0 = 1e -8 , µ s = 1.05, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104024 297 1170.13 K = 1, µ 0 = 5e -9 , µ s = 1.05, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104027 418 1602.25 K = 1, µ 0 = 1e -9 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104022 293 1089.25 K = 1, µ 0 = 1e -9 , µ s = 1.05, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 273 1030.21 K = 1, µ 0 = 1e -9 , µ s = 1.05, κ = 0.05 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104024 254 893.40 K = 1, µ 0 = 1e -9 , µ s = 1.01, κ = 0.05 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104024 254 798.32 K = 1, µ 0 = 1e -9 , µ s = 1.01, κ = 0.01 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104024 241 761.37 K = 1, µ 0 = 1e -9 , µ s = 1.001, κ = 0.01 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 303 1150.23 K = 1, µ 0 = 1e -9 , µ s = 4.0, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104025 391 1456.13 K = 1, µ 0 = 1e -10 , µ s = 1.05, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 329 1193.51 K = 1, µ 0 = 1e -10 , µ s = 1.05, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104028 595 2340.03 K = 1, µ 0 = 1e -10 , µ s = 2.0, κ = 0.7 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104020 360 1348.14 K = 1, µ 0 = 1e -10 , µ s = 2.0, κ = 0.1 Alg. [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] -104026 325 1250.17 In order to determine the impact on the convergence, we varied the precision of Genz code. This test also allows us to check the potential of varying this precision along the iterations. We took the best method, Alg.PB.SEV, with the same parameter setting, for different oracle precision. Table 4.3 reports the corresponding results. The solution obtained when using Genz' code with precision 1e -2 is slightly infeasible. This explains the "over"-optimal objective function value. From this table, we see that if the oracle accuracy is too high, CPU times can reach inacceptably large values.

Continuing with our analysis of the best variant, Alg.PB.SEV with fixed settings, in Table 4.4 we consider different values of β k , ranging between the severe noise test (close to -1) to the permissive one (close to 1), and likewise for α k , which varies from a severe serious step test (close to 0) to a permissive on (close to 2). Higher α values lead to more serious steps, until the extreme of declaring all iterates serious (α = 2 -2ε m ). In our tests, a severe noise test has resulted in a more stable management of the prox-parameter. Indeed, we have observed that if noise gets detected early on, the value of µ k remains unchanged a significant number of iterations. By contrast, the permissive choice for β k leads to some chaotic changes throughout the iterative process.

We have also tested the two alternative stopping criteria and the alternative serious step conditions. Table 4.5 reports these results. ) is less sensitive to the scaling parameter K. In particular one could make an alternative choice for the Armijo parameter to boost feasibility steps early on. The stopping criteria clearly does impact results, mainly because the varying tolerances, although equivalent choices exist, are hard to link together.

Finally, we also examined the impact of penalty parameter ρ k , comparing taking ρ k = 0 (as in the previous runs in this subsection) with an update as for Alg. [START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF] 4 Results are provided in Table 4.6. Instead, we observe that with a nonnull ρ k a feasible stability center is found in half the number of iterations. Unfortunately this quest for feasibility strongly deteriorates the objective function value, and eventually more iterations are required to converge to very similar solutions in the end.

We conclude that, at least for our runs, the best variant is Alg.PB.SEV with the settings in Table 4.4.

Solution Quality

To assess the obtained solutions, we simulate the reservoirs evolution and check if the stipulated probability level is satisfied numerically. Figure 4.2 shows the reservoir levels for 100 simulated inflow scenarios, using the expected-value strategy obtained from Except for the deterministic solution, which violates constraints for almost all scenarios, the various methods in our benchmark provide in fact very similar solutions. When slight differences arise, they account for increased robustness (with respect to the deterministic solution) and for increased optimality (with respect to various solution methods).

A level Bundle Method

Following the discussion of [START_REF] Oliveira | A doubly stabilized bundle method for nonsmooth convex optimization[END_REF] at least three types of bundle methods can be identified. The classical proximal bundle method ( [START_REF] Lemaréchal | Variable metric bundle methods: from conceptual to implementable forms[END_REF] and references therein) of which a special version for probabilistically constrained programming was derived in Section 4.1. The level bundle method ( [START_REF] Lemaréchal | New variants of bundle methods[END_REF]) and the trust-region bundle method ( [START_REF] Schramm | A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results[END_REF]). In the last version the cutting planes model of the convex function (to be minimized) is minimized under a trust-region constraint as its name implies. In the level method we orthogonally project the current stability center onto a level set defined by the current cutting planes model. For a given iteration, we can theoretically find parameter settings such that each of these methods provides the same next iterate. From a practical viewpoint these three methods differ in the ease with which the parameters are updated. For instance, the level bundle method does not have a proximal parameter to update. Since this is one of the most tricky parts for obtaining an efficient proximal bundle method, this can be seen as an advantage. In order to investigate this, we will derive a level bundle method for probabilistically constrained programming in this section. We will repeat some of the earlier notation for clarity.

Once again, let us consider problems of the type min

x f 0 (x) s.t. x ∈ X ⊂ R n and f j (x) 0 for j = 1, ..., m , (4.2.1)
where f j : R n → R, j = 0, ..., m are convex functions and X = ∅ a polyhedral bounded set in R n . We will make the assumption that some (or all) functions f j are hard to evaluate. We therefore assume that the mappings f j , j = 1, ..., m can be inexactly evaluated by some oracle (or black-box) providing approximate values and subgradients, hereafter denoted by oracle information. The precision of the oracle information might be unknown, but it is assumed bounded.

Applications in which a decision maker has to make a decision hedging risks often lead to optimization problems with nonlinear convex constraints that are difficult to deal with. For instance, if such constraints (f j , with j = 1, . . . , m) are setup as joint chance constraints (see Example 4.2.2 below). Evaluating the functions for a given point involves computing numerically a multidimensional integral. This is a difficult task when uncertainty variables are high-dimensional (say dimension 100), see [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF]. A second difficulty is obtaining an efficient estimate of a subgradient. This can be done in many cases of interest, but equally involves imprecisions (see [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF] for a special case and [START_REF] Uryas'ev | A differentation formula for integrals over sets given by inclusion[END_REF][START_REF] Uryas'ev | Derivatives of probability functions and some applications[END_REF][START_REF] Uryas'ev | Introduction to the Theory of Probabilistic Functions and Percentiles (Value-at-Risk) Chapter 1 in[END_REF] for very general results).

The last few years have seen the occurrence of a new generation of bundle methods, capable to handle inexact oracle information. For unconstrained (or polyhedral constrained) convex non-smooth optimization problems we refer to [START_REF] Hintermüller | A proximal bundle method based on approximate sub-gradients[END_REF] and [START_REF] Kiwiel | A proximal bundle method with approximate subgradient linearizations[END_REF] for general approaches, [START_REF] Kiwiel | An inexact bundle variant suited to column generation[END_REF] for a combinatorial context, and [START_REF] Fábián | Bundle-type methods for inexact data[END_REF], [START_REF] Fábián | Solving two-stage stochastic programming problems with level decomposition[END_REF], [START_REF] Oliveira | Inexact bundle methods for two-stage stochastic programming[END_REF] and [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF] for a stochastic optimization framework. For an encompassing approach we refer to the recent work [START_REF] Oliveira | Bundle methods in depth: a unified analysis for inexact oracles[END_REF].

For constrained convex optimization problems like (4.2.1) more sophisticated methods need to come into play. We refer to [START_REF] Fábián | Bundle-type methods for inexact data[END_REF] and [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] for level bundle methods capable to deal with asymptotically exact oracle information. Such oracle information is assumed to underestimate the exact value of the functions, i.e., the oracle is of the lower type. Since [START_REF] Fábián | Bundle-type methods for inexact data[END_REF] and [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] are based on the pioneering work [START_REF] Lemaréchal | New variants of bundle methods[END_REF], they are not readily implementable because they might require unbounded storage, i.e., all the linearizations built along iterations must be kept. The work presented in section 4.1 overcomes this drawback and furthermore allows for upper oracles, i.e., oracles that provide inexact information which might overestimate the exact values of the functions. In contrast to [START_REF] Fábián | Bundle-type methods for inexact data[END_REF] and [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] that employ level bundle methods, that work applies a proximal bundle method, see also [START_REF] Hiriart-Urruty | Convex Analysis and Minimization Algorithms II[END_REF].

In all these approaches we define different improvement functions (see (4.2.8) below) to measure the improvement of each iterate towards a solution of the problem. Level bundle methods have at disposal lower bounds (self-built) to define improvement functions, in contrast to proximal bundle that must find other alternatives to the (in-available) lower bounds. We refer to [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF], [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF] and [START_REF] Karas | A bundle-filter method for nonsmooth convex constrained optimization[END_REF] for some improvement function alternatives.

Due to the improvement function definition, level bundle methods are likely to be preferable to proximal bundle methods to handle (nonlinear) constrained convex optimization problems. For this reason we follow some features of the restricted memory level bundle method proposed in [START_REF] Kiwiel | Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities[END_REF] and extend [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] to handle upper oracles and to keep the storage information bounded; this is what we call restricted memory. Moreover, we extend [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF] by considering upper oracles with on-demand accuracy (see the formal definition in Section 4.2.3.2), and like [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] we prove convergence for lower oracles with on-demand accuracy without assuming a Slater point.

This section is organized as follows: Section 4.2.1 presents two important examples of optimization problems that cast into the problem formulation (4.2.1), with difficult to evaluate objective and/or constraint functions. In Section 4.2.2 we propose our algorithm and prove its convergence for exact oracle information. The convergence analysis therein is the key for proving convergence of the algorithm when inexact oracle information come into play. Section 4.2.3 deals with inexact oracle information. We consider upper, lower, and on-demand oracles, depending on the assumptions made. In Section 4.2.4 we provide numerical experiments on realistic joint chance constrained energy problems, arising when dealing with robust cascaded-reservoir management. These are problems from Électricité de France. We compare the proposed approaches with some algorithms considered in [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF].

To easy notation, in what follows we consider a short formulation to problem (4.2.1).

Without loss of generality we may assume that m = 1 in (4.

f min := min x∈X⊂R n f (x) s.t. c(x) 0 , (4.2.2)
which is assumed to have a solution; therefore, its optimal value f min is finite.

Some examples coming from stochastic optimization

We now provide two examples coming from stochastic optimization that can be cast into the above scheme, wherein f and/or c are difficult to evaluate.

Example 4.2.1 (Risk-averse two-stage stochastic programming). Let Z ⊂ R n be a bounded polyhedral set. Let also Ξ ⊂ R d be a finite set containing elements (scenarios) ξ i , i = 1, . . . , N ; and suppose that ϕ : Z → R is a convex function. A two-stage stochastic problem with mean-CV@R aversion model and finitely many scenario can be written as

   min ϕ(z) + N i=1 p i Q(z, ξ i ) s.t. t + 1 β N i=1 p i [Q(z, ξ i ) -t] + ρ z ∈ Z, t ∈ Re , (4.2.3)
where 1 β > 0 and ρ 0 are given parameters, p i > 0 is the probability associate to the scenario ξ i , [•] + denotes the positive part of a real number, and Q : Z × Ξ → R is a convex function on Z defined for instance by the linear problem

Q(z, ξ) :=    min q, y s.t. W (ξ)y + T (ξ)z = h(ξ) y ∈ Re n 2 + .
The components q, W, T and h are assumed to be mesurable mappings with appropriate dimensions and well defined in Ξ.

Problem (4.2.
3) is a risk-averse two-stage stochastic programming problem. The nonlinear and difficult to evaluate constraint t + 1

β N i=1 p i [Q(z, ξ i ) -t] +
ρ is the so called conditional value-at-risk (CV@R β ) with confidence level β; see [START_REF] Rockafellar | Optimization of conditional value-at-risk[END_REF], [START_REF] Rockafellar | Conditional value at risk: optimization approach[END_REF], [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF].

By bounding the variable t, writing x = (z, t) ∈ X ⊂ Y × Re, and by taking

f (x) = f (z, t) = ϕ(z) + N i=1 p i Q(z, ξ i ) and c(x) = c(z, t) = t + 1 β N i=1 p i [Q(z, ξ i ) -t] + -ρ problem (4.2.
3) corresponds to (4.2.2). Notice that computing the value of the functions f (x) and c(x) for each given point x ∈ X amounts to solving N linear problems Q(x, ξ i ), i = 1, . . . , N . After having solved those linear problems, subgradients for f and c are easily at hand; see [START_REF] Shapiro | Lectures on Stochastic Programming. Modeling and Theory, volume 9 of MPS-SIAM series on optimization[END_REF] for more details. Therefore, for large N (say N > 10000) the functions f and c are hard to evaluate, and by making use of inexact oracle information (e.g., solving inexactly or just skipping some linear problems) problem (4.2.3) can be numerically tractable even for a large N .

These problems are taken into account in [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF], by considering lower oracles with ondemand accuracy (see Section 4.2.3.4 for the formal definition). Besides the conditional value-at-risk approach of two-stage stochastic programs, [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] considers more risk-aversion concepts such as stochastic dominance, proposed in [START_REF] Dentcheva | Two-stage stochastic optimization problems with stochastic ordering constraints on the recourse[END_REF]. In all cases, no upper oracles are studied.

f (x) = q, x and c(x) = log(p) -log(P[Ax + a ξ Ax + b]) .
The oracle for the function c(x) employs multidimensional numerical integration and quasi-Monte Carlo techniques for which the error sign is unknown (but a bound is computable).

Take x ∈ X. Since the inexact oracle information can overestimate the exact value c(x) (upper oracle), the constrained level bundle methods proposed in [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] is not suitable.

In what follows we propose a new algorithm to deal with (in-)exact oracle information.

Level Bundle Method for Exact Oracles

In this section we assume that both functions f and c in (4.2.2) can be exactly evaluated by an oracle that provides for each x k ∈ X:

f -oracle information: f (x k ) and g f k ∈ ∂f (x k ) c-oracle information: c(x k ) and g c k ∈ ∂c(x k ) . (4.2.6)
With such information from the oracle, we consider the approximate linearizations

f k (x) := f (x k ) + g f k , x -x k ck (x) := c(x k ) + g c k , x -x k .
At iteration k, two polyhedral cutting-plane models are available: It follows from convexity that f k (x) f (x) and čk (x) c(x) for all k and for all x ∈ X.

f k (x) := max j∈J k f f j (x) with J k f ⊂ {1,
Given a lower bound f k low for f min , we define the so called improvement function at iteration k by

h k (x) := max{f (x) -f k low , c(x)} . (4.2.8)
As already mentioned, the above definition of improvement function is not possible for proximal bundle methods, since the lower bounds f k low are not available. Some alternatives for f k low must be taken instead. We have the following important result. Lemma 4.2.3. Consider the improvement function h k given in (4.2.8). Then, h k (x) 0 for all x ∈ X and for each iteration k. If there exists a sequence {x k rec } ⊂ X such that lim k h k (x k rec ) = 0, then any cluster point of the sequence {x k rec } is a solution to problem (4.2.2). In particular, if h k (x k rec ) = 0 for some k then x k rec is a solution to problem (4.2.2).

Proof. Let x ∈ X and k be arbitrary. If c(x) 0, then f (x) f k low by definition of f k low ; on the other hand, if f (x) < f k low then x cannot be feasible, i.e., c(x) > 0. It thus follows trivially that h k (x) 0 for all x ∈ X, as x and k were taken arbitrarily.

Assume that a sequence {x

k rec } ⊂ X such that lim k h k (x k rec ) = 0 is given. It follows from (4.2.8) that h k (x k rec ) ≥ f (x k rec ) -f k low f (x k rec ) -f min . We can thus conclude (i) 0 lim k (f (x k rec ) -f k low ) lim k (f (x k rec ) -f min )
and similarly (ii) 0 lim k c(x k rec ). Let x be a cluster point of the sequence {x k rec } ⊂ X. Then x ∈ X and by (ii) c(x) 0. Therefore, since x is feasible, by (i) we have that 0 lim k (f (x k rec )-f min ) = f (x)-f min 0. Hence, x is a solution to problem (4.2.2).

In particular, if h k (x k rec ) = 0 for some k then 0 f (x k rec )f min and 0 c(x k rec ), showing that x k rec is a solution to problem (4.2.2).

That having been said, we will provide an algorithm that generates a sequence of iterates {x k } ⊂ X, a subsequence of recorded iterates {x k rec } ⊂ {x k }, a nondecreasing sequence of lower bounds {f k low } for f min , and a sequence of levels

f k lev ≥ f k low such that f k low ↑ f min , f k lev → f min and h k (x k rec ) → 0 .
In order to do so, at iteration k we obtain a new iterate x k+1 by projecting a given stability center xk ∈ X (not necessary feasible for c) onto the level set

X k := {x ∈ X : f k (x) f k lev , čk (x) 0} (4.2.9)
whenever it is nonempty. Obtaining x k+1 amounts to solving the following quadratic problem (QP)

x k+1 := arg min

x∈X k 1 2 |x -xk | 2 , or just x k+1 := P X k (x k ) for short. (4.2.10)
Whenever X k is empty, the current level f k lev and lower bound f k low require updating. The following proposition provides us with properties of the minimizer x k+1 . We will use the notation N X for the normal cone of convex analysis of a set X. Proposition 4.2.4. The point x k+1 solves (4.2.10) if and only if

x k+1 ∈ X, f k (x k+1 ) f k lev , čk (x k+1
) 0, and there exist vectors

ĝk f ∈ ∂ f k (x k+1 ), ĝk c ∈ ∂č k (x k+1 ), s k ∈ N X (x k+1 ) and stepsizes µ k f , µ k c 0, such that x k+1 = xk -(µ k f ĝk f +µ k c ĝc +s k ) , µ k f ( f k (x k+1 )-f k lev ) = 0 and µ k c čk (x k+1 ) = 0 . (4.2.11)
In addition, the aggregate linearizations

f a f (k) (•) := f k (x k+1 ) + ĝk f , • -x k+1 satisfies f a f (k) (x) f k (x) f (x) for all x ∈ X (4.2.12a) ca c (k) (•) := čk (x k+1 ) + ĝk c , • -x k+1 satisfies ca c (k) (x) čk (x) c(x) for all x ∈ X. (4.2.12b) Moreover, P X k (x k ) = P X a(k) (x k ) (4.2.13)
holds, where X a(k) is the aggregate level set defined as X a(k) := {x ∈ X :

f a f (k) (x) f k lev , ca c (k) (x) 0}.
Proof. Let i X be the indicator function of the polyhedral set X, i.e., i X (x) = 0 if x ∈ X and i X (x) = ∞ otherwise. Remembering that the set X = ∅ is polyhedral and by [197, p.215] ∂i X (x) = N X (x) for x ∈ X, the first claim results from the KKT conditions for problem (4.2.10) rewritten as

   min x∈R n 1 2 |x -xk | 2 + i X (x) s.t. f k (x) f k lev čk (x) 0 .
The subgradient inequality gives (4.2.12).

We now proceed to show (4.2.13). Notice that P X a(k) (x k ) is the solution to the following QP:

   min x∈R n 1 2 |x -xk | 2 + i X (x) s.t. f a f (k) (x) f k lev ca c (k) (x) 0 . (4.2.14)
Moreover, x solves the above problem if, and only if, there exist ρ, λ 0 and s ∈ N X (x) such that

x = xk -(ρĝ k f + λĝ c + s) , ρ( f a f (k) (x) -f k lev ) = 0 and λc a c (k) (x) = 0 .
Notice that by taking ρ = µ k f , λ = µ k c and s = s k above, the resulting x = x k+1 satisfies the optimality conditions for problem (4.2.14). Since (4.2.14) has a unique solution, we conclude that x k+1 = P X a(k) (x k ), as stated.

The interest of the aggregate level set is that X a(k) condenses all the past information relevant for defining x k+1 . This property is crucial for the bundle compression mechanism. Indeed, this property makes it possible to change the feasible set X k in (4.2.10), possibly with many constraints, by any smaller set containing X a(k) . In this manner, the size of the bundles J k f and J k c in (4.2.7) can be kept controlled, thus making (4.2.10) easier to solve; see comment (g) below.

We are now in position to give the algorithm with full details. In the algorithm we will keep track of special iterations called "critical iterations" that allow us to either progress significantly in the quest of solving (4.2.2) or improve our currently proven lower bound for f min in (4.2.2). Critical iterations are counted with l. All iterations are counted with k. Iterations between the change of the critical iteration counter from l to l + 1 are said to belong to the l-th cycle. We will moreover denote with k(l) the critical iteration at the start of the l-th cycle and hence K l = {k(l), k(l) + 1, . . . , k(l + 1) -1} is the index set gathering the iterations in the l-th cycle. The recorded sequence {x k rec } is constructed by defining x 0 rec equal to the first iterate x 0 and

x k rec :=    x k-1 rec if min j∈J k f ∩J k c h k (x j ) > h k-1 (x k-1 rec ) and k 1 or {J k f ∩ J k c } = ∅, arg min j∈J k f ∩J k c h k (x j ) otherwise. ( 4

.2.15)

Algorithm 4.2 (Level Bundle Method). We assume given an oracle computing f /c values as in (4.2.6) for any x ∈ X.

Step 0 (Input and Initialization) Choose a parameter γ ∈ (0, 1) and a stopping tolerance δ Tol > 0. Given x 0 ∈ X and a lower bound f 0 low f min , set x0 ← x 0 . Call the oracle to obtain (f (x 0 ), g f 0 ) and (c(x 0 ), g c 0 ) and set l ← 0, k ← 0, k(l) ← 0, J 0 f ← {0} and J 0 c ← {0}.

(x k rec ) ≤ (1 -γ)h k(l) (x k(l)
rec ). If so we declare a critical iteration and we set l

← l + 1, k(l) ← k and choose xk ∈ {x j : j ∈ J k f ∩ J k c }. Step 4 (Level Updating) Set f k lev ← f k low + γh k (x k rec
) and update the feasible set X k given in (4.2.9).

Step 5 (Quadratic Program) Try to solve the quadratic program (4.2.10). If no feasible solution is found then declare a critical iteration, set It may happen by Step 3 and 5 that K l is an empty set. In the convergence analysis given below we simply assume that K l is nonempty for each l. This strategy eases the calculations without loosing generality. low because the stopping test in Step 2 fails. Since f min < ∞ is an upper bound on this sequence, we must therefore have that h k (x k rec ) → 0. Hence, if δ Tol > 0 there cannot be an infinite loop between Steps 5 and 1. (e) In order to avoid excessively short cycles, we can solve the following linear problem to update f k low at Step 5:

l ← l + 1, k(l) ← k, f k low ← f k lev and choose xk ∈ {x j : j ∈ J k f ∩ J k c }.
(c) If the level set X k is empty, this means that f k lev < f k (x) for all x ∈ {z ∈ X : čk (z) 0}. Since čk (•) c(•), it follows that {z ∈ X : c(z) 0} ⊆ {z ∈ X : čk (z) 0} and thus, f k lev < f k (x) for all x ∈ {z ∈ X : c(z) 0}. As f k (•) f (•) we conclude that f k
f k low = min x∈X f k (x) s.t. čk (x) 0 . ( 4

.2.17)

As f k lev > f k low , then X k will be nonempty. (f) If problem (4.2.17) is solved at each iteration at Step 3, then the level set X k is nonempty. However, this procedure might not ensure (4.2.16) and thus Lemma 4.2.7 below holds only if there is no bundle compression along each cycle K l , i.e., J k+1

f = J k f ∪ {k + 1} and J k+1 c = J k c ∪ {k + 1} for all k ∈ K l .
For more details see [136, p. 121] or [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF]Lemma 3.5]. (g) In fact, the aggregate linearizations (4.2.12) do not need to be included in the bundle at each iteration but only when the bundle is compressed. This provides a versatile framework for dealing with bundle compression and avoid encumbering memory.

Considering that the bundle is full (we have saved more elements than some a priori bound), a quite practical rule would consist of keeping only the active linearization index sets:

Jk f := {j ∈ J k f : f j (x k+1 ) = f k lev } and Jk c := {j ∈ J k c : cj (x k+1 ) = 0}
. If the bundle is still considered full, we can replace any two indices i, j ∈ Jk f by a f (k) (respectively i, j ∈ Jk c by a c (k)) until the Bundle is sufficiently cleaned up. Then we update as follows:

J k+1 f = Jk f ∪ {k + 1} and J k+1 c = Jk c ∪ {k + 1} . (h) After having solved problem (4.2.10), each constraint f j (x k+1 ) f k lev with j ∈ J k f (respec. ci (x k+1 ) 0 with i ∈ J k c ) has a Lagrange multiplier α j f 0 (respec. α i c 0
). Furthermore, it follows from the cutting-plane definition that g f j ∈ ∂f (x k+1 ) for all j ∈ J k f such that α j f > 0 (respec. g c i ∈ ∂c(x k+1 ) for all i ∈ J k c such that α i c > 0). In this manner, by taking

µ k f = j∈J k f α j f and ĝk f = 1 µ k f j∈J k f α j f g f j (respec. µ k c = j∈J k c α j c and ĝk c = 1 µ k c j∈J k c α j c g c j )
the aggregate linearizations (4.2.12) can be easily obtained.

If µ k f = 0, then all constraints f j (x k+1 ) f k lev are inactive (the inequality holds strictly) and all the bundle J k f information is useless. Then the aggregate linearization f a(k) is also inactive. The equivalent conclusion holds if µ k c = 0. Lemma 4.2.6 below ensures that µ k c = µ k f = 0 cannot happen, otherwise we would have x k+1 = xk . Steps 6 and 7 in Algorithm 4.2 are optional in a way. We can consider two important alternatives which will be particularly useful when either the f or the c oracle is costly.

A variant of Algorithm 4.2 for costly c-oracles

The following strategy is useful if the constraint oracle information (c(x k+1 ), g c k+1 ) is more difficult to obtain than the information (f (x k+1 ), g f k+1 ). This is the case of Example 4.2.2.

Goal Avoid computing the c-oracle information if x k+1 does not give a significant decrease.

Step 6(f) (Oracle) Set f k+1 low ← f k low , xk+1 ← xk and call the oracle to obtain (f (x k+1 ), g f k+1 ). If f (x k+1 ) f k lev + (1 -γ)h k (x k rec ) go to Step 7(f). Otherwise, before going to Step 7(f) compute (c(x k+1 ), g c k+1 ).
Step 7(f) (Bundle Management) Manage the bundle freely as long as:

• (If both f (x k+1 ) and c(x k+1 ) were computed at Step 6(f))

J k+1 f ⊃ {k + 1, a f (k)} and J k+1 c ⊃ {k + 1, a c (k)} hold true at each bundle com- pression. • (If c(x k+1 ) was not computed at Step 6(f)) J k+1 f ⊃ {k + 1, a f (k)} and J k+1 c ⊃ {a c (k)} hold true at each bundle compression.
We emphasize that the test f (x k+1 )

f k lev + (1 -γ)h k (x k rec ) implies that the c-oracle information is computed only if f (x k+1 ) < max{f (x k rec ), f k low +c(x k rec )}, i.e.
, when f (x k+1 ) provides decrease with respect to at least one of the two thresholds f (x k rec ) or f k low +c(x k rec ).

A variant of Algorithm 4.2 for costly f -oracles

If (f (x k+1 ), g f k+1
) is more difficult to compute than (c(x k+1 ), g c k+1 ) we may consider the following strategy in Algorithm 4.2:

Goal Avoid computing the f -oracle information if x k+1 is far from feasibility.

Step 6(c) (Oracle) Set

f k+1 low ← f k low , xk+1 ← xk and call the oracle to obtain (c(x k+1 ), g c k+1 ). If c(x k+1 ) (1 -γ)h k (x k rec ) go to
Step 7(c). Otherwise, before going to Step 7(c) compute (f (x k+1 ), g f k+1 ).

Step 7(c) (Bundle Management) Manage the bundle freely as long as:

• (If both f (x k+1
: j ∈ J k f ∩ J k c } if J k f ∩ J k c = ∅. Otherwise, xk ∈ {x j : j ∈ J k f and f (x j ) f j-1 lev + (1 -γ)h j-1 (x j-1 rec )}. -At Steps 3 and 5 of Algorithm 4.2(c), choose xk ∈ {x j : j ∈ J k f ∩ J k c } if J k f ∩ J k c = ∅. Otherwise, xk ∈ {x j : j ∈ J k c and c(x j ) (1 -γ)h j-1 (x j-1 rec )}.
Here comes an interesting feature of Algorithm 4.2: differently from the constrained level bundle methods [START_REF] Lemaréchal | New variants of bundle methods[END_REF], [START_REF] Kiwiel | Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities[END_REF], [START_REF] Fábián | Bundle-type methods for inexact data[END_REF], [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] and version (fc), the versions (f) and (c) of Algorithm 4.2 do not need to include the two linearizations f k+1 and ck+1 into the bundle at each iteration k. As shown in Theorem 4.2.9 below, even considering less oracle information, versions (f) and (c) provide the same complexity result than version (fc).

Convergence analysis of the above three versions ((fc), (f) and (c)) of Algorithm 4.2 is given in the following subsection.

Convergence analysis

Throughout this section we suppose the feasible set X in (4.2.2) is compact with diameter D, and denote by Λ f > 0 (respectively Λ c ) a Lipschitz constant for the objective function (respectively constraint) over the set X. Therefore, Λ := max{Λ f , Λ c } is a Lipschitz constant for h k , defined in (4.2.8). In what follows, both constants D and Λ can be unknown, since they are not used by the algorithm but only to give upper bounds for the length of cycles K l .

In the following lemma, we extend the result given in [127, Lemma 3.1] to our setting.

Lemma 4.2.6. Let l ≥ 0 be arbitrary, Λ = max{Λ f , Λ c } and γ ∈ (0, 1) be given. At iteration k of any version (fc), (f ) or (c) of Algorithm 4.2, the following estimates hold:

|x k+1 -x k | (1 -γ) Λ h k (x k rec ) if k > k(l) |x k+1 -xk | (1 -γ) Λ h k (x k rec ) if k = k(l) .
Proof. Let us split this proof in three cases: we consider Algorithm 4.2(fc) first, then Algorithm 4.2(f) and finally Algorithm 4.2(c). We now study the first case.

Let k be arbitrary and j ∈ J k f ∩ J k c be given. By (4.2.9) combined with (4.2.10), we get

f (x j ) + g f j , x k+1 -x j f k lev c(x j ) + g c j , x k+1 -x j 0 .
By applying the Cauchy-Schwarz inequality we thus derive

f (x j ) -f k lev |g f j ||x k+1 -x j | Λ f |x k+1 -x j | c(x j ) |g c j ||x k+1 -x j | Λ c |x k+1 -x j | . Since Λ = max{Λ f , Λ c }, f k lev = f k low + γh k (x k rec ) and h k (x k rec ) 0, we conclude that Λ|x k+1 -x j | max{f (x j ) -f k low -γh k (x k rec ), c(x j )} max{f (x j ) -f k low -γh k (x k rec ), c(x j ) -γh k (x k rec )} = -γh k (x k rec ) + max{f (x j ) -f k low , c(x j )} = -γh k (x k rec ) + h k (x j ) (4.2.18) (1 -γ)h k (x k rec ) .
Assuming that k > k(l), then the Oracle and Bundle Management steps assure that k ∈ J k f ∩ J k c . Hence the estimate (4.2.18) with j = k provides the result. When k = k(l), the stability center xk is chosen among the bundle members. Hence, there exists a j ∈ J k f ∩ J k c such that x j = xk and (4.2.18) allows us to conclude this proof for the first case.

For versions 4.2(f) and 4.2(c) of Algorithm

, if k ∈ J k f ∩ J k c or j ∈ J k f ∩ J k c with x j = xk nothing more needs to be shown. Consider Algorithm 4.2(f) and j ∈ J k f but j / ∈ J k c together with f (x j ) -f j-1 lev (1 -γ)h j-1 (x j-
(x j ) -f k lev (1 -γ)h k (x k rec ) .
Since x k+1 is feasible for (4.2.10), then f (x j ) + g f j , x k+1x j f k lev . All together we conclude that

Λ|x k+1 -x j | g f j , x j -x k+1 f (x j ) -f k lev (1 -γ)h k (x k rec )
, which was to be shown. Consider Algorithm 4.2(c) and j ∈ J k c but j / ∈ J k f such that c(x j ) (1γ)h j-1 (x j-1 rec ), which holds for j = k and j s.t. x j = xk . In this case we obtain c(

x j ) (1 - γ)h j-1 (x j-1 rec ) (1 -γ)h k (x k rec )
, where the last inequality follows from (4.2.15). Since x k+1 is feasible for (4.2.10), then 0 c(x j ) + g c j , x k+1x j . It follows from Cauchy-Schwarz inequality that

Λ|x k+1 -x j | g c j , x j -x k+1 c(x j ) (1 -γ)h k (x k rec )
, which concludes the proof. Lemma 4.2.6 shows that consecutive iterates are different. The following result implies that each cycle K l is finite. Lemma 4.2.7. Let l ≥ 0 be arbitrary, D be the diameter of the feasible set X and Λ = max{Λ f , Λ c }. Assume that at each bundle compression in the versions (fc), (f ) and (c) of Algorithm 4.2 we include the aggregate indices in the bundle during the l-th cycle. Then, for all three versions (fc), (f ) and (c) of Algorithm 4.2, any iteration k in the l-th cycle (i.e., k ∈ K l ) with h k (x k rec ) > δ Tol may differ no more from k(l) than the following bound:

k -k(l) + 1 ΛD (1 -γ)h k (x k rec ) 2
Proof. Let k ∈ K l be such that k > k(l). It follows from (4.2.10) that

x k = P X k-1 (x k-1 ); thus xk-1 -x k , x -x k 0 for all x ∈ X k-1 . (4.2.19)
If no bundle compression took place between step k -1 and step k, it holds that f k (x) f k-1 (x), čk (x) čk-1 (x) for all x ∈ R n . Since f k lev ≤ f k-1 lev by (4.2.16), it follows that X k ⊆ X k-1 . Since k belongs to the l-th cycle, X k is nonempty and x k+1 ∈ X k . This fact combined with xk-1 = xk and (4.2.19) allows us to deduce that xkx k , x k+1x k ≤ 0. When bundle compression took place between steps k -1 and k, the aggregate indices belong to the bundle for each of the three considered versions, i.e., a f (k -1) 1) . Similarly as before we can use (4.2.20) to obtain xkx k , x k+1x k 0 for each of the versions (fc), (f) and (c). Therefore, by developing squares in the identity

∈ J k f and a c (k -1) ∈ J k c for Algorithms 4.2(fc), 4.2(f) and 4.2(c). This implies f k (x) f a f (k-1) (x), čk (x) ca c (k-1) (x) for all x ∈ R n . It follows from (4.2.13) that x k = P X a(k-1) (x k-1 ) and thus, xk-1 -x k , x -x k 0 for all x ∈ X a(k-1) . (4.2.20) Since f k lev ≤ f k-1 lev by (4.2.16), it follows that X k ⊂ X a(k-
|x k+1 -xk | 2 = |x k+1 -x k + (x k -xk )| 2 we have that |x k+1 -xk | 2 |x k -xk | 2 + |x k+1 -x k | 2 .
As h k (x rec ) > δ Tol for all k ∈ K l , Algorithm 4.2 does not stop and xk = xk(l) . A recursive application of the above inequality implies

|x k+1 -xk | 2 |x k -xk | 2 + |x k+1 -x k | 2 = |x k -xk-1 | 2 + |x k+1 -x k | 2 |x k-1 -xk-1 | 2 + |x k -x k-1 | 2 + |x k+1 -x k | 2 . . . |x k(l)+1 -xk(l) | 2 + k j=k(l)+1 |x j+1 -x j | 2 .
Since X is compact with diameter D and both x k+1 and xk are in X, then D |x k+1xk |.

Together with Lemma 4.2.6, this yields the bounds

D 2 (1 -γ) Λ h k (x k rec ) 2 + k j=k(l)+1 (1 -γ) Λ h k (x k rec ) 2 = (1 -γ) Λ h k (x k rec ) 2 (k-k(ℓ)+1) ,
and result follows as k was arbitrary.

If the stopping test tolerance is set to zero, we now show that any cluster point of the recorded sequence of points generated by any version of Algorithm 4.2 solves problem (

Theorem 4.2.8. Consider any version (fc), (f ) or (c) of Algorithm 4.2. Suppose that δ Tol = 0 and that the algorithm does not terminate. Then lim k h k (x k rec ) = 0 and any cluster point of the sequence {x k rec } is a solution to problem (4.2.2).

Proof. If any version of Algorithm 4.2 stops at iteration k with h k (x k rec ) = 0, by Lemma 4.

x k

rec is an optimal solution to problem (4.2.2). If the algorithms do not stop, then by Lemma 4.2.7 (each cycle K l is finite) we conclude that l → ∞. Let W (respec. L) be an index set gathering counters l updated at Step 3 (respec. Step 5) of (any version of) Algorithm 4.2. Let l i be the i-th index in W = {l 1 , l 2 . . . , }. Therefore, the inequality h k(l i ) (x

k(l i ) rec ) (1 -λ)h k(l i-1 ) (x k(l i-1 ) rec
) holds even if there exists some l ∈ L such that l i-1 < l < l i . A recursive application of the previous inequality gives us

0 h k(l i ) (x k(l i ) rec ) (1 -λ)h k(l i-1 ) (x k(l i-1 ) rec ) . . . (1 -γ) i h k(0) (x k(0) rec ) .
As both function f and c are finite valued, if W has infinitely many indices then

lim i h k(l i ) (x k(l i )
rec ) = 0. Moreover, lim k h k (x k rec ) = 0 by monotonicity; see (4.2.15). Let us now suppose that W is finite. As l → ∞, then the index set L has infinitely many indices. As f k(l+1) low is increased at Step 5 of the algorithm by an amount of γh k (x k rec ) 0 for each l ∈ L, we obtain that lim k h k (x k rec ) = 0 (otherwise we would have

f k low ↑ ∞, which contradicts f k low f min < ∞).
We have shown in both cases (W finite or infinite) that lim k h k (x k rec ) = 0. Hence, the stated result follows from Lemma 4.2.3.

Complexity results

We now provide an upper bound for the maximum number of iterations performed by versions (fc), (f) and (c) of Algorithm 4.2 in order to reach a given tolerance δ Tol > 0.

To do so, we consider all the versions of Algorithm 4.2 without the optional Step 3. Theorem 4.2.9. Let D be the diameter of the feasible set X and Λ = max{Λ f , Λ c }. Let -∞ < f 1 low f min be given in all versions (fc), (f ) and (c) of Algorithm 4.2 and let the Step 3 be deactivated. Assume that at each bundle compression the aggregate indices are included in the bundle. Then, to reach an optimality measure h k (x k rec ) smaller than δ Tol > 0 Algorithms 4.2(fc), 4.2(f ) and 4.2(c) perform at most

1 + f min -f 1 low γδ Tol ΛD (1 -γ)δ Tol 2 iterations.
Proof. Consider any arbitrary but fixed version of Algorithm 4.2. Notice that every time that X k = ∅, the lower bound f k low for the optimal value f min is increased by an amount of γh k (x k rec ) (> γδ Tol ). Since f 1 low is finite, the maximum number of cycles l mx times the stepsize γδ Tol is less than f minf 1 low , i.e.,

l mx f min -f 1 low γδ Tol .
It follows from Lemma 4.2.7 that each cycle K l has at most

ΛD (1-γ)δ Tol 2
iterations, where we have used that h k (x k rec ) > δ Tol since the algorithm did not stop at iteration k. Let K(δ Tol ) := {1, 2, . . . , } be the index set for which h k (x k rec ) > δ Tol . We have thus shown that K(δ Tol ) is finite and if k δ Tol is its maximal element, then:

k δ Tol 1 + f min -f 1 low γδ Tol ΛD (1 -γ)δ Tol 2 ,
as was to be shown.

Notice that the better is the initial lower bound f 1 low for the optimal value f min , the lower is the upper bound for the maximum number of iterations performed by any considered version of Algorithm 4.2. If a lower bound f 1 low is not available, we might obtain an initial lower bound by solving a linear program, as follows.

Corollary 4.2.10. In the setting of Theorem 4.2.9, suppose that the initial lower bound f 1 low is defined as

f 1 low = min f 1 (x) s.t. č1 (x) 0 , x ∈ X . Then, f min -f 1 low 2Λ f D
and to reach an optimality measure h k (x k rec ) smaller than δ Tol > 0 Algorithms 4.2(fc), 4.2(f ) and 4.2(c) perform at most

1 + 2Λ f D γδ Tol ΛD (1 -γ)δ Tol 2 iterations.
Proof. We have that f min = f (x min ) f (x 1 ) + Λ f D. Let x be a solution to the above linear program. Then

f 1 low = f 1 (x) = f (x 1 ) + g f 1 , x -x 1 f (x 1
) -Λ f D by Cauchy-Schwarz inequality. The result follows by combining the preceding inequalities and invoking Theorem 4.2.9.

So far we have considered only exact oracle information. In the next section we deal with inexactness from the oracle.

Level Bundle Method for Inexact Oracles

In this section we assume that both functions f and c in (4.2.2) are inexactly evaluated. In what follows we will assume that the oracle provides us for each x ∈ X with

           f -oracle information f x = f (x) -η x f and gf x ∈ R n such that f (•) f x + gf x , • -x -ǫ x f c-oracle information c x = c(x) -η x c and gc x ∈ R n such that c(•) c x + gc x , • -x -ǫ x c (4.2.21) for some unknown η x f , η x c , ǫ x f , ǫ x c ∈ R. By substituting f x = f (x) -η x
f in the second inequality and evaluating at x we derive that η

x f -ǫ x f . Similarly we observe that η x c -ǫ x c . Therefore, gf x ∈ ∂ (η x f +ǫ x f ) f (x) and gc x ∈ ∂ (η x c +ǫ x c ) c(x)
. Throughout this section we will make the assumption that the error on each of these estimates is bounded, i.e., there exist constants η f , η c , ǫ f , ǫ c 0 such that

|η x f | η f , ǫ x f ǫ f , |η x c | η c , ǫ x c ǫ c for all x ∈ X . (4.2.22)
With such information from the oracle, we consider the approximate linearizations

f k (x) := f k x + gf k , x -x k and ck (x) := c k x + gc k , x -x k (4.2.23)
to set up the cutting-plane models in (4.2.7), which gives Given a (approximate) lower bound f k low for the optimal value f min in (4.2.2), we define the inexact improvement function by

f k (x) f (x) + ǫ f and čk (x) c(x) + ǫ c for all k and all x ∈ X. ( 4
h k i (x) = max{f x -f k low , c x } . ( 4
(i) 0 lim k (f x k rec -f k low ) lim k (f (x k rec ) -η f -f k low ) lim k (f (x k rec ) -η f -f min -ǫ f ) , (ii) 0 lim k c x k rec lim k c(x k rec ) -η c .
Let x be a cluster point of the sequence {x k rec } ⊂ X. Then x ∈ X and we conclude by (i) and (ii) that f (x)

f min + η f + ǫ f and c(x) η c , i.e, x is a η-solution to problem (4.2.2).
We now show that Lemma 4.2.6 still holds when all the three versions of Algorithm 4.2 are applied with the inexact improvement function h i of (4.2.26). Lemma 4.2.12. Let l ≥ 0 be arbitrary, Λ be the bound in (4.2.25) and γ ∈ (0, 1) be given. At iteration k of all versions (fc), (f ) and (c) of Algorithm 4.2 applied to improvement function h i of (4.2.26), assume that h k i (x k rec ) ≥ 0 holds. Then the following estimates are true:

|x k+1 -x k | (1 -γ) Λ h k i (x k rec ) if k > k(l) |x k+1 -xk | (1 -γ) Λ h k i (x k rec ) if k = k(l) .
Proof. In the following we focus on the version (fc), i.e., Algorithm 4.2(fc). The proofs for the remaining two versions can be obtained in a similar manner to the proof of Lemma 4.2.6.

Let k be arbitrary and j ∈ J k f ∩ J k c be given. By (4.2.9) combined with (4.2.10) and assumptions (4.2.21), we get

f x j + gf j , x k+1 -x j f k lev c x j + gc j , x k+1 -x j 0 .
By applying the Cauchy-Schwarz inequality we thus derive

f x j -f k lev | gf j ||x k+1 -x j | Λ|x k+1 -x j | c x j | gc j ||x k+1 -x j | Λ|x k+1 -x j | . Since f k lev = f k low + γh k i (x k rec ) and h k i (x k rec ) 0 by assumption, we conclude that Λ|x k+1 -x j | max{f x j -f k low -γh k i (x k rec ), c x j } max{f x j -f k low -γh k i (x k rec ), c x j -γh k i (x k rec )} = -γh k i (x k rec ) + max{f x j -f k low , c x j } = -γh k i (x k rec ) + h k i (x j ) (4.2.27) (1 -γ)h k i (x k rec ) . Assuming that k > k(l), then the Oracle and Bundle Management steps assure that k ∈ J k f ∩ J k c .
Hence the estimate (3.4.14) with j = k provides the result. When k = k(l), the stability center xk is chosen among the bundle members. Hence, there exists a j ∈ J k f ∩ J k c such that x j = xk and (4.2.27) allows us to conclude this proof.

Similarly Lemma 4.2.7 can be derived by a slight change of conditions, as shown below.

Lemma 4.2.13. Let l ≥ 0 be arbitrary, D be the diameter of the feasible set X and Λ be the bound of (4.2.25). Consider the versions (fc), (f ) and (c) of Algorithm 4.2, with improvement function h i given in (4.2.26). Assume that at each bundle compression in all versions (fc), (f ) and (c) of Algorithm 4.2 we include the aggregate indices in the bundle during the l-th cycle. Then, any iteration k in the l-th cycle (i.e., k ∈ K l ) may differ no more from k(l) than the following bound, provided that

h j i (x j rec ) > δ Tol > 0 for all j ≤ k: k -k(l) + 1 ΛD (1 -γ)h k i (x k rec ) 2
The proof of the above lemma is identical to the proof of Lemma 4.2.

In what follows we make a distinction on the possible oracle types (4.2.21). We start by considering the most general case: upper oracles.

Inexact constrained level methods for upper oracles

An upper oracle is an oracle of type (4.2.21) wherein ǫ f > 0 and/or ǫ c > 0 in (4.2.22) might occur. This in particular implies that η x f , η x c can be negative too and that the f -(c-) values can be overestimated.

Since in this case (4.2.24) holds, we can no longer keep Step 5 of Algorithm 4.2 as we can no longer assure that f k lev remains a lower bound for f min . We thus suggest to change Step 5 as follows:

Step 5 (Upper) Try to solve the quadratic program (4.2.10). If no feasible solution is found:

Step 5.1 solve the linear problem

val LP := min j∈J k c s j s.t. x ∈ X, cj (x) -s j 0 ∀ j ∈ J k c . (4.2.28)
If val LP > 0, stop (the problem is ǫ c -infeasible); otherwise, and that the oracle error is such that c(x s ) < -ǫ c as in [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF] would eliminate this case and the need for solving (4.2.28). However, how could we know if the inequality c(x s ) < -ǫ c holds when x s is unknown? In fact, as shown below we do not need to verify (at least explicitly) this inequality.

Let (x, s) be an optimal solution of problem (4.2.28). We say that cj is a wrong linearization if sj > 0. That having been said, by assuming the existence of a Slater point x s ∈ X we may modify Step 5 (Upper) above in order to eliminate wrong linearizations cj (•) that cause ǫ c -infeasibility. Eliminating indefinitely wrong linearizations from the bundle J k c may deteriorate the convergence. In order to avoid this problem, the adopted strategy is to clean the bundle J k c while requesting more accuracy from the c-oracle in (4.2.21). It follows from (4.2.29) that

{x ∈ X : čk (x) 0} = ∅ if ǫ c -c(x s ) . (4.2.30)
Hence, if the accuracy ǫ c is decreased until it satisfies ǫ c -c(x s ), no wrong linearization will be identified and the algorithm will not stop before satisfying the stopping test. In what follows, we formalize the above idea:

Step 5 (Upper On-demand) Try to solve the quadratic program (4.2.10). If no feasible solution is found:

Step 5.1 Solve the linear problem (4. Every time Step 5.2 is accessed, the maximal c-oracle error is decreased by half. Hence, after finitely many applications of Step 5.2 the algorithm will determine ǫ c -c(x s ), and by (4.2.30) the existence of a Slater point ensures that no infinite loop between Steps 5 and 5.2 can occur. Therefore, Theorem 4.2.14 holds without any change. We emphasize that only the assumption of such Slater point x s ∈ X is enough for our strategy. In fact, we need to know neither x s nor c(x s ). Since Step 5.2 requires more accuracy from the c-oracle by checking ǫ c -infeasibility, the above approach can be considered as the on-demand accuracy type. Up to our knowledge, upper oracles with on-demand accuracy have not been studied so far in the literature of inexact bundle methods.

In what follows we no longer assume a Slater point. Furthermore, we focus on a particular case of oracle (4.2.21): lower oracles.

Inexact constrained level methods for lower oracles

A lower oracle is an oracle consistent with setting (4.2.21) allowing for one-sided errors only: in (4.2.22), we therefore assume ǫ f ≤ 0, ǫ c ≤ 0.

We thus derive η

x f -ǫ x f 0, η x c -ǫ x c
0 for all x ∈ X. In particular, the cuttingplane models set up from approximate linearizations will remain below the true functions: using definition (4.2.7), the linearizations (4.2.23) provide for lower oracles cutting-plane models satisfying f k (x) f (x) and čk (x) c(x) for all iteration counter k and x ∈ X.

In order to state our convergence result, we provide the oracle assumptions formally: for each given x ∈ X, an inexact oracle delivers the same information as oracle (4.2.21), however with

ǫ x f 0, ǫ x c 0 . ( 4
:= max{η f , η c }.
Proof. Lower oracles have the special property that

x ∈ R n : f k (x) f k lev ∩ {x ∈ R n : c(x) 0} ⊆ X k , due to the inequality čk (•) c(•). If X k is found to be empty, we also have f k lev ≤ f k (x)
f (x) for all x ∈ X with c(x) 0, thus showing that f k lev is indeed a proven lower bound for f min . Moreover, since for lower oracles ǫ f = 0, Lemma 4.2.11 holds with η := max{η f , η c }.

We thus rely on Lemma 4.2.15 and Theorem 4.2.14 to give an asymptotic convergence result.

Theorem 4.2.16. Consider any version (fc), (f ) or (c) of Algorithm 4.2 applied with oracle (4.2.31) and improvement function h k i given in (4.2.26). Suppose that δ Tol = 0 and that the algorithm does not terminate. Then lim k h k i (x k rec ) 0 and any cluster point of the sequence {x k rec } is a η-solution to problem (4.2.2), with η := max{η f , η c }.

Since lower oracles are likely to be less noisy than upper oracles, it is natural to expect that a solution obtained by any version of Algorithm 4.2 employing a lower oracle has a better quality. This is confirmed by comparing Theorems 4.2.14 and 4.2.16. Notice that the only difference between these two theorems is in the solution error η: max{η f +ǫ f , η c } for upper oracles and max{η f , η c } for lower oracles.

If Algorithm 4.2 terminates at iteration k with h k i (x k rec ) δ Tol , then x k rec is a (max{η f , η c }+ δ Tol )-solution to problem (4.2.2). A more refined solution can be ensured if the oracle errors η x k f and η x k c vanish asymptotically for special iterates x k . This case is addressed in the following section.

Constrained level bundle methods for lower oracles with on-demand accuracy

Following the lead of [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] we denote x k ∈ X a f -substantial iterate if the inexact value of the function f k x meets a descent target f k tar ∈ R. Similarly, we denote x k ∈ X a csubstantial iterate if the inexact constraint value c k

x meets a feasibility target c k tar ∈ R. We call an iterate substantial if it is both f -and c-substantial. Moreover, we call substantial set S, the index set gathering iterations that provide substantial iterates.

The aim of this section is to make a few changes in Algorithm 4.2 to deal with lower oracles that cast into the setting of (4.2.21), with errors that vanish for substantial iterates. Once again, ǫ f ≤ 0, ǫ c ≤ 0 in (4.2.22). We also assume that for each given point x ∈ X, upper bound η 0 for the oracle error, and targets f tar and c tar , an inexact oracle delivers

      
the same information as oracle (4.2.21), with:

ǫ x f ≤ 0, ǫ x c ≤ 0 η x f η if f x f tar η x c η if c x c tar . (4.2.32)
Notice that the upper bound for the oracle error η is given as input; therefore, it is known and controllable. Moreover, oracle (4.2.32) provides information with accuracy up to η for substantial iterates. We denote an oracle satisfying (4.2.32) by oracle with on-demand accuracy. This kind of oracles was introduced in [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF] for unconstrained (or polyhedral constrained) convex nonsmooth optimization problems. The recent work [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] extends this concept to the constrained setting. Differently from the method in [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF], our proposal allows for bundle compression, an important feature for practical applications. Moreover, in [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] the additional linear problem (4.2.17) is solved at each iteration k, while in Algorithm 4.2 it is optional.

We emphasize that oracle (4.2.32) is still quite general. Notice that for given x k = x and η k = η = 0, the oracle provides exact information if both f k x and c k x meet the targets. This is a kind of partially exact oracle introduced in [START_REF] Kiwiel | An inexact bundle approach to cutting stock problems[END_REF] for an unconstrained proximal bundle method, and further studied in [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF] and [START_REF] Oliveira | Bundle methods in depth: a unified analysis for inexact oracles[END_REF]. Moreover, if both targets f k tar and c k tar are set as +∞ for all iterations, then oracle (4.2.32) is an exact one. An asymptotically exact version of oracle (4.2.32) is obtained if, again, f k tar = c k tar = ∞ for all k and η k → 0. A combination between partially exact and asymptotically exact versions of oracle (4.2.32) is possible. For this setting, in order to get an exact solution to problem (4.2.2) the algorithm must force η k → 0 only for substantial iterates.

Let f k low be a proven lower bound for f min at any iteration k. At any substantial iterate j ∈ S, the oracle bound η j is known and can be exploited in the definition of the improvement function. We therefore define:

h k ae (x j ) := max{f x j -f k low , c x j } + η j if j ∈ S h j-1 ae (x j-1 rec ) otherwise. (4.2.33)
Since we force j = 0 ∈ S, the above rule is well defined.

Lemma 4.2.17. Consider the improvement function h k ae given in (4.2.33), and S the substantial index set. It follows that h k ae (x j ) h k (x j ) 0 for all x j ∈ X and j ∈ S, with h k defined in (4.2.8). Besides, if there exists an index set K ⊂ S such that {x k rec } k∈K ⊂ X and lim k∈K h k ae (x k rec ) = 0, then any cluster point of the sequence {x k rec } k∈K is a solution to problem (4.2.2).

Proof. Note that for each j ∈ S, h k ae (x j ) = max{f

x j +η j -f k low , c x j +η j } max{f x j +η x j f - f k low , c x j + η x j c }. By the oracle assumptions, h k ae (x j ) max{f (x j ) -f k low , c(x j )} = h k (x j ). Therefore, lim k∈K h k ae (x k rec ) = 0 implies that lim k∈K h k (x k rec ) = 0. As function h k is monotone, then lim k h k (x k
rec ) = 0 and the stated result follows from Lemma 4.2.3.

With such a new setting, in order to show convergence of Algorithm 4.2 to an exact solution of problem (4.2.2) we need to force η k to zero, for substantial iterates. This is done by controlling the oracle error at Step 5 of Algorithm 4.2 as follows:

Step 5 (On-demand) This step is similar as Step 5, however, if problem (4.2.10) is solved go to Step 5.1 before going to Step 6.

Step 5.1 (Noise Updating) Update the oracle error η k+1 ← θh

k(l) ae (x k(l) rec ), for a given θ ∈ (0, (1 -γ) 2 ). Step 5.2 (Target Updating) Set f k+1 tar ← f k lev + (1 -γ)h k ae (x k rec ) and c k+1 tar ← (1 - γ)h k ae (x k rec )
. Go to Step 6. We now consider the following result based on Lemma 4.2.6.

|x k+1 -x k | [(1 -γ) 2 -θ] Λ h k ae (x k rec ) if k ∈ K l is such that k > k(l) .
Proof. We can proceed as in Lemma 4.2.12 to show that version (fc) satisfies

f k x -f k lev | gf k ||x k+1 -x k | Λ|x k+1 -x k | (4.2.34) c k x | gc k ||x k+1 -x k | Λ|x k+1 -x k | . (4.2.35) Suppose first that x k is a substantial iterate: k ∈ S. Since f k lev = f k low + γh k ae (x k rec ) and h k ae (x k rec ) 0 by Lemma 4.2.17, we conclude that Λ|x k+1 -x k | max{f k x -f k low -γh k ae (x k rec ), c k x } max{f k x -f k low -γh k ae (x k rec ), c k x -γh k ae (x k rec )} = -γh k ae (x k rec ) + max{f k x -f k low , c k x } = -γh k ae (x k rec ) + h k ae (x k ) -η k ,
where the equality follows from (4.2.33), due to the assumption that k ∈ S. Since k ∈ K l is such that k > k(l), we have by Step 5 (On-demand) that η k+1 = η k , and by Step 3

h k ae (x k rec ) > (1 -γ)h k(l) ae (x k(l)
rec ). All together with the identity η k+1 = θh

k(l) ae (x k(l) rec ) by Step 5 (On-demand) we conclude that Λ|x k+1 -x k | (1 -γ)h k ae (x k rec ) -θh k(l) ae (x k(l) rec ) > (1 -γ) 2 h k(l) ae (x k(l) rec ) -θh k(l) ae (x k(l) rec ) [(1 -γ) 2 -θ]h k(l) ae (x k(l) rec ) [(1 -γ) 2 -θ]h k ae (x k rec )
where the last inequality is due to the monotonicity of {h k ae (x k rec )}. Hence, the result follows for k ∈ S.

Under the assumption that k / ∈ S, i.e., x k is not a substantial iterate, we have (a) In what follows, D denotes again the diameter of compact set X. Lemma 4.2.19. In the setting of Lemma 4.2.18, consider the versions (fc), (f ) and (c) of Algorithm 4.2, with improvement function h k ae given in (4.2.33) and Step 5 replaced by Step 5 (On-demand). Assume that at each bundle compression in all versions (fc), (f ) and (c) of Algorithm 4.2 we include the aggregate indices in the bundle during the l-th cycle. Then, any iteration k in the l-th cycle (i.e., k ∈ K l ) may differ no more from k(l) than the following bound, provided that h j ae (x j rec ) > δ Tol > 0 for all j ≤ k:

f k x > f k tar , and/or (b) c k x > c k tar . -Case (a): the inequality f k x > f k tar implies f k x -f k-1 lev > (1 -γ)h k-1 ae (x k-1 rec ). Since k ∈ K l is such that k > k(l), we have that f k lev f k-1 lev . Therefore, it follows by monotonicity of h k ae that f k x -f k lev f k x -f k-1 lev > (1 -γ)h k-1 ae (x k-1 rec ) (1 -γ)h k ae (x k rec ) [(1 -γ) 2 -θ]h k ae (x k rec ) , and the result follows from (4.2.34). -Case (b): in this version, c k x > (1 -γ)h k-1 ae (x k-1 rec ) (1 -γ)h k ae (x k rec ) [(1 -γ) 2 - θ]h k ae (x k rec ),
k -k(l) + 1 1 + ΛD [(1 -γ) 2 -θ]h k ae (x k rec ) 2
Proof. Since (4.2.16) holds, we can proceed as in Lemma 4.2.7 to conclude that

D 2 |x k+1 -xk | 2 |x k(l)+1 -xk(l) | 2 + k j=k(l)+1 |x j+1 -x j | 2 , for k ∈ K l , l 0 .
Together with Lemma 4.2.18, this yields the bounds

D 2 k j=k(l)+1 [(1 -γ) 2 -θ] Λ h k ae (x k rec ) 2 = [(1 -γ) 2 -θ] Λ h k ae (x k rec ) 2 (k -k(ℓ)) ,
and result follows.

The main convergence result is given in the following theorem.

Theorem 4.2.20. Consider any version (fc), (f ) or (c) of Algorithm 4.2 applied with oracle (4.2.32) and improvement function h k ae given in (4.2.33). Suppose that Step 5 is replaced by Step 5 (On-demand), and that parameters are chosen to satisfy γ ∈ (0, 1) and θ ∈ (0, (1γ) 2 ). Suppose also that δ Tol = 0 and that the algorithm does not terminate. Then lim k h k ae (x k rec ) = 0 and any cluster point of the sequence {x k rec } is a solution to problem (4.2.2).

Proof. If any version of Algorithm 4.2 stops at iteration k with h k ae (x k rec ) = 0, then by (4.2.33) we have that k ∈ S. Lemma 4.2.17 ensures that (0 =)

h k ae (x k rec ) h k (x k rec ) 0; thus x k
rec is an optimal solution to problem (4.2.2). If the algorithm does not stop, then by Lemma 4.2.19 (each cycle K l is finite) we conclude that l → ∞. Let W (respec. L) be an index set gathering counters l updated at Step 3 (respec. Step 5 (On-demand)) of (any version of) Algorithm 4.2. Let l i be the i-th index in W = {l 1 , l 2 . . . , }. Therefore, the inequality h

k(l i ) ae (x k(l i ) rec ) (1 -λ)h k(l i-1 ) ae (x k(l i-1 ) rec
) holds even if there exists some l ∈ L such that l i-1 < l < l i . A recursive application of the previous inequality gives us

0 h k(l i ) ae (x k(l i ) rec ) (1 -λ)h k(l i-1 ) ae (x k(l i-1 ) rec ) . . . (1 -γ) i h k(0) ae (x k(0) rec ) .
As both function f and c are finite valued and the oracle error is bounded by η 0 < ∞, if W has infinitely many indices then lim i h

k(l i ) ae (x k(l i ) rec ) = 0. Moreover, lim k h k ae (x k rec ) = 0 by monotonicity; see (4.2.

33).

Let us now suppose that W is finite. As l → ∞, then the index set L has infinitely many indices. Since we are dealing with lower oracles, Lemma 4.2.15 shows that f k low is a proven lower bound for f min < ∞. As f k(l+1) low is increased at Step 5 (On-demand) of the algorithm by an amount of γh k ae (x k rec ) for each l ∈ L, we obtain that lim k h k ae (x k rec ) = 0 (otherwise we would have

f k low ↑ ∞, which contradicts f k low f min < ∞).
We have shown in both cases (W finite or infinite) that lim k h k ae (x k rec ) = 0. Hence, it follows from (4.2.33) that there exists an index set K ⊂ S such that lim k∈K h k ae (x k rec ) = 0, and together with Lemma 4.2.17 we have the stated result. Theorem 4.2.20 ensures that if δ Tol = 0, then the algorithm eventually finds an exact solution to problem (4.2.2). This is not necessary the case if the algorithm is employed with upper oracles with on-demand accuracy; see Theorem 4.2.14 and comments in Section 4.2.3.2.

Numerical experiments 4.2.4.1 Benchmark Instance

For Benchmarking purposes we will investigate a Joint-Chance-Constrained Programming problem coming from cascaded reservoir management. We refer to section 4.1.5 for an analysis of this problem and its adequacy with the structure exhibited in this section. We also refer to section 5.1 for a full description of the model. One could also consult [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF].

We recall, once again, that these problems fit in the abstract structure (4.2.4), which we provide here with a few more details:

       min q, x s.t. P[a r + A r x ξ b r + A r x] p Ãx ≤ b x 0 . (4.2.36)
In (4.2.36), ξ ∈ R m is a Gaussian random vector with variance-covariance matrix Σ and zero mean (we have explicitly extracted the non-zero average in a r , b r ). The Joint Chance Constraint arises since we wish to make sure that the volumes in the reservoirs remain within bounds with high enough probability. The latter volumes are impacted by random inflows and the turbining strategy. The latter will thus be rendered robust by (4.2.36). The polyhedral constraint set Ãx ≤ b results from modelling flow constraints, bounds on turbining, pumping and water valuation. Our first numerical experience consists of optimizing a hydro valley as in Figure 4.1(a). The dimension of the vector x in this instance is 672 and ξ has dimension 48. The key difficulty of problems of the type (4.2.36) is the dimension of ξ. The dimension of x can readily be increased without much impact on computation time.

In order to compute a (sub-)gradient of the chance constraint involved in (4.2.36), we use the formulae derived in Lemma 3.2.1. That formulae builds a link between computing a component of the partial derivative and the evaluation of probabilities of the form appearing in (4.2.36). Evaluating the latter probabilities requires using efficient numerical software such as Genz' Code [START_REF] Genz | Numerical computation of multivariate normal probabilities[END_REF][START_REF] Genz | Computation of multivariate normal and t probabilities[END_REF]. The latter code takes as an entry a precision ε g , up to which the probabilities are evaluated. For instance picking ε g = 1e -4 implies that all probabilities are exact up to this precision. This precision allows us to control the "inexactness" of our Oracle.

A second instance of problem (4.2.36) consists of the Hydro valley appearing in Figure 4.1(b). We will consider 3 variants of this latter instance, wherein the dimension of the random vector ξ appearing in (4.2.36) varies from 48, 96 up to 168. The dimension of vector x is 888 in all variants of this instance. The first case occurs when random inflows are considered on reservoirs "COCHE" and "MONTRIGON". The random vector has dimension 96 when reservoirs "BREVIERES" and "LA RAIE" are also impacted by uncertainty. Finally in the last case all reservoirs are assumed to have random inflows.

Preconditioning

Looking carefully at the quadratic program (QP) (4.2.10), one readily observes that the objective function has approximate order of magnitude O( 1 2 x 2 ). Indeed, xk is typically proportional to the last iterate, so

1 2 x -xk 2 = 1 2 x T x -(x k ) T x + (x k ) T xk ≈ -1 2 x T
x, by eliminating the constant. If x has large components, this may generate numerical problems for the QP solver (even with CPLEX 12.2). We have therefore scaled the objective function by an appropriate constant in quite a similar way as one would do with a proximal bundle method. Returning to section 4.1 it appears that µ 0 = 1e -6 would make a nice choice.

A second problem might occur when the polyhedral system Ãx ≤ b has disproportionate components with respect to the constraint čk (x) ≤ 0. We have thus computed b ratio =

max v i=1 | bi| min v i=1 | bi| +1
, where b ∈ R v . Let g T x ≤ e be an arbitrary constraint figuring in čk (x) ≤ 0. We have replaced it with sg T x ≤ se, where s = b ratio g ∞ . This scaling also has to be used for the supporting hyperplane approach, shortly described below.

The various compared algorithms

An elementary method for solving problems of the type (4.2.36) is the supporting hyperplane method, which we will label Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. We refer to section 2.8.4 for some historical comments on this method and to paragraph 4.1.6.1.3 for more specific information on this algorithm. The stopping criteria is based on obtaining a solution with proven optimality gap δ Tol . This algorithm requires a Slater point, which is computationally expensive to obtain.

We will also compare our Level Method with the Algorithm of section 4.1. We will refer to it as Alg.PB. Neither algorithm has used bundle compression. Since the considered inexact oracle is of the Upper type, the algorithm suggested in [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] is not suitable for our setting. This is why we have decided not to incorporate the algorithm of [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] in our numerical runs.

The numerical results show that Alg.LB:

1. Converges faster than Alg.PB to a solution with similarly optimality when parameters are appropriately set. This especially holds if the dimension is big. This might be due to the naive way in which the proximal parameter is updated. Nonetheless, even when parameters are not-fine tuned Alg.LB does equally well as Alg.PB.

2.

Both the proximal and level bundle method outperform Alg. [START_REF] Prékopa | Probabilistic programming[END_REF].

An extensive discussion of numerical results

It was shown in section 4.1 that an appropriate scaling of the constraints was needed in order to obtain correct numerical behaviour. This meant that instead of looking at the constraints c(x) ≤ 0 we numerically deal with Kc(x) ≤ 0 for an appropriately chosen constant K > 0. For the Algorithm of section 4.1 the constant K impacts both the constraints of the quadratic program and the definition of the improvement function.

For our Algorithm 4.2 it turns out that scaling is only needed for the definition of the improvement function. It is then clear that K > 0 allows us to define an implicit preference for improving first feasibility (large K values) or the objective function (small K values). This is also illustrated in Table 4.7.

K = 1e 3 , γ = 0.9, δ Tol = 5 Alg.LB -104160 139 [1] 20.4 K = 1e 6 , γ = 0.9, δ Tol = 5 Alg.LB * * - K = 1e 2 , γ = 0.9, δ Tol = 5 Alg.LB * * - K = 1e 1 , γ = 0.9, δ Tol = 5
One can also observe that Alg.LB is less sensitive to K than Alg.PB, which is a strong advantage. Nonetheless too small values of K make the algorithm stop with infeasible solutions. This is understandable in the setting of problem (4.2.36) since the constraint, a chance constraint, has a rather small value compared to the objective function. Very small K values make this term negligible for the Algorithm. The observed "divergence" of Alg.PB with a small K value is of another order. The algorithm does not manage to produce solutions with improved feasibility at all. The proximal term then slowly converges to infinity and induces numerical problems in solving the quadratic program. If one directly compares the nominal solution with K = 5e 4 and compares CPU time with the one figuring in Table 4.4 (first line) it turns out that Alg.PB is found to be faster here. This is induced by the fact that Genz' code was recoded (in C++) and an approximate 30% speed up was obtained in that manner.

The effect of γ on the convergence is investigated in Table 4.8. One can observe that the behaviour of the Algorithm with γ < 0.5 and γ > 0.5 is slightly different, but Algorithm 4.2 is not very sensitive to this choice. The number of iterations increases when γ is small since we will detect empty level sets more often. These iterations are however not very costly as we do not call the oracle when an empty level set is detected. The number of iterations seems to be rather stable when γ > 0.5.

In the standard variant of Alg.LB the level set (4.2.9) is detected to be empty when the quadratic solver attempting to solve (4.2.10) produces an infeasible solution. The lower bound f k low is then updated. Alternatively the lower bound can be updated by solving the additional linear program (4.2.17) in Step 5 of Algorithm 4.2. Since in these numerical experiments there is no bundle compression, solving (4.2.17) at each iteration does not Table 4.8: Effect of γ (nne x stands for nn10 x ). Precision of Genz' code ε g = 5e -4 . The number of iterations at which (4.2.10) is found infeasible is indicated in between brackets. impact the convergence analysis of the algorithm; see comment (f) after Algorithm 4.2. The effect of this choice is reported in Table 4.9. In the setting of problem (4.2.36) the c-oracle is costly to call and hence solving an additional linear program (or even quadratic program) has a negligible effect on computation time. A further effect of adding (4.2.17) to step 5 of the Algorithm is that (4.2.9) will never be empty anymore. In Figure 4.3 we compare the iterative process of the first case in Table 4.9 with its counterpart in Table 4.8. Top graphics show the evolution of the sequences {f (x k )}, {f k low } and {f k lev } along the iterations. The bottom left graphic compares the two sequences of {f k low } corresponding to the following two cases: LP (4.2.17) is never solved, and (4.2.17) is solved at each iteration of Algorithm 4.2. Remember that when LP (4.2.17) is never solved, the lower bound f k low is updated only when the level set (4.2.9) is empty. We thus conclude by the stairs in the bottom right graphic in Figure 4.3 that the level set was detected to be empty six times in a total of sixty two iterations (as shown at line 5 of Table 4.8). The Bottom right graphic shows the sequences {c(x k )} for both cases. Notice that when solving LP (4.2.17) the resulting sequence {x k } is nearer to feasibility.

Up until now we have worked with an reasonably precise oracle: ε g = 5e -4 . It is of interest to examine how results are when we are working with a precise Oracle: ε g = 1e -4 . In table 4.3 it was shown that a precision ε g = 1e -5 led to unreasonable computation times. We will therefore not investigate this choice here. Table 4.10 provides these results.

When looking at the results of Table 4.10 one can observe that both Alg.LB and Alg.PB outperform Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] in an important way. In particular we can note that a solution with similar optimality as Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] is produced by Alg.PB as early as the 67th iteration, which would correspond to an approximate computation time of around 53 minutes. The level method does not produce any intermediately feasible solutions and directly stops at optimality.

We now turn ourselves towards the instance consisting of the Isère Valley (Figure 4.1(b)). For these instances no Slater point was available so if we wish to use Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] we need to compute such a point. We have obtained this point by applying our Level Method with an increased p level (See (4.2.36) and (4.2.5)). We have taken p = 0.95 as a numeric value. Now if we use the objective function q of (4.2.36) too in such an algorithm, it is clear that we end up with a very strong Slater point. For instance, in the Isr96 instance below it turns out that the Slater point is only 0.3% suboptimal. Clearly the Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] algorithm will only need to do a few iterations to converge and obtain the optimal solution. Such a comparison would therefore be incorrect with respect to the other Algorithms, which would in fact boil down to a comparison between Alg.LB with hybrid stopping criteria and the other algorithms. Moreover in practice, such as in Unit-Commitment problems wherein cascaded reservoir management are a sub-problem, one would typically compute a Slater point against an early poor estimate of the true "price vector". See section 5.2 for such a Unit-Commitment decomposition setting. Then the above advantage would go away. We have thus arbitrarily set q = 0 when computing the Slater point. This time is not integrated in the results obtained below.

Returning to the work of section 4.1, we can notice that two parameters (α and β) allow the user to have a control over the severity of the descent test and the noise test respectively. In this benchmark we make them vary to a setting which was found beneficial there. As Alg.LB is concerned, we will pick the best parameter settings of Table 4.7. The scaling parameter K was investigated in the work [START_REF] Van Ackooij | Estimating the probabilistic contents of gaussian rectangles faster in joint chance constrained programming for hydro reservoir management[END_REF] and this led to the choice K = 1e5. Again such a choice can be argued to be intuitive since it is roughly half the order of magnitude of the objective function. It appears that Alg.LB works best with a choice of K roughly 10 times smaller. This is again confirmed in Table 4.11.

When making the comparison on instance Isr96, some comments are in order. One should note that Alg.PB with parameter α = 1 produces a solution with similar quality as Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] in the 129th iteration, which corresponds roughly to 90 minutes of CPU time. Algorithm Alg.PB with parameter α = 0 produces such a solution in the 167th iteration, which corresponds to 118 minutes of CPU time. So both version greatly outperform Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. Finally Alg.LB in its best setting largely outperforms both Alg.PB and Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. This is quite an encouraging result.

Now we turn our attention to the analysis of instance Isr168. When looking at Alg.PB, with α = 1, a solution with similar optimality as Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] is produced in the 153th iteration, which would correspond to about 480 minutes of CPU time. A substantial advantage over Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. The setting with α = 0 produces such a solution in the 177th iteration, with about 553 minutes of CPU time. The level method Alg.LB with K = 1e 4 produces a solution of such quality as early as the 42nd iteration, but slightly infeasible. Still feasibility is easily restored when a Slater point is available. Globally both bundle methods have a significant advantage over Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. The level method again offers advantages over the proximal variant since fine tuning the initial proximal parameter µ 0 is not very easy. A key difference between the proximal and the level bundle variant is that the former produces a feasible solution quickly and all successive stability centers are feasible solutions. The level method produces a feasible solution quite late, often near to the convergence.

As explained above the behaviour of Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] is heavily dependent on the choice of the Slater point. Surprisingly enough Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] performs quite well on instance Isr48, even though the obtained solution is sub-optimal. Furthermore, observing Table 4.10 and Table 4.7 it clearly appears that Alg. [START_REF] Prékopa | Probabilistic programming[END_REF] suffers far more from increased oracle accuracy than Alg.PB or Alg.LB. Indeed the performance ratio moves from Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]/ Alg.PB = 3.2 to 13.5 and from 6.8 to 39.7 respectively. And this comparison does not integrate the loss of optimality incurred for Alg. [START_REF] Prékopa | Probabilistic programming[END_REF]. This is confirmed by the results of Table 4.12.

Comments on Parameter Settings

The previous subsection was devoted to a comparison of Bundle Methods and the Supporting Hyperplane method for solving convex joint chance constrained programs. The latter method requires a Slater point, which might be tricky to obtain, but on the other hand the method has few parameters. This can be perceived as an advantage, since one might be afraid to have to spend much time in setting up appropriate choices. However this is untrue, since natural parameter choices for the Bundle methods led to good results. Indeed, both Bundle Methods have parameter settings to play with. The proximal variant, has choices for the definition of the nominal decrease, the noise test, the "magic" scaling parameter K and the initial proximal parameter µ 0 . The Level Method also has the scaling parameter K, γ and the stopping tolerance to set. We have however shown that results are relatively robust to changes in such parameters provided they are in the right zone. Also when moving from one instance to another (both coming from cascaded reservoir management) the same rules of thumb seem to give quite excellent initial choices. Globally for joint chance constraint programming we would suggest to solve the following deterministic equivalent of (4.2.36):

           min q, x s.t. a r + A r x 0 0 b r + A r x Ãx ≤ b x 0 (4.2.37)
and call x d its optimal solution. Then f 0 low = q, x d is a good lower bound for (4.2.36), K = 1 2 q, x d an appropriate choice for the magic Scaling parameter in the proximal Bundle Method. For the level method we should pick around 1 10 th of that value. The initial proximal parameter should be set up according to what has been explained in section 4.2.4.2. According to our experience the quadratic term xxk 2 should not exceed 1e 11 and µ 0 can be set accordingly.

Chapter 5

Applications of JCCP in large scale Unit-Commitment through decomposition

In Energy Management, a key problem known as "unit-commitment" deals with finding a minimal cost production schedule. This schedule has to satisfy the operational constraints of each of the production units and meet customer load as closely as possible. Depending on the detail with which the operational constraints are modelled, many variants of this problem can be set up. Since operational constraints involve delays (start-up delays, etc...), the computed production schedule is often determined quite ahead of real-time. This allows for uncertainty to have a key impact on the "feasibility" and "optimality" of the executed production schedule. In practice, spinning reserves and intra-daily changes to the schedule allow the operator to partially account for uncertainty. Highly binding operational constraints might give rise to difficult situations, wherein the quest for "feasibility" induces a heavy cost. As such, computing a schedule having seen at least part of the plausible uncertainty, might turn out less costly eventually. This would be a key goal of Robust Unit-Commitment. However, unit-commitment problems are already very challenging in a deterministic setting. This is the consequence of the following facts:

1. each unit is subject to many complex technical constraints.

2. units exist in many varieties (thermal, hydraulic, contracts). Each type comes with a very specific set of constraints. Most of them requiring specific techniques for an efficient resolution.

the offer-demand equilibrium constraints couple all the units together

In order to tackle these large scale problems, the coupling constraints are often dualized, using Lagrangian techniques, leading to an effective price decomposition scheme ( [START_REF] Cohen | Decomposition-coordination methods in large-scale optimization problems. the non-differentiable case and the use of augmented Lagrangians[END_REF][START_REF] Lemaréchal | An approach to variable metric bundle methods[END_REF][START_REF] Frangioni | Sequential Lagrangian-MILP Approaches for Unit Commitment Problems[END_REF][START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF]). Since the global unit-commitment problem is already challenging to solve in a deterministic setting due to its non-convex feasible sets and large scale, uncertainty is often neglected, even though decisions are taken at least one day in advance. Uncertainty in unit-commitment problems comes at least from the following sources: customer load, renewable generation, inflows, unit availability. Integrating uncertainty in global unitcommitment will be quite challenging for the reasons outlined above. Hence, as a first necessary step, we will focus on hydro valley optimization. This will be investigated in section 5.1. In section 5.2 we will consider a global robust unit-commitment problem.

Robust Cascaded Reservoir Management

In the Lagrangian dualization setting of a unit-commitment problem, hydro valley optimization can be seen as a sub-problem. Alternatively, one can interpret this sub-problem as an optimization against market-prices. Complex dynamic constraints on watershed controls introduce combinatorial aspects in this sub-problem, making it difficult to solve. The focus of this section will therefore be on integrating uncertainty in hydro valley management.

The aforementioned combinatorial aspects result from formulating smoothness requests on watershed. From an engineering perspective it is undesirable to have turbining output increase and decrease rapidly over short time spans as this induces a strain on material. Other combinatorial elements can arise when modelling very realistic efficiency curves. We refer to [START_REF] Diniz | A four-dimensional model of hydro generation for the short-term hydrothermal dispatch problem considering head and spillage effects[END_REF] for an approach to deal (i.e., remove) the latter combinatorial elements.

In hydro dominated systems, such as in Brazil, Scandinavia and Canada, the emphasis of accurate modelling lies on hydro generation and combinatorial optimization is common for cascaded reservoir management. We refer to [10,[START_REF] Finardi | Solving the hydro unit commitment problem via dual decomposition and sequential quadratic programming[END_REF][START_REF] Ponrajah | Systems to optimise conversion efficiencies at ontario hydro's hydroelectric plants[END_REF][START_REF] Nilsson | Variable splitting applied to modelling of start-up costs in short term hydro generation scheduling[END_REF] for more details on such models. In thermal dominated systems, such as the French system, the modelling emphasis lies more on thermal generation. In these large-scale unit-commitment problems, such additional combinatorial elements are often neglected in order to have an acceptable computional burden (see [START_REF] Dubost | A primal-proximal heuristic applied to french unitcommitment problem[END_REF]). We will make the same assumption. Integrating uncertainty and combinatorial elements in a cascaded reservoir model is quite challenging and will be investigated in future work. A potential entry point for such an approach would be the decomposition idea investigated in [START_REF] Finardi | Solving the hydro unit commitment problem via dual decomposition and sequential quadratic programming[END_REF].

Uncertainty in cascaded reservoir management results from uncertainty on inflows and impacts the physical constraints of the system. Since decisions are taken prior to the observation of uncertainty, appropriate modelling approaches for integrating uncertainty have to be considered. The two main approaches are chance constrained programming and robust optimization. The main focus here is on the former, for the latter we refer to [START_REF] Ben-Tal | Robust Optimization[END_REF]7,[START_REF] Apparigliato | Règles de décision pour la gestion du risque: Application á la gestion hebdomadaire de la production électrique[END_REF]. In dynamic decision processes, i.e., when decisions in later time periods are allowed to adapt to earlier observed uncertainty, the main approaches are Stochastic Dynamic Programming and SDDP (see [START_REF] Pereira | Multi-stage stochastic optimization applied to energy planning[END_REF]). Often the convenient hypothesis is made that uncertainty within the transition problem is known. This essentially makes the transition problem a deterministic problem. The latter choice is especially questionable when the transition problem covers a time span of a week such as assumed in [START_REF] Philpott | On cutting plane algorithms and dynamic programming for hydroelectricity generation[END_REF].

Introduced by [START_REF] Charnes | Chance-constrained programming[END_REF], probability constraints are quite an appealing tool for dealing with uncertainty. In particular, when uncertainty arises in physical constraints, since they also offer a simple interpretation. Since their first introduction, chance constraints have become quite common in hydro valley management ([143, 63, 65, 142, 155, 265, 264, 244]), but often individual chance constraints are used and not joint chance constraints. Though a very appealing approximation, individual chance constraints unfortunately do not offer sufficient robustness (see [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF]). Hydro reservoir models with joint chance constraints have been considered, for instance, in [START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Prékopa | A new multivariate gamma distribution and its fitting to empirical streamflow data[END_REF][START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF]. In [1] even a dynamic approach has been developed in this context. However, these models were comparatively simple from their structure (no serially linked reservoirs, no delay time between reservoirs, no realistic water value condition, no time series modelling of statistical data, small dimension). The main focus of [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF] is on deriving an efficient gradient formula for joint chance constraints of a specific form. The latter form arises naturally in hydro reservoir management. The interest of the formula is then illustrated on a stylized hydro reservoir optimization problem.

This section is organized as follows. In section 5.1.1, we present our model for hydro reservoir management, where combinatorial constraints are neglected and random inflows are introduced. We give a detailed description of a real hydro valley, and present the main optimization problem. As the uncertainty on inflows is concerned, many statistical models are based on a deterministic trend (potentially dependent on explanatory variables) and a causal noise process. Since convexity results exist for specific classes of randomness and in particular Gaussian ones, it seems tempting to place ourselves in such a setting. Restricting uncertainty laws to such a setting, might seem restrictive at first. However, we will show that a large class of models is available, i.e., the class of causal time series models with Gaussian innovations ( [START_REF] Shumway | Time Series Analysis and Its Applications[END_REF]).

In section 5.1.2, we present several alternatives to the joint chance constraint problem.

In particular, we present an individual chance constraint problem and a robust model. We report results obtained when solving these various models on a realistic instance of a hydro valley management problem in section 5.1.3. The interest of joint chance constrained programming is illustrated by comparing results obtained on this hydro valley with those obtained from the alternative models.

Problem Description

In this section we will give a description of the hydro reservoir management problem. We will consider a discretized time horizon. To this end let τ = {1, ..., T } denote the set of (homogeneous) time steps, where T denotes the last time step. Let ∆t be the time step size expressed in hours. We will begin by providing problem constraints and the objective function. We will conclude with a paragraph highlighting the structure of the problem.

Problem Constraints

5.1.1.1.1 Topology A hydro valley can be seen as a set of connected reservoirs and associated turbines. We can therefore represent this with a directed graph. Let N be the set of nodes and let A (of size |N |×|N |) be the connection matrix, i.e., A n,m = 1 whenever water released from reservoir n will flow into reservoir m. We will assume that D is the flow duration vector, i.e., D m is the amount of time (measured in time steps) it takes for water to flow from upper reservoir m to its unique child. It is assumed that pumping is (nearly) instantaneous. Let T := {g i , i = 1, ..., N T } denote the set of turbines and P := {p i , i = 1, ..., N P } denote the set of pumping stations. We furthermore introduce the mapping σ T : {1, ..., N T } → N (σ P : {1, ..., N P } → N ) attributing to each turbine (pumping station) the reservoir number to which it belongs. We will also introduce the sets A(n) = {m ∈ N : A m,n = 1} and F(n) = {m ∈ N : A n,m = 1}. The set A(n) is empty for uphill reservoirs and the set F(n) for downhill reservoirs. To each reservoir n ∈ N and for each time step t ∈ τ we associate its volume V n (t) in cubic hectometers hm 3 . The initial volume of each reservoir n ∈ N is denoted by V n (0), lower and upper bounds are V n min (t) and V n max (t) respectively.

Controls

We will assume that each turbine (and pumping station) can be controlled for each time step. To this end we introduce the variables x i (t) for each t ∈ τ and i = 1, ..., N T . In a similar way we introduce the variables y i (t) for the pumping stations. The units are in cubic meters per hour, i.e., m 3 /h. Furthermore we assume that each of these variables are bounded: 0 ≤ x i (t) ≤ x i (t), ∀t ∈ τ, i = 1, ..., N T (5.1.1) 0 ≤ y i (t) ≤ y i (t), ∀t ∈ τ, i = 1, ..., N P .

Random Inflows

We will assume that inflows (in m 3 /h) in reservoirs are the result of some stochastic process. Let A n (t) denote this stochastic process for reservoir n. Not all reservoirs will have stochastic inflows, some of them will have deterministic inflows. This can be explained by the fact that top reservoirs have random inflows due to the melting of snow in the high mountains, whereas rain can be neglected for lower reservoirs. Let N r ⊆ N denote the set of reservoirs receiving random inflows. We will assume that the stochastic inflow process is the sum of a deterministic trend s n t and a causal process ( [START_REF] Shumway | Time Series Analysis and Its Applications[END_REF]) generated by Gaussian innovations. To this end, let ζ n (t) be a Gaussian white noise process, where (ζ k 1 (t), ..., ζ k l (t)) is a Gaussian random vector of zero average and variance-covariance matrix Σ(t) ({k 1 , ..., k l } = N r ). This last assumption means that the innovations for several reservoirs are correlated. This correlation alone will bring forth the correlation structure on inflows across reservoirs. This assumption is quite similar to the one made in mathematical finance wherein Brownian motions are correlated and thus generate correlated commodities. We will assume independence between time steps of the ζ vector. Since A n (t) is a causal process, we can write it as follows

A n (t) = s n t + ∞ j=0 ψ n j ζ n (t -j) = s n t + ∞ j=t ψ n j ζ n (t -j) + t-1 j=0 ψ n j ζ n (t -j), ∀n ∈ N r , t ∈ τ
for some coefficient vector ψ n and infinite past before t = 0 (the beginning of the optimization horizon). We will assume that randomness before (and including) t = 0 is known and as such we can assume w.l.o.g. that the random inflow process can be written as

A n (t) = s n t + t-1 j=0 ψ n j ζ n (t -j), ∀n ∈ N r , t ∈ τ. (5.1.2)
For reservoirs n ∈ N \ N r , we simply have A n (t) = s n t .

5.1.1.1.4 Flow constraints and Volume bounds Each reservoir is subject to flow constraints induced by pumping and turbining. The following balance constraint applies

V n (t) = V n (t -1) + m∈A(n) i∈σ -1 T [m] x i (t -D m )∆t - i∈σ -1 T [n]
x i (t)∆t

+ m∈F (n) i∈σ -1 P [m] y i (t)∆t - i∈σ -1 P [n]
y i (t)∆t (5.1.3)

+ s n t ∆t + t-1 j=0 ψ n j ζ n (t -j)∆t, ∀t ∈ τ, n ∈ N .
The above equation is entirely deterministic except for the reservoirs n ∈ N r . In order to deal with this randomness and reservoir bounds we will therefore add the following constraints

P[V n min (t) ≤ V n (t) ≤ V n max (t) ∀t ∈ τ, n ∈ N r ] ≥ p (5.1.4) V n min (t) ≤ V n (t) ≤ V n max (t) ∀t ∈ τ, n ∈ N \ N r , (5.1.5)
where P is a probability measure and p a security level. Constraint (5.1.4) is a joint chance constraint. This means that we wish to satisfy all linear inequalities of the stochastic system simultaneously with high enough probability. This can be compared to a model with individual chance constraints, which is a model wherein we wish to satisfy each inequality with high enough probability, but taken separately. We will show (see section 5.1.3) that the latter model offers insufficient robustness.

Water Values

In short term optimization problems (with time horizons ranging from several days up to a month) water values provide a way to associate a cost with used water. Incorporating no such cost in a short term optimization problem would inevitably lead to a maximum use of water on this specific time horizon, whereas water might be needed in later time periods. Water might be used to reduce the use of costly thermal generation or as a security to avoid "black-outs" in difficult situations. Water values are obtained as the by product of (stochastic) dynamic programming approaches in mid term (time horizons ranging from 1 to 5 years).

In full generality water values depend on time, a multivariate random vector, the current water levels in all reservoirs and other quantities that can be considered as inventories or stocks (such as customer interruption options (see [START_REF] Zorgati | Optimizing financial and physical assets with chance-constrained programming in the electrical industry[END_REF] for more details), i.e., an inventory globally very similar to the number of remaining exercise rights in swing options).

As the effect of uncertainty is concerned, it is often averaged out on a set of reasonable scenarios in order to integrate unconditional water values in short term optimization.

The stochastic dynamic programming algorithms typically deal with uncertainty effects rarely integrated in short term optimization such as stochastic fuel prices.

The multivariate stock dependency is only known approximately, if at all, since one quickly hits the curse of dimensionality of dynamic programming. In such cases, approaches such as approximate dynamic programming (ADP) ( [START_REF] De Farias | Approximate dynamic programming via linear programming[END_REF]), approximate dual dynamic programming (ADDP) ( [START_REF] Girardeau | Résolution de grands problèmes en optimisation stochastique dynamique et synthèse de lois de commande[END_REF]), SDDP ( [START_REF] Pereira | Multi-stage stochastic optimization applied to energy planning[END_REF][START_REF] Philpott | On the convergence of stochastic dual dynamic programming and related methods[END_REF]) or aggregation approaches ( [START_REF] Turgeon | Optimal operation of multi-reservoir power systems with stochastic inflows[END_REF][START_REF] Torrion | Comparaison de différentes méthodes d'optimisation appliquées à la gestion annuelle du système offre-demande d'électricité français[END_REF]) are applied in order to approximately solve the dynamic programming problem. In the ADP approach, it is commonly assumed that the continuation function of dynamic programming decomposes as a sum of 1 dimensional functions. Each function depending on a unique stock only. This then automatically results in single stock dependent water values. Even if water values would be available as multivariate functions, they would only be known on a set of grid points. If this is to be incorporated in short term optimization one surely needs interpolation techniques very similar to those explained in [START_REF] Ambrosio | Piecewise linear approxmation of functions of two variables in milp models[END_REF]. This interpolation approach leads to the introduction of binary variables in the optimization problem. Since multivariate effects in water values are only rarely known and integrating them induces combinatorial aspects, we will focus on single stock dependent water values.

As the temporal dependency is concerned it is often daily or intra-daily. Due to the average effect of climate on unit-commitment, some specific weeks are far more costly than surrounding weeks. Such weeks have peaking customer load and high risk of black outs. Such effects get reflected in the water values as well. These effects are moreover strengthened by averaging out stochastically dependent water values as explained above.

If we wish to incorporate water values in short term optimization, the latter temporal effect can either be neglected or taken into account. In the first case, we would value the differential between the end and the initial volume of a reservoir against water values at that time step. In the second approach we would either value volumes against water values at each time step or value local volumetric differences. The first approach would consider indifferently any two storage paths leading to the same end volume. When the short term time horizon is close to a month and one of the above difficult weeks is within this time horizon, from an operational view point two paths leading to the same end volume are not necessarily considered equivalent. It is therefore of interest to integrate the temporal dependence in order to reflect this feature. A second reason for integrating this effect is to provide a model that fits better with current practice. In practice, in order to control the storage path, a selection of time steps τ s ⊆ τ is made where artificially we force V n min (t) ≈ V n max (t) for t ∈ τ s . Integrating the temporal dependency of water values in short term optimization is a natural way to have control over the storage path without risking to have an empty feasible set.

In this section we present a model for incorporating water values without reflecting temporal dependencies as the focus of the section is on Chance Constrained programming for hydro reservoir management. Upon valuing the volume at each time step against local water values, the presented model allows for a straightforward extension for incorporating the above discussed temporal effect of water values. We also discuss how valuing volumetric differences leads to a non-convex model.

5.1.1.1.5.1 Volume dependent water values Our aim is to set up a model which evaluates the expected amount of water in the reservoir at the end of the optimization horizon 1 . This is necessary in order not to carry out the optimization at the expense of later periods of time. A possible way to do so is to subdivide the levels of each reservoir into a finite number of values from bottom to top as follows:

V n 0 , . . . , V n K n ∀n ∈ N . Each compartment V n i-1 , V n i is assigned a water value W n i (in e/m 3 ) such that W n i-1 > W n i ≥ 0 ∀n ∈ N ∀i = 1, . . . , K n . (5.1.6)
The value of the expected final water level E (V n (T )) of reservoir n is then simply the cumulative value of water in the compartments below:

i≤i * W n i (V n i -V n i-1 ) + W n i * (E (V n (T )) -V n i * ), i * := max{i| E (V n (T )) ≥ V n i }.
Note that this value is an increasing function of the expected final level EV n (T ) despite the fact that water values are strictly decreasing from bottom to top. Now, in order to avoid combinatorial arguments concerning the index i * , we introduce auxiliary variables z n i indicating for each reservoir n the amount of water in compartment

V n i-1 , V n i .
Of course, since all compartments have to be completely filled up to i * , one has that

z n i =    V n i -V n i-1 i = 1, . . . , i * E (V n (T )) -V n i * i = i * + 1 0 i = i * + 2 . . . , K n ∀n. (5.1.7)
Then, the value of the final water level in reservoir n equals

K n i=1 W n i z n i
∀n.

(5.1.8)

We claim that the relations (5.1.7) for variables z n i can be replaced by the following relations in which the crucial index i * is absent:

K n i=1 z n i = E (V n (T )) -V n 0 ∀n (5.1.9) 0 ≤ z n i ≤ V n i -V n i-1 ∀n∀i = 1, . . . , K n .
(5.1.10)

The argument is as follows: as part of the overall objective function in our problem, we shall maximize the value of the final water level (5.1.8). Given the strictly decreasing order of water levels in (5.1.6) (from bottom to top), it is clear from (5.1.9) that the upper inequality in (5.1.10) will be satisfied as an equality as long as possible and that only the most upper compartment may not be completely filled. This of course is equivalent with (5.1.7) but avoiding the explicit description of that most upper compartment.

Since the initial volume V n (0) is known in advance, one can define variables z n 0,i in a similar way as z n i . It then follows that

K n i=1 W n i (z n 0,i -z n i ) (5.1.11)
is the cost of used water for reservoir n ∈ N . The valuation induced by K n i=1 W n i z n 0,i is in fact a constant and can be omitted. 5.1.1.1.5.2 Time and Volume dependent water values : Volumetric differences Our aim is to set up a model in which (expected) volumetric difference of adjacent time steps are valued with a water value that depends on time and volume. We will show that such a model is non-convex. We therefore suggest to extend the model of paragraph 5.1.1.1.5.1 instead. We begin by recalling the subdivision of the levels of each reservoir in (5.1.1.1.5.1). To each of these compartments we assign a water value W i (t) n (in e/m 3 ), i = 1, ..., K n , n ∈ N . We assume that these are decreasing:

W n i-1 (t) > W n i (t) ≥ 0 ∀i = 1, ..., K n , n ∈ N , t ∈ τ.
(5.1.12)

Assuming for simplicity of reasoning that the dependence on the compartments is absent.

Then we wish to value

n∈N t∈τ W n (t) E (V n (t) -V n (t -1 )) . (5.1.13)
For a fixed reservoir n ∈ N we define i * n,t = max {i E (V n (t)) ≥ V n i }. Let i = 1, ..., K n moreover be arbitrary and define z n

x,i (t) as negative variations of volume with respect to expected volume E (V n (t)) (i.e., turbining). Similarly, we define z n y,i (t) as the positive variations (i.e., pumping). Both variations are restricted to the interval [V n i-1 , V n i ). This means that:

V n (t + 1) = V n (t) + K n i=1 (z n y,i (t) -z n x,i (t)).
Since z n x,i (t) is defined as the negative variations with respect to the expected volume E (V n (t)), one has z n

x,i (t) = 0 for all i > i * n,t + 1. By similar arguments we readily observe that z n y,i (t) = 0 for all i < i * n,t -1. Moreover, one has:

z n x,j (t) > 0 ⇒ z n x,i (t) = min {V n i , E (V n (t))} -V n i-1 , i = j + 1, ..., i * n,t + 1 (5.1.14) z n y,j (t) > 0 ⇒ z n y,i (t) = V n i -max V n i-1 , E (V n (t)) , i = i * n,t -1, ..., j -1. (5.1.15)
It therefore follows that the appropriate generalization of (5.1.13) accounting for the compartments is:

n∈N t∈τ K n i=1 W n i (t)(z n x,i (t) -z n y,i (t)).
(5.1.16)

The following constraints:

0 ≤ z n x,i (t) ≤ max E (V n (t)) -V n i-1 , 0 , i = 1, ..., K n , n ∈ N (5.1.17) 0 ≤ z n y,i (t) ≤ max {V n i -E (V n (t)), 0} , i = 1, ..., K n , n ∈ N (5.1.18)
assure that z n x,i (t) = 0 for all i > i * n,t + 1 and z n y,i (t) = 0 for all i < i * n,t -1. One can also see that the requirements (5.1.14) and (5.1.15) are automatically satisfied by hypothesis (5.1.12) and objective function (5.1.21). The above non-linear constraints (5.1.17 If moreover, both g and h are linear, the complementarity constraint appearing in (5.1.20) is quadratic. In the particular case, when g takes the form g(x, z) = z and h(x, z) = a T x, x ∈ R n , z ∈ R, the matrix defining the quadratic form of the inequality is given by

Q =   0 -0.5a 0 -0.5a T 1 0.5 0 0.5 0  
and has n zero eigenvalues and 2 non-zero ones, 0.5 ± 0.5 a 2 + 2. The resulting quadratic constraint is non-convex.

Proof. For each feasible x for problem (5.1.19), one can find some λ ≥ 0 such that (x, λ) is feasible for (5.1.20). This can be seen by case distinction on the sign of h(x), g(x). By case distinction on the sign of g(x) one can see that each feasible (x, λ) for problem Let us consider equation (5.1.3) and apply it recursively to establish the identity

V n (t) = V n (0) + t u=1 m∈A(n) i∈σ -1 T [m] x i (u -D m )∆t - t u=1 i∈σ -1 T [n]
x i (u)∆t

+ t u=1 m∈F (n) i∈σ -1 P [m] y i (u)∆t - t u=1 i∈σ -1 P [n]
y i (u)∆t (5.1.22)

+ t u=1 s n u ∆t + t u=1 u-1 j=0 ψ n j ζ n (u -j)∆t,
holding for all t ∈ τ and n ∈ N . In what follows we will denote with V n ∈ R T the vector V n = (V n (1), ..., V n (T )). It is of interest to explicitly establish the way in which V n depends on the vector ζ n in order to identify the correlation structure of the global underlying uncertainty vector. One easily observes that V n depends linearly on x and y.

In order to establish the correlation structure of the vector ζ, we introduce the matrix mapping C : R T → M T ×T . Here M T ×T stands for the set of T × T real matrices and C as applied to the sequence ψ := (ψ 0 , ..., ψ T -1 ) ∈ R T is defined as:

C(ψ) =      ψ 0 0 0 • • • 0 ψ 0 + ψ 1 ψ 0 0 • • • 0 . . . . . . . . . T -1 j=0 ψ j • • • • • • ψ 0      .
It will be convenient to extend the definition of C to a sequence ψ shorter than T by appending with zero entries.

Following equation (5.1.22) for each n ∈ N we can find a T × T N T matrix M n T and T × T N P matrix M n P such that

V n = V 0 -∆tM n T x + M n P y + ∆tC(1)s n + ∆tC(ψ n )ζ n , (5.1.23)
where s n is the vector formed from the deterministic trend s n t of equation (5.1.2). Equations (5.1.9), (5.1.10) can be written easily in linear form by extracting the last line from equation (5.1.23) without the term in ζ n .

Models for dealing with uncertainty

In this section we will provide our main model, which is a joint chance constrained programming problem (JCCP). We will also provide several alternative models.

Expectation model

In a classic version of cascaded reservoir management in short term optimization, uncertainty is assumed to be absent or sufficiently characterized by a forecast. This amounts to the choice of replacing ζ n in equation (5.1.23) or equivalently (5.1.2) by its expectation, i.e., ζ n (t) = E (ζ n (t)) = 0 ∀t ∈ τ . This substitution in turn impacts equation (5.1.4).

When combining equations (5.1.9), (5.1.23) and relations (5.1.10), (5.1.5) we know that we can find some extended decision vector (also noted x ∈ R n ) containing (x, y, z) and some matrix A, vector b such that the system Ax ≤ b models all the deterministic constraints (including bounds on x) found in section 5.1.1. One can moreover find a matrix A r and vectors a r , b r such that equation (5.1.4) wherein we have substituted the expectation of ζ for ζ is reflected by a

r + A r x ≤ 0 ≤ A r x + b r .
Combined, this gives the following linear program: .1.24) This model can be identified with the model considered in a classical deterministic unitcommitment setting.

min x∈R n ,x≥0 c T x s.t. Ax ≤ b -A r x ≤ b r A r x ≤ -a r . ( 5

A Joint Chance Constraint Model (JCCP)

In contrast to the expectation model wherein the effect of uncertainty is neglected, incorporating uncertainty fully in equation (5.1.4) leads to a joint chance constrained program. Indeed, by combining equation (5.1.23) with (5.1.4), we can see that the problem of section 5.1.1 can be cast into the following form, where ξ ∈ R m is a Gaussian random vector with variance-covariance matrix Σ and zero mean (we have explicitly extracted the non-zero average in eq.(5.1.23)):

min x∈R n ,x≥0 c T x s.t. Ax ≤ b p ≤ P[a r + A r x ≤ ξ ≤ b r + A r x]. (5.1.25)
In fact the feasible set of (5.1.25) is convex due to the Gaussian character of ξ ∈ R m and a theorem by Prékopa ([181]). This makes the previous optimization problem a convex one. For convenience we define ϕ :

R n → [0, 1] as ϕ(x) = P[a r + A r x ≤ ξ ≤ b r + A r x].

Link with the Expectation Problem

The chance constrained model can be seen as an extension of the expectation model model since it takes into account the available stochastic information on the distribution of randomness, whereas model (5.1.24) only uses a single parameter. The following Lemma shows that any feasible solution of (5.1.25) is feasible for (5.1.24). Physically this can be explained by the fact that a "robust" control has to work well in the average situation.

Lemma 5.1.2. Assume that p > 0.5 and that ξ ∈ R m is a symmetric random variable, i.e., P[ξ ∈ A] = P[ξ ∈ -A] for any measurable set A ⊆ R m . The feasible set of (5.1.25) is contained in the feasible set of (5.1.24). As a consequence the optimal value of (5.1.24) is lower than that of (5.1.25).

Proof. Assume that x ∈ R n is not feasible for (5.1.24), for instance not a r + A r x ≤ 0, i.e., there is at least one strictly positive component. By rearranging we may assume that this is the first one. Now

P[a r + A r x ≤ ξ ≤ b r + A r x] ≤ P[a r + A r x ≤ ξ] ≤ P[e T 1 (a r + A r x) ≤ e T 1 ξ] ≤ P[0 < ξ 1 ] < 0.5
, where ξ 1 is (also) a centered one dimensional symmetric random variable, and e 1 is a standard unit-vector of R m . This shows that x can't be feasible for (5.1.25).

As mentioned the expectation model is a simple linear program. It is therefore much easier to solve than problem (5.1.25). Despite this fact and the fact that it yields solutions with low optimal values, it will be shown later that the solutions are useless since they violate constraints almost surely.

5.1.2.2.2 An algorithm for solving JCCP In order to solve problem (5.1.25) we will use the supporting hyperplane method. This method was originally introduced by [START_REF] Veinott | The supporting hyperplane method for unimodal programming[END_REF] and adapted to the context of joint chance constrained programming by [START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF][START_REF] Szántai | A computer code for solution of probabilistic-constrained stochastic programming problems[END_REF]. This algorithm converges in a finite number of steps as shown in [START_REF] Prékopa | Flood control reservoir system design using stochastic programming[END_REF]. We repeat the algorithm for completeness.

1. (Initialization) Let x 0 be the solution of (5.1.24), x s a Slater point for (5.1.25). Set A 0 = A, b 0 = b and k = 0 and pick some tolerance tol, e.g., tol = 10 -2 . Let ε > 0 be a tolerance on the evaluation of ϕ.

(Interpolation

) Find λ * such that x * k = (1 -λ * )x k + λ * x s and p -ε ≤ ϕ(x * k ) ≤ p. 3. (Add Cut) Add constraint -∇ϕ(x * k ) T x ≤ -∇ϕ(x * k ) T x * k to the matrix system A k x ≤ b k . 4. (Solve LP) Solve min x∈R n ,x≥0 c T x s.t. A k x ≤ b k to find x k+1 . 5. (Stopping Test) If c T (x * k -x k+1 ) c T x k+1 < tol then stop, x *
k is sufficiently optimal, else set k = k + 1 and go to step 2.

For the previous algorithm to function we require a Slater point, i.e., some x s such that Ax s ≤ b, and ϕ(x s ) > p. It can be obtained by solving the "max-p" problem (see section 5.1.2.3). Moreover, we should be able to efficiently evaluate ϕ and ∇ϕ. As shown in Lemma 3.2.1 and Corollary 3.2.3, evaluating the gradient can be analytically reduced to computing function values in smaller dimension. Finally computing function values such as ϕ(x) can be done by using the code of Genz ([83]). Evaluating ϕ and ∇ϕ requires 2n + 1 calls to Genz' code.

Max-P Problem

We define the "max-p" problem as the following optimization problem:

max x∈R n ,x≥0 ϕ(x) := P[a r + A r x ≤ ξ ≤ b r + A r x]
Ax ≤ b.

(5.1.26)

Clearly any solution x s of the previous problem with objective function value strictly bigger than p is a Slater point for problem (5.1.25). This "max-p" problem is not only an auxiliary problem for obtaining Slater points, but can also be interpreted as the problem of a decision-maker looking for maximum robustness, regardless of the costs. As a matter of fact if the optimal solution of (5.1.26) is strictly below one, then almost surely satisfying the "random" physical constraints (5.1.4) is not possible. The "max-p" problem therefore also provides us with information on the maximum robustness level p that is "possible".

Individual Chance Constraint Model (ICCP)

We consider a simplification of the joint chance constrained model ( 5

min x∈R n ,x≥0 c T x s.t. Ax ≤ b P[e T i (a r + A r x) ≤ ξ i ] ≥ p ∀i = 1, ..., m P[ξ i ≤ e T i (b r + A r x)] ≥ p ∀i = 1, ..., m, (5.1.27)
where e i ∈ R m is the i-th standard unit vector.

As a matter of fact, model (5.1.27) can be reduced to a simple linear program since the inverse of F ξ i (z) = P[ξ i ≤ z] can be evaluated easily. It also offers improved robustness with respect to the expectation model (5.1.24) that offered none. However it can't guarantee a probability level of p for the whole stochastic inequality system and therefore offers far less robustness than the joint model (5.1.25) ( [START_REF] Van Ackooij | On probabilistic constraints induced by rectangular sets and multivariate normal distributions[END_REF]). This will become apparent in the numerical experience.

A Robust Model

We would like to identify an uncertainty set E p ⊆ R m for our random inflow process ξ ∈ R m in such a way that the probability of ξ falling in this set is approximately p. We will then enforce the constraints of problem (5.1.25) to hold for all ξ in this set rather than in probability. We will use a specific ellipsoidal form for the uncertainty set and show that the thus obtained robust optimization problem then boils down to a linear program, once two conic quadratic problems have been solved (see [START_REF] Lobo | Applications of second-order cone programming[END_REF] and references therein).

In order to determine E p , let LL T = Σ be the Cholesky decomposition of Σ. Let y ∈ R m be defined as y = L -1 ξ and assume that we dispose of a statistical estimate of E y 4 i (in the Gaussian case these are known exactly) for i = 1, ..., m. Whenever the law of ξ is unknown, we can use the variance covariance matrix Σ obtained from statistic estimates. By construction, y is uncorrelated, we will make the (wrong) approximation that this is the same as independence. Now by the Lindeberg-Feller Central Limit Theorem ( [START_REF]Limit Theorems of Probability Theory[END_REF]) we obtain that y T y is approximately normally distributed with mean m and standard

deviation σ C , i.e., y T y ≈ N (m, σ C ), with σ C = m i=1 E y 4 i -m. We now define E p = z ∈ R m : z T Σ -1 z ≤ m + Φ -1 (p)σ C .
It follows in the case that ξ follows a multivariate Gaussian law that P[ξ ∈ E p ] = p. This will be true approximately when ξ follows another multivariate law. This is a very elementary way of deriving an uncertainty set. We refer to [START_REF] Bandi | Tractable stochastic analysis in high dimensions via robust optimization[END_REF][START_REF] Ben-Salem | Gestion Robuste de la production électrique à horizon court-terme[END_REF] for more sophisticated approaches.

We therefore consider the following robust version of problem (5.1.25):

min x∈R n ,x≥0 c T x s.t. Ax ≤ b a r + A r x ≤ inf E p b r + A r x ≥ sup E p , (5.1.28)
where inf E p ∈ R m denotes the vector whose components are the coordinate-wise minima of E p (sup E p ∈ R m is defined similarly). Both inf E p and sup E p are solutions of a conic quadratic optimization problem. Indeed model (5.1.28) is equivalent with

min x∈R n ,x≥0 c T x s.t. Ax ≤ b a r + A r x ≤ ξ ≤ b r + A r x ∀ξ ∈ E p .
Since model (5.1.28) basically looks at the smallest rectangle containing E p and requires satisfaction of constraints for all elements in the rectangle, one could also look at alternative ways to obtain such a rectangle. Basically, we are looking for some ξ and ξ such that P[ξ ≤ ξ ≤ ξ] ≈ p. These would then give better bounds than inf E p and sup E p as above, since in general

P[inf E p ≤ ξ ≤ sup E p ] > p.
In the Gaussian case considered here we can exactly evaluate the probabilistic contents of such rectangles and hence fine-tune the rectangle. Clearly any feasible solution of problem (5.1.25) will also provide such vectors. This last way of obtaining those vectors offers no computational advantage to (5.1.28) other than prematurely ending the algorithm that solves (5.1.25). An alternative would be to take some q < p, such that P[inf E q ≤ ξ ≤ sup E q ] ≈ p. This is computationally not intensive, but requires evaluations of probabilistic contents. In order to investigate the impact of the choice of this rectangle we have made some runs with model (5.1.28) wherein the rectangle was made to fit perfectly. In practice, we have obtained ξ and ξ by taking some ad-hoc convex combinations between the Slater point and the solution of (5.1.24). These results will be referred to as Robust-Calibrated (Robust-Calib) or (5.1.28)-Calib.

Numerical Example

In this section we consider a numerical example from the industry. The instance size is moderate but realistic. The nominal inflows, i.e., s n t in equation (5.1.2), are considered constant through time. Finally, the water values are not assumed to depend on the volume, and thereby correspond to the V 0 level. It was shown in section 5.1.1.1.5 that adding the volume dependency induces no substantial difficulties. The focus of this numerical example is the impact of uncertainty. We will consider 24 time steps of 2 hours each. Figures 4.1(c) and 4.1(a) show further data of our example. This implies the following dimensions for our problem: the Gaussian vector dimension m is 48, the decision vector has 700 elements and the polyhedral constraints are defined by about 1000 linear inequalities.

The system topology, that is the relation between the six power reservoirs in the valley, is expressed by the reservoir connection matrix

A =         0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0         .
Likewise, the vector σ T = (1, 1, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 5, 6, 6, 6) associates each one of the 16 turbines in the valley to a specific reservoir.

As uncertainty is concerned we will assume that reservoirs 1 ("Vouglans") and 2 ("Saut Mortier") have random inflows. The standard deviations of the innovations ζ of the inflow process A n in equation (5.1.2) are taken to be equal to 20% of the nominal inflow values (0.3 m 3 /s for reservoir 2). We will consider two instances, one wherein inflows on both reservoirs follow an AR(1) process with coefficient 0.9. A second instance is one wherein we assume that inflows on reservoir 2 follow an AR(3) process with coefficients (0.9, 0.7, -0.7). In this instance inflows on reservoir 1 still follow an AR(1) process. The required probability level p in (5.1.25) is taken to be 0.8.

To express the hydro valley relations in the presence of uncertainty, we need to write down the variance-covariance matrix of the stochastic process. This needs the introduction of some notation. Specifically, we let ψ be a 2 × 24 matrix such that ψ 1,j = 0.9 j and ψ 2,j = 0.9ψ 2,j-1 + 0.7ψ 2,j-2 -0.7ψ 2,j-3 , j = 1, . . . , 24, where in the last formula any negative indexed terms are assumed 0. We also define the elementary covariance matrix Σ a : Σ a = 4.24 0 0 0.

Robust Unit-Commitment

We begin by recalling that deterministic unit-commitment problems are often decomposed using Lagrangian techniques. Since many unit-commitment problems involve nonconvex modelling features, such a Lagrangian decomposition scheme often leads to a non-feasible primary solution. This obtained schedule then needs to be fixed. This can be done by applying heuristics derived from Augmented Lagrangian ( [START_REF] Beltran | Unit commitment by augmented lagrangian relaxation: Testing two decomposition approaches[END_REF][START_REF] Wang | Short-term generation scheduling with transmission and environmental constraints using an augmented lagrangian relaxation[END_REF]) formulations or other techniques (see [START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF] for some suggestions or [START_REF] Dubost | A primal-proximal heuristic applied to french unitcommitment problem[END_REF][START_REF] Frangioni | Solving Unit Commitment Problems with General Ramp Contraints[END_REF]). We wish to investigate the effects of using chance constraints for robustifying the production schedule in a unit-commitment framework. As a first step we wish to avoid the complex issues raised by using heuristics as explained above. As a numerical example, we will therefore consider a variant of Unit-Commitment wherein thermal units have a simple (convex) representation. The investigated approaches can clearly be applied unchanged to nonconvex Unit-Commitment problems, but the above discussed heuristics might be needed. This will be the object of future work.

Two difficult sub-problems are those related to the thermal generation units ( [START_REF] Frangioni | Solving non-linear single-unit commitment problems with ramping constraints[END_REF]) and the hydraulic valleys (see Section 5.1 above or (for instance) [START_REF] Dubost | A primal-proximal heuristic applied to french unitcommitment problem[END_REF] and references therein).

Complex dynamic constraints on watershed controls introduce combinatorial aspects in the hydro sub-problem making it even more challenging to solve ( [START_REF] Doukopoulos-Hechmé | The short-term electricity production management problem at EDF[END_REF]).

Ideally, a Robust unit-commitment problem considers at least the following sources of uncertainty:

1. uncertainty on inflows for the hydro reservoirs 2. uncertainty on customer load 3. uncertainty on renewable generation 4. uncertainty on unit availability.

In this section we will consider the first two sources of uncertainty. It can be argued that the third source can be appended to uncertainty of load. It might however be of more interest to consider both sources separately as information on their individual laws might be more readily available than on their aggregated law. Since the deterministic unit-commitment models are already large-scale, their robust counterparts are clearly at least as difficult to solve.

Obtaining a globally robust production schedule means that one has to incorporate uncertainty on load as well. In the literature "stochastic unit-commitment" models are far less common than their deterministic counterparts. Most approaches considered so far use scenario trees in one way or another (e.g. [START_REF] Philpott | Hydro-electric unit commitment subject to uncertain demand[END_REF][START_REF] Takriti | Incorporating fuel constraints and electricity spot prices into the stochastic unit commitment problem[END_REF][START_REF] Takriti | A stochastic model for the unit commitment problem[END_REF][START_REF] Nowak | Stochastic lagrangian relaxation applied to power scheduling in a hydro-thermal system under uncertainty[END_REF][START_REF] Wu | Stochastic security-constrained unit commitment[END_REF]). Uncertainty in each node of the scenario tree is considered known when the decision is taken. When the trunk of the tree covers a whole day (as in [START_REF] Takriti | A stochastic model for the unit commitment problem[END_REF]) the obtained schedule has seen future uncertainty (past the first day) and can be applied (i.e., sent to the grid operator). Yet uncertainty during the first day is neglected. When the trunk covers less time, it is unclear what schedule has to be sent to the grid-operator ( [START_REF] Nowak | Stochastic lagrangian relaxation applied to power scheduling in a hydro-thermal system under uncertainty[END_REF]). Recently robust optimization has been applied to stochastic Unit-Commitment ( [START_REF] Zhao | Robust unit commitment problem with demand response and wind energy[END_REF][START_REF] Bertsimas | Adaptive robust optimization for the security constrained unit commitment problem[END_REF]). But the suggested model decouples commitment decisions (starting / stopping status of each unit) from dispatch decisions (power output). The latter decisions are then taken when uncertainty is fully known. Though very interesting, it is unclear still what schedule (including power output) has to be sent to the grid operator. Moreover, in practice, commitment decisions can be adapted (at least to some extent) in a 2nd stage.

Here we make the assumption that both commitment and dispatch decisions have to be taken prior to observing uncertainty. Though it is true that recourse stages exist that allow us to make changes to the previously committed schedule, we do not consider that option in the current model. It is left for future work. We suggest a model that uses a joint-chance constraint to model the deviation between production and load and request that such a deviation remains between bounds with a specific probability. In this way the production schedule becomes robust against uncertainty. Since this chance constraint renders the offer-demand equilibrium constraints highly non-linear (even implicit) it prevents straightforward application of classic decomposition schedules to tackle the large-scale structure. Decomposition schemes are clearly a prerequisite if Robust Unitcommitment is ever to be used in practice. Therefore, we will examine to what extent efficient decomposition schemes can be derived in order to solve the proposed Robust Unit-Commitment problem. We emphasize that our approach differs from the individual chance constrained approaches considered in [START_REF] Ding | Studies on stochastic unit commitment formulation with flexible generating units[END_REF][START_REF] Ozturk | A solution to the stochastic unit commitment problem using chance constrained programming[END_REF]. Indeed the individual chance constraints on the offer-demand equilibrium with uncertainty on load have an equivalent linear formulation with an additional safety term.

This section is organized as follows: In section 5.2.1 we introduce the specific structure of robust unit-commitment problems. We then propose several decomposition schedules in section 5.2.2. Some theoretical insights on obtaining slater points for bilateral joint chance constraints, useful for a numerical solution of joint chance constrained programming, are given in section 5.2.3. Section 5.2.4 contains the description of the model consistent with the highlighted structure. The numeric results obtained when applying the various decomposition schedules and a discussion thereof are provided in section 5.2.6.

Problem Structure

In this section we describe the specific block structure of unit-commitment problems with joint chance constraints. Section 5.2.1.1 contains a short description of our application showing how this structure arises in practice. Section 5.2.1.2 provides a global view of the problem structure.

Hydro-Thermal Unit-Commitment Problems

In Hydro-Thermal systems, each production unit generates power to meet some overall equilibrium between customer load and generated power. It is very natural to assume that units generate power without influencing any other units, at least for thermal production units. Clearly hydraulic turbines and pumps influence each other through a network structure and flow constraints. Hydraulic "units" are therefore typically entire hydro valleys for which it becomes reasonable to assume that they can be operated independently. Key constraints are bilateral bounds on volume in each reservoir, flow constraints and potentially complex technical constraints on turbining/pumping operation. Cost, i.e., the value of water is typically computed by a mid-term planning (optimization) tool, often based on dynamic programming. Since inflows are uncertain and decisions taken before observing uncertainty, one has to come up with a reasonable way to model the above bilateral volume constraints. One way of doing so, is to use chance constrained programming (see Section 5.1). When considering that a Hydro-Valley acts on a price signal, this would imply the following problem structure:

min x∈R k c T x s.t. P[A r x + a r ≤ ξ ≤ A r x + b r ] ≥ p (5.2.1) Ax ≤ b,
where k is the problem dimension, i.e., the dimension of the decision vector as the hydraulic valley is concerned. Moreover, l r specifies the number of lines of matrix A r in the joint probabilistic constraint with safety level p and l d specifies the number of lines of the matrix A. In the specific case of a hydraulic valley, l r would typically be the number of reservoirs receiving random inflows times the number of time steps. The size of the parameter l d would also be linked to various auxiliary variables and related constraints. In applications one has l d > l r . The vectors a r , b r , b, c are of appropriate dimension. The vector ξ ∈ R l r follows some appropriate multi-variate law. In the case of a Hydro-Valley, the cost vector c would reflect costs of using water as computed with loss of opportunity costs and gain as induced by a price signal. The latter price signal might be one coming from a decomposition schedule. The polyhedral constraints Ax ≤ b would contain bounds on turbining, pumping and bilateral bounds on the volume in reservoirs not impacted by uncertain inflows. The joint chance constraint would deal with bilateral bounds on volume for reservoirs impacted by uncertain inflows. As such the matrix vector product A r x represents the way in which the volume in each reservoir and for each time step is impacted by the previous turbining/pumping decisions. Roughly speaking a r would be the minimal volume bound for each reservoir and time step minus the initial volume and to which we also integrate any deterministically cumulated inflows. The vector b r is defined similarly but with respect to a maximal volume bound. It is important to note that the joint chance constraint is implicit and highly non-linear. Many assumptions (e.g. ξ Gaussian) can be made under which the feasible set induced by the chance constraint is convex and an appropriate transform of the joint chance constraint a concave function. Then problem (5.2.1) can be either solved by the supporting hyperplane method (see Section 5.2.3) or the bundle methods of Sections 4.1 and 4.2. Again, the main difficulty lies in computing gradients of the joint chance constraint.

As far as the thermal units are concerned, they are subject to complex technical constraints on power variations, starts, ramping rates, minimum up/down times etc... (see [START_REF] Frangioni | Solving non-linear single-unit commitment problems with ramping constraints[END_REF][START_REF] Langrene | Dynamic constraints for aggregated units: Formulation and application[END_REF] for instance). Typical constraints imply that adjacent power levels are sufficiently close (ramping constraints), the unit remains online or offline for a specific amount of time (minimum up/down times), the number of starts is limited, starting costs depend on plant status etc... Most of these constraints imply non-convexities and typical modelling involves binary variables. Therefore, when considering that a thermal unit acts on a price signal, in full generality this would imply a problem structure of the following form:

min x∈R k ,z∈Z r f (x, z) s.t. Ax + Bz ≤ b, (5.2.2)
where k + r is the dimension of the decision vector for the thermal unit. The objective function f is typically a separable linear or quadratic function and the polyhedral inequality system Ax + Bz ≤ b modelling the constraints, contains vectors and matrices of appropriate dimension.

In fact the only constraints that couple the units together are the equilibrium constraints. In a deterministic setting, these simply state that deviation between production and customer load has to remain small. These constraints then have the typical form

s d ≤ D -A l x ≤ s u , (5.2.3)
where s l , s u ∈ R T are operator chosen bounds, T is the number of time steps in the considered time horizon, D customer load and A l the T × n matrix summing up the production of each of the m units aggregated in the decision vector x = (x 1 , ..., x m ). When D is considered to be uncertain and x decided upon before observing D, an appropriate model for (5.2.3) results from requiring the decision vector x to satisfy

P[s d ≤ D -A l x ≤ s u ] ≥ p. ( 5

.2.4)

A very similar structure is investigated in [START_REF] Römisch | Distribution sensitivity for certain classes of chanceconstrained models with application to power dispatch[END_REF], where stability theory for probabilistically constrained problems is developed. In particular, the authors explicitly consider stability results for probabilistically constrained power dispatch models, showing that the models are stable for several underlying distributions of the load, such as discrete or multi-variate Gaussian. However, no computational results are presented.

Putting these elements together, one ends up with a typical block structure:

min x:=(x 1 ,...,x m )∈R n m i=1 f i (x i ), s.t. x i ∈ X i ⊆ R n i , i = 1, ..., m (5.2.5) s d ≤ D -A l x ≤ s u ,
where m i=1 n i = n and the set X i integrates the constraints of problems (5.2.2) or (5.2.1) depending on whether i ∈ {1, ..., m} refers to a hydro valley or thermal unit. Finally the mappings f i : R n i → R associate a cost to each decision vector x i ∈ R n i for each i = 1, ..., m.

Large-scale (or difficult) instances of problem (5.2.5) arise typically when either the number of units m is very large or the constraint sets X i are complex for some i ∈ {1, ..., m}. In practice the instances coming from French Unit-commitment problems have both these features and therefore require decomposition methods in order to be solved.

A bird's view of the structure

Following the discussion in section 5.2.1.1, unit-commitment problems in Energy management have the following typical abstract structure (see also [START_REF] Dubost | A primal-proximal heuristic applied to french unitcommitment problem[END_REF] and references therein):

min x∈R n f (x) s.t. x ∈ X 1 ∩ X 2 (5.2.6) x ∈ B := {y : x ≤ x ≤ x} ,
where n is the size of the decision making vector, X 1 is the set modelling all constraints on the individual m units and X 2 the offer-demand equilibrium constraints. This first set is typically structured as a cartesian product of smaller sets. This means that X

1 = m i=1 X 1 i , with X 1 i ⊆ R n i , m i=1 n i = n.
Moreover, the objective function f also allows for a decomposition along the sets X 1 i , i.e., f (x) = m i=1 f i (x i ) and x i ∈ X 1 i for all i = 1, ..., m. Each of the sets X i 1 , i = 1, ..., m roughly contains the feasible production schedules for each unit. In practice a "unit" would be anything ranging from a thermal generation unit, a hydro valley to a specific financial contract.

In many classical applications the set X 2 is a polyhedral set and the sets X i 1 are defined by many constraints, possibly involving non-linear, non-convex or combinatorial constraints and/or variables.

We will consider uncertainty on both inflows and customer load. This impacts problem (5.2.6) in two ways. Firstly, some of the sets X i 1 will be defined by additional joint bilateral chance constraints. Secondly, the set X 2 , modelling the offer-demand equilibrium constraints will no longer be polyhedral (as in (5.2.3)), but defined by a joint bilateral chance constraint itself (as in (5.2.4)). As such, classical decomposition schedules based on Lagrangian decomposition ( [START_REF] Lemaréchal | An approach to variable metric bundle methods[END_REF]) can no longer be applied in a straightforward manner. The second difficulty is that even though it has been shown ( [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF]) that the robust hydro valley sub-problems (those with the joint bilateral chance constraint due to uncertainty on inflows) are solvable, the computational effort required to handle them is still substantial. Any algorithm requiring that computational effort at each iteration/step is incompatible with any operational requirement.

According to the model (5.2.4) it turns out that the problem structure of the problem

min x∈R n f (x) s.t x ∈ X 2 , x ∈ B (5.2.7)
is of the same nature as problem (5.2.1), when f is linear. Indeed the chance constraint (5.2.4) is of the same nature of the chance constraint appearing in problem (5.2.1). Data clearly depends on the specific sub-problem. The convexity of the feasible set of Problem (5.2.1) is of-course important for an efficient solution. In particular whenever the vector ξ ∈ R l r follows a (centered) multi-variate Gaussian law with covariance matrix Σ, convexity can be ensured. Such an assumption is fairly realistic for load as shown in [START_REF] Bruhns | A non-linear regression model for midterm load forecasting and improvements in seasonality[END_REF]. Indeed, it follows from a classic result by Prékopa (Theorem 4.2.4 [START_REF] Prékopa | Stochastic Programming[END_REF]) that the feasible set induced by the Probabilistic Constraint is convex in that case. In such a setting, similar algorithms as those found in [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF] can therefore be used in order to solve problems (5.2.1).

We will investigate decomposition methods in order to solve (5.2.6) efficiently in section 5.2.2.

Decomposition Methods

We will investigate decomposition methods based on moving the problem structure of problem (5.2.6) artificially into a full cartesian product structure. This can be done by adding additional variables y ∈ R ñ in an appropriate dimension (this technique is called "variable duplication" in [START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF] and dates back at least to [START_REF] Soenen | Contribution à l'étude des systèmes de conduite en temps réel en vue de la commande d'unités de fabrication[END_REF]) playing the same role as x ∈ R n . An additional constraint is added to enforce their equality. We will speak of "lift", because the problem dimension is increased. Many different lift and decomposition schedules can be set up. They depend on the choice of the dimension ñ and the way in which we enforce equality with the original problem variables. In this section we will specify several such schedules and investigate their efficiency on a numerical example in section 5.2.6.

Lift and Dualize

Applying the previously described "duplication" technique directly, yields the following problem formulation for any α ∈ [0, 1]:

min x∈R n ,y∈R n αf (x) + (1 -α)f (y) s.t. x ∈ X 1 (5.2.8) y ∈ X 2 x = y x ∈ B, y ∈ B.
It is easily seen that problem (5.2.8) is equivalent with problem (5.2.6) for all α ∈ [0, 1]. Moreover problem (5.2.8) will admit a Slater point if and only if (5.2.6) admits such points. It does not however exploit any knowledge of the specific nature of the X 2 set. This can be done as explained in section 5.2.2.1.1. The result of which is a lift in smaller dimension, most likely allowing for a more efficient solution.

In order to solve this new problem formulation, we will consider the Lagrangian dual of (5.2.8) where the constraint x = y is relaxed (Duplicating variables + Lagrangian Decomposition is referred to as Lagrangean Decomposition in [START_REF] Guignard | Lagrangean relaxation[END_REF] and references therein). This implies that for each fixed µ ∈ R n , the following primal problem has to be solved:

min x∈R n ,y∈R n αf (x) + (1 -α)f (y) + µ, x -y s.t. x ∈ X 1 (5.2.9) y ∈ X 2 x ∈ B, y ∈ B.
This primal problem can be naturally decomposed into sub-problems by unit and an additional load-equilibrium sub-problem. Let x(µ), y(µ) denote an arbitrary optimal solution. By construction x(µ) is feasible for all operational constraints and y(µ) for the offer-demand equilibrium constraint.

The dual function Θ : R n → R defined as

Θ(µ) := αf (x(µ)) + (1 -α)f (y(µ)) + µ, x(µ) -y(µ) , (5.2.10)
is a concave typically non-differentiable function. We can use a Bundle Method ( [START_REF] Hiriart-Urruty | Convex Analysis and Minimization Algorithms II[END_REF][START_REF] Bonnans | Numerical Optimization: Theoretical and Practical Aspects[END_REF]) in order to maximize (5.2.10), i.e., optimize the problem dual to (5.2.9). It is known that the solution (x(µ * ), y(µ * )) at the optimal dual signal µ * is not necessarily an optimal solution for the primal problem (5.2.9). However, it has been shown that one can take an optimal convex combination of oracle responses (along the iterations) and that this is the optimal solution of the bi-dual problem (see [START_REF]Lagrangian relaxation[END_REF][START_REF] Briant | Comparison of bundle and classical column generation[END_REF]). The convex multipliers are dual variables belonging to the cuts in the last quadratic program solved by the Bundle Method. If problem (5.2.6) is convex, then this approach allows us to obtain the optimal primal solution.

In the specific situation wherein X 1 is a closed convex set, constraint (5.

∈ X2 := z ∈ R T : P[s d ≤ D -z ≤ s u ] ≥ p and A l x = z. This alternative descrip- tion of the set X 2 , called X2 is a subset of R T . Typically T is much smaller than n.
This therefore leads to the following alternative, wherein the problem dimension is equal to the number of lines in the probabilistic offer-demand equilibrium constraint, i.e., T . That formulation is:

min x∈R n ,z∈R T f (x) s.t. x ∈ X 1 (5.2.11) z ∈ X2 A l x = z x ∈ B, z ≥ 0, z ≤ z,
where z is a natural maximum limit on installed capacity of generation and the matrix A l as defined in equations (5.2.3) and (5.2.4).

The constraint A l x = z is relaxed and the following dual function considered Θ 2 : R T → R:

Θ 2 (µ) := f (x(µ)) + µ, A l x(µ) -z(µ) .
(5.2.12)

The corresponding dual problem to (5.2.11) is that of maximizing (5.2.12).

Lift and Dualize -Augmented Lagrangian

A classic idea to overcome the fact that the primal solution of (5.2.8) at the optimal dual signal µ * in (5.2.10) is not optimal, is the use of Augmented Lagrangians. A further reason for considering such approaches is the fact that in [START_REF] Batut | Daily scheduling with transmission constraints: A new class of algorithms[END_REF] it is reported that they deal better with a non-convex unit-commitment problem. Non-convexity typically comes from complex constraints on thermal generation.

Considering an Augmented Lagrangian, the primal problem formulation becomes:

min x∈R n ,y∈R n αf (x) + (1 -α)f (y) + µ, x -y + c 2 x -y 2 s.t. x ∈ X 1 (5.2.13) y ∈ X 2 x ∈ B, y ∈ B,
for some appropriate constant c > 0. The dual problem would consist of maximizing (5.2.13) over µ ∈ R n .

Unfortunately this problem no longer naturally decomposes directly into sub-problems. One can however apply the auxiliary problem principle ( [START_REF] Cohen | Decomposition-coordination methods in large-scale optimization problems. the non-differentiable case and the use of augmented Lagrangians[END_REF][START_REF] Cohen | Auxiliairy problem principle and decomposition of optimization problems[END_REF]) to obtain an effective decomposition scheme, whenever the problem satisfies appropriate hypotheses. Let (x k , y k ) ∈ X 1 × X 2 be a solution generated by an iterative process trying to solve Problem (5.2.13). In order to decompose problem (5.2.13) according to the auxiliary problem principle, let us define the kernel

K k : R n × R n → R + as follows K k (x, y) := ĉ 2 x -x k 2 + ĉ 2 y -y k 2
, for some constant ĉ > 0. Pick a further set of constants ε k > 0 and let these constants satisfy ĉ > 2ε k c, then this principle shows that solving

min x∈R n ,y∈R n ε k (αf (x) + (1 -α)f (y) + µ, x -y ) + ĉ 2 x -x k 2 + + ĉ 2 y -y k 2 + cε k x k -y k , x -y s.t.
x ∈ X 1 (5.2.14) y ∈ X 2 x ∈ B, y ∈ B, to obtain (x k+1 , y k+1 ) will yield a sequence converging to the optimal solution. Moreover the function values will decrease in each iteration. The advantage is that problem (5.2.14) can be decomposed naturally. Moreover µ can be updated by using Uzawa's method, i.e., µ k+1 = µ k + ρ(x ky k ), (5.2.15) with an appropriate step size ρ ∈ (0, 2c). No stopping criteria is exhibited in ( [START_REF] Cohen | Decomposition-coordination methods in large-scale optimization problems. the non-differentiable case and the use of augmented Lagrangians[END_REF][START_REF] Cohen | Auxiliairy problem principle and decomposition of optimization problems[END_REF]) or later documents. We will however choose the stopping criteria x ky k < tol, for some user defined tolerance tol.

Subproblem Structure

When applying the augmented Langrangian idea, the sub-problem structure is no longer that of (5. where Q is a k × k semi-definite matrix (typically even diagonal) and other data is unaltered with respect to problem (5.2.1). In a very similar way as above we also have to add a quadratic term to the objective function of sub-problem (5.2.2).

These sub-problems can be solved by adapting the supporting hyper-planes methods ( [START_REF] Prékopa | Stochastic Programming[END_REF]) idea in a straightforward manner. A similar change can be operated as the Bundle Methods (of Sections 4.1 and 4.2) for solving (5.2.1) are concerned. It however requires solving convex-quadratic programs at each iteration instead of linear programs. Bundle methods already solve such quadratic programs at each iteration, so the overall computational difficulty is not increased.

Dimension Reduction

In problem (5.2.13), the dual signal µ is again of dimension n, i.e., the global problem dimension. It can be reduced in a very similar way as explained in section 5.2.2.1.1. As such, the Reduced Augmented Lagrangian Problem that we would like to solve becomes:

min x∈R n ,z∈R T f (x) + µ, A l x -z + c 2 A l x -z 2 s.t.
x ∈ X 1 (5.2.17) z ∈ X2 x ∈ B, z ≥ 0, z ≤ z, for an appropriate constant c > 0. The dual problem would consist of maximizing (5.2.17) over µ ∈ R T .

Let (x k , z k ) ∈ X 1 × X2 be the current iterate. By picking the Kernel K k : R n × R T → R + defined as follows K k (x, z) := ĉ 2 xx k 2 + ĉ 2 zz k 2 , for ĉ > 0, the auxiliary problem principle requires solving the following problem:

min x∈R n ,z∈R T ε k f (x) + µ, A l x -z + ĉ 2 x -x k 2 + ĉ 2 z -z k 2 + cε k (A l ) T A l x k -(A l ) T z k , x +cε k z k -A l x k , z ) s.t.
x ∈ X 1 (5.2.18) z ∈ X2 x ∈ B, z ≥ 0, z ≤ z, where ε k > 0 is a set of constants. When the parameters satisfy the inequality ĉ > 2 max { A , 1} ε k c, solving (5.2.18) yields a sequence (x k+1 , z k+1 ) converging to the optimal solution of (5.2.17). The dual signal µ ∈ R T can again be updated by applying Uzawa's updating schedule and requires ρ ∈ (0, 2c) as before. Alternatively one can update it by considering (x k+1 , z k+1 ) as an inexact solution of problem (5.2.17) and using a Bundle Method with inexact oracle (see [START_REF] Oliveira | Level bundle methods for oracles with on demand accuracy[END_REF] and reference therein) to update µ.

Diagonal Quadratic Approximation

As suggested in [START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF] we will also investigate the Diagonal Quadratic Approximation (DQA) technique ( [START_REF] Ruszczyński | On convergence of an augmented lagrangian decomposition method for sparse convex optimization[END_REF]). It is called a Block Coordinate Descent Method (or non-linear Gauss-Seidel method in [START_REF] Beltran | Unit commitment by augmented lagrangian relaxation: Testing two decomposition approaches[END_REF]). The authors of [START_REF] Beltran | Unit commitment by augmented lagrangian relaxation: Testing two decomposition approaches[END_REF] find it to be far more efficient than the auxiliary problem principle. It's main idea is to solve problems (5.2.13) or (5.2.17) by alternating steps between optimizing over x or y (z), while keeping the other fixed to its current value. In [START_REF]Divide to conquer: Decomposition methods for energy optimization[END_REF] it is suggested to interpret the obtained solution as an inexact solution to (5.2.13) or (5.2.17) and update the dual signal µ according to an inexact Bundle Method. We will therefore follow these suggestions as well and apply the principle on Problem (5.2.17). The impact of using DQA in that setting on the sub-problems (5.2.16) is that the matrix Q is no longer diagonal.

Algorithmic and Numerical Considerations

In this section we provide some Algorithmic and Numerical considerations in order to solve Joint Chance Constrained Programs (JCCP) as (5.2.1) or (5.2.16) as efficiently as possible in order to have efficient decomposition methods. First of all, many algorithms for solving (5.2.1) require the knowledge of a Slater point. Such a point can, in the general case, be computed by applying a very similar solution algorithm to a problem wherein the joint probability is maximized. This is often very involved and computationally quite long, especially when approaching the maximum feasible level. In section 5.2.3.1 we provide some results that allow us to obtain such a point in some cases, by solving a linear program only. Section 5.2.3.2 provides some considerations on how to obtain an efficient decomposition scheme.

Slater Points

The key observation in the following Lemmas is that symmetric random variables maximize probability when evaluating symmetric rectangles in bilateral chance constraints.

We recall Definitions 2.5.2 and 2.5.6 and state our key result: Remark 5.2.4. The assumption concerning the set X in Corollary 5.2.2 and 5.2.3 can be easily checked if for instance the matrix A r is surjective. It then has a right inverse G. Then indeed x = -Gα is a solution to the first condition and the set X is not empty if -AGα ≤ b. If the condition fails to be satisfied, one could look at x minimizing A r x + α 2 under the constraint Ax ≤ b. This may turn out to be a reasonable Slater point.

Remark 5.2.5. The interest of Corollary 5.2.3 lies in the clear link between problem (5.2.19) and that of our application (5.2.1). One could attempt to use Corollary 5.2.3 or 5.2.2 in order to obtain a Slater point for problem (5.2.1). In our application considered in section 5.2.6 it turns out that Corollary 5.2.3 allows us indeed to obtain a Slater point for both Hydro Reservoir problems and the load sub-problems. This obtained Slater point is not the optimal Solution of problem (5.2.1) and appropriate algorithms still have to be used.

Algorithmic Considerations

In the decomposition schemes proposed in section 5.2.2 each iteration requires the solution of the sub-problems, some of which are joint chance constraint problems. The latter problems are solvable, but can take some time. Now brutally plugging those solution methods inside a decomposition scheme might lead to enormous computation times. But one can do this far more efficiently. Solving such joint chance constrained programming problems can be done by forming a cutting plane approximation of the feasible set (or a cutting plane model for an appropriate transform of the joint probabilistic constraint). If one uses a supporting hyperplane method ( [START_REF] Prékopa | Probabilistic programming[END_REF], see also [START_REF] Van Ackooij | Joint chance constrained programming for hydro reservoir management[END_REF]) in order to solve a joint chance constrained program, one would start with computing the convex combination of the current iterate and the Slater point that exactly meets the required probability level. One would then compute a new cutting plane to improve the current cutting plane model of the constraint set. This cut removes the previous iterate from the solution set. Depending on whether the structure is that of (5.2.1) or (5.2.16) one would solve a linear or quadratic program to come up with the following iterate. If it is feasible one would stop the algorithm. In such a procedure, most time is spent computing the gradients of the joint-chance constraint (i.e., the elements of the Bundle).

When solving problems (5.2.1) or (5.2.16) we thus form a cutting plane model of (a transform of) the joint chance constraint. A change of objective function can be operated and the latter model would remain valid. Since CPU time is spent setting up a cutting plane model of the joint chance constraint, it is clearly of interest to improve this cutting plane model over the course of the iterations of the decomposition schedule. We thus assume that a first cutting plane model is generated and stored to disk (or computed in a first lengthy iteration) for each JCCP sub-problem. It can also be observed that the current solution for a sub-problem solution is feasible for its constraints and can be used as an initial solution for future iterations. Together with the trivial continuity property of Lemma 5.2.6, a small change in the objective function would lead to a near optimal initial solution for two consecutive iterations. Indeed if x ∈ R k is an optimal solution for a sub-problem inside of a decomposition scheme and the following iteration of this decomposition scheme induces a minor change in the objective function of this sub-problem, then x is nearly optimal for that sub-problem. In particular this means that only few iterations are required in order to appropriately solve the sub-problem. Few iterations imply few costly improvements of the cuttings plane model of the joint chance constraint. Such improvements are of interest only in the vicinity of the global optimal solution. Lemma 5.2.6. Let C ⊆ R n be a bounded connected set and let f, g : C → R be bounded functions admitting global minima on C. Let ε > 0 be an arbitrary constant and assume that g satisfies gf ∞ < ε. Then an optimal solution of the problem min x∈C f (x) is 2ε optimal for min x∈C g(x).

Moreover by solving a linear programming problem using the current cutting planes model, one can obtain a lower bound on the objective function. The current best feasible point, provides an upper bound. We can therefore terminate any solution algorithm for JCCP early if the current best feasible point is sufficiently close (relatively) to the best available bound.

This argument shows that it can be preferable to have small updates in the sub-problem objective functions in order to have fast decomposition iterates. In this section we will give a detailed description of the simplified Robust Unit-Commitment problem that we will consider in the numerical example. The model considers a Hydro-Thermal Energy mix such as one could encounter in the French system. We will consider a Robust version of a realistic hydro valley model (see Section 5.1 for details) since we consider inflows to be random. We will however consider simplified thermal generation units in order to keep their feasible set convex as the focus is on dealing with robust models as defined by joint chance constraints and not on non-convexities induced by very realistic thermal generation models. Clearly, the suggested methods of section 5.2.2 can easily be applied to models involving far complexer sub-problem models. The applied decomposition method would need to be followed by a Heuristic approach in order to recover an interesting Primal Solution. This is not trivial as shown in [START_REF] Frangioni | Solving Unit Commitment Problems with General Ramp Contraints[END_REF] and will require substantial additional work. It will be left for future investigations. Finally we will consider randomness on load and suggest a robust model based on joint chance constraints in order to generate sufficient, but not excess energy. The obtained problem is labelled robust as the obtained production schedule protects against uncertainty on inflows and load. The obtained problem structure is exactly that of (5.2.6). The set X 1 will be detailed in sections 5.1 and 5.2.5.1. The set X 2 will be specified in section 5.2.5.2.

We will consider a discretized time horizon. To this end let τ denote the set of (homogeneous) time steps. Let ∆t be this time step size expressed in hours. Occasionally T will denote the number of (or last) time steps.

The Thermal Generation Unit sub-problem

We will consider a very simple thermal sub-problem, wherein power output of thermal units is considered to be continuous between 0 and some maximum power level P mx . We moreover assume that we have some ramping rate s (M W/h), such that any adjacent power levels may differ no more than s∆t. Finally thermal units have a cost c p (e/M W h).

With initial power level P 0 and price signal λ ∈ R T , this gives the following problem.

min x∈R T c T x s.t.      -s∆t + P 0 -s∆t . . . -s∆t      ≤      1 0 0 • • • 0 -1 1 0 • • • 0 . . . 0 • • • 0 -1 1      x ≤      s∆t + P 0 s∆t . . . s∆t     
, where c = c p (1, ..., 1) Tλ. This is a linear programming model, and therefore convex.

Some more detailed thermal constraints can be found in [START_REF] Langrene | Dynamic constraints for aggregated units: Formulation and application[END_REF]. These will however imply a non-convex solution set. We wish to focus only on the difficulties induced by trying to decompose block structured Chance Constrained Programs. As such, for the moment, we will avoid difficulties generated by non-convex thermal sub-problems. Such additional difficulties will be addressed in future work.

Offer Demand Equilibrium constraint

We will assume that we have I h ∪ I t = {1, ..., m} units, where I h indicates the hydrovalleys and I t the thermal units. Let R n i be the dimension of the sub-problem i ∈ {1, ..., m}, n h = i∈I h n i and n t = i∈I t n i . By construction the thermal sub-problems have dimension n i = T . Hydro sub-problems have a decision vector x i ∈ R n i far larger. By placing the global decision vector x in the order x = (x h 1 , ..., x h | I h | , x t 1 , ..., x t |I t | ) ∈ R n h × R n t = R n and by doing some bookkeeping one can readily identify the T × n matrix A l such that A l x is the amount of generated power.

It seems reasonable to assume that load D ∈ R T follows a model of the form

D t = µ t + ξ t , (5.2

.20)

where ξ t is independent of whatever preceded the first time step t 0 and t ∈ τ . This implies that E (D t |F t 0 ) = µ t for all t ∈ τ , where F t 0 is the first element of a filtration {F t } t∈τ with which we can equip our probability space. The latter filtration would model information in a dynamic setting, which is not considered here. This makes µ t the best load forecast at time t 0 . We moreover assume that ξ t follows a causal time series model with Gaussian Innovations. This is a fairly realistic assumption as [START_REF] Bruhns | A non-linear regression model for midterm load forecasting and improvements in seasonality[END_REF] shows. We also assume that we are given some bounds s u (t) ≥ 0 and s d (t) ≤ 0 that define acceptable load mismatches. Typically one would allow production levels higher that D t in peak hours and lower than D t in off-peak hours. The following Joint chance constraint models the offer demand equilibrium constraint:

P[s d (t) ≤ D t -(A l x) t ≤ s u (t) ∀t ∈ τ ] ≥ p, (5.2.21)
because of the above assumptions, one can find a Gaussian random variable ξ with variance covariance matrix Σ and 0 mean, such that this constraint is equivalent with

P[s d + A l x -µ ≤ ξ ≤ s u + A l x -µ] ≥ p.
(5.2.22)

Because of earlier stated results, this implies a convex feasible set for x.

Numerical Example: Data and Results

In this section we provide the results of applying each of the decomposition schedules in section 5.2.2 to the problem detailed in section 5.2.4. Section 5.2.6.1 provides the specific numeric data of this test problem, whereas section 5.2.6.3 provides the numeric results.

Numerical Data

We will consider a production park with two hydro valleys, and 9 thermal units. We will consider 24 time steps of 2 hours each. These two hydro valleys are submitted to a joint probability constraints with uncertainty on inflows. This gives a global decision vector dimension of size n = 1776. Moreover there are three joint probability constraints, two of which with a random variable in dimension 48 and one with a random variable in dimension 24.

Numerical results

The numerical results, of which an extensive discussion is provided in section 5.2.6.3 show that Robust Unit-Commitment with joint probabilistic constraints:

1. Can be readily solved to optimality in large scale systems under a convexity assumption. The computational effort is controlled except for a first lengthy iteration in which the cutting plane models for the joint chance constraints have to be set up.

2. Intuitively the augmented Lagrangian approaches with Bundle Methods for the dual form an interesting starting point for the non-convex models.

An extensive discussion of Numerical results

In this section we will compare the various decomposition methods suggested in section 5.2.2 on the numeric instance. The goal is to show that an efficient decomposition scheme can be obtained. Two key elements will be examined: feasibility for all constraints of the obtained solution and cost. We will also provide computation time for each method. As observed in section 5.2.3.2 some methods may have lengthy iterations as they can imply "chaotic" changes in the sub-problems objective functions. Especially the robust sub-problem models then require some iterations to converge. Since each such iteration involves the computation of a gradient of a joint-chance constraint, this may take some time.

Since the model of section 5.2.4 is convex by construction, it is known that the lift and dualize method of section 5.2.2.1 converges. Moreover, the optimal convex combination of primal oracle responses in the Lagrangian decomposition (5.2.9) should provide the optimal primal solution. The same holds for the reduced lift and dualize method of section 5.2.2.1.1. We will use a Bundle method to maximize the dual functions (5.2.10) and (5.2.12). The specific Bundle method employed is a variant of the method given in [START_REF] Lemaréchal | Variable metric bundle methods: from conceptual to implementable forms[END_REF] being able to cope with inexact oracles. It turns out that the method of section 5.2.2.1.1 provides an optimal solution indeed. Table 5.4 provides these results: It can be noted here that early iterations for maximizing function (5.2.12) imply some iterations for each of the sub-problems, meaning a significant, but not too large change in the objective functions. Later iterations however require almost no further iterations on solving the sub-problems. This means that the Bundle of Information (containing the local joint-chance constraint cuts) is sufficiently rich and only minor changes in the objective function are incurred. This makes sense, since Bundle Methods for maximizing the dual function (5.2.12) will come up with a new signal µ ′ not far from the current stability center μ. It also turns out that it is indeed preferable to keep the dual signal in small dimension, since problem 5.2.2.1 still did not converge after 5000 iterations.

Since the alternative decomposition methods require parameter settings, we can use these results from Lagrangian decomposition to benchmark those decomposition methods. We recall that these alternative methods are considered since they are reported (e.g., in [START_REF] Batut | Daily scheduling with transmission constraints: A new class of algorithms[END_REF]) to work better in a non-convex (deterministic) setting. Table 5.5 provides the results for solving problem (5.2.13) by applying principle (5.2.14). One can conclude from table 5.5 that selecting a correct parameter setting for the auxiliary problem principle is delicate. Indeed only one choice of the reported parameters (c, ĉ, ρ, ε) allows the algorithm to terminate before reaching the maximum number of iterations. Nonetheless this principle allows us to ultimately generate a feasible solution for many different parameter settings. These solutions are however approximately 6.2% more costly. Figure 5.3 shows how the augmented Lagrangian term xy 2 decreases along the number of iterations for a few selected parameter settings of table 5.5. In a very similar fashion in which the dimension of the Langrangian dual problem could be reduced, the same thing can be done for the Augmented Lagrangian technique as shown in section 5.2.2.2.2. Table 5.5 provides numerical results for applying principle (5.2.18) for solving problem (5.2.17). It turns out that setting up a convergent scheme for the reduced form of the Augmented Lagrangian (5.2.17), decomposed by using the auxiliary problem principle is delicate. Only one set of parameters of the over 50 tested allows us to obtain a feasible solution in the end. Moreover, convergence is very slow compared to the already slow convergence observed for the full version of the Augmented Lagrangian formulation (5.2.13) (decomposed by using the auxiliary problem principle). It seems to be particularly difficult to rightfully set the ρ parameter for Uzawa's updating schedule of the dual signal µ. For this reason we have also attempted to let a Bundle method with inexact oracle update the dual signal. In such a setting we interpret the solution (x k , z k ) given by the auxiliary problem principle as an inexact solution of (5.2.17). Results are reported in Table 5.7. One can observe that this allows us to generate a feasible solution for all the tested settings (c, ĉ, ε) showing the potential for such an idea. The last suggested decomposition schedule was based on decomposing the augmented Lagrangian formulation (5.2.17) according to the diagonal quadratic approximation and using a Bundle method to update the dual of (5.2.17). Results of applying this suggestion can be found in table 5.8. When looking at the results of table 5.8 we again observe the typical additional cost implied by the augmented Lagrangian formulations already observed with the other decomposition schedules. One can also see that when c is small the cost is nearly that of the optimal solution. This result is clearly as expected, since for very small c, the DQA schedule actually boils down to the reduced lagrangian dual decomposition of section 5.2.2.1.1.

In figures 5.4, 5.5 and 5.6 we show the optimal solutions produced by the various decomposition methods.

In figure 5.4 and 5.5 we can see that the optimal turbining strategies differ quite a lot depending on the decomposition schedule used. It is interesting to observe that the turbining strategies produced by the Reduced Lagrangian dual and the Reduced Augmented Lagrangian + DQA are somewhat similar of aspect, whereas those produced by using the auxiliary problem principle are all quite different. One can also see that the solution obtained from using a Bundle method to update the dual signal for the reduced augmented Lagrangian + auxiliary problem principle yields a very similar result when using Uzawa's updating schedule. The latter having been found quite hard to fine tune, empirically. Figure 5.6 shows that all solutions use the thermal units in quite a similar way. Finally we can also look at the feasibility for the offer-demand equilibrium constraint (5.2.22). As such we have generated 100 scenarios for the random vector ξ in (5.2.22) and plotted the bounds obtained from the solution for each of the decomposition schedules. These results are shown in figure 5.7. We have also plotted what would happen if we would generate exactly the average load, i.e., A l x = µ in (5.2.22). One can observe that the Reduced Lagrangian Dual solution follows closely the general allure of the scenarios and that the lower bound mainly becomes binding near the end of the time horizon. The solutions generated by the augmented Lagrangian formulations and there respective decomposition schedules have some additional bumps that can offer no additional feasibility, since they stick with the RLD solution elsewhere, but do account for the additional cost. Altogether as this aspect is concerned, the different solutions are quite satisfactory. Chapter 6

Concluding Remarks

In this Thesis we have investigated probabilistic programming from a theoretical, algorithmic and applicative point of view. The theoretical work provides new efficient gradient formulae and extends eventual convexity results for specially structured probabilistic constraints. Bundle methods were investigated and applied to probabilistic programming. This led to the definition of the so-called upper oracle, which might provide linearizations that locally overestimate the (convex) function. Rendering unit-commitment robust through the use of probability constraints was the key driver of the applications. In particular we show that realistically sized problems can, in principle, be dealt with.

As perspectives are concerned, the first one involves an extension of the provided gradient formulae to the case wherein we are dealing with a random inequality system having several components. We should examine to what extent those hypothetical results can equally be used for Student (or T -) uncertainty. It is also important to provide counterexamples for differentiability. In particular this implies a thorough investigation of the here formulated growth condition. Finally, it would seem that an alternative derivation would lead to less restrictive conditions for the Student case. Déak's provided sampling scheme could surely also be improved upon and would potentially lead to improved accuracy. This would be of interest for reducing computation time.

As convexity is concerned, we have provided a new insight in the eventual convexity results obtained in [START_REF] Henrion | Convexity of Chance Constraints with Dependent Random Variables: the use of Copulae[END_REF]. We have shown that those results can be extended to a larger class of Copulae and that sharper probability thresholds can be obtained. However, these results are still largely incomplete. It is of particular interest to examine if some elliptical copulae (Gaussian, T , etc...) exhibit the δ-γ-concavity property as well. Moreover, since Archimedean Copulae decompose as the product of one-dimensional radial distribution and a uniform distribution over the unit-simplex it is readily conjectured that most of these Copulae should be δ-γ-concave. Counter examples should also be looked at to figure out to what extent the provided probability thresholds are tight.

The use of p-efficient points for discrete distributions might allows us to provide conditions under which the feasible set of a joint probabilistic constraint can be approximated by the simpler v-individual probabilistic constraints introduced in this Thesis. When such positive cases are sufficiently identified it becomes interesting to examine if a separation oracle can be provided. The latter would allow us to identify a violated v-individual constraint for a current candidate solution.

Throughout this work we have remained very attached to the convexity of the feasible set. This is of course important in order to show convergence to a global solution of the optimization problem at hand. However many applications simply do not have convexity at all. Mixed integer programming is a flourishing field in the industry. Unit-commitment problems are modelled in a non-convex way for many years. This shows that convexity might not be as important after all. In order to continu the classic approach of probabilistic programming and considerably extend its applications, convexity could be replaced by local Lipschitz continuity. Such results already exist for multi-variate distribution functions, but would need to be extended to probabilistic constraints involving more general random inequality systems. The gradient can then readily be replaced by a sub-gradient in the sense of Clarke. Bundle algorithms for locally Lipschitz functions already exist and would be a key candidate for dealing with such a non-smooth constraint. It is likely that those methods would need to be extended to constrained optimization with inexact (upper) oracles.

From modelling perspectives, we could integrate the combinatorial constraints on the decision variables, potentially without many difficulties in the cascaded reservoir management problem (e.g. [START_REF] Finardi | Solving the hydro unit commitment problem via dual decomposition and sequential quadratic programming[END_REF][START_REF] Arnold | A mixed-integer stochastic nonlinear optimization problem with joint probabilistic constraints[END_REF]). Also the impact of more realistic models for thermal units (with typically non-convex feasible sets) should be investigated. The investigated decomposition approaches can be applied directly to such a setting, but might need to be followed by heuristics in order to recover an interesting primal solution. Investigating such heuristics offers many research perspectives. We should also consider uncertainty on renewable generation and unit availability. The latter aspect might be particularly tricky as it could induce a non-convex feasible set for the offer-demand equilibrium constraint. Then either we try to deal with such a setting as explained above or we set up a hybrid robust-probabilistically constrained model in order to circumvent the difficulty. Ultimately, a thorough comparison will need to be done in order to investigate the strengths and weaknesses of each approach. This of course includes robust optimization in the sense of Ben-Tal and penalized robust optimization as investigated by [START_REF] Minoux | Solving some multistage robust decision problems with huge implicitly defined scenario trees[END_REF][START_REF] Ben-Salem | Gestion Robuste de la production électrique à horizon court-terme[END_REF]. In the latter works, second stage decisions are seen in a simplified manner through the use of a penalty function. The worst case penalty costs are then minimized along with the first stage objective function. It is of particular interest to examine if probabilistically constrained (or robust) optimization can be used instead of a true (robust) multi-stage approach. The advantage of the former over the latter is evidently its improved "tractability".

  t (x, ξ) ≥ 0 ∀t ∈ T ] = lim inf x→x P[g inf (x, ξ) ≥ 0] = = lim inf x→x P[g inf (x, ξ) > 0] =lim sup x→x -P[g inf (x, ξ) > 0] = =lim sup x→x (P[g inf (x, ξ) ≤ 0] -1) = 1lim sup x→x P[g inf (x, ξ) ≤ 0] ≥ 1 -P[g inf (x, ξ) ≤ 0] = P[g inf (x, ξ) > 0] = P[g inf (x, ξ) ≥ 0] = ϕ(x).

Lemma 2 . 5 . 1 .

 251 Consider the situation of (2.0.1), define ϕ : R n → [0, 1] as ϕ(x) = P[g(x, ξ) ≥ 0] ≥ p and let M (p) be defined as M (p) := {x ∈ R n : P[g(x, ξ) ≥ 0] ≥ p}.

3 .

 3 Uniform distributions are log-concave 4. Wishart distributions (on the set of square matrices) are log-concave 5. Beta distributions (on the set of square matrices) are log-concave 6. Multi-variate Gamma distributions are log-concave 7. The m-variate Cauchy distributions has an -1 m -concave density. Its distribution is therefore quasi-concave. 8. The Pareto distribution has a -1

Theorem 2 . 5 .

 25 25 ([110]). Let A be a m × n matrix having a non degenerate multivariate Gaussian density in R nm with independent rows. Let b ≥ 0 be a fixed vector. The set M (p) := {x ∈ R n : P[Ax ≤ b] ≥ p} is convex for all p > p * , where p * = Φ(max √ 3, u * ). Here Φ is the one dimensional standard Gaussian distribution function and u * = max i=1,...,m

Lemma 3 . 1 . 3 .

 313 Consider the situation of Definition 3.1.1. We have : M (p) ⊆ M SI (p) ⊆ M I (p). (3.1.4)

Lemma 3 . 1 . 5 .

 315 Let . be an arbitrary norm in R k . The set M SI (p) of definition 3.1.1 is invariant when restricting ourselves in its definition to all

.1. 8 )

 8 It directly follows that s Ĉp (v) = 0 when v ∈ ker A T . Similarly we get v T Aξ = (A T v) T ξ = 0. Fix an arbitrary v / ∈ ker A T . From (3.1.8) and (3.1.5) we derive:

.1. 9 )

 9 where a, b ∈ R m and the m × n matrices A, B are deterministic. Let ξ ∈ R m be a nondegenerate multivariate Gaussian random variable with mean µ and variance-covariance matrix Σ. Then for each p >1 2 we have M SI (p) = M I (p).

Remark 3 . 3 . 2 .

 332 If ξ has a general nondegenerate Gaussian distribution, i.e., ξ ∼ N (µ, Σ) for some mean vector µ ∈ R m and some positive definite covariance matrix Σ of order (m, m), then one may define ξ := D (ξµ), where D is the diagonal matrix with elements Σ -1/2 ii . Then, clearly, ξ ∼ N (0, R), where R is the correlation matrix associated with Σ. Defining g : R n × R m → R as g (x, z) := g x, D -1 z + µ , (3.3.4) can be rewritten as ϕ (x) = P g(x, ξ) ≥ 0 ,

  3.13 allows us to derive (3.3.20) which along with the formula for ∇ x e above yields (3.3.22).

  Now, let r ≥ 0 be arbitrary and let ζ have a Fisher-Snedecor (F-) distribution with m and ν degrees of freedom. Then, ζ = (νU m ) / (mU ν ), where U m and U ν are independent and follow χ-squared distributions with m and ν degrees of freedom, respectively. Denoting by F m,ν the distribution function of ζ, we derive that

Figure ( 3 . 1 )Figure 3 . 1 :

 3131 Figure(3.1) shows the results of the supporting hyperplane approach. The red ellipsoid is the feasible set for ϕ(x) ≥ 0.8. The dashed red ellipsoid is the feasible set for ϕ(x) ≥ 0.79, the red lines correspond to the feasible set of Ax ≤ b and the black star is the Slater point x s . Successive iterates x k are indicated by their number and a cross, the successive feasible points x c are indicated by circles. Finally the green lines are the generated supporting hyperplanes and the diamond indicates the found optimal solution.

Remark 3 . 4 . 8 .

 348 The reason the sets are of the specific form in definition 3.4.7 and not their opposite will become apparent in Theorems 3.4.18,3.4.19 and 3.4.20.

  ≥ q} etc... Then the density of the Gaussian Copula has been shown to be 0-0-concave on all sets C 2 , ..., C n . Furthermore each C i is a convex set. Hence the restriction of the density of C(e u ) to each C i , i = 2, ..., n is a log concave density by Corollary 3.4.15. Let fi denote the restriction of the density of u → C(e u ) to C i , i = 1, ..., n. Defining ϕ i : [0, 1] m → [0, 1] as the integral of this restricted density fi , it follows by Theorem 4.15[START_REF] Dentcheva | Optimisation Models with Probabilistic Constraints[END_REF] that ϕ i is log-concave, i = 2, ..., n. Now clearly

  (4.1.21) and (4.1.31) below).

  .1.13) for a parameter α k satisfying (4.1.15) below.

Proposition 4 . 1 . 1 (

 411 Consequences of penalization updates). When Algorithm 4.1 uses the rule (4.1.20) for the penalties, the following holds.

1 . 2 , 4 . 4 . 1 . 2 (

 124412 the second case in Lemma 4.1.3 and the last case in Lemma 4.1.Lemma Infinitely many serious iterations). Consider solving (4.1.1) with Algorithm 4.1 using an oracle satisfying (4.1.3) and (4.1.21), with parameters α k , β k satisfying (4.1.15). Let K s denote the set gathering indices of serious iterations. If there are infinitely many of such indices, and the prox-parameter sequence satisfies (4.1.19a), then h k xk ≤ M k (y) + o(1/k) for all y ∈ X and k ∈ K s sufficiently large. Proof. Consider k ∈ K s . We first show that δ k → 0. If for all serious steps we have c k+1 > 0 then (4.1.16) implies that ĉk+1 = c k+1 ≤ c kmδ k = ĉkmδ k , (4.1.23)

Lemma 4 . 1 . 3 (

 413 Infinitely many noisy iterations). Consider solving (4.1.1) with Algorithm 4.1 using an oracle satisfying (4.1.3) and (4.1.21), with parameters α k , β k satisfying (4.1.15) and penalties as in (4.1.20).

  by the third item in the proposition, for some constant M -E k ≤ M for all k ∈ K a . From (4.1.25) and the condition β k ≥ b from (4.1.15) we obtain the inequality E k < -1 2 (1 + b)µ k x k+1xk 2 . Moreover, using (4.1.7) and the fact that 1 + b > 0 by (4.1.15), this means that 2 1 + b

  0 and, furthermore, x k+2x k+1 2 → 0 (4.1.30) because µ k ≥ µk +1 by(4.1.19c).

  x remains fixed. By the same three lemmas, and the first item in Proposition 4.1.1, max(ĉ k , 0)(1σ k ) = h k xk ≤ M k (y) + o(1/k) for all y ∈ X and k ∈ K ′ sufficiently large. By (4.1.31), M k (y) ≤ H τ k (y) + ε k for the exact improvement function (4.1.2) written with target τ = τ k , so

  .1.33) By (4.1.4), the target is τ k = ( f k + ρ k max(ĉ k , 0), σ k max(ĉ k , 0)) and by (4.1.21) the sequences { f k } and {ĉ k } are bounded. Extracting from K ′ a further subsequence K if needed, we let f = lim k∈K f k , c = lim k∈K ĉk , ρ = lim k∈K ρ k , σ = lim k∈K σ k , noting that ρ is not necessarily finite. Passing to the limit as K ∋ k → ∞ in (4.1.33) yields (4.1.32).

Corollary 4 . 1 . 6 (

 416 Convergence for lower oracles). In the setting of Theorem 4.1.5, suppose ε = 0 in (4.1.31). If problem (4.1.1) has a Slater point and ρ is sufficiently large, then c ≤ 0 and f ≤ f (y) for all y feasible in (4.1.1).

Corollary 4 . 1 . 7 (

 417 Convergence for partly asymptotically exact lower oracles). In the setting of Theorem 4.1.5, suppose ε = 0 in (4.1.31) and calculations are eventually exact for serious steps: f (x) = f and c(x) = c. The following holds: (i) Either (4.1.1) is feasible and c(x) ≤ 0 with x solving (4.1.1). (ii) Or c(x) > 0 and (4.1.1) is unfeasible with x minimizing infeasibility. As a result, when (4.1.1) has a Slater point, only item (i) is possible.

Figure 4 . 1 :

 41 Figure 4.1: Numerical Instance Data

  3'a (Interpolation and Oracle) Determine γ k ∈ [0, 1] for which x k+1 c = (1γ k )x k+1 + γ k x s satisfies c x k+1 c = 0. Call the c-oracle and add the linearization to the cutting-plane model. Step 3'b (Stopping Test and Loop) If f, x k+1 c x k+1 < f, x k+1 tol then stop. Otherwise, set k = k + 1 and loop to Step 1'.

K = 1 ,

 1 µ 0 = 1e -10 , µ s = 4.0, κ = 0.1 for all methods.

25 K 52 K 1 4. 1 . 6 . 4

 25521164 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0= 1e 4 , µ 0 = 1e -5 , µ s = 2.0, κ = 0= 1e 4 , µ 0 = 1e -5 , µ s = 2.0, κ = 0.Different variants of Alg.PB.SEV

Figure 4 . 2 :

 42 Figure 4.2: Evolution of reservoir "Saut Mortier", for the expected-value estrategy (left) and for Alg.PB.SEV strategy (right).

2 . 1 )

 21 by defining the (nonsmooth and convex) function c : R n → R c(x) := max j=1,...,m f j (x) , and call f 0 just f . With this notation, the problem we are interested in solving is (4.1.1):

lev is a lower

  bound for the optimal value f min . Therefore, the updating rule f k low ← f k lev at Step 4 gives a valid lower bound for f min . (d) Whenever the level set is empty at Step 5, the lower bound f k low is increased by at least an amount of γh k (x k rec ) > 0 and the cycle-counter l is incremented by one. Moreover f k low f min for all k 0. Assume that δ Tol = 0. If there is an infinite loop between Steps 5 and 1, one concludes that f

2 :

 2 ) and c(x k+1 ) were computed at Step 6(c))J k+1 f ⊃ {k + 1, a f (k)} and J k+1 c ⊃ {k + 1, a c (k)} remain true at each bundle compression. • (If f (x k+1 ) was not computed at Step 6(c)) J k+1 c ⊃ {k+1, a c (k)} and J k+1 f ⊃ {a f (k)} remain true at each bundle compression.According to the above two strategies, we consider the following three versions for Algorithm 4.Algorithm 4.2(fc) Algorithm 4.2 as it is stated; Algorithm 4.2(f) Algorithm 4.2 with Steps 6 and 7 replaced by Steps 6(f) and 7(f); Algorithm 4.2(c) Algorithm 4.2 with Steps 6 and 7 replaced by Steps 6(c) and 7(c).

Remark 4 . 2 . 5 .

 425 Notice that the index set J k f ∩ J k c can be empty for versions (f ) and (c) of Algorithm 4.2. If it is the case, the choice of the next stability center xk at Steps 3 and 5 is not well defined. We may proceed as follows according to Steps 6(f ) and 6(c):-At Steps 3 and 5 of Algorithm 4.2(f ), choose xk ∈ {x j

2 . 28 )

 228 to get val LP . If val LP > 0 continue; otherwise go to Step 5.3. Step 5.2 (Bundle Cleaning) Let (x, s) be a solution to (4.2.28). Set J k c ← {j ∈ J k c : sj 0}, ǫ c ← ǫ c 2 and go back to Step 5. Step 5.3 declare a critical iteration, set l ← l + 1, k(l) ← k, f k low ← f k lev and choose xk ∈ {x j : j ∈ J k f ∩ J k c }. Return to Step 1. If problem (4.2.10) is solved then move to Step 6.

Lemma 4 . 2 . 18 .

 4218 Let Λ > 0 given in (4.2.25) and parameters γ ∈ (0, 1) and θ ∈ (0, (1γ) 2 ) in Algorithm 4.2, with Step 5 replaced by Step 5 (On-demand) and improvement function h k replaced by h k ae given in (4.2.33). Then, at iteration k of any version (fc), (f ) or (c) of Algorithm 4.2, the following estimates hold:

  and the result follows from (4.2.35). Since the three considered possibilities (k ∈ S, case (a) and case (b)) cover Algorithms 4.2(fc), 4.2(f) and 4.2(c), the stated result follows.

method 6 K

 6 Obj. Value Nb. Iter. [¬(4.2.10)] CPU time parameters = 5e 4 , γ = 0.667, δ Tol = 5 Alg.LB -104159 76 [2] 8.2 K = 5e 4 , γ = 0.800, δ Tol = 5 Alg.LB -104159 90 [2] 11.4 K = 5e 4 , γ = 0.900, δ Tol = 5

  k ) (for f k low fixed in the cicles) c(x k ) (for f k low updated at each iteration)

Figure 4 . 3 :

 43 Figure 4.3: Comparison of Algorithm 4.2 when solving LP (4.2.17).

  ) and (5.1.18) can be transformed into quadratic constraints by applying Lemma 5.1.1. The same lemma then allows us to conclude the non-convexity of the model involving valuation (5.1.16). Lemma 5.1.1. Let f, g, h : R n → R be finite-valued mappings. Then the following problems are equivalent: min x∈R n f (x) : g(x) ≤ [h(x)] + (5.1.19) and min x∈R n ,λ≥0 {f (x) : (g(x)h(x) + λ)g(x) ≤ 0} (5.1.20)

Figure 5 . 1 : 6 Figure 5 . 2 :

 51652 Figure 5.1: Trajectories of filling levels in reservoir "Saut Mortier" and instance 2 for 100 simulated inflow scenarios. From top left to bottom right, solutions of problems (5.1.24), (5.1.25), (5.1.27), (5.1.28), (5.1.28)-Calib and (5.1.26)

  2.1), but min x∈Rk c T x + 1 2 x T Qx s.t. P[A r x + a r ≤ η ≤ A r x + b r ] ≥ p (5.2.16) Ax ≤ b,

Proposition 5 . 2 . 1 .

 521 Let ξ ∈ R m be a random variable inducing a symmetric γ-concave probability measure for any γ ∈ [-∞, ∞]. Here symmetric means that P[ξ ∈ A] = P[(-ξ) ∈ A] for any Lebesgue measurable set A. Then for any b ∈ R m with b ≥ 0, and α ∈ R m it holds that P[-b + α ≤ ξ ≤ b + α] ≤ P[-b ≤ ξ ≤ b]. Proof. Let α, b ∈ R m with b ≥ 0 be arbitrary and define A := {y ∈ R m : -b ≤ y ≤ b}. It follows from symmetry and the definition of A that P[-b + α ≤ ξ ≤ b + α] = P[ξ ∈ (A + α)] = P[ξ ∈ (Aα)].

5. 2 . 4

 24 Numerical Example: a (simplified) Robust Unit-Commitment Problem 5.2.5 Robust System-Dispatch Problem

  Figures 4.1(a) and 4.1(b) provide the hydro valley data. Thermal plant data is given in table 5.3.

Figure 5 . 3 :

 53 Figure 5.3: Iteration Number vs. xy 2 for Method 5.2.2.2

6 Figure 5 . 4 :

 654 Figure 5.4: Turbined volumes (m3 ) for each reservoir of the Ain Valley. From top left to bottom right, solutions produced by Reduced Lagrangian Dual ( §5.2.2.1.1), Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2), Reduced Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2.2), Reduced Augmented Lagrangian with Bundle Updating and Reduced Augmented Lagrangian + DQA ( §5.2.2.3)

Figure 5 . 7 :

 57 Figure 5.7: Deviations from Load. 100 Scenarios for η in eq.(5.2.22) and lower/upper bounds derived from each of the solutions. Reduced Lagrangian Dual (RLD), Augmented Lagrangian + Auxiliary Problem principle (AL+APP), Reduced Augmented Lagrangian + Auxiliary Problem principle (RAL+APP), Reduced Augmented Lagrangian with Bundle Updating (RAL+APP+Bdl) and Reduced Augmented Lagrangian + DQA (RAL+DQA).

  2. Under notation of Definition 3.1.1, we say that x ∈ R n is individual chance constrained feasible (ICC-feasible) if and only if

  Upon redefining a and b we may assume without loss of generality that µ = 0. A vector x ∈ R n satisfies (3.1.11) if and only if there exists y ∈ R m , y = Ax and P[a + y ≤ ξ ≤ b + y] ≥ p. It is thus sufficient to consider the case A = I. Let i = 1, ..., m be arbitrary but fixed and let y ∈ R m be e i -ICC feasible. By definition we have P[a i + y

	Lemma 3.1.10. Let g appearing in (3.1.1) be defined in such a way that (3.1.1) is
	equivalent with:
	(3.1.11) where a, b ∈ R m and the m × n matrix A are deterministic. Let ξ ∈ R m be a non-P[a + Ax ≤ ξ ≤ b + Ax] ≥ p, degenerate multivariate Gaussian random variable with mean µ and variance-covariance
	matrix Σ. Then for each p > 1 2 there exist y(p), y(p) ∈ R m such that
	M I (p) = x ∈ R n : y(p) ≤ Ax ≤ y(p) .	(3.1.12)
	Moreover the set [y(p), y(p)] is bounded.
	Proof.

  by 2. in Lemma 3.3.5. The arguments of Lemma 3.3.9 allow us to deduce that for k sufficiently large the estimates (3.3.10) and (3.3.12) still hold. Using(3.3.35), we may combine Corollary 3.3.24 with (3.3.10) and (3.3.12) in order to derive that

  Remark 3.4.22. The advantage of Theorem 3.4.19 over Theorem 3.4.18 is that it reinserts the dependence of p * on the Copula and not just its generalized concavity property. We will illustrate this effect in later examples. If we call p * 1 the critical level obtained in Theorem 3.4.18 and p * 2 that of Theorem 3.4.19, then it follows always p * 2 ≤ p * 1 . Indeed picking p ≥ p * 1 , x ∈ M (p) one derives from equation (3.4.10) that x ∈ D, i.e., x ∈ M (p) ∩ D. The latter set is shown to be convex whenever p ≥ p * 2 , implying p * 1 ≥ p * 2 .

  1.1 is devoted to bundle methods for upper inexact oracles. After giving the initial bundle setting in § 4.1.1.1, the attenuation step required for the method to converge in the presence of oracle noise is explained in § 4.1.1.2. The new bundle algorithm is given in full details in § 4.1.1.3. Based on Section 4.1.2 asymptotic results, Section 4.1.3 proves convergence of the method to a solution of (4.1.1), up to the accuracy provided by the oracle. In particular, we show that for lower oracles that are asymptotically exact, the method finds an exact minimizer whenever (4.1.1) has a Slater point. Since our setting is more general than previous work (cf. the discussion in § 4.1.4.1), our approach significantly generalizes and extends results in the literature. The specific unit-commitment application is considered in the last two sections. Section 4.1.5 describes the hydro valley considered for the numerical tests, formulates the joint chance constrained problem to be solved, and discusses the upper inexact oracle employed. The different solvers used for comparison and a thorough set of numerical tests showing the interest of the approach are given in Section 4.1.6.

  .1.15) Only when noise is declared acceptable, that is when(4.1.14) does not hold, the algorithm examines if the new iterate is good enough to become the next center by checking When the relation holds, the iteration is declared serious, because it provides a new algorithmic center: xk+1 = x k+1 . Otherwise, the center is maintained and the iteration is declared null.The rationale behind (4.1.16) is to measure progress towards minimization of (4.1.1) by focusing either in reducing the objective value without losing feasibility if the center is approximately feasible. Otherwise, when ĉk > 0, the emphasis is put in reducing infeasibility by checking the second condition in(4.1.16).

either if f k+1 ≤ f kmδ k and c k+1 ≤ 0 when ĉk ≤ 0 , or if c k+1 ≤ ĉkmδ k when ĉk > 0 . (

4

.1.16)

  xk 2 and, therefore, E k ≤ δ k at all iterations (4.1.17) because α k ≤ 2, by (4.1.15).

	Second, by adding δ k to both sides of the identity (4.1.17), we see that

inequality (4.1.14) holds ⇐⇒ δ k + E k < -(α k + 2β k ) 2 µ k x k+1xk 2 . (

4

.1.18) 4.1.1.3 A Non-smooth Optimization Solver for Inexact Oracles

  Step 3 (Stopping Test and New Oracle Information) If δ k+1 ≤tol then stop. Otherwise call the oracle to obtain f k+1 , c k+1 , g f k+1 and g c k+1 .Step 4 (Serious step test) Check the descent condition (4.1.16). If this condition is true, declare a serious iteration and set xk+1 = x k+1 . Otherwise, declare a null step and maintain the center: xk+1 = xk . Step 5 (Bundle Management and updates) Choose a new prox-parameter µ k+1 satisfying (4.1.19a) if the iteration was declared serious or satisfying (4.1.19c) whenever the iteration was declared null. In all cases choose parameters α k+1 , β k+1 satisfying (4.1.15) and penalties ρ k+1 , σ k+1 satisfying (4.1.20) below. Define the new bundle B k+1 , for example by appending to the index set the last iterate information: J k+1 = J k ∪ {k + 1}. Increase k by 1 and loop to Step 1. Both in Step 2 and Step 5 there is some freedom in the choice of the new bundle B k+1 . When noise is excessive, as in Step 2, the conservative choice of keeping the same cuttingplane models for both f and c seems reasonable. Alternative choices for managing the bundle in Step 5 are discussed after Lemmas 4.1.2, 4.1.3, and 4.1.4, for the cases of serious, noisy, and null iterations, respectively.

1 2 µ k yxk 2 : y ∈ X}. Define the predicted decrease δ k+1 as in (4.1.13). Step 2 (Noise attenuation test) If condition (4.1.14) is true, noise is too large: decrease the prox-parameter as in (4.1.19b) below; maintain the center, the bundle, and the penalties: xk+1 , B k+1 , ρ k+1 , σ k+1 = xk , B k , ρ k , σ k ; choose parameters α k+1 , β k+1 satisfying (4.1.15), and loop to Step1. Otherwise, if (4.1.14) does not hold, proceed to Step 3.

  Theorem 4.1.5 (Asymptotic Bounds for Upper Oracles). Consider solving (4.1.1) with Algorithm 4.1 using an oracle (4.1.3) such that (4.1.21) and (4.1.31) hold, with parameters α k , β k satisfying (4.1.15) and penalties as in (4.1.20). Suppose the prox-parameter is updated according to (4.1.19), and the model satisfies (4.1.26).

.

1.31)

  mδ k the iteration is declared serious (4.1.35) (noting that the test (4.1.16) is also mentioned in [129] as a possibility). In view of the second item in Proposition 4.1.1, the criterion (4.1.16) is stronger for null steps. Depending on the problem, one criterion or the other might be preferable. By checking the proofs of Lemma 4.1.2 and 4.1.4, it is not difficult to see that the asymptotic results in Section 4.1.2 still hold with the alternative test.

Table 4 .

 4 best objective function value in this test was -104162, found by Alg.PB.SEV for different settings. When K = 1e 4 Alg.PB.SEV found in 239 minutes a similar 3 solution than Alg.[START_REF] Prékopa | Probabilistic programming[END_REF], which took approximately the same CPU time. Taking K = 5e 4 the same point is reached in only 193 minutes and with K = 1e 4 that point is reached in 178 minutes. Table4.1 shows the importance of the scaling parameter in the bundle method. It also shows that the supporting hyperplane method reaches good points early, but takes a very long time to converge. Basically each iteration takes approximately 15 to 20 minutes. Each bundle iteration takes less time (approximately 3 to 4 minutes per The modification was done to prevent the methods from stalling in the early stages of the algorithmic process and can only be active a finite number of iterations. Alg.PB.SEV does not stall and needs no modification. Once more, this variant is the best one in the benchmark, since it finds a feasible point with objective function value -104153 in only 56 iterations (comparable to the other two algorithms in the comparison). The best objective value of -104162 was obtained at the stake of many iterations. This table shows that important speed ups can be gained when starting with infeasible points. In particular, Alg.[START_REF] Prékopa | Probabilistic programming[END_REF] requiring a Slater point and reaching a feasible solution with objective function value -104154 in 2079.17 minutes (see Table 4.1) is definitely out-performed by the Bundle method settings of Table 4.2. 1: Comparison of Algorithms (nne x stands for nn10 x ), assuming a Slater Point available.

	method	Obj. Value Nb. Iter. CPU time	parameters
				(mins)
	Alg.[182]	-103197	17	247.7	tol = 1e -2
	Alg.[182]	-104070	45	940.3	tol = 1e -3
	Alg.[182]	-104154	94	2079.17	tol = 1e -4
	Alg.PB.SEV	-104162	294	1028.43	K = 1e 5 , µ 0 = 1e -6 , µ s = 2, κ = 0.7
	Alg.PB.SEV	-104160	300	1060.35	K = 2e 5 , µ 0 = 1e -6 , µ s = 2, κ = 0.7
	Alg.PB.SEV	-104162	286	991.33	K = 5e 4 , µ 0 = 1e -6 , µ s = 1.05, κ = 0.95
	Alg.PB.SEV	-104162	243	846.20	K = 5e 4 , µ 0 = 1e -6 , µ s = 2, κ = 0.7
	Alg.PB.SEV	-104160	215	723.55	K = 5e 4 , µ 0 = 1e -6 , µ s = 2, κ = 0.25
	Alg.PB.SEV	-104162	255	887.07	K = 5e 4 , µ 0 = 1e -5 , µ s = 2, κ = 0.1
	Alg.PB.SEV	-104162	257	907.20	K = 5e 4 , µ 0 = 1e -6 , µ s = 2, κ = 0.1
	Alg.PB.SEV	-104162	239	827.26	K = 5e 4 , µ 0 = 1e -6 , µ s = 4, κ = 0.5
	Alg.PB.SEV	-104162	265	932.19	K = 1e 4 , µ 0 = 1e -6 , µ s = 2, κ = 0.7
	Alg.PB.NUL	-104109	417	1686.12	K = 1e 5 , µ 0 = 1e -6 , µ s = 1.05, κ = 0.95
	Alg.PB.NUL	-104096	402	1539.16	K = 5e 4 , µ 0 = 1e -6 , µ s = 1.05, κ = 0.95
	Alg.PB.NUL	-104102	395	1639.40	K = 5e 4 , µ 0 = 1e -6 , µ s = 2, κ = 0.7
	Alg.PB.NUL	-104089	277	1106.43	K = 5e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.25
	Alg.PB.NUL	-104091	323	1257.09	K = 5e 4 , µ 0 = 1e -6 , µ s = 4.0, κ = 0.5
	Alg.PB.NUL	-104078	366	1480.21	K = 1e 4 , µ 0 = 1e -6 , µ s = 1.05, κ = 0.95
	Alg.PB.NUL	-104077	395	1591.13	K = 1e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.7
	Alg.PB.NUL	-104078	278	1063.49	K = 1e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.5
	Alg.PB.NUL	-104076	234	887.38	K = 1e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.25
	Alg.PB.NUL	-104078	253	981.13	K = 1e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.35
	Alg.PB.NUL	-104079	144	549.19	K = 1e 4 , µ 0 = 1e -5 , µ s = 2.0, κ = 0.1
	Alg.PB.NUL	-104072	208	792.17	K = 1e 4 , µ 0 = 1e -6 , µ s = 2.0, κ = 0.1
	Alg.PB.NUL	-104078	218	828.58	K = 1e 4 , µ 0 = 1e -7 , µ s = 2.0, κ = 0.1
	Alg.PB.NUL	-104078	278	1126.51	K = 1e

iteration). This can be explained by the fact that the supporting hyperplane method computes the exact interpolation. Now typically the cutting-plane method would yield a new iterate x k far from the probability boundary P[a r + A r x ≤ ξ ≤ b r + A r x] = p and it would seem that Genz' code takes more time on such points than on strictly feasible points. It is also interesting to note that each stability center from the bundle methods is feasible. Thereby empirically providing support for the initial discussion in Section 4.1.5.2. Since in this setting Alg.

[START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF]

and Alg.PB.NUL would have similar results, we did not include them in this comparison.

The results in Table

4

.2 were obtained with an infeasible starting point, x0 = x d from (4.1.41), using for each algorithm the best parameter settings in Table

4

.1.

For these runs, both Alg.PB.NUL and Alg.

[START_REF] Kiwiel | A method of centers with approximate subgradient linearizations for nonsmooth convex optimization[END_REF]

modify the serious step test in Step 4, as follows. When the current stability center is infeasible, in addition to the usual test for acceptance, any improvement in feasibility by at least the c-oracle precision ("Genz precision" ε g ) declares the current iterate a new stability center.

The comparison above shows a strong dependence of the CPU time on the initial point,

  4

Table 4 .

 4 2: Comparison of Algorithms (nne x stands for nn10 x ), infeasible starting point. Precision of Genz' code ε g = 1e -4 .

	method	Obj. Value Nb. Iter. CPU time (mins)	parameters
	Alg.PB.SEV	-104162	133	379.45	K = 5e

Table 4 .

 4 3: Effect of "Genz Precision" (nne x stands for nn10 x ), Alg.PB.SEV with K = 5e 4 , µ 0 = 1e -6 , µ s = 2.0κ = 0.5.

	Starting Point Genz Precision objective Value Nb. Iterations CPU time (mins)
	Feasible	1e -2	-104177	216	83.55
	Feasible	1e -3	-104163	216	87.13
	Feasible	1e -4	-104162	222	722.59
	Feasible	1e -5	-104161	261	54390.5
	Infeasible	1e -2	-104177	71	27.48
	Infeasible	1e -3	-104163	108	41.36
	Infeasible	1e -4	-104162	128	338.37

Table 4 .

 4 4: Effect of noise test (nne x stands for for nn10 x ), Alg.PB.SEV with K = 5e 4 , µ 0 = 1e -6 , µ s = 2.0κ = 0.5, tol = 0.5, and Genz precision 5e-4

	Starting Point	α	β	objective Value Nb. Iterations CPU time (mins)
	Infeasible Infeasible	0 0	-1 + ε m 0	-104160 -104157	88 87	21.33 21.0
	Infeasible Infeasible Infeasible Infeasible	0 1 1 2 -2ε m -1 + ε m 1 -ε m -1 + ε m -ε m	-104159 -104158 -104158 -104077	111 60 70 24	30.30 12.8 15.57 5.35

Table 4 .

 4 5: Effect of Descent test & Stopping Criteria (αe x stands for α10 x ). Alg.PB.SEV with parameter setting K = 5e 4 , µ 0 = 1e -6 , µ s = 2.0κ = 0.5. Precision of Genz code is taken to be 5e -4 , ρ k = 0 α = 0, β = -1 + ε m .

	Starting Point Descent Test	Stopping Criteria	objective Value Nb. It CPU time (mins)
	Infeasible Infeasible Infeasible	(4.1.16) (4.1.35) (4.1.16)	δ k ≤ 0.5 δ k ≤ 0.5 (4.1.36) ,tol = 1e -4	-104160 -104160 -104153	93 93 69	23.33 22.49 16.38
	Infeasible	(4.1.16)	(4.1.37) ,tol = 1e -4	-104156	70	17.39

One can observe that in our specific case the choice of Descent Test (4.1.16) or (4.1.35) makes no difference. This can be intuited since for a currently feasible stability center with feasible candidate the check (4.1.35) is indeed identical to that of

(4.1.16)

. Since this situation seems to occur most often in this application, the result of Table

4

.5 are indeed natural. It could however be argued that descent test

(4.1.16

Table 4 .

 4 6: Effect of updating ρ k and the noise test (αe x stands for α10 x ). Alg.PB.SEV with parameter setting K = 5e 4 , µ 0 = 1e -6 , µ s = 2.0κ = 0.5, tol = 0.5. Precision of Genz code is taken to be 5e-4

	Starting Point	α	β	objective Value Nb. Iterations CPU time (mins)
	Infeasible Infeasible	0 0	-1 + ε m 0	-104160 -104160	93 131	23.33 31.3
	Infeasible Infeasible Infeasible Infeasible	0 1 1 2 -2ε m -1 + ε m 1 -ε m -1 + ε m -ε m	-104157 -104158 -104159 -104095	115 72 68 27	32.40 16.31 15.57 6.57
	solving problem (4.1.41) and Alg.PB.SEV. The figure clearly shows the importance of
	integrating uncertainty in order to obtain robust turbining strategies.

  has a feasible set that is convex for d-dimensional random variables ξ with log-concave probability distribution (but in many other settings too). In this formulation, A is a fixed matrix and a, b are fixed vectors with appropriate dimensions. Evaluating the constraint and computing a gradient with high precision is possible but potentially very costly, especially when the dimension d is large (say d ≥ 100).

	Example 4.2.2 (Chance-constrained programming). Given a probability level p, the
	optimization problem
	 	min q, x
		s.t. P[Ax + a ξ Ax + b] p	(4.2.4)
		x ∈ X
	By a classical result (Theorem 2.5.13), not only the feasible set induced by the joint chance
	constraint in (4.2.4) is convex but also the function
	log(p) -log(P[Ax + a ξ Ax + b]) .	(4.2.5)
	Therefore, problem (4.2.4) corresponds to (4.2.2) by taking

  Step 1 (Best Current Minimizer) Update the recorded sequence x k rec according to (4.2.15). Step 2 (Stopping Test) Perform the stopping test h k (x k rec ) δ Tol . If satisfied return x k rec and f (x k rec ). Step 3 (Descent Test) In this optional Step we test if h k

  Step 6 (Oracle) Call the oracle to obtain (f (x k+1 ), g f k+1 ) and (c(x k+1 ), g c k+1 ). Set f k+1 low ← f k low and xk+1 ← xk .Step 7 (Bundle Management) Manage the bundle freely as long as J k+1 Set k = k + 1 and go to Step 1. Let K l be the index set belonging to the l-th cycle, it then follows that both the stability center xk and lower bound f k low are fixed for all k ∈ K l . As a consequence, for each fixed l ≥ 0, the sequence {f j lev } j∈K l is nonincreasing. (4.2.16)This property is essential for Lemma 4.2.7.

	f ⊃ {k + 1, a c (k)} remain true at each bundle compression. Step 8 (Loop) We can provide the following remarks concerning Algorithm 4.2: 1, a f (k)} and J k+1 c	⊃ {k +
	rec )} is monotonically decreasing by (4.2.8) and (4.2.15). (a) The sequence {h k (x k (b)

Return to Step 1. If problem (4.2.10) is solved then move to Step 6.

  .2.24) Since both functions f and c in (4.2.2) are convex, the feasible set X is compact, and the oracle errors in (4.2.21) are bounded (it follows from (4.2.21) and (4.2.22) that -η f

	ǫ x f such that ǫ f and -η c ǫ x c	ǫ c ), [114, Prop.XI.4.1.2] ensures that there exists a constant Λ
		Λ max{| gf	x |, | gc	x |}; for all x ∈ X .	(4.2.25)

  Lemma 4.2.11. Consider the improvement function h k i given in (4.2.26). If there exists a sequence {x k rec } ⊂ X such that lim k h k i (x k rec ) 0, then any cluster point of the sequence {x k rec } is a η-solution to problem (4.2.2), with a possibly unknown error η := max{η f + ǫ f , η c }. Proof. By (4.2.24) the lower bound updating at Step 5.2 ensures that f k low f min + ǫ f .

	If lim k h k i (x k rec ) 0, it follows by (4.2.26), the definition of η and oracle assumptions that

.2.26)

Due to inaccuracy from the oracle, we can have h k i (x) < 0.

  .2.31) As before, the upper bounds η f and η c in (4.2.22) can be unknown. Lemma 4.2.15. For lower oracles, the updating procedure of f k low at Step 5 of any version of Algorithm 4.2 assures f k low f min . Moreover, Lemma 4.2.11 holds with η

Table 4 .

 4 7: Comparison of Algorithms (nne x stands for nn10 x ). Precision of Genz' code ε g = 5e -4 . The number of iterations at which (4.2.10) is found infeasible is indicated in between brackets.

	method	Obj. Value Nb. Iter. [¬(4.2.10)] CPU time parameters
				(mins)
	Alg.[182]	-103363	16	7.2	δ Tol = 1e -2
	Alg.[182]	-104077	49	25.5	δ Tol = 1e -3
	Alg.[182]	-104156	99	43.9	δ Tol = 1e -4
	Alg.PB	-104160	94	13.6	K = 5e 4
	Alg.PB	-104160	91	12.3	K = 1e 4
	Alg.PB	-104159	99	14.4	K = 1e 5
	Alg.PB	*	*	-	K = 1e 3
	Alg.LB	-104159	90 [2]	11.4	K = 5e 4 , γ = 0.9, δ Tol = 5
	Alg.LB	-104158	94 [3]	12.3	K = 1e 4 , γ = 0.9, δ Tol = 5
	Alg.LB	-104159	81 [2]	10.4	K = 1e 5 , γ = 0.9, δ Tol = 5
	Alg.LB	-104160	58 [4]	6.5

Table 4 .

 4 9: Effect of adding (4.2.17) to Step 5. (nne x stands for nn10 x ). Precision of Genz' code ε g = 5e -4 .

	method Obj. Value Nb. Iter. CPU time parameters
				(mins)
	Alg.LB	-104161	35	4.4	K = 1e 3 , γ = 0.8, δ Tol = 5
	Alg.LB	-104158	65	9.2	K = 5e 4 , γ = 0.8, δ Tol = 5

Table 4 .

 4 10: Effect of precise Oracle. (nne x stands for nn10 x ). Precision of Genz' code ε g = 1e -4 .

	method	Obj. Value Nb. Iter. CPU time parameters
				(mins)
	Alg.[182]	-104154	107	961.5	δ Tol = 1e -4
	Alg.PB	-104160	89	71.2	K = 5e4
	Alg.LB	-104162	33	24.2	K = 1e 3 , γ = 0.8, δ Tol = 5

Table 4 .

 4 11: Comparison of Alg.PB and Alg.LB on the Isère Valley. (nne x stands for nn10 x ). Precision of Genz' code ε g = 5e -4 . The number of iterations at which (4.2.10) is found infeasible is indicated in between brackets. The indication [¬(4.2.17)] means that auxiliary problem (4.2.17) is not solved.

	Instance method	Obj. Value Nb. Iter.	CPU time parameters
				[¬(4.2.10)]	(mins)
	Isr48	Alg.[182]	-175031	35	10.5 δ Tol = 1e -4
	Isr48 Isr48 Isr48 Isr48	Alg.PB Alg.PB Alg.PB Alg.LB	-175043 -175043 -175042 -175039	115 88 69 66	17.5 K = 1e 5 , µ 0 = 1e -5 , α = 0, β = -1 11.2 K = 1e 5 ,µ 0 = 1e -5 , α = 1, β = -1 8.3 K = 1e 5 ,µ 0 = 1e -6 , α = 1, β = -1 10.0 K = 1e 5 , γ = 0.8, δ Tol = 5
	Isr48	Alg.LB	-175040	38	5.4 K = 1e 4 , γ = 0.8, δ Tol = 5
	Isr48	Alg.LB	-175040	63 [3]	8.6 K = 1e 5 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]
	Isr48	Alg.LB	-175037	38 [4]	5.2 K = 1e 4 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]
	Isr96	Alg.[182]	-175708	143	217.4 δ Tol = 1e -4
	Isr96 Isr96 Isr96 Isr96	Alg.PB Alg.PB Alg.PB Alg.LB	-175714 -175715 -175715 -175713	214 159 177 122	152.3 K = 1e 5 ,µ 0 = 1e -5 , α = 0, β = -1 110.9 K = 1e 5 ,µ 0 = 1e -5 ,α = 1, β = -1 123.5 K = 1e 5 ,µ 0 = 1e -6 ,α = 1, β = -1 82.5 K = 1e 5 , γ = 0.8, δ Tol = 5
	Isr96	Alg.LB	-175713	94	48.4 K = 1e 4 , γ = 0.8, δ Tol = 5
	Isr96	Alg.LB	-175710	115 [3]	75.3 K = 1e 5 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]
	Isr96	Alg.LB	-175697	76 [4]	44.3 K = 1e 4 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]
	Isr168	Alg.[182]	-175222	190	1504.7 δ Tol = 1e -4
	Isr168 Isr168 Isr168 Isr168	Alg.PB Alg.PB Alg.PB Alg.LB	-175236 -175237 -175237 -175235	284 219 188 161	888.4 K = 1e 5 ,µ 0 = 1e -5 ,α = 0, β = -1 687.4 K = 1e 5 ,µ 0 = 1e -5 ,α = 1, β = -1 573.5 K = 1e 5 ,µ 0 = 1e -6 ,α = 1, β = -1 529.6 K = 1e 5 , γ = 0.8, δ Tol = 5
	Isr168	Alg.LB	-175232	110	352.3 K = 1e 4 , γ = 0.8, δ Tol = 5
	Isr168	Alg.LB	-175235	165 [3]	423.2 K = 1e 5 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]
	Isr168	Alg.LB	-175220	127 [5]	353.5 K = 1e 4 , γ = 0.8, δ Tol = 5, [¬(4.2.17)]

Table 4 .

 4 12: Effect of precise Oracle. (nne x stands for nn10 x ). Precision of Genz' code ε g = 1e -4 .

	Instance method	Obj. Value Nb. Iter. CPU time parameters
					(mins)
	Isr48	Alg.[182]	-175037	39	58.5	δ Tol = 1e -4
	Isr48 Isr48	Alg.PB Alg.LB	-175041 -175039	81 39	35.1 16.2	K = 1e5,µ 0 = 1e -6 ,α = 1, β = -1 K = 1e 4 , γ = 0.8, δ Tol = 5

  only incited to turbine if this allows us to improve robustness) and most reservoirs, except for "Saut Mortier". This reservoir has tight volume bounds and is most heavily impacted by the stochastic inflows. The solution (5.1.25) turbines a bit less in the beginning to avoid violations in time steps 8-10, a bit more during time steps 12-15 to avoid violations there and stops earlier to avoid violations for the last time steps. Solution (5.1.27) offers an intermediate solution. The solution (5.1.28) heavily increases turbining during steps 10-15 and drastically reduces during steps 15-20 for additional robustness. Indeed, even though the uncertainty E p is very well calibrated, the solution is over-robust. Unfortunately for larger values of p (in fact p > 0.85) this will lead to an empty feasible set of problem (5.1.28), whereas solutions of (5.1.25) can be found. It also shows the difficulty of getting the robust rectangle well calibrated for problem (5.1.28)-Calib. Indeed, even though the rectangle is calibrated to give exactly the same probabilistic contents in both instances, one gives over-robust results (3.6% away from deterministic solution), whereas the other gives more reasonable results as the number of violations is concerned, but still at a large cost (2.2 % away from deterministic solution).

	and is hence
		1 0.5	4.24 0
	3	0.5 1	0 0.3

  2.4) is dealt with in the form log(p)-log(P[s d ≤ D-A l x ≤ s u ]) ≤ 0 , D follows a non-degenerate multivariate Gaussian distribution, and either f is linear or α = 1, it follows from[START_REF] Lemaréchal | A geometric study of duality gaps, with applications[END_REF] Theorem 4.4] that maximizing (5.2.10) might increase the duality gap with respect to a direct dualization of the probabilistic constraint under the above form. Since, unit-commitment problems generally have a non-convex set X 1 , neither Theorem 4.4 or Theorem 4.5 of[START_REF] Lemaréchal | A geometric study of duality gaps, with applications[END_REF] can be used and the duality gap may either increase or decrease. Since constraint (5.2.4) is highly non-linear, this might be the price to pay in order to decompose the Lagrangian dual over the m units.5.2.2.1.1 Dimension ReductionOne difficulty in the formulation of problem (5.2.8), and its dual "maximize (5.2.10)" is the dimension of the dual signal µ. Indeed, it is of the global problem dimension n, which might be (very) large. This may in particular imply some difficulties for convergence of non-smooth optimization tools such as Bundle methods in order to maximize (5.2.10). In our setting we know more about the structure of the X 2 set, since X 2 := {x ∈ R n : s.t (5.2.4) holds }. Let T be the size of the random vector D, i.e., D ∈ R T . Now it is clear that x ∈ X 2 if and only if z

Table 5 .

 5 3: Thermal Plant data

	Data Type				Plants
	Initial Generation 700 700 700	150	150	0	0	0	0
	Cost	30	35	37	45	47	60 100 110 150
	Max Power	900 900 900	300	300 200 200 200 100
	Gradient	100 100 100	30	30	20	20	20	10

Table 5 .

 5 4: Results for the Lift and Dualize Method

	Method	objective Value Nb. Iterations CPU time (mins)
	Reduced L&D 5.2.2.1.1	3.69963e 6	280	62.5

Table 5 .

 5 5: Results for the Lift and Dualize -Augmented Lagrangian Methods (The objective function value is shown if a feasible solution is generated) , 2e -2 , 1e -2 , 0.4)

	Method	Obj. Value Nb. Iter. CPU (mins)	Parameters (c, ĉ, ρ, ε)
	AL APP 5.2.2.2	3.9304e 6	5000*	484.54	(1.8e -2 , 3.6e -2 , 1.8e -2 , 1.0)
	AL APP 5.2.2.2	3.9283e 6	5000*	593.43	(0.1, 0.5, 0.1, 1.0)
	AL APP 5.2.2.2	3.9288e 6	5000*	475.47	(1, 5, 0.5, 1.0)
	AL APP 5.2.2.2	3.9410e 6	5000*	508.56	(10, 20, 5, 1.0)
	AL APP 5.2.2.2	-	5000*	553.3	(1e -3 , 2e -3 , 1e -3 , 1.0)
	AL APP 5.2.2.2	3.9213e 6	4646	429.0	(2e -2 , 5e -2 , 2e -2 , 1.0)
	AL APP 5.2.2.2	3.9172e 6	5000*	528.39	(1, 2, 2e -2 , 1.0)
	AL APP 5.2.2.2	3.9162e 6	5000*	600.42	(0.5, 1.0, 0.02, 1.0)
	AL APP 5.2.2.2	3.9252e 6	5000*	824.17	(1.8e -2 , 1.8e -2 , 1.8e -2 , 0.4)
	AL APP 5.2.2.2	3.9049e 6	5000*	449.46	(2e -3 , 2e -3 , 2e -3 , 0.4)
	AL APP 5.2.2.2	3.9285e 6	5000*	682.18	(2e -1 , 2e -1 , 2e -1 , 0.4)
	AL APP 5.2.2.2	-	5000*	455.22	(2e -2 , 2e -2 , 3e -2 , 0.4)
	AL APP 5.2.2.2	3.9342e 6	5000*	689.11	(2, 2, 2, 0.4)
	AL APP 5.2.2.2	3.9339e 6	5000*	527.45	(2e -1 , 2e -1 , 2e -1 , 0.1)
	AL APP 5.2.2.2	3.9248e 6	5000*	524.27	(2e -1 , 2e -1 , 1e -1 , 0.4)
	AL APP 5.2.2.2	3.9250e 6	5000*	541.13	(2e -2

Table 5 .

 5 6: Results for the Lift and Dualize -Augmented Lagrangian Methods -Reduced (The objective function value is shown if a feasible solution is generated)

	Method	Obj. Value Nb. Iter. CPU (mins)	Parameters (c, ĉ, ρ, ε)
	AL APP 5.2.2.2.2	-	5000*	300.32	(2e -2 , 4e -2 , 2e -2 , 1.0)
	AL APP 5.2.2.2.2	-	5000*	280.40	(2e -2 , 2e -2 , 2e -2 , 0.4)
	AL APP 5.2.2.2.2	-	5000*	407.09	(2e -2 , 5e -2 , 2e -2 , 1.0)
	AL APP 5.2.2.2.2	-	5000*	349.47	(2e -2 , 1e -1 , 2e -2 , 1.0)
	AL APP 5.2.2.2.2	-	5000*	323.50	(2e -2 , 5e -2 , 2e -2 , 0.4)
	AL APP 5.2.2.2.2	-	5000*	318.19	(0.1, 1.0, 0.05, 1.0)
	AL APP 5.2.2.2.2	-	5000*	376.09	(4.0, 10.0, 0.5, 1.0)
	AL APP 5.2.2.2.2	-	5000*	331.28	(40.0, 100.0, 0.5, 1.0)
	AL APP 5.2.2.2.2	-	5000*	355.01	(4.0, 10.0, 1.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	412.09	(40.0, 100.0, 10.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	397.44	(4.0, 10.0, 2.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	345.23	(40.0, 100.0, 1.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	361.27	(10.0, 50.0, 2.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	337.46	(40.0, 100.0, 2.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	406.36	(4.0, 10.0, 3.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	373.22	(40.0, 100.0, 3.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	376.28	(40.0, 100.0, 5.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	stalls	(400.0, 1000.0, 25.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	450.51	(4.0, 10.0, 5.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	372.24	(40.0, 100.0, 6.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	371.4	(40.0, 100.0, 7.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	405.59	(4.0, 10.0, 6.0, 1.0)
	AL APP 5.2.2.2.2	-	5000*	368.15	(60.0, 150.0, 6.0, 1.0)
	AL APP 5.2.2.2.2	3.9400e 6	5000*	381.55	(50.0, 120.0, 6.0, 1.0)

Table 5 .

 5 7: Results for the Lift and Dualize -Augmented Lagrangian Methods -Reduced -Updating with Bundle Method (The objective function value is shown if a feasible solution is generated)

	Method	Obj. Value Nb. Iter. CPU (mins) Parameters (c, ĉ, ε)
	AL APP Bdl 5.2.2.2.2	3.9686e 6	296	1120.58	(4.0, 10.0, 1.0)
	AL APP Bdl 5.2.2.2.2	4.0390e 6	83	514.7	(2.0, 5.0, 1.0)
	AL APP Bdl 5.2.2.2.2	3.9321e 6	194	520.56	(1.0, 3.0, 1.0)
	AL APP Bdl 5.2.2.2.2	3.8576e 6	287	171.42	(0.5, 2.0, 1.0)

Table 5 .

 5 8: Results for the Diagonal Quadratic Approximation (The objective function value is shown if a feasible solution is generated)

	Method	Obj. Value Nb. Iter. CPU (mins) Parameters c
	AL DQA 5.2.2.3	3.89674e 6	30	321.59	1e -1
	AL DQA 5.2.2.3	4.00927e 6	21	318.26	2e -1
	AL DQA 5.2.2.3	3.70072e 6	208	284.55	2e -2

  3 ) for each reservoir of the Ain Valley. From top left to bottom right, solutions produced by Reduced Lagrangian Dual ( §5.2.2.1.1), Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2), Reduced Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2.2), Reduced Augmented Lagrangian with Bundle Updating and Reduced Augmented Lagrangian + DQA ( §5.2.2.3) Figure 5.5: Turbined volumes (m 3 ) for each reservoir of the Isere Valley. From top left to bottom right, solutions produced by Reduced Lagrangian Dual ( §5.2.2.1.1), Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2), Reduced Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2.2), Reduced Augmented Lagrangian with Bundle Updating and Reduced Augmented Lagrangian + DQA ( §5.2.2.3)

				18	x 10 4							18	x 10 4
				16								16
				14								14
				12								12
				10								10
				8								8
				6						turbine Res.1	6				turbine Res.1
										turbine Res.2					turbine Res.2
				4						turbine Res.3	4				turbine Res.3
										turbine Res.4					turbine Res.4
				2						turbine Res.5 turbine Res.6	2				turbine Res.5 turbine Res.6
										turbine Res.7					turbine Res.7
				0 0	5	10		15	20		0 0	5	10	15	20
	18	x 10 4						18	x 10 4							18	x 10 4
	16							16								16
	14							14								14
	12							12								12
	10							10								10
	8							8								8
	6					turbine Res.1	6						turbine Res.1	6	turbine Res.1
						turbine Res.2							turbine Res.2	turbine Res.2
	4					turbine Res.3	4						turbine Res.3	4	turbine Res.3
						turbine Res.4							turbine Res.4	turbine Res.4
	2					turbine Res.5 turbine Res.6	2						turbine Res.5 turbine Res.6	2	turbine Res.5 turbine Res.6
						turbine Res.7							turbine Res.7	turbine Res.7
	0 0	5	10	15	20		0 0	5	10		15	20		0 0	5	10	15	20

  Figure 5.6: Thermal Generation (M W ) for each thermal unit.From top left to bottom right, solutions produced by Reduced Lagrangian Dual ( §5.2.2.1.1), Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2), Reduced Augmented Lagrangian + Auxiliary Problem principle ( §5.2.2.2.2), Reduced Augmented Lagrangian with Bundle Updating and Reduced Augmented Lagrangian + DQA ( §5.2.2.3)

										900
										800
										700
										600
										500
									1	400					Power Th.1
									Power Th.2						Power Th.2
									Power Th.3	300					Power Th.3
									Power Th.4						Power Th.4
									Power Th.5	200					Power Th.5
									Power Th.6						Power Th.6
									Power Th.7 Power Th.8	100					Power Th.7 Power Th.8
									Power Th.9						Power Th.9
										0 0	5	10	15	20	25
	900						900						900
	800						800						800
	700						700						700
	600						600						600
	500						500						500
	400					Power Th.1	400					Power Th.1	400					Power Th.1
						Power Th.2						Power Th.2						Power Th.2
	300					Power Th.3	300					Power Th.3	300					Power Th.3
						Power Th.4						Power Th.4						Power Th.4
	200					Power Th.5	200					Power Th.5	200					Power Th.5
						Power Th.6						Power Th.6						Power Th.6
	100					Power Th.7 Power Th.8	100					Power Th.7 Power Th.8	100					Power Th.7 Power Th.8
						Power Th.9						Power Th.9						Power Th.9
	0 0	5	10	15	20	25	0 0	5	10	15	20	25	0 0	5	10	15	20	25

called linear matrix inequalities

(a i ,a j ) (D j m-1 D i m a, D j m-1 D i m b) ∀ ĵ = i

x

+y 2 +0.1 and h 2 (x, y) = 1 (x+y) 2 +0.1 from R 2 to R are chosen. We moreover pick any 0-0-concave Copula, i.e., some log-exp concave Copula, such as for instance the independent, Gumbel or maximum Copula. In order to be able to apply

Theorem 3.4.18 and derive convexity of the set M (p) for chance constraints structured as (3.4.2), we have to show that

See also section 5.1

In this case one requires the c-oracle to be exactly evaluated at the Slater point

i.e., feasible with same objective function value

That is: we multiply ρ k by two at each iteration when the current stability center is infeasible. We take ρ k = 0 whenever the current stability center is feasible. Since all following stability centers remain feasible from that moment onward there is no further effect of ρ k .

In practice, one would evaluate the difference of the final and initial volume. The latter adds a constant to the objective function and can theoretically be omitted. In practice, it may generate some numerical difficulties, especially when large volumes are valued and turbining/pumping capacity is small compared to the volume. In that case, relative changes in valuation induced by the controls are easily considered negligible. Moreover, the constant can easily be added.

Acknowledgements

List of Figures

The right figure provides a zoom near the optimal solution. . . . . . . . . case, the model is required to satisfy the following conditions whenever the iteration k is declared null:

M k+1 (y) ≥M k (y) and (4.1.26a)

The result below uses standard arguments in bundle methods [START_REF] Correa | Convergence of some algorithms for convex minimization[END_REF], taking advantage of the boundedness relations in (4. 1.21) and (4. 1.22) to adapt those arguments to the inexact improvement function setting. Suppose at iteration k there is a last serious step, denoted by x and after an iteration k > k there are no more noise attenuation steps: eventually only null steps occur. Let K n denote the set gathering indices of iterations larger than k. If there are infinitely many of such indices and both (4. 1.19c) and (4.1.26) hold, then

Proof. Consider k ∈ K n . Once again, the stated inequality will follow from (4.1.11) by boundedness of X, if we show that both G k + ν k → 0 and E k → 0. In turn, these results follow from showing that δ k → 0.

To see that δ k → 0, we start by expanding squares and using (4.1.7) to write the identity

where in the last equality we define

) from (4.1.8) (the value OV k is the optimal value of the null-steps.

The logarithm being a uniformly continuous mapping on [Φ, 1], X being a compact set and ϕ : X → [0, 1] also being uniformly continuous, it is clear that for any ε ′ > 0 a precision ε g can be chosen such that c(x k )c x k ≤ ε ′ for any iteration k. As discussed at the end end of Section 4.

By the Cauchy-Schwarz inequality and compactness of X we then obtain that

for an appropriate constant M > 0 and any y ∈ X. We conclude that (4.1.31) holds with ε ≥ ε ′ + M ε g . Moreover, whenever ε g is taken small enough, ε ′ can also be made arbitrarily small. We are in the conditions of Theorem 4.1.5. Since in the hydro valley application it is reasonable to assume the existence of a Slater point x s , we pick ε g , the user-defined precision of Genz' code in such a way that c(x s ) ≤ -2ε. This ensures that the set X ε in Theorem 4.1.5 is not empty. In the setting of the theorem, let x be the limiting point generated by Algorithm 4.1.

When x is infeasible according to the oracle information, c > 0 and item (i) of Theorem 4.1.5 leads to the contradiction c < c(x s ) + ε ≤ -ε < 0. Therefore, x is approximately feasible (c < 0) and satisfies the relation f ≤ f (y) + ε for all y ∈ X ε by item (ii) of the same theorem. Consequently, x approximately solves (4.1.1).

Numerical experience

For our numerical experience, the starting point uses x d , a solution to the linear program

the "deterministic" counterpart of (4.1.39) wherein the random vector ξ is replaced by its expectation. In general x d will not be feasible for (4.1.39) (unless it solves the problem).

Since improvement functions are scale-dependent, [START_REF] Sagastizábal | An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter[END_REF], it is useful to scale the constraint. Accordingly, we consider the constraint Kp ≤ KP[a r +A r x ≤ ξ ≤ b r +A r x] for some value of K > 0. A natural choice for K would be f, x d . Finally rules (4.1.19a), (4.1.19c) and (4.1.19b) are dealt with in the following way. In serious steps we take µ k+1 = µ k without making any changes. When noise is detected, we take µ k+1 = κµ k for a parameter κ ∈ (0, 1). Finally when null steps are made we choose µ k+1 = min {µ s µ k , µ max } for a parameter µ s > 1.

Step 5.2 Declare a critical iteration, set l

Proof. Given the assumptions Lemma 4.2.13 holds. Proceed similarly as in the proof of Theorem 4.2.8 to show that lim k h k i (x k ) 0, where f min should be substituted by f + η f . Here f is defined as f := min x :c(x)≤-ǫ c f (x). It is easily observed that f k low is a proven lower bound for f + η f and hence indeed h k i (x k rec ) must tend to zero. We can now use Lemma 4.2.11 to conclude.

Since the bounds given in Lemmas 4.2.7 and 4.2.13 are the same, we conclude that the complexity result of Theorem 4.2.9 also holds for inexact oracle satisfying (4.2.21). Up to our knowledge, general inexact oracles satisfying (4.2.21) have not been considered for (nonlinear) constrained level bundle methods. In [START_REF] Fábián | Bundle-type methods for inexact data[END_REF] and [START_REF] Fábián | Computational aspects of risk-averse optimisation in two-stage stochastic models[END_REF] only lower oracles (see Section 4.2.3.3 below) are considered. The work [START_REF] Van Ackooij | Constrained bundle methods for upper inexact oracles with application to joint chance constrained energy problems[END_REF] deals with upper oracles by applying a proximal bundle method. Theorem 4.2.14 assumes that the algorithm does not terminate. This is not the case when problem (4.2.2) is detected to be ǫ c -infeasible, i.e., {x ∈ X : čk (x) 0} = ∅ in Step 5.1.

In the following section we deal with ǫ c -infeasibility by making additional assumptions on problem (4.2.2) and oracle (4.2.21).

Inexact constrained level methods for upper oracles with on-demand accuracy

In this subsection we additionally assume that:

-problem (4.2.2) has a Slater point, i.e., there exists a point x s ∈ X such that c(x s ) < 0; -demanding more accuracy on the c-information from (4.2.21) is possible (the upper bound ǫ c in (4.2.22) can be decreased).

From (4.2.21) one directly derives that čk (x) c(x) + ǫ c for all x ∈ X. This shows that upper oracles have nonetheless (5.1.20), yields a feasible x for problem (5.1.19). It is easily seen that the quadratic constraint is y T Qy ≤ 0, with y := (x, z, λ). Clearly any vector (b, 0, 0) with b T a = 0 is an eigenvector with zero eigenvalue. This means that Q has at least n -1 zero eigenvalues. Picking v = (x, z, λ), equating Qv = µv, gives that x = -1 2 z µ a, λ = 1 2 z µ and shows that non zero eigenvalues µ must satisfy the equation µ 2µ -( 1 4 a T a + 1 4 ) = 0. Solving this equation yields the two non zero eigenvalues summing up to one as given in the lemma. This implies that we have identified n + 1 out of n + 2 eigenvalues. Since Tr(Q) = 1, the last eigenvalue has to be zero too. The non-convexity follows since one eigenvalue of Q is negative.

Objective function

Often, in reality, each reservoir only has a single turbine. The power output of turbining x, in cubic meters per second m 3 /s, is given by a function ρ(x). This function is strictly increasing and concave, i.e., ρ ′ (x) ≥ 0 and ρ ′′ (x) ≤ 0. In our model we have split this range into several subsections (hence several turbines), each with efficiency ρ i = ρ ′ (s * i )/3600 (M W h/m 3 ) for some s * i in each section. We can thus remark that for any two turbines i 1 and i 2 belonging to the same reservoir we have

This approximation comes down to approximating ρ(x) by a piece-wise linear function.

We assume given a time dependent price signal λ(t) (in e/MWh). The following objective function has to be minimized, when integrating the cost of used water according to equation (5.1.11):

y i (t)), (5.1.21) where, θ i (t) is the efficiency of pumping and the auxiliary variables z n i satisfy equations (5.1.9),(5.1.10).

Matrix formulation

In this section we show that (5.1.4) can be written as bilateral joint chance constraint. This means that the model we are interested in is a bilateral joint chance constrained program with linear objective function and some polyhedral constraints. and let Σ ζ be the block diagonal matrix containing 24 copies of Σ a . The expression for matrix Σ can be obtained by using matrix C(ψ) as defined above. As a result, if R is the 48 × 48 matrix with R i,mod(i-1,24)2+1+ ⌊ i-1 24 ⌋ = 1, it follows that

is the variance-covariance matrix of the global random inflow vector, that we shall shorten to ξ. The vector ξ is a Gaussian multi-variate random variable in dimension 48.

Solving the problems introduced in Section 5.1.2, we obtain the results as given in Table 5.1, Figures 5.1 and 5.2. We have set a tolerance of 10 -2 for the supporting hyperplane algorithm for joint chance constrained programming. It should be stated that the true optimal solution of problem (5.1.25) for instance 2 gives a cost, only 0.6% away from the deterministic cost. Indeed the price of chance-constrained robustness is cheap here.

Table 5.1 shows optimal costs and number of violations. In order to compute the latter information, we have made an a posteriori check of empirical probabilities by generating 100 scenarios and counting the number of violations. The volume trajectories resulting from these scenarios are shown in Figure 5.1. Clearly we observe the advantage of using joint chance constrained programming. The additional cost with respect to the deterministic solution is only small, but robustness can be fine tuned. A full robust solution turns out quite costly. Finally individual chance constrained programming can not be used to mimic joint chance constraints as we have no control over the number of violations over a period of time. Cost (e) -1.0478e 5 -1.0340e 5 -1.0422e 5 -1.0282e 5 -1.0251e 5 -9.9176e 4

The results of Table 5.1 are readily confirmed by examining the instance of figure 4.1(b) (see also section 4.2.4). We then get: When comparing the turbined volumes in Figure 5.2, one can observe that they are quite similar for most solutions (except for max-p which does not see the cost vector From Theorem 4.2.4 in [START_REF] Prékopa | Stochastic Programming[END_REF] it follows that f (α) :

where the mapping m γ is as in Definition 2.5.2 proving the Proposition. where the latter inequality holds by Proposition 5.2.1 for all x. Moreover we have equality whenever x ∈ X proving the optimality.