Tomas Dominguez

Jean-Christophe Mourrat

Statistical mechanics of mean-field disordered systems A Hamilton-Jacobi approach

Course outline

We describe here how the series of lectures in Zurich were organized. We do not believe that this is the optimal organization, and some of the results discussed in the book were not known at the time (in particular Theorem 6.12), but we hope that it can still be of some interest. This series of lectures comprised 12 sessions of 2 × 45 min. The first session was an overview of motivations and a description of the plan for the course. The second session started with a refresher on large deviations concluding with Theorem 2.16, followed by the contents of Section 1.1. The third session covered the rest of Chapter 1 and finished with Theorem 2.19. The fourth session covered Sections 3.1 and 3.2. The fifth session contained the proof of the comparison principle in Theorem 3.5 in the simpler case where the space domain R d is replaced by the torus, the statements of the variational formulas in Theorems 3.8 and 3.13 without proofs, and concluded with Corollary 2.20 recovered using the Hamilton-Jacobi approach. The plan for the sixth session was to cover Section 3.6, with a direct proof of the convex selection principle in Theorem 3.21 that bypasses Lemma 3.23, and also to present some material from Subsection 2.1.3 on subdifferentials. This actually spilled over to the seventh session. The rest of this session was spent discussing the setup of the statistical inference problem in Chapter 4, and motivating it informally in relation with the problem of community detection discussed in Section 4.5. The eighth session covered Section 4.1 and Section 4.3 up to (4.83), mostly skipping Section 4.2 except for the simple Gaussian integration by parts in (4.29) and some simple generalization of it. The ninth session was meant to cover the rest of Section 4.3, taking the concentration of the free energy for granted (any estimate stating that (4.101) tends to zero with N will do, and one can show an upper bound of the order of N -1 3 on this quantity using simple arguments based on the Efron-Stein and Gaussian Poincaré inequalities in Exercises 4.7 and 4.8). This actually spilled over a bit to the tenth session. Also, the proof of Theorem 4.9 was only obtained under the additional assumption that Ex 1 = 0, since the convex selection principle was only shown in the form of Theorem 3.21, but the more refined Lemma 3.22 is needed to conclude in general here. The major part of the tenth session was spent covering Section 6.2 and its analogue for the bipartite model, and discussing the new difficulties that show up. The eleventh session started with a discussion of the random energy model from Section 6.3, and then presented the main results of Section 5, in particular Proposition 5.13, as well as elementary properties of the Poisson-Dirichlet process from Section 5.5. In the last session, the Poisson-Dirichlet cascades from Section 5.6 were defined, the contents of Section 6.4 were covered without proving everything, and with some hand-waving for the remainder of Chapter 6. A recording of these lectures can be found at https://tinyurl.com/HJ-ETHZ.

Overview

The central goal of this book is to present new techniques for studying the behaviour of mean-field systems with disordered interactions. We will focus in particular on certain problems of statistical inference, and on spin glasses. We will also consider simpler models of statistical mechanics for which the interactions are not disordered, as a useful training ground. We will mostly be interested in computing the asymptotic behaviour of a fundamental quantity called the free energy, in the limit of large system size. In the context of statistical inference, the free energy is essentially the mutual information between the observations and the signal that we wish to recover; knowing its limit behaviour allows us to identify how much of the signal can be recovered from the observations. The purpose of this work is to present a way to approach this class of problems using techniques from partial differential equations, and specifically, the theory of Hamilton-Jacobi equations. We strove to make this book self-contained, and in particular, no prior knowledge of Hamilton-Jacobi equations or other partial differential equations is assumed.

To get a sense of what spin glasses are, let us start by describing the simplest such model, namely the Sherrington-Kirkpatrick (SK) model [START_REF] Sherrington | Solvable model of a spin-glass[END_REF]. This model is often motivated by considering the problem of splitting N individuals into two groups. An assignment of individuals into two groups can be encoded by a vector σ ∈ Σ N ∶= {-1,+1} N , with the understanding that σ i represents the group to which the individual indexed by i is assigned. For each pair of individuals (i, j), we are given a number g i j that encodes the quality of the interaction between individuals i and j, with g i j being large if i and j get along very well, and g i j being very negative if they cannot stand each other. We would like to find an assignment into groups that maximizes the sum total of the interactions, that is, we want to maximize, over the set Σ N , the "comfort function" c N (σ ) ∶= N i, j=1 g i j σ i σ j .

(0.1)

Given the problem we are trying to encode, it would probably have felt more natural 2 Contents to write 1 {σ i =σ j } in place of σ i σ j above. However, the form σ i σ j is more standard, partly for historical reasons having to do with the modelling of magnetic materials. Since σ i σ j = 21 {σ i =σ j } -1, writing one or the other is inconsequential. It will be convenient not to impose any symmetry on the matrix (g i j ) 1⩽i, j⩽N , so the interaction between i and j, with i ≠ j, is actually encoded by g i j + g ji . It will also be convenient to include some diagonal terms (g ii ) 1⩽i⩽N ; these terms only change the comfort function by an additive constant, and their contribution will in any case be of lower order.

In order to gain some insight into a typical instance of this optimization problem, we assume a particular structure on the interaction parameters (g i j ) 1⩽i, j⩽N . Namely, we assume that the (g i j ) 1⩽i, j⩽N are independent standard Gaussian random variables. By "standard Gaussian", we always mean Gaussian random variables with zero mean and unit variance. One quickly realizes that the optimization of the comfort function (0.1) is not going to be easy because the coefficients (g i j ) 1⩽i, j⩽N do not have a sign. A glimpse of the difficulty can already be perceived when we consider situations in which three individuals i, j and k are as depicted in Figure 0.1, that is, with g i j > 0 and g ik > 0, but with g jk < 0. In view of the sign of g i j , we would rather want i and j to belong to the same group, and similarly with i and k. But the sign of g ik suggests to rather have i and k in different groups, and all of these "local preferences" cannot be reconciled at once. Physicists use the word frustration to describe situations of this nature, and, a glass is a system that is subject to such frustrations. More generally, the presence of these frustrations suggests that it is likely to be difficult to optimize the function (0.1). For instance, a naive method that would try to optimize each spin σ i one at a time in order to decrease the value of (0.1) is unlikely to reach the global optimum. The word "glass" points to the fact that this phenomenology is also present for regular window glass, as well as a number of other materials. Indeed, the making of a glass requires the very rapid cooling of an initially liquid material. A slow cooling would have allowed the material to find its preferred crystal structure, but the fast cooling has blocked the particles in a highly disordered configuration, and the complex geometric arrangement of the particles is believed to create jams that render the finding of the preferred crystalline state very difficult to achieve.

In order to gain insight into the problem of optimizing the comfort function (0.1), it is useful to first focus on the evaluation of the asymptotic behaviour of the maximum itself, and more generally of quantities of the form

1 N Elog σ ∈Σ N exp β √ N N i, j=1 g i j σ i σ j , (0.2)
where β ⩾ 0 is a tunable parameter, and E denotes the expectation with respect to the randomness of the (g i j ). The quantity in (0.2) is called the free energy, and it is closely related to the random probability measure, called the Gibbs measure, that assigns to each configuration σ ∈ Σ N a probability proportional to

exp β √ N N i, j=1 g i j σ i σ j . (0.3)
To understand the presence of the factor of N -1 2 in the exponential above, one needs to realize that the maximum of the comfort function (0.1) will typically be of order N 3 2 , see Exercise 6.1. With the factor of N -1 2 in the exponential in (0.2), we thus ensure that the terms that have the largest contribution to the sum in (0.2) are exponential in N. Since we are then summing over 2 N terms, this allows us to interpolate between a situation in which the entropy dominates, for small β , where the Gibbs measure resembles the uniform law over Σ N , and a situation in which the energy dominates, for large β , where the Gibbs measure concentrates on the configurations that essentially realize the maximal value of the comfort function (0.1). In particular, a good approximation of β N -3 2 times the maximum of the comfort function (0.1) is obtained by choosing β sufficiently large, not depending on N, in (0.2). This is made precise in Exercise 6.3. The problem of identifying the large-N limit of the free energy (0.2) turns out to be surprisingly difficult. Starting in the late 1970's, Giorgio Parisi and collaborators [START_REF] Mézard | Replica symmetry breaking and the nature of the spin glass phase[END_REF][START_REF] Mézard | Nature of the spin-glass phase[END_REF][START_REF] Mézard | Spin glass theory and beyond[END_REF][START_REF] Parisi | Infinite number of order parameters for spin-glasses[END_REF][START_REF] Parisi | The order parameter for spin glasses: a function on the interval 0-1[END_REF][START_REF] Parisi | A sequence of approximated solutions to the S-K model for spin glasses[END_REF][START_REF] Parisi | Order parameter for spin-glasses[END_REF][START_REF] Parisi | Nobel prize lecture: Giorgio Parisi, Nobel prize in physics[END_REF] proposed a way to solve this problem using sophisticated non-rigorous techniques, and the limit of (0.2) is now known as the Parisi formula. A rich phenomenology progressively emerged concerning the structure of the associated Gibbs measure, which turns out to organize itself along an ultrametric structure. Some key elements of this picture that were uncovered in the physics literature were then progressively put on a rigorous mathematical footing [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF][START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF][START_REF] Talagrand | The Parisi formula[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF]. Although many interesting questions remain, the mathematical understanding of the SK model, and of some generalizations thereof, is by now very substantial [START_REF] Barra | Multi-species mean field spin glasses. Rigorous results[END_REF][START_REF] Bates | Free energy in multi-species mixed p-spin spherical models[END_REF][START_REF] Bovier | The Aizenman-Sims-Starr and Guerra's schemes for the SK model with multidimensional spins[END_REF][START_REF] Guerra | The infinite volume limit in generalized mean field disordered models[END_REF][START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF][START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF][START_REF] Panchenko | The free energy in a multi-species Sherrington-Kirkpatrick model[END_REF][START_REF] Panchenko | Free energy in the Potts spin glass[END_REF][START_REF] Panchenko | Free energy in the mixed p-spin models with vector spins[END_REF][START_REF] Talagrand | A general form of certain mean field models for spin glasses[END_REF].

There are however many seemingly innocent generalizations of the SK model that mostly remain mathematically mysterious. Motivated in part by considerations that relate to artificial neural networks (see e.g. [START_REF] Tubiana | Emergence of compositional representations in restricted Boltzmann machines[END_REF]), we would like for instance to consider generalizations of the SK model in which the spins are organized over Contents two (or more) layers. To encode this precisely, we can represent a configuration as a pair σ = (σ 1 ,σ 2 ) ∈ Σ 2 N , where σ 1 = (σ 1,i ) 1⩽i⩽N and σ 2 = (σ 2,i ) 1⩽i⩽N represent the spins in each of the two layers, and where we choose the layers to be of the same size for convenience of notation. The energy function can then be written as

H N (σ ) ∶= 1 √ N N i, j=1
g i j σ 1,i σ 2, j . (0.4)

We will refer to this as the bipartite model. An illustration of the graph of interactions between the coordinates of σ is in Figure 0.2. Perhaps surprisingly, this model is much less understood than the SK model. In particular, the limit free energy of this model has not yet been identified rigorously. Inspired by [START_REF] Barra | Mean field spin glasses treated with PDE techniques[END_REF][START_REF] Barra | Replica symmetry breaking in mean-field spin glasses through the Hamilton-Jacobi technique[END_REF][START_REF] Brankov | On the description of the phase transition in the Husimi-Temperley model[END_REF][START_REF] Guerra | Sum rules for the free energy in the mean field spin glass model[END_REF][START_REF] Newman | Shock waves and mean field bounds: concavity and analyticity of the magnetization at low temperature. Appended to: Percolation theory: a selective survey of rigorous results[END_REF], we propose to approach this problem using a point of view based on partial differential equations [START_REF] Mourrat | Hamilton-Jacobi equations for mean-field disordered systems[END_REF][START_REF] Mourrat | The Parisi formula is a Hamilton-Jacobi equation in Wasserstein space[END_REF]. It turns out that one can identify the limit of the free energy (0.2) in the SK model as lim

N→+∞ - 1 N Elog σ Σ N exp 2t N N i, j=1
g i j σ i σ j -Nt = f (t,0), (0.5)

where f = f (t,q) ∶ R ⩾0 × Q 2 (R ⩾0 ) → R is the solution to the infinite-dimensional Hamilton-Jacobi equation

∂ t f (t,q) - ˆ1 0 ∂ q f (t,q,u) 2 du = 0 on R >0 × Q 2 (R ⩾0 ), (0.6)
with Q 2 (R ⩾0 ) being the space of square-integrable non-decreasing paths from [0,1) to R ⩾0 , and ∂ q f (t,q,⋅) denoting the Gateaux derivative of f at (t,q), see (6.74). The initial condition f (0,⋅) to this equation is described by a functional transform of the Bernoulli measure δ -1 +δ +1 encoding the "reference" law of one spin. Theorems 6.7 and 6.8 state this result more precisely. One of the most interesting aspects of this statement is that it suggests a very natural candidate for the limit free energy of the bipartite model and its generalizations, also phrased in the language of Hamilton-Jacobi equations, see Questions 6.9 and 6.11 for precise statements. An inequality between the limit free energy and this candidate limit is proved in [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF][START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF], but the converse bound remains an open problem.

The same approach has already led to more complete results concerning certain problems of statistical inference, as will be explained in detail in this book. For clarity of exposition, we will focus on the following relatively simple situation. We observe a noisy version of a rank-one matrix of the form xx * , where x = (x 1 ,...,x N ) is a vector of independent and identically distributed (i.i.d.) random variables, and the superscript * denotes the transposition operator. Precisely, we assume that we observe the matrix

Y ∶= 2t N xx * +W, (0.7)
where t ⩾ 0 is a free parameter that allows us to vary the signal-to-noise ratio, and W = (W i j ) 1⩽i, j⩽N is a matrix of independent standard Gaussian random variables. We will see in Chapter 4 that this problem is closely related to that of community detection in random networks of Erdős-Rényi type with diverging average degree.

We wish to answer the following question: given the observation of Y , can we recover meaningful information about the signal xx * ? One way to assess this is to monitor the minimal mean-square error

mmse N (t) ∶= 1 N 2 inf g E xx * -g(Y ) 2 = 1 N 2 E xx * -E[xx * Y ] 2 (0.8)
between the signal xx * and its noisy observation Y . Here, the infimum is taken over the set of measurable functions g, and for a matrix a, we write a ∶= tr(aa * ), with tr denoting the trace operator. Assuming that Ex 1 = 0 for simplicity, we can compare this minimal mean-square error with the error one would make by simply using the null estimator, defining

var N (t) ∶= 1 N 2 E xx * 2 .
(0.9)

It turns out that there exists a critical parameter t c ∈ (0,+∞) such that the following holds: for t < t c , essentially no information can be obtained about xx * , in the sense that the difference between mmse N (t) and var N (t) becomes vanishingly small as N becomes large; while for t > t c , this difference remains bounded away from zero as N tends to infinity. In fact, we will be able to characterize exactly the large-N limit of mmse N (t). This result can be obtained using a variety of methods [START_REF] Barbier | Mutual information for symmetric rank-one matrix estimation: a proof of the replica formula[END_REF][START_REF] Barbier | The adaptive interpolation method: a simple scheme to prove replica formulas in bayesian inference[END_REF][START_REF] Barbier | The layered structure of tensor estimation and its mutual information[END_REF][START_REF] Deshpande | Information-theoretically optimal sparse PCA[END_REF][START_REF] Krzakala | Estimation in the spiked Wigner model: a short proof of the replica formula[END_REF][START_REF] Lelarge | Fundamental limits of symmetric low-rank matrix estimation[END_REF][START_REF] Thibault Lesieur | Phase transitions in sparse PCA[END_REF]. We will use a Hamilton-Jacobi approach developed in [START_REF] Chen | Hamilton-Jacobi equations for nonsymmetric matrix inference[END_REF][START_REF] Chen | Hamilton-Jacobi equations for inference of matrix tensor products[END_REF][START_REF] Chen | Free energy of multi-layer generalized linear models[END_REF][START_REF] Chen | Statistical inference of finite-rank tensors[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for finite-rank matrix inference[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for mean-field disordered systems[END_REF], and identify the large-N limit of the minimal meansquare error (0.8) to be lim

N→+∞ mmse N (t) = E x 1 2 2 -∂ t f (t,0), (0.10)
where f ∶ R ⩾0 × R ⩾0 → R is the solution to the Hamilton-Jacobi equation

∂ t f (t,h) -∂ h f (t,h) 2 = 0 on R >0 × R >0 , (0.11)
subject to some explicit initial condition. This Hamilton-Jacobi approach has been extended to a very large class of other models of statistical inference in [START_REF] Chen | Statistical inference of finite-rank tensors[END_REF] (alternative approaches have not yet reached this level of generality). Before going into this, we will train our skills on the analysis of the much simpler Curie-Weiss model and its generalizations. The Curie-Weiss model is the non-disordered version of the SK model: its energy function is obtained by setting all the interaction parameters g i j to be equal to 1 in (0.1). The identification of the limit free energy of this model is a straightforward consequence of Cramér's theorem on large deviations for sums of i.i.d. random variables. This shortcut through large deviations does not seem to be available for problems in statistical inference or spin glasses. Our first goal will be to recover the limit free energy of the Curie-Weiss model and its variants using the Hamilton-Jacobi approach. This will give us the opportunity to develop this approach and its associated toolbox in the simplest possible context. This toolbox will then be ready for us to use, with no further addition necessary, when we turn to the problems of statistical inference.

Organization of the book

This book is organized as follows. We start in Chapter 1 with an introduction to the basics of statistical mechanics, where we motivate the notion of a Gibbs measure on physical grounds, and introduce the Curie-Weiss model as well as its generalizations. In Chapter 2, we develop the fundamentals of convex analysis and large deviation principles, and use these to compute the limit of the free energy in the Curie-Weiss model and its generalizations. This large-deviation approach is not applicable to the disordered models we want to consider next, so we need to develop an alternative approach.

In Chapter 3, we define the notion of viscosity solution to a Hamilton-Jacobi equation, and use it to recover the limit free energy of the Curie-Weiss model. We discover technical challenges to applying the same method to generalized versions of the Curie-Weiss model, and develop a new "selection principle" based on convexity to overcome these. We then turn to statistical inference in Chapter 4, focusing on the problem of recovering a large symmetric rank-one matrix from a noisy observation. We discover that the tools developed in the previous chapter apply to this setting as well, and allow us to give a closed-form description of its phase transitions. Chapter 5 is preparatory work for a discussion of the more challenging case of spin glasses. The first half of this chapter is a self-contained introduction to Poisson point processes, including limit theorems on extreme values of independent and identically distributed random variables, which we believe to be of wide interest. We finally turn to the setting of spin glasses in Chapter 6. For the Sherrington-Kirkpatrick model, we show how to relate the Parisi formula with the Hamilton-Jacobi approach. We conclude with a more informal discussion on the status of current research for more challenging models. Appendix A is a self-contained presentation of many of the basic results in real analysis and probability theory that are used throughout the book. Solutions to the exercises are also provided.

Related works

Our main goal with this book is to provide the reader with an inviting presentation of some results and open problems in mean-field disordered systems which we find fascinating, following the common thread of the Hamilton-Jacobi approach. We wish to stress however that we do not claim this to be the one-and-only definitive way to approach these problems.

Trying to survey the range of works and topics related to those discussed in this book is a daunting task. We will attempt to fulfill it to the best of our abilities, hoping that the references listed here will at least provide the reader with good entry points to explore further according to their interests. Apologies to all those whose work is not included here.

Chapters 1 and 2 cover very classical material on statistical mechanics, convex analysis, and large deviations. References on these topics include [START_REF] Friedli | Statistical mechanics of lattice systems: a concrete mathematical introduction[END_REF], [START_REF] Bertsekas | Convex optimization theory[END_REF][START_REF] Ekeland | Convex analysis and variational problems[END_REF]137,[START_REF] Mordukhovich | An easy path to convex analysis and applications[END_REF][START_REF] Rockafellar | Convex analysis[END_REF], and [START_REF] Dembo | Large deviations techniques and applications[END_REF][START_REF] Frank Den Hollander | Large deviations[END_REF][START_REF] Srinivasa Varadhan | Large deviations[END_REF] respectively.

Concerning the theory of viscosity solutions to Hamilton-Jacobi equations discussed in Chapter 3, the most classical reference on the topic is probably [START_REF] Michael | User's guide to viscosity solutions of second order partial differential equations[END_REF]; we also mention [START_REF] Barles | Solutions de viscosité des équations de Hamilton-Jacobi[END_REF][START_REF] Barles | An introduction to the theory of viscosity solutions for first-order Hamilton-Jacobi equations and applications[END_REF][START_REF] Evans | Partial differential equations[END_REF] for more accessible presentations that focus on firstorder equations. The notion of viscosity solution was introduced in [START_REF] Michael | Some properties of viscosity solutions of Hamilton-Jacobi equations[END_REF][START_REF] Michael | Viscosity solutions of Hamilton-Jacobi equations[END_REF][START_REF] Evans | On solving certain nonlinear partial differential equations by accretive operator methods[END_REF], and the Hopf-Lax and Hopf formulas were introduced in [START_REF] Hopf | Generalized solutions of non-linear equations of first order[END_REF][START_REF] Peter | Hyperbolic systems of conservation laws II[END_REF] and proved to be viscosity solutions in [START_REF] Lions | Generalized solutions of Hamilton-Jacobi equations[END_REF] and [START_REF] Bardi | On Hopf's formulas for solutions of Hamilton-Jacobi equations[END_REF][START_REF] Lions | Hopf formula and multitime Hamilton-Jacobi equations[END_REF] respectively. The infinite-dimensional Hamilton-Jacobi equations discussed in Chapter 6 are developed in [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF][START_REF] Chen | Hamilton-Jacobi equations with monotone nonlinearities on convex cones[END_REF][START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF].

The problems of statistical inference discussed in Chapter 4 are also explored in the surveys [START_REF] Montanari | A short tutorial on mean-field spin glass techniques for non-physicists[END_REF][START_REF] Zdeborová | Statistical physics of inference: Thresholds and algorithms[END_REF], along with several aspects not covered here. We also mention the monograph [START_REF] Mézard | Information, physics, and computation[END_REF] presenting the interplay between statistical mechanics, information theory, and combinatorial optimization, with extensive coverage of a class of methods called message-passing or belief-propagation algorithms that have shown their usefulness in a wide variety of circumstances.

The calculation of the limit free energy (or mutual information) for the inference problem discussed in Chapter 4 and its generalizations has been approached using a large variety of techniques. Together with an interpolation in the spirit of [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Guerra | The thermodynamic limit in mean field spin glass models[END_REF][START_REF] Guerra | The infinite volume limit in generalized mean field disordered models[END_REF], these may involve algorithmic approaches in [START_REF] Barbier | Mutual information for symmetric rank-one matrix estimation: a proof of the replica formula[END_REF][START_REF] Deshpande | Asymptotic mutual information for the balanced binary stochastic block model[END_REF][START_REF] Deshpande | Information-theoretically optimal sparse PCA[END_REF][START_REF] Kadmon | Statistical mechanics of low-rank tensor decomposition[END_REF][START_REF] Thibault Lesieur | Phase transitions in sparse PCA[END_REF], a cavity method in the spirit of [START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF] in [START_REF] Lelarge | Fundamental limits of symmetric low-rank matrix estimation[END_REF][START_REF] Thibault Lesieur | Statistical and computational phase transitions in spiked tensor estimation[END_REF][START_REF] Mayya | Mutual information in community detection with covariate information and correlated networks[END_REF][START_REF] Miolane | Fundamental limits of low-rank matrix estimation: the non-symmetric case[END_REF][START_REF] Reeves | The geometry of community detection via the mmse matrix[END_REF], concentration of measure in [START_REF] Krzakala | Estimation in the spiked Wigner model: a short proof of the replica formula[END_REF], or adaptive interpolation in [START_REF] Barbier | The adaptive interpolation method: a simple scheme to prove replica formulas in bayesian inference[END_REF][START_REF] Barbier | The layered structure of tensor estimation and its mutual information[END_REF][START_REF] Barbier | Information-theoretic limits of a multiview lowrank symmetric spiked matrix model[END_REF][START_REF] Luneau | Mutual information for low-rank even-order symmetric tensor estimation[END_REF][START_REF] Luneau | High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case[END_REF][START_REF] Reeves | Information-theoretic limits for the matrix tensor product[END_REF]. The Hamilton-Jacobi approach presented in Chapter 4 has been developed in [START_REF] Chen | Hamilton-Jacobi equations for nonsymmetric matrix inference[END_REF][START_REF] Chen | Hamilton-Jacobi equations for inference of matrix tensor products[END_REF][START_REF] Chen | Free energy of multi-layer generalized linear models[END_REF][START_REF] Chen | Statistical inference of finite-rank tensors[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for finite-rank matrix inference[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for mean-field disordered systems[END_REF]. Examples of models that are covered by this approach but currently not by other techniques are discussed in Section 7 of [START_REF] Luneau | Mutual information for low-rank even-order symmetric tensor estimation[END_REF].

In Chapter 4 on statistical inference, we are mostly concerned with determining Contents whether or not one can recover meaningful information about the signal from a noisy observation, and if so how much. In practice, it is also fundamental to know whether or not one can recover this information in a reasonable amount of time. We barely scratch the surface on this point in Section 4.4. The example in Proposition 4.21 shows a situation in which for some regime of parameters, it is theoretically possible to recover meaningful information about the signal, but the standard (and rapid) PCA method completely fails. In fact, one expects that in this regime, there is no polynomial-time algorithm that recovers meaningful information about the signal. This difference between the moment when one can theoretically infer non-trivial information about the signal, and the one when one can do so efficiently, is often called the statistical-to-computational gap. It is believed to exist in a wide variety of situations, and we refer to [START_REF] Afonso | Notes on computationalto-statistical gaps: predictions using statistical physics[END_REF][START_REF] Gamarnik | The overlap gap property: a topological barrier to optimizing over random structures[END_REF][START_REF] Gamarnik | Disordered systems insights on computational hardness[END_REF][START_REF] Zdeborová | Statistical physics of inference: Thresholds and algorithms[END_REF] for surveys on this. In Section 4.5, we discuss the relationship between the inference problem (0.7) and the problem of detecting communities from the observation of a random graph where the probability for an edge to be drawn between two nodes depends on the community of each node. A survey on this problem of community detection is [START_REF] Abbe | Community detection and stochastic block models: recent developments[END_REF]; we also refer to [START_REF] Decelle | Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications[END_REF] where many predictions were put forward at the physics level of rigour. In the setting in which the average degree of the graph of connections remains bounded, the identification of the regime of parameters for which one can reconstruct non-trivial information about the community structure from the observation of the random graph was achieved in [START_REF] Massoulié | Community detection thresholds and the weak Ramanujan property[END_REF][START_REF] Mossel | Reconstruction and estimation in the planted partition model[END_REF][START_REF] Mossel | A proof of the block model threshold conjecture[END_REF] for models with two communities. To the best of our knowledge, the identification of a closedform description of this regime of parameters remains open in settings with more than two communities, and the difficulties seem to be of a similar nature as those encountered in the analysis of the bipartite spin-glass model (0.4). We refer in particular to [START_REF] Abbe | Proof of the achievability conjectures for the general stochastic block model[END_REF] for positive results in this direction, and again to [START_REF] Decelle | Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications[END_REF] for several predictions. A related problem concerns the determination of the asymptotic mutual information between the community structure and the graph of connections. This problem was successfully resolved in "convex" cases [START_REF] Abbe | Conditional random fields, planted constraint satisfaction, and entropy concentration[END_REF][START_REF] Coja-Oghlan | Information-theoretic thresholds from the cavity method[END_REF] and even in some "non-convex" cases [START_REF] Abbe | Stochastic block model entropy and broadcasting on trees with survey[END_REF][START_REF] Gu | Uniqueness of BP fixed point for the Potts model and applications to community detection[END_REF][START_REF] Varun Kanade | Global and local information in clustering labeled block models[END_REF][START_REF] Mossel | Belief propagation, robust reconstruction and optimal recovery of block models[END_REF][START_REF] Mossel | Local algorithms for block models with side information[END_REF][START_REF] Yu | Ising model on locally tree-like graphs: Uniqueness of solutions to cavity equations[END_REF], but a fully general solution has still not been identified [START_REF] Gu | Uniqueness of BP fixed point for the Potts model and applications to community detection[END_REF]. (What counts as a "convex" or a "non-convex" model should hopefully become clear upon reading Chapter 6.) This problem seems very similar to those presented in Questions 6.9 and 6.11 in the context of spin glasses, as discussed further in [START_REF] Dominguez | Mutual information for the sparse stochastic block model[END_REF][START_REF] Kireeva | Breakdown of a concavity property of mutual information for non-Gaussian channels[END_REF].

Books with a focus on spin glasses include [START_REF]Spin glasses[END_REF][START_REF] Bovier | Statistical mechanics of disordered systems: a mathematical perspective[END_REF][START_REF] Bovier | Mathematical aspects of spin glasses and neural networks[END_REF][START_REF] Charbonneau | Spin glass theory and far beyond[END_REF][START_REF] Contucci | Perspectives on spin glasses[END_REF][START_REF] De | Random fields and spin glasses: a field theory approach[END_REF][START_REF] Mézard | Spin glass theory and beyond[END_REF][START_REF] Nishimori | Statistical physics of spin glasses and information processing[END_REF][START_REF]Advanced mean field methods[END_REF][START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF][START_REF] Daniel | Spin glasses and complexity[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF]. The construction of Poisson-Dirichlet cascades in Sections 5.5 and 5.6 essentially follows [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF]. We also refer to [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF] for a complete proof of the Parisi formula (6.8).

The idea that the limit free energy of the SK model should be described by the Parisi formula emerged in the series of works [START_REF] Parisi | Infinite number of order parameters for spin-glasses[END_REF][START_REF] Parisi | The order parameter for spin glasses: a function on the interval 0-1[END_REF][START_REF] Parisi | A sequence of approximated solutions to the S-K model for spin glasses[END_REF] based on the non-rigorous replica method. The physical understanding of the model progressed further in many contributions including [START_REF] Mézard | Nature of the spin-glass phase[END_REF][START_REF] Mézard | Replica symmetry breaking and the nature of the spin glass phase[END_REF][START_REF] Mézard | Spin glass theory and beyond[END_REF][START_REF] Parisi | Order parameter for spin-glasses[END_REF]. A mathematical proof of the Parisi formula was then obtained in [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Talagrand | The Parisi formula[END_REF]. A more robust proof was later found in [START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF][START_REF] Panchenko | The Parisi formula for mixed p-spin models[END_REF] using the idea of ultrametricity discussed in Section 5.7, and also inspired by [START_REF] Aizenman | On the stability of the quenched state in mean-field spin-glass models[END_REF][START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF][START_REF] Arguin | On the structure of quasi-stationary competing particle systems[END_REF][START_REF] Bolthausen | On Ruelle's probability cascades and an abstract cavity method[END_REF][START_REF] Ghirlanda | General properties of overlap probability distributions in disordered spin systems. Towards Parisi ultrametricity[END_REF][START_REF] Panchenko | A connection between the Ghirlanda-Guerra identities and ultrametricity[END_REF][START_REF] Panchenko | Ghirlanda-Guerra identities and ultrametricity: an elementary proof in the discrete case[END_REF][START_REF] Panchenko | A unified stability property in spin glasses[END_REF]. We refer to [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF] for a more thorough presentation of historical developments on this topic. The Parisi formula for spherical models was obtained in [START_REF] Chen | The Aizenman-Sims-Starr scheme and Parisi formula for mixed p-spin spherical models[END_REF][START_REF] Talagrand | Free energy of the spherical mean field model[END_REF], and takes a form that was predicted in [START_REF] Crisanti | The spherical p-spin interaction spin glass model: the statics[END_REF].

Early contributions in the physics literature on the bipartite spin-glass model (0.4) include [START_REF] Fyodorov | Antiferromagnetic ising spin glass[END_REF][START_REF] Fyodorov | Phase transitions in frustrated metamagnets[END_REF][START_REF] Kincaid | Phase diagrams of liquid helium mixtures and metamagnets: experiment and mean field theory[END_REF][START_REF] Ya | Spin glass in an lsing two-sublattice magnet[END_REF]. On the mathematical side, the limit free energy of spin glasses with multiple types has been identified in a number of cases. For Hamiltonians such that (6.163) holds, the limit free energy has been identified in [START_REF] Barra | Multi-species mean field spin glasses. Rigorous results[END_REF][START_REF] Panchenko | Free energy in the generalized Sherrington-Kirkpatrick mean field model[END_REF][START_REF] Panchenko | The free energy in a multi-species Sherrington-Kirkpatrick model[END_REF][START_REF] Panchenko | Free energy in the Potts spin glass[END_REF][START_REF] Panchenko | Free energy in the mixed p-spin models with vector spins[END_REF] when ξ is convex over R D×D ; see also [START_REF] Bates | Crisanti-Sommers formula and simultaneous symmetry breaking in multi-species spherical spin glasses[END_REF][START_REF] Bates | Free energy in multi-species mixed p-spin spherical models[END_REF][START_REF] Ko | Free energy of multiple systems of spherical spin glasses with constrained overlaps[END_REF][START_REF] Panchenko | On the overlap in the multiple spherical SK models[END_REF] in the spherical case. The more general case when ξ is convex over the space of positive semi-definite matrices is obtained in [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF][START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF][START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF][START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF] using some of the ideas presented here. For fully general models in the form of (6.163), Theorem 6.12 from [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] imposes strong constraints on what the limit free energy can be; it is at present unclear whether this statement is a complete characterization of the limit free energy or not (even though there are choices of (t,q) such that (6.171) is satisfied for multiple pairs (q ′ ,p), it may still be the case that, for instance, there is only one continuous function f that satisfies the properties listed in Theorem 6.12). Spin glasses with multiple types were already present in the first proof of the Parisi formula from [START_REF] Talagrand | The Parisi formula[END_REF][START_REF] Talagrand | Mean field models for spin glasses: some obnoxious problems[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF], in the form of two copies of the original system coupled together through a constraint on their overlap. That there exist connections between limit free energies of statistical-mechanics systems and solutions to Hamilton-Jacobi equations dates back at least to [START_REF] Brankov | On the description of the phase transition in the Husimi-Temperley model[END_REF][START_REF] Newman | Shock waves and mean field bounds: concavity and analyticity of the magnetization at low temperature. Appended to: Percolation theory: a selective survey of rigorous results[END_REF]; see also [START_REF] Bauerschmidt | Stochastic dynamics and the Polchinski equation: an introduction[END_REF] for a survey of related contemporary research topics. In the context of spin glasses, heuristic connections between limit free energies and Hamilton-Jacobi equations were first pointed out in [START_REF] Agliari | Notes on the p-spin glass studied via Hamilton-Jacobi and smooth-cavity techniques[END_REF][START_REF] Barra | Mean field spin glasses treated with PDE techniques[END_REF][START_REF] Barra | Replica symmetry breaking in mean-field spin glasses through the Hamilton-Jacobi technique[END_REF][START_REF] Guerra | Sum rules for the free energy in the mean field spin glass model[END_REF], under a replica-symmetric or one-step replica symmetry breaking assumption. The possibility to rephrase the Parisi formula in terms of a Hamilton-Jacobi equation as in Theorems 6.7 and 6.8 is from [START_REF] Mourrat | The Parisi formula is a Hamilton-Jacobi equation in Wasserstein space[END_REF][START_REF] Mourrat | Extending the Parisi formula along a Hamilton-Jacobi equation[END_REF]. Theorems 6.10 and 6.12 are from [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF] (generalized in [START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF]) and [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] respectively. A high-temperature version of Theorem 6.12 is in [START_REF] Dey | Fluctuation results for multi-species Sherrington-Kirkpatrick model in the replica symmetric regime[END_REF].

For spherical models with multiple types, the limit free energy is identified in some non-convex cases in [START_REF] Subag | The free energy of spherical pure p-spin models -computation from the TAP approach[END_REF][START_REF] Subag | TAP approach for multi-species spherical spin glasses I: general theory[END_REF][START_REF] Subag | On the second moment method and RS phase of multi-species spherical spin glasses[END_REF][START_REF] Subag | TAP approach for multi-species spherical spin glasses II: the free energy of the pure models[END_REF] using an approach inspired by the early work [START_REF] Thouless | Solution of 'solvable model of a spin glass[END_REF], and in [START_REF] Baik | Free energy of bipartite spherical Sherrington-Kirkpatrick model[END_REF] for the bipartite spherical SK model. We also mention [START_REF] Ben Arous | Exponential growth of random determinants beyond invariance[END_REF][START_REF] Huang | Strong topological trivialization of multi-species spherical spin glasses[END_REF][START_REF] Kivimae | The ground state energy and concentration of complexity in spherical bipartite models[END_REF][START_REF] Mckenna | Complexity of bipartite spherical spin glasses[END_REF] for a geometric analysis of the energy landscape of spin glasses with multiple types; see also [START_REF] Auffinger | Complexity of random smooth functions on the high-dimensional sphere[END_REF][START_REF] Auffinger | Random matrices and complexity of spin glasses[END_REF][START_REF] Fyodorov | Complexity of random energy landscapes, glass transition, and absolute value of the spectral determinant of random matrices[END_REF][START_REF] Subag | The complexity of spherical p-spin models-a second moment approach[END_REF] in the single-type case.

By taking a low-temperature limit, the determination of the limit free energy of spin glasses allows one to infer the asymptotic value of the maximum of the Hamiltonian; see Exercise 6.3 and [26,[START_REF] Chen | On p -Gaussian-Grothendieck problem[END_REF][START_REF] Dominguez | The p -Gaussian-Grothendieck problem with vector spins[END_REF]. In the context of the Sherrington-Kirkpatrick model, with H N as in (6.3), this would amount to determining the Contents asymptotic behaviour of the maximum of H N (σ ) N over σ ∈ Σ N ; we denote the limit by OPT. One may ask whether there exists an efficient algorithm for actually finding a configuration σ ∈ Σ N such that H N (σ ) N is close to OPT. A specific value ALG described by an analogue of the Parisi formula was identified such that there exists an efficient algorithm that can identify a configuration σ with H N (σ ) N approximately equal to ALG with high probability [START_REF] Alaoui | Optimization of mean-field spin glasses[END_REF][START_REF] Montanari | Optimization of the Sherrington-Kirkpatrick Hamiltonian[END_REF][START_REF] Sellke | Optimizing mean field spin glasses with external field[END_REF][START_REF] Subag | Following the ground states of full-RSB spherical spin glasses[END_REF]. Moreover, it was argued in [START_REF] Huang | Tight Lipschitz hardness for optimizing mean field spin glasses[END_REF] that no algorithm within a broad class can exist which improves upon this value. Whether or not ALG = OPT depends on the specifics of the spin-glass model at hand. Remarkably, these results on algorithmic thresholds have been extended to non-convex models such as the bipartite model in (0.4), despite the fact that we do not know of a characterization of OPT or of the free energy in this case [START_REF] Huang | Algorithmic threshold for multi-species spherical spin glasses[END_REF][START_REF] Huang | Optimization algorithms for multi-species spherical spin glasses[END_REF]. The reference [START_REF] Auffinger | Optimization of random high-dimensional functions: Structure and algorithms[END_REF] surveys a number of results related to optimization algorithms for spin glasses.

The study of mean-field spin glasses has inspired developments in many other contexts. Works that explore connections between spin glasses and neural networks include [START_REF] Agliari | Multitasking associative networks[END_REF][START_REF] Barra | On the equivalence of Hopfield networks and Boltzmann machines[END_REF][START_REF] Barra | The replica symmetric approximation of the analogical neural network[END_REF][START_REF] Barra | Phase transitions in restricted Boltzmann machines with generic priors[END_REF][START_REF] Barra | Phase diagram of restricted Boltzmann machines and generalized Hopfield networks with arbitrary priors[END_REF][START_REF] Decelle | Thermodynamics of restricted Boltzmann machines and related learning dynamics[END_REF][START_REF] Gabrié | Training Restricted Boltzmann Machine via the Thouless-Anderson-Palmer free energy[END_REF][START_REF] Hartnett | Replica symmetry breaking in bipartite spin glasses and neural networks[END_REF][START_REF] Marullo | Boltzmann machines as generalized Hopfield networks: a review of recent results and outlooks[END_REF][START_REF] Tubiana | Emergence of compositional representations in restricted Boltzmann machines[END_REF]; see [START_REF] Advani | Statistical mechanics of complex neural systems and high dimensional data[END_REF][START_REF] Daniel | Modeling brain function[END_REF][START_REF] Decelle | Restricted Boltzmann machine: recent advances and mean-field theory[END_REF][START_REF] Engel | Statistical mechanics of learning[END_REF][START_REF] Gabrié | Mean-field inference methods for neural networks[END_REF][START_REF] Hertz | I of Santa Fe Institute Studies in the Sciences of Complexity[END_REF][START_REF] Peretto | An introduction to the modeling of neural networks, volume 2 of Collection Aléa-Saclay: Monographs and Texts in Statistical Physics[END_REF][START_REF] Tubiana | Restricted Boltzmann machines: from compositional representations to protein sequence analysis[END_REF] for overviews. For a suitable choice of the reference measure denoted by P N in Chapter 6, the law of the spins of a given type in the bipartite model (0.4) is the same as the law of the spins in the Hopfield model [START_REF] Barra | On the equivalence of Hopfield networks and Boltzmann machines[END_REF]. Some versions of the perceptron model can be obtained similarly. The Hopfield and perceptron models were introduced in [START_REF] Shun | Learning patterns and pattern sequences by self-organizing nets of threshold elements[END_REF][START_REF] Hopfield | Neural networks and physical systems with emergent collective computational abilities[END_REF][START_REF] Little | The existence of persistent states in the brain[END_REF][START_REF] Mcculloch | A logical calculus of the ideas immanent in nervous activity[END_REF][START_REF] Minsky | Perceptrons[END_REF] as toy models for memory storage and retrieval or classification tasks. Early works on the statistical mechanics of these models include [START_REF] Daniel | Spin-glass models of neural networks[END_REF][START_REF] Daniel | Statistical mechanics of neural networks near saturation[END_REF][START_REF] Gardner | The space of interactions in neural network models[END_REF][START_REF] Gardner | Optimal storage properties of neural network models[END_REF][START_REF] Pastur | Exactly soluble model of a spin glass[END_REF][START_REF] Pastur | Theory of disordered spin systems[END_REF][START_REF] Pastur | Infinite-range limit for a class of disordered spin systems[END_REF]; recent rigorous works include [START_REF] Bolthausen | Gardner formula for Ising perceptron models at small densities[END_REF][START_REF] Ding | Capacity lower bound for the ising perceptron[END_REF][START_REF] Talagrand | Mean field models for spin glasses[END_REF][START_REF] Xu | Sharp threshold for the Ising perceptron model[END_REF]. Among many other topics related to spin glasses, we mention random constraint satisfaction problems [START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF][START_REF] Ding | Satisfiability threshold for random regular NAE-SAT[END_REF][START_REF] Krzakała | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF][START_REF] Mézard | Information, physics, and computation[END_REF][START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Monasson | Determining computational complexity from characteristic phase transitions[END_REF], the random assignment and travelling salesman problems [START_REF] Aldous | Asymptotics in the random assignment problem[END_REF][START_REF] Aldous | The ζ (2) limit in the random assignment problem[END_REF][START_REF] Mézard | A replica analysis of the travelling salesman problem[END_REF], error correcting codes in information theory [START_REF] Richardson | Modern coding theory[END_REF], and combinatorial problems such as graph colouring [START_REF] Coja-Oghlan | Phase transitions in discrete structures[END_REF][START_REF] Ding | Maximum independent sets on random regular graphs[END_REF][START_REF] Mulet | Coloring random graphs[END_REF]. A recent book on "spin glass theory and far beyond" is [START_REF] Charbonneau | Spin glass theory and far beyond[END_REF].

Chapter 1 Introduction to statistical mechanics

In this chapter, we introduce the basic objects from statistical mechanics that we will explore throughout the rest of the book. In Section 1.1, we introduce and motivate the notion of a Gibbs measure from a physical point of view, as well as the notion of free energy. The rest of the book will be devoted to computing this fundamental quantity for models of increasing complexity. In Section 1.2, we consider the classical Ising model, and give a brief historical overview of its developments. Finally, in Section 1.3, we trivialize the geometry of the Ising model to obtain the Curie-Weiss model. The Curie-Weiss model is simple enough that its free energy can be computed exactly using the classical large deviation principles discussed in Chapter 2, but it is rich enough to capture many of the challenges present in more sophisticated models. For this reason, we will then use it as a test-bed for the development of the Hamilton-Jacobi approach.

Gibbs measures

We start by motivating the notion of a Gibbs measure on physical grounds. In this section we will avoid delving into too many technicalities, so we will allow ourselves to not be as rigorous as in the rest of the book. Consider a system of N units, which can each be in one of K states {1,2,...,K}. We can think of these units as being particles. We think of K as being fixed, while N is very large and will be sent to infinity. Each state k ∈ {1,...,K} has an associated energy e k > 0, and we assume that the system is "isolated" in the sense that its total energy is fixed, say at Ne for some constant e in the convex hull of {e 1 ,...,e K }. The state of the system can be represented by a vector x = (x 1 ,...,x N ) ∈ {1,...,K} N , and we denote by We assume that the system is uniformly distributed among all possible configurations x that satisfy the constraint (1.2). This assumption can be related to Hamiltonian dynamics and the Boltzmann hypothesis, but we simply take it for granted here.

N k (x) ∶=
A natural question is to determine the probability of finding a particular unit in state k,

P{x i = k} = 1 N N n=1 P{x n = k} = 1 N EN k (x), (1.3)
when the number N of units in the system is very large. We will argue, without full rigour nor attention to taking integer parts at the right places, that N k (x) N converges in probability to a constant as N tends to infinity, and we will identify this constant. In order to do so, we fix a probability vector

p = (p 1 ,..., p K ) ∈ R K ⩾0 with K k=1 p k = 1, (1.4)
and we determine the asymptotic behaviour of the number of configurations x such that (N 1 (x),...,N K (x)) = (N p 1 ,...,N p K ). For a fixed N, this number is given by the multinomial coefficient which is called the entropy of p. The calculation leading to (1.6) shows that, if p and p ′ are probability vectors with S(p) > S(p ′ ), then there are exponentially more configurations with (N 1 (x),...,N K (x)) = (N p 1 ,...,N p K ) than there are configurations with (N 1 (x),...,N K (x)) = (N p ′ 1 ,...,N p ′ K ). In other words, if S(p) > S(p ′ ), then the configurations consistent with p vastly outnumber those that are consistent with p ′ , and so if we pick one configuration uniformly at random, it is overwhelmingly more likely that we observe p rather than p ′ . To be more precise, with overwhelming probability, we will observe (N 1 (x),...,N K (x)) ≃ (N p * 1 ,...,N p * K ), for the vector p * that maximizes the entropy (1.7) within the set

N N p 1 N -N p 1 N p 2 ⋯ N -∑ K-1 k=1 N p k N p K = N! (N p 1 )!⋯(N p K )! . ( 1
Λ ∶= p ∈ [0,1] K K k=1 p k = 1 and K k=1
p k e k = e .

(1.8)

Using the strict concavity of the entropy, one can show that this maximizer p * exists and is unique. Our initial question regarding the probability of finding a particular unit in a given state has therefore been answered: asymptotically as N tends to infinity, the proportion of particles in state k ∈ {1,...,K} is about p * k . We can in fact go a bit further and make p * more explicit. By the Lagrange multiplier theorem, there exist constants α,β ∈ R such that for every k ∈ {1,...,K}, ∂ p k S(p * ) = α + β e k .

(1.9)

Rearranging gives a constant Z such that for every k ∈ {1,...,K},

p * k = exp(-β e k ) Z , (1.10)
and since ∑ K k=1 p * k = 1, the constant Z can be rewritten as

Z = K k=1 exp(-β e k ). (1.11)
The probability measure on {1,...,K} with probability vector p * given by (1.10) is called the Gibbs measure at inverse temperature β . When one builds the theory of thermodynamics from first principles, this Lagrange multiplier β is defined to be the inverse temperature of the physical system (up to multiplication by the Boltzmann constant which we set equal to 1 here). The parameter β is fixed in such a way that the average energy of the system with respect to the Gibbs measure is e, In most physical systems, there is room for very high energy levels, so we typically have β > 0; however, this is not guaranteed in our current context, and will depend on how e compares with the arithmetic average of (e 1 ,...,e K ). A direct calculation allows us to relate the entropy, the average energy and the inverse temperature of the system,

S(p * ) = K k=1
p * k log Z exp(β e k ) = log(Z) + β e.

(1.13)

The quantity log(Z) should be called the free entropy; however, throughout the book we will call any quantity of this form the free energy of the system. In the language of physics, the free energy should be minus the free entropy divided by β , so that it is indeed expressed in the same units as e. But since the interpretation of the inverse temperature will become less transparent as we proceed, and since the term free energy is much more commonly used, we will stick to the less proper terminology.

The quantity Z itself is often called the partition function. We mention in passing that the identity (1.13) is a shadow of a convex-duality relationship between the entropy and the free energy of a system, if we think of them as functions of e and -β respectively; in particular, -S(p * ) = sup

β ′ ∈R -β ′ e -log K k=1 exp(-β ′ e k ) . (1.14)
We also refer to Corollaries 4.14 and 4.15 in [START_REF] Boucheron | Concentration inequalities: a nonasymptotic theory of independence[END_REF] for a more general view on this.

We have therefore been able to show that the probability of finding a microscopic unit of the system in a particular state is given by the Gibbs measure at inverse temperature β , and that the inverse temperature β can be measured from macroscopic quantities of the system such as the free energy, the entropy and the average energy. Although we have considered a setting in which the microscopic and macroscopic parts of the system are made of identical units, a refined version of the argument allows to lift this restriction. To frame our findings in concrete terms, suppose we have a piece of material at equilibrium in a room at inverse temperature β , and assume the accessible energy levels of the material are (e 1 ,...,e K ). We should then expect to find the piece of material in state k with probability proportional to exp(-β e k ). For further motivation on the physical terminology and the notion of a Gibbs measure, we refer the interested reader to Chapter 1 in [START_REF] Friedli | Statistical mechanics of lattice systems: a concrete mathematical introduction[END_REF].

Exercise 1.1. Let (a k ) k⩾1 be a sequence of real numbers. (ii) How rapidly can K grow with N for (1.15) to still remain valid?

The Ising model

The Ising model is undoubtedly the most famous model in the field of statistical mechanics. It was introduced by Willhelm Lenz and his student Ernst Ising in 1920 to gain a theoretical understanding of the phase transition from ferromagnetic to paramagnetic behaviour first observed in 1895 by Pierre Curie. In the lab, Curie found that, when subjected to a magnetic field, certain materials have a "memory" of the magnetic field they have been exposed to, while others do not. Materials of the former kind are said to exhibit ferromagnetic behaviour, while materials of the latter kind are said to exhibit paramagnetic behaviour. Perhaps surprisingly, Curie observed that materials can be made to transition from ferromagnetic to paramagnetic behaviour by increasing their temperature [START_REF] Curie | Propriétés magnétiques des corps à diverses températures[END_REF]. Shortly after these empirical observations, Auguste Piccard and Pierre Weiss gave a partly phenomenological explanation of what they called the magnetocaloric phenomenon, and proposed some formulas for the magnetic susceptibility of materials near the critical temperature at which the phase transition occurs [START_REF] Weiss | Le phénomène magnétocalorique[END_REF]. It was following this line of investigation that Wilhelm Lenz and Ernst Ising introduced the Ising model in 1920.

In the Ising model, atoms are arranged on a finite box in the integer lattice Z d . Each atom carries a magnetic moment called a spin which can either be in the -1 or in the +1 orientation. The atoms interact with their nearest neighbours in such a way as to favour the alignment of spins. To formalize this model, let B N ∶= {-N,...,N} d (1. [START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF] denote the box of side-length 2N in the integer lattice. Given two lattice points i, j ∈ B N , we write i ∼ j to mean that i and j are nearest-neighbours in the sense that they differ by one unit in at most one coordinate. A configuration of spins may be encoded by a vector σ ∈ {±1} B N ∶= {-1,+1} B N , and the interactions between the spins can be described by the energy function, or Hamiltonian, of the system, H N (σ ) ∶= i, j∈B N i∼ j σ i σ j .

(1.17)

Physicists would typically add a minus sign to the Hamiltonian, but here we omit this minus sign for convenience of notation. With our sign convention, the system has a preference for larger values of the energy function as opposed to smaller ones. The discussion in Section 1.1 implies that, at inverse temperature β > 0, the probability of finding the system in the configuration σ is the Gibbs weight

G N (β ,σ ) ∶= exp(β H N (σ )) Z N (β ) (1.18)
with partition function

Z N (β ) ∶= σ ∈{±1} B N
exp(β H N (σ )).

(1. [START_REF] Daniel | Spin-glass models of neural networks[END_REF] In view of the Hamiltonian in (1.17), this means that the system will have a preference for configurations in which neighbouring spins are equal. In the presence of an external magnetic field of intensity h ∈ R, the Hamiltonian at inverse temperature β > 0 is enriched with the term h∑ i∈B N σ i to obtain

H N (β ,h,σ ) ∶= β i, j∈B N i∼ j σ i σ j + h i∈B N σ i (1.20)
whose associated Gibbs measure is

G N (β ,h,σ ) ∶= expH N (β ,h,σ ) Z N (β ,h) (1.21)
for the enriched partition function

Z N (β ,h) ∶= σ ∈{±1} B N expH N (β ,h,σ ). (1.22)
From a physical perspective, it would be more natural that the inverse temperature β multiplies both terms in (1.20), in other words to replace h by β h in (1.20), but the parametrization chosen in (1.20) is much more convenient to work with; and this is why we include it into the definition of H N as opposed to encoding the dependency in β as in (1.18). To study the empirical observations of Curie from a theoretical point of view, we need to understand the mean magnetization m N (β ,h) ∶=

σ ∈{±1} B N 1 B N i∈B N σ i G N (β ,h,σ ), (1.23)
where B N denotes the cardinality of the box B N . One possible approach to understand the behaviour of the mean magnetization (1.23) when the number of particles N tends to infinity is to start by studying the free energy

F N (β ,h) ∶= 1 B N logZ N (β ,h). (1.24)
Indeed, a direct computation shows that m N (β ,h) = ∂ h F N (β ,h). Moreover, using a box-decomposition argument, it is possible to show that the sequence (F N ) N⩾1 admits a limit f ∶ R ⩾0 × R → R, and that if (β ,h) is a point of differentiability of f , then m(β ,h) ∶= lim N→+∞ m N (β ,h) = ∂ h f (β ,h).

(1.25)

The limit free energy f therefore encodes much of the physical information about the system. It also has the advantage of being more "robust" to perturbations of the model than the mean magnetization m N (β ,h) -for instance, it does not depend on boundary conditions one might introduce to encode some interaction between the spins on the inner boundary of the box B N and the outside environment. In contrast to the empirical evidence found by Curie, Ising [START_REF] Ising | Contribution to the theory of ferromagnetism[END_REF] was able to show that in the one-dimensional setting, d = 1, the limit magnetization curve h ↦ m(β ,h) remains smooth for every fixed β > 0. He also suggested that the same would happen in higher dimensions, d ⩾ 2. However, in 1936, Rudolf Peierls was able to prove that in any dimension d ⩾ 2, the function h ↦ m(β ,h) does in fact have a discontinuity at h = 0 when β is sufficiently large [START_REF] Peierls | On Ising's model of ferromagnetism[END_REF]. This is depicted in Figure 1.1 and is in agreement with Curie's empirical observations. Indeed, it implies that at low temperature and for h > 0, the difference between the fraction of spins in the +1 and in the -1 directions remains bounded away from zero, no matter how small h is.

In other words, the system retains some global ordering even in the limit of h > 0 going to zero. In contrast, in the high temperature regime, the random fluctuations dominate and no ordering can be preserved as h > 0 is sent to zero.

-1 The one-dimensional and two-dimensional Ising models are explored in Exercises 1.2 and 1.3. In Exercise 1.2, we compute the limit of the free energy (1.24) in the one-dimensional Ising model explicitly. In higher dimensions, d ⩾ 2, it is by no means guaranteed that a self-contained description of the limit free energy should exist. Surprisingly, in the 1940's, Lars Onsager and others managed to give an explicit description of the limit free energy of the two-dimensional Ising model [START_REF] Kaufman | Crystal statistics. II. Partition function evaluated by spinor analysis[END_REF][START_REF] Kaufman | Crystal statistics. III. Short-range order in a binary Ising lattice[END_REF][START_REF] Onsager | Crystal statistics. I. A two-dimensional model with an order-disorder transition[END_REF][START_REF] Szegő | On certain Hermitian forms associated with the Fourier series of a positive function[END_REF][START_REF] Ning | The spontaneous magnetization of a two-dimensional Ising model[END_REF]. This allowed them to determine several critical exponents which were completely unexpected. By opening a first window into the study of critical phenomena and discovering many unexpected results, their work became extremely influential. The approach developed in this series of works rested on an ambitious generalization of the method used in Exercise 1.2 for the one-dimensional Ising model based on "transfer matrices". Among other hurdles, it required the asymptotic analysis of the determinants of large Toeplitz matrices, which itself appealed to considerations involving orthogonal polynomials. Other approaches to determine the limit free energy were developed later, for instance by leveraging connections with domino tilings [START_REF] Chelkak | Revisiting the combinatorics of the 2D Ising model[END_REF][START_REF] Montroll | Lattice statistics[END_REF]. The calculation of the limit free energy for the two-dimensional Ising model is, in some sense, miraculous, and generalizations to higher dimensions, d ⩾ 3, are unlikely to be found. In contrast, the models we consider in the rest of the book are of mean-field type. This means that the rich geometry of the integer lattice is replaced by a much simpler geometry, typically that of the complete graph, making the models invariant under any permutation of the index set. This very high degree of symmetry gives more hope that the identification of the limit free energy will be amenable to robust analytic techniques. For much more about the Ising model, we refer the interested reader to [START_REF] Friedli | Statistical mechanics of lattice systems: a concrete mathematical introduction[END_REF].

1 h mN (β, h) - 1
1 h mN (β, h)
Exercise 1.2. For every inverse temperature β > 0 and magnetic intensity h ∈ R, consider the one-dimensional free energy (1.24),

F N (β ,h) ∶= 1 N log σ ∈{±1} N exp β N-1 i=1 σ i σ i+1 + β σ N σ 1 + h N i=1 σ i , (1.26)
(i) By rewriting F N in terms of the N th power of a 2-by-2 matrix, show that lim N→+∞ F N (β ,h) = log e β cosh(h) + e 2β cosh 2 (h) -2sinh(2β )

(ii) Deduce that the limit (1.25) of the mean magnetization is continuous in h. It may be helpful to refer to Propositions 2.11 and 2.15 as well as Exercise 2.6.

Exercise 1.3. Through a slight abuse of notation, consider the Hamiltonian H N (σ ) ∶= i, j∈B N+1 i∼ j σ i σ j (1.28) on {±1} B N with the boundary condition σ i ∶= 1 for i ∉ B N and the convention that the sum (1.28) is taken over the edge set of the lattice B N+1 as opposed to its vertex set. For each pair of neighbours i ∼ j ∈ B N+1 with σ i σ j = -1, draw an edge centred at the midpoint i+ j 2 in the direction orthogonal to the edge (i, j). These edges create contours that delimit regions in which the sign of the spins is constant. Denote by Γ(σ ) the set of these edges and consider the Gibbs measure defined for a bounded and measurable f = f (σ ) by

⟨ f (σ )⟩ ∶= ∑ σ ∈{±1} B N f (σ )exp(β H N (σ )) ∑ σ ∈{±1} B N exp(β H N (σ ))
.

(1.29) (i) Show that H N (σ ) + 2 Γ(σ ) does not depend on σ ∈ {±1} B N .

(ii) Show that if σ 0 = -1, then the origin must be surrounded by at least one contour in Γ(σ ).

(iii) For any contour γ, show that ⟨1 {γ⊆Γ(σ )} ⟩ ⩽ exp(-2β γ ).

(iv) Show that for β sufficiently large liminf N→+∞ ⟨σ 0 ⟩ > 0.

(v) Conclude that for large β the Ising model exhibits ferromagnetic behaviour.

The Curie-Weiss model

The Curie-Weiss model is the non-disordered mean-field model obtained by trivializing the geometry of the Ising model. In the Curie-Weiss model, the energy associated with each configuration σ ∈ {±1} N ∶= {-1,+1} N at inverse temperature t ⩾ 0 and with external magnetic field of intensity h ∈ R is

H N (t,h,σ ) ∶= t N N i, j=1 σ i σ j + h N i=1 σ i . (1.30)
Up to relabelling the spin coordinates, this corresponds to the one-dimensional Ising model on the box B N 2 where the restriction that a spin only interacts with its nearest neighbours is dropped, and a spin can instead interact with all other spins. To ensure that the Hamiltonian (1.30) remains of order N despite these additional terms, the interaction term in (1.30) is divided by N. We have chosen to denote the inverse temperature parameter by t ⩾ 0 as opposed to β ⩾ 0 as this parameter can be interpreted as a time variable in the partial differential equation that will appear later. Just like in the Ising model, the probability of finding the Curie-Weiss model in the configuration σ ∈ {±1} N is the Gibbs weight

G N (t,h,σ ) ∶= expH N (t,h,σ ) Z N (t,h) , (1.31)
with partition function

Z N (t,h) ∶= σ ∈{±1} N
expH N (t,h,σ ).

(1.32)

As in Exercise 1.3, we denote the Gibbs average of any bounded and measurable function f ∶ {±1} N → R by .33) Notice that the bracket ⟨⋅⟩ depends on N, t and h, although this dependence is kept implicit in the notation. As discussed in Section 1.2, a key goal of ours is to understand the mean magnetization

⟨ f (σ )⟩ ∶= ∑ σ ∈{±1} N f (σ )expH N (t,h,σ ) ∑ σ ∈{±1} N expH N (t,h,σ ) . ( 1
m N (t,h) ∶= ⟨ 1 N N i=1 σ i ⟩ (1.34)
as a function of the model parameters, in the regime of large N. To understand this object in the setting of the Curie-Weiss model, we will first study the free energy

F N (t,h) ∶= 1 N log 1 2 N σ ∈{±1} N
expH N (t,h,σ ).

(1.35)

The factor of 2 -N in (1.35) has been introduced so that we can interpret the normalized sum 2 -N ∑ σ ∈{±1} N as integration against the uniform probability measure on {±1} N ; its presence is inconsequential as it amounts to subtracting log2 from F N .

In Chapter 2 we will use the classical theory of large deviations to explicitly compute the limit of the free energy (1.35). This will be possible as the Hamiltonian (1.30) depends on a spin configuration σ ∈ {±1} N only through the sample average

S N (σ ) ∶= 1 N N i=1 σ i . (1.36)
Indeed, the Hamiltonian (1.30) can be written as H N (t,h,σ ) = N tS N (σ ) 2 + hS N (σ ) .

(1.37)

From the formula for the limit free energy, we will be able to leverage the envelope theorem discussed in Section 2.4 to understand the magnetization (1.34).

Unfortunately, for more complex mean-field models, it is not possible to determine the limit of the free energy by expressing the Hamiltonian as a function of a simple quantity such as S N (σ ) whose large deviations can be figured out separately. In Chapter 3 we will therefore develop a Hamilton-Jacobi approach independent of large deviation principles to determine the limit of the free energy (1.35). To face as many of the difficulties presented by more complex models as possible in a simple setting where large deviation principles can be used to verify our results, we will also consider variants of the Curie-Weiss model which generalize it in two ways:

(i) we will replace the term S N (σ ) 2 in the Hamiltonian (1.37) by ξ (S N (σ )) for an arbitrary smooth function ξ ∈ C ∞ (R;R);

(ii) we will replace the uniform probability measure on {±1} N by an arbitrary probability measure P N on R N whose support is contained in the closed Euclidean ball of radius √ N centred at the origin.

The free energy of the generalized Curie-Weiss model is therefore

F N (t,h) ∶= 1 N log
ˆRN expN tξ (S N (σ )) + hS N (σ ) dP N (σ ).

(1.38)

The free energy (1.35) can be recovered by setting ξ (x) = x 2 and by choosing P N to be the uniform probability measure on {±1} N . Once we have understood how to use the Hamilton-Jacobi approach to determine the limit of the free energy in the generalized Curie-Weiss model, we will see that the techniques developed along the way can be used to study many interesting models such as rank-one matrix estimation, community detection, and, at least partially, spin glass models.

is convex in the sense (2.2). Conversely, if a function f ∶ R d → R ∪ {+∞} is convex in the sense (2.2), then it is convex in the classical sense on its effective domain

dom f ∶= {x ∈ R d f (x) < +∞}, (2.3)
which is a convex set. We say that a function f ∶ R d → R ∪ {+∞} is proper if its effective domain is not empty. The convexity of functions and sets are intimately related. A function

f ∶ R d → R ∪ {+∞} is convex if and only if its epigraph epi f ∶= (x,λ ) ∈ R d × R f (x) ⩽ λ (2.4)
is convex, while a set C ⊆ R d is convex if and only if its indicator function

I C (x) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 0 if x ∈ C +∞ otherwise (2.5)
is convex. The notion of lower semi-continuity will also play an important role.

We say that a function f ∶ R d → R ∪ {+∞} is lower semi-continuous if its epigraph is closed. It is readily verified that a function f ∶ R d → R ∪ {+∞} is lower semicontinuous if and only if, for every sequence (x n ) n⩾1 ⊆ R d converging to some point x ∈ R d , we have f (x) ⩽ liminf n→+∞ f (x n ).

Exercise 2.1. Let C 1 ,C 2 ⊆ R d be convex sets. Show that the set difference

C 2 -C 1 ∶= {x 2 -x 1 x 1 ∈ C 1 and x 2 ∈ C 2 } (2.6)
is also convex.

Exercise 2.2. Let C be a convex set whose interior is not empty. Show that

int(C) = int(C), (2.7)
where we use the notation int(A) to denote the interior of a set A. Deduce that ∂C = ∂C. λ i = 1, and n ⩾ 1 (2.9) its convex hull. Show that any x ∈ conv(A) is a convex combination of at most d + 1 points of A.

Exercise 2.5. Let A be a compact subset of R d . Prove that its convex hull conv(A) is also compact. Assuming that g takes values in R ∪ {+∞}, show that g is convex.

Exercise 2.9. We fix a function f ∶ R d → R ∪ {+∞} and define L ∶= g ∶ R d → R ∪ {+∞} g ⩽ f and g is lower semi-continuous .

(2.12)

Assuming that the set L is not empty, we define the lower-semicontinuous envelope f ∶ R d → R ∪ {+∞} of the function f as the supremum of all functions in L, that is f = sup g∈L g. For every x ∈ R d , we write liminf y→x f (y) ∶= lim r↘0 inf{ f (y) yx ⩽ r}.

(2.13) (i) Show that f is lower semi-continuous.

(ii) Show that for every x ∈ R d , we have f (x) = liminf y→x f (y).

(iii) Show that if f is convex, then f is convex.

The Fenchel-Moreau duality theorem

The main result in convex analysis that we will discuss is the Fenchel-Moreau theorem. This result ensures that a convex and lower semi-continuous function is equal to its convex bi-dual. It will play an important role in obtaining variational formulas for limit free energies from the Hamilton-Jacobi approach. For instance, it will be used in Chapter 3 to recover the formula for the limit free energy in the Curie-Weiss model that we will prove in this chapter. The convex dual of a proper function

f ∶ R d → R ∪ {+∞} is the function f * ∶ R d → R ∪ {+∞} defined by f * (λ ) ∶= sup x∈R d (λ ⋅ x -f (x)), (2.14)
If f * is proper, then we can iterate this operation and obtain the convex bi-dual of f , which is the function

f * * ∶ R d → R ∪ {+∞} defined by f * * (x) ∶= ( f * ) * (x) = sup λ ∈R d (x ⋅ λ -f * (λ )). (2.15)
For instance, if f (x) ∶= p ⋅ x + a is the affine function with normal vector p ∈ R d and intercept a ∈ R, then its convex dual is given by

f * (λ ) ∶= ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -a if λ = p +∞ otherwise (2.16)
while its convex bi-dual is f itself, f * * (x) = p ⋅ x + a = f (x).

(2.17)

This is a very important special case of the Fenchel-Moreau theorem. Indeed, to prove the Fenchel-Moreau theorem we will first show that a convex and lower semicontinuous function f ∶ R d → R ∪ {+∞} is the supremum of its affine minorants, and we will then leverage this observation as well as the basic properties of the convex dual established in Exercise 2.10. A function g ∶ R d → R is an affine minorant of f if g is an affine function and g ⩽ f . This representation result for convex functions will be deduced from the supporting hyperplane theorem, which itself is a special case of the Hahn-Banach separation theorem. The Hahn-Banach separation theorem is the geometrically intuitive statement that any two disjoint convex sets C 1 ,C 2 ⊆ R d may be separated by a hyperplane. Although this result is classical, to keep the book as self-contained as possible, we will provide a full proof. We will first use projection operators onto closed and convex sets to show that convex sets can be linearly separated from singletons that they do not contain, and that this separation is strict if the convex set is closed. We will then deduce the Hahn-Banach separation theorem from this special case.

Lemma 2.1 (Projection). If C ⊆ R d is a non-empty closed convex set and x ∈ R d , then there exists a unique point P C (x) ∈ C with

x -P C (x) 2 = inf x -y 2 y ∈ C . (2.18)
Moreover, a point y ∈ C is the projection P C (x) of x onto C if and only if for all z ∈ C,

(x -y) ⋅ (z -y) ⩽ 0. (2.19)
Proof. Fix z ∈ C, and observe that

inf x -y 2 y ∈ C = inf{ x -y 2 y ∈ C and x -y ⩽ x -z .
The infimum on the right side is achieved at some point P C (x) as it consists in minimizing the continuous function y ↦ xy 2 over a compact set. To prove the uniqueness of this minimizer, suppose that y 1 ,y 2 are two minimizers, and let y = y 1 +y 2 2 be their average. Introduce the differences x 1 ∶= y 1x and x 2 ∶= y 2x, and observe that

x 1 2 + x 2 2 = 1 2 x 1 + x 2 2 + 1 2 x 2 -x 2 2 .
This implies that

y 1 -x 2 + y 2 -x 2 = 2 y -x 2 + y 2 -y 1 2 ⩾ y 1 -x 2 + y 2 -x 2 + y 2 -y 1 2 ,
where the second inequality uses the fact that yx 2 ⩾ y ix 2 for i = 1 and i = 2. Rearranging shows that y 2 = y 1 and establishes the uniqueness of the projection P C (x).

We now show that the projection y = P C (x) satisfies (2.19). Fix z ∈ C and α ∈ (0,1).

The convexity of C implies that P C (x) + α(z -P C (x)) ∈ C, so the definition of P C (x) reveals that

P C (x) -x 2 ⩽ P C (x) -x + α(z -P C (x)) 2 = P C (x) -x 2 + 2α(P C (x) -x) ⋅ (z -P C (x)) + α 2 z -P C (x) 2 .
Rearranging, dividing by α and letting α tend to zero shows that y = P C (x) satisfies (2.19

⩾ (x -y) ⋅ (z -y) = (x -y) ⋅ (z -x + x -y) = (x -y) ⋅ (z -x) + x -y 2 .
It follows by the Cauchy-Schwarz inequality that

xy 2 ⩽ xy zx .

Dividing by xy ≠ 0 and leveraging the uniqueness of P C (x) shows that y = P C (x), and completes the proof. ∎ Theorem 2.2 (Supporting hyperplane). If C ⊆ R d is a non-empty convex set and x ∉ C, then there exists a non-zero vector

v ∈ R d with v ⋅ x ⩽ inf{v ⋅ y y ∈ C}. (2.20)
Moreover, if C is closed, then the vector v can be chosen so that the inequality in (2.20) is strict.

Proof. We decompose the proof into two steps.

Step 1: closed C. The vector v ∶= P C (x)x is well-defined and non-zero by Lemma 2.1. Moreover, its characterization (2.19) ensures that for any z ∈ C,

0 ⩾ (x -P C (x)) ⋅ (z -P C (x)) = -v ⋅ (z -v -x) = -v ⋅ z + v ⋅ x + v 2 .
Rearranging shows that v⋅z ⩾ v⋅x+ v 2 which establishes (2.20) with strict inequality.

Step 2: general C. Without loss of generality, we may assume that the interior of the set C is not empty, that is, int(C) ≠ ∅. Indeed, if int(C) is empty, then C must lie in an affine set of dimension less than d, and the normal vector v ∈ R d to any hyperplane containing this affine set satisfies (2.20), provided that we adjust the sign of v so that v ⋅ x ⩽ 0. We now distinguish two cases. First, if x ∉ C, then applying Step 1 to C gives (2.20) for C. On the other hand, if x ∈ C ∖C ⊆ ∂C, then Exercise 2.2 yields a sequence (x n ) n⩾1 ⊆ R d ∖C converging to x. Applying Step 1 to C and x n gives a sequence (v n ) n⩾1 of non-zero vectors with v n ⋅ x n ⩽ v n ⋅ z for all z ∈ C. Since v n ≠ 0, the normalized vector

v n ∶= v n v n
is well-defined and lies in the unit ball. It follows by compactness of the unit ball that the sequence (v n ) n⩾1 admits a subsequential limit v ≠ 0 with v ⋅ x ⩽ v ⋅ z for all z ∈ C. This completes the proof. ∎

Theorem 2.3 (Hahn-Banach). If C 1 ,C 2 ⊆ R d are disjoint non-empty convex sets, then there exists a non-zero vector v ∈ R d with sup{v ⋅ x 1 x 1 ∈ C 1 } ⩽ inf{v ⋅ x 2 x 2 ∈ C 2 }. (2.21) Proof. Consider the set C ∶= C 2 -C 1 = {x 2 -x 1 x 1 ∈ C 1 and x 2 ∈ C 2 },
and recall that it is convex by Exercise 2.1. Moreover, as C 1 and C 2 are disjoint, we have 0 ∉ C. It follows by the supporting hyperplane theorem that there exists a non-zero vector v ∈ R d with 0 ⩽ v ⋅ x for every x ∈ C, or equivalently v ⋅ x 1 ⩽ v ⋅ x 2 for all x 1 ∈ C 1 and x 2 ∈ C 2 . This completes the proof. ∎ Proposition 2.4 (Envelope of affine minorants). A function f ∶ R d → R ∪ {+∞} is convex and lower semi-continuous if and only if it is the supremum of its affine minorants.

Proof. Every affine function is convex and lower continuous, and the supremum of a family of convex and lower semi-continuous functions is also convex and lower semi-continuous by Exercise 2.7. This establishes the converse implication. Notice also that the equivalence is clearly valid when f is identically +∞. From now on, we therefore assume that f is proper, convex and lower semi-continuous, we fix x ∈ R d as well as µ < f (x), and we aim to find an affine function g such that g ⩽ f and g(x) ⩾ µ. Since f is convex and lower semi-continuous, its epigraph (2.4) is convex and closed. Moreover, it does not contain the pair (x, µ). It follows by the supporting hyperplane theorem for closed sets that there exist p ∈ R d , a ∈ R, and c 1 < c 2 such that for every (y,λ ) ∈ epi f ,

p ⋅ x + aµ = c 1 < c 2 ⩽ p ⋅ y + aλ . (2.22)
Recall that we assume that f is proper; that is, there exists

x 0 ∈ R d such that f (x 0 ) < +∞. Since {x 0 } × [ f (x 0 ),+∞) is a subset of epi f , we must have that a ⩾ 0.
We decompose the rest of the proof into two steps. First we treat the case a > 0, and then the case a = 0.

Step 1: a > 0. Dividing (2.22) by a > 0, we find that f (y) ⩾ a -1 p ⋅ (xy) + µ for every y ∈ dom f , and this inequality clearly extends to every y ∈ R d . The mapping y ↦ a -1 p ⋅ (xy) + µ is thus an affine minorant of f , and takes the value µ at x, as desired.

Step 2: a = 0. As a preliminary step we show that the set of affine minorants of f is not empty. Applying the same reasoning as that leading to (2.22) with x replaced by x 0 , and, say, with µ replaced by f (x 0 ) -1 < +∞, we get the existence of p 0 ∈ R d and a 0 ⩾ 0 such that

p 0 ⋅ x 0 + a 0 ( f (x 0 ) -1) < p 0 ⋅ y + a 0 λ
for every (y,λ ) ∈ epi f . Since (x 0 , f (x 0 )) ∈ epi f , it cannot be that a 0 = 0. Arguing as in Step 1, we thus deduce that the mapping g 0 (y) ∶= a -1 0 p 0 ⋅ (x 0y) + f (x 0 ) -1 is an affine minorant of f . Coming back to the main thread of the argument, in the case when a = 0, we obtain from (2.22) that for every y ∈ dom f ,

c 2 -c 1 + p ⋅ (x -y) ⩽ 0.
Using the affine minorant g 0 just constructed, we deduce that for every M > 0, and

y ∈ dom f , f (y) ⩾ g 0 (y) + M c 2 -c 1 + p ⋅ (x -y) , (2.23)
and this inequality is in fact valid for every y ∈ R d . Since c 2c 1 > 0, we can make sure that the value on the right side of (2.23) at y = x is as large as desired, in particular larger than µ, by choosing M sufficiently large. Once M is thus chosen, the right side of (2.23) defines a suitable affine minorant of f . This completes the proof. ∎

The Fenchel-Moreau theorem extends this result by identifying an explicit set of affine minorants of f whose supremum is f . In essence, for each slope p, we optimize the intercept to build the affine minorant with slope p that just touches f . In Exercise 2.10, it is shown that the convex bi-dual of a proper function f ∶ R d → R ∪ {+∞} is a convex and lower semi-continuous minorant of f . The Fenchel-Moreau theorem states that as soon as f is convex and lower semi-continuous, the inequality f ⩾ f * * is in fact an equality.

Theorem 2.5 (Fenchel-Moreau). A proper function f ∶ R d → R ∪ {+∞} is convex and lower semi-continuous if and only if it is equal to its convex bi-dual, f = f * * .
(2.24)

Remark 2.6. Throughout this section, we chose to avoid considering functions that may also take the value -∞; as a result, we required that f * be proper in order to define the bi-dual f * * . When stating the identity (2.24), we understand that it implicitly implies that f * * is well-defined, or in other words, that f * is proper. If f is not proper, that is if f is constant and equal to +∞, then we could decide that f * = -∞ and f * * = +∞, which is indeed equal to f in this case as well.

Proof of Theorem 2.5. Since f * * is convex and lower semi-continuous by (i) in Exercise 2.10, the converse implication is clear. To show the direct implication, we fix a proper convex and lower semi-continuous function f ∶ R d → R ∪ {+∞}, and aim to show that f * * = f . We start by showing that f * * is well-defined and that f * * ⩾ f . Let x ∈ R d and ε > 0. Invoking Proposition 2.4 gives an affine minorant g of f with g(x) ⩾ f (x)ε. Since g ⩽ f , we can use (iii) in Exercise 2.10 to obtain that g * ⩾ f * . Recalling from (2.16) that the dual of an affine function is proper, we obtain that f * is proper as well. It thus follows that f * * is well-defined, and another application of (iii) in Exercise 2.10 yields that g * * ⩽ f * * . Remembering that an affine function satisfies the Fenchel-Moreau theorem by (2.17) shows that in fact

f (x) -ε ⩽ g(x) ⩽ f * * (x).
Letting ε tend to zero completes the proof of the inequality f ⩽ f * * . The converse inequality is immediate, see part (ii) of Exercise 2.10. ∎ Remark 2.7. As was pointed out in the previous remark, we chose throughout to exclude the possibility that convex functions take the value -∞. If we were to allow for this possibility, we would then modify the definition and require that (2.2) holds whenever it is unambiguous; or equivalently, that it holds for every x,y ∈ R d such that f (x) < +∞ and f (y) < +∞. If a convex function f takes the value -∞ at some point x ∈ R d , then on any half-line emanating from x, the function f will either be -∞ all the way; or else it must take the value -∞ for a while, and then jump to +∞, with the value of f at the transition point being some arbitrary element of R ∪ {-∞,+∞}. If f is also lower semi-continuous, then it must be -∞ on some closed convex set, and +∞ outside of it. There cannot be any affine minorant to a function that takes the value -∞ somewhere, and in particular, if we were to define the bi-dual for such a function, it could only be the constant function equal to -∞.

Exercise 2.10. Let f ∶ R d → R ∪ {+∞} be a proper function. Show that its convex dual satisfies the following properties.

(i) f * is convex and lower semi-continuous.

(ii) f * * ⩽ f . (iii) If f ⩽ g for some proper function g ∶ R d → R ∪ {+∞}, then f * ⩾ g * . Exercise 2.11. Consider the norm function f (x) ∶= 1 2 x 2 . Show that f * = f . Exercise 2.12. Let f ∶ R d → R be a real-valued Lipschitz continuous convex function with Lipschitz constant L. Show that f * (x) = +∞ for all x ∈ R d with x > L.
Exercise 2.13. Let f ∶ R d → R ∪ {+∞} be a convex function, and denote by f the lower semi-continuous envelope of f , as defined in Exercise 2.9. Show that we have f * = ( f ) * and f * * = f . Exercise 2.14. Let K be a closed convex cone -this means that K is a closed convex set with the additional property that, for all x ∈ K and λ > 0, we have λ x ∈ K. We also assume that K is not empty. The polar of K is the closed convex cone

K ○ ∶= {v ∈ R d v ⋅ x ⩽ 0 for all x ∈ K}.
(2.25)

Show that K = K ○○ .
Exercise 2.15. We use the notion of cone and of the polar of a set K as defined in Exercise 2.14. Let C be a closed convex set. The normal to C at a point x ∈ C is the closed convex cone

n C (x) ∶= {v ∈ R d v ⋅ (x ′ -x) ⩽ 0 for all x ′ ∈ C}. (2.26)
The tangent to C at a point x ∈ C is the closed convex cone

T C (x) ∶= {λ (x ′ -x) x ′ ∈ C and λ ⩾ 0}. (2.27) (i) Show that for all x ∈ int(C), we have n C (x) = {0} and T C (x) = R d . (ii) Show that for all x ∈ C, we have n C (x) = T C (x) ○ and T C (x) = n C (x) ○ . (iii) Fix x ∈ C. Prove that v ∈ T C (x)
if and only if there is a sequence (x i ) i⩾1 ⊆ C converging to x and a sequence (t i ) i⩾1 ⊆ R >0 decreasing to 0 with t -1 i (x ix) → v. Exercise 2.16. The purpose of this exercise is to generalize Lemma 2.1 to the Hilbert space setting. Let H be a Hilbert space and let C ⊆ H be a closed convex subset of H.

(i) Show that for every x ∈ H, there is a unique point P C (x) ∈ C with (ii) Let f ∶ H → R be a continuous linear functional on H. Show that there exists a unique y ∈ H with f (x) = x ⋅ y for all x ∈ H.

x -P C (x) 2 = inf x -y 2 y ∈ C . ( 2

Continuity and almost everywhere differentiability

In this section we will show that a convex function is locally Lipschitz continuous on the interior of its effective domain. Although this does not pertain to convex analysis, we will also show Rademacher's theorem stating that a locally Lipschitz continuous function is almost everywhere differentiable. The multi-dimensional version of Rademacher's theorem relies on its one-dimensional counterpart whose proof is the content of Exercise 2.18. We start by showing that a convex function that is locally bounded from above is in fact locally bounded and locally Lipschitz continuous. For every r > 0 and x ∈ R d , we denote by B r (x) the open Euclidean ball of radius r centred at x. (i) There exists m ∈ R such that for every x ∈ B 2δ (x * ), we have m ⩽ f (x) ⩽ M.

(ii) For every x,y ∈ B δ (x * ), we have f (x)f (y) ⩽ M-m δ xy . Proof. The convexity of f reveals that for any x ∈ B 2δ (x * ),

f (x * ) = f 1 2 (2x * -x) + 1 2 x ⩽ 1 2 f (x * + (x * -x)) + 1 2 f (x) ⩽ 1 2 M + 1 2 f (x).
This implies that f (x) ⩾ 2 f (x * )-M, so the first assertion holds with m ∶= 2 f (x * )-M.

To prove the second assertion, fix x,y ∈ B δ (x * ) and consider the point z ∶= y + δ y-x y-x in B 2δ (x * ). Since y lies on the line segment joining x and z,

y = y -x δ + y -x z + δ δ + y -x x,
the convexity of f implies that

f (y) -f (x) ⩽ y -x δ + y -x ( f (z) -f (x)) ⩽ M -m δ y -x .
Interchanging the roles of x and y completes the proof. ∎ Proposition 2.9 (continuity). Let f ∶ R d → R ∪ {+∞} be a proper and convex function. If C ⊆ int(dom( f )) is a compact set, then there exists L < +∞ such that for every x,y ∈ C, f (x)f (y) ⩽ L xy .

(2.31)

Proof. For each x ∈ R d and r > 0, denote by C r (x) ∶= ∏ d i=1 [x ir,x i + r] the cube of side-length 2r centred at x, or equivalently the closed ∞ -ball of radius r centred at x. We fix x * ∈ C ⊆ int(dom f ), and pick δ = δ (x * ) > 0 and r > 0 such that x * ∈ B 2δ (x * ) ⊆ C r (x * ) ⊆ dom( f ). Denote by (v i ) i⩽2 d the vertices of C r (x * ), and observe that any point in the cube C r (x * ) may be written as a convex combination of the vertices (v i ) i⩽2 d . In particular, for any y ∈ B 2δ (x * ), there exist non-negative scalars

(α i ) i⩽2 d with y = 2 d i=1 α i v i and 2 d i=1 α i = 1.

It follows by convexity of

f that f (y) ⩽ max{ f (v 0 ),..., f (v n )}. Invoking Lemma 2.8
shows that f is Lipschitz continuous on B δ (x * ), and remembering that C is compact implies that f is in fact Lipschitz continuous on C. This completes the proof. ∎ Together with Rademacher's theorem on the almost everywhere differentiability of Lipschitz functions, this result implies that a real-valued convex function is almost everywhere differentiable.

Theorem 2.10 (Rademacher). If U ⊆ R d is open and f ∶ U → R is Lipschitz continu- ous, then f is differentiable almost everywhere on U.
Proof. The proof proceeds in three steps. First, we use the one-dimensional Rademacher theorem to show that for every direction vector ν ∈ R d , the directional derivative

D ν f (x) ∶= d dt t=0 f (x +tν) (2.32)
exists almost everywhere on U, then we show that it is almost everywhere given by ∇ f (x)⋅ν, and finally we use the separability and compactness of the unit sphere and the Lipschitz continuity of f to conclude that f is differentiable almost everywhere.

Step 1: almost everywhere existence of the directional derivative. Fix a direction vector ν ∈ R d ∖ {0}, and denote by

A ν ∶= {x ∈ U D ν f (x) exists}
the set of points where the derivative of f in the direction ν exists. Denote by ν ⊥ the orthogonal complement of the subspace Rν, so that by Exercise 2.17, we have the decomposition R d = Rν ⊕ ν ⊥ . By the one-dimensional Rademacher theorem established in Exercise 2.18, for each y ∈ ν ⊥ , the intersection of the set U ∖ A ν with the line parallel to Rν through the point y is of zero measure. It follows by the Fubini-Tonelli theorem that U ∖ A ν is of zero measure, and therefore that the directional derivative D ν f exists almost everywhere on U.

Step 2: D ν f (x) = ∇ f (x) ⋅ ν almost everywhere. Fix a direction vector ν ∈ R d as well as a smooth function φ ∈ C ∞ c (U;R) of compact support. A linear change of variables reveals that for any t > 0 small enough,

ˆU f (x +tν) -f (x) t φ (x)dx = -ˆU f (x) φ (x -tν) -φ (x) -t dx.
Since f is Lipschitz continuous and φ is compactly supported, both integrands in this equality are uniformly bounded and vanish outside a compact set. Using the dominated convergence theorem to let t tend to zero shows that

ˆU D ν f (x)φ (x)dx = -ˆU f (x) ∇φ (x) ⋅ ν dx.
In particular, choosing ν = e i for the canonical basis vector

e i ∈ R d reveals that ˆU ∂ x i f (x)φ (x)dx = -ˆU f (x)∂ x i φ (x)dx. It follows that ˆU D ν f (x)φ (x)dx = - d i=1 ν i ˆU f (x)∂ x i φ (x)dx = ˆU ∇ f (x) ⋅ ν φ (x)dx, (2.33)
where here we just use ∇ f (x) as a notation for

∇ f (x) = (∂ x 1 f (x),...,∂ x d f (x)) = (D e 1 f (x),...,D e d f (x)),
which is well-defined almost everywhere on U by the previous step. Since the function D ν f -∇ f ⋅ ν is uniformly bounded by the Lipschitz continuity of f , applying the equality (2.33) along a sequence (φ n ) n⩾1 of smooth functions of compact support that approximate

D ν f -∇ f ⋅ ν in L 1 (R d ;R) shows that D ν f = ∇ f ⋅ ν almost everywhere on U.
Step 3: concluding that f is differentiable almost everywhere. Let (ν i ) i⩾1 ⊆ S d-1 be a countable set of direction vectors that is dense in the unit sphere S d-1 ⊆ R d . By the additivity of measure as well as Steps 1 and 2, the complement of the set

A ∶= ∞ ⋂ i=1 x ∈ U ∇ f (x) exists, D ν i f (x) exists and D ν i f (x) = ∇ f (x) ⋅ ν i
has zero measure. Here we understand the phrase "∇ f (x) exists" to mean that ∂ x i f (x) = D e i f (x) exists for every i ∈ {1,...,d}.

We now fix x ∈ A and show that f is differentiable at x. If we introduce the error function

R(x,ν,t) ∶= f (x +tν) -f (x) t -∇ f (x) ⋅ ν,
this comes down to proving that, given ε > 0, it is possible to find δ = δ (ε,x) > 0 such that for all ν ∈ S d-1 and t > 0 with t < δ , we have R(x,ν,t) ⩽ ε.

(2.34)

Writing L for the Lipschitz constant of f , we have that ∂ x i f (x) ⩽ L and ∇ f (x) ⩽ √ dL, so the Cauchy-Schwarz inequality implies that R is Lipschitz continuous in ν. Indeed, for any ν,ν ′ ∈ S d-1 , we have

R(x,ν,t) -R(x,ν ′ ,t) ⩽ ( √ d + 1)L ν -ν ′ .
By the compactness of S d-1 , there is a finite sub-collection (ν i ) i⩽n of direction vectors with the property that the open balls (B ε (ν i )) i⩽n cover the unit sphere S d-1 . Since each error term R(x,ν i ,t) vanishes as t tends to zero, it is possible to find δ > 0 such that R(x,ν i ,t) ⩽ ε whenever t < δ and 1

⩽ i ⩽ n. Now, if ν ∈ S d-1 belongs to the open ball B ε (ν i ) and t < δ , then R(x,ν,t) ⩽ R(x,ν i ,t) + R(x,ν i ,t) -R(x,ν,t) ⩽ ε + ( √ d + 1)ε.
Redefining ε establishes (2.34) and completes the proof. ∎ (i) For any step function of the form φ ∶= ∑

n i=1 φ i 1 [a i-1 ,a i ) , with φ 1 ,...,φ n ∈ R and a ⩽ a 0 ⩽ ⋯ ⩽ a n ⩽ b, define T (φ ) ∶= n i=1 φ i F(a i ) -F(a i-1 ) . (2.35)
Prove that the operator T admits a unique extension to a continuous functional on L 2 ([a,b];R).

(ii) Use Exercise 2.17 to find f ∈ L 2 ([a,b];R) such that, for all x ∈ [a,b], we have (i) Slightly adapting the notation in (2.5), for every set A, we may write I {x∈A} instead of I A (x); for instance, we understand that I {x⩾0} is equal to 0 if x ⩾ 0, and is +∞ otherwise. What is the convex dual of the mapping x ↦ I {x⩾0} ?

F(x) = F(a) + ˆx a f (t)dt. ( 2
(ii) For every x,z ∈ R d , we write φ (x) ∶= I {x⩾0} + I {Ax⩾c} and

ψ(z) ∶= inf y∈R k c ⋅ y + I {y⩽0} + I {A * y⩾z} . (2.38)
Show that φ = ψ * .

(iii) For each z ∈ R d , we define the set

K z ∶= y ∈ R k y ⩾ 0 and A * y ⩽ z . (2.39)
Assuming that K z is a compact set for every z in a neighbourhood of b, show that the two variational problems in (2.37) are equal.

The subdifferential of a convex function

The local Lipschitz continuity of a real-valued convex function and the Rademacher theorem imply that a real-valued convex function is differentiable almost everywhere. It is at times useful to also be able to study the differentiability of a convex function at every single point of its effective domain using the notion of subdifferential. The subdifferential of a convex function

f ∶ R d → R ∪ {+∞} at a point x ∈ dom f is the set ∂ f (x) ∶= p ∈ R d f (y) ⩾ f (x) + p ⋅ (y -x) for all y ∈ R d . (2.40)
We start by showing that the subdifferential of f at a point in the interior of its effective domain is not empty.

Proposition 2.11. Let f ∶ R d → R ∪ {+∞} be a convex function. For every point x ∈ int(dom f ), the subdifferential ∂ f (x) is not empty. Proof. We consider the convex set C ∶= {(y, µ) ∈ dom( f ) × R f (y) < µ}, and fix x ∈ int(dom f ). Since (x, f (x)) ∉ C the supporting hyperplane theorem gives a non-zero vector (v,b) ∈ R d × R with 0 ⩽ v ⋅ (y -x) + b(µ -f (x))
for every (y, µ) ∈ C. Since µ can be arbitrarily large, we must have b ⩾ 0. If we had b = 0, then we would have 0 ⩽ v⋅(y-x) for all y in a neighbourhood of x ∈ int(dom f ), which is not possible since (v,b) is non-zero. This means that b > 0 so the vector p ∶= -v b is well-defined and satisfies

µ ⩾ f (x) + p ⋅ (y -x)
for all (y, µ) ∈ C. Letting µ tend to f (y) reveals that p ∈ ∂ f (x), which means that the subdifferential ∂ f (x) is not empty. ∎

It turns out that a convex function is differentiable at a point x in the interior of its effective domain if and only its subdifferential at x consists of a singleton. To prove this, we will leverage the fact that a convex function is differentiable at a point in the interior of its effective domain if and only if its directional derivative is a linear function of the direction. The directional derivative of a real-valued

convex function f ∶ R d → R ∪ {+∞} at x ∈ int(dom f ) in the direction of ν ∈ R d is the function D ν f (x) ∶= lim t↘0 f (x +tν) -f (x) t . (2.41)
The convexity of f implies that the difference quotient defining the directional derivative is a decreasing function of t, so the directional derivative is well-defined as a monotone limit. In general, the linearity of the directional derivative does not suffice to characterize the differentiability of a function; however, it does suffice for locally Lipschitz continuous functions, a fact that we already used at least implicitly in the proof of the Rademacher theorem. Combining this with the local Lipschitz continuity of convex functions yields the following result.

Lemma 2.12.

A convex function f ∶ R d → R∪{+∞} is differentiable at x ∈ int(dom f ) if and only if the map ν ↦ D ν f (x) is linear. In this case, we have D ν f (x) = ∇ f (x)⋅ν.
Proof. On the one hand, if f is differentiable, then for any direction vector ν ∈ R d , we have

D ν f (x) = ∇ f (x) ⋅ ν. Conversely, suppose that ν ↦ D ν f (x) is linear, that is, there exists a ∈ R d such that D ν f (x) = a ⋅ ν for every ν ∈ R d .
We assume for the sake of contradiction that f is not differentiable at x. Let (ν n ) n⩾1 with ν n = 1 for all n ⩾ 1 and (t n ) n⩾1 ⊆ R >0 be a sequence converging to 0 such that the error term

R(x,ν n ,t n ) ∶= f (x +t n ν n ) -f (x) t n -a ⋅ ν n
does not converge to zero. Up to passing to a subsequence, assume that (ν n ) n⩾1 converges to some ν 0 in the unit sphere. Remembering Proposition 2.9, denote by L < +∞ the Lipschitz constant of f in a neighbourhood of x. The triangle inequality implies that

R(x,ν n ,t n ) ⩽ R(x,ν 0 ,t n ) + (L + a ) ν n -ν 0 .
Leveraging the assumption that D ν 0 f (x) = a ⋅ ν 0 to let n tend to infinity contradicts the absurd hypothesis that the sequence (R(x,ν n ,t n )) n⩾1 does not converge to zero. This completes the proof.

∎ Theorem 2.13. A convex function f ∶ R d → R ∪ {+∞} is differentiable at a point x ∈ int(dom f ) if and only if ∂ f (x) consists of a singleton. In this case, we have ∂ f (x) = {∇ f (x)}.
Proof. We first show the direct implication. Recall from Proposition 2.11 that the subdifferential ∂ f (x) is not empty. We fix p ∈ ∂ f (x). By definition of the subdifferential, for every v ∈ R d and λ > 0,

f (x + λ v) -f (x) ⩾ λ v ⋅ p.
Dividing by λ and letting λ tend to zero shows that

(∇ f (x) -p) ⋅ v ⩾ 0. Choosing v = p -∇ f (x) reveals that p = ∇ f (x), so ∂ f (x) = {∇ f (x)}.
Conversely, suppose that the subdifferential is a singleton, ∂ f (x) = {p}, and fix a direction vector ν ∈ R d . The convexity of f and the definition of the directional derivative imply that for all λ ∈ R,

f (x) + λ D ν f (x) ⩽ f (x + λ ν).
This implies that the convex sets

C ∶= {(x + λ ν, f (x) + λ D ν f (x)) λ ∈ R} and C ′ ∶= {(y, µ) ∈ dom f × R f (y) < µ}
are disjoint. It follows by the Hahn-Banach separation theorem that there exists a non-zero vector

(a,b) ∈ R × R d with a( f (x) + λ D ν f (x)) + b ⋅ (x + λ ν) ⩽ aµ + b ⋅ y (2.42)
for all (y, µ) ∈ C ′ and λ ∈ R. Taking λ = 0 shows that

a f (x) + b ⋅ x ⩽ aµ + b ⋅ y (2.43)
for all (y, µ) ∈ C ′ . Since µ can be arbitrarily large, we must have a ⩾ 0. If we had a = 0, then we would have 0 ⩽ b⋅(y-x) for all y in a neighbourhood of x ∈ int(dom f ), which is not possible since (a,b) is non-zero. Dividing through by a and letting µ tend to f (y) in (2.43) shows that -b a ∈ ∂ f (x), and therefore b a = -p. Combining this with (2.42) and letting µ tend to f (y) in the resulting bound gives

f (x) + λ D ν f (x) -p ⋅ (x + λ ν) ⩽ f (y) -p ⋅ y
for all y ∈ dom f and λ ∈ R. Taking y = x reveals that λ (D ν f (x)p ⋅ ν) ⩽ 0 for all λ ∈ R, which implies that D ν f (x) = p ⋅ ν. In particular, the map ν ↦ D ν f (x) is linear. Invoking Lemma 2.12 completes the proof. ∎

The next proposition gives a sort of continuity property of the subdifferential as we vary the base point.

Proposition 2.14. Let f ∶ R d → R ∪ {+∞} be a convex function, and let (x n , p n ) n⩾1 be a sequence of points in dom f ×R d with p n ∈ ∂ f (x n ) for each n ⩾ 1 that converges to some point (x, p) ∈ dom f × R d . If f is continuous at x ∈ dom f , then p ∈ ∂ f (x). Proof. Fix y ∈ R d as well as n ⩾ 1. Since p n ∈ ∂ f (x n ), we have f (y) ⩾ f (x n ) + p n ⋅ (y -x n )
Letting n tend to infinity and using the continuity of f at x completes the proof. ∎ We recall that if the point x in Proposition 2.14 belongs to int(dom f ), then the continuity of f at x is automatically satisfied by Proposition 2.9. Although we will not use this fact, we also mention that one can use Proposition 2.14 to show that if a convex function f is uniformly Lipschitz continuous on its effective domain, then the subdifferential of f at every point x ∈ dom f is not empty, including at points x that sit on the boundary of dom f . Proposition 2.15. Let x ∈ R d , and for each integer n [START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF]. It follows by definition of the subdifferential that, for every y ∈ R d ,

⩾ 1, let f n ∶ R d → R ∪ {+∞} be a convex function such that x ∈ int(dom f n ) and with f n differentiable at x ∈ R d . If ( f n ) n⩾1 converges pointwise to some function f ∶ R d → R ∪ {+∞} with x ∈ dom f , and if the sequence of derivatives (∇ f n (x)) n⩾1 converges to some vector p ∈ R d , then p ∈ ∂ f (x). Proof. Since f n is differentiable at x ∈ int(dom f ), we have ∇ f n (x) ∈ ∂ f n (x) by Theorem 2.
f n (y) ⩾ f n (x) + ∇ f n (x)(y -x).
Letting n tend to infinity and using the pointwise convergence of f n to f completes the proof. ∎

For a more in-depth discussion of convex analysis, we refer the interested reader to [START_REF] Bertsekas | Convex optimization theory[END_REF][START_REF] Ekeland | Convex analysis and variational problems[END_REF]137,[START_REF] Mordukhovich | An easy path to convex analysis and applications[END_REF][START_REF] Rockafellar | Convex analysis[END_REF].

Exercise 2.20. We use the notation conv(C) from Exercise 2.4 for the convex hull of a set C, and the notation n C (x) from Exercise 2.15 for the normal cone at

x ∈ C. Let f ∶ R d → R ∪ {+∞} be a convex function such that dom f has non-empty interior. For each x ∈ dom f , let S(x) ∶= lim i→+∞ ∇ f (x i ) (x i ) i⩾1 ⊆ intdom f converges to x and
for every i ⩾ 1, the function f is differentiable at x i (2.44) be the set of limits of gradients of f along sequences that converge to x. Suppose that f is Lipschitz on dom f , and show that

∂ f (x) = n dom f (x) + conv S(x) .
(2.45)

Although we do not prove this, let us mention that the Lipschitz assumption is not necessary. Indeed, if f ∶ R d → R ∪ {+∞} is a convex and lower semicontinuous function such that dom f has non-empty interior, then

∂ f (x) = n dom f (x) + conv S(x) . (2.46)
We refer the interested reader to Theorem 25.6 in [START_REF] Rockafellar | Convex analysis[END_REF] or Theorem 4.5 in [START_REF] Heinz | Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces[END_REF] for a proof of this more general formula.

Large deviation principles

One of the first quantities that one tries to understand in any basic course in probability theory is the sample average of a sequence of i.i.d. coin tosses. More specifically, given a collection of independent Bernoulli random variables (X n ) n⩾1 with mean p ∈ [0,1], one is often presented with two main results about the sample average

S N ∶= 1 N N n=1 X n (2.47)
of N of these random variables. The first is the law of large numbers, which states that the sample average S N converges almost surely to the mean p as the number of trials N goes to infinity. The second is the central limit theorem, which is the assertion that the typical deviations of the sample average from the mean p are of order 1 √ N and are normally distributed. Large deviation principles are concerned with the rare deviations, of order one, of the sample average S N from its mean p.

In the setting of i.i.d. coin tosses, studying large deviations comes down to analyzing the asymptotic behaviour of the probabilities

P S N = k N = N k p k (1 -p) N-k (2.48)
for k ∈ {0,...,N}. Stirling's formula implies that for a fixed x = k N ∈ (0,1), we have

log N k = -N xlog(x) + (1 -x)log(1 -x) + O(log(N)). (2.49)
This means that

logP S N = k N = -NI(x) + O(log(N)) (2.50)
for the rate function

I(x) ∶= xlog x p + (1 -x)log 1 -x 1 -p . ( 2.51)
The probability that S N lies above x ∈ [p,1) is therefore bounded from below by

P{S N ⩾ x} ⩾ P S N = ⌈Nx⌉ N = exp(-NI(x) + O(log(N)) , (2.52)
and bounded from above by

P{S N ⩾ x} ⩽ N k=⌊xN⌋ P S N = k N ⩽ exp -NI(x) + O(log(N)) , (2.53)
where we have used that I is increasing on the interval [p,1) in the second inequality.

Combining these bounds and letting N tend to infinity reveals that for every x ∈ [p,1),

lim N→+∞ 1 N logP{S N ⩾ x} = -I(x). (2.54)
Similar arguments can be used to show that for x ∈ (0, p].

lim N→+∞ 1 N logP{S N ⩽ x} = -I(x).
(2.55)

In particular, the large deviations of (S N ) N⩾1 from its mean p are entirely described by the rate function I. As shown in Exercise 2.21, the equalities (2.54) and (2.55) are equivalent to the fact that for every Borel set A ⊆ R,

-inf x∈int(A) I(x) ⩽ liminf N→+∞ 1 N logP{S N ∈ A} ⩽ limsup N→+∞ 1 N logP{S N ∈ A} ⩽ -inf x∈A I(x), (2.56)
where we recall that we write int(A) to denote the interior of the set A. This string of inequalities is more amenable to generalization than (2.54) and (2.55) as it requires neither that we stipulate the mean p of the random variables (S N ) N⩾1 , nor that the underlying space be ordered. We will therefore say that a sequence (S N ) N⩾1 of random variables taking values in a topological space S satisfies a large deviation principle with rate function I ∶ S → R if, for every Borel set A ⊆ S, we have limsup

N→+∞ 1 N logP{S N ∈ A} ⩽ -inf x∈A I(x), (2.57)
and liminf

N→+∞ 1 N logP{S N ∈ A} ⩾ -inf x∈int(A) I(x).
(2.58)

Since int(A) ⊆ A ⊆ A, there is no loss in generality in restricting our attention to closed sets in the upper bound (2.57), and to open sets in the lower bound (2.58). Heuristically, if (S N ) N⩾1 satisfies a large deviation principle, then for every x ∈ S,

P{S N ≃ x} ≃ exp(-NI(x)), (2.59)
so S N should concentrate around the minima of its rate function. This is consistent with the fact that the rate function (2.51) is minimized at the mean p. Although the theory of large deviations can be developed for rather general spaces S, we will restrict our attention to the case S = R here, and derive a general criterion involving log-Laplace transforms to determine when a sequence (S N ) N⩾1 satisfies a large deviation principle. Extending this result to the case of S = R d is an interesting exercise which we leave to the curious reader. The log-Laplace transform of a random variable X is the function

ψ ∶ R → R ∪ {+∞} defined by ψ(λ ) ∶= logEexp(λ X). (2.60)
To motivate the general large deviation principle that we will prove, and see how the log-Laplace transform is related to it, consider a sequence (X n ) n⩾1 of bounded and centred i.i.d. random variables and denote by

S N ∶= 1 N N n=1 X n (2.61)
the sample average of N of these random variables. Chebyshev's inequality implies that for any x ∈ R and λ ⩾ 0, we have

P{S N ⩾ x} ⩽ exp(-λ Nx)Eexp(λ NS N ) = exp -N λ x - 1 N ψ N (λ N) , (2.62)
where ψ N denotes the log-Laplace transform of S N . If we write ψ = ψ 1 for the log-Laplace transform of X 1 , then the independence of the random variables

(X n ) n⩾1 reveals that 1 N ψ N (λ N) = 1 N logEexp(λ NS N ) = logEexp(λ X 1 ) = ψ(λ ). (2.63)
Substituting this into (2.62) and taking the infimum over all λ ⩾ 0 in the resulting bound shows that 1

N logP{S N ⩾ x} ⩽ -sup λ ⩾0 (λ x -ψ(λ )). (2.64)
This suggests that the sample mean S N might satisfy a large deviation principle with rate function given by the convex dual ψ * . This is indeed correct and known as Cramér's theorem. It is a special case of the general large deviation principle that we now prove.

Theorem 2.16 (General LDP on R). Let (S N ) N⩾1 be a sequence of real random variables, and for each integer N ⩾ 1 denote by ψ N ∶ R → R the log-Laplace transform of S N . If there exists ψ ∈ C 1 (R;R) such that for all λ ∈ R, we have

lim N→+∞ 1 N ψ N (λ N) = ψ(λ ), (2.65)
then (S N ) N⩾1 satisfies a large deviation principle with rate function ψ * .

The knowledgeable reader will recognize in Theorem 2.16 a version of the Gärtner-Ellis theorem (see for instance Theorem 2.3.6 of [START_REF] Dembo | Large deviations techniques and applications[END_REF]), with a slightly less general but significantly simpler assumption. Although we will not prove this here, we point out that Theorem 2.16 is also valid as stated for random variables taking values in R d .

Proof of Theorem 2.16. We start by deriving a handful of convenient properties of the convex dual

ψ * (x) = sup λ ∈R (λ x -ψ(λ ) . (2.66)
Evaluating (2.66) at λ = 0, and using that ψ(0) = 0, we see that ψ * is non-negative,

ψ * ⩾ 0. (2.67)
We set m ∶= ψ ′ (0). Since ψ is convex by Exercise 2.6 and the fact that a limit of convex functions is convex, we have that ψ is bounded from below by the linear function λ ↦ mλ (this can be seen as a consequence of Theorem 2.13). It follows that ψ * (m) = 0 and sup

λ ⩽0 (λ x -ψ(λ )) ⩽ sup λ ⩽0 (λ x -λ m) = 0 (2.68)
for every x ⩾ m. In particular, for every x ⩾ m, the supremum in (2.66) can be restricted to

R ⩾0 , ψ * (x) = sup λ ⩾0 (λ x -ψ(λ )). (2.69)
We decompose the rest of the proof into four steps. First we derive an upper bound for the probability that S N is greater than some x ⩾ m, and we use this bound to establish the large deviation upper bound. We then obtain a lower bound for the probability that S N is an a small neighbourhood of some value x ∈ R, and finally we use this lower bound to establish the large deviation lower bound.

Step 1: upper bound on P{S N ⩾ x}. In this step, we show that for every x ⩾ m,

limsup N→+∞ 1 N logP{S N ⩾ x} ⩽ -ψ * (x).
(2.70)

Given λ ⩾ 0, Chebyshev's inequality implies that

P{S N ⩾ x} ⩽ exp(-λ Nx)Eexp(λ NS N ) = exp -λ Nx + ψ N (λ N) .
It follows by the assumption (2.65) that limsup

N→+∞ 1 N logP{S N ⩾ x} ⩽ ψ(λ ) -λ x.
Taking the supremum over λ ⩾ 0 and remembering (2.69) establishes (2.70).

Step 2: large deviation upper bound. We now show the large deviation upper bound; that is, we prove that for every closed set A ⊆ R,

limsup N→+∞ 1 N logP{S N ∈ A} ⩽ -inf x∈A ψ * (x). (2.71)
First, we observe that by the same reasoning as in Step 1, for every x ⩽ m,

limsup N→+∞ 1 N logP{S N ⩽ x} ⩽ -ψ * (x). (2.72)
Given a closed set A ⊆ R, we use the value m = ψ ′ (0) defined before Step 1 to partition the probability that S N lies in A, ) is not empty, then we denote by x ⩾ m its infimum. This infimum belongs to A since A is closed, and

P{S N ∈ A} ⩽ P S N ∈ A ∩ (-∞,m] + P S N ∈ A ∩ [m,+∞) . (2.73) If A ∩ [m,+∞
P S N ∈ A ∩ [m,+∞) ⩽ P{S N ⩾ x}. It follows by (2.70) that limsup N→+∞ 1 N logP S N ∈ A ∩ [m,+∞) ⩽ -ψ * (x) ⩽ -inf x∈A ψ * (x).
(2.74)

The last inequality remains valid if A ∩ [m,+∞) is empty provided that we understand that log0 = -∞, and that if A itself is empty, then the infimum over A is +∞.

Similar observations based on (2.72) as opposed to (2.70) reveal that limsup

N→+∞ 1 N logP S N ∈ A ∩ (-∞,m] ⩽ -inf x∈A ψ * (x).
Combining this with (2.74) and (2.73), we obtain (2.71).

Step 3: lower bound for P{S N ≃ x}. Since ψ * is convex, non-negative and vanishes at m by Exercise 2.10 and equations (2.67) and (2.68), it must be non-increasing on (-∞,m] and non-decreasing on [m,+∞). Moreover, as ψ ′ is continuous, its image must be an interval of R whose endpoints we denote by

d -⩽ d + ∈ [-∞,+∞].
In this step, we fix x ∈ (d -,d + ) and show that for every ε > 0,

liminf N→+∞ 1 N logP S N ∈ [x -ε,x + ε] ⩾ -ψ * (x). (2.75)
Looking at the optimality condition for the supremum in (2.69), it is natural to define λ ∈ R to be such that x = ψ ′ (λ ). Such a λ exists since x belongs to (d -,d + ).

For definiteness, we assume that λ ⩾ 0 and observe that

P S N ∈ [x -ε,x + ε] ⩾ exp -λ N(x + ε) Eexp(λ NS N )1 {S N ∈[x-ε,x+ε]} . (2.76)
In order to conclude, we would like to remove the indicator function in the expectation above. Loosely speaking, we would like to verify that our choice of λ ensures that

Eexp(λ NS N )1 {S N ∉[x-ε,x+ε]} Eexp(λ NS N ) ≪ 1.
This is a question about a probability upper bound very much like the one studied in Step 1. For every µ ⩾ 0, we have

Eexp(λ NS N )1 {S N ⩾x+ε} ⩽ exp(-Nµ(x + ε))Eexp((λ + µ)NS N ) = exp -Nµ(x + ε) + ψ N (Nλ + Nµ)) . (2.77)
Similarly, for every µ ⩾ 0, we have

Eexp(λ NS N )1 {S N ⩽x-ε} ⩽ exp(Nµ(x -ε))Eexp((λ -µ)NS N ) = exp Nµ(x -ε) + ψ N (Nλ -Nµ)) . (2.78)
Since ψ ∈ C 1 (R;R), any µ > 0 sufficiently small will be such that

ψ(λ + µ) -ψ(λ ) -µψ ′ (λ ) + ψ(λ -µ) -ψ(λ ) + µψ ′ (λ ) ⩽ ε µ 4 .
Fixing such a µ > 0, recalling the convergence (2.65) and remembering that we chose ψ ′ (λ ) = x, we obtain that for every N sufficiently large,

ψ N (Nλ + Nµ) -ψ N (Nλ ) -Nµx + ψ N (Nλ -Nµ) -ψ N (Nλ ) + Nµx ⩽ Nε µ 2 .
Combining this with (2.77) and (2.78) reveals that for every N sufficiently large,

Eexp(λ NS N )1 {S N ∉[x-ε,x+ε]} ⩽ 2exp - Nε µ 2 Eexp(λ NS N ),
and in particular,

Eexp(λ NS N )1 {S N ∈[x-ε,x+ε]} ⩾ 1 2 Eexp(λ NS N ).
Substituting this into (2.76), we get that, for every N sufficiently large,

P S N ∈ [x -ε,x + ε] ⩾ 1 2 exp -λ N(x + ε) Eexp(λ NS N ).
Observing that the left side of (2.75) only gets smaller as ε decreases, letting N tend to infinity and then ε tend to zero establishes (2.75).

Step 4: large deviation lower bound. Finally, we show the large deviation lower bound; that is, we prove that for every open set A ⊆ R,

limsup N→+∞ 1 N logP{S N ∈ A} ⩾ -inf x∈A ψ * (x). (2.79)
To start with, we extend (2.75) to every x ∈ R.

∎

To finish our discussion of large deviation principles, we explore the necessity of assuming that the rate function in Theorem 2.16 be continuously differentiable. This possibly surprising assumption will also appear in the Hamilton-Jacobi approach when we prove the convex selection principle in Section 3.6.

Example 2.18. In this example we show that the assumption ψ ∈ C 1 (R;R) in Theorem 2.16 is necessary. Consider a sequence (S N ) N⩾1 of Rademacher random variables,

P{S N = 1} = P{S N = -1} = 1 2 ,
and denote by ψ N the log-Laplace transform of S N (which here does not actually depend on N). A direct computation shows that ψ N (λ ) = logcosh(λ ), and by Exercise 1.1,

ψ(λ ) ∶= lim N→+∞ 1 N ψ N (λ N) = lim N→+∞ 1 N log e Nλ 2 + e -Nλ 2 = λ .
Since ψ is Lipschitz with Lipschitz constant equal to one, Exercise 2.12 implies that

ψ * (x) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 0 x ∈ [-1,1], +∞ otherwise.
If Theorem 2.16 were true, then for all N large enough, we would have

1 N logP S N ∈ (-1 2,1 2) ⩾ - inf x∈(-1 2,1 2) ψ * (x) - 1 2 = - 1 2 ,
and therefore

0 = P{S N ∈ (-1 2,1 2)} ⩾ exp(-N 2).
This contradiction establishes the necessity of the assumption ψ ∈ C 1 (R;R).

We briefly discuss further the phenomenon at play in this example, without going into details -the reader can safely skip this imprecise paragraph. Suppose that a sequence (S N ) N⩾1 of random variables with log-Laplace transform ψ N satisfies a large deviation principle with rate function I that may or may not be convex. Informally, this means that (2.59) holds. In Example 2.18 we would need to have I(x) = 0 for x ∈ {-1,+1} and I(x) = +∞ otherwise. In general, when (2.59) holds, we should expect that lim

N→+∞ 1 N ψ N (λ N) = I * (λ ), (2.80)
and Theorem 2.16 states that if ψ = I * is C 1 , then I * * is the correct rate function.

Using the Fenchel-Moreau theorem and Exercise 2.10, one can show that in general, the convex bi-dual I * * is the convex envelope, or largest convex minorant, of the function I. The assumption in Theorem 2.16 that the function ψ is continuously differentiable can be seen to be equivalent to the statement that the convex dual ψ * has no flat piece, a flat piece being an open interval over which ψ * is affine (see also Exercise 2.23 for a related statement). This means that whenever (2.59) holds, the assumption that ψ is continuously differentiable implies that the convex envelope of I has no flat piece. This can only occur if I was convex in the first place, and then indeed I * * = I by the Fenchel-Moreau theorem.

For a more general and in-depth discussion of large deviation principles, we refer the interested reader to [START_REF] Dembo | Large deviations techniques and applications[END_REF][START_REF] Frank Den Hollander | Large deviations[END_REF][START_REF] Srinivasa Varadhan | Large deviations[END_REF]

ψ * (x) = xlog x p + (1 -x)log 1 -x 1 -p . (2.81)

Analyzing the Curie-Weiss model

The large deviation principle in Theorem 2.16 allows us to compute the limit free energy in the generalized Curie-Weiss model introduced in Section 1.3. We recall that this model is defined in terms of a probability measure P N on R N with the property that σ ⩽ √ N for P N -a.e. sample σ , as well as a smooth function ξ ∈ C ∞ (R;R) on the real line. For every σ ∈ R N , we denote by

S N (σ ) ∶= 1 N N i=1 σ i (2.82)
its sample average, or magnetization. For every σ in the support of P N , we have that

S N (σ ) 2 = 1 N N i=1 σ i 2 ⩽ 1 N N i=1 σ 2 i = σ 2 N ⩽ 1. (2.83)
This means that the magnetization lies in the interval [-1,1],

S N (σ ) ∈ [-1,1]. (2.84)
The free energy in the generalized Curie-Weiss model is given by

F N (t,h) ∶= 1 N log ˆRN expN tξ (S N (σ )) + hS N (σ ) dP N (σ ). (2.85)
To see how a large deviation principle for the sequence (S N ) N⩾1 can shed light on the asymptotic behaviour of the free energy (2.85), we denote by ψ N the log-Laplace transform of S N and suppose that the limit

ψ(h) ∶= lim N→+∞ 1 N ψ N (hN) = lim N→+∞ F N (0,h) (2.86)
exists and defines a continuously differentiable function ψ ∈ C 1 (R;R). The general large deviation principle in Theorem 2.16 then implies that the sequence (S N ) N⩾1 satisfies a large deviation principle with rate function ψ * . Heuristically, this means that for every m ∈ [-1,1],

P{S N ≃ m} ≃ exp(-Nψ * (m)).
(2.87)

If we fix an integer K ⩾ 1 and denote by -1 = m 0 < m 1 < ... < m K = 1 a partition of the interval [-1,1] into sub-intervals of width 2 K, then we can localize the free energy according to the approximate magnetization of its spin configurations,

F N (t,h) ≃ 1 N log K k=0 ˆRN 1 {S N (σ )∈[m k ,m k+1 )} expN tξ (m k ) + hm k dP N (σ ). (2.88)
Here m K+1 can be any value larger than one, say m K+1 = +∞. The large deviation principle (2.87) implies that for K large enough and 1

⩽ k ⩽ K, ˆRN 1 {S N (σ )∈[m k ,m k+1 )} dP N (σ ) ≃ exp(-Nψ * (m k )). (2.89)
It follows by Exercise 1.1 that for large enough values of K,

F N (t,h) ≃ max 0⩽k⩽K ξ (m k ) + hm k -ψ * (m k ) . (2.90)
We should therefore be able to leverage a large deviation principle for the magnetization (2.82) to show that the limit free energy (2.85) in the generalized Curie-Weiss model is given by

lim N→+∞ F N (t,h) = sup m∈[-1,1] ξ (m) + hm -ψ * (m) . (2.91)
By choosing ξ (x) = x 2 and P N to be uniform on {±1} N ∶= {-1,+1} N , we will be able to deduce a formula for the limit of the free energy (1.35) exists. If ψ ∈ C 1 (R;R), then the limit free energy f ∶ R ⩾0 × R → R in the generalized Curie-Weiss model is given by

f (t,h) ∶= lim N→+∞ F N (t,h) = sup m∈[-1,1] tξ (m) + hm -ψ * (m) . (2.93)
Proof. For each integer N ⩾ 1, we denote by ψ N the log-Laplace transform of the magnetization S N defined in (2.82). The function ψ ∈ C 1 (R;R) can be written as

ψ(h) = lim N→+∞ F N (0,h) = lim N→+∞ 1 N ψ N (hN), (2.94)
so the general large deviation principle in Theorem 2.16 implies that the sequence (S N ) N⩾1 satisfies a large deviation principle with rate function ψ * . This means that for every Borel set A ⊆ R, we have limsup

N→+∞ 1 N logP N {S N ∈ A} ⩽ inf x∈A ψ * (x) (2.95)
and liminf

N→+∞ 1 N logP N {S N ∈ A} ⩾ -inf int(A) ψ * (x). (2.96)
The rest of the proof proceeds in two steps. First we establish the upper bound in (2.93), and then we obtain the matching lower bound.

Step 1: the upper bound. We fix an integer K ⩾ 1, and for each k ∈ {-K,...,K -1}, we introduce the set

A k ∶= k K , k + 1 K .
The bound (2.84) implies that

S N (σ ) ∈ [-1,1] = K-1 ⋃ k=-K A k
for every σ in the support of P N . It follows by monotonicity of the logarithm that

F N (t,h) ⩽ 1 N log K-1 k=-K ˆRN 1 {S N ∈A k } expN(tξ (S N ) + hS N )dP N ⩽ 1 N log K-1 k=-K max m∈A k expN tξ (m) + hm P N {S N ∈ A k } ⩽ log(2K) N + max -K⩽k<K max m∈A k (tξ (m) + hm) + 1 N logP N {S N ∈ A k } .
Using the large deviation upper bound (2.95) to let N tend to infinity reveals that

limsup N→+∞ F N (t,h) ⩽ max -K⩽k<K max m∈A k (tξ (m) + hm) -inf m∈A k ψ * (m) . (2.97)
We next observe that since ξ ∈ C ∞ (R;R), there exists a constant C < +∞ such that for every m,m

′ ∈ [-1,1], ξ (m) -ξ (m ′ ) ⩽ C m -m ′ .
For each k ∈ {-K,...,K -1}, and m ∈ A k , we therefore have that max

m ′ ∈A k tξ (m ′ ) + hm ′ -ψ * (m) ⩽ tξ (m) + hm -ψ * (m) + C + h K .
Taking the supremum over m ∈ A k and over k ∈ {-K,...,K -1}, and then combining this with (2.97), we obtain that

limsup N→+∞ F N (t,h) ⩽ sup m∈[-1,1] tξ (m) + hm -ψ * (m) + C + h K .
Letting K tend to infinity then yields the upper bound

limsup N→+∞ F N (t,h) ⩽ sup m∈[-1,1] tξ (m) + hm -ψ * (m) . (2.98)
Step 2: the lower bound. We fix ε > 0, m 0 ∈ [-1,1], δ > 0, and consider the open set U δ ∶= (m 0 -δ ,m 0 + δ ). The monotonicity of the logarithm implies that

F N (t,h) ⩾ 1 N log ˆRN 1 {S N ∈U δ } expN tξ (S N ) + hS N dP N ⩾ inf m∈U δ tξ (m) + hm + 1 N logP N {S N ∈ U δ }.
Using the large deviation lower bound (2.96) to let N tend to infinity reveals that

liminf N→+∞ F N (t,h) ⩾ inf m∈U δ tξ (m) + hm -ψ * (m 0 ).
Letting δ tend to zero and leveraging the continuity of the map m ↦ tξ (m) + hm gives the lower bound

liminf N→+∞ F N (t,h) ⩾ tξ (m 0 ) + hm 0 -ψ * (m 0 ).
Since m 0 ∈ [-1,1] was arbitrary, this completes the proof. ∎

Corollary 2.20. The limit of the free energy (1.35) in the (standard) Curie-Weiss model is given by

f (t,h) ∶= lim N→+∞ F N (t,h) = sup m∈[-1,1] tm 2 + hm -ψ * (m) (2.99)
for the function ψ * ∶ R → R defined by

ψ * (m) ∶= 1 + m 2 log(1 + m) + 1 -m 2 log(1 -m), for m ∈ [-1,1], (2.100)
with the understanding that 0log(0) = 0, and

ψ * = +∞ on R ∖ [-1,1].
Proof. The Curie-Weiss model corresponds to the generalized Curie-Weiss model with ξ (x) = x 2 and P N uniform on the set {±1} N . This means that for every h ∈ R,

F N (0,h) = 1 N log 1 2 N σ ∈{±1} N exp h N i=1 σ i = logcosh(h).
In particular, the limit

ψ(h) = lim N→+∞ F N (0,h) = logcosh(h)
exists and defines a continuously differentiable function ψ ∈ C 1 (R;R). It follows by Theorem 2.19 that the limit free energy is

f (t,h) = sup m∈[-1,1] tm 2 + hm -ψ * (m)
for the convex dual

ψ * (m) = sup h∈R hm -ψ(h) = sup h∈R hm -logcosh(h) .
The symmetry of ψ implies that of ψ * , so it suffices to compute the convex dual for m ∈ [0,1]. The derivative of the function g(h

) ∶= hm -ψ(h) is g ′ (h) = m -tanh(h).
Since tanh(h) ∈ (-1,1), if m = 1 the function g is increasing and

ψ * (1) = lim h→+∞ (h -logcosh(h)) = log(2).
On the other hand, if m ∈ [0,1), then g admits a unique critical point h * which satisfies tanh(h * ) = m.

Since g is concave, we must have ψ * (m) = g(h * ). A direct computation shows that

e 2h * = 1 -m 1 + m from which it is readily verified that ψ * (m) ∶= 1 + m 2 log(1 + m) + 1 -m 2 log(1 -m). (2.101)
(The attentive reader will have observed that this must coincide with the rate function (2.51), up to a change of variables.) This completes the proof. ∎

The envelope theorem and the Curie-Weiss magnetization

The original motivation for introducing the free energy (1.35) was to understand the asymptotic behaviour of the mean magnetization

m N (t,h) ∶= ⟨ 1 N N i=1 σ i ⟩ (2.102)
in the Curie-Weiss model, and to show that for large enough values of the inverse temperature parameter t, it exhibits ferromagnetic behaviour. Recall that ⟨⋅⟩ denotes the average under the Gibbs measure (1.31), see (1.33). A direct computation reveals that the mean magnetization is the derivative of the free energy,

m N (t,h) = ∂ h F N (t,h). (2.103)
The sequence (F N ) N⩾1 is convex in h by Exercise 2.6 and, by Corollary 2.20, it converges to the limit free energy

f (t,h) ∶= sup m∈[-1,1] tm 2 + hm -ψ * (m) , (2.104)
where ψ * ∶ R → R is given by

ψ * (m) ∶= 1 + m 2 log(1 + m) + 1 -m 2 log(1 -m). (2.105)
We would like to pass to the limit in (2.103), and to study the properties of ∂ h f . This should hopefully allow us to identify the phase transition of the model from paramagnetic to ferromagnetic behaviour as we increase t, similarly to (v) of Exercise 1.3.

The problem of differentiating a function which, like the function f in (2.104), is defined as the envelope, or supremum, of a family of functions, is sufficiently important that we will discuss it in general. Given a continuous function

g ∈ C(R d × R d ;R) that
O x ∶= y ∈ R d f (x) = g(x,y) . (2.107)
This holds in particular whenever O x is a singleton. In this case, the derivative of f can be computed by simply bringing the derivative into the supremum.

Theorem 2.21 (Envelope). Let g ∈ C(R d × R d ;R) be a continuous function that is continuously differentiable in its first variable, let f ∶ R d → R be its envelope (2.106), and let O x be as in (2.107). We fix x ∈ R d , and assume that there exists a compact set K which contains O x ′ ≠ ∅ for every x ′ in a small enough neighbourhood of x. As x ′ ∈ R d tends to x, we have the expansion

f (x ′ ) = f (x) + sup y∈Ox (x ′ -x) ⋅ ∇ x g(x,y) + o( x -x ′ ). (2.108)
In particular, the envelope function

f ∶ R d → R is differentiable at x ∈ R d if and only if the set D x ∶= ∇ x g(x,y) y ∈ O x (2.109)
is a singleton. In this case, for any y ∈ O x , we have ∇ f (x) = ∇ x g(x,y).

Proof. We decompose the proof into three steps.

Step 1: lower bound for (2.108). Since g is continuously differentiable, we have for every y ∈ O x that, as x ′ tends to x, g(x ′ ,y) = g(x,y) + (x ′x) ⋅ ∇ x g(x,y) + o( xx ′ ).

(2.110) Moreover, since O x is contained in the compact set K, the expansion (2.110) holds uniformly over y ∈ O x . By the definition of f as a supremum and the fact that y ∈ O x , we obtain that, as x ′ tends to x,

f (x ′ ) ⩾ f (x) + (x ′ -x) ⋅ ∇ x g(x,y) + o( x -x ′ ). (2.111)
Taking the supremum over y ∈ O x yields the lower bound for (2.108).

Step 2: upper bound for (2.108). We now turn to the converse bound. Let (x n ) n⩾1 ⊆ R d ∖ {x} be a sequence converging to x. Our goal is to show that and letting n tend to infinity, we obtain that g(x,y ′ ) ⩽ g(x,y) -in other words, we have that y ∈ O x . By the definition of f as a supremum and the fundamental theorem of calculus, we have

limsup n→+∞ 1 x n -x f (x n ) -f (x) -sup y∈Ox {(x n -x) ⋅ ∇ x g(x,y)} ⩽ 0. ( 2
g(x n ,y n ) -f (x) ⩽ g(x n ,y n ) -g(x,y n ) = ˆ1 0 (x n -x) ⋅ ∇ x g(tx n + (1 -t)x,y n )dt.
Since y n ∈ O xn and y ∈ O x , we obtain that

f (x n ) ⩽ f (x) + sup y∈Ox (x n -x) ⋅ ∇ x g(x,y) + x n -x ˆ1 0 ∇ x g(tx n + (1 -t)x,y n )dt -∇ x g(x,y) . (2.113)
Since ∇ x g is continuous, we have

lim n→+∞ ˆ1 0 ∇ x g(tx n + (1 -t)x,y n )dt = ∇ x g(x,y),
so letting n tend to infinity in (2.113) yields (2.112).

Step 3: differentiability condition. By (2.108), it is clear that if the set D x in (2.109) is a singleton, then f is differentiable at x, and ∇ f (x) = ∇ x g(x,y) for every y ∈ O x . Conversely, suppose that the function f is differentiable at x. Using (2.111) and the differentiability assumption, we obtain that, for every y ∈ O x and as x ′ tends to x,

(x ′ -x) ⋅ ∇ f (x) -∇ x g(x,y) ⩾ o( x ′ -x ).
This means that ∇ x g(x,y) = ∇ f (x), so D x is a singleton as announced. ∎

To apply this result to determine the limit of the mean magnetization (2.102) as N tends to infinity, given t ∈ R ⩾0 and h ∈ R, we need to study the set of maximizers of the function g t,h ∶ [-1,1] → R defined by g t,h (m) ∶= tm 2 + hm -ψ * (m).

(2.114)

We will focus in particular on the case when h is close to zero. Figure 2.1 displays the graph of this function for (t,h) = (0.1,0) and (t,h) = (0.6,0), and it suggests that there should be a transition at some critical inverse temperature t c ∈ (0.1,0.6), where the number of maximizers jumps from one to two.

m 0.1m 2 -ψ * (m) m 0.6m 2 -ψ * (m)
∂ m g t,h (m) = 2tm + h + 1 2 log 1 -m 1 + m .
It follows that the critical points m of g t,h are the solutions to the critical point equation

m = e 2(h+2tm) -1 e 2(h+2tm) + 1 = tanh(h + 2tm).
Differentiating one more time, wee see that, for every m ∈ [-1,1],

∂ 2 m g t,h (m) = 2t - 1 1 -m 2 .
We now distinguish two cases. First, if t ⩽ We also assume that the function f is strictly convex, that is, for every x ≠ y ∈ R d and α ∈ (0,1), we have

f (αx + (1 -α)y) < α f (x) + (1 -α) f (y). (2.120)
Show that the effective domain of f * is R d and that f * is continuously differentiable.

Exercise 2.24. Denote by f ∶ R ⩾0 × R → R the limit free energy (2.104) in the Curie-Weiss model and by t c ∶= 1 2 its critical inverse temperature parameter. We recall from Proposition 2.22 that if t ⩽ t c , then the function (2.114) for h = 0 has a unique maximizer m 0 (t) = 0, while it has two maximizers ±m 0 (t) if t > t c .

(i) Identify ∂ t f (t,0), and show that it is a continuous function of t.

(ii) Identify an exponent δ ⩾ 0 such that, as h tends to zero and in a sense to be specified,

f (t c ,h) ≃ h 1+1 δ .
(iii) Identify an exponent β ⩾ 0 such that, as t tends to t c and in a sense to be specified, m 0 (t) ≃ (t -t c ) β + , where x + ∶= max(0,x) denotes the positive part.

The notation used here for the exponents δ and β follows the convention of the physics literature. In dimensions d ⩾ 4, the Ising model on Z d has the same critical exponents as the Curie-Weiss model, with a logarithmic correction when d = 4 [START_REF] Michael Aizenman | Geometric analysis of ϕ 4 fields and Ising models[END_REF][START_REF] Aizenman | On the critical behavior of the magnetization in high-dimensional Ising models[END_REF]. For the two-dimensional Ising model, we have δ = 15 and β = 1 8 [START_REF] Kaufman | Crystal statistics. II. Partition function evaluated by spinor analysis[END_REF][START_REF] Kaufman | Crystal statistics. III. Short-range order in a binary Ising lattice[END_REF][START_REF] Onsager | Crystal statistics. I. A two-dimensional model with an order-disorder transition[END_REF][START_REF] Szegő | On certain Hermitian forms associated with the Fourier series of a positive function[END_REF][START_REF] Ning | The spontaneous magnetization of a two-dimensional Ising model[END_REF]. There is no known or conjectured closed-form description of these exponents in three dimensions; sophisticated numerical estimates are derived in [START_REF] El-Showk | Solving the 3d Ising model with the conformal bootstrap[END_REF][START_REF] El-Showk | Solving the 3d Ising model with the conformal bootstrap II. c-minimization and precise critical exponents[END_REF][START_REF] Kos | Precision islands in the Ising and O(N) models[END_REF].

Chapter 3 Hamilton-Jacobi equations

In Chapter 2, we computed the limit of the free energy in the Curie-Weiss model via large deviation principles. Unfortunately, this approach seems inapplicable for the models coming from statistical inference or spin glass theory which we are ultimately interested in. In this chapter, we introduce the Hamilton-Jacobi approach, and we use it to compute the limit of the free energy in the Curie-Weiss model and its generalization. The approach centres around the observation that the finite-volume free energy satisfies an approximate Hamilton-Jacobi equation, up to a small error term. Section 3.1 presents this computation in the context of the Curie-Weiss and generalized Curie-Weiss models, and justifies the need to introduce a precise notion of solution to a Hamilton-Jacobi equation. The notion of a viscosity solution is defined in Section 3.2, and we observe that any subsequential limit of the free energy in the Curie-Weiss model must be a viscosity solution. We then show in Section 3.3 that there can be at most one viscosity solution to a Hamilton-Jacobi equation with a prescribed initial condition. This already guarantees that we have identified the limit free energy of the Curie-Weiss model uniquely. In Section 3.4, in order to recover the explicit formula for the limit free energy found in the previous chapter, we develop variational representations of viscosity solutions whenever the initial condition or the non-linearity in the equation is convex (or concave). In Section 3.5, we explore whether any aspect of this variational structure is preserved in the absence of convexity or concavity. Finally, in Section 3.6, we focus on a new difficulty that emerges when trying to analyze the generalized Curie-Weiss model. To overcome this, we present a convenient criterion for deciding when a given convex function is the viscosity solution to a Hamilton-Jacobi equation.

A Hamilton-Jacobi approach to Curie-Weiss

Recall from Section 1.3 that, given a smooth function ξ ∈ C ∞ (R;R) and a probability measure P N on R N with the property that σ ⩽ √ N for P N -a.e. σ , the Hamiltonian of the generalized Curie-Weiss model at the point (t,h,σ ) ∈ R ⩾0 × R × R N is defined by

H N (t,h,σ ) ∶= Ntξ 1 N N i=1 σ i + h N i=1 σ i . (3.1)
The free energy in the generalized Curie-Weiss model is given by

F N (t,h) ∶= 1 N log ˆRN expH N (t,h,σ )dP N (σ ), (3.2)
and the average of any bounded and measurable function f ∶ R N → R with respect to the Gibbs measure (1.31) is denoted by

⟨ f (σ )⟩ ∶= ´RN f (σ )expH N (t,h,σ )dP N (σ ) ´RN expH N (t,h,σ )dP N (σ ) . (3.3)
We recall that, although this is kept implicit in the notation, the bracket ⟨⋅⟩ depends on the choice of parameters t and h. The (standard) Curie-Weiss model corresponds to the choice ξ (x) = x 2 and P N uniform on the hypercube {±1} N ∶= {-1,+1} N . To illustrate the Hamilton-Jacobi approach in the simplest possible setting, let us first focus on this situation.

In the context of the Curie-Weiss model, a direct computation shows that

∂ t F N (t,h) = ⟨ 1 N N i=1 σ i 2 ⟩ and ∂ h F N (t,h) = ⟨ 1 N N i=1 σ i ⟩. (3.4)
It follows that

∂ t F N (t,h) -∂ h F N (t,h) 2 = ⟨ 1 N N i=1 σ i 2 ⟩ -⟨ 1 N N i=1 σ i ⟩ 2 = Var 1 N N i=1 σ i . (3.5)
Another direct computation reveals that

∂ 2 h F N (t,h) = ⟨ 1 N N i=1 σ i 2 ⟩ - 1 N ⟨ N i=1 σ i ⟩ 2 = N Var 1 N N i=1 σ i . (3.6)
Together with (3.5), this implies that the finite-volume free energy (3.2) in the Curie-Weiss model satisfies

∂ t F N (t,h) -∂ h F N (t,h) 2 = 1 N ∂ 2 h F N (t,h). (3.7)
Since P N is uniform on the hypercube {±1} N , the free energy at "time" 0 is independent of N. Indeed,

F N (0,h) = 1 N log 1 2 N σ ∈{±1} N exp h N i=1 σ i = 1 N log 1 2 e h + e -h N = logcosh(h).
We introduce the notation

ψ(h) ∶= logcosh(h) = F 1 (0,h) = F N (0,h). (3.8)
These observations suggest that the limit free energy f ∶ R ⩾0 × R → R in the Curie-Weiss model should satisfy the Hamilton-Jacobi equation

∂ t f -(∂ h f ) 2 = 0 on R >0 × R, (3.9)
subject to the initial condition f (0,⋅) = ψ. In Corollary 2.20, we showed that

f (t,h) = sup m∈[-1,1] tm 2 + hm -ψ * (m) . (3.10)
By the envelope theorem (Theorem 2.21), one can show that at every point (t,h) of differentiability of f , we have

∂ t f (t,h) = m 2 0 (t,h) and ∂ h f (t,h) = m 0 (t,h), (3.11)
for a maximizer m 0 (t,h) of the right side of (3.10). In particular, the limit free energy f does indeed satisfy the Hamilton-Jacobi equation (3.9) at all its points of differentiability. Together with Rademacher's theorem (Theorem 2.10), this implies that f satisfies the equation (3.9) almost everywhere. Notice that f is Lipschitz continuous because each F N is, by (3.4). A natural question arises at this point. Could we identify the limit free energy f as the unique Lipschitz continuous function with f (0,⋅) = ψ which satisfies the equation (3.9) almost everywhere?

Unfortunately, the answer is no. In fact, the following construction shows that this Hamilton-Jacobi equation admits infinitely many almost-everywhere solutions which are Lipschitz continuous.

Example 3.1. For simplicity, we will construct infinitely many Lipschitz functions that satisfy the Hamilton-Jacobi equation (3.9) almost everywhere and are constant equal to zero at the initial time. Similar constructions can be performed for more general initial conditions. Temporarily disregarding the question of the initial condition, we start by noticing that the functions (t,h) ↦ 0, (t,h) ↦ t + h and (t,h) ↦ th are all solutions to (3.9). It thus follows that the Lipschitz function

f (t,h) ∶= ⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ t + h if h ∈ [-t,0] t -h if h ∈ [0,t] 0 otherwise (3.12)
displayed in Figure 3.1 is an almost everywhere solution to (3.9), as it is obtained by "gluing together" these different solutions, and we can disregard the measure-zero set of points where they are joined together. We also have f (0,⋅) = 0.

-t t t h f (h) t -h -t + h Figure 3.1 Graph of the function h ↦ f (t,h), for a fixed value of t > 0.
Evidently, the null solution also satisfies these properties, so we already have two almost-everywhere solutions with the same initial condition. Moreover, any translation in space of f also satisfies this property; and we can also delay the "emergence of the corner" to some arbitrary time. So in fact there are uncountably many Lipschitz functions that solve the equation (3.9) almost everywhere and vanish at the initial time.

We could try to impose uniqueness of solutions by strengthening the regularity assumptions; for instance, we could impose that a solution f to (3.9) be in C 1 (R ⩾0 × R;R) and satisfy the equation everywhere. The problem with this idea is that in this case, the set of solutions can be empty, and indeed, we have already seen in (2.117) and (2.118) that our candidate solution f in (3.10) is not everywhere differentiable. We therefore need to identify a notion of solution that is more stringent than the "almost-everywhere solutions" explored above, but less stringent than asking the solution to be in C 1 (R ⩾0 × R;R). In a nutshell, we will also require the function f to satisfy a certain form of the maximum principle. We observe that whenever two functions f and g satisfy the Hamilton-Jacobi equation (3.10) with "viscosity" parameter ε > 0,

∂ t f -(∂ h f ) 2 = ε∆ f on R >0 × R,
if their initial conditions are ordered, say f (0,⋅) ⩽ g(0,⋅), then this ordering is preserved at all later times t ⩾ 0 as well, f (t,⋅) ⩽ g(t,⋅). While we will obtain this monotonicity property as a consequence of the definition of solution explained below, we point out that one can essentially also go the other way around [START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF].

Before introducing this appropriate notion of solution, let us see what the computations leading to (3.7) would look like in the context of the generalized Curie-Weiss model. In this setting, one can readily see that

∂ t F N (t,h) = ⟨ξ 1 N N i=1 σ i ⟩ and ∂ h F N (t,h) = ⟨ 1 N N i=1 σ i ⟩, (3.13)
and that (3.6) still holds. It follows that

∂ t F N (t,h) -ξ ∂ h F N (t,h) = ⟨ξ 1 N N i=1 σ i ⟩ -ξ ⟨ 1 N N i=1 σ i ⟩ . (3.14)
If the variance of the magnetization is small, we therefore expect ∂ t F Nξ (∂ h F N ) to be close to zero. Recall that for every σ in the support of P N , the magnetization is bounded by one,

1 N N i=1 σ i ⩽ 1 N N i=1 σ 2 i 1 2 = σ √ N ⩽ 1, (3.15)
and so this bound also holds with probability one under the Gibbs measure. Moreover, as ξ is smooth, there exists a constant C < +∞ such that for every x,y ∈ [-1,1],

ξ (y) -ξ (x) -(y -x)ξ ′ (x) ⩽ C(x -y) 2 . This implies that ξ 1 N N i=1 σ i -ξ ⟨ 1 N N i=1 σ i ⟩ -ξ ′ ⟨ 1 N N i=1 σ i ⟩ 1 N N i=1 σ i -⟨ 1 N N i=1 σ i ⟩ ⩽ C 1 N N i=1 σ i -⟨ 1 N N i=1 σ i ⟩ 2 . (3.16)
Averaging with respect to the Gibbs measure (3.3), substituting the resulting bound into (3.14), and recalling (3.6) yields that

∂ t F N (t,h) -ξ ∂ h F N (t,h) ⩽ CVar 1 N N i=1 σ i = C N ∂ 2 h F N (t,h). (3.17)
If we assume that the limit

ψ(h) ∶= lim N→+∞ F N (0,h) (3.18)
of the initial conditions exists, which is for instance the case when P N is a product measure, then we expect the limit free energy f ∶ R ⩾0 × R → R in the generalized Curie-Weiss model to satisfy the Hamilton-Jacobi equation

∂ t f -ξ (∂ h f ) = 0 on R >0 × R (3.19)
subject to the initial condition f (0,⋅) = ψ. This will however be harder to prove rigorously than for the Curie-Weiss model due to the presence of the absolute value in the inequality (3.17). The problem is that functions that are only known to satisfy the inequality (3.17) do not necessarily satisfy the maximum principle, while the notion of viscosity solutions takes this property as its foundation. We will explain how to circumvent this difficulty by leveraging the convexity of the solutions.

Viscosity solutions to Hamilton-Jacobi equations

In this section, given a locally Lipschitz continuous non-linearity

H ∶ R d → R and a Lipschitz continuous initial condition ψ ∶ R d → R, we define a notion of solution f = f (t,x) ∶ R ⩾0 × R d → R to the Hamilton-Jacobi equation ∂ t f -H(∇ f ) = 0 on R >0 × R d , (3.20)
subject to the initial condition f (0,⋅) = ψ. We understand the gradient to be taken with respect to the x variable, leaving the t variable aside:

∇ f (x) = (∂ x 1 f ,...,∂ x d f ).
The equations arising in the context of the Curie-Weiss and generalized Curie-Weiss models correspond to the one-dimensional case, d = 1 and, respectively, the non-linearities H(p) = p 2 and H(p) = ξ (p).

A natural way to define a solution to the equation (3.20) is to add a small "viscosity" parameter ε > 0, and to consider the second-order parabolic equation

∂ t f ε -H ∇ f ε = ε∆ f ε on R >0 × R d (3.21)
subject to the initial condition f ε (0,⋅) = ψ. That smooth solutions exist for (3.21) can be shown by classical techniques, because the Laplacian term is dominant on very small scales, being of higher order than the non-linear term in the equation. The solution to the Hamilton-Jacobi equation (3.20) can then be defined as the limit of the solutions to (3.21) as the viscosity parameter ε tends to zero. Although we will not pursue this route rigorously here, let us suppose for a moment that for each ε > 0, we have been able to define a smooth solution (3.21), and that we have been able to show that, as ε tends to zero, the sequence ( f ε ) ε>0 converges to some function f ∶ R ⩾0 × R d → R in the topology of local uniform convergence. We would like to say, in some sense to be discovered, that the limit thus obtained is a solution to (3.20). As was seen in the previous section, the main difficulty is that we do not want to impose a solution to (3.19) to be differentiable everywhere; but we do not obtain unique solutions if we simply ignore the points of non-differentiability. In analogy with the notion of weak solutions, we would like to introduce smooth test functions and somehow move the derivatives of f onto the test functions, so that we could introduce some constraint for what a solution is allowed to do at points of non-differentiability. This transfer of the derivatives onto the test functions will not be obtained by some integration by parts here. Rather, we seek a strategy that accords well with the fact that our approximations ( f ε ) ε>0 satisfy the maximum principle. We pick a test function φ ∈ C ∞ (R >0 × R d ;R), and assume that fφ achieves a strict local maximum at the point

f ε ∶ R ⩾0 × R d → R to
(t * ,x * ) ∈ R >0 × R d . If f is smooth at (t * ,x * ), then we clearly have ∂ t φ (t * ,x * ),∇φ (t * ,x * ) = ∂ t f (t * ,x * ),∇ f (t * ,x * ) , (3.22)
so we expect that

∂ t φ (t * ,x * ) -H ∇φ (t * ,x * ) = 0. (3.23)
This identity may no longer be valid when f is not differentiable at (t * ,x * ), but we will now argue that the quantity on the left side of (3.23) must always be non-positive. Using Exercise 3.1, we can find a sequence (t * ε ,x * ε ) ε>0 converging to (t * ,x * ) with the property that, for every ε > 0 sufficiently small, the function f εφ achieves a local maximum at

(t * ε ,x * ε ) ∈ R >0 × R d , and therefore ∂ t ( f ε -φ )(t * ε ,x * ε ) = 0, ∇( f ε -φ )(t * ε ,x * ε ) = 0, and ∆( f ε -φ )(t * ε ,x * ε ) ⩽ 0. (3.24)
It follows by (3.21) that

∂ t φ -H(∇φ ) (t * ε ,x * ε ) = ∂ t f -H(∇ f ) (t * ε ,x * ε ) = ε ∆ f ε (t * ε ,x * ε ) ⩽ ε ∆φ (t * ε ,x * ε ).
Letting ε tend to zero and leveraging the smoothness of φ shows that

∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0. (3.25)
We have thus shown that whenever a smooth function φ is such that fφ has a strict local maximum at (t * ,x * ) ∈ R >0 × R d , the inequality (3.25) holds at (t * ,x * ).

An analogous argument shows that whenever a smooth function φ is such that fφ has a strict local minimum at (t * ,x * ) ∈ R >0 × R d , we have

∂ t φ -H(∇φ ) (t * ,x * ) ⩾ 0. (3.26)
As we will see in the next section, these properties of f we just derived are sufficient to determine it uniquely, once the initial condition f (0,⋅) is fixed. We will therefore take them as the definition of being a solution to the Hamilton-Jacobi equation (3.20). When fφ has a local maximum at (t * ,x * ), we will often say that "φ touches f from above at (t * ,x * )". The reason is that, for our purposes, adding a constant to the test function φ is irrelevant to the discussion, so we may as well assume indeed that ( fφ )(t * ,x * ) = 0. For this reason, we may also call the point (t * ,x * ) the "contact point". We feel that the wording has some intuitive appeal, as we can imagine taking some arbitrary smooth function φ that is way above f , and then progressively "sliding it down" until the graphs of φ and f touch -at the contact point. This is illustrated in Figure 3.2. Notice that not every point can be "touched" in this way. For instance, there is no smooth function that touches the absolute value function from above at the origin.

f (t, x) ϕ+(t, x) ϕ-(t, x) (t * , x * ) R ≥0 × R d R Figure 3.2
The function φ + touches the function f from above at the point (t * ,x * ) while the function φ -touches the function f from below at the point (t * ,x * ). As shown in Exercises 3.2-3.4, in the definition of a viscosity subsolution or viscosity supersolution, replacing "local maximum" by "strict local maximum" or by "global maximum", or replacing the requirement that φ ∈ C ∞ (R >0 ×R d ;R) by the requirement that φ ∈ C 1 (R >0 × R d ;R) lead to equivalent definitions. In Exercise 3.5, we verify that a C 1 function that satisfies the equation (3.20) everywhere is indeed a viscosity solution, so the notion of viscosity solution is more "permissive" than prescribing the function to be C 1 (R ⩾0 ×R d ;R) and to solve the equation everywhere. One can also show that a viscosity solution must satisfy the equation (3.20) at every point of differentiability, see for instance Theorem 10.1.2.1 in [START_REF] Evans | Partial differential equations[END_REF]. By the Rademacher theorem (Theorem 2.10), a Lipschitz viscosity solution must therefore satisfy the equation (3.20) almost everywhere. In other words, the notion of viscosity solution is indeed more stringent than that of "almost-everywhere solution" explored in Section 3.1. This is corroborated by the fact that the solution constructed in Example 3.1 is not a viscosity solution to the Hamilton-Jacobi equation (3.9). Example 3.3. Recall the definition of the function f in Example 3.1, fix t 0 > 0, and consider the smooth function φ (t,x) = t. Observe that fφ ⩽ 0, and f (t 0 ,0)φ (t 0 ,0) = 0, so f -φ has a local maximum at (t 0 ,0). If f were a viscosity subsolution to the Hamilton-Jacobi equation (3.9), we would have

Definition 3.2. A continuous function

u ∶ R ⩾0 × R d → R is a viscosity subsolution to the Hamilton-Jacobi equation (3.20) if, for every (t * ,x * ) ∈ R >0 × R d and φ ∈ C ∞ (R >0 × R d ;R) with the property that u -φ has a local maximum at the point (t * ,x * ) ∈ R >0 × R d , we have ∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0. (3.27) A continuous function v ∶ R ⩾0 × R d → R
(∂ t φ -(∂ x φ ) 2 )(t 0 ,0) ⩽ 0. However, we see that (∂ t φ -(∂ x φ ) 2 )(t 0 ,0) = 1.
This shows that f is not a viscosity solution to (3.9).

We now verify that any subsequential limit of the sequence (F N ) N⩾1 of free energies in the Curie-Weiss model is a viscosity solution to the Hamilton-Jacobi equation (3.9). Notice first that the sequence of free energies (F N ) N⩾1 is precompact in the topology of local uniform convergence, by the Arzelà-Ascoli theorem. Indeed, the function F N (0,⋅) does not depend on N by (3.8), and ∂ h F N and ∂ t F N take values in the compact interval [-1,1] by (3.4). Proposition 3.4. For each N ⩾ 1, let F N denote the free energy (1.35) of the (standard) Curie-Weiss model, and let f ∶ R ⩾0 × R → R be a subsequential limit of (F N ) N⩾1 in the topology of local uniform convergence. The function f is a viscosity solution to the Hamilton-Jacobi equation (3.9) with initial condition f (0,h) = ψ(h) ∶= logcosh(h).

Proof. This is essentially the argument that led us to (3.25). Recall from (3.7) that

∂ t F N -(∂ h F N ) 2 = 1 N ∂ 2 h F N on R ⩾0 × R. (3.29)
Fix a smooth function φ ∈ C ∞ (R >0 × R;R) with the property that fφ has a strict local maximum at (t * ,h * ) ∈ R >0 × R. Since (F N ) N⩾1 converges locally uniformly to f , using Exercise 3.1 it is possible to find a sequence (t N ,h N ) N⩾1 ⊆ R >0 × R converging to (t * ,h * ) with the property that, for every N large enough, F Nφ has a local maximum at (t N ,h N ). Since t * > 0, we have that t N > 0 for every N sufficiently large, so the derivatives in t and h of F Nφ must vanish at (t N ,h N ), and we must also have that

∂ 2 h (F N -φ )(t N ,h N ) ⩽ 0.
Using also the identity (3.29), we obtain

∂ t φ -(∂ h φ ) 2 (t N ,h N ) = 1 N ∂ 2 h F N (t N ,h N ) ⩽ 1 N ∂ 2 h φ (t N ,h N ).
Since φ is smooth, letting N tend to infinity shows that f is a viscosity subsolution to the Hamilton-Jacobi equation (3.9). A similar argument shows that it is also a viscosity supersolution to this equation. By (3.8) (3.20) everywhere, then f is a viscosity solution.

"φ ∈ C ∞ (R >0 × R d ;R)" by "φ ∈ C 1 (R >0 × R d ;R)" yields an equivalent definition. Exercise 3.5. Show that if f ∈ C 1 (R ⩾0 × R d ;R) satisfies

Uniqueness of solutions via the comparison principle

The result in Proposition 3.4 ensures that any subsequential limit of the free energy in the Curie-Weiss model is a viscosity solution to (3.9). In particular, this shows that a viscosity solution to this equation with initial condition ψ exists. To show that the finite-volume free energies indeed converge to the solution to (3.9), we need to assert the uniqueness of solutions to Hamilton-Jacobi equations with a prescribed initial condition. We will in fact show a more general result known as the comparison principle. The comparison principle formalizes the idea that the Hamilton-Jacobi equation (3.20) should preserve the ordering of initial conditions. In fact, since the equation is invariant under the addition of a constant to the solution, we will show that if u is a subsolution and v is a supersolution to (3.20), then the function uv achieves its supremum at time zero. In particular, if u(0,⋅) ⩽ v(0,⋅), then this ordering is preserved by the evolution.

If we assume for a moment that u and v are smooth functions, then we can argue heuristically to get a sense of why this result might be true. For smooth functions u and v, saying that u and v are a subsolution and a supersolution to the Hamilton-Jacobi equation (3.20) amounts to saying that ∂ t u -H(∇u) ⩽ 0 and ∂ t v -H(∇v) ⩾ 0.

(3.30)

Arguing by contradiction, suppose that sup

R ⩾0 ×R d (u -v) > sup {0}×R d (u -v). (3.31)
Up to subtracting a small increasing function of t to u, such as εt for some sufficiently small ε > 0, we can make sure that (3.31) is still valid, and also that we have improved (3.30) into

∂ t u -H(∇u) < 0 and ∂ t v -H(∇v) ⩾ 0. (3.32)
Now, if we assume that the supremum on the left side of (3.31) is achieved at some point (t * ,x * ), then we must have t * > 0, so the first order derivatives of u and v must coincide at this point. But this contradicts (3.32), so (3.31) cannot be true. Of course, there is much left to be desired with this argument, since we assumed that u and v are smooth, and also that the supremum on the left side of (3.31) is achieved at some point. We start by discussing how one can go around the second problem.

To simplify the discussion, we will temporarily assume that the space R d is replaced by the unit torus T d = R d Z d , so that the variable x lives in a compact space without boundary. We take u,v ∶ R ⩾0 × T d → R to be a viscosity subsolution and supersolution to (3.20) respectively, and we assume that they are smooth. Our starting point is again to argue by contradiction, assuming that there exists some time T > 0 with sup

[0,T ]×T d (u -v) > sup {0}×T d (u -v). (3.33)
Denoting χ(t) ∶= ε T -t , we can select ε > 0 sufficiently small that sup

[0,T ]×T d (u -v -χ) > sup {0}×T d (u -v -χ). (3.34)
Since uv is uniformly bounded over [0,T ] × T d , it is clear that approximate optimizers of the left side of (3.34) will remain away from the final time T . Using also that [0,T ] × T d is compact, we can construct an optimizer (t * ,x * ) ∈ [0,T ] × T d for this supremum, and it is clear that we must in fact have t * ∈ (0,T ). Once this is verified, we can use the differential condition at the maximum to ascertain that

∂ t (u -v)(t * ,x * ) - ε (T -t * ) 2 = ∂ t (u -v -χ)(t * ,x * ) = 0 (3.35) and ∇(u -v)(t * ,x * ) = 0. (3.36)
Since v is a smooth supersolution to the equation, we can combine this with the second inequality in (3.30) to obtain that

∂ t u -H(∇u) (t * ,x * ) - ε (T -t * ) 2 = ∂ t v -H(∇v) (t * ,x * ) ⩾ 0. (3.37)
This contradicts the assumption (3.30) that u is a smooth subsolution to the Hamilton-Jacobi equation (3.20). The role of the perturbation function χ is therefore two-fold.

First, it ensures that the optimum t * is detached from the right endpoint of the interval [0,T ], that is, t * < T . The function χ also allows us to strengthen the inequalities (3.30) into those in (3.32), if we understand that the function u is redefined to be uχ.

In order to establish the comparison principle rigorously, we need to resolve two main problems. The first, and most fundamental, is of course that u and v cannot be assumed to be differentiable everywhere. The second is that the spatial variable takes values in R d rather than the torus. To tackle the first of these problems, we will "double the variables", and rather optimize a function that involves u(t,x)v(t ′ ,x ′ ) plus a smooth penalty term that strongly encourages (t,x) and (t ′ ,x ′ ) to stay close together. This will naturally provide us with smooth test functions that touch u and v from above and below respectively. The second problem, that the variable x lives in an unbounded space, will be tackled by introducing another "cutoff" function, similar to the function χ used above, but in the space variable. This additional spatial cutoff will in fact allow us to prove a somewhat stronger result than the comparison principle. To state this concisely, we denote the Lipschitz semi-norm of a function h ∶ R d → R by

h Lip = sup x≠x ′ h(x ′ ) -h(x) x ′ -x , (3.38)
and introduce the space of uniformly Lipschitz functions in space For every R,M ∈ R with M > 2L, the mapping

L = u ∶ R ⩾0 × R d → R
(t,x) ↦ u(t,x) -v(t,x) -M x +V t -R + (3.41)
achieves its supremum at a point in {0} × R d .

In the statement above, we use the notation r + ∶= max(0,r) to denote the positive part of a real number r ∈ R. Since the proof of Theorem 3.5 is a bit long, we suggest that the reader ignore any term related to the cutoff in space M( x +V t -R) + and its smoothed variants on first reading, in effect showing the comparison principle with the unbounded space domain R d replaced by the compact space T d .

Proof of Theorem 3.5. Suppose for the sake of contradiction that there exists T > 0 with sup

[0,T ]×R d (u -v -ϕ) > sup {0}×R d (u -v -ϕ), (3.42)
where ϕ(t,x) ∶= M x + V t -R + . The proof proceeds in three steps. First we smoothen and perturb (3.42), then we use a variable doubling argument to obtain a system of inequalities, and finally we contradict this system of inequalities.

Step 1: smoothing and perturbing. Let ε 0 ∈ (0,1) be a parameter to be determined, and let θ ∈ C ∞ (R;R) be a non-decreasing function such that, for every r ∈ R,

(r -ε 0 ) + ⩽ θ (r) ⩽ r + .
We introduce the function

Φ(t,x) ∶= Mθ ε 0 + x 2 1 2 +V t -R defined on R ⩾0 × R d .
The choice of θ and the bound (a + b)

+ ⩽ a + + b + imply that ϕ(t,x) ⩽ Φ(t,x) + Mε 0 ⩽ ϕ(t,x) + Mε 1 2 0 + Mε 0 ,
where we have used that (a + b)

1 2 ⩽ a 1 2 + b 1 2 for a,b > 0. It follows by (3.42) that sup {0}×R d (u -v -Φ) < sup [0,T ]×R d (u -v -Φ) + Mε 0 + Mε 1 2 0 ,
so choosing ε 0 > 0 small enough guarantees that sup

[0,T ]×R d (u -v -Φ) > sup {0}×R d (u -v -Φ). (3.43)
This is a smoothed version of the hypothesis (3.42) we aim to contradict. We now also add a cutoff function in time. For a small parameter ε > 0 to be determined, we introduce the function

χ(t,x) ∶= Φ(t,x) + ε T -t , defined on R ⩾0 × R d . Choosing ε > 0 small enough ensures that sup [0,T ]×R d (u -v -χ) > sup {0}×R d (u -v -χ . (3.44)
Step 2: system of inequalities. For each α ⩾ 1, we define the function

Ψ α ∶ [0,T ] × R d × [0,T ] × R d → R ∪ {-∞} by Ψ α (t,x,t ′ ,x ′ ) ∶= u(t,x) -v(t ′ ,x ′ ) - α 2 t -t ′ 2 + x -x ′ 2 -χ(t,x). (3.45)
We now argue that the function Ψ α achieves its supremum at a point (t α ,x α ,t ′ α ,x ′ α ) which remains bounded as α tends to infinity. In order to do so, we write C < +∞ to denote a constant whose value might change as we proceed through the argument, and which may depend on T , M, R, V , sup t⩽T u(t,0) , sup t⩽T v(t,0) and L. For every

x ∈ R d with x > R + 1, the bound Φ(t,x) ⩾ M x +V t -R -1 + reveals that Ψ α (t,x,t ′ ,x ′ ) ⩽ L x + x ′ + u(t,0) -v(t ′ ,0) - α 2 x -x ′ 2 -Φ(t,x) ⩽ L x + x ′ - α 2 x -x ′ 2 -M x +C ⩽ (2L -M) x + L x -x ′ - α 2 x -x ′ 2 +C ⩽ (2L -M) x + L 2 2α +C.
We have used that the function y ↦ Ly-α 2 y 2 achieves its maximum at y = L α . Observe also that the supremum of (3.45) is bounded from below by Ψ α (0,0,0,0), which does not depend on α, and that M > 2L. This implies that x α remains bounded as α tends to infinity, and that

α t α -t ′ α 2 + x α -x ′ α 2 + χ(t α ,x α ) ⩽ C. (3.46)
It follows that, up to the extraction of a subsequence, there exist t 0 ∈ [0,T ] and x 0 ∈ R d such that t α → t 0 , t ′ α → t 0 , x α → x 0 and x ′ α → x 0 as α → +∞. By (3.46), we have that χ(t α ,x α ) remains bounded, and thus in particular, it must be that t 0 ∈ [0,T ). On the other hand, the continuity of u, v and χ together with the bounds sup

[0,T ]×R d (u -v -χ) ⩽ Ψ α (t α ,x α ,t ′ α ,x ′ α ) ⩽ u(t α ,x α ) -v(t ′ α ,x ′ α ) -χ(t α ,x α ) imply that (u -v -χ)(t 0 ,x 0 ) = sup [0,T ]×R d (u -v -χ).
By (3.44), we thus deduce that t 0 ∈ (0,T ). This means that t α ,t ′ α ∈ (0,T ) for all α large enough. We have therefore found a quadruple (t α ,x α ,t ′ α ,x ′ α ) such that Ψ α achieves its supremum at (t α ,x α ,t ′ α ,x ′ α ), and with t α ,t ′ α ∈ (0,T ) for α large enough. With this in mind, we fix α ⩾ 1 large enough, and introduce the smooth functions

φ ,φ ′ ∈ C ∞ ((0,T ) × R d ;R) defined by φ (t,x) ∶= v(t ′ α ,x ′ α ) + α 2 t -t ′ α 2 + x -x ′ α 2 + χ(t,x), φ ′ (t ′ ,x ′ ) ∶= u(t α ,x α ) - α 2 t ′ -t α 2 + x ′ -x α 2 -χ(t α ,x α ).
Since (t α ,x α ,t ′ α ,x ′ α ) maximizes Ψ α , the function uφ achieves its maximum at the point (t α ,x α ) ∈ R >0 × R d , while the function vφ ′ achieves its minimum at

(t ′ α ,x ′ α ) ∈ R >0 × R d .
It follows by the definition of a viscosity subsolution and supersolution that

∂ t φ -H(∇φ ) (t α ,x α ) ⩽ 0 and ∂ t φ ′ -H(∇φ ′ ) (t ′ α ,x ′ α ) ⩾ 0. (3.47)
This is the system of inequalities that we now strive to contradict.

Step 3: reaching a contradiction. A direct computation shows that

∂ t φ -H(∇φ ) (t α ,x α ) = α(t α -t ′ α ) + ∂ t Φ(t α ,x α )+ ε (T -t α ) 2 -H ∇φ (t α ,x α ) , (3.48)
and

∂ t φ ′ -H(∇φ ′ ) (t ′ α ,x ′ α ) = α(t α -t ′ α ) -H ∇φ ′ (t ′ α ,x ′ α ) . ( 3
φ (t α ,x α + εz) -φ (t α ,x α ) ⩾ u(t α ,x α + εz) -u(t α ,x α ) ⩾ -εL z .
Dividing by ε and letting ε tend to zero reveals that ∇φ (t α ,x α ) ⋅ z ⩾ -L z . Choosing z = -∇φ (t α ,x α ) gives the first inequality in (3.50); the second inequality is obtained in an identical manner. Together with (3.49) and the definition of V , this implies that (3.49) is bounded from above by

α(t α -t ′ α ) -H ∇φ (t α ,x α ) +V ∇φ (t α ,x α ) -∇φ ′ (t ′ α ,x ′ α ) = α(t α -t ′ α ) -H ∇φ (t α ,x α ) +V ∇Φ(t α ,x α ) . A direct computation shows that V ∇Φ(t α ,x α ) ⩽ ∂ t Φ(t α ,x α ), so in fact ∂ t φ ′ -H(∇φ ′ ) (t ′ α ,x ′ α ) ⩽ α(t α -t ′ α ) + ∂ t Φ(t α ,x α ) -H ∇φ (t α ,x α ) < ∂ t φ -H(∇φ ) (t α ,x α ) ⩽ 0,
where the strict inequality is due to the term ε (T -t) 2 in (3.48), and the final inequality leverages the first inequality in (3.47). This contradicts the second inequality in (3.47) and completes the proof. ∎

R ⩾0 ×R d (u -v) = sup {0}×R d (u -v). (3.51)
Proof. Suppose for the sake of contradiction that there is a point

(t * ,x * ) ∈ R >0 × R d such that (u -v)(t * ,x * ) > sup {0}×R d (u -v). (3.52)
In the notation of Theorem 3.5, we choose M ∶= 2L + 1 and R ∶= x * +V t * , so that

u(t * ,x * ) -v(t * ,x * ) -M x * +V t * -R + = (u -v)(t * ,x * ).
By the assumption (3.52), this is strictly greater than sup

x∈R d u(0,x) -v(0,x) ⩾ sup x∈R d u(0,x) -v(0,x) -M( x -R) + .
This contradicts Theorem 3.5 and thus completes the proof. ∎

Finally, we verify that the comparison principle implies the uniqueness of a viscosity solution with a given initial condition. Proof. Since u is a viscosity subsolution and v is a viscosity supersolution to the Hamilton-Jacobi equation (3.20), the comparison principle in Corollary 3.6 implies that u ⩽ v. A symmetric argument reveals that v ⩽ u, and completes the proof. ∎

To relate this uniqueness result back to the limit free energy in the Curie-Weiss model, we recall what we have done so far. Using the Arzelà-Ascoli theorem, we argued that the sequence (F N ) N⩾1 of free energies in the Curie-Weiss model is precompact. In Proposition 3.4, leveraging the approximate Hamilton-Jacobi equation (3.7), we showed that any subsequential limit must be a viscosity solution to the Hamilton-Jacobi equation (3.9) with initial condition ψ(h) ∶= logcosh(h). Together with the uniqueness result in Corollary 3.7, this implies that (F N ) N⩾1 must itself converge to the unique solution f to the Hamilton-Jacobi equation (3.7).

We have thus succeeded in showing the convergence of the free energy in the Curie-Weiss model, and we have found an intrinsic characterization of its limit in terms of Hamilton-Jacobi equations. We could go about and study properties of this limit using PDE techniques, most importantly the comparison principle we have just seen. For instance, we can observe that the initial condition ψ = logcosh for the Curie-Weiss model is non-negative and bounded from above by a parabola: there exists C < +∞ such that ψ(h) ⩽ Ch 2 . Using the comparison principle, we deduce that the solution f will remain non-negative and bounded from above by the solution with the initial condition h ↦ Ch 2 . This solution can be identified explicitly as (t,h) ↦ Ch 2 1-4Ct , for t < 1 (4C). So we deduce in particular that f will remain differentiable at h = 0, with null derivatives, at least up until the time t = 1 (4C).

Of course, we also know from the arguments based on large deviation principles that the limit free energy can be written as a variational formula, see Corollary 2.20. As we will now see, this variational formula can be recovered using general results for solutions to Hamilton-Jacobi equations of the form (3.20), under suitable convexity assumptions.

Exercise 3.6. Let H ∶ R d → R be a uniformly Lipschitz non-linearity with Lipschitz constant V > 0, let f be a viscosity subsolution to (3.20), and let

Φ ∈ C ∞ (R >0 × R d ;R) be a smooth function with ∂ t Φ ⩾ V ∇Φ . Show that f -Φ is a subsolution to (3.20).
Exercise 3.7. Let f be a Lipschitz viscosity solution to (3.20), and let L ∶= f (0,⋅) Lip denote the Lipschitz constant of its initial condition. Show that sup t⩾0 f (t,⋅) Lip ⩽ L.

Exercise 3.8. Fix L ∈ R. Let f and g be two Lipschitz viscosity solutions to (3.20) with sup t⩾0 f (t,⋅) Lip ⩽ L, sup t⩾0 g(t,⋅) Lip ⩽ L, and let V be defined by (3.40). Suppose that there exist x ∈ R d and T > 0 with f (0,⋅) = g(0,⋅) on B V T (x). Prove that f (T,x) = g(T,x). Exercise 3.9. Let K ∶ R d → R be a Lipschitz function, let f be a Lipschitz viscosity solution to (3.20), and let g be a Lipschitz viscosity solution to ∂ t g -K(∇g) = 0 on R >0 × R d . Suppose that f (0,⋅) = g(0,⋅), that L denotes a Lipschitz constant of f and g, and that H and K coincide on B L (0). Prove that f = g.

Variational representations of viscosity solutions

The Hamilton-Jacobi approach we have developed so far is able to identify the limit free energy in the Curie-Weiss model as the unique viscosity solution to the Hamilton-Jacobi equation (3.9). In this section, we develop variational representations for the solution to the Hamilton-Jacobi equation (3.20), under appropriate convexity assumptions. This will allow us to recover the variational formula for the limit free energy in the Curie-Weiss model obtained in Corollary 2.20 using the Hamilton-Jacobi approach. We will prove two variational formulas, the Hopf-Lax formula in the setting when the non-linearity H is convex, and the Hopf formula in the setting when the initial condition ψ is convex. This implies in particular that solutions to the Hamilton-Jacobi equation (3.20) do exist under the stated convexity assumptions. A general result of existence of solutions for these equations can be obtained using the classical Perron method, but we will not develop this point here, and only refer the interested reader to [START_REF] Bardi | Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations[END_REF][START_REF] Michael | User's guide to viscosity solutions of second order partial differential equations[END_REF][START_REF] Dominguez | Infinite-dimensional Hamilton-Jacobi equations for statistical inference on sparse graphs[END_REF] for more on this. In the context of the problems of statistical mechanics that we aim to study, the existence of solutions should come for free since the limit free energy is expected to be such a solution.

As we will see, under different assumptions, the Hopf-Lax and the Hopf formulas allow us to write the solution to the Hamilton-Jacobi equation (3.20) with initial condition ψ as a saddle-point problem for the functional defined, for each Since the proofs in this section are a bit long and the ideas developed there will not reappear later, the reader may consider skipping these proofs on first reading.

(t,x) ∈ R ⩾0 × R d and (y, p) ∈ R d × R d , by J t,x (y, p) ∶= ψ(y) + p ⋅ (x -y) +tH(p). ( 3

Hopf-Lax formula

The Hopf-Lax formula is a variational representation for the unique solution to the Hamilton-Jacobi equation (3.20) in the setting when the non-linearity H is convex. It reads as follows.

Theorem 3.8 (Hopf-Lax formula [START_REF] Lions | Generalized solutions of Hamilton-Jacobi equations[END_REF]). If ψ ∶ R d → R is a Lipschitz continuous initial condition and H ∶ R d → R is a convex and locally Lipschitz continuous nonlinearity, then the Hopf-Lax function

f (t,x) ∶= sup y∈R d inf p∈R d J t,x (y, p) = sup y∈R d ψ(y) -tH * y -x t (3.54)
is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20).

We recall from (2.14) that H * denotes the convex dual of H. If we assume that the non-linearity H is concave instead of convex, then we can appeal to Theorem 3.8 to obtain a variational representation of the viscosity solution to (3.20) as well. Indeed, this follows from the observation that a function f is a viscosity solution to (3.20) if and only if the function g ∶=f is a viscosity solution to ∂ t g + H(-∇g) = 0.

(3.55)

In the case when H is concave, we thus obtain that the solution f to (3.20) with initial condition ψ is given by

f (t,x) = inf y∈R d sup p∈R d J t,x (y, p). (3.56)
To prove Theorem 3.8, we first verify that the function (3.54) satisfies the right initial condition, and that the supremum in its definition is attained. We then show that it satisfies a semigroup property from which we deduce that it belongs to the space L. Finally, we show that it is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20) with initial condition ψ. (3.57)

Proof. We interpret the formula in (3.54) at t = 0 as

f (0,x) = sup y∈R d inf p∈R d ψ(y) + p ⋅ (x -y) .
Taking y = x on the right side of this expression gives the lower bound f (0,x) ⩾ ψ(x).

On the other hand, given y ∈ R d , choosing

p = p(y) ∶= ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -ψ Lip x-y x-y if y ≠ x 0 otherwise
gives the upper bound

f (0,x) ⩽ sup y∈R d ψ(y) + p(y) ⋅ (x -y) = sup y∈R d ψ(y) -ψ Lip x -y ⩽ ψ(x),
where we have used the Lipschitz continuity of the initial condition ψ. This confirms that tH * ( y-x t ) should be interpreted as +∞ when t = 0 and y ≠ x. It also implies that, given a point (t,x) ∈ R ⩾0 × R d , it is possible to find R > 0 large enough such that for all y ∈ R d with yx > tR, we have

tH * y -x t ⩾ ( ψ Lip + 1) y -x . (3.60)
It follows by the Lipschitz continuity of ψ that for all y ∈ R d with yx > tR, we have ψ(y

) -tH * y -x t ⩽ ψ(x) -y -x .
This means that the supremum defining the Hopf-Lax function in (3.54) may be restricted to a bounded set. Together with the fact that the function y ↦ ψ(y) -tH * ( y-x t ) is upper semicontinuous and locally bounded from above, as H * (z) ⩾ -H(0), this implies that the supremum on the right side of (3.54) is achieved at some point y ∈ R d . This completes the proof. ∎ Lemma 3.11 (Semigroup property). Under the assumptions of Theorem 3.8, for every pair t > s ⩾ 0 and x ∈ R d ,

f (t,x) = sup y∈R d f (s,y) -(t -s)H * y -x t -s . (3.61) Proof. Fix y,z ∈ R d . Since H * is convex by Exercise 2.

10, we have

H * y -x t ⩽ s t H * y -z s + t -s t H * z -x t -s .
Substituting this bound into (3.54) yields

f (t,x) ⩾ ψ(y) -sH * y -z s -(t -s)H * z -x t -s .
Taking the supremum over all y ∈ R d gives

f (t,x) ⩾ f (s,z) -(t -s)H * z -x t -s ,
and taking the supremum over all z ∈ R d establishes the lower bound

f (t,x) ⩾ sup y∈R d f (s,y) -(t -s)H * y -x t -s .
To obtain the matching upper bound, we invoke Lemma 3.10 to find y ∈ R d with

f (t,x) = ψ(y) -tH * y -x t . Defining z ∶= s t x + t-s t y, we observe that z -x t -s = y -x t = y -z s .
In particular, taking

y ∈ R d in (3.54) gives f (s,z) -(t -s)H * z -x t -s ⩾ ψ(y) -sH * y -z s -(t -s)H * z -x t -s = ψ(y) -tH * y -x t = f (t,x).
Taking the supremum over z ∈ R d establishes the matching upper bound and completes the proof. ∎ Lemma 3.12. Under the assumptions of Theorem 3.8, we have f ∈ L with

sup t⩾0 f (t,⋅) Lip = ψ Lip . (3.62) Proof. Fix (t,x,x ′ ) ∈ R ⩾0 × R d × R d and invoke Lemma 3.10 to find y ∈ R d with f (t,x) = ψ(y) -tH * y -x t . Taking y -x + x ′ ∈ R d in (3.54) gives the lower bound f (t,x ′ ) ⩾ ψ(y -x + x ′ ) -tH * y -x t .
It follows that

f (t,x) -f (t,x ′ ) ⩽ ψ(y) -ψ(y -x + x ′ ) ⩽ ψ Lip x -x ′ .
Reversing the roles of x and x ′ gives y ′ ∈ R d with

f (t,x ′ ) -f (t,x) ⩽ ψ(y ′ ) -ψ(y ′ -x ′ + x) ⩽ ψ Lip x -x ′ .
Combining these two bounds establishes the spatial Lipschitz continuity (3.62) of the Hopf-Lax function. To conclude that f ∈ L, there remains to show that f is also continuous in time; we show that it is in fact Lipschitz continuous in time as well. Fix x ∈ R d and t > s ⩾ 0. The semigroup property in Lemma 3.11 implies that

f (t,x) ⩾ f (s,x) -(t -s)H * (0) ⩾ f (s,x) -(t -s)H(0). (3.63)
Combining the semigroup property in Lemma 3.11 with the spatial Lipschitz continuity (3.62) reveals that

f (t,x) ⩽ f (s,x) + sup y∈R d ψ Lip y -x -(t -s)H * y -x t -s = f (s,x) + (t -s) sup z∈R d ψ Lip z -H * (z) ⩽ f (s,x) + (t -s) sup p ⩽ ψ Lip sup z∈R d z ⋅ p -H * (z) ,
where the final inequality uses that ψ Lip z = z ⋅ ψ Lip z z

. Invoking the Fenchel-Moreau theorem (Theorem 2.5) and remembering (3.63) establishes the temporal Lipschitz continuity of the Hopf-Lax function,

f (t,x) -f (s,x) ⩽ t -s sup p ⩽ ψ Lip H(p) .
(3.64)

The convexity of the non-linearity H has played its part. This completes the proof. ∎

Proof of Theorem 3.8. The proof proceeds in two steps. First we show that the Hopf-Lax function (3.54) is a viscosity supersolution to the Hamilton-Jacobi equation (3.20), and then that it is also a viscosity subsolution to this equation. Together with Lemma 3.12 and the uniqueness result in Corollary 3.7, this proves that the Hopf-Lax function (3.54) is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20).

Step 1: viscosity supersolution.

Consider a smooth function φ ∈ C ∞ (R >0 × R d ;R) with the property that f -φ has a local minimum at the point (t * ,x * ) ∈ R >0 × R d .
By definition of a local minimum, for every s ∈ (0,t * ) sufficiently small and y ∈ R d , we have

φ (t * ,x * ) -φ (t * -s,x * + sy) ⩾ f (t * ,x * ) -f (t * -s,x * + sy).
It follows by the semigroup property in Lemma 3.11 that

φ (t * ,x * ) -φ (t * -s,x * + sy) ⩾ sH * sy s = sH * (y).
Dividing by s and letting s tend to zero reveals that

∂ t φ (t * ,x * ) -y ⋅ ∇φ (t * ,x * ) + H * (y) ⩾ 0.
Taking the infimum over y ∈ R d and invoking the Fenchel-Moreau theorem (Theorem 2.5) shows that

∂ t φ -H(∇φ ) (t * ,x * ) = ∂ t φ (t * ,x * ) -H * * ∇φ (t * ,x * ) ⩾ 0.
The convexity of the non-linearity H has played its part. This verifies the supersolution criterion for the Hopf-Lax function f .

Step 2: viscosity subsolution.

Consider a smooth function φ ∈ C ∞ (R >0 × R d ;R)
with the property that fφ has a local maximum at the point

(t * ,x * ) ∈ R >0 × R d ,
and suppose for the sake of contradiction that there is δ > 0 such that for all (t ′ ,x ′ ) sufficiently close to (t * ,x * ), we have

∂ t φ -H(∇φ ) (t ′ ,x ′ ) ⩾ δ > 0.
Using the convexity of H and the Fenchel-Moreau theorem, this may be recast as the assumption that for all (t ′ ,x ′ ) sufficiently close to (t * ,x * ) and all y ∈ R d , we have

∂ t φ (t ′ ,x ′ ) -y ⋅ ∇φ (t ′ ,x ′ ) + H * (y) ⩾ δ . (3.65)
Leveraging the semigroup property in Lemma 3.11 and arguing as in the proof of Lemma 3.10, it is possible to find R > 0 with the property that for all s > 0 sufficiently small there is y s ∈ R d with x *y s ⩽ Rs and

f (t * ,x * ) = f (t * -s,y s ) -sH * y s -x s . Writing u(r) ∶= (rt * + (1 -r)(t * -s),rx * + (1 -r)y s )
, it follows by the fundamental theorem of calculus and the absurd assumption (3.65) with y = ys-x * s that

φ (t * ,x * ) -φ (t * -s,y s ) = ˆ1 0 d dr φ (u(r))dr = ˆ1 0 s∂ t φ + (x * -y s ) ⋅ ∇φ (u(r))dr ⩾ sδ -sH * y s -x * s = sδ + f (t * ,x * ) -f (t * -s,y s ).
Rearranging shows that for s sufficiently small,

f (t * -s,y s ) -φ (t * -s,y s ) ⩾ sδ + f (t * ,x * ) -φ (t * ,x * ).
This contradicts the local maximality of fφ at (t * ,x * ) and completes the proof. ∎

Hopf formula

The Hopf formula is a variational representation for the unique solution to the Hamilton-Jacobi equation (3.20) in the setting when the initial condition ψ is convex. It reads as follows.

Theorem 3.13 (Hopf formula [START_REF] Bardi | On Hopf's formulas for solutions of Hamilton-Jacobi equations[END_REF][START_REF] Lions | Hopf formula and multitime Hamilton-Jacobi equations[END_REF]). If ψ ∶ R d → R is a Lipschitz continuous and convex initial condition, and

H ∶ R d → R is a locally Lipschitz continuous non-linearity, then the Hopf function f (t,x) ∶= sup p∈R d inf y∈R d J t,x (y, p) = sup p∈R d inf y∈R d ψ(y) + p ⋅ (x -y) +tH(p) (3.66)
is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20).

Compared with the Hopf-Lax formula in (3.54), notice that the supremum is now taken over p while the infimum is taken over y. Assuming that the infimum over y is achieved and that ψ is smooth, a differentiation in y shows that p = ∇ψ(y) at the optimum. For each fixed y, the mapping (t,x) ↦ ψ(y)+∇ψ(y)⋅(x-y)+tH(∇ψ(y)) is the solution to the Hamilton-Jacobi equation (3.20) with initial condition given by the tangent of ψ at y. Since ψ is convex, the comparison principle tells us that the solution to (3.20) should indeed lie above this family of solutions.

If we assume instead that the initial condition ψ is concave, then we can again use the observation around (3.55) and appeal to Theorem 3.13 to obtain that the viscosity solution to (3.20) is given by

f (t,x) = inf p∈R d sup y∈R d J t,x (y, p).
(3.67)

We now turn to the proof of Theorem 3.13. In order to do so, we first verify that the function (3.66) satisfies the right initial condition, and that the supremum in its definition is attained. We next argue that this function is jointly convex and belongs to the space L. Finally, we show that it satisfies a semigroup property, and deduce that it is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20). It will be convenient to notice that the Hopf function (3.66) can be written as Proof. The first equality of (3.68) is a representation of f as a supremum of affine functions of (t,x), so f is jointly convex by Exercise 2.7. To prove Lipschitz continuity, we fix t,t ′ ⩾ 0 as well as x,x ′ ∈ R d , and invoke Lemma 3.15 to find p ∈ R d with p ⩽ ψ Lip and f (t,x) = p ⋅ x + tH(p)ψ * (p). The Cauchy-Schwarz inequality implies that

f (t,x) = sup p∈R d p ⋅ x +tH(p) -ψ * (p) = ψ * -tH * (x). ( 3
f (t,x) -f (t ′ ,x ′ ) ⩽ p ⋅ (x -x ′ ) + H(p)(t -t ′ ) ⩽ p x -x ′ + H(p) t -t ′ ⩽ ψ Lip x -x ′ + sup q ⩽ ψ Lip H(q) t -t ′ .
An identical argument with the roles of (x,t) and (x ′ ,t ′ ) reversed establishes the Lipschitz continuity (3.71) of the Hopf function. It also shows that f is continuous in (t,x), and thus that f ∈ L. ∎ Lemma 3.17 (Semigroup property). Under the assumptions of Theorem 3.13, for every t,s ⩾ 0 and x ∈ R d ,

f (t + s,⋅) = ( f * (t,⋅) -sH) * . (3.72)
Proof. We will use properties (i)-(iii) of the convex dual established in Exercise 2.10. Property (ii) implies that (ψ * -tH) * * ⩽ ψ * -tH. Combining this with the representation (3.68) of the Hopf function reveals that

f * (t,⋅) -sH = (ψ * -tH) * * -sH ⩽ ψ * -(t + s)H,
and leveraging property (iii) gives the upper bound

( f * (t,⋅) -sH) * ⩾ (ψ * -(t + s)H) * = f (t + s,⋅). (3.73)
Similarly, property (ii) implies that

ψ * -(t + s)H ⩾ (ψ * -(t + s)H) * * .
Multiplying through by t t+s = 1 -s t+s reveals that

ψ * -tH ⩾ s t + s ψ * + t t + s (ψ * -(t + s)H) * * . (3.74)
The functions ψ * and (ψ * -(t + s)H) * * are both convex by property (i), so, as a positive linear combination of convex functions, the right side of (3.74) defines a convex function. It follows by the Fenchel-Moreau theorem and two applications of property (iii) that

(ψ * -tH) * * ⩾ s t + s ψ * + t t + s (ψ * -(t + s)H) * * ,
and therefore,

(ψ * -(t + s)H) * * -(ψ * -tH) * * ⩽ s t + s (ψ * -(t + s)H) * * -ψ * ⩽ s t + s ψ * -(t + s)H -ψ * = -sH,
where the second inequality uses property (ii). Leveraging the dual representation (3.68), this inequality may be rewritten as

f * (t + s,⋅) ⩽ f * (t,⋅) -sH.
Remembering that f is convex by Lemma 3.16, appealing to the Fenchel-Moreau theorem, and using property (iii) once more gives the lower bound

f (t + s,⋅) = f * * (t + s,⋅) ⩾ ( f * (t,⋅) -sH) * .
Together with the upper bound (3.73), this completes the proof. ∎

Proof of Theorem 3.13. The proof proceeds in two steps. First we show that the Hopf function (3.66) is a viscosity subsolution to the Hamilton-Jacobi equation (3.20), and then that it is also a viscosity supersolution to this equation. Together with Lemma 3.16 and the uniqueness result in Corollary 3.7, this proves that the Hopf function (3.66) is the unique viscosity solution in L to the Hamilton-Jacobi equation (3.20).

Step 1: viscosity subsolution.

Consider a smooth function φ ∈ C ∞ (R >0 × R d ;R) with the property that f -φ has a local maximum at the point (t * ,x * ) ∈ R >0 × R d .
Invoking Lemma 3.15, we may find p ∈ R d with p ⩽ ψ Lip and

f (t * ,x * ) = p ⋅ x * +t * H(p) -ψ * (p).
Together with the definition of a local maximum, this implies that for s ⩾ 0 small enough, z ∈ R d and ε ⩾ 0 small enough,

φ (t * ,x * ) -φ (t * -s,x * + εz) ⩽ f (t * ,x * ) -f (t * -s,x * + εz) ⩽ -p ⋅ εz + sH(p), (3.75)
where we used the expression (3.68) for f (t *s,x * + εz) in the second step. Taking s = 0, dividing by ε, and letting ε tend to zero shows that (∇φ

(t * ,x * ) -p) ⋅ z ⩾ 0. Since z ∈ R d is arbitrary, this means that p = ∇φ (t * ,x * ). (3.76)
On the other hand, taking ε = 0, dividing by s, and letting s tend to zero in (3.75) reveals that

∂ t φ (t * ,x * ) ⩽ H(p).
Rearranging and remembering (3.76) shows that ∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0, which is the required subsolution criterion.

Step 2: viscosity supersolution.

Consider a smooth function φ ∈ C ∞ (R >0 × R d ;R) with the property that f -φ has a local minimum at the point (t * ,x * ) ∈ R >0 × R d . We fix λ ∈ (0,1] and (t ′ ,x ′ ) ∈ R ⩾0 × R d .
The joint convexity of f established in Lemma 3.16 and the definition of a local minimum imply that, for λ small enough,

f (t ′ ,x ′ ) -f (t * ,x * ) ⩾ 1 λ f (t * + λ (t ′ -t * ),x * + λ (x ′ -x * )) -f (t * ,x * ) ⩾ 1 λ φ (t * + λ (t ′ -t * ),x * + λ (x ′ -x * )) -φ (t * ,x * ) .
Letting λ tend to zero shows that

f (t ′ ,x ′ ) -f (t * ,x * ) ⩾ (t ′ -t * )∂ t φ (t * ,x * ) + ∇φ (t * ,x * ) ⋅ (x ′ -x * ).
Together with the semigroup property in Lemma 3.17 and two applications of property (ii) in Exercise 2.10, this implies that

f (t * ,x * ) = f * (t * -s,⋅) -sH * (x * ) ⩾ f (t * ,x * ) -s∂ t φ (t * ,x * ) + ∇φ (t * ,x * ) ⋅ (⋅ -x * ) * -sH * (x * ).
To leverage this bound, observe that

f (t * ,x * ) -s∂ t φ (t * ,x * ) + ∇φ (t * ,x * ) ⋅ (⋅ -x * ) * (q) = s∂ t φ (t * ,x * ) -f (t * ,x * ) + ∇φ (t * ,x * ) ⋅ x * + sup y∈R d y ⋅ q -∇φ (t * ,x * ) ,
which is infinite unless q = ∇φ (t * ,x * ). It thus follows from the definition of the convex dual that

f (t * ,x * ) ⩾ x * ⋅ ∇φ (t * ,x * ) -s∂ t φ (t * ,x * ) -f (t * ,x * ) + ∇φ (t * ,x * ) ⋅ x * -sH(∇φ (t * ,x * )) .
Rearranging and dividing by s > 0 shows that ∂ t φ -H(∇φ ) (t * ,x * ) ⩾ 0 and completes the proof. ∎

Of course, when both the non-linearity H and the initial condition ψ are convex, the uniqueness result in Corollary 3.7 ensures that the Hopf-Lax and the Hopf variational representations coincide. It is also possible to show this directly using the Fenchel-Moreau theorem. Proof. Using the definition of the convex dual g * and then applying the Fenchel-Moreau theorem to the convex function f shows that sup

x∈R d inf y∈R d f (x) + g(y) -x ⋅ y = sup x∈R d f (x) -g * (x) = sup x∈R d sup y∈R d x ⋅ y -f * (y) -g * (x) .
Since this expression is symmetric in the pair ( f ,g), this establishes (3.77). More explicitly, we can write sup

y∈R d sup x∈R d x ⋅ y -f * (y) -g * (x) = sup y∈R d g(y) -f * (y) = sup y∈R d inf x∈R d f (x) + g(y) -x ⋅ y .
When both H and ψ are convex, we can make the change of variables z = xy in (3.54) and (3.66) to rewrite the statement that the Hopf-Lax and Hopf formulas coincide as sup

z∈R d inf p∈R d ψ(x + z) +tH(p) -p ⋅ z = sup p∈R d inf z∈R d ψ(x + z) +tH(p) -p ⋅ z .
The fact that this identity is valid follows from (3.77). This completes the proof. ∎ Applying the Hopf formula to the non-linearity H(p) = p 2 and the convex initial condition ψ(h) ∶= logcosh(h) in the context of Proposition 3.4, we find that the limit free energy in the Curie-Weiss model is given by

f (t,h) = sup m∈R d mh +tH(m) -ψ * (m) = sup m∈[-1,1] mh +tm 2 -ψ * (m) , (3.78)
where the second equality combines the fact that ψ is Lipschitz continuous with Lipschitz constant one with Exercise 2.12. We have therefore given a proof of Corollary 2.20 on the limit of the free energy in the Curie-Weiss model using only the Hamilton-Jacobi approach.

Variational representations in the absence of convexity

In this section, we explore whether any aspect of the variational structures discussed in the previous section remains in the absence of any convexity or concavity assumption on the non-linearity or the initial condition. The considerations discussed here will not reappear until Section 6.6, which is the last section of the book, so the reader may consider skipping this section on first reading. For the generalized Curie-Weiss model and the model from statistical inference studied in Chapter 4, as well as for the broader class of models from statistical inference considered in [START_REF] Chen | Hamilton-Jacobi equations for inference of matrix tensor products[END_REF] and discussed briefly at the end of Section 4.3, the free energy is always convex. If we can show that it converges to the solution to a Hamilton-Jacobi equation, then we can represent its limit variationally using the Hopf variational formula. This convexity property of the free energy is however lost in the realm of spin glasses. In some cases, the non-linearity appearing in the relevant Hamilton-Jacobi equation is convex. This will allow us to appeal to a Hopf-Lax formula instead and to still represent the solution variationally; we will see in Chapter 6 that this variational representation is closely related to the Parisi formula. However, there are many spin-glass models of interest for which the nonlinearity in the equation is neither convex nor concave. It is therefore interesting to wonder whether, for general solutions to Hamilton-Jacobi equations, any aspect of the variational structure displayed in the Hopf and Hopf-Lax formulas is preserved in the absence of any convexity or concavity assumption on the initial condition or the non-linearity. Recall that the Hopf-Lax and Hopf formulas are expressed as variational problems for the function J t,x defined in (3.53). The specific property we want to discuss is whether we can always represent the solution f (t,x) to the Hamilton-Jacobi equation (3.20) as a critical value of the function J t,x ; in other words, we ask whether we can find a point (y * , p * ) such that ∇J t,x (y * , p * ) = 0 and f (t,x) = J t,x (y * , p * ). The next result shows that this property is indeed valid as soon as the initial condition or the non-linearity in the equation is convex (or concave). The gradient ∇J t,x is understood as being jointly in the two arguments of J t,x , that is,

∇J t,x = (∇ y J t,x ,∇ p J t,x ). Theorem 3.19. Let ψ ∈ C 1 (R d ;R) be Lipschitz continuous and let H ∈ C 1 (R d ;R).
Suppose that among the functions ψ and H, at least one of them is convex or concave, and let f ∶ R ⩾0 × R d → R be the viscosity solution to the Hamilton-Jacobi equation (3.20) with initial condition f (0,⋅) = ψ. Then there exists (y

* , p * ) ∈ R d × R d with ∇J t,x (y * , p * ) = 0 such that f (t,x) = J t,x (y * , p * ). (3.79)
We recall that under the assumptions of Theorem 3.19, the viscosity solution to (3.20) with initial condition ψ exists and admits a variational representation in terms of J t,x , by Theorems 3.8 and 3.13. For instance, if H is convex, then by Theorem 3.8, we have

f (t,x) = sup y∈R d inf p∈R d J t,x (y, p). (3.80)
We stress that the proof of Theorem 3.19 must leverage the convexity assumption, say on H here, beyond the validity of the representation (3.80). In other words, in general, it is not true that the mapping

(t,x) ↦ sup y∈R d inf p∈R d J t,x (y, p) (3.81)
will satisfy the property in (3.79), even in cases when the supremum and infimum are achieved. To give an example, consider the one-dimensional case with ψ(y) = logcosh(y) and H(p) = (p 2 -1) 2 . One can then verify that for every x ∈ R and t ⩾ 0,

ψ(x) = sup y∈R inf p∈R J t,x (y, p), (3.82)
and that the supremum and infimum are achieved. Yet, there will typically be no (y * , p * ) with ∇J t,x (y * , p * ) = 0 such that ψ(x) = J t,x (y * , p * ). For small t ⩾ 0, such a critical point (y * , p * ) = (y * (t,x), p * (t,x)) is unique, and the mapping (t,x) ↦ J t,x (y * (t,x), p * (t,x)) is in fact the solution to (3.20), as discussed further below in relation with the method of characteristics.

Proof of Theorem 3.19. We show the result assuming that H is convex. The other cases are identical, up to a change of sign or an interchange of the roles of ψ and H. We recall that since H is convex, the function f satisfies (3.80). We give ourselves a small parameter δ ∈ (0,1], and define the perturbed non-linearity

H δ (p) ∶= H(p) + δ p 2 ,
so that the mapping H δ is strictly convex, see (2.120) for a definition of strict convexity. We denote by J (δ )

t,x the functional J t,x with the non-linearity H replaced by the perturbed non-linearity H δ . We decompose the rest of the proof into four steps. First, we show that the saddle-point problem (3.81) associated with the perturbed functional J (δ ) t,x admits an optimizer. We then prove that the family (H * δ ) δ >0 satisfies a lower semicontinuity property as δ tends to zero, and use this to show that the saddle-point problem (3.81) associated with the perturbed functional J (δ ) t,x is continuous at zero with respect to the perturbation parameter. We then combine these results to establish (3.79).

Step 1: identifying an optimizer to the perturbed saddle-point problem. In this step, we show that for each δ ∈ (0,1],

there exists (y * δ , p * δ ) ∈ R d × R d with ∇J (δ ) t,x (y * δ , p * δ ) = 0 such that sup y∈R d inf p∈R d J (δ ) t,x (y, p) = J (δ ) t,x (y * δ , p * δ ). (3.83)
We will also make sure that (y * δ , p * δ ) is chosen in a fixed compact set not depending on δ > 0, so that we can then extract a converging subsequence as δ → 0.

It follows from (3.59)-(3.60) that we can find a compact set K such that for every δ ∈ [0,1],

sup y∈R d inf p∈R d J (δ ) t,x (y, p) = sup y∈K inf p∈R d J (δ ) t,x (y, p). (3.84)
Moreover, as was argued below (3.60), the supremum on the right side of (3.84) is achieved, say at some y * δ ∈ K. Using that H δ grows at least quadratically, one can also make sure that, for every y ∈ R d , there exists p δ (y) such that inf

p∈R d J (δ ) t,x (y, p) = J (δ ) t,x (y, p δ (y)). (3.85)
Since H is differentiable, we must have that ∇ p J (δ )

t,x (y, p δ (y)) = 0. Since H δ is strictly convex, there is in fact exactly one p δ (y) that realizes (3.85) (see the solution to Exercise 2.23 for a detailed argument). The envelope theorem (Theorem 2.21) therefore ensures that the mapping

y ↦ inf p∈R d J (δ ) t,x (y, p)
is differentiable, and that its gradient is ∇ y J (δ ) t,x (y, p δ (y)). By the optimality condition for y * δ , we conclude that

∇ y J (δ ) t,x (y δ , p δ (y δ )) = 0.
This means that ∇ψ(y δ ) = p δ (y δ ). In particular, we must have that p δ (y) ⩽ Ψ Lip .

We have therefore obtained (3.83), with

y * δ ∈ K and p * δ ∶= p δ (y * δ ) such that p * δ ⩽ Ψ Lip .
Step 2: establishing a lower semicontinuity property of (H * δ ). Let (q δ ) δ >0 be a family of points in R d that converge to q ∈ R d . In this step, we show that liminf

δ →0 H * δ (q δ ) ⩾ H * (q). (3.86) More explicitly, the claim is that liminf δ →0 sup p∈R d p ⋅ q δ -H(p) -δ p 2 ⩾ sup p∈R d p ⋅ q -H(p) . (3.87)
Assuming first that H * (q) is finite, we can find p ∈ R d such that p ⋅ q -H(p) ⩾ H * (q)ε.

Using p as a candidate in the variational problem on the left side of (3.87) yields the desired inequality, up to an error of ε. Since ε > 0 was arbitrary, this completes the argument. If instead H * (q) is infinite, then we can argue similarly, since for each M ∈ R, we can find p ∈ R d with p ⋅ q -H(p) ⩾ M. This completes the proof of (3.86).

Step 3: showing the perturbed saddle-point problem is continuous at zero. In this step, we show that lim

δ →0 sup y∈R d inf p∈R d J (δ ) t,x (y, p) = sup y∈R d inf p∈R d J t,x (y, p). (3.88)
We can rewrite (3.88) as lim

δ →0 sup y∈R d ψ(y) -H * δ y -x t = sup y∈R d ψ(y) -H * y -x t . (3.89)
Since for each y ∈ R d , the family (H * δ (y)) δ >0 is non-increasing, the limit on the left side of (3.89) exists, and since H * δ ⩽ H * , the statement of (3.89) with the equality sign replaced by ⩾ is valid. For the converse inequality, we recall that the supremum on the left side of (3.89) is achieved at some y * δ ∈ K, where K is a fixed compact set.

Up to the extraction of a subsequence, we may assume that y * δ converges to some y * ∈ K. By the result of the previous step, we obtain that lim

δ →0 sup y∈R d ψ(y) -H * δ y -x t = lim δ →0 ψ(y * δ ) -H * δ y * δ -x t ⩽ ψ(y * ) -H * y * -x t ⩽ sup y∈R d ψ(y) -H * y -x t .
This completes the proof of (3.89).

Step 4: concluding. We are now ready to conclude the proof. Up to the extraction of a subsequence, we may assume that the points y * δ and p * δ constructed in Step 1 converge as δ > 0 tends to zero, say to y * ∈ K and p * ⩽ Ψ Lip . By the result of the previous step, we have

f (t,x) = sup y∈R d inf p∈R d J t,x (y, p) = lim δ →0 sup y∈R d inf p∈R d J (δ ) t,x (y, p) = lim δ →0 J (δ ) t,x (y * δ , p * δ ) = J t,x (y * , p * ).
Since the functions H and ψ are continuously differentiable, we can also verify by continuity that ∇J t,x (y * , p * ) = 0, as desired. ∎

In fact, for smooth initial condition ψ, say ψ ∈ C 2 (R d ;R), the property in (3.79) remains valid for small t ⩾ 0 even in the absence of any convexity or concavity assumption on ψ or H. To explain this best, we start by explaining a classical method for solving the Hamilton-Jacobi equation (3.20) for a short time called the method of characteristics.

Suppose for a moment that, for some T ∈ (0,+∞], we have found a smooth solution f ∈ C 2 ([0,T ) × R d ;R) to the Hamilton-Jacobi equation (3.20) with initial condition ψ ∈ C 2 (R d ;R), and that the non-linearity H is also in C 2 (R d ;R). We fix i ∈ {1,...,d} and write u i ∶= ∂ x i f . Differentiating (3.20), we find that

∂ t u i (t,x) -∇H(∇ f (t,x)) ⋅ ∇u i (t,x) = 0. (3.90) For each t ∈ [0,T ) and x ∈ R d , we define X(t,x) such that X(0,x) = x and X(⋅,x) solves the ordinary differential equation ∂ t X(t,x) = -∇H(∇ f (t,X(t,x))). The curve t ↦ X(t,x) (or also t ↦ (t,X(t,x))) is called the characteristic starting from x ∈ R d . We find using (3.90) that d dt u i (t,X(t,x)) = ∂ t u i (t,X(t,x)) -∇H(∇ f (t,X(t,x))) ⋅ ∇u i (t,X(t,x)) = 0. (3.91)
In other words, the derivatives of f are constant along the curve t ↦ (t,X(t,x)); and in fact, this curve is therefore a straight line, with ∂ t X(t,x) = -∇H(∇ψ(x)). Finally, since

d dt f (t,X(t,x)) = ∂ t f (t,X(t,x)) -∇H(∇ψ(x)) ⋅ ∇ f (t,X(t,x)) = H(∇ f (t,X(t,x))) -∇H(∇ψ(x)) ⋅ ∇ f (t,X(t,x)) = H(∇ψ(x)) -∇H(∇ψ(x)) ⋅ ∇ψ(x), (3.92)
we conclude that

f (t,X(t,x)) = ψ(x) +t H(∇ψ(x)) -∇H(∇ψ(x)) ⋅ ∇ψ(x) . (3.93)
To sum up, we have argued that if a solution to the Hamilton-Jacobi equation (3.20) is sufficiently smooth, then it must satisfy (3.93). Let us now try to use this observation to actually construct the function f . Given the non-linearity H and the initial condition ψ, both assumed to be in C 2 (R d ;R) and with ∇ψ and ∇ 2 ψ bounded, we set

X(t,x) ∶= x -t∇H(∇ψ(x)). (3.94)
Using a fixed-point argument, one can check that for every t ⩾ 0 sufficiently small, the mapping x ↦ X(t,x) is bijective. This allows us to define the function f according to the formula (3.93). One can then verify that f is a classical solution to (3.20) on [0,T ) × R d for a sufficiently small T > 0, as will be explored in more detail in Exercise 3.10 below. This perspective thus allows us to understand the possible low regularity of the viscosity solution in terms of characteristic lines intersecting each other.

The statement (3.79) we derived in Theorem 3.19 also has a very natural interpretation in terms of characteristic lines. Indeed, the condition that the pair (y, p) ∈ R d × R d satisfies ∇J t,x (y, p) = 0 can be rewritten as p = ∇ψ(y) and x = y -t∇H(p).

(3.95)

A pair (y, p) therefore satisfies (3.95) if and only if p = ∇ψ(y) and the characteristic line t ↦ X(t,y) = y -t∇H(∇ψ(y)) satisfies X(t,y) = x. In short, there is a one-toone correspondence between critical points of J t,x and characteristic lines that pass through the point (t,x). Moreover, for a pair (y, p) satisfying (3.95), we have

J t,x (y, p) = ψ(y) + p ⋅ (x -y) +tH(p) (3.96) = ψ(y) +t H(∇ψ(y)) -∇H(∇ψ(y)) ⋅ ∇ψ(y) , (3.97)
in agreement with (3.93). This connection between the method of characteristics and critical points of J t,x has at least two interesting consequences. First, we learn that if ψ and H are in C 2 (R d ;R) with ∇ψ and ∇ 2 ψ bounded, then the property (3.79) is valid for every t ⩾ 0 sufficiently small, even in the absence of any convexity of concavity assumption on H or ψ. Second, we can also reinterpret Theorem 3.19 as saying that if at least one function among ψ or H is convex or concave, then at every (t,x) ∈ R ⩾0 × R d , including possibly large t ⩾ 0, there exists a characteristic line that prescribes the correct value for f (t,x). To state this more precisely, we may introduce the wavefront

W ∶= t, x -t∇H(∇ψ(x)), ψ(x) +t H(∇ψ(x)) -∇H(∇ψ(x)) ⋅ ∇ψ(x) t ⩾ 0, x ∈ R d . (3.98)
Theorem 3.19 states that whenever H or ψ is convex or concave, letting f denote the viscosity solution to (3.20), we have that the graph of f is a subset of the wavefront. Since we have just observed that this property is also valid for short times without any convexity assumption, one may expect that the graph of the viscosity solution to (3.20) is in fact always a subset of the wavefront. This is however not the case. Indeed, it was shown in [START_REF] Roos | Nonexistence of global characteristics for viscosity solutions[END_REF] that for any fixed non-linearity H ∈ C 2 (R d ;R) which is neither convex nor concave, there exists a smooth and Lipschitz continuous function ψ ∶ R d → R such that the graph of the viscosity solution to (3.20) with initial condition ψ is not a subset of the wavefront.

On the other hand, for any given H ∈ C 2 (R d ;R) and Lipschitz continuous ψ ∶ R d → R, one can show that there always exists a Lipschitz function f ∶ R ⩾0 × R d → R whose graph belongs to the wavefront [START_REF] Roos | Variational and viscosity operators for the evolutionary Hamilton-Jacobi equation[END_REF]. In particular, such a function f satisfies the Hamilton-Jacobi equation (3.20) almost everywhere, and f (0,⋅) = ψ. Such a function is however not unique in general.

In the context of the general non-convex spin-glass models discussed in Section 6.6, one can show that any subsequential limit of the free energy must satisfy a natural Hamilton-Jacobi equation "almost everywhere". (The quotes are due because the state space is infinite-dimensional.) It is not known whether the limit free energy is the viscosity solution to the equation. But surprisingly, it is shown in [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] that, assuming that the limit free energy exists, its graph must belong to the wavefront. We do not know whether the graph of the viscosity solution belongs to the wavefront in this case. We refer to Section 6.6 for a more precise discussion on this point.

Exercise 3.10. The goal of this exercise is to justify the method of characteristics for solving the Hamilton-Jacobi equation (3.20) for a short time. We assume that the non-linearity H and the initial condition ψ both belong to C 2 (R d ;R). We also assume that the first and second-order derivatives of ψ are uniformly bounded.

(i) Show that there exists T > 0 such that for every t ∈ [0,T ), the mapping

φ t ∶ x ↦ x -t∇H(∇ψ(x)) (3.99) is a C 1 diffeomorphism of R d ;
in other words, it is bijective and both φ t and its inverse belong to C 1 (R d ;R d ).

(ii) We define the function u ∶ [0,T ) × R d → R implicitly by setting, for every t ∈ [0,T ) and

x ∈ R d , u(t,φ t (x)) ∶= ψ(x) +t H(∇ψ(x)) -∇H(∇ψ(x)) ⋅ ∇ψ(x) . (3.100) Show that u ∈ C 1 ([0,T ) × R d ;R
) and that for every t ∈ [0,T ) and x ∈ R d , the gradient of u is constant along the characteristic line t ↦ (t,φ t (x)).

(iii) Deduce that u satisfies the Hamilton-Jacobi equation (3.20) at every point in [0,T ) × R d .

Leveraging convexity to identify viscosity solutions

The Hopf formula and Proposition 3.4 allowed us to recover the formula obtained in Corollary 2.20 for the limit free energy in the Curie-Weiss model using the Hamilton-Jacobi approach. However, if we now try to apply the same reasoning to recover the formula obtained in Theorem 2.19 for the limit free energy in the generalized Curie-Weiss model, then we run into a problem. The upper bound

∂ t F N -ξ ∂ h F N ⩽ C N ∂ 2 h F N (3.101)
implied by (3.17) should suffice to show that the limit free energy is a viscosity subsolution to the Hamilton-Jacobi equation (3.19); however the lower bound

∂ t F N -ξ ∂ h F N ⩾ - C N ∂ 2 h F N (3.102)
implied by (3.17) should ring alarm bells since the sign in front of the Laplacian seems to be wrong. As discussed in Section 3.2, the notion of viscosity solutions is entirely built around the validity of a maximum principle; but changing the sign in front of the Laplacian destroys this property. To see this clearly, one can think about solutions of the backwards heat equation, ∂ t f = -∆ f . Solutions to this equation may blow up in finite time, and it is not difficult to construct examples of initial conditions that are ordered, f (0,⋅) ⩽ g(0,⋅), and such that this ordering is no longer valid at some subsequent time.

One may wonder whether the negative sign in front of the Laplacian in (3.102) is just an artefact; maybe a more refined analysis would allow us to obtain a better estimate. In the case when the function ξ is convex, we have

⟨ξ 1 N N i=1 σ i ⟩ ⩾ ξ ⟨ 1 N N i=1 σ i ⟩ , (3.103)
by Jensen's inequality. This gives the improved lower bound

∂ t F N (t,h) -ξ ∂ h F N (t,h) ⩾ 0. (3.104)
Arguing exactly as in the proof of Proposition 3.4, we thereby obtain the following extension of Theorem 2.19 in the convex setting. Notice that for convex ξ , this is indeed an extension since we no longer require that ψ ∈ C 1 (R;R).

Proposition 3.20. For each integer N ⩾ 1, denote by F N ∶ R ⩾0 × R → R the free energy (3.
2) in the generalized Curie-Weiss model, and suppose that for every h ∈ R the limit

ψ(h) ∶= lim N→+∞ F N (0,h) (3.105)
exists. If ξ is convex, then the limit free energy f ∶ R ⩾0 × R → R in the generalized Curie-Weiss model is the unique viscosity solution to the Hamilton-Jacobi equation (3.19), and it admits the Hopf representation

f (t,h) ∶= lim N→+∞ F N (t,h) = sup m∈[-1,1] tξ (m) + hm -ψ * (m) . (3.106)
Proof. The sequence (F N ) N⩾1 of free energies is precompact by the Arzelà-Ascoli theorem and the Lipschitz bounds (3.13). Using the inequalities (3.101) and (3.104), we can follow the proof of Proposition 3.4 to show that any subsequential limit of (F N ) N⩾1 must be a viscosity solution to the Hamilton-Jacobi equation (3.19).

Invoking the uniqueness result in Corollary 3.7, we conclude that (F N ) N⩾1 converges to the unique solution f to the Hamilton-Jacobi equation (3.19). Since each of the initial conditions h ↦ F N (0,h) is convex, their limit ψ must also be convex. Invoking the Hopf formula in Theorem 3.13 yields that

f (t,h) = sup m∈R tξ (m) + hm -ψ * (m) . (3.107)
To restrict this supremum to the interval [-1,1], observe that ∂ h F N is uniformly bounded by one by the derivative computation (3.13), so the initial condition ψ must be Lipschitz continuous with Lipschitz constant one. It follows by Exercise 2.12 that ψ * is infinite outside the interval [-1,1]. This completes the proof. ∎

When ξ is convex, we were therefore able to improve upon the bound (3.102) and conclude. One may wonder if this could be achieved for general ξ . This is however not possible, as is most clearly demonstrated by considering the case of ξ (p) = -p 2 . Indeed, we obtain in this case that

∂ t F N + ∂ h F N 2 = - 1 N ∂ 2 h F N . (3.108)
So there really is a fundamental problem with the sign here; it is not just that the bounds we have derived are too crude. And yet, we know from Theorem 2.19

and the Hopf formula that the sequence (F N ) N⩾1 of free energies does indeed converge to the viscosity solution to the Hamilton-Jacobi equation (3.19) even in this case, at least when the initial condition ψ is in C 1 (R;R)! To prove this using the Hamilton-Jacobi approach, we now introduce a new tool, which we call the convex selection principle, to identify when a convex function f is a viscosity solution to the Hamilton-Jacobi equation (3.20). In Example 3.1, we described a function that satisfies a Hamilton-Jacobi equation almost everywhere but is not a viscosity solution to this equation. Notice that this counterexample had corner singularities "in both directions"; formally, the second derivative was neither bounded from above nor from below. A convex function cannot look like this, since its Hessian must be non-negative. Roughly speaking, the convex selection principle states that imposing the function to be convex completely rules out the emergence of non-viscosity-type singularities, and thus restores uniqueness. We will establish this result in the setting of the Hamilton-Jacobi equation (3.20). Recall that the non-linearity When we say that f satisfies the Hamilton-Jacobi equation (3.20) on a dense subset of R ⩾0 × R d , we mean that the set

H ∶ R d → R is locally Lipschitz continuous. Theorem 3.21 (Convex selection principle [75]). If f ∶ R ⩾0 × R d → R
(t,x) ∈ R >0 × R d f differentiable at (t,x) and ∂ t f -H(∇ f ) (t,x) = 0 (3.109) is dense in R ⩾0 × R d .
Naturally, Theorem 3.21 also holds if we replace "convex" by "concave" in the statement. We already stressed that, in some sense, the notion of viscosity solution is tailored to approximations in which a small positive term times the Laplacian of f appears on the right side of (3.20). In particular, the notion of viscosity solution is sensitive to the orientation of time; in general, it is not the case that the time-reversed viscosity solution to some equation will be the viscosity solution to the time-reversed equation. However, superficially, the statement of Theorem 3.21 looks invariant under time reversal. The only assumption that breaks this symmetry is that f (0,⋅) ∈ C 1 (R d ;R). This already hints at the fact that this assumption is necessary. In other words, Theorem 3.21 states that under the convexity assumption, pathological solutions cannot spontaneously emerge if we start from a smooth initial condition. However, if we start from a Lipschitz function that is not in C 1 (R d ;R), then we may be able to exploit the singularities of the initial condition to create solutions that differ from the viscosity solution.

To prove the convex selection principle, we first show that the function f in its statement must actually satisfy (3.20) at all its points of differentiability.

We then prove a convex selection principle with a somewhat weaker version of the assumption that f (0,⋅) ∈ C 1 (R d ;R) which will be convenient in Chapter 4 to analyze the model from statistical inference in full generality. On first reading, the reader may want to work out a simpler proof using directly the assumption that f (0,⋅) ∈ C 1 (R d ;R) -this stronger assumption will be valid in the most interesting cases discussed in Chapter 4. We recall that for a convex function

f ∶ R ⩾0 × R d → R, we denote by ∂ f (t,x) ⊆ R × R d the subdifferential of f at the point (t,x) ∈ R ⩾0 × R d , as defined in (2.40). Lemma 3.22. If f ∶ R ⩾0 ×R d → R
R >0 × R d . Moreover, for every (t,x) ∈ R ⩾0 × R d , there exists (a, p) ∈ ∂ f (t,x) such that a -H(p) = 0. Proof. Fix (t,x) ∈ R ⩾0 × R d , and let (t n ,x n ) n⩾1 ⊆ R >0 × R d be a sequence of points of differentiability of f converging to (t,x) at which ∂ t f -H(∇ f ) (t n ,x n ) = 0. (3.110) Since f is differentiable at the interior point (t n ,x n ) ∈ R >0 × R d , Theorem 2.13 implies that ∂ f (t n ,x n ) = {(∂ t f (t n ,x n ),∇ f (t n ,x n ))}.
The joint Lipschitz continuity of f implies that, up to the extraction of a subsequence, we may assume that the sequence of gradients

(∂ t f (t n ,x n ),∇ f (t n ,x n )) n⩾1 converges to some vector (a, p) ∈ R × R d .
∂ f (t,x) = {(∂ t f (t,x),∇ f (t,x))}. It must therefore be the case that (a, p) = (∂ t f (t,x),∇ f (t,x))
, and thus that f satisfies the Hamilton-Jacobi equation (3.20) at the point (t,x). This completes the proof. ∎

We now state and show the refined version of Theorem 3.21. Proof. We decompose the proof into two steps. First we show that f is a viscosity subsolution to the Hamilton-Jacobi equation (3.20), and then that it is a supersolution to this equation. The assumption on the initial condition will only play a role in showing that f is a viscosity supersolution to (3.20).

Lemma 3.23. Let f ∶ R ⩾0 × R d → R
Step 1: viscosity subsolution. Consider a smooth function φ

∈ C ∞ (R >0 × R d ;R) with the property that f -φ has a local maximum at the point (t * ,x * ) ∈ R >0 × R d .
We are going to show that f is differentiable at (t * ,x * ). By Proposition 2.11, the subdifferential ∂ f (t * ,x * ) contains at least one element, say

(a, p) ∈ R × R d .
By the definition of the subdifferential and of a local maximum, for every (t ′ ,x ′ ) sufficiently close to (t * ,x * ), we have

a(t ′ -t * ) + p ⋅ (x ′ -x * ) ⩽ f (t ′ ,x ′ ) -f (t * ,x * ) ⩽ φ (t ′ ,x ′ ) -φ (t * ,x * ). (3.111)
It follows from the smoothness of φ that, as

(t ′ ,x ′ ) tends to (t * ,x * ), (t ′ -t * ) a -∂ t φ (t * ,x * ) + (x ′ -x * ) p -∇φ (t * ,x * ) ⩽ o t ′ -t * + x ′ -x * .
This implies that (a, p) = (∂ t φ ,∇φ )(t * ,x * ). Using (3.111) once more, we obtain that f is differentiable at (t * ,x * ), and that

(∂ t f ,∇ f )(t * ,x * ) = (∂ t φ ,∇φ )(t * ,x * ). It follows by Lemma 3.22 that ∂ t φ -H(∇φ ) (t * ,x * ) = ∂ t f -H(∇ f ) (t * ,x * ) = 0.
This completes the verification that f is a viscosity subsolution to the Hamilton-Jacobi equation (3.20).

Step 2: viscosity supersolution.

Consider a smooth function φ ∈ C ∞ (R >0 × R d ;R)
with the property that fφ has a local minimum at the point

(t * ,x * ) ∈ R >0 × R d .
Together with the convexity of f , this implies that for every

(t ′ ,x ′ ) ∈ R ⩾0 × R d and ε > 0 small enough, f (t ′ ,x ′ ) -f (t * ,x * ) ⩾ ε -1 f (t * ,x * ) + ε(t ′ -t * ,x ′ -x * ) -f (t * ,x * ) ⩾ ε -1 φ (t * ,x * ) + ε(t ′ -t * ,x ′ -x * ) -φ (t * ,x * ) .
Letting ε tend to zero shows that

(∂ t φ ,∇φ )(t * ,x * ) ∈ ∂ f (t * ,x * ). It therefore suffices to fix (a, p) ∈ ∂ f (t * ,x * ) and prove that a -H(p) ⩾ 0. (3.112) Since (a, p) ∈ ∂ f (t * ,x * ), we have f (0,y) ⩾ f (t * ,x * ) -at * + (y -x * ) ⋅ p for every y ∈ R d . Rearranging shows that for every y ∈ R d , f (0,y) -y ⋅ p ⩾ f (t * ,x * ) -at * -p ⋅ x * . (3.113)
Inspired by the Hopf formula, we would like to consider an optimizer of the minimization problem inf

y∈R d f (0,y) -y ⋅ p . (3.114)
Since we cannot guarantee the existence of this optimizer, we introduce a small parameter ε > 0 and consider instead the perturbed minimization problem inf

y∈R d f (0,y) -y ⋅ p + ε 1 + y 2 . (3.115)
We see from (3.113) that we may restrict the infimum in (3.115) to values of y that range in a compact set (which depends on ε). Together with the continuity of the functions involved, this shows that the infimum in (3.115) is achieved, say at

y ε ∈ R d . We observe that lim ε→0 inf y∈R d f (0,y) -y ⋅ p + ε 1 + y 2 = inf y∈R d f (0,y) -y ⋅ p . (3.116)
Indeed, the existence of the limit on the left side of (3.116) and the fact that it is lower bounded by the right side are immediate. Conversely, for each δ > 0, we can find

y * δ ∈ R d such that f (0,y * δ ) -y * δ ⋅ p ⩽ inf y∈R d f (0,y) -y ⋅ p + δ ,
and we obtain the upper bound in (3.116) up to an error of δ > 0 by using y * δ as a candidate in the infimum on the left side of (3.116). Since δ > 0 was arbitrary, this shows (3.116). Since inf

y∈R d f (0,y) -y ⋅ p + ε 1 + y ε 2 ⩽ f (0,y ε ) -y ε ⋅ p + ε 1 + y ε 2 = inf y∈R d f (0,y) -y ⋅ p + ε 1 + y 2 ,
we deduce from (3.116) that lim

ε→0 ε y ε = 0. (3.117)
Using the convexity of f and that y ε is a minimizer of (3.115), we have that, for every y ∈ R d and λ ∈ (0,1],

f (0,y ε + y) -f (0,y ε ) ⩾ λ -1 f (0,y ε + λ y) -f (0,y ε ) ⩾ λ -1 λ y ⋅ p + ε 1 + y ε 2 -1 + y ε + λ y 2 .
Letting λ tend to zero shows that for every

y ∈ R d , f (0,y ε + y) -f (0,y ε ) ⩾ y ⋅ p -ε y ε 1 + y ε 2 , which means that p ε ∶= p -ε y ε 1 + y ε 2 ∈ ∂ ψ(y ε ). (3.118)
Invoking the assumption on the initial condition gives b

ε ∈ R with (b ε , p ε ) ∈ ∂ f (0,y ε ) and b ε -H(p ε ) ⩾ 0. (3.119) Since (b ε , p ε ) ∈ ∂ f (0,y ε ) and (a, p) ∈ ∂ f (t * ,x * ), we have f (t * ,x * ) ⩾ f (0,y ε ) + b ε t * + p ε ⋅ (x * -y ε ), f (0,y ε ) ⩾ f (t * ,x * ) -at * + p ⋅ (y ε -x * ).
Combining these two inequalities reveals that

b ε t * + p ε ⋅ (x * -y ε ) ⩽ at * + p ⋅ (x * -y ε ).
Using (3.117) and that p εp ⩽ ε yields that b ε ⩽ a + o ε (1), and using also (3.119), we obtain that a -H(p ε ) ⩾ o ε (1). Since H is continuous, this establishes (3.112) and completes the proof. ∎

Proof of Theorem 3.21. We write ψ ∶= f (0,⋅). By Lemma 3.23, it suffices to show that for every

x ∈ R d and p ∈ ∂ ψ(x), there exists b ∈ R such that (b, p) ∈ ∂ f (0,x) and b -H(p) ⩾ 0. Invoking Lemma 3.22 gives (b,q) ∈ ∂ f (0,x) with b -H(q) = 0. Since ψ ∈ C 1 (R d ;R), Theorem 2.
13 implies that p = q = ∇ψ(x). This completes the proof. ∎

To apply the convex selection principle to the generalized Curie-Weiss model, it will be convenient to observe that the set of points (t * ,x * ) ∈ R ⩾0 × R d for which there exists a smooth function that touches f from above at (t * ,x * ) is dense. This will allow us to replace the assumption that f satisfies the Hamilton-Jacobi equation (3.20) on a dense set by the assumption that whenever a smooth function φ touches the function f from above at a point (t * ,x * ), the function φ should satisfy the Hamilton-Jacobi equation (3.20) at the point of contact (t * ,x * ). This version of the convex selection principle will be well-suited for the generalized Curie-Weiss model due to the bound (3.17); it is also useful in other contexts such as statistical inference.

Corollary 3.24. Let f ∶ R ⩾0 × R d →
R be a jointly convex and jointly Lipschitz continuous function. Suppose that for any point

(t * ,x * ) ∈ R >0 × R d and smooth function φ ∈ C ∞ (R ⩾0 × R d ;R) such that f -φ has a strict local maximum at (t * ,x * ), we have (∂ t φ -H(∇φ ))(t * ,x * ) = 0. If moreover f (0,⋅) ∈ C 1 (R d ;R), then f is a viscosity solution to the Hamilton-Jacobi equation (3.20). Proof. Consider a smooth φ ∈ C ∞ (R >0 × R d ;R) with the property that f -φ admits a strict local maximum at the point (t * ,x * ) ∈ R >0 × R d . Following Step 1 of the proof of Lemma 3.23, we see that f is differentiable at the contact point (t * ,x * ) with (∂ t f ,∇ f )(t * ,x * ) = (∂ t φ ,∇φ )(t * ,x * ).
By the convex selection principle in Theorem 3.21, it therefore suffices to show that the set

A ∶= (t * ,x * ) ∈ R >0 × R d there exists φ ∈ C ∞ (R >0 × R d ;R) such that f -φ has a strict local maximum at (t * ,x * ) is dense in R >0 × R d . Fix (t 0 ,x 0 ) ∈ R >0 × R d ,
and let V be a compact neighbourhood of (t 0 ,x 0 ). For each α ⩾ 1, consider the mapping

φ α (t,x) ∶= f (t,x) - α 2 t -t 0 2 + x -x 0 2 ,
and denote by (t α ,x α ) a maximizer of φ α on V . Writing L for the joint Lipschitz constant of f and rearranging the bound

f (t α ,x α ) - α 2 t α -t 0 2 + x α -x 0 2 = φ α (t α ,x α ) ⩾ φ α (t 0 ,x 0 ) = f (t 0 ,x 0 ) reveals that t α -t 0 + x α -x 0 2 ⩽ 2 α f (t α ,x α ) -f (t 0 ,x 0 ) ⩽ 2L α t α -t 0 + x α -x 0 . It follows that t α -t 0 + x α -x 0 ⩽ 2L α ,
so the sequence (t α ,x α ) α⩾1 tends to (t 0 ,x 0 ) as α tends to infinity. In particular, for α sufficiently large, the point (t α ,x α ) is in the interior of V , and so is a local maximum of φ α as a function on

R ⩾0 × R d . It is a strict local maximum for the mapping (t,x) ↦ φ α (t,x) -t -t α 2 + x -x α 2 .
We have thus shown that for every α sufficiently large, the point (t α ,x α ) belongs to A. Recalling that (t α ,x α ) tends to (t 0 ,x 0 ) as α tends to infinity, we conclude that the set A is dense in R >0 × R d , as desired. ∎

We are finally in a position to use the Hamilton-Jacobi approach to prove Theorem 2.19 on the limit free energy in the generalized Curie-Weiss model.

Theorem 3.25. For each integer N ⩾ 1, let F N ∶ R ⩾0 × R → R denote the free en- ergy (3.2) in the generalized Curie-Weiss model. Suppose that for every h ∈ R, the limit ψ(h) ∶= lim N→+∞ F N (0,h) (3.120)
exists, and that ψ ∈ C 1 (R;R). Then, for every t ⩾ 0 and h ∈ R, we have that F N (t,h) converges to f (t,h) as N tends to infinity, where f ∶ R ⩾0 × R → R is the viscosity solution to the Hamilton-Jacobi equation (3.19) with initial condition ψ. Moreover, this function admits the Hopf representation

f (t,h) = lim N→+∞ F N (t,h) = sup m∈[-1,1] tξ (m) + hm -ψ * (m) . (3.121)
Proof. The sequence (F N ) N⩾1 of free energies is precompact by the Arzelà-Ascoli theorem and the Lipschitz bounds (3.13). We denote by f any subsequential limit, and observe that it must be jointly Lipschitz continuous by (3.13), and jointly convex by the joint convexity of each free energy F N . We may therefore use the convex selection principle in Corollary 3.24 to prove that f is a viscosity solution to the Hamilton-Jacobi equation

(3.19). Fix φ ∈ C ∞ (R >0 × R d ;R) with the property that f -φ has a strict local maximum at the point (t * ,h * ) ∈ R >0 × R. Since (F N ) N⩾1 converges
∂ t F N (t N ,x N ) -ξ ∂ h F N (t N ,x N ) ⩽ C N ∂ 2 h F N (t N ,x N ).
At the local maximum (t N ,h N ) we have

∂ t (F N -φ )(t N ,x N ) = 0, ∂ h (F N -φ )(t N ,x N ) = 0, ∂ 2 h (F N -φ )(t N ,x N ) ⩽ 0, from which it follows that ∂ t φ (t N ,x N ) -ξ ∂ h φ (t N ,x N ) ⩽ C N ∂ 2 h φ (t N ,h N ).
Letting N tend to infinity reveals that We have thus been able to recover the key result from Section 2.3 obtained using large deviation principles. It is worth mentioning that Theorem 3.25 essentially allows us to deduce a large-deviation principle as a consequence. Indeed, if we consider a smooth function ξ that is essentially zero near a point m 0 ∈ R, and essentially -∞ elsewhere, then F N (t,0) is essentially computing the probability that the mean magnetization is around m 0 , and the formula (3.121) states that this is essentially exp(-Nψ * (m 0 )).

∂ t φ -ξ (∂ h φ ) (t * ,h * ) = 0,
We have thus presented two methods, one based on large deviation principles and one based on the Hamilton-Jacobi approach, to identify the limit free energy in the generalized Curie-Weiss model. We do not know of another approach for doing this.

We close this chapter with a counterexample that shows that the differentiability assumption ψ ∈ C 1 (R d ;R) in the convex selection principle cannot be dropped in general, despite the result in Proposition 3.20.

Example 3.26. Consider the probability measure

P N ∶= 1 2 δ (1,...,1) + 1 2 δ (-1,...,-1)
and the non-linearity ξ (p) ∶= -p 2 . The generalized Curie-Weiss free energy (3.2) associated with this non-linearity and this probability measure is given by

F N (t,h) = 1 N log 1 2 e -tN+hN + 1 2 e -tN-hN .
By Exercise 1.1, we have

lim N→+∞ F N (t,h) = h -t. (3.122)
In particular, the limiting initial condition ψ(h) ∶= h is not in C 1 (R;R). On the other hand, the unique viscosity solution to the Hamilton-Jacobi equation

∂ t f + ∂ h f 2 = 0 (3.123)
with initial condition ψ is given by the Hopf formula

f (t,h) = sup m∈R -tm 2 + hm -ψ * (m) .
A direct computation shows that

ψ * (m) = sup p∈R (pm -p ) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 0 if m ⩽ 1, +∞ if m > 1.
This means that

f (t,h) = sup m ⩽1 -tm 2 + hm = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ h 2 4t if h ⩽ 2t h -t if h > 2t.
Since (t,h) ↦ h -t also satisfies the equation (3.123) at every point of differentiability and has the same initial condition, we have thus obtained a counter-example to the convex selection principle when the differentiability assumption ψ ∈ C 1 (R d ;R) is dropped. We also see that the limit free energy (3.122) is not the viscosity solution in this case. Interestingly, we can identify another reference measure P N such that the associated free energy does converge to f . Indeed, consider now the probability measure

P N ∶= 1 2 ˆ1 -1 δ (u,...,u) du.
In words, if we pick a uniform random variable U over [-1,1], then the law of the vector (U,...,U) ∈ R N is P N . In this case, and still with ξ (p) = -p 2 , we have

F N (t,h) = 1 N log 1 2 ˆ1 -1 expN -tu 2 + hu du,
which, as N tends to infinity, converges to

sup u∈[-1,1] -tu 2 + hu = f (t,h).
In particular, we see with these examples that, in situations in which the initial condition ψ is not in C 1 (R;R), it is not possible in general to identify the limit free energy from the knowledge of ψ ∶= lim N→+∞ F N (0,⋅) only.

Chapter 4 Statistical inference

So far, we have focused on models coming from statistical mechanics, and we have developed the Hamilton-Jacobi approach to compute the limit of their associated free energy. It turns out that certain problems of statistical inference share a very similar structure. In this chapter we apply the Hamilton-Jacobi approach to the problem of recovering information about a symmetric rank-one matrix given a noisy observation of it. To be more precise, we assume that, for some vector x = (x 1 ,...,x N ) with independent and identically distributed coordinates, we observe xx * +W , where the noise term W is made of independent Gaussian random variables, independent from x. Here and throughout this chapter, the superscript * denotes the transposition operator. Our goal will be to study whether it is possible to recover meaningful information about the signal xx * from this noisy observation, in the regime of large N. The answer to this question will depend on the magnitude of the signal-to-noise ratio. When Ex 1 = 0, it turns out that there exists a strictly positive and finite threshold such that if the signal-to-noise ratio is below this value, then essentially no information can be reconstructed about the signal from the observation, while if it is above this value, then there exists an estimator that has a non-trivial correlation with the signal itself (see Exercise 4.9 for a precise statement). This critical threshold is analogous to the critical inverse temperature at which phase transitions occur in the Ising and Curie-Weiss models, and we will determine it using the techniques presented in the previous chapters and progressively developed in [START_REF] Chen | Hamilton-Jacobi equations for nonsymmetric matrix inference[END_REF][START_REF] Chen | Hamilton-Jacobi equations for inference of matrix tensor products[END_REF][START_REF] Chen | Free energy of multi-layer generalized linear models[END_REF][START_REF] Chen | Statistical inference of finite-rank tensors[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for finite-rank matrix inference[END_REF][START_REF] Mourrat | Hamilton-Jacobi equations for mean-field disordered systems[END_REF]. In particular, we will appeal to the convex selection principle introduced in Section 3.6 to identify the limit free energy of this problem. We point out that, for this particular problem of statistical inference, a more standard approach similar to the proof of Proposition 3.20 would also work. We prefer to present an approach based on the convex selection principle since it generalizes to a much broader class of problems [START_REF] Chen | Statistical inference of finite-rank tensors[END_REF]. Alternative methods that so far have not reached the level of generality of [START_REF] Chen | Statistical inference of finite-rank tensors[END_REF] include [START_REF] Barbier | Mutual information for symmetric rank-one matrix estimation: a proof of the replica formula[END_REF][START_REF] Barbier | The adaptive interpolation method: a simple scheme to prove replica formulas in bayesian inference[END_REF][START_REF] Barbier | The layered structure of tensor estimation and its mutual information[END_REF][START_REF] Barbier | Information-theoretic limits of a multiview lowrank symmetric spiked matrix model[END_REF][START_REF] Deshpande | Asymptotic mutual information for the balanced binary stochastic block model[END_REF][START_REF] Deshpande | Information-theoretically optimal sparse PCA[END_REF][START_REF] Krzakala | Estimation in the spiked Wigner model: a short proof of the replica formula[END_REF][START_REF] Kadmon | Statistical mechanics of low-rank tensor decomposition[END_REF][START_REF] Lelarge | Fundamental limits of symmetric low-rank matrix estimation[END_REF][START_REF] Thibault Lesieur | Phase transitions in sparse PCA[END_REF][START_REF] Thibault Lesieur | Statistical and computational phase transitions in spiked tensor estimation[END_REF][START_REF] Luneau | Mutual information for low-rank even-order symmetric tensor estimation[END_REF][START_REF] Luneau | High-dimensional rank-one nonsymmetric matrix decomposition: the spherical case[END_REF][START_REF] Mayya | Mutual information in community detection with covariate information and correlated networks[END_REF][START_REF] Miolane | Fundamental limits of low-rank matrix estimation: the non-symmetric case[END_REF][START_REF] Reeves | Information-theoretic limits for the matrix tensor product[END_REF][START_REF] Reeves | The geometry of community detection via the mmse matrix[END_REF].

In Section 4.1, we link the problem of statistical inference with the framework developed in earlier chapters, using the language of Gibbs measures and free energies to encode the relevant information-theoretic quantities. Section 4.2 contains a general discussion on Gaussian integration by parts and concentration results. We then combine these results with the Hamilton-Jacobi approach in Section 4.3 to identify the limit free energy. The critical signal-to-noise ratio is determined by analyzing the derivatives of the limit free energy as in Section 2.4. In Section 4.4, we test the performance of the classical principal-component analysis estimator for the symmetric rank-one matrix estimation problem, by comparing its associated mean-square error with the minimal mean-square error determined in Section 4.3. Finally, in Section 4.5 we see how, from an information-theoretic perspective, the symmetric rank-one matrix estimation problem is equivalent to the problem of detecting community structures in a dense random graph for which the probability of two nodes being connected depends only on the community of each node.

From statistical inference to statistical mechanics

We give ourselves a random vector x ∶= (x 1 ,...,x N ) ∈ R N with independent coordinates sampled from a bounded probability measure P 1 on the real line. We write P N ∶= (P 1 ) ⊗N to denote the law of x. The problem of symmetric rank-one matrix estimation that we consider consists in observing

Y ∶= 2t N xx * +W ∈ R N×N , (4.1)
where W = (W i j ) 1⩽i, j⩽N ∈ R N×N is made of independent standard Gaussian random variables independent of the vector x, and we think of it as noise. We call the parameter t ⩾ 0 the signal-to-noise ratio. Our main focus will be on understanding how much information we can recover about the symmetric rank-one matrix xx * from its noisy observation Y . In particular, we will aim to determine the asymptotic behaviour, as N tends to infinity, of the minimal mean-square error,

mmse N (t) ∶= 1 N 2 inf g E xx * -g(Y ) 2 = 1 N 2 E xx * -E[xx * Y ] 2 , (4.2)
between the signal xx * and its noisy observation Y . Here, the infimum is taken over the set of measurable functions g, and for any two matrices a,b of the same size, we write a ⋅ b ∶= tr(ab * ) and a ∶= (a ⋅ a)

1 2 (4.3)
for their entry-wise scalar product and their Euclidean norm, respectively. In the L 2 sense of (4.2), the best estimator for recovering xx * is the conditional expectation of this quantity given Y . Notice that in order to compute this conditional expectation, one needs to know the laws of x and W , so we implicitly assume that these laws are known to the observer. We will also be interested in computing the mutual information,

I N (t) ∶= 1 N E ˆRN log dP x Y dP N (x) dP x Y (x), (4.4)
between the vector x and the observation Y . Here P x Y denotes the conditional law of x given Y (this is a random object that depends on the realization of Y ; in other words, it is the evaluation at Y of the mapping which sends each y to the conditional law of x given that Y = y). The quantity I N (t) is a precise measure of the amount of information obtained about the vector x when we observe Y ; see for instance [START_REF] Cover | Elements of information theory[END_REF] for more on this interpretation, and Exercise 4.3.

In order to analyze quantities such as the minimal mean-square error (4.2) or the mutual information (4.4), it will be useful to get a better understanding of the conditional law of the signal x given the observation Y . For every x ∈ R N and y ∈ R N×N , a formal calculation suggests that

P{x = x Y = y} = P{x = x and Y = y} P{Y = y} = exp -1 2 y -2t N xx * 2 dP N (x) ´RN exp -1 2 y -2t N x ′ x ′ * 2 dP N (x ′ ) . (4.5)
More precisely, denoting

H ○ N (t,x) ∶= 2t N x ⋅Y x - t N x 4 = 2t N x ⋅W x + 2t N (x ⋅ x) 2 - t N x 4 , (4.6)
we have for every bounded measurable function

f ∶ R N → R that E f (x) Y = ´RN f (x)expH ○ N (t,x)dP N (x) ´RN expH ○ N (t,x)dP N (x) . (4.7)
This is verified rigorously in Exercise 4.2. In other words, the conditional law of x given Y is the Gibbs measure associated with the Hamiltonian H ○ N (t,⋅). Notice also that the term x ⋅W x in (4.6) is highly reminiscent of the quantity (0.1) appearing in the definition of the Sherrington-Kirkpatrick spin-glass model. As in previous chapters, it will be convenient to focus first on studying the asymptotic behaviour of the free energy

F ○ N (t) ∶= 1 N log ˆRN expH ○ N (t,x)dP N (x). (4.8)
Compared with the Curie-Weiss model, one important novelty in the symmetric rankone matrix estimation problem is that the Hamiltonian H ○ N (t,⋅) is itself a random quantity, as it depends on x and W through Y . In particular, the free energy (4.8) is also random, and we denote its average by

F ○ N (t) ∶= EF ○ N (t) = 1 N Elog ˆRN expH ○ N (t,x)dP N (x). (4.9)
We will often slightly abuse terminology and refer to the average free energy (4.9) as simply the free energy. Just like in the Curie-Weiss model, we introduce notation for a random variable whose law is the random Gibbs measure (4.7). For every bounded and measurable function f ∶ R N → R, we write

⟨ f (x)⟩ ∶= ˆRN f (x)dP x Y (x) = ´RN f (x)expH ○ N (t,x)dP N (x) ´RN expH ○ N (t,x)dP N (x) . (4.10)
Although this is kept implicit in the notation, we stress that the bracket ⟨⋅⟩ is a random quantity that depends on t and on Y . Whenever we write expressions such as ⟨g(x,x)⟩, we understand that the variable x is integrated against the conditional probability measure P x Y , while keeping x fixed. In more explicit notation,

⟨g(x,x)⟩ = ˆRN g(x,x)dP x Y (x) = ´RN g(x,x)expH ○ N (t,x)dP N (x) ´RN expH ○ N (t,x)dP N (x) , (4.11)
not to be confused with E[g(x,x) Y ] for instance. If we have no information about the signal x, when t = 0, then x is simply an independent copy of x. On the other hand, if we have perfect information about x, for instance if we had instead observed x with no noise, then x would be equal to x. It will also be convenient to introduce independent copies of x under the Gibbs average ⟨⋅⟩, often called replicas, which we denote by x ′ , x ′′ , or also x 1 , x 2 , x 3 , and so on if an arbitrary number of replicas needs to be considered. Explicitly, for every bounded and measurable function

f ∶ R N × R N → R, ⟨ f (x,x ′ )⟩ = ´RN ´RN f (x,x ′ )exp H ○ N (t,x) + H ○ N (t,x ′ ) dP N (x)dP N (x ′ ) ´RN expH ○ N (t,x)dP N (x) 2 , (4.12)
with the natural generalization of this expression in the case of more replicas. Compared with the setting of spin glasses explored in Chapter 6, the fact that the Gibbs measure (4.7) is a conditional expectation will fundamentally simplify the analysis.

In the language of physics, the symmetric rank-one matrix estimation problem is always replica symmetric -this terminology will be clarified in Chapter 6. The replica symmetry will be derived from the Nishimori identity (named after [START_REF] Nishimori | Exact results and critical properties of the ising model with competing interactions[END_REF]). This identity allows us to replace one replica x by the ground-truth signal x, provided that we average over all sources of randomness.

Proposition 4.1 (Nishimori identity). For all bounded and measurable functions

f ∶ R N × R N×N → R and g ∶ R N × R N × R N×N → R, we have E⟨ f x,Y )⟩ = E⟨ f x,Y ⟩, (4.13) E⟨g x,x ′ ,Y )⟩ = E⟨g x,x,Y ⟩, (4.14)
and so on with more replicas, that is, for every integer ⩾ 1 and bounded measurable function h for some bounded measurable functions h 1 ,h 2 ,h 3 . We can then write

∶ (R N ) × R N×N → R, E⟨h x 1 ,x 2 ,...,x ,Y )⟩ = E⟨h x 1 ,x 2 ,...,x -1 ,x,Y )⟩. ( 4
E⟨h(x 1 ,...,x ,Y )⟩ = E ⟨h 1 (x 1 ,...,x -1 ,Y )⟩⟨h 2 (x )⟩h 3 (Y ) = E ⟨h 1 (x 1 ,...,x -1 ,Y )⟩E[h 2 (x) Y ]h 3 (Y ) ,
where we used that the Gibbs measure (4.10) is the conditional law of x given the observation Y in the last identity. Recalling that the measure ⟨⋅⟩ depends on the randomness only through Y , and thus that ⟨h

1 (x 1 ,...,x -1 ,Y )⟩ is Y -measurable, we obtain that E⟨h(x 1 ,...,x ,Y )⟩ = E E[⟨h 1 (x 1 ,...,x -1 ,Y )⟩h 2 (x)h 3 (Y ) Y ] = E ⟨h 1 (x 1 ,...,x -1 ,Y )⟩h 2 (x)h 3 (Y ) = E⟨h 1 (x 1 ,...,x -1 ,Y )h 2 (x)h 3 (Y )⟩.
Combining this with the definition (4.16) of the function h completes the proof. ∎

Using this identity, we can rewrite the minimal mean-square error (4.2) and the mutual information (4.4) in terms of the Gibbs average (4.10) and the free energy (4.9). Proposition 4.3. For every t ⩾ 0 and integer N ⩾ 1, the minimal mean-square error (4.2) is given by

mmse N (t) = 1 N 2 E x 4 - 1 N 2 E⟨(x ⋅ x) 2 ⟩. (4.17)
Proof. We expand the square to find that

N 2 mmse N (t) = E xx * -⟨xx * ⟩ 2 = E x 4 -2E xx * ⋅ ⟨xx * ⟩ + E ⟨xx * ⟩ ⋅ ⟨xx * ⟩ ,
and we can rewrite

⟨xx * ⟩ ⋅ ⟨xx * ⟩ = ⟨x ′ x ′ * ⋅ xx * ⟩.
The Nishimori identity (4.15) yields that

N 2 mmse N (t) = E x 4 -E⟨xx * ⋅ xx * ⟩ = E x 4 -E⟨(x ⋅ x) 2 ⟩.
This completes the proof. ∎ Proposition 4.4. For every t ⩾ 0 and integer N ⩾ 1, the mutual information (4.4) is given by

I N (t) = t N 2 E x 4 -F ○ N (t). (4.18)
Proof. Remembering that the conditional distribution of x given Y is the Gibbs measure (4.7), we can write the mutual information in terms of the Gibbs average (4.10) and the free energy (4.9) as

I N (t) = 1 N E⟨log expH ○ N (t,x) ´RN expH ○ N (t,x ′ )dP N (x ′ ) ⟩ = 1 N E⟨H ○ N (t,x)⟩ -F ○ N (t).
Invoking the Nishimori identity (4.15) reveals that

E⟨H ○ N (t,x)⟩ = EH ○ N (t,x) = 2t N Ex ⋅Y x - t N E x 4 = 2t N E x 4 - t N E x 4 = t N E x 4 .
This completes the proof. ∎

Together with the fact that the coordinates of x are independent and identically distributed, these results imply that understanding the minimal mean-square error (4.2) and the mutual information (4.4) comes down to understanding the asymptotic behaviour of the quantity E⟨(x ⋅ x) 2 ⟩ and of the free energy (4.9). Indeed, we have

1 N 2 E x 4 = E x 1 2 2 + 1 N E x 1 4 -E x 1 2 2 = E x 1 2 ) 2 + o(1). (4.19)
To understand the limit of the free energy (4.9), we will use the Hamilton-Jacobi approach. There are two issues that we need to resolve to be able to do this. The first is that we need to be able to simplify the time derivative of the free energy (4.9),

∂ t F ○ N (t) = 1 2tN 3 E⟨x ⋅W x⟩ + 2 N 2 E⟨(x ⋅ x) 2 ⟩ - 1 N 2 E⟨ x 4 ⟩, (4.20)
by integrating out the Gaussian noise W ; the second is that we need to "enrich" the free energy (4.9) so that it depends on an additional variable that allows us to close the equation. The first of these issues will be addressed through the Gaussian integration by parts formula discussed in the next section, where we will show that

∂ t F ○ N (t) = 1 N 2 E⟨(x ⋅ x) 2 ⟩. (4.21)
The second issue will be tackled in Section 4.3 by enriching the observation Y . The equality (4.21) reveals that determining the limit free energy is in fact sufficient to understand both the asymptotic mutual information and the asymptotic minimal mean-square error [START_REF] Guo | Mutual information and minimum mean-square error in Gaussian channels[END_REF].

Exercise 4.1. Let A ∈ R d×d be a symmetric rank-one matrix. Show that there exists a unit vector u ∈ R d , unique up to a sign, and a unique scalar λ ∈ R with A = λ uu * .

Exercise 4.2. Show that in the context of the symmetric rank-one matrix estimation problem, the conditional law of x given Y is the Gibbs measure (4.7) associated with the Hamiltonian (4.6).

Exercise 4.3. Fix two probability measures P and Q on a separable metric space S, and write P ≪ Q to mean that P is absolutely continuous with respect to Q. The relative entropy of P with respect to Q is defined by

H(P Q) ∶= ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ ´S dP dQ log dP dQ dQ if P ≪ Q, +∞ otherwise. (4.22)
The mutual information between two random variables X and Z on S is defined by

I(X;Z) ∶= H(P X,Z P X ⊗ P Z ), (4.23)
where P V denotes the law of a random variable V . We now place ourselves in the context of the symmetric rank-one matrix estimation problem.

(i) Prove that the mutual information (4.4) is given by

I N (t) = 1 N I(x;Y ). (4.24) (ii) Prove that H(P x,Y P x,W ) = H(P Y P W ) + I(x;Y ), (4.25)
(iii) Show that

F ○ N (t) = 1 N H(P Y P W ) and H(P x,Y P x,W ) = t N E x 4 . (4.26)
(iv) Deduce the relationship (4.18) between the mutual information (4.4) and the free energy (4.9).

Gaussian integration by parts and concentration inequalities

In this section, we temporarily move away from the symmetric rank-one matrix estimation problem to discuss two classical results about Gaussian random variables, namely integration by parts and concentration inequalities. In the context of statistical inference and spin glasses, the Gaussian integration by parts formula is very helpful to compute derivatives of the free energy, while the Gaussian concentration inequality ensures that the free energy concentrates around its average. In the next section we will see how to leverage these results to tackle the symmetric rank-one matrix estimation problem using the Hamilton-Jacobi approach. We start by presenting the Gaussian integration by parts formula. In its simplest form, let g be a centred Gaussian random variable with variance ν 2 , so with density

ϕ ν (x) ∶= 1 √ 2πν 2 exp - x 2 2ν 2 . (4.27)
For every differentiable and bounded function F ∈ C 1 (R;R), we have

EgF(g) = ˆR xF(x)ϕ ν (x)dx = -ν 2 ˆR F(x)ϕ ′ ν (x)dx, (4.28)
so integrating by parts reveals that

EgF(g) = ν 2 ˆR F ′ (x)ϕ ν (x)dx = ν 2 EF ′ (g) = Eg 2 EF ′ (g). (4.29)
To see what this formula yields in the context of statistical inference, let us fix a pair 1 ⩽ i, j ⩽ N, and denote F(W,x) ∶= ⟨x i x j ⟩. By (4.29),

E⟨W i j x i x j ⟩ = EW i j F(W,x) = E∂ W i j F(W,x). (4.30)
A direct computation shows that ∂ W i j F(W,x) is given by

∂ W i j ⎛ ⎝ ´RN x i x j exp 2t N ∑ N , ′ =1 W , ′ x x ′ dµ(x) ´RN exp 2t N ∑ N , ′ =1 W , ′ x x ′ dµ(x) ⎞ ⎠ = 2t N ⟨x 2 i x 2 j ⟩ -⟨x i x j ⟩ 2 , (4.31)
where we used the shorthand notation

dµ(x) ∶= exp 2t N (x ⋅ x) 2 - t N x 4 dP N (x). (4.32)
Summing over all i and j, we thus obtain that

E⟨x ⋅W x⟩ = 2t N E⟨ x 4 ⟩ -E⟨(x ⋅ x ′ ) 2 ⟩ , (4.33)
which combined with (4.20) and the Nishimori identity yields (4.21).

When it comes to integration by parts, the identity (4.29) is all we need for the purposes of this chapter. With a view towards subsequent chapters on spin glasses, we now present a multivariate version of this Gaussian integration by parts formula, also allowing for more general functions F; and then also another variant that is tailored to the setting of Gibbs measures. Readers who are not interested in subsequent chapters on spin glasses can safely skip to the end of the proof of Theorem 4.6, at which place we turn to discussing concentration inequalities. Theorem 4.5 (Gaussian integration by parts). Let g = (g 1 ,...,g d ) be a centred Gaussian vector in R d and F ∈ C 1 (R d ;R) be such that E g 1 F(g) + E ∇F(g) < +∞. We have

Eg 1 F(g) = d i=1 Eg 1 g i E∂ x i F(g). (4.34)
Proof. Denote by C the covariance matrix of the Gaussian vector g. The proof proceeds in three steps. First we prove that when C = I d , we have for any bounded function

G ∈ C 1 (R d ;R d ) with E ∇G(g) < +∞ that Eg ⋅ G(g) = E∇ ⋅ CG(g). (4.35)
Then we use a change of variables to deduce (4.35) for arbitrary C, and finally we prove (4.34) by choosing an appropriate G and using an approximation argument.

Step 1: proving (4.35) for C = I d . In the case when the covariance matrix C is the identity, the Gaussian vector g has density

ϕ(x) ∶= 1 (2π) d exp - 1 2 x 2 .
This implies that for any bounded function

G ∈ C 1 (R d ;R d ) with E ∇G(g) < +∞, we have Eg ⋅ G(g) = ˆRd x ⋅ G(x)ϕ(x)dx = - ˆRd ∇ϕ(x) ⋅ G(x)dx,
where we used that ∇ϕ(x) = -xϕ(x). Integrating by parts reveals that

Eg ⋅ G(g) = ˆRd ϕ(x)∇ ⋅ G(x)dx = E∇ ⋅ G(g).
The boundedness of G ensures that the boundary term in this integration by parts vanishes. This establishes (4.35) when C = I d .

Step 2: deducing (4.35) for general C. Letting z be a standard Gaussian random vector in R d and g ∶= C 1 2 z, the vector g is centred Gaussian with covariance C. It follows by symmetry of C 1 2 and the previous step that

Eg ⋅ G(g) = Ez ⋅C 1 2 G(C 1 2 z) = Ez ⋅ G(z) for the function G(z) ∶= C 1 2 G(C 1 2 z). A direct computation reveals that ∇ ⋅ G(z) = d i, j,k=1 C 1 2 i j C 1 2 ki ∂ x k G j (g) = d j=1 ∂ x j (CG) j (g) = ∇ ⋅ CG(g)
which gives (4.35) for general C.

Step 3: obtaining (4.34). Applying (4.35) to the function G(g) ∶= F(g)e [START_REF] Abbe | Community detection and stochastic block models: recent developments[END_REF] , where e 1 ∈ R d denotes the first canonical basis vector, and observing that (CG) i = C i,1 F(g) for 1 ⩽ i ⩽ d establishes (4.34) for F bounded. To lift this assumption, given ε > 0, define the function

φ ε ∶ R → R by φ ε (x) ∶= x √ 1 + εx 2 .
Observe that φ ε is bounded by ε -1 2 , so φ ε ○ F is differentiable, bounded, and

satisfies E ∇(φ ε ○ F) < +∞ since ∇φ ε ∞ ⩽ 1. It follows that Eg 1 φ ε (F(g)) = d i=1 Eg 1 g i Eφ ′ ε (F(g))∂ x i F(g) = d i=1 Eg 1 g i E ∂ x i F(g) (1 + εF(g) 2 ) 3 2 .
Invoking the dominated convergence theorem to let ε tend to zero completes the proof. ∎

As is suggested by the example in (4.33), the Gaussian integration by parts formula admits a convenient reformulation that is adapted to the context of Gibbs measures. With a view towards using it for spin glasses in subsequent chapters, we now state and prove it in a general context. We reiterate that the reader only interested in statistical inference models can safely skip this generalization and directly move to right after the proof of Theorem 4.6.

Consider a centred Gaussian process made of the two families (y(σ )) σ ∈Σ and (z(σ )) σ ∈Σ indexed by some complete and separable metric space of indices Σ, and let µ be a finite measure on Σ. Suppose that the covariances of (y(σ )) σ ∈Σ and (z(σ )) σ ∈Σ are given by

C y (σ 1 ,σ 2 ) ∶= Ey(σ 1 )y(σ 2 ) and C z (σ 1 ,σ 2 ) ∶= Ez(σ 1 )z(σ 2 ) (4.36)
for some bounded and continuous functions

C y ∶ Σ 2 → R and C z ∶ Σ 2 → R. The process (y(σ )) σ ∈Σ defines the Gibbs measure dG(σ ) ∶= expy(σ )dµ(σ ) ´Σ expy(τ)dµ(τ) (4.37)
on Σ, and we denote by ⟨⋅⟩ the Gibbs average associated with the Gibbs measure G, with canonical random variable σ . We also let (σ ) ⩾1 be independent copies of σ under ⟨⋅⟩. This means that for any integer ⩾ 1 and any bounded and measurable

function f ∶ Σ → R, ⟨ f (σ 1 ...,σ )⟩ ∶= ´Σ f (σ 1 ,...,σ )∏ i=1 expy(σ i )dµ(σ i ) ´Σ expy(σ )dµ(σ ) . (4.38)
The regularity assumptions on the covariance functions (4.36) ensure that the integral defining the Gibbs average is well-defined. Indeed, the continuity of C y can be used to show that the process (expy(σ )) σ ∈Σ is continuous in mean and therefore in probability. In the context of the symmetric rank-one matrix estimation problem, we would have

Σ ∶= R N , y(x) ∶= 2t N x ⋅W x, and dµ(x) = exp 2t N (x ⋅ x) 2 - t N x 4 dP N (x), (4.39)
and for the calculation leading to (4.33), we would choose z(x) ∶= x ⋅W x. In general, the Gaussian integration by parts formula in Theorem 4.5 can be used to compute Gibbs averages of Gaussian processes as follows.

Theorem 4.6 (Gibbs Gaussian integration by parts). Let C(σ 1 ,σ 2 ) ∶= Ez(σ 1 )y(σ 2 ) denote the covariance between the Gaussian processes (z(σ )) σ ∈Σ and (y(σ )) σ ∈Σ . For every n ⩾ 1 and bounded measurable Φ ∶ Σ n → R, we have

E⟨Φ(σ 1 ,...,σ n )z(σ 1 )⟩ = E⟨Φ(σ 1 ,...,σ n ) n i=1 C σ 1 ,σ i -nC σ 1 ,σ n+1 ⟩. (4.40)
Proof. The definition of the Gibbs average (4.38) implies that

E⟨Φ(σ 1 ,...,σ n )z(σ 1 )⟩ = ˆΣn Φ(σ 1 ,...,σ n )Ez(σ 1 ) n =1 expy(σ ) ´Σ expy(τ)dµ(τ) dµ(σ ). (4.41)
To simplify this expression, we fix σ 1 ,...,σ n ∈ Σ and introduce the Gaussian process (y ′ (σ )) σ ∈Σ defined by

y ′ (σ ) ∶= y(σ ) -λ σ 1 ,σ z(σ 1 ) for λ σ 1 ,σ ∶= Ey(σ )z(σ 1 ) Ez(σ 1 ) 2 .
Since Ey ′ (σ )z(σ 1 ) = 0 and y and z are jointly Gaussian, the process (y ′ (σ )) σ ∈Σ is independent of z(σ 1 ). If we condition on (y ′ (σ )) σ ∈Σ and define the function

F ∶ R → R by F(x) ∶= n =1 exp y ′ (σ ) + λ σ 1 ,σ x ´Σ exp y ′ (τ) + λ σ 1 ,τ x dµ(τ) ,
then the Gaussian integration by parts formula in Theorem 4.5 implies that

Ez(σ 1 ) n =1 expy(σ ) ´Σ expy(η)dµ(η) = Ez(σ 1 ) 2 EF ′ (z(σ 1 )).
A direct computation shows that

F ′ (x) = n =1 λ σ 1 ,σ F(x) -nF(x) ´Σ λ σ 1 ,σ n+1 exp(y ′ (σ n+1 ) + λ σ 1 ,σ n+1 x)dµ(σ n+1 )
´Σ exp(y ′ (τ) + λ σ 1 ,τ x)dµ(τ) .

It follows that conditionally on (y ′ (σ )) σ ∈Σ as well as the replicas σ 1 ,...,σ n ∈ Σ,

Ez(σ 1 ) n =1 expy(σ ) ´Σ expy(τ)dµ(τ) = n =1 C σ 1 ,σ EF(z(σ 1 )) -nEF(z(σ 1 ))⟨C σ 1 ,σ n+1 ⟩ ′ ,
where ⟨⋅⟩ ′ denotes the Gibbs average (4.38) conditionally on σ 1 ,...,σ n , and therefore with the average only taken with respect to the randomness of σ n+1 . Since (y ′ (σ )) σ ∈Σ is independent of z(σ 1 ), this equality also holds unconditionally on the randomness of (y ′ (σ )) σ ∈Σ . Substituting it into the right side of (4.41) yields

E ˆΣn Φ(σ 1 ,...,σ n ) n =1 C σ 1 ,σ -n⟨C σ 1 ,σ n+1 ⟩ ′ n =1 expy(σ ) ´Σ expy(τ)dµ(τ) dµ(σ ).
Remembering that the Gibbs average (4.38) is a product measure allows us to absorb the Gibbs average ⟨⋅⟩ ′ into the Gibbs average ⟨⋅⟩ to obtain (4.40). This completes the proof. ∎

We now turn to the Gaussian concentration inequality. This inequality states that a Lipschitz function of a standard Gaussian random vector will concentrate around its mean, with a standard deviation of the order of the Lipschitz constant. To motivate this statement, let us take g a centred Gaussian random variable with variance ν 2 , and observe that by Chebyshev's inequality, we have for every t,λ > 0 that

P{ g > t} ⩽ 2P{g > t} ⩽ 2exp(-λt)Eexp(λ g) = 2exp λ 2 ν 2 2 -λt . (4.42)
Optimizing over λ to find λ = t ν 2 reveals that

P{ g > t} ⩽ 2exp - t 2 2ν 2 . (4.43)
Notice that ν is the Lipschitz constant of the function F ν (g) ∶= νg which maps a standard Gaussian random variable to a centred Gaussian random variable with variance ν 2 . The elementary concentration inequality (4.43) can therefore be written in terms of a standard Gaussian random variable g as

P F ν (g) -EF ν (g) > t ⩽ 2exp - t 2 2 F ν 2 Lip , (4.44)
where the Lipschitz semi-norm ⋅ Lip is defined in (3.38). It turns out that this formula generalizes to all Lipschitz continuous functions of a Gaussian random vector.

Theorem 4.7 (Gaussian concentration inequality). If g is a standard Gaussian vector in R d and F ∶ R d → R is a Lipschitz continuous function, then for any t ⩾ 0,

P{ F(g) -EF(g) ⩾ t} ⩽ 2exp - t 2 2 F 2 Lip . (4.45)
Proof. Replacing F by F -EF(g) if necessary and using the union bound, it suffices to show that for every Lipschitz continuous function F ∶ R d → R with EF(g) = 0 and every t ⩾ 0, we have

P{F(g) ⩾ t} ⩽ exp - t 2 2 F 2 Lip . (4.46)
The proof proceeds in two steps. First we prove (4.46) when F is continuously differentiable, and then we use an approximation argument to establish it in general.

Step 1: F continuously differentiable. By Chebyshev's inequality, we have for every λ > 0 that P{F(g) ⩾ t} ⩽ exp(-λt)Eexp(λ F(g)).

(4.47)

To bound the function h(λ ) ∶= Eexp(λ F(g)), we use a symmetrization trick. Let g be an independent copy of g, and for each s

∈ [0,1], introduce the Gaussian random variable Z s ∶= √ sg+ √ 1 -sg. Define the function G λ (x) ∶= exp(λ F(x))
, and observe that by the fundamental theorem of calculus,

h ′ (λ ) = EF(g)G λ (g) = ˆ1 0 d ds EF(g)G λ (Z s )ds.
The independence of g and g as well as the assumption EF(g) = 0 have played their part. This implies that

h ′ (λ ) = ˆ1 0 EF(g)∇G λ (Z s ) ⋅ g 2 √ s - g 2 √ 1 -s ds.
For each 1 ⩽ i ⩽ d, the Gaussian integration by parts formula reveals that

Eg i F(g)∂ x i G(Z s ) = E∂ x i F(g)∂ x i G λ (Z s ) = λ E ∂ x i F(g) 2 G λ (Z s ) + √ sEF(g)∂ 2 x i G λ (Z s ),
while

Eg i F(g)∂ x i G λ (Z s ) = √ 1 -sEF(g)∂ 2 x i G λ (Z s ).
It follows by the Cauchy-Schwarz inequality and the fact that Z s is equal in distribution to g for every s ∈ [0,1] that

h ′ (λ ) ⩽ 1 2 λ F 2 Lip EG λ (g) ˆ1 0 1 √ s ds ⩽ λ F 2 Lip h(λ ).
Dividing both sides of this equation by h(λ ) and integrating the resulting expression shows that

logh(λ ) ⩽ logh(0) + λ 2 F 2 Lip 2 = λ 2 F 2 Lip 2 .
Substituting this into (4.47) and optimizing over λ to find λ = t Step 2: F general. For each ε > 0, we introduce the function F ε (x) ∶= EF(x + ε g), where g is an independent copy of g. Since the random vector x + ε g is standard Gaussian, the function F ε is smooth. Moreover, it is such that F ε Lip ⩽ F Lip . It follows by (4.46) applied to F ε that for every integer n ⩾ 1,

P F ε (g) > t + n -1 ⩽ exp - (t + n -1 ) 2 2 F 2 Lip .
Since (F ε ) ε>0 converges to F uniformly, the sequence (F ε (g)) ε>0 of random variables converges in law to F(g). Invoking the Portmanteau theorem (Theorem A.17) reveals that

P F(g) > t + n -1 ⩽ liminf ε→0 P F ε (g) > t + n -1 ⩽ exp - (t + n -1 ) 2 2 F 2 Lip ,
and using the continuity of measure to let n tend to infinity completes the proof. ∎

The Gaussian concentration inequality has many important consequences in the context of statistical mechanics. For instance, as shown in Exercise 4.4, it gives the concentration of the maximum of the coordinates of a Gaussian vector around its average. As we will see in Exercise 6.2 of Chapter 6, this implies that the maximum of the Hamiltonian in the Sherrington-Kirkpatrick model concentrates around its average. In the language of physics, one could say that the ground-state energy is self-averaging. Another quantity that is generally self-averaging is the free energy. In the context of the symmetric rank-one matrix estimation problem, we can think of the free energy (4.8) as a function of the Gaussian noise W and the signal x. To make this dependence explicit, let us temporarily change notation and write the free energy as

F ○ N (t,W,x) ∶= F ○ N (t). (4.48)
Letting a > 0 be such that the support of the measure P 1 is contained in the interval [-√ a, √ a], we see that the measure P N is supported in the ball B √ aN (0) of radius √ aN centred at the origin. We thus deduce that for t ⩾ 0,

x ∈ R N and W 1 ,W 2 ∈ R N×N , F ○ N (t,W 1 ,x) ⩽ 2t N 3 sup x∈B √ aN (0) x ⋅W 1 x -x ⋅W 2 x + F ○ N (t,W 2 ,x). (4.49) Using that x ⋅ (W 1 -W 2 )x = (W 1 -W 2 ) ⋅ (x * x
) and the Cauchy-Schwarz inequality, we obtain that

F ○ N (t,W 1 ,x) -F ○ N (t,W 2 ,x) ⩽ 2t N 3 sup x∈B √ aN (0) x 2 W 1 -W 2 ⩽ a 2t N W 1 -W 2 .
(4.50) Since the right side of (4.50) is symmetric in the pair (W 1 ,W 2 ), this estimate gives us an upper bound on the Lipschitz semi-norm of the map W ↦ F ○ N (t,W,x). It thus follows by the Gaussian concentration inequality applied conditionally on the randomness of the signal x that, for any λ > 0,

P F ○ N (t,W,x) -E W F ○ N (t,W,x) ⩾ λ ⩽ 2exp - Nλ 2 4a 2 t , (4.51)
where E W denotes the average only with respect to the randomness of the Gaussian noise W . To obtain the full concentration of the free energy, it remains to establish the concentration of E W F ○ N (t,W,x) about its average with respect to the signal x. This can be done using the McDiarmid inequality.

Theorem 4.8 (McDiarmid inequality). Let X = (X 1 ,...,X n ) be a vector of independent random variables taking values in the measurable space S 1 × ⋯ × S n , let c 1 ,...,c n ⩾ 0, and let F ∶ S 1 × ⋯ × S n → R be a measurable function such that, for every i ∈ {1,...,n}, x 1 ∈ S 1 ,...,x n ∈ S n , and

x ′ i ∈ S i , F(x 1 ,...,x i-1 ,x ′ i ,x i+1 ,...,x n ) -F(x 1 ,...,x i-1 ,x i ,x i+1 ,...,x n ) ⩽ c i . (4.52)
For every t ⩾ 0, we have

P F(X) -EF(X) ⩾ t ⩽ 2exp - 2t 2 ∑ n i=1 c 2 i . (4.53)
Proof. We decompose the proof into two steps.

Step 1: Laplace transform estimate. Let a < b ∈ R. In this step, we show that if X is a random variable taking values in the interval

[a,b], then logEexpλ (X -EX) ⩽ λ 2 (b -a) 2 8 . (4.54)
As a preliminary observation, any random variable X taking values in the interval [a,b] must be such that

E(X -EX) 2 = inf c∈R E(X -c) 2 ⩽ E X - a + b 2 2 ⩽ b -a 2 2 .
We denote by ψ(λ ) the left side of (4.54). Since X is bounded, the function ψ is infinitely differentiable, and a direct calculation gives that

ψ ′ (λ ) = ⟨X -EX⟩ and ψ ′′ (λ ) = ⟨(X -EX) 2 ⟩ -⟨X -EX⟩ 2 ,
where we introduced the notation

⟨Z⟩ ∶= E Z expλ (X -EX) Eexpλ (X -EX)
to denote the Gibbs measure associated with the Hamiltonian λ (X -EX). Although this is kept implicit in the notation, we point out that this Gibbs measure depends on λ ; in particular, the quantities ⟨X⟩ and EX are different in general. We see that ψ ′′ is expressed as the variance of a random variable that takes values in an interval of length ba. By the preliminary observation, we deduce that ψ ′′ is uniformly bounded by (ba) 2 4. Since ψ(0) = ψ ′ (0) = 0, integrating this bound yields (4.54).

Step 2: Chebyshev inequality. We now take up the notation and assumptions of the theorem, and introduce, for every i ∈ {1,...,n}, the difference

∆ i ∶= E f (X) X 1 ,...,X i -E f (X) X 1 ,...,X i-1 ,
with the understanding that

∆ 1 = E[ f (X) X 1 ] -E f (X). We clearly have f (X) -E f (X) = n i=1 ∆ i .
For every λ ⩾ 0, we now look for an upper bound on

Eexp λ n i=1 ∆ i = E n i=1 expλ ∆ i .
Conditionally on X 1 ,...,X n-1 , the random variable ∆ n is centred, and by (4.52), it takes values in an interval of length c n . Since the random variables ∆ 1 ,...,∆ n-1 are measurable with respect to X 1 ,...,X n-1 , we can apply the result of the previous step conditionally on these variables, so that

Eexp λ n i=1 ∆ i ⩽ Eexp λ n-1 i=1 ∆ i exp λ 2 c 2 n 8 .
Continuing inductively from i = n -1 down to i = 1, we conclude that

Eexp λ n i=1 ∆ i ⩽ exp λ 2 8 n i=1 c 2 i .
By Chebyshev's inequality, for every t ⩾ 0 and λ ⩾ 0, we have

P{F(X) -EF(X) ⩾ t} ⩽ exp -λt + λ 2 8 n i=1 c 2 i .
Optimizing over λ yields that

P{F(X) -EF(X) ⩾ t} ⩽ exp - 2t 2 ∑ n i=1 c 2 i .
By symmetry and the union bound, we thus obtain (4.53). This completes the proof. ∎

To control the deviations of E W F ○ N (t,W,x) about its average using the McDiarmid inequality, fix t ⩾ 0,W ∈ R N×N and x ∈ R N . Recalling (4.6), a direct computation reveals that for any 1 ⩽ i ⩽ N,

∂ x i F ○ N (t,W,x) = 4t N 2 ⟨x i (x ⋅ x)⟩. (4.55)
Letting a > 0 be such that the support of the measure

P 1 is contained in the interval [- √ a, √ a], this implies that ∂ x i F ○ N (t,W,x) ⩽ 4ta 3 2 N . (4.56)
Averaging over the randomness of the Gaussian noise W and using Jensen's inequality shows that this upper bound also holds for the averaged free energy

E W F ○ N (t,W,x).
Combining this with the mean value theorem, we find that for any i ∈ {1,...,N} and

x 1 ,...,x N ,x ′ i ∈ [- √ a, √ a], E W F ○ N (t,W,x 1 ,...,x i-1 ,x ′ i ,x i+1 ,...,x n ) -E W F ○ N (x 1 ,...,x i-1 ,x i ,x i+1 ,x n ) ⩽ 4ta 3 2 N x i -x ′ i = 8ta 2 N . (4.57)
It follows by the McDiarmid inequality that, for every λ ⩾ 0,

P E W F ○ N (t,W,x) -EF ○ N (t,W,x) ⩾ λ ⩽ 2exp - Nλ 2 32t 2 a 4 . (4.58)
Together with the triangle inequality and the upper bound (4.51), we find, for every T < +∞, a constant C < +∞ such that for every t ∈ [0,T ] and λ ⩾ 0,

P F ○ N (t) -F ○ N (t) ⩾ λ ⩽ 2exp - Nλ 2 C . (4.59)
This establishes the exponential concentration of the free energy about its average. There are many other concentration inequalities for Gaussian and non-Gaussian random variables which are often used in statistical mechanics. For instance, alternatively to the McDiarmid inequality, the Efron-Stein inequality discussed in Exercise 4.7 can be used to establish the concentration of the free energy in non-Gaussian contexts, such as in Appendix B of [START_REF] Dominguez | Mutual information for the sparse stochastic block model[END_REF] for community detection on sparse graphs. Exercise 4.8 presents the Gaussian Poincaré inequality, which can be used to obtain concentration results in L 2 under a weaker assumption than Lipschitz continuity of the functional. For much more on concentration inequalities we refer the interested reader to [START_REF] Boucheron | Concentration inequalities: a nonasymptotic theory of independence[END_REF].

Exercise 4.4. Let a ∈ R ⩾0 and let (g 1 ,...,g d ) be a d-dimensional centred Gaussian vector with Eg 2 i ⩽ a for every i ∈ {1,...,d}. Show that for every t ⩾ 0,

P max 1⩽i⩽d g i -E max 1⩽i⩽d g i ⩾ t ⩽ 2exp - t 2 2a . (4.60)
Exercise 4.5. For W = (W i j ) 1⩽i, j⩽N ∈ R N×N a matrix of independent standard Gaussian random variables, we introduce the norm W * ∶= sup x ⩽1 W x . The purpose of this exercise is to show that there exists a constant C < +∞ such that for every a ⩾ C,

P W 2 * ⩾ aN ⩽ exp - aN C . (4.61)
(i) Show that there exists a constant C < +∞ such that for every x ∈ R N of unit norm and all a ⩾ 0,

P W x 2 ⩾ aN ⩽ exp C - a C N . (4.62) (ii) Consider a set A ⊆ R N with the property that for every x ∈ R N with x ⩽ 1, there exists y ∈ A such that x -y ⩽ 1 2 . Show that W * ⩽ 2sup y∈A Wy . (4.63)
(iii) Show that one can find such a set A of size exponential in N, and conclude that (4.61) holds.

(iv) Deduce that for every q ⩾ 1, there exists a constant C < +∞ with

E W q * ⩽ CN q 2 . (4.64)
Exercise 4.6 (Approximate Gaussian integration by parts). Let X be a centred random variable with finite third moment E X 3 < +∞. Show that for every

F ∈ C 2 (R;R) with F ′′ ∞ < +∞, EXF(X) -EX 2 EF ′ (X) ⩽ 3 2 F ′′ ∞ E X 3 . (4.65)
Exercise 4.7 (Efron-Stein). Let X ∈ R d be a vector of independent random variables, and let X ′ ∈ R d be an independent copy of X. Show that for every bounded and measurable

f ∶ R d → R, Var f (X) ⩽ 1 2 d i=1 E f (X) -f (X 1 ,...,X i-1 ,X ′ i ,X i+1 ,...,X N ) 2 . ( 4
(Z) ⩽ E f ′ (Z) 2 .

A Hamilton-Jacobi approach to rank-one matrix estimation

To determine the limit free energy in the symmetric rank-one matrix estimation problem using the Hamilton-Jacobi approach, we need to find a partial differential equation satisfied by the free energy (4.9) up to an error that vanishes with N. Notice that the limits of the random free energy (4.8) and its average (4.9), provided that they exist, are the same. Indeed, by the free energy concentration inequality (4.59), there exists a constant C < +∞ that depends only on an upper bound on t and on the support of P 1 such that for every λ > 0,

P F ○ N (t) -F ○ N (t) ⩾ λ ⩽ 2exp - Nλ 2 C . (4.67)
Since the right side of this expression is summable, the Borel-Cantelli lemma implies that almost surely,

limsup N→+∞ F ○ N (t) -F ○ N (t) = 0. (4.68)
Using the Gaussian integration by parts formula, we saw in (4.33) that

E⟨x ⋅W x⟩ = 2t N E⟨ x 4 ⟩ -E⟨(x ⋅ x ′ ) 2 ⟩ . (4.69)
Together with the derivative computation in (4.20) this implies that

∂ t F ○ N (t) = 1 N 2 E⟨(x ⋅ x) 2 ⟩. (4.70)
Unfortunately, there is no way of closing this equation if we can only compute derivatives in t of the free energy. Indeed, the situation is analogous to that encountered if we were studying a Curie-Weiss model without any magnetization part, where there would be no parameter h with respect to which to differentiate and close the equation. To overcome this issue, we will define an "enriched" free energy that also depends on an additional parameter h. The enriched free energy F N (t,h) should extend the free energy F ○ N (t), in the sense that there is some h 0 with F N (⋅,h 0 ) = F ○ N (⋅); it should be simple enough that we can explicitly compute the limit of its initial condition F N (0,h); and it should be rich enough that it allows us to close the equation, up to a small error term. In the context of statistical inference models, we also want to ensure that the enrichment does not destroy the fact that the Gibbs measure is a conditional expectation. Indeed, this property gives us access to the Nishimori identity, which plays a fundamental role in simplifying statistical inference models and distinguishing them from the more complicated spin glass models we will discuss in Chapter 6.

In the context of the symmetric rank-one matrix estimation problem, the appropriate enrichment of the free energy is obtained by assuming that, in addition to observing the noisy rank-one matrix Y in (4.1), we also observe a noisy version Ỹ of the signal vector x,

Ỹ ∶= √ 2hx + z. (4.71)
The noise vector z = (z i ) i⩽N ∈ R N is made of independent standard Gaussian random variables independent of the vector x and the noise matrix W , and the parameter h ⩾ 0 is a signal-to-noise ratio. The enriched symmetric rank-one matrix estimation problem is to infer the rank-one matrix xx * from the observation of Y ∶= (Y, Ỹ ).

Applying Bayes' formula as in Exercise 4.2 shows that the law of the signal x given the observation of Y is the Gibbs measure whose Hamiltonian on R N is

H N (t,h,x) ∶= H ○ N (t,x) + √ 2h Ỹ ⋅ x -h x 2 . (4.72)
In other words, for any bounded and measurable function f ∶ R N → R, we have

E f (x) Y = ´RN f (x)expH N (t,h,x)dP N (x) ´RN expH N (t,h,x)dP N (x) . (4.73)
The free energy

F N (t,h) ∶= 1 N log ˆRN expH N (t,h,x)dP N (x) (4.74)
of this model is again random, as it depends on x, W and z, and we denote its average by

F N (t,h) ∶= EF N (t,h) = 1 N Elog ˆRN expH N (t,h,x)dP N (x). (4.75)
We will again often refer to the average free energy (4.75) as simply the free energy. Through a slight abuse of notation, we will, as before, write ⟨⋅⟩ for the average with respect to the Gibbs measure (4.73), and write x,x ′ ,x ′′ , and so on, for independent random variables sampled according to this probability measure. That is, for every bounded and measurable function f ∶ R N → R, we write

⟨ f (x)⟩ ∶= ´RN f (x)expH N (t,h,x)dP N (x) ´RN expH N (t,h,x)dP N (x) , (4.76)
and so on as in (4.12) with more replicas. The derivation of the identity (4.70) is unchanged for this more general Gibbs measure: we have for all t,h ⩾ 0 that

∂ t F N (t,h) = 1 N 2 E⟨(x ⋅ x) 2 ⟩. (4.77)
Recalling from (4.71) and (4.72) that

H N (t,h,x) = H ○ N (t,x) + 2hx ⋅ x + √ 2hz ⋅ x -h x 2 , (4.78)
we next observe that for all t,h ⩾ 0,

∂ h F N (t,h) = 1 N √ 2h E⟨z ⋅ x⟩ + 2 N E⟨x ⋅ x⟩ - 1 N E⟨ x 2 ⟩. (4.79)
To integrate out the Gaussian noise z, we fix 1 ⩽ i ⩽ N, we write F(z,W,x) ∶= ⟨x i ⟩, and we observe that by the Gaussian integration by parts formula in (4.29),

E⟨z i x i ⟩ = Ez i F(z,W,x) = E∂ z i F(z,W,x) = √ 2h E⟨x 2 i ⟩ -E⟨x i ⟩ 2 . (4.80) It follows that E⟨z ⋅ x⟩ = √ 2h E⟨ x 2 ⟩ -E⟨x ⋅ x ′ ⟩ , (4.81)
which together with the Nishimori identity implies that

∂ h F N (t,h) = 1 N E⟨x ⋅ x⟩. (4.82)
It is reasonable to expect the variance of the inner product, or overlap, N -1 x ⋅ x between a sample x from the Gibbs measure (4.73) and the ground-truth signal x to be small, simply because it is the average of a large number of variables. If this is so, then the difference between the time derivative (4.77) and the square of the spatial derivative (4.82) would also be small, since

∂ t F N (t,h) -∂ h F N (t,h) 2 = Var x ⋅ x N . (4.83)
This suggests that the enriched free energy (4.75) might converge to the function f solving the Hamilton-Jacobi equation

∂ t f -∂ h f 2 = 0 on R >0 × R >0 (4.84)
with the initial condition

ψ(h) ∶= lim N→+∞ F N (0,h) = F 1 (0,h). (4.85)
We have used the assumption that P N is a product measure to assert that the initial condition is independent of N. Notice that the Hamilton-Jacobi equation (4.84) is the same as the Hamilton-Jacobi equation (3.9) that appeared in the context of the Curie-Weiss model, with the sole difference that it is posed on

R >0 × R >0 as opposed to R >0 × R.
Because of this difference, the Hamilton-Jacobi equation (4.84) that we would like to use to describe the limit free energy in the symmetric rank-one matrix estimation problem does not exactly fall within the scope of the Hamilton-Jacobi equations studied in Chapter 3. Indeed, now the spatial variable is restricted to the space R >0 , which unlike R is a domain with a boundary. The right way forward would be to extend the analysis in Chapter 3 to equations posed on more general domains. How to proceed along these lines and conclude for the identification of the limit free energy in the symmetric rank-one matrix estimation problem is explained in details in [START_REF] Dominguez | A Hamilton-Jacobi approach to the stochastic block model[END_REF], in a setting that closely matches our current one. This approach is also taken up in [START_REF] Chen | Hamilton-Jacobi equations with monotone nonlinearities on convex cones[END_REF][START_REF] Chen | Statistical inference of finite-rank tensors[END_REF] in great generality in finite dimension, in [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF][START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF] for the infinite-dimensional equations arising in Chapter 6 in relation with spin glasses, and in [START_REF] Dominguez | Infinite-dimensional Hamilton-Jacobi equations for statistical inference on sparse graphs[END_REF] for community detection on sparse graphs. The upshot of this analysis is that, for all equations of relevance to us, we do not have to prescribe any boundary condition for the solution, and in effect, we can proceed by simply ignoring the boundary. Intuitively, this is possible because the characteristic lines discussed below (3.90) always go towards the boundary rather than away from it.

Instead of explaining this, we prefer to use a simpler but less general workaround here, by taking advantage of the simple geometry of the domain R ⩾0 through a symmetrization trick. Although this will be convenient in our context, the reader should not surmise that this is a standard way to handle equations posed on domains with boundaries, and there are indeed more complicated models of statistical inference or spin glasses, where R ⩾0 is replaced by the space of positive semi-definite matrices, for which this trick does noes seem to be applicable.

So we will content ourselves with saying that a function f ∶ R ⩾0 × R ⩾0 → R is a viscosity solution to the Hamilton-Jacobi equation (4.84) with initial condition

ψ if its symmetrization f ∶ R ⩾0 × R → R defined by f (t,h) ∶= f (t, h ) (4.86)
is a viscosity solution the Hamilton-Jacobi equation

∂ t f -∂ h f 2 = 0 on R >0 × R (4.87)
subject to the symmetrized initial condition ψ(h) ∶= ψ( h ). As will be seen, the main reason why this approach makes sense is that the functions we care about are non-decreasing in the variable h, when this variable varies in R ⩾0 . If the function ψ was decreasing, then the symmetrization would instead create an unphysical singularity at the origin h = 0 -by this we mean that we would be able to contradict the definition of viscosity solution for (4.87) by using any point in R ⩾0 × {0} as a contact point.

Our main result will be that the limit free energy (4.75) is indeed a solution to the Hamilton-Jacobi equation (4.84). Together with the Hopf-Lax formula in Theorem 3.8 and the observation that the free energy (4.9) can be recovered from the enriched free energy (4.75),

F ○ N (t) = F N (t,0), (4.88)
this will give us the following result for the limit free energy in the symmetric rank-one matrix estimation problem.

Theorem 4.9 (Identification of limit free energy). For every N ⩾ 1, we denote by

F N ∶ R ⩾0 ×R ⩾0 → R
ψ(h) ∶= F 1 (0,h) = Elog ˆR exp √ 2hxz 1 + 2hxx 1 -hx 2 dP 1 (x).
(4.89)

The limit free energy f admits the Hopf-Lax representation, for every t,h ⩾ 0,

f (t,h) = sup h ′ ⩾0 ψ(h + h ′ ) - (h ′ ) 2 4t . (4.90)
In particular, the limit of the free energy F ○ N defined in (4.9) is given by

lim N→+∞ F ○ N (t) = sup h⩾0 ψ(h) - h 2 4t . (4.91)
This result will be proved by applying the convex selection principle to the symmetrization f of any subsequential limit f of the sequence (F N ) N⩾1 of enriched free energies. To begin with, we will verify that the enriched free energy F N is jointly convex, Lipschitz continuous and non-decreasing, and that the first two of these three properties are inherited by f . We say that a function

g ∶ R ⩾0 × R ⩾0 → R is non-decreasing if, for all t,h,h ′ ⩾ 0, we have h ⩽ h ′ ⇒ g(t,h) ⩽ g(t,h ′ ).
(4.92)

We will then show that each free energy F N satisfies the Hamilton-Jacobi equation (4.84) up to some explicit error term, and use this to deduce that any smooth function that touches f from above must satisfy the Hamilton-Jacobi equation (4.84) at the contact point. This will then allow us to conclude that f is the unique viscosity solution to the Hamilton-Jacobi equation (4.87) using the convex selection principle in Lemma 3.23. We will rely on this more general version of the convex selection principle, as opposed to the cleaner form in Theorem 3.21, because the symmetrized initial condition ψ is not necessarily continuously differentiable at the origin. As can easily be seen, e.g. from (S.27), the symmetrized initial condition ψ does belong to C 1 (R;R) when Ex 1 = 0, and the reader may consider working out a simpler proof based on Theorem 3.21 under this additional assumption.

We start as announced by showing that the enriched free energy F N is jointly convex, non-decreasing in the sense of (4.92), and Lipschitz continuous uniformly over N.

Proposition 4.10. The enriched free energy (4.75) in the symmetric rank-one matrix estimation problem is jointly convex, non-decreasing, and Lipschitz continuous uniformly over N.

The proof that the free energy F N is convex involves a somewhat lengthy calculation. This may come as a surprise, since for the simpler models studied in previous chapters, the convexity of the free energy was a consequence of the very general observation presented in Exercise 2.6 that the log-Laplace transform of a random variable is convex. This argument cannot be applied here however. One may wonder whether this comes from the fact that we placed some square roots acting on the parameters t and h in the definition of the problem; and in the context of spin glasses discussed in Chapter 6, it will be immediate to see that the function FN ∶ (t,h) ↦ F N (t 2 ,h 2 ) is indeed convex. So are we making the question uselessly complicated here? The main reason we want to insist on showing the convexity property is that this is a requirement for the validity of the convex selection principle. The proof of this result relies on the fact that the underlying Hamilton-Jacobi equation does not explicitly depend on the parameters t and h. If we were to write down a Hamilton-Jacobi equation for FN instead, then the parameters t and h would have to be explicitly present in the equation; and it turns out that the convex selection principle is actually false in this more general setting.

In the context of statistical inference, there is however a fundamental informationtheoretic reason to expect that the free energy F N as defined in (4.75) is indeed convex. For the interested reader, we briefly sketch this argument and why it does not actually allow us to show the joint convexity of F N . Recall from (4.24) that we denote by I N (t) the mutual information between the signal x and the observation Y . By Proposition 4.4, we see that the convexity of F N (⋅,0) is equivalent to the concavity of the mutual information I N . Moreover, one can show that when we observe Y , we gain exactly as much information about the signal as if we were observing, for two independent copies W 1 and W 2 of W , the quantities

t N xx * +W 1 and t N xx * +W 2 . (4.93)
Notice that compared with the definition of Y in (4.1), the variable t has been replaced by t 2 in each of the two quantities above. Finally, once we have observed the first of the two quantities in (4.93), one can verify that we are going to gain at most as much information when we subsequently observe the second quantity in (4.93). In other words, the mutual information I N satisfies a sort of subadditivity property, and elementary properties of the mutual information allow us to upgrade this to the fact that I N is a concave function. The argument we just sketched therefore leads to the conclusion that the function F N (⋅,0) is convex. Minor variants of this argument yield that the function F N is convex in each of the variables separately.

It may seem plausible that this argument can be generalized and lead to a conceptual information-theoretic proof that the function F N is convex jointly in (t,h). However, pushing this argument to a setting with multiple variables yields instead that every entry of the Hessian of F N is non-negative. To our surprise, for the problem of community detection on sparse graphs, which unlike the situation considered in this chapter is not reducible to a problem with Gaussian noise, one can indeed show that the relevant free energy in this context is not convex in general [START_REF] Kireeva | Breakdown of a concavity property of mutual information for non-Gaussian channels[END_REF].

To sum up this informal discussion, the proof that F N is jointly convex does have to use some aspects of the particular structure of the class of problems we consider, and thus at least some calculations do need to be made. We also refer to Proposition 3.1 of [START_REF] Kireeva | Breakdown of a concavity property of mutual information for non-Gaussian channels[END_REF] for a somewhat more general view on such calculations.

Proof of Proposition 4.10. The derivative computations (4.77) and (4.82) as well as the boundedness of the support of P 1 imply that the first order derivatives of the enriched free energy F N are uniformly bounded. This establishes the uniform Lipschitz continuity of F N . That F N is non-decreasing follows from the derivative computation (4.82) and the Nishimori identity, ∂ h F N (t,h) = N -1 E⟨x⟩ 2 ⩾ 0. To prove the convexity of F N , we show that its Hessian is non-negative definite. Differentiating the expression (4.82) in h and recalling (4.78), we can write

N∂ 2 h F N (t,h) = E⟨(x ⋅ x)∂ h H N (t,h,x)⟩ -E⟨(x ⋅ x)∂ h H N (t,h,x ′ )⟩ = 1 √ 2h E⟨(x ⋅ x)(z ⋅ x -z ⋅ x ′ )⟩ + 2E⟨(x ⋅ x)(x ⋅ x -x ′ ⋅ x)⟩ -E⟨(x ⋅ x) x 2 -x ′ 2 ⟩.
We now use the Gaussian integration by parts formula in (4.29) to integrate out the noise z, as in the calculations leading to (4.33) or to (4.81). Recalling also that when the Gibbs measure is an average over the two variables x and x ′ , the underlying Hamiltonian is H N (t,h,x) + H N (t,h,x ′ ), we obtain

E⟨(x ⋅ x)(z ⋅ x)⟩ = √ 2hE⟨(x ⋅ x) x 2 -x ⋅ x ′ ⟩, E⟨(x ⋅ x)(z ⋅ x ′ )⟩ = √ 2hE⟨(x ⋅ x) x ′ 2 + x ⋅ x ′ -2x ′ ⋅ x ′′ ⟩.
It follows by the Nishimori identity that

N∂ 2 h F N (t,h) = 2E⟨(x ⋅ x ′ ) 2 ⟩ -4E⟨(x ⋅ x ′ )(x ⋅ x ′′ )⟩ + 2E⟨(x ⋅ x ′ )(x ′′ ⋅ x ′′′ )⟩. (4.94)
We introduce the re-scaled and centred variable

y ∶= 1 √ N (x -⟨x⟩),
with y ′ ,y ′′ ,y ′′′ denoting independent copies of y under the measure ⟨⋅⟩, so that

N 2 E⟨(y ⋅ y ′ ) 2 ⟩ = E⟨((x -⟨x⟩) ⋅ (x ′ -⟨x⟩)) 2 ⟩ = E⟨(x ⋅ x ′ -x ⋅ ⟨x⟩ -x ′ ⋅ ⟨x⟩ + ⟨x⟩ ⋅ ⟨x⟩) 2 ⟩ = E⟨(x ⋅ x ′ ) 2 ⟩ -2E⟨(x ⋅ x ′ )(x ⋅ x ′′ )⟩ + E⟨(x ⋅ x ′ )(x ′′ ⋅ x ′′′ )⟩ = N 2 ∂ 2 h F N (t,h). (4.95)
This already shows that F N is convex in the h variable. To compute the second derivative in t, we can rewrite (4.77) in the form of

N 2 ∂ t F N (t,h) = E⟨xx * ⋅ xx * ⟩, (4.96)
and follow through the same calculation as for the second derivative in h; the only difference is that each occurrence of x is replaced by xx * , each occurrence of x is replaced by xx * , and so on, with z being replaced by W . This leads to

N 3 ∂ 2 t F N (t,h) = 2E⟨(xx * ⋅ x ′ x ′ * ) 2 ⟩ -4E⟨(xx * ⋅ x ′ x ′ * )(xx * ⋅ x ′′ x ′′ * )⟩ + 2E⟨(xx * ⋅ x ′ x ′ * )(x ′′ x ′′ * ⋅ x ′′′ x ′′′ * )⟩,
so in terms of the re-scaled and centred variable

ξ ∶= 1 N (xx * -⟨xx * ⟩), we get 1 2N ∂ 2 t F N (t,h) = E⟨(ξ ⋅ ξ ′ ) 2 ⟩.
For the cross-derivative, we start from (4.96) and differentiate in h to obtain

N 2 ∂ h ∂ t F N (t,h) = 1 √ 2h E⟨(xx * ⋅ xx * )(z ⋅ x -z ⋅ x ′ )⟩ + 2E⟨(xx * ⋅ xx * )(x ⋅ x -x ′ ⋅ x)⟩ -E⟨(xx * ⋅ xx * ) x 2 -x ′ 2 ⟩.
A Gaussian integration by parts allows us to rewrite the first term on the right side above as

E⟨(xx * ⋅ xx * )( x 2 -x ⋅ x ′ -( x ′ 2 + x ⋅ x ′ -2x ′ ⋅ x ′′ ))⟩.
An application of the Nishimori identity therefore yields that

N 2 ∂ h ∂ t F N (t,h) = 2E⟨(xx * ⋅ x ′ x ′ * )(x ⋅ x ′ )⟩ -4E⟨(xx * ⋅ x ′ x ′ * )(x ⋅ x ′′ )⟩ -2E⟨(xx * ⋅ x ′ x ′ * )(x ′′ ⋅ x ′′′ )⟩,
which can be rewritten in terms of the re-scaled and centred variables as

1 2N ∂ h ∂ t F N (t,h) = E⟨(ξ ⋅ ξ ′ )(y ⋅ y ′ )⟩.
To see that the Hessian of F N is non-negative definite, we take w = (a,b) ∈ R 2 and write

1 2N w ⋅ ∇ 2 F N (t,h)w = a 2 E⟨(ξ ⋅ ξ ′ ) 2 ⟩ + b 2 E⟨(y ⋅ y ′ ) 2 ⟩ + 2abE⟨(ξ ⋅ ξ ′ )(y ⋅ y ′ )⟩ = E⟨(aξ ⋅ ξ ′ + by ⋅ y ′ ) 2 ⟩ ⩾ 0, as desired. ∎ Lemma 4.11. If f ∶ R ⩾0 × R ⩾0 → R is a jointly convex and Lipschitz continuous function that is non-decreasing, then its symmetrization f ∶ R ⩾0 × R → R defined by f (t,h) ∶= f (t, h
) is also jointly convex and Lipschitz continuous, with the same Lipschitz constant.

Proof. The uniform Lipschitz continuity of the symmetrization f is an immediate consequence of the uniform Lipschitz continuity of f and the reverse triangle inequality. To show that the symmetrization is also jointly convex, fix α ∈ [0,1], t,t ′ ⩾ 0 and h,h ′ ∈ R. By non-decreasingness of f ,

f αt + (1 -α)t ′ ,αh + (1 -α)h ′ = f αt + (1 -α)t ′ , αh + (1 -α)h ′ ⩽ f αt + (1 -α)t ′ ,α h + (1 -α) h ′ .
It follows by convexity of f that

f αt + (1 -α)t ′ ,αh + (1 -α)h ′ ⩽ α f (t, h ) + (1 -α) f (t ′ , h ′ ) = α f (t,h) + (1 -α) f (t ′ ,h ′ ).
This completes the proof. ∎

We now proceed analogously to our treatment of the Curie-Weiss model and aim to estimate the error term in the approximate Hamilton-Jacobi equation (4.83) in terms of quantities involving the free energy itself. Compared with (3.7) or (3.17), a new term appears that relates to the fluctuations of the free energy itself.

Proposition 4.12. The enriched free energy (4.75) in the symmetric rank-one matrix estimation problem satisfies the approximate Hamilton-Jacobi equation

0 ⩽ ∂ t F N (t,h) -∂ h F N (t,h) 2 ⩽ 1 N ∂ 2 h F N (t,h) + E ∂ h F N -∂ h F N 2 (t,h). (4.97)
Proof. The lower bound in (4.97) is immediate from (4.83) and the non-negativity of the variance. To establish the upper bound, by (4.83), it suffices to show that

Var x ⋅ x N ⩽ 1 N ∂ 2 h F N (t,h) + E ∂ h F N -∂ h F N 2 (t,h). (4.98)
As in the Curie-Weiss model, it is reasonable to expect this variance term to be related to ∂ 2 h F N (t,h). Although the value of this derivative was stated in (4.94), we will compute it slightly differently here for convenience. We write

H ′ N (h,x) ∶= 1 √ 2h z ⋅ x + 2x ⋅ x -x 2 , (4.99)
so that the derivative of the free energy (4.74) prior to averaging may be expressed concisely as

∂ h F N (t,h) = 1 N ⟨H ′ N ⟩. Differentiating this expression reveals that ∂ 2 h F N (t,h) = 1 N ⟨∂ h H ′ N ⟩ + 1 N ⟨(H ′ N ) 2 ⟩ - 1 N ⟨H ′ N ⟩ 2 = 1 N ⟨(H ′ N ) 2 ⟩ - 1 N ⟨H ′ N ⟩ 2 - 1 N(2h) 3 2 ⟨z ⋅ x⟩.
Together with (4.81) and the Nishimori identity, this implies that

∂ 2 h F N (t,h) = 1 N E⟨(H ′ N ) 2 ⟩ - 1 N E⟨H ′ N ⟩ 2 - 1 2hN E⟨ x 2 ⟩ -E⟨x ⋅ x⟩ = 1 N E⟨(H ′ N ) 2 ⟩ - 1 N E⟨H ′ N ⟩ 2 - 1 2hN E⟨ x 2 ⟩ -E ⟨x⟩ 2 .
Notice that the variance of H ′ N may be written as

Var H ′ N = E⟨ H ′ N -⟨H ′ N ⟩ 2 ⟩ + E ⟨H ′ N ⟩ -E⟨H ′ N ⟩ 2 = E⟨(H ′ N ) 2 ⟩ -E⟨H ′ N ⟩ 2 + N 2 E ∂ h F N -∂ h F N 2 (t,h).
Up to lower-order terms, the proof consists in showing that the variance of x ⋅ x is bounded from above by the variance of H ′ N , which in turn is essentially the right side of (4.97) up to scaling. To justify this precisely, we write

1 N 2 Var H ′ N = 1 N ∂ 2 h F N (t,h) + E ∂ h F N -∂ h F N 2 + 1 2hN 2 E⟨ x 2 ⟩ -E ⟨x⟩ 2 .
To relate this back to the variance term in (4. [START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF], observe that by the derivative computation (4.82),

1 N 2 Var H ′ N - 1 N 2 Var(x ⋅ x) = 1 N 2 E⟨(H ′ N ) 2 ⟩ - 1 N 2 E⟨(x ⋅ x) 2 ⟩. It follows that Var x ⋅ x N ⩽ 1 N ∂ 2 h F N (t,h) + E ∂ h F N -∂ h F N 2 (t,h) + 1 2hN 2 E⟨ x 2 ⟩ - 1 N 2 E⟨(H ′ N ) 2 ⟩ + 1 N 2 E⟨(x ⋅ x) 2 ⟩. (4.100)
What is important for the sequel is to verify that the second line in this expression is of lower order in N due to a cancellation between E⟨(H ′ N ) 2 ⟩ and E⟨(x ⋅ x) 2 ⟩. But in fact we can show that this second line in (4.100) is non-positive. We observe that

E⟨(H ′ N ) 2 ⟩ = 1 2h E⟨(z ⋅ x) 2 ⟩ + 2 √ 2h E⟨z ⋅ x(2x⋅x -x 2 )⟩ + 4E⟨x ⋅ x(x ⋅ x -x 2 )⟩ + E⟨ x 4 ⟩.
Fixing i, j ∈ {1,...,N}, two applications of the Gaussian integration by parts formula in (4.29) reveal that for i ≠ j,

E⟨z i z j x i x j ⟩ = √ 2hE⟨z j x j (x 2 i -x i x ′ i )⟩ = 2hE⟨(x 2 i -x i x ′ i )(x 2 j + x j x ′ j -2x j x ′′ j )⟩.
while for i = j,

E⟨z 2 i x 2 i ⟩ = √ 2hE⟨z i x i (x 2 i -x i x ′ i )⟩ + E⟨x 2 i ⟩ = 2hE⟨(x 2 i -x i x ′ i )(x 2 i + x i x ′ i -2x i x ′′ i )⟩ + E⟨x 2 i ⟩.
Together with the Nishimori identity, this shows that

1 2h E⟨(z ⋅ x) 2 ⟩ = E⟨ x 4 ⟩ -2E⟨ x 2 (x ⋅ x)⟩ -E⟨(x ⋅ x) 2 ⟩ + 2E⟨(x ⋅ x)(x ⋅ x ′ )⟩ + 1 2h E⟨ x 2 ⟩.
A similar computation using the Gaussian integration by parts formula gives

2 √ 2h E⟨z ⋅ x(2x ⋅ x -x 2 )⟩ = 2E⟨(2x ⋅ x -x 2 )( x 2 -(x ⋅ x ′ )⟩ = 6E⟨ x 2 (x ⋅ x)⟩ -4E⟨(x ⋅ x)(x ⋅ x ′ )⟩ -2E⟨ x 4 ⟩.
It follows by the Cauchy-Schwarz inequality that

E⟨(H ′ N ) 2 ⟩ = 3E⟨(x ⋅ x) 2 ⟩ -2E⟨(x ⋅ x)(x ⋅ x ′ )⟩ + 1 2h E⟨ x 2 ⟩ ⩾ E⟨(x ⋅ x) 2 ⟩ + 1 2h E⟨ x 2 ⟩.
This shows that the second line in (4.100) is non-positive and establishes the upper bound (4.98), thereby completing the proof. ∎

To control the error term on the right side of the approximate Hamilton-Jacobi equation (4.97), at first glance, it seems like we need to establish the concentration of the derivative of the enriched free energy (4.74) about its average. However, even in the simpler setting of the Curie-Weiss model with a random external field, we do not expect this concentration of the derivative to be valid at every single point. But in any case, we do not expect the term N -1 ∂ h F N (t,h) in (4.97) to tend to zero at every single point (t,h) either. The whole point of the notion of viscosity solution is that it suffices to control the size of the error term in (4.97) at appropriate contact points. In particular, we see from the proof of Theorem 3.25 that it suffices to establish the smallness of the error term at every point at which the free energy is touched from above by a smooth function. Leveraging convexity again, we will show that at such points, the variance of the derivative of the free energy can be controlled in terms of

E sup (t,h)∈[0,M] 2 F N (t,h) -F N (t,h) 2 , (4.101)
for an adequate choice of M < +∞. Leveraging the Gaussian concentration inequality and a covering argument, we will show that (4.101) is essentially of order N -1 . We start by establishing a Gaussian-type estimate on the tail of the random variable in (4.101).

Lemma 4.13. For each M ∈ R ⩾0 , there exists a constant C < +∞ such that for all λ > CN -1 2 log(N), we have

P sup (t,h)∈[0,M] 2 F N (t,h) -F N (t,h) ⩾ λ ⩽ exp - Nλ 2 C . (4.102)
Proof. We write C < +∞ for a constant that does not depend on N or λ , but may depend on M, and whose value may change as we proceed through the proof. We introduce the random norm

W * ∶= sup x ⩽1 W x .
To run a covering argument, we begin by establishing the Hölder continuity of the free energy F N . We fix t,t ′ ,h,h ′ ∈ [0,M] as well as x in the support of P N , and observe that by the Cauchy-Schwarz inequality and the boundedness of the support of P 1 , we have

H N (t,h,x) -H N (t ′ ,h ′ ,x) ⩽ 2t N - 2t ′ N x ⋅W x + 1 N t -t ′ 2(x ⋅ x) 2 + x 4 + √ 2h - √ 2h ′ x ⋅ z + h -h ′ 2 x ⋅ x + x 2 ⩽ C √ N √ t - √ t ′ W * + √ h - √ h ′ z +CN t -t ′ + h -h ′ .
Together with the observation that for y ⩾ y ′ , we have √ y -√ y ′ ⩽ yy ′ 1 2 as well as yy ′ ⩽ 2 y yy ′ 1 2 , this bound on the Hamiltonian implies that the free energy (4.74) is Hölder continuous on [0,M] 2 with

F N (t,h) -F N (t ′ ,h ′ ) ⩽ C t -t ′ 1 2 + h -h ′ 1 2 X for the random variable X ∶= 1 + W * √ N + z √ N .
We also recall from Proposition 4.10 that the averaged free energy F N is Lipschitz continuous, uniformly over N. We thus deduce that for every λ > 0 and ε > 0,

P sup [0,M] 2 F N (t,h) -F N (t,h) ⩾ λ ⩽ P sup Aε F N (t,h) -F N (t,h) ⩾ λ 2 + P X ⩾ ε -1 2 λ C for the set A ε ∶= εZ 2 ∩ [0,M] 2 .
Indeed, every point (t,h) ∈ [0,M] 2 is at distance at most ε from a point in A ε . A union bound and the fact that the cardinality of A ε is bounded by Cε -2 yield that this is further bounded by

Cε -2 sup Aε P F N (t,h) -F N (t,h) ⩾ λ 2 + P X ⩾ ε -1 2 λ C .
A simple extension of the proof of the free energy concentration inequality (4.59) yields that for every λ ⩾ 0 and

(t,h) ∈ [0,M] 2 , P F N (t,h) -F N (t,h) ⩾ λ ⩽ 2exp - Nλ 2 C .
On the other hand, Exercise 4.5 and (4.43) imply that for some constant C ′ < +∞, and any ε > 0 and λ > C ′ √ ε,

P{X ⩾ ε -1 2 λ C ⩽ exp - Nλ 2 εC .
Combining these two bounds and choosing ε = N -1 reveals that for any λ > C ′ N -1 2 , P sup

[0,M] 2 F N (t,h) -F N (t,h) ⩾ λ ⩽ CN 2 exp - Nλ 2 C .
Whenever λ > C ′ N -1 2 log(N) for some sufficiently large constant C ′ < +∞, the term N 2 can be absorbed into the exponential to obtain (4.102) and complete the proof. ∎

We are now ready to verify the key assumption of the convex selection principle besides the convexity property itself: that any subsequential limit of F N must satisfy the equation on a dense subset. It is convenient to phrase this in terms of contact points, just like we did in Section 3.6, see in particular Corollary 3.24.

Lemma 4.14. Let f be any subsequential limit of the sequence (F N ) N⩾1 of enriched free energies in the symmetric rank-one matrix estimation problem, and fix t * ,h

* > 0. If φ ∈ C ∞ (R >0 × R;R) is a smooth function with the property that f -φ has a strict local maximum at (t * ,h * ) ∈ R >0 × R >0 , then ∂ t φ -(∂ h φ ) 2 (t * ,h * ) = 0 (4.103)
Proof. Abusing notation, we do not denote the subsequence along which the convergence of F N to f occurs explicitly, in effect pretending that the convergence occurs along the whole sequence. Using Exercise 3.1, we find a sequence (t N ,h N ) N⩾1 converging to (t * ,h * ) such that F Nφ has a local maximum at (t N ,h N ); this exercise also guarantees that the neighbourhood over which (t N ,h N ) is a local maximum can be chosen independently of N. To control the right side of the approximate Hamilton-Jacobi equation (4.97) at the contact point (t N ,h N ), the idea will be to argue similarly to the proof of Proposition 2.15 where it is shown that whenever a sequence of convex functions converges, the sequence of derivatives also converges to the derivative of the limit at every point of differentiability of the limit. The subsequential limit f of the sequence (F N ) N⩾1 of convex functions will be differentiable at the contact point (t * ,h * ). The sequence (F N ) N⩾1 also converges to f in the sense of second moments by Lemma 4.13, and although F N itself is not necessarily convex, its second derivative is bounded from below by a term of order one which will suffice to control the error term in (4.97). The proof therefore proceeds in three steps. First we control the upper deviation of F N from its tangent at (t N ,h N ) by a parabola, then we control the lower deviation of F N from its tangent at (t N ,h N ) by a random parabola, and finally we combine these two ingredients to control the right side of the approximate Hamilton-Jacobi equation (4.97). Throughout the proof, we understand that the constant C < +∞ may change from one occurrence to the next, only making sure that it does not depend on N.

Step 1: Hessian of F N upper bound. Since F Nφ has a local maximum at (t N ,h N ) and φ is smooth, there exists C < +∞ such that for every

h ′ ∈ R with h ′ ⩽ C -1 , F N (t N ,h N + h ′ ) -F N (t N ,h N ) ⩽ φ (t N ,h N + h ′ ) -φ (t N ,h N ) ⩽ h ′ ∂ h φ (t N ,h N ) +C h ′ 2 .
For every N sufficiently large, we have h N > 0, and therefore

∂ h (F N -φ )(t N ,h N ) = 0. It follows that for every h ′ ∈ R with h ′ ⩽ C -1 , F N (t N ,h N + h ′ ) -F N (t N ,h N ) ⩽ h ′ ∂ h F N (t N ,h N ) +C h ′ 2 . (4.104)
In particular, we have

∂ 2 h F N (t N ,h N ) ⩽ C, (4.105)
which is also immediate from the observation that

∂ 2 h (F N -φ )(t N ,h N ) ⩽ 0.
Step 2: Hessian of F N lower bound. Recalling the definition of the Hamiltonian H ′ N in (4.99), we can write

∂ 2 h F N (t N ,h N + h ′ ) = 1 N ⟨(H ′ N ) 2 ⟩ - 1 N ⟨H ′ N ⟩ 2 - 1 N(2(h N + h ′ )) 3 2 ⟨z ⋅ x⟩.
Since h N converges to h * > 0, it remains bounded away from zero for N sufficiently large. Using also the non-negativity of the variance, the Cauchy-Schwarz inequality and the fact that the measure P 1 has bounded support, we deduce that for every h ′ ⩽ C -1 , we have

∂ 2 h F N (t N ,h N + h ′ ) ⩾ - C z √ N .
It follows by Taylor's theorem that for every

h ′ ⩽ C -1 , F N (t N ,h N + h ′ ) -F N (t N ,h N ) -h ′ ∂ h F N (t N ,h N ) ⩾ - C z √ N h ′ 2 . (4.106)
Step 3: controlling the right side of (4.97). Combining (4.104) and (4.106) with the fact that (t N ,h N ) N⩾1 converges to (t * ,h * ) gives a neighbourhood V of (t * ,h * ) with

h ′ ∂ h F N -∂ h F N (t N ,h N ) ⩽ 2sup V F N -F N +C h ′ 2 1 + z √ N . (4.107)
In particular, given a deterministic λ ∈ [0,C -1 ], the bound (4.107) for

h ′ = λ ∂ h F N -∂ h F N ∂ h F N -∂ h F N (t N ,h N ) reads λ ∂ h F N -∂ h F N (t N ,h N ) ⩽ 2sup V F N -F N +Cλ 2 1 + z √ N .
Squaring both sides of this inequality, taking expectations and leveraging the concentration inequality in Lemma 4.13 yields

λ 2 E ∂ h F N -∂ h F N 2 (t N ,h N ) ⩽ 8Esup V F N -F N 2 +Cλ 4 E 1 + z √ N 2 ⩽ C N 1 2 +Cλ 4 ,
where we have used that E z 2 = NEz 2 1 = N. Choosing λ ∶= N -1 8 leads to the upper bound

E ∂ h F N -∂ h F N 2 (t N ,h N ) ⩽ C N 1 4
. Substituting this into the approximate Hamilton-Jacobi equation in Proposition 4.12 and remembering the Hessian bound (4.105) gives

0 ⩽ ∂ t φ -(∂ h φ ) 2 (t N ,h N ) = ∂ t F N -(∂ h F N ) 2 (t N ,h N ) ⩽ C N + C N 1 4 .
Letting N tend to infinity and using the smoothness of φ completes the proof. ∎

We are now ready to prove Theorem 4.9 using the convex selection principle. We will appeal to the version of this result given in Lemma 3.23. We point out that under the additional assumption that Ex 1 = 0, this proof can be simplified, as Step 3 becomes unnecessary, and we can instead rely on the simpler version of the convex selection principle given by Theorem 3.21. The key observation is that under this additional assumption, the symmetrized initial condition ψ is in C 1 (R;R), by (S.27).

Proof of Theorem 4.9. The Arzelà-Ascoli theorem together with the derivative computations (4.77) and (4.82) and the boundedness of the support of P 1 imply that the sequence (F N ) N⩾1 is precompact. Denoting by f a subsequential limit, the idea is to apply the convex selection principle in Lemma 3.23 to the symmetrization f of f and show that it is the unique viscosity solution to the Hamilton-Jacobi equation (4.87). The proof therefore proceeds in four steps. First, we show that f is jointly convex and Lipschitz continuous; then we prove that it satisfies the Hamilton-Jacobi equation (4.87) on a dense subset of R ⩾0 × R, and that the initial condition ψ ∶= f (0,⋅) satisfies the assumption in Lemma 3.23; and finally we conclude using the uniqueness of viscosity solutions and the Hopf-Lax formula.

Step 1: f is jointly convex and Lipschitz continuous. Each free energy F N is jointly convex, Lipschitz continuous and non-decreasing by Proposition 4.10. Moreover, the Lipschitz constant of F N is bounded uniformly over N. It follows by Lemma 4.11 that the symmetrization of each free energy F N is also jointly convex and Lipschitz continuous uniformly over N. Since this sequence of symmetrizations converges to f , the function f must also be jointly convex and Lipschitz continuous.

Step 2: f satisfies (4.87) on a dense set. Following the proof of Corollary 3.24, we deduce from Lemma 4.14 that the set

A ∶= (t,h) ∈ R >0 × R >0 f is differentiable at (t,h) and ∂ t f -(∂ h f ) 2 (t,h) = 0 is dense in R >0 × R >0 .
We also observe that if t > 0 and h < 0 are such that (t, h ) is a point of differentiability of f , then f is differentiable at (t,h) and we have

∂ t f (t,h) = ∂ t f (t, h ) and ∂ h f (t,h) 2 = ∂ h f (t, h ) 2 .
This implies that f satisfies the Hamilton-Jacobi equation (4.87) on a dense subset of R ⩾0 × R.

Step 3: ψ satisfies the assumption of Lemma 3.23. We first verify that the initial condition ψ satisfies the assumption of Lemma 3.23 at any point h ≠ 0. That is, for every h ≠ 0 and p ∈ ∂ ψ(h), we find b ∈ R such that (b, p) ∈ ∂ f (0,h) and bp 2 ⩾ 0. By the results of the previous steps and Lemma 3.22, there exists (a,q) ∈ ∂ f (0,h) such that aq 2 = 0. We must have in particular that q ∈ ∂ ψ(h), so by Theorem 2.13, we have q = p, and the choice of b = a = p 2 satisfies our requirements. For future reference, we observe that for every h > 0, we thus have that p = ∂ h ψ(h) and (∂ h ψ(h) 2 ,∂ h ψ(h)) ∈ ∂ f (0,h). Letting h > 0 tend to zero and using Proposition 2.14, we deduce that

∂ h ψ(0) 2 ,∂ h ψ(0) ∈ ∂ f (0,0). (4.108)
We now consider the case h = 0. Letting p ∈ ∂ ψ(0), we look for b ∈ R such that (b, p) ∈ ∂ f (0,0) and bp 2 ⩾ 0. Since p ∈ ∂ ψ(0), we have that for every y ∈ R,

ψ(y) -ψ(0) = ψ( y ) -ψ(0) ⩾ py.
Letting y tend to zero from the left and from the right of zero, we see that this implies that p ⩽ ∂ h ψ(0). Letting b ∶= ∂ h ψ(0) 2 , we therefore have that

b -p 2 ⩾ b -∂ h ψ(0) 2 = 0.
It remains to verify that (b, p) ∈ ∂ f (0,0). By (4.108), for every (t,h) ∈ R ⩾0 × R, we have that f (t,h) ⩾ f (0,0) +tb + h∂ h ψ(0).

Using that f (t,-h) = f (t,h), and then that p ⩽ ∂ h ψ(0), we deduce that for every

(t,h) ∈ R ⩾0 × R, f (t,h) ⩾ f (0,0) +tb + h ∂ h ψ(0) ⩾ f (0,0) +tb + hp.
This shows that (b, p) ∈ ∂ f (0,0), as desired.

Step 4: conclusion. The results of the previous steps ensure that we can invoke the convex selection principle in Lemma 3.23 to conclude that f is a viscosity solution to the Hamilton-Jacobi equation (4.87) with initial condition ψ. Together with the uniqueness result in Corollary 3.7, this implies that the symmetrization of the sequence (F N ) N⩾1 converges to the unique viscosity solution f to the Hamilton-Jacobi equation (4.87) with initial condition ψ. In particular, the sequence (F N ) N⩾1 of functions from R ⩾0 × R ⩾0 to R converges to some limit f , which is the restriction of f to R ⩾0 × R ⩾0 . The Hopf-Lax formula and the computation for the convex dual of the square function in Exercise 2.11 imply that for all t,h ⩾ 0,

f (t,h) = f (t,h) = sup h ′ ∈R ψ( h ′ ) - (h ′ -h) 2 4t .
Using that h ⩾ 0, we see that it is always better to choose h ′ ⩾ 0 in the optimization problem above, since for every h ′ ⩾ 0,

ψ( -h ′ ) - (-h ′ -h) 2 4t ⩽ ψ( h ′ ) - (h ′ -h) 2 4t ,
and we thus obtain that

f (t,h) = sup h ′ ⩾0 ψ(h ′ ) - (h ′ -h) 2 4t = sup h ′ ⩾-h ψ(h + h ′ ) - (h ′ ) 2 4t . (4.109)
Recall also from Proposition 4.10 that the function ψ is non-decreasing. In the supremum on the right side of (4.109), we may therefore restrict the supremum to h ′ ⩾ 0, since when h ′ ∈ [-h,0], we have

ψ(h + h ′ ) - (h ′ ) 2 4t ⩽ ψ(h).
We have thus shown (4.90). Remembering (4.88) and evaluating this at h = 0 gives the representation (4.91) and completes the proof. ∎

We can now combine the formula (4.91) for the limit free energy in the symmetric rank-one matrix estimation problem obtained in Theorem 4.9 with Propositions 4.3 and 4.4 to determine the two information-theoretic quantities introduced at the beginning of this chapter, the minimal mean-square error (4.2) and the mutual information (4.4). It will be convenient to denote by

f (t) ∶= sup h⩾0 ψ(h) - h 2 4t (4.110)
the limit free energy in the symmetric rank-one matrix estimation problem, and to write D for its points of differentiability, Together with the derivative computation (4.77), this allows us to express the minimal mean-square error in terms of any maximizer h * (t). We also recall from the envelope theorem that for every t ⩾ 0, if there exists a unique maximizer to the right side of (4.110), then t ∈ D.

D ∶= t ∈ R ⩾0 f is differentiable at t . ( 4
Proposition 4.15. At every point of differentiability t ∈ D of the limit free energy (4.110), the limit minimal mean-square error is given by

mmse(t) ∶= lim N→+∞ mmse N (t) = E x 1 2 2 - h 2 * (t) 4t 2 , (4.113)
where h * (t) denotes any maximizer of the right side of (4.110).

Proof. Proposition 4.3, the asymptotic expansion (4. [START_REF] Daniel | Spin-glass models of neural networks[END_REF], and the derivative computation (4.77) imply that for every t ⩾ 0,

mmse N (t) = E x 1 2 2 - 1 N 2 E⟨(x ⋅ x) 2 ⟩ + o(1) = E x 1 2 2 -∂ t F N (t,0) + o(1).
Together with Proposition 2.15, Theorem 2.13 and the consequence (4.112) of the envelope theorem, this implies that at any point of differentiability t ∈ D, we have

mmse(t) = E x 1 2 2 -∂ t f (t,0) = E x 1 2 2 - h 2 * (t) 4t 2 ,
as required. ∎ A variational formula for the limit mutual information can be obtained directly from the variational representation (4.110) of the limit free energy. Proposition 4.16. For every t ⩾ 0, the limit mutual information is given by

I(t) ∶= lim N→+∞ I N (t) = t E x 1 2 2 -sup h⩾0 ψ(h) - h 2 4t . ( 4
I N (t) = t E x 1 2 2 -F ○ N (t) + o(1).
Letting N tend to infinity and using the variational formula (4.110) for the limit free energy completes the proof. ∎

Before moving to the next section, we mention more general models that can also be handled by the techniques we used to analyze the symmetric rank-one matrix estimation problem.

A first natural generalization consists in relaxing the symmetry assumption, and in considering matrices of higher rank. For illustration, we may imagine a recommendation system collecting ratings for each customer and product. One may model the fundamental characteristics of a customer by a vector with K coordinates, for some K relatively small, say K ≃ 10, and represent the list of customers as an M-by-K matrix u, and where we can postulate that the rows representing the customers are independent and identically distributed. We also assume that the products on sale can be similarly represented by an N-by-K matrix v, with i.i.d. rows representing each product, in such a way that the ratings that users attribute to the products are essentially a noisy version of the M-by-N matrix uv * . We would like to retrieve information about the matrix uv * given the noisy observations we have of it, in the regime in which M and N are both large and of the same order of magnitude.

We can generalize this model further, so that we can represent, for instance, a situation in which we have multiple different observations of the interactions between the rows of u and v, or we have higher-order interactions. To encode this larger class of problems, we fix integers p,K,L ⩾ 1, and a matrix A ∈ R K p ×L that are deterministic (and known to the observer). Let X be an N-by-K random matrix, and denote by X ⊗p the p-fold tensor product of X, which we think of as an N p -by-K p matrix. The more general model consists in observing a noisy version of the matrix X ⊗p A, and asking again whether meaningful information can be recovered about X ⊗p A or X from this observation, in the regime of large N. Under mild assumptions on the law of X (e.g. the rows are i.i.d. and almost surely bounded in norm by a fixed constant), the techniques that we have used to analyze the symmetric rank-one matrix estimation problem allow us to answer this more general question precisely [START_REF] Chen | Statistical inference of finite-rank tensors[END_REF].

We have seen in Chapter 3, and in particular in Proposition 3.20, that the standard Curie-Weiss model can be analyzed more directly, without appealing to the convex selection principle. The inequality on the left side of (4.97) can be used in a similar way to analyze the symmetric rank-one matrix estimation problem. We chose to present a proof based on the convex selection principle because this is the only proof we are aware of that is capable of handling the more general inference problems discussed in the previous paragraph. To obtain this generalization, the main difficulty is that the variable h ⩾ 0 must be replaced by a positive semi-definite matrix, so the symmetrization trick we have used to handle the domain R ⩾0 no longer works. In this case, we must therefore make sense of Hamilton-Jacobi equations posed on domains with boundaries; we refer to [START_REF] Chen | Hamilton-Jacobi equations with monotone nonlinearities on convex cones[END_REF][START_REF] Chen | Statistical inference of finite-rank tensors[END_REF][START_REF] Dominguez | A Hamilton-Jacobi approach to the stochastic block model[END_REF] for more on this. The non-linearity appearing in the Hamilton-Jacobi equation is the mapping q ↦ (AA * ) ⋅ q ⊗p . This mapping is not necessarily convex, so the Hopf-Lax formula no longer applies in this case. But the free energy is still jointly convex, and the Hopf representation of solutions can be adapted to this context with a boundary. We thus end up with a variational representation of the limit free energy, and can thereby conveniently analyze the minimal mean-square error and mutual information of the model.

Exercise 4.9. The goal of this exercise is to prove the existence of a "phase transition" for the limit of the minimal mean-square error (4.2) as we vary the signalto-noise ratio, as was announced in the introduction to this chapter. Notice that the mean-square error of the null estimator (0.9) converges to (E x 1 2 ) 2 as N tends to infinity. Assuming that Ex 1 = 0 and that x 1 is not constant, show that there exists a critical parameter t c ∈ R >0 such that for every t < t c , lim

N→+∞ mmse N (t) = E x 1 2 2 , while (4.115)
for every t > t c , limsup

N→+∞ mmse N (t) < E x 1 2 2 . (4.116)
(What happens at t = t c depends on the law of x 1 .)

Exercise 4.10. Let (t,h) ∈ R >0 × R >0 be a point of differentiability of the limit free energy f ∶ R ⩾0 × R ⩾0 → R in the symmetric rank-one matrix estimation problem. Denote by h * (t,h) any maximizer of the right side of (4.90). Prove that for the choice of λ 0 ⩾ 0 that minimizes the mean-square error

lim N→+∞ 1 N E x -E[x Y] 2 = Ex 2 1 + h -h * (t,h) 2t -∂ h ψ(h * (t,h)). ( 4
mse N ∶= 1 N 2 inf λ ⩾0 E xx * -λ vv * 2 . (4.119)
By comparing the limit mean-square error of the PCA estimator to the limit minimal mean-square error computed in Proposition 4.15 for a Gaussian prior, a Rademacher prior and a sparse prior, we will see that the performance of PCA depends greatly on the identity of this prior. By "prior", we mean the law of the signal P N = (P 1 ) ⊗N .

In this section, we always normalize this measure so that Ex 1 = 0 and Ex 2 1 = 1. To study the performance of the PCA estimator, we start by determining its asymptotic mean-square error. We will rely on a result in random matrix theory which states that, in probability, lim

N→+∞ x ⋅ v 2 x 2 v 2 = max 0,1 - 1 4t . (4.120)
We will take this convergence for granted and refer the interested reader to Section 3.1 in [START_REF] Benaych | The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices[END_REF] for its proof in a more general setting (see also [START_REF] Baik | Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices[END_REF]).

Proposition 4.17. For every t ⩾ 0, the asymptotic mean-square error associated with the PCA estimator is given by

mse(t) ∶= lim N→+∞ mse N (t) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1 if t ⩽ 1 4 , 1 4t 2 -1 4t if t ⩾ 1 4 . (4.121)
Proof. We start by writing, for any λ ⩾ 0,

E xx * -λ vv * 2 = E x 4 -2λ E(x ⋅ v) 2 + λ 2 E v 4 .
Optimizing over λ and recalling that v 2 = N reveals that the optimal λ is

λ 0 ∶= E(x ⋅ v) 2 E v 4 = E(x ⋅ v) 2 N 2 .
It follows by the asymptotic expansion (4.19) and the assumption Ex

2 1 = 1 that mse N (t) = E x 4 N 2 - E(x ⋅ v) 2 N 2 2 = 1 -E x ⋅ v 2 x 2 v 2 x 2 N 2 + o(1)
.

By the law of large numbers, we have that x 2 N converges to 1 almost surely as N tends to infinity. Together with (4.120), this gives the convergence in probability lim

N→+∞ x ⋅ v 2 x 2 v 2 x 2 N = max 0,1 - 1 4t .
Remembering that the support of P 1 is bounded and invoking the dominated convergence theorem for convergence in probability reveals that lim

N→+∞ mse N (t) = 1 -max 0,1 - 1 4t 2 .
Expanding and distinguishing cases completes the proof. ∎

This result implies that the critical signal-to-noise ratio of the PCA estimator is always t * PCA = 1 4 . It does not depend on the prior, except to the extent that we have imposed Ex 1 = 0 and Ex 2 1 = 1. By considering three different priors, we will see that the PCA estimator can sometimes be optimal, sometimes be sub-optimal but with a critical signal-to-noise ratio coinciding with the optimal one, and sometimes be sub-optimal and with a critical signal-to-noise ratio differing from the optimal one.

In the context of the Gaussian prior

dP 1 (x) ∶= 1 √ 2π exp - x 2 2 dx, (4.122)
the PCA estimator is optimal.

Proposition 4.18. For the Gaussian prior (4.122), the limit minimal mean-square error is given by

mmse(t) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 1 if t ⩽ 1 4 , 1 4t 2 -1 4t if t ⩾ 1 4 . (4.123)
Partial proof. Although the Gaussian prior (4.122) does not have bounded support, the formula (4.110) for the limit free energy still holds in this setting. This can be shown by approximation, but we will simply admit it. In the Gaussian setting, the initial condition ψ can be computed explicitly. Recall that

ψ(h) = Elog ˆ+∞ -∞ exp √ 2hxz 1 + 2hxx 1 -hx 2 dP 1 (x),
ψ(h) = 1 2(1 + 2h) E 2hz 2 1 + 4h √ 2hz 1 x 1 + 4h 2 x 2 1 - 1 2 log(1 + 2h) = h - 1 2 log(1 + 2h).
Any maximizer h * (t) of (4.110) must therefore satisfy the critical point equation

1 - 1 1 + 2h * (t) = h * (t) 2t .
Rearranging reveals that h * (t) = 2t -1 2 or h * (t) = 0. If t ⩽ 1 4 , the only non-negative solution is h * (t) = 0. This implies that the limit free energy is differentiable at t and, by Proposition 4.15,

mmse(t) = 1 - h 2 * (t) 4t 2 = 1.
On the other hand, if t > 1 4 , then

ψ 2t - 1 2 - (2t -1 2 ) 2 4t = t - 1 2 log(4t) - 1 16t > 1 4 - 1 2 log(1) - 1 4 = 0 so (4.110) is maximized by h * (t) = 2t -1 2 .
Together with the envelope theorem (Theorem 2.21), this shows that for t > 1 4 , the free energy is differentiable at t, and

mmse(t) = 1 - h 2 * (t) 4t 2 = 1 - (2t -1 2 ) 2 4t 2 = 2t 4t 2 - 1 16t 2 = 1 4t 2 - 1 4t .
This completes the proof. ∎

In the context of a Rademacher prior

P 1 ∶= 1 2 δ 1 + 1 2 δ -1 (4.124)
the PCA estimator is sub-optimal but identifies the correct critical signal-to-noise ratio. In other words, the PCA estimator recovers part of the signal exactly in the region where this is feasible, but it does not recover as much of the signal as the optimal estimator. This is illustrated in Figure 4.1. While we expect the PCA estimator to be sub-optimal in the entire regime t > 1 4 , we show this rigorously only for t sufficiently large.

∂ h ψ(h) = E⟨x ⋅ x 1 ⟩ = E⟨x⟩ 2 = Etanh 2 √ 2hz 1 + 2hx 1 .
Averaging with respect to the randomness of x 1 , and using the symmetry of tanh 2 and z 1 reveals that

∂ h ψ(h) = 1 2 Etanh 2 √ 2hz 1 + 2h + 1 2 Etanh 2 √ 2hz 1 -2h = Etanh 2 √ 2hz 1 + 2h .
Any maximizer h * (t) of (4.110) must therefore satisfy the equation

h * (t) = 2tEtanh 2 2h * (t)z 1 + 2h * (t) . (4.126)
The second derivative computation (4.94), the Nishimori identity, and the fact that x 2 = 1 for any x in the support of P 1 imply that

∂ 2 h ψ(h) = 2E⟨(xx) 2 ⟩ -4E⟨xxxx ′ ⟩ + 2E⟨xx⟩ 2 = 2 -4E⟨x⟩ 2 + 2E⟨x⟩ 4 = 2E 1 -⟨x⟩ 2 2 .
This means that 0 ⩽ ∂ 2 h ψ(h) ⩽ 2. Together with the fact that ∂ h ψ(0) = ψ(0) = 0, this implies that 0 ⩽ ψ(h) ⩽ h 2 . It follows that for t ⩽ 1 4 and h > 0,

ψ(h) - h 2 4t ⩽ h 2 1 - 1 4t ⩽ 0.
This means that for t < 1 4 , we have h * (t) = 0. On the other hand, if t > 1 4 and h > 0 is small enough, then

ψ(h) - h 2 4t = h 2 1 - 1 4t + o(h 2 ) > 0, so h * (t) > 0.
Step 2: h * (t) is essentially 2t for t large. For any x ∈ R,

1 -tanh 2 (x) = 1 - e 2x -1 e 2x + 1 2 = 4e 2x e 4x + 2e 2x + 1 ⩽ 4 e 2x + e -2x ⩽ 4e -2 x .
It follows by the critical point equation (4.126) that

h * (t) -2t ⩽ 2tE tanh 2 2h * (t)z 1 + 2h * (t) -1 ⩽ 8tEexp -2 2h * (t)z 1 + 2h * (t) .
To bound this further, introduce the set A h * (t) ∶= { 2h * (t)z 1 > h * (t)}, and observe that

h * (t) -2t ⩽ 8tP A h * (t) + 8tEexp -2 2h * (t)z 1 + 2h * (t) 1 A c h * (t) ⩽ 8tP A h * (t) + 8t exp -2h * (t) .
Since z 1 is a Gaussian random variable, for any c > 0, we have

P{ z 1 > c} = 2 √ 2π ˆ+∞ c exp - x 2 2 dx ⩽ 2 π ˆ+∞ c x c exp - x 2 2 dx ⩽ √ 2 c √ π exp - c 2 2 .

Applying this with

mmse(t) = 1 - h 2 * (t) 4t 2 ⩽ 1 - 2t -e -t 4 2 4t 2 = e -t 4 2t - e -t 2 4t 2 .
Increasing t if necessary to ensure that e -t 2 ⩽ 1 4 e -t 4 gives the strict upper bound

mmse(t) ⩽ e -t 4 4t 2 - 1 4t < 1 4t 2 - 1 4t
and completes the proof. ∎

In the context of a sparse prior defined for some p ∈ (0,1) by

P 1 ∶= (1 -p)δ 0 + p 2 δ 1 √ p + p 2 δ -1 √ p , (4.127)
the PCA estimator can be sub-optimal in a stronger sense: for p > 0 sufficiently small, there is a region of signal-to-noise ratios for which a non-trivial recovery of the signal is possible but for which the PCA estimator is trivial.

Proposition 4.21. For the sparse prior (4.124) with p > 0 small enough, there exists a signal-to-noise ratio t * (p) <

ψ(h) = Elog (1 -p) + p 2 exp 2h p z 1 + 2h √ p x 1 - h p + p 2 exp - 2h p z 1 - 2h √ p x 1 - h p .
The monotonicity of the logarithm together with the independence of z 1 and x 1 give the lower bound

Elog p 2 exp 2h p z 1 + 2h √ p x 1 - h p 1 {x 1 =1 √ p} = p 2 log p 2 + h 2 .
Similarly,

Elog p 2 exp - 2h p z 1 - 2h √ p x 1 - h p 1 {x 1 =-1 √ p} = p 2 log p 2 + h 2 .
Combining these two bounds reveals that

ψ(h) ⩾ plog p 2 + h.
If we set h = 1 2 and choose p * small enough so -1 16 ⩽ plog(p 2) ⩽ 0 whenever 0 ⩽ p ⩽ p * , we have that

ψ 1 2 - ( 1
2) 2 4t ⩾ 1 2 - 1 16 - 1 16t = 1 16 7 - 1 t .
It follows that any maximizer h * (t) of (4.110) must be strictly positive for any t > 1 7 , and therefore mmse(t) < 1. Setting t * (p) = 1 7 completes the proof. ∎

The community detection problem

In this section, we analyze the information-theoretic properties of the stochastic block model. We do so by leveraging a universality property of the free energy (4.9) that will allow us to map the stochastic block model onto the symmetric rankone matrix estimation problem. The stochastic block model is a simple model for networks with a community structure. It was first introduced in the machine learning and statistics literature [START_REF] Fienberg | Statistical analysis of multiple sociometric relations[END_REF][START_REF] Holland | Stochastic blockmodels: first steps[END_REF][START_REF] Wang | Stochastic blockmodels for directed graphs[END_REF][START_REF] Harrison | Social structure from multiple networks. I. Blockmodels of roles and positions[END_REF], and also emerged independently in a variety of other scientific disciplines. In the theoretical computer science community it is often called the planted partition model [START_REF] Ravi | Eigenvalues and graph bisection: an average-case analysis[END_REF][START_REF] Nguyen Bui | Graph bisection algorithms with good average case behavior[END_REF][START_REF] Dyer | The solution of some random NP-hard problems in polynomial expected time[END_REF], while the mathematics literature also refers to it as the inhomogeneous random graph model [START_REF] Bollobás | The phase transition in inhomogeneous random graphs[END_REF]. This model has been used as a test bed for clustering and community detection algorithms used in a variety of contexts including social networks [START_REF] Mark | Random graph models of social networks[END_REF], protein-to-protein interaction networks [START_REF] Chen | Detecting functional modules in the yeast proteinprotein interaction network[END_REF], recommendation systems [START_REF] Linden | Amazon.com recommendations: itemto-item collaborative filtering[END_REF], medical prognosis [START_REF] Sørlie | Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications[END_REF],

DNA folding [START_REF] Cabreros | Detecting community structures in Hi-C genomic data[END_REF], image segmentation [START_REF] Shi | Normalized cuts and image segmentation[END_REF], and natural language processing [START_REF] Ball | Efficient and principled method for detecting communities in networks[END_REF].

In this section we focus on the dense stochastic block model with two communities, which we now define precisely.

Consider N individuals, each belonging to exactly one of two communities. We encode the individuals as elements of {1,...,N}, and represent the community structure using a vector

σ ∶= σ 1 ,...,σ N ∈ Σ N ∶= {-1,+1} N . (4.129)
For convenience, we have changed our notation of the hypercube {-1,+1} N from {±1} N to Σ N ; we will keep this new notation when we discuss spin glasses in Chapter 6. We understand that individuals i and j belong to the same community if and only if σ i = σ j . We sample the labels (σ i ) i⩽N independently from a Bernoulli distribution P 1 with probability of success p ∈ (0,1) and expectation m,

p ∶= P 1 {1} = P{σ i = 1} and m ∶= Eσ 1 = 2p -1. (4.130)
The assignment vector σ is thus distributed according to the product law

P N ∶= P ⊗N 1 , (4.131)
and the expected sizes of the communities are N p and N(1p). Using the assignment vector σ , we construct a random undirected graph G N = (G i j ) i, j⩽N with vertex set {1,...,N} by stipulating that an edge between node i and node j is present with conditional probability

P G i j = 1 σ ∶= ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ a N if σ i = σ j , b N if σ i ≠ σ j , (4.132)
for some a N ,b N ∈ (0,1), independently of all other edges. In other words, the probability that an edge is present between node i and node j depends only on whether or not the individuals i and j belong to the same community. To express (4.132) more succinctly, it is convenient to introduce the average and the gap of a N and b N ,

c N ∶= a N + b N 2 and ∆ N ∶= a N -b N 2 ∈ (-c N ,c N ), (4.133)
in such a way that

P G i j = 1 σ = c N + ∆ N σ i σ j . (4.134)
Our inference task is to try to reconstruct the community structure σ as best we can, given the observation of the network of interactions G N = (G i j ) i, j⩽N . This is illustrated in Figure 4.3. When p = 1 2, the symmetry between the two communities makes it clear that we can only hope to recover σ up to a change of sign. We may also consider the problem of recovering the matrix σ σ * , which encodes whether or not two individuals belong to the same community or not for each pair of individuals.

In the case when ∆ N ⩽ 0, it is more likely for an edge to be present between nodes in different communities, and the model is called disassortative. When ∆ N > 0, connections are more likely between individuals in the same community, and the model is termed assortative.

The recovery task in the stochastic block model can be interpreted in at least two different ways, often called exact recovery and detection. The exact recovery task aims to determine the regimes of a N and b N , or equivalently of c N and ∆ N , for which there exists an algorithm that completely recovers the two communities with high probability, up to a global change of sign. A necessary condition for exact recovery is that the random graph G N be connected; this makes exact recovery impossible in the sparse regime when the average degree of a node remains bounded as N diverges. The sharp threshold for exact recovery was obtained in [START_REF] Abbe | Exact recovery in the stochastic block model[END_REF][START_REF] Mossel | Consistency thresholds for the planted bisection model[END_REF], where it was shown that in the symmetric dense regime, p = 1 2, a N = alog(N) N and b N = blog(N) N, exact recovery is possible, and efficiently so, if and only if √ a -√ b ⩾ 2. On the other hand, the detection task is to construct a partition of the graph G N that is positively correlated with the assignment vector σ with high probability, possibly up to a global change of sign. This is the problem that we will focus on, in the dense regime where the average degree of a node diverges with N. More specifically, we will work under the two following assumptions on c N and the quantity

λ N ∶= N∆ 2 N c N (1 -c N ) . (4.135)
A1 The sequence (λ N ) N⩾1 converges to some value λ ⩾ 0.

A2 The sequence (Nc N (1c N )) N⩾1 diverges to infinity.

The second of these assumptions implies that the average degree of a node i,

Edeg(i) = (N -1)(c N + m∆ N ), (4.136)
diverges with N. The sparse regime, when the average degree of a node remains bounded even in the limit of large system size, is more challenging. In this case, identifying the regimes of c N and ∆ N for which the detection of the communities is possible has been achieved in [START_REF] Massoulié | Community detection thresholds and the weak Ramanujan property[END_REF][START_REF] Mossel | Reconstruction and estimation in the planted partition model[END_REF][START_REF] Mossel | A proof of the block model threshold conjecture[END_REF] (in a setting with two communities).

Concerning the precise identification of the limit mutual information between the community structure and the random graph G N , we refer the reader interested in the sparse disassortative regime to [START_REF] Abbe | Conditional random fields, planted constraint satisfaction, and entropy concentration[END_REF][START_REF] Coja-Oghlan | Information-theoretic thresholds from the cavity method[END_REF][START_REF] Decelle | Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications[END_REF], and the reader interested in the harder sparse assortative regime to [START_REF] Abbe | Stochastic block model entropy and broadcasting on trees with survey[END_REF][START_REF] Decelle | Asymptotic analysis of the stochastic block model for modular networks and its algorithmic applications[END_REF][START_REF] Dominguez | Mutual information for the sparse stochastic block model[END_REF][START_REF] Gu | Uniqueness of BP fixed point for the Potts model and applications to community detection[END_REF][START_REF] Varun Kanade | Global and local information in clustering labeled block models[END_REF][START_REF] Mossel | Belief propagation, robust reconstruction and optimal recovery of block models[END_REF][START_REF] Mossel | Local algorithms for block models with side information[END_REF][START_REF] Yu | Ising model on locally tree-like graphs: Uniqueness of solutions to cavity equations[END_REF].

As in the symmetric rank-one matrix estimation problem, we may be interested in studying the large-N limit of the minimal mean-square error for the recovery of the matrix σ σ * , that is,

mmse N ∶= 1 N E σ σ * -E[σ σ * G N ] 2 , (4.137)
or similarly for the vector σ itself. We will here focus on studying the asymptotic behaviour of the mutual information

I N ∶= 1 N E ˆΣN log dP σ G N dP N (σ ) dP σ G N (σ ) (4.138)
between the graph G N and the vector σ . The asymptotic behaviour of the minimal mean-square error in (4.137) can also be identified using similar ideas, in analogy with the results of Propositions 4.3 and 4.4 and (4.21); see also [START_REF] Deshpande | Asymptotic mutual information for the balanced binary stochastic block model[END_REF][START_REF] Lelarge | Fundamental limits of symmetric low-rank matrix estimation[END_REF]. Observing that

P G N = (G i j ) σ = σ = i< j (c N + ∆ N σ i σ j ) G i j (1 -c N -∆ N σ i σ j ) 1-G i j , (4.139)
we can use Bayes' formula to obtain the law of the signal σ conditionally on the observation of G N . We write it in the form of a Gibbs measure,

P σ = σ G N = (G i j ) = expH SBM N (σ )P N (σ ) ´ΣN expH SBM N (τ)dP N (τ) , (4.140)
for the Hamiltonian

H SBM N (σ ) ∶= i< j log 1 + ∆ N c N σ i σ j G i j 1 - ∆ N 1 -c N σ i σ j 1-G i j . (4.141)
We denote the associated free energy by

F SBM N ∶= 1 N Elog ˆΣN expH SBM N (σ )dP N (σ ). (4.142)
In the limit of large N, this free energy coincides with the mutual information (4.138), up to an additive constant.

Proposition 4.23. Under assumptions (A1) and (A2), the limits of the free energy (4.142) and of the mutual information (4.138) differ by an additive constant,

I N = λ 4 -F SBM N + o(1). (4.143)
Proof. The explicit form of the likelihood in (4.139) and the definition of the Hamiltonian in (4.141) imply that

I N = 1 N EH SBM N (σ ) -F SBM N . (4.144)
Since the coordinates of the assignment vector σ are i.i.d. the first term simplifies to

1 N EH SBM N (σ ) = 1 N ⋅ N 2 ⋅ EG 12 log 1 + ∆ N c N σ 1 σ 2 + E(1 -G 12 )log 1 - ∆ N 1 -c N σ 1 σ 2 . (4.145)
Averaging with respect to the randomness of G 12 conditionally on the randomness of the assignment vector σ reveals that this is equal to

N -1 2 c N E 1+ ∆ N c N σ 1 σ 2 log 1 + ∆ N c N σ 1 σ 2 + (1 -c N )E 1 - ∆ N 1 -c N σ 1 σ 2 log 1 - ∆ N 1 -c N σ 1 σ 2 , (4.146)
and Taylor expanding the logarithm gives

1 N EH SBM N (σ ) = N -1 4 ⋅ ∆ 2 N c N (1 -c N ) + O ∆ 3 N c 2 N + ∆ 3 N (1 -c N ) 2 . (4.147)
Recalling the definition of the constant λ N in (4.135), observing that c N ∈ (0,1) and remembering (4.144) reveals that

I N = N -1 N ⋅ λ N 4 -F SBM N + O N -1 N ⋅ λ 3 2 N Nc N (1 -c N ) . (4.148)
Invoking assumptions (A1) and (A2) completes the proof. ∎

To determine the limit of the free energy F SBM N

, we will follow [START_REF] Deshpande | Asymptotic mutual information for the balanced binary stochastic block model[END_REF][START_REF] Lelarge | Fundamental limits of symmetric low-rank matrix estimation[END_REF] and argue that it is asymptotically equivalent to the free energy of a rank-one estimation problem of just the form we have been studying in earlier sections of this chapter. To see this, we start by simplifying the expression of the Hamiltonian H SBM N and discarding some lower-order terms. Taylor expanding the logarithm in the definition of the Hamiltonian (4.141) reveals that

H SBM N (σ ) = HSBM N (σ ) + i< j G i j ∆ 2 N 2(1 -c N ) 2 - ∆ 2 N 2c 2 N - ∆ 2 N 2(1 -c N ) 2 + i< j O G i j ∆ 3 N c 3 N + (1 -G i j ) ∆ 3 N (1 -c N ) 3 (4.149)
for the Hamiltonian

HSBM N (σ ) ∶= i< j ∆ N c N (1 -c N ) G i j -c N σ i σ j . (4.150)
If we introduce the free energy

FSBM N ∶= 1 N E ˆΣN exp HSBM N (σ )dP N (σ ) (4.151)
associated with the Hamiltonian (4.150), then the equality (4.149) implies that

F SBM N = FSBM N + 1 N i< j E G i j ∆ 2 N 2(1 -c N ) 2 - ∆ 2 N 2c 2 N - ∆ 2 N 2(1 -c N ) 2 + i< j O EG i j ∆ 3 N c 3 N + (1 -EG i j ) ∆ 3 N (1 -c N ) 3 . (4.152)
Noticing that EG i j = c N + ∆ N m 2 and remembering the definition of λ N in (4.135), this becomes

F SBM N = FSBM N - N -1 N ⋅ λ N 4 + O N -1 N ⋅ λ 3 2 N Nc N (1 -c N ) . (4.153)
Leveraging the assumptions (A1) and (A2) shows that

F SBM N = FSBM N - λ 4 + o(1) (4.154)
so the free energies (4.142) and (4.151) are equal up to an additive constant. The free energy (4.151) starts to look more like the free energy (4.9) in the symmetric rank-one matrix estimation problem. Indeed, if we introduce the centred random variables

Gi j ∶= ∆ N c N (1 -c N ) G i j -c N -∆ N σ i σ j , (4.155)
then the Hamiltonian (4.150) may be written as

HSBM N (σ ) = i< j Gi j σ i σ j + λ N N σ i σ j σ i σ j . (4.156)
A direct computation shows that the random variables Gi j have variance

E G2 i j = λ N Nc N (1 -c N ) E 1 -c N -∆ N σ i σ j c N + ∆ N σ i σ j (4.157) = λ N N + O λ N N ⋅ λ N Nc N (1 -c N ) + λ N N . (4.158)
Recall that under our assumption (A2), the average degree of a node diverges as N tends to infinity. This gives some credence to the idea that in the limit of large system size, a sort of central limit theorem takes place and we may as well substitute the random variables ( Gi j ) i j by centred Gaussian random variables with the same variance. In other words, we may expect the free energy (4.151) to be asymptotically equivalent to the Gaussian free energy

F gauss N (λ ) ∶= 1 N Elog ˆΣN expH gauss N (λ ,σ )dP N (σ ) (4.159)
associated with the Hamiltonian

H gauss N (λ ,σ ) ∶= i< j λ N W i j σ j σ j + λ N σ i σ j σ i σ j .
As shown in Exercise 4.11, up to an additive constant, this Gaussian free energy is the free energy (4.9) in the symmetric rank-one matrix estimation problem,

F gauss N (λ ) = F ○ N λ 4 + λ 4 - λ 2N . (4.160)
To show rigorously that the free energies (4.151) and (4.159) are asymptotically equivalent up to an additive constant, and therefore that the free energy (4.142) in the stochastic block model is asymptotically equivalent to the free energy (4.9) in the symmetric rank-one matrix estimation problem for an appropriate choice of prior and signal-to-noise ratio t, we use an interpolation argument. To alleviate notation, we will keep the dependence on λ implicit. We proceed by interpolation. For each t ∈ [0,1], we define the interpolating Hamiltonian

H N,t (σ ) ∶= i< j √ t Gi j + √ 1 -t λ N W i j σ i σ j + t ⋅ λ N N + (1 -t) ⋅ λ N σ i σ j σ i σ j
and the interpolating free energy

F N (t) ∶= 1 N Elog ˆΣN expH N,t (σ )dP N (σ )
in such a way that by the fundamental theorem of calculus

FSBM N -F gauss N = F N (1) -F N (0) ⩽ sup t∈[0,1] F ′ N (t) . (4.163)
To compute this derivative, write ⟨⋅⟩ t for the average with respect to the Gibbs measure associated with the interpolating Hamiltonian H N,t , and observe that

F ′ N (t) = 1 2N √ t i< j E Gi j ⟨σ i σ j ⟩ t - √ λ 2N 3 2 √ 1 -t i< j EW i j ⟨σ i σ j ⟩ t + λ N -λ ⋅ 1 N 2 i< j E⟨σ i σ j σ i σ j ⟩ t . (4.164)
At this point, fix indices i < j and introduce the function F( Gi j ) = ⟨σ i σ j ⟩ t . A direct computation reveals that

∂ Gi j F = ⟨σ i σ j ∂ Gi j H N,t (σ )⟩ t -⟨σ i σ j ⟩ t ⟨∂ Gi j H N,t (σ )⟩ t = √ t 1 -⟨σ i σ j ⟩ 2 t , ∂ 2 Gi j F = -2 √ t⟨σ i σ j ⟩ t ⋅ ∂ Gi j F = -2t ⟨σ i σ j ⟩ t -⟨σ i σ j ⟩ 3 t ,
so the approximate Gaussian integration by parts formula in Exercise 4.6 gives

E Gi j ⟨σ i σ j ⟩ t = √ tE G2 i j 1 -E⟨σ i σ j ⟩ 2 t + O E Gi j 3 .
On the other hand, the Gaussian integration by parts formula as in (4.29) shows that

EW i j ⟨σ i σ j ⟩ t = √ 1 -t ⋅ λ N ⋅ 1 -E⟨σ i σ j ⟩ 2 t .
It follows by (4.164) that

F ′ N (t) = 1 2N i< j E G2 i j 1 -E⟨σ i σ j ⟩ 2 t - λ 2N 2 i< j 1 -E⟨σ i σ j ⟩ 2 t + O NE G12 3 + λ N -λ .
Together with (4.158) and (4.163), this implies that

FSBM N = F gauss N + O λ N ⋅ λ N Nc N (1 -c N ) + λ N N + NE G12 3 + λ N -λ .
Observing that

E G12 3 ⩽ ∆ 3 N c 3 N (1 -c N ) 3 E (1 -c N -∆ N σ i σ j )(c N + ∆ N σ i σ j ) ⩽ 4λ 3 2 N N Nc N (1 -c N )
, and remembering the assumptions (A1) and (A2) establishes (4.162) and completes the proof. ∎

Together with Theorem 4.9 on the limit free energy in the symmetric rank-one matrix estimation problem and Proposition 4.23 on the relationship between the mutual information and the free energy in the stochastic block model, this allows us to deduce the following variational representation for the limit free energy and limit mutual information in the stochastic block model. Theorem 4.25. Under assumptions (A1) and (A2), the limit of the free energy in the stochastic block model admits the variational representation

lim N→+∞ F SBM N = sup h⩾0 ψ(h) - h 2 λ , where ψ(h) ∶= Elog ˆΣ1 exp √ 2hσ z 1 + 2hσ σ 1 -h dP 1 (σ ), (4.165)
with z 1 a standard Gaussian random variable independent of any other randomness.

The limit mutual information in the stochastic block model admits the variational representation

lim N→+∞ I N = λ 4 -sup h⩾0 ψ(h) - h 2 λ . (4.166)
Exercise 4.11. Show that the Gaussian free energy (4.159) and the free energy (4.9) are related by

F gauss N (λ ) = F ○ N λ 4 + λ 4 - λ 2N . ( 4

.167)

Exercise 4.12. In the symmetric dense stochastic block model with p = 1 2 , show that

lim N→+∞ I N = inf h⩾0 λ 4 + h 2 λ + h -Elogcosh √ 2hz 1 + 2hσ 1 .

Chapter 5 Poisson point processes and extreme values

The main goal of this chapter is to lay important groundwork for our study of meanfield spin glass models in Chapter 6. The material we wish to present involves the consideration of Poisson point processes. In the first four sections of this chapter, we therefore give a self-contained introduction to this topic. This part culminates with limit theorems for extreme values of independent and identically distributed random variables, which in our view should be of interest to a very broad audience. We then turn to aspects that are specifically geared towards the analysis of mean-field spin glasses in the last three sections. In Section 5.1, we study the topological properties of the space of point measures on a locally compact and separable metric space. We then define point processes, which are random variables on the space of point measures, in Section 5.2. In Section 5.3, we introduce Poisson point processes and study their fundamental properties. We use these objects to determine the asymptotic behaviour of extremes of i.i.d. random variables in Section 5.4. We then proceed in Section 5.5 with a thorough analysis of a special Poisson point process known as the Poisson-Dirichlet point process. This process plays a fundamental role in the analysis of mean-field spin glasses, as a building block in the hierarchical construction of the Poisson-Dirichlet cascades discussed in Section 5.6. Finally, in Section 5.7, we discuss a characterization of Poisson-Dirichlet cascades in terms of certain distributional identities, and the fundamental role played by the notion of ultrametricity.

The space of point measures

A Poisson point process is a special type of point process, which in turn is a random variable taking values in a space of point measures. In this section, we study the basic topological properties of the space of point measures on a locally compact and separable metric space (S,d), endowed with its Borel σ -algebra B(S). In practice, we will mostly be interested in the case when S is R d or an open subset of R d equipped with the Euclidean distance, but this level of generality does not cause much additional difficulty, and is at times handy.

A measure λ on S is said to be locally finite if for every compact set K ⊆ S, we have λ (K) < +∞. A point measure on S is a measure λ on S that is locally finite and for which there exists a collection of points (x i ) i∈I with x i ∈ S satisfying

λ = i∈I δ x i , (5.1)
where δ x i denotes the Dirac measure at x i . The space of point measures on S is denoted by M δ (S). Notice that a point x i in the representation (5.1) could be repeated. In particular, measures of the form kδ x for some k ∈ N and x ∈ S belong to M δ (S). By Exercise 5.1, the index set I is always countable. We endow M δ (S) with the topology of vague convergence. This is the coarsest topology that makes the mappings

λ ↦ ˆS f dλ (5.2)
continuous for every f ∈ C c (S;R). Here C c (S;R) is the space of real-valued continuous functions with compact support in S. Specifically, this means that a sequence (λ n ) n⩾1 ⊆ M δ (S) converges to a measure λ ∈ M δ (S) if and only if for every con-

tinuous function f ∈ C c (S;R) of compact support, lim n→+∞ ˆS f dλ n = ˆS f dλ . (5.3)
Notice that this form of convergence does not preserve the total mass of a point measure. For instance, the Dirac mass δ n , seen as an element of M δ (R), converges to the null measure as n tends to infinity. We will mainly be concerned with random elements taking values in M δ (S) and their convergence. For the classical convergence theory discussed in Sections A.5 and A.6 of the appendix to apply, it will be useful to know that the topology of vague convergence on M δ (S) is metrizable. We will deduce this from the following result regarding the separability of the space of compactly supported continuous functions on S.

Lemma 5.1. There exists a countable family F ⊆ C c (S;R) with the property that, for any f ∈ C c (S;R) and ε > 0, there exists g ∈ F with

sup x∈S f (x) -g(x) ⩽ ε and suppg ⊆ x ∈ S d(x,supp f ) ⩽ ε . (5.4)
In particular, the space C c (S;R) is separable.

Proof. Let D be a countable and dense subset of S. We denote by F + the set of functions in C c (S;R ⩾0 ) that can be written in the form

x ↦ max y∈Y λ y (1 -kd(x,y)) + , (5.5)
where Y is a finite subset of D, (λ y ) y∈Y are non-negative rational numbers, k is a positive integer, and we recall that d is the distance function on S and that r + ∶= max(r,0). Up to decomposing a function f ∈ C c (S;R) into its positive and negative parts, and setting F to be the set of all functions that can be written as the difference between two functions in F + , it suffices to fix a non-negative function f ∈ C c (S;R ⩾0 ) as well as ε > 0 and to construct g ∈ F + satisfying (5.4). Moreover, up to replacing ε by ε f ∞ in (5.4), we can assume without loss of generality that f is bounded by one. Since f is supported on a compact set K and is continuous, it is uniformly continuous. Hence, there exists δ > 0 such that for every x,y ∈ S, we have f (x)f (y) ⩽ ε whenever d(x,y) ⩽ δ . Up to reducing δ if necessary, we can also assume that δ ⩽ ε. We let k be the smallest integer larger than δ -1 . For every y ∈ D, we pick λ y a rational number in the interval [ f (y)ε, f (y) + ε]. For every η ∈ (0,δ ), we denote by Y η a finite subset of D, made of points that are at distance at most η from K, such that every x ∈ K is at distance at most η from a point in Y η . The set Y η can be constructed by extracting a finite sub-cover from the open cover of the compact set K formed by open balls of radius η centred at the points in D.

We write g η for the function (5.5) with these choices of parameters, and with Y replaced by Y η . The function g η is Lipschitz continuous, with Lipschitz constant at most k + kε. For every x ∈ K, we can find y ∈ Y η such that d(x,y) ⩽ η ⩽ δ , and thus

f (x) -g η (x) ⩽ f (y) -g η (y) + ε + kη + kεη ⩽ 2ε + kη + kεη.
On the other hand, for x ∉ K but in the support of g η , there is y ∈ Y η with d(x,y) ⩽ k -1 ⩽ δ and g η (x) = λ y (1kd(x,y)) + .

It follows that

f (x) -g η (x) = g η (x) ⩽ λ y ⩽ f (y) + ε ⩽ f (x) + 2ε = 2ε.
Choosing η to be smaller than the minimum between ε k and 1 k, we have shown that for every x ∈ S, f (x)g η (x) ⩽ 4ε.

Notice also that every point in the support of g η is at distance at most η + k -1 ⩽ η + δ ⩽ 2δ ⩽ 2ε from a point in K. We have therefore constructed a function satisfying (5.5), up to replacing ε by 4ε. In order to conclude, it remains to verify that the support of the function g η is compact. In order to do so, it suffices to check that, for ε > 0 sufficiently small, the set K ε ∶= x ∈ S there exists y ∈ K with d(x,y) ⩽ ε (5.6) is compact. Since S is locally compact, we can cover K with open balls having compact closure, simply by picking an open ball having compact closure for each point in K. Since K is compact, we can extract a finite sub-cover of K. As soon as ε > 0 is smaller than the radius of the smallest ball in this finite sub-cover, we see that the set K ε is compact. This completes the proof. ∎

This result suggests that vague convergence on M δ (S) should be determined by the set F. If we denote by ( f n ) n⩾1 an enumeration of the countable set F, this would ensure that the metric ρ ∶ M δ (S) × M δ (S) → R ⩾0 defined by

ρ(λ ,λ ′ ) ∶= +∞ n=0 2 -n ´S f n dλ -´S f n dλ ′ 1 + ´S f n dλ -´S f n dλ ′ (5.7)
metrizes the vague topology on M δ (S). The fact that (5.7) defines a metric is the content of Exercise 5.4. Theorem 5.2. A sequence (λ n ) n⩾1 ⊆ M δ (S) converges vaguely to a point measure λ ∈ M δ (S) if and only if (5.3) holds for all functions f ∈ F. In particular, the topology induced by the metric in (5.7) is the vague topology on M δ (S).

Proof. The direct implication is immediate. Conversely, we assume that (5.3) holds for every f ∈ F, and fix f ∈ C c (S;R) as well as ε ∈ (0,1). Letting K be the support of f , we invoke Lemma 5.1 to find g ∈ F with sup x∈S f (x)g(x) ⩽ ε and suppg ⊆ K ε .

Here K ε denotes the ε-neighbourhood of K defined in (5.6). By the triangle inequality,

ˆS f dλ -ˆS f dλ n ⩽ ε λ (K ε ) + sup n⩾1 λ n (K ε ) + ˆS gdλ n -ˆS gdλ ,
so letting n tend to infinity and leveraging the assumption that (5.3) holds for g ∈ F reveals that

limsup n→+∞ ˆS f dλ -ˆS f dλ n ⩽ ε λ (K ε ) + sup n⩾1 λ n (K ε ) .
To bound this further, recall from the end of the proof of Lemma 5.1 that, decreasing ε if necessary, the set K ε is compact. Decreasing ε further if necessary, and invoking Lemmas A.7 and 5.1, we can find φ ∈ F with φ ⩾ 1 on K ε . Since ( ´S φ dλ n ) n⩾1 converges by assumption, it is uniformly bounded, so

M ∶= sup n⩾1 λ n (K ε ) ⩽ sup n⩾1 ˆS φ dλ n < +∞. (5.8) It follows that limsup n→+∞ ˆS f dλ -ˆS f dλ n ⩽ ελ (K ε ) + εM.
Recalling that K ε is compact for ε > 0 sufficiently small, and that λ (K ε ) is therefore finite, and observing that λ (K ε ) is non-increasing in ε, we obtain the desired result. ∎ Theorem 5.2 allows us to appeal to the basic convergence results recalled in Sections A.5 and A.6 of the appendix for random elements taking values in M δ (S) endowed with its Borel σ -algebra B(M δ (S)). In Exercise 5.2, we give a characterization of the Borel σ -algebra B(M δ (S)) as the smallest σ -algebra on M δ (S) which makes the evaluation map λ ↦ λ (A) measurable for every measurable set A ∈ B(S). In the next section, we will want to characterize the convergence of random elements on M δ (S) through their Laplace transform. To do this, it will be convenient to know what relatively compact sets in M δ (S) look like. Notice that the notions of compactness and sequential compactness in M δ (S) are equivalent by Theorem 5.2.

Lemma 5.3. A set M ⊆ M δ (S) is relatively compact if and only if for all f ∈ C c (S;R), sup λ ∈M ˆS f dλ < +∞.
(5.9)

Proof. For the direct implication, we take a relatively compact set M ⊆ M δ (S), and for each f ∈ C c (S;R) we define the continuous functional

T f ∶ M δ (S) → R by T f (λ ) ∶= ˆS f dλ .
Since M is compact, the set T f (M) ⊆ R is compact, and therefore bounded. This means that sup

λ ∈M ˆS f dλ ⩽ sup λ ∈M ˆS f dλ < +∞,
as desired.

For the converse implication, we assume that (5.9) holds for all f ∈ C c (S;R), and we fix a sequence (λ n ) n⩾1 ⊆ M. We aim to identify an element λ ∈ M δ (S) and a subsequence of (λ n ) n⩾1 that converges vaguely to λ . We decompose the rest of the proof into three steps.

Step 1: convergence on a given compact subset. In this step, we show that for any given compact set K ⊆ S, one can find λ ∈ M δ (S) and a subsequence (n(k)) k⩾1 such that for every f ∈ C c (S;R) with compact support in K,

lim k→+∞ ˆS f dλ n(k) = ˆS f dλ .
(5.10)

We denote by λ n K the restriction of λ n to K. Arguing as in the proof of Theorem 5.2, we see that the sequence (λ n (K)) n⩾1 must remain bounded. Up to the extraction of a subsequence, we can thus assume that it converges to some j ⩾ 0, and since the sequence (λ n (K)) n⩾1 only takes integer values, the subsequence is ultimately constant. That is, for n large enough and along the subsequence, we can write

λ n K = j i=1 δ x n,i ,
for some x n,1 ,...,x n, j ∈ K. Up to the extraction of a further subsequence, we can also assume that each of these points converges as n tends to infinity, and we denote by x 1 ,...,x j ∈ K their respective limits. We define λ ∶= ∑ j i=1 δ x i . Let f ∈ C c (S;R) be a continuous function with compact support in K. For every n sufficiently large,

ˆS f dλ n = j i=1 f (x n,i ),
so letting n tend to infinity along the subsequence reveals that

lim n→+∞ ˆS f dλ n = j i=1 f (x i ) = ˆS f dλ .
This is (5.10).

Step 2: covering S with compact sets. In order to patch together the different measures obtained in the previous step for different choices of the compact set K, we now build a countable collection U of open sets with compact closure which covers S. Since S is assumed to be separable, there exists a countable set χ ⊆ S that is dense in S. We denote by U the collection of all open balls with centre a point in χ, with radius a rational number, and whose closure is compact. The collection U is clearly countable; we claim that it covers S. To see this, let x ∈ S. Since S is locally compact, for r > 0 sufficiently small, the closed ball B r (x) is compact. Since χ is dense, one can find y ∈ χ whose distance to x is less than r 4. Any open ball centred at x and with radius some rational number in (r 4,r 2) contains x, and it belongs to U since its closure still lies in the compact ball B r (x).

Step 3: constructing λ . By the two previous steps and a diagonal extraction argument, we can find a subsequence (n(k)) k⩾1 and, for every U ∈ U, a measure λ U ∈ M δ (S) such that for every f ∈ C c (S;R) with compact support in U, we have

lim k→+∞ ˆS f dλ n(k) = ˆS f dλ U .
It is clear that the restrictions of the measures λ U and λ V to U ∩V must coincide. We can thus build a point measure λ ∈ M δ (S) such that for every U ∈ U, the restrictions of λ and λ U to U coincide. Since U covers S, for every f ∈ C c (S;R), we can find a finite cover of the support of f made of elements of U, say U 1 ,...,U n . Letting h 1 ,...,h n be an associated partition of unity as in Lemma A.8, we can write

ˆS f dλ n(k) = n i=1 ˆS h i f dλ n(k) ,
and since each h i f has compact support in U i , we conclude that

lim k→+∞ ˆS f dλ n(k) = n i=1 ˆS h i f dλ = ˆS f dλ .
The proof is thus complete. ∎ Exercise 5.1. Show that in the expression (5.1), the index set I must be countable.

Exercise 5.2. Show that, for every A ∈ B(S), the mapping

M δ (S) → R ∪ {+∞} λ ↦ λ (A) (5.11)
is measurable. Show that the smallest σ -algebra such that each of these mappings is measurable is the Borel σ -algebra of M δ (S).

Exercise 5.3. Let (λ n ) n⩾1 be a sequence of measures in M δ (S), and let λ be a locally finite measure on S such that, for every f ∈ C c (S;R), we have lim n→+∞ ˆS f dλ n = ˆS f dλ .

(5.12)

Show that λ must be a point measure.

Exercise 5.4. Show that the function in (5.7) defines a metric on M δ (S).

Exercise 5.5. Show that the space M δ (S) endowed with the metric (5.7) is complete.

Point processes

A point process on S is a random variable taking values in M δ (S), and it serves as a model for a random distribution of points in S. That is, a point process Λ on S is a measurable map from some abstract probability space (Ω,A,P) to M δ (S) equipped with its Borel σ -algebra B(M δ (S)). For each ω ∈ Ω, the weight assigned by the point measure Λ(ω) ∈ M δ (S) to any measurable set A ⊆ S gives the number of points Λ(ω)(A) ⊆ [0,+∞] in the support of Λ(ω) that belong to A. The law of the random process Λ is the probability measure P ○ Λ -1 on M δ (S). Two point processes Λ and Λ ′ have the same law if, for every bounded and measurable function

F ∶ M δ (S) → R, EF(Λ) = EF(Λ ′ ). (5.13)
A sequence (Λ n ) n⩾1 of point processes on S converges in law to a point process Λ on S if, for every bounded and continuous function F ∶ M δ (S) → R,

lim n→+∞ EF(Λ n ) = EF(Λ).
(5.14)

In the same spirit as Theorem A.23, the next two results simplify these criteria by showing that it suffices to consider Laplace transform functions of the form

F(λ ) ∶= exp -ˆS f dλ (5.15)
for functions f ranging in the set C c (S;R ⩾0 ). Notice that the quantity ´S f dλ is well-defined and finite for every f ∈ C c (S;R ⩾0 ) and λ ∈ M δ (S), since elements of M δ (S) are assumed to be locally finite.

Proposition 5.4. Two point processes Λ,Λ ′ on S have the same law if and only if, for every f ∈ C c (S;R ⩾0 ),

Eexp -ˆS f dΛ = Eexp -ˆS f dΛ ′ . (5.16)
Proof. The direct implication is immediate. Conversely, we assume that (5.16) holds for every f ∈ C c (S;R ⩾0 ). Consider the collection of sets of the form

λ ∈ M δ (S) ˆS f i d(λ -λ 0 ) < r i for all 1 ⩽ i ⩽ k ,
where k ∈ N, f 1 ,..., f k ∈ C c (S;R), r 1 ,...,r k > 0, and λ 0 ∈ M δ (S) are arbitrary. This collection is closed under finite intersections, and the smallest σ -algebra containing it is the Borel σ -algebra. By Dynkin's π-λ theorem (Theorem A.5), it therefore suffices to show that, for every f 1 ,..., f k ∈ C c (S;R), the random vectors

Y ∶= ˆS f i dΛ 1⩽i⩽k and Z ∶= ˆS f i dΛ ′ 1⩽i⩽k
have the same law. Decomposing each function into its positive and negative parts, and thus considering a vector of length 2k instead of k, we can without loss of generality assume that f 1 ,..., f k take non-negative values. Fix t ∈ R k ⩾0 , and observe that applying the assumption (5.16) to the function f ∶= ∑ k i=1 t i f i reveals that Ee -t⋅Y = Ee -t⋅Z . Invoking the uniqueness theorem for Laplace transforms in Theorem A. [START_REF] Auffinger | Complexity of random smooth functions on the high-dimensional sphere[END_REF] ). Here C ∞ c (S;R ⩾0 ) denotes the space of non-negative smooth functions with compact support in S. Indeed, this follows by a density argument similar to that in Theorem 5.2. In short, denoting by K the support of f , we first find K ε a compact enlargement of K, and argue as in (5.8) that we can make sup n⩾1 P{Λ n (K ε ) ⩾ M} arbitrarily small by taking M sufficiently large. We then conclude by observing that we can find a sequence of functions in C ∞ c (S;R ⩾0 ) with support in K ε that converges to f in the uniform norm. More generally, in the statement of Proposition 5.4, we can replace C c (S;R ⩾0 ) by any subset F ⊆ C c (S;R ⩾0 ) that satisfies the approximation property stated in Lemma 5.1.

We next aim to use Proposition 5.4 to argue that the convergence in law of a sequence of point processes can be assessed by looking only at the Laplace transform of the point processes. We start with a preliminary observation clarifying the notion of convergence in law of point processes with respect to the vague topology.

Lemma 5.6. A sequence (Λ n ) n⩾1 of point processes on S converges in law to a point process Λ on S if and only if, for every f ∈ C c (S;R ⩾0 ), the sequence ´S f dΛ n n⩾1 converges in law to ´S f dΛ.

Proof. The direct implication is immediate. To show the converse implication we first decompose every f ∈ C c (S;R) into its positive and negative parts, f + and f -, and assert the joint convergence in law of ´S f + dΛ n , ´S f -dΛ n n⩾1 towards ´S f + dΛ, ´S f -dΛ by appealing to Proposition A.24. In particular, we have that for every f ∈ C c (S;R), the sequence ´S f dΛ n n⩾1 converges in law to ´S f dΛ. We now proceed in two steps to show that (Λ n ) n⩾1 converges to Λ. First we argue that (Λ n ) n⩾1 is tight using Prokhorov's theorem, and then we prove that Λ is its only subsequential limit using Proposition 5.4. The result then follows from Lemma A.19.

Step 1: (Λ n ) n⩾1 is tight. We fix ε > 0 and denote by ( f k ) k⩾1 an enumeration of the countable set F in Lemma 5.1. Since ´S f k dΛ n n⩾1 converges in law to ´S f k dΛ as n tends to infinity, for every k ⩾ 1, it is possible to find a constant c k > 0 such that, for every n ⩾ 1,

P ˆS f k dΛ n > c k ⩽ ε 2 k .
This can be shown directly, or by appealing to the converse of Prokhorov's theorem (Theorem A.21) applied to the complete and separable space R. With the sequence (c k ) k⩾1 at hand, we introduce the set of point measures

M ∶= λ ∈ M δ (S) ˆS f k dλ ⩽ c k for all k ⩾ 1 .
An identical argument to that in Theorem 5.2 reveals that for every g ∈ C c (S;R),

sup λ ∈M ˆS gdλ < +∞.
It follows by the characterization of relatively compact sets in M δ (S) provided in Lemma 5.3 that M is relatively compact. Moreover, a union bound and the choice of the sequence (c k ) k⩾1 imply that, for every n ⩾ 1,

P Λ n ∉ M ⩽ P Λ n ∉ M ⩽ k⩾1 P ˆS f k dΛ n > c k ⩽ ε.
This shows that the sequence (Λ n ) n⩾1 is tight. By Prokhorov's theorem (Theorem A.20), every subsequence of (Λ n ) n⩾1 admits a further subsequence that converges in law.

Step 2: (Λ n ) n⩾1 admits a unique subsequential limit. Let (Λ n(k) ) k⩾1 be a subsequence of (Λ n ) n⩾1 converging in law to some point process Λ ′ . For every f ∈ C c (S;R ⩾0 ), we have

Eexp -ˆS f dΛ = lim k→+∞ Eexp -ˆS f dΛ n(k) = Eexp -ˆS f dΛ ′ .

Poisson point processes

A special class of point processes known as Poisson point processes play a particularly important role in the study of extremes of i.i.d. random variables, spin glasses, and many other areas. Given a locally finite measure µ on S, we say that a point process Λ on S is a Poisson point process on S with intensity measure µ if, for every integer n ⩾ 1 and pairwise disjoint measurable sets (A k ) 1⩽k⩽n ⊆ S, the random variables (Λ(A k )) 1⩽k⩽n are independent Poisson random variables, with

Λ(A k ) having mean µ(A k ) for 1 ⩽ k ⩽ n.
In this definition, we understand that Λ(A) is equal to +∞ almost surely if µ(A) = +∞. A Poisson point process is therefore a point process Λ for which we stipulate that the number of points in the support of Λ that fall in any measurable set A ⊆ S follows a Poisson distribution with mean µ(A).

We also require that these Poisson counts be independent for pairwise disjoint sets. It is not clear from the description of a Poisson point process that such an object should exist or be uniquely defined. We now describe how to generate a Poisson point process with intensity measure µ. For simplicity, we will assume that µ(S) is finite; otherwise, we can decompose µ into a sum of measures of finite total mass, construct independent Poisson point processes for each of these measures, and verify that the sum of these point processes is a Poisson point process with intensity measure µ. Let (X n ) n⩾1 be independent random variables with law

µ ∶= µ µ(S) , (5.18)
let N be a Poisson random variable with mean µ(S), and define

Λ ∶= N i=1 δ X i . (5.19)
The following proposition states that this is the unique Poisson point process with intensity measure µ.

Proposition 5.9 (Poisson point process). Let µ be a locally finite measure on S. There exists a Poisson point process with intensity measure µ; when µ is finite, the point process (5. [START_REF] Daniel | Spin-glass models of neural networks[END_REF]) is one such. Moreover, given a Poisson point process Λ with intensity measure µ and a measurable function f ∶ S → R ⩾0 ,

Eexp -ˆS f dΛ = exp - ˆS 1 -e -f dµ . (5.20)
In particular, any two Poisson point processes with intensity measure µ have the same law.

Proof. We start by proving the existence of a Poisson point process with intensity measure µ. We restrict our attention to the case when µ is finite; otherwise, we can decompose µ into a sum of measures of finite total mass, construct independent Poisson point processes for each of these measures, and verify that the sum of these point processes is a Poisson point process with intensity measure µ by following the argument we now present, with slightly heavier notation. Let Λ be the point process defined in (5.19). For every measurable function f ∶ S → R ⩾0 , we have

Eexp -ˆS f dΛ = Eexp - N i=1 f (X i ) = e -µ(S) +∞ n=0 (µ(S)) n n! ˆS e -f dµ n = exp -ˆS(1 -e -f )dµ .
In particular, if A 1 ,...,A k ⊆ S are disjoint measurable sets and t 1 ,...,t k ⩾ 0, then

Eexp - k i=1 t i Λ(A i ) = k i=1 exp -(1 -e -t i )µ(A i ) .
Together with the injectivity of the Laplace transform in Theorem A. [START_REF] Auffinger | Complexity of random smooth functions on the high-dimensional sphere[END_REF] for some pairwise disjoint measurable sets A i , and some non-negative constants t i ⩾ 0. The definition of a Poisson point process implies that

Eexp -ˆS f dΛ = k i=1 exp (e -t i -1)µ(A i ) = exp - ˆS 1 -e -f dµ ,
where we have used the explicit form of the moment generating function of a Poisson random variable. For a general measurable function f , we can find a sequence of functions of the form (5.21) that converge to f monotonically, see for instance Theorem 1.17 in [START_REF] Rudin | Real and complex analysis[END_REF], so (5.20) follows from the monotone convergence theorem. Invoking Proposition 5.4 establishes the uniqueness in law of the Poisson point process with intensity measure µ and completes the proof. ∎

The set of Poisson point processes is preserved by a number of transformations. To begin with, let us fix another locally compact separable metric space (S ′ ,d ′ ) endowed with its Borel σ -algebra B(S ′ ), and show that mapping the points in the support of a Poisson point process on S to the space S ′ again yields a Poisson point process. Given a Poisson point process Λ of the form (5.19) and a measurable map f ∶ S → S ′ , we write

f (Λ) ∶= N i=1 δ f (X i ) (5.22)
for the image point process on S ′ .

Proposition 5.10 (Mapping theorem). We fix a locally finite measure µ on S and a measurable map f ∶ S → S ′ such that µ ○ f -1 is a locally finite measure on S ′ . If Λ is a Poisson point process on S with intensity measure µ, then f (Λ) is a Poisson point process on S ′ with intensity measure µ ○ f -1 .

Proof. Proposition 5.9 implies that for any measurable function g ∈ C c (S ′ ;R ⩾0 ),

Eexp -ˆS gd f (Λ) = Eexp -ˆS g ○ f dΛ = exp - ˆS 1 -e -g○ f dµ .
Changing variables and combining Propositions 5.4 and 5.9 completes the proof. ∎

In addition to using a Poisson point process Λ on S to construct a Poisson point process on S ′ , we can also use it to construct a Poisson point process on the product space S × S ′ by appending a random coordinate to each point in its support. The simplest instance of this construction is to fix a sequence (Y i ) i⩾1 of i.i.d. random variables independent of the random variables (X i ) i⩾1 and N appearing in the representation (5.19) of Λ, and to define the point process

Λ * ∶= N i=1 δ (X i ,Y i ) .
(5.23)

Here we assume again that the measure µ is finite to simplify a bit the discussion, but this restriction can easily be lifted. If we write µ for the intensity measure of Λ and P Y for the law of Y 1 , then the explicit construction of a Poisson point process shows that Λ * is a Poisson point process on S × S ′ with intensity measure µ ⊗ P Y . More generally, if we use a transition function K ∶ S × B(S ′ ) → [0,1] to generate, independently for each i ∈ I, a point m(X i ) ∈ S ′ from the distribution K(X i ,⋅), and define the marked point process

Λ * ∶= N i=1 δ (X i ,m(X i )) (5.24)
on S × S ′ , then the marking theorem ensures that this is a Poisson point process with intensity measure µ * (s,s ′ ) ∶= µ(ds) ⊗ K(s, ds ′ ). We call a function K ∶ S × B(S ′ ) → [0,1] a transition function if, for each s ∈ S, the section K(s,⋅) is a probability measure on B(S ′ ), while, for each A ∈ B(S ′ ), the section K(⋅,A) is measurable on S.

The marking theorem is therefore a generalization of the i.i.d. construction in (5.23).

Proposition 5.11 (Marking theorem). If Λ is a Poisson point process on S with intensity measure µ, then the point process Λ * in (5.24) is a Poisson point process on S × S ′ with intensity measure µ * (s,s ′ ) ∶= µ(ds) ⊗ K(s, ds ′ ).

Proof. Fix a measurable function f ∈ C c (S × S ′ ;R ⩾0 ), and observe that

Eexp -ˆS f dΛ * = Eexp - N i=1 f X i ,m(X i ) = e -µ(S) +∞ n=0 (µ(S)) n n! ˆS Ee -f (s,m(s)) dµ(s) n = e -µ(S) +∞ n=0 (µ(S)) n n! ˆS ˆS′ e -f (s,s ′ ) K(s, ds ′ )dµ(s) n = exp - ˆS ˆS′ 1 -e -f (s,s ′ ) K(s, ds ′ )dµ(s) ,
where the last equality uses the fact that K(s,⋅) is a probability measure. Invoking Propositions 5.4 and 5.9 completes the proof. ∎

For later usage, we now record the following explicit formula translating averages involving Poisson point processes into averages involving their intensity measure.

Proposition 5.12 (Poisson average). If Λ is a Poisson point process on S with intensity measure µ, then for any measurable function f ∶ S → R ⩾0 , E ˆS f dΛ = ˆS f dµ.

(5.25)

Proof. Approximating f by a sequence of simple functions as in Proposition 5.9, it suffices to consider the case when f ∶ S → R ⩾0 is of the form

f ∶= k i=1 t i 1 A i
for some pairwise disjoint measurable sets A i and non-negative constants

t i ∈ R ⩾0 .
The definition of a Poisson point process implies that

E ˆS f dΛ = k i=1 t i EΛ(A i ) = k i=1 t i µ(A i ) = ˆS f dµ,
as required. ∎ Exercise 5.6. Let (Λ n ) n⩾1 be a sequence of point processes that converges in law to a Poisson point process Λ with intensity measure µ, and fix a measurable set A ⊆ S with µ(∂ A) = 0. Show that the sequence of random variables (Λ n (A)) n⩾1 converges in law to Λ(A).

Extremes of i.i.d. random variables

We now analyze the asymptotic behaviour of extreme values of i.i.d. random variables, which is most conveniently phrased in terms of the convergence to a Poisson point process. We view this as a fundamental result in probability theory, essentially on par with the central limit theorem, and we hope that the reader will appreciate to find a brief and self-contained presentation of this result. It will also inform our analysis of spin glasses, as will be discussed in the next sections and chapter. Given a sequence (X n ) n⩾1 of i.i.d. random variables, we denote their running maximum by M n ∶= max 1⩽k⩽n X k .

(5.26)

The most basic question in the study of extreme values is whether there exist sequences (a n ) n⩾1 ⊆ R >0 and (b n ) n⩾1 ⊆ R such that the normalized running maximum,

a -1 n (M n -b n ), (5.27)
converges in law to a non-trivial limit. Under suitable assumptions on the law of the random variables, it is elementary to obtain the answer to this question. Poisson point processes allow us to extend this result and capture the joint convergence in law of the largest values taken by the sequence (X k ) k⩽n , in the limit of large n.

The next proposition makes this precise in the case of random variables with a polynomial tail distribution.

Proposition 5.13 (Polynomial tail). Let (X n ) n⩾1 be i.i.d. random variables, and suppose that there exist c > 0,ζ > 0 such that, as x tends to infinity,

P{X 1 ⩾ x} ∼ c x ζ .
(5.28)

Then the point process

Λ n ∶= n k=1 δ n -1 ζ X k (5.29)
converges in law on R >0 to the Poisson point process with intensity dµ(x) ∶= cζ x ζ +1 dx. In Proposition 5.13, and similarly throughout, we say for convenience that a sequence of point processes converges in law on R >0 to mean that it converges in law in the sense of the vague topology on M δ (R >0 ). We implicitly understand that we have equipped R >0 with the Euclidean topology. One may take issue with the fact that the definition of Λ n may involve Dirac masses at locations outside of R >0 if X 1 can take values in (-∞,0]. These Dirac masses can be discarded if desired; in any case, they are not seen by the topology of M δ (R >0 ), so their presence is irrelevant to the question at hand. The display in (5.28) means that x ζ P{X 1 ⩾ x} converges to c as x tends to infinity.

Proof. By Proposition 5.7, it suffices to investigate the convergence of the Laplace transform of Λ n , and by Remark 5.8 it suffices to verify (5.17) for a given smooth and compactly supported function

f ∈ C ∞ c (R >0 ;R ⩾0 ). The independence of (X k ) 1⩽k⩽n implies that Eexp - ˆ+∞ 0 f dΛ n = Eexp - n k=1 f n -1 ζ X k = Eexp -f n -1 ζ X 1 n .
The slight difficulty of this proof is that the assumption (5.28) concerns the cumulative distribution function of X 1 , while it would, for instance, be easier to directly have access to an assumption on the density of X 1 . We resolve this through integration by parts,

E 1 -exp -f n -1 ζ X 1 = E ˆ+∞ 0 f ′ (y)e -f (y) 1 {y⩽n -1 ζ X 1 } dy = ˆ+∞ 0 f ′ (y)e -f (y) P X 1 ⩾ n 1 ζ y dy.
The first equality leverages the fact that f (0) = 0. We now fix ε > 0, and observe that by assumption (5.28), there exists x ε > 0 such that for every x ⩾ x ε ,

x ζ P{X 1 ⩾ x} -c ⩽ ε. (5.30)
Recalling also that f has compact support in R >0 , we see that, for n = n(ε) sufficiently large, we have ˆ+∞

0 f ′ (y)e -f (y) P X 1 ⩾ n 1 ζ y dy - c n ˆ+∞ 0 f ′ (y)e -f (y) y -ζ dy ⩽ ε n ˆ+∞ 0 f ′ (y) e -f (y) y -ζ dy.
The last integral is finite, and can be absorbed up to a reparametrization of ε > 0. We can now "undo" the integration by parts, ˆ+∞

0 f ′ (y)e -f (y) y -ζ dy = ˆ+∞ 0 f ′ (y)e -f (y) ˆ+∞ 0 ζ x ζ +1 1 {x⩾y} dxdy = ˆ+∞ 0 ˆx 0 f ′ (y)e -f (y) dy ζ x ζ +1 dx = ˆ+∞ 0 (1 -e -f (x) ) ζ x ζ +1 dx.
We have thus shown that, for every ε > 0 and every n sufficiently large,

Eexp -f n -1 ζ X 1 -1 - c n ˆ+∞ 0 (1 -e -f (x) ) ζ x ζ +1 dx ⩽ ε n .
In other words, we have the asymptotic expansion as n tends to infinity

Eexp -f n -1 ζ X 1 = 1 - c n ˆ+∞ 0 1 -e -f (x) ζ x ζ +1 dx + o 1 n . (5.31)
It thus follows that

lim n→+∞ Eexp -f n -1 ζ X 1 n = exp - ˆ+∞ 0 1 -e -f (x) cζ
x ζ +1 dx . By Proposition 5.9, the latter expression is the Laplace transform of a Poisson point process on R >0 with intensity dµ(x) = cζ x ζ +1 dx. This completes the proof. ∎

Since ´+∞ 1

x -(ζ +1) dx < +∞, the limit Poisson point process appearing in Proposition 5.13 contains only a finite number of points in the interval [1,+∞). Heuristically, it thus makes sense to think of Proposition 5.13 as capturing the law of the largest values of (X k ) k⩽n . However, Proposition 5.13 actually falls short of allowing us to assert this; in particular, one cannot infer the asymptotic behaviour of the running maximum M n from Proposition 5.13 as stated. The reason is that the topology of M δ (R >0 ) does not allow us to test for properties that require "observing" the point process arbitrarily far away towards infinity; for instance, if we add a Dirac mass at position n to Λ n , then this does not affect the convergence of Λ n in M δ (R >0 ). Fortunately, it is easy to amend for this shortcoming by choosing a topology more adapted to our needs. Indeed, we can compactify the interval R >0 at +∞ to obtain the set (0,+∞], equipped for instance with the distance d(x,y) ∶= e -xe -y , (5.32)

with the understanding that e -∞ = 0. Writing dµ(x) ∶= cζ x ζ +1 dx, we note that the Poisson point process with intensity µ appearing in Proposition 5.13 is a valid point process in M δ ((0,+∞]). Indeed, since µ([1,+∞]) = µ([1,+∞)) < +∞, the measure µ is locally finite also on (0,+∞], and likewise for the Poisson point process. We can strengthen the statement of Proposition 5.13 so that it is now stated with respect to the more demanding topology of M δ ((0,+∞]). Proposition 5.14. Let (X n ) n⩾1 be i.i.d. random variables, and suppose that there exist c > 0, ζ > 0 such that (5.28) holds as x tends to infinity. The point process Λ n in (5.29) converges in law on (0,+∞] to the Poisson point process with intensity dµ(x) ∶= cζ x ζ +1 dx. Proof. The proof is almost identical to that of Proposition 5.13. The only difference is that the set C c ((0,+∞];R ⩾0 ) contains functions that may not have compact support in R >0 , but instead converge to some limit at +∞. As before, it suffices to verify (5.17) for functions f ∈ C ∞ c ((0,+∞];R ⩾0 ), and one can also impose that f is constant over [M,+∞] for M is sufficiently large, by density. The proof of Proposition 5.13 then applies without further modification. ∎ Propositions 5.13 and 5.14 can be refined so that we also learn about the index at which a given extreme value is occurring; see Exercise 5.7.

To further clarify that Proposition 5.14 indeed captures the asymptotic behaviour of the extremes of the sequence (X n ) n⩾1 , we now deduce from it the asymptotic behaviour of the k largest values of (X ) ⩽n , for each fixed k. To state this result, we first make a simple observation concerning the Poisson point process Λ appearing in Proposition 5.14. We recall that the Poisson point process Λ has only a finite number of points falling in the interval [1,+∞), so we can enumerate its support decreasingly. Moreover, since ´+∞ 0

x -(ζ +1) dx is infinite, the support of Λ is infinite almost surely. So with probability one, there exist

u 1 ⩾ u 2 ⩾ ⋯ such that Λ = ∑ +∞ n=1 δ un .
Proposition 5.15. Let (X n ) n⩾1 be i.i.d. random variables, and suppose that there exist c > 0, ζ > 0 such that (5.28) holds as x tends to infinity. Denote by X 1,n ⩾ X 2,n ⩾ ⋯ ⩾ X n,n the decreasing ordering of the sequence (X k ) 1⩽k⩽n . Let Λ be a Poisson point process with intensity dµ(x) ∶= cζ x ζ +1 dx, and let u 1 ⩾ u 2 ⩾ ⋯ be such that Λ = ∑ +∞ n=1 δ un . For each integer k ⩾ 1, the vector

n -1 ζ X 1,n ,X 2,n ,...,X k,n (5.33)
converges in law to (u 1 ,...,u k ) as n tends to infinity.

Proof. We introduce the set

C ∶= λ ∈ M δ ((0,+∞]) λ = +∞ n=1
δ yn for a strictly decreasing sequence (y n ) n⩾1 ⊆ R >0 of point processes with no repeated points in their support. Recall that every measure in M δ ((0,+∞]) must be locally finite, and in particular, every λ ∈ M δ ((0,+∞]) has a finite number of points in [1,+∞]. Fix a positive integer k, and denote by F ∶ M δ ((0,+∞]) → R k the mapping such that F(λ ) outputs the vector of the k largest elements in the support of λ ; while this will be irrelevant for our purposes, we can decide to complete F(λ ) with zeros in case λ has fewer than k elements in its support. The key step of the proof is to observe that the mapping F is continuous at every λ ∈ C. In other words, if (λ n ) n⩾1 ⊆ M δ ((0,+∞]) converges to some λ ∈ C, then it must be that F(λ n ) converges to F(λ ). Indeed, let (y n ) n⩾1 ⊆ R >0 be a strictly decreasing sequence with λ = ∑ , with k > 0 sufficiently large, it is not difficult to conclude that the largest point in λ n converges to the largest point in λ . We can repeat the procedure recursively, choosing b = y 2 ± k -1 , then b = y 3 ± k -1 , and so on, thereby concluding that (F(λ n )) n⩾1 converges to F(λ ) as claimed. At this point, recall the notation Λ n in (5.29). Since the intensity measure dµ(x) ∶= cζ x ζ +1 dx does not contain any atoms, there are no repeated points in Λ, and the point process Λ thus belongs to C with probability one. By Proposition 5.14 and Exercise A.10, we deduce that F(Λ n ) converges in law to F(Λ), as desired.

∎

Recall the definition of the random variable M n in (5.26). As a simple consequence of Proposition 5.15, which can also be established by a very direct and elementary computation, we obtain the convergence in law of n -1 ζ M n . The limit law is characterized by the fact that, for every x > 0,

lim n→+∞ P M n ⩽ n 1 ζ x = exp -c ˆ+∞ x ζ y ζ +1 dy = exp -cx -ζ .
(5.34)

A random variable whose cumulative distribution function is given, for x > 0, by

x ↦ exp(-x -ζ ) is called a Fréchet law of parameter ζ > 0. For i.i.d. random variables (X n ) n⩾1 , in order that there exist sequences (a n ) n⩾1 ⊆ R >0 and (b n ) n⩾1 ⊆ R such that the normalized running maximum a -1 n (M n -b n )
converges in law to a Fréchet law of parameter ζ > 0, it is necessary and sufficient that, for every t > 0, we have

lim x→+∞ P{X 1 ⩾ tx} P{X 1 ⩾ x} = t -ζ , (5.35)
see for instance Theorem 1.6.2 in [START_REF] Leadbetter | Extremes and related properties of random sequences and processes[END_REF]. In fact, under this assumption, we can impose that b n = 0 and ζ log(a n ) ∼ log(n), and show analogues of Propositions 5.13, 5.14, and 5.15, provided that we re-scale the random variables by a -1 n in place of n -1 ζ . The proof can be obtained by combining the arguments we have seen with those in the proof of Theorem 1.6.2 in [START_REF] Leadbetter | Extremes and related properties of random sequences and processes[END_REF].

Besides Fréchet laws, there are two other possible classes of limit laws for the re-scaled maximum of i.i.d. random variables. The first class is associated with random variables with a light tail, such as exponential or Gaussian random variables, or the logarithm of a random variable satisfying (5.28). For simplicity, we only give the example of Gaussian random variables here, but the reader can work out other examples as well.

Proposition 5.16 (Light tail, example of Gaussians). Let (X n ) n⩾1 be independent standard Gaussian random variables, and let a n ∶= 2lognloglognlog(4π) (5.36)

The point process

Λ n ∶= n k=1
δ an(X k -an) (5.37) converges in law on R to a Poisson point process with intensity dµ(x) ∶= e -x dx.

Proof. Fix a function f ∈ C c (R;R ⩾0 ) of compact support, and observe that

Eexp -ˆR f dΛ n = Eexp - n k=1 f a n (X k -a n ) = 1 2πa 2 n ˆR e -f (x) exp - (x + a 2 n ) 2 2a 2 n dx n = exp nlog 1 - 1 2πa 2 n ˆR 1 -e -f (x) exp - (x + a 2 n ) 2 2a 2 n dx .
Using the definition of the constant a n in (5.36) and the fact that f is of compact support reveals that for x in the support of f ,

n 2πa 2 n exp - (x + a 2 n ) 2 2a 2 n = n 4π log(n) exp -x - 1 2 a 2 n + o(1) = e -x + o(1).

It follows by a Taylor expansion of the logarithm that

Eexp -ˆR f dΛ n = exp -ˆR 1e -f (x) e -x dx + o(1).

Invoking Propositions 5.4 and 5.9 completes the proof. ∎

As before, we can upgrade this statement by compactifying R at +∞, and then work out the corresponding version of Proposition 5.15. The last possible class of limit laws concerns random variables whose support is bounded to the right, and with a probability to fall close to the top of the support that decays in a power-law fashion. Examples include the uniform law on an interval, or the law of -1 X 1 for X 1 satisfying (5.28).

Proposition 5.17 (Polynomial tail near a point). Let (X n ) n⩾1 be i.i.d. random variables, and suppose that there exist c > 0, α > 0 such that, as x < 0 tends to 0,

P{X 1 ⩾ x} ∼ c x α .
(5.38)

Then the point process

Λ n ∶= n k=1 δ n 1 α X k (5.39)
converges in law on R ⩽0 to a Poisson point process with intensity dµ(x) ∶= cα x α-1 dx.

Proof. For a change, we prove this by appealing to Proposition 5.14 and the mapping theorem (Proposition 5.10). Letting x increase to zero in (5.38) shows that X 1 takes values in R <0 , so the random variable Y 1 ∶= -1 X 1 is well-defined and takes values in R >0 with probability one. Moreover, as x tends to infinity,

P{Y 1 ⩾ x} = P{X 1 ⩾ -1 x} ∼ c x α .
It follows by Proposition 5.14 that the point process

n k=1 δ n -1 α Y k
converges in law on (0,+∞] to a Poisson point process with intensity dν(x) ∶= cα x α+1 dx. Together with the mapping theorem applied to the function f (x) ∶= -1

x , this implies that the point process

Λ n ∶= n k=1 δ f (n -1 α Y k ) = n k=1 δ n 1 α X k converges in law on R ⩽0 to a Poisson point process with intensity µ ∶= ν ○ f -1 . Observe that for any 0 < a < b, µ([a,b]) = ˆ-1 b -1 a cα x α+1 dx = ˆb a cα(-y) α+1 dy y 2 = ˆb a cα y α-1 dy,
where we have used the change of variables y = -1 x. Invoking Propositions 5.4 and 5.9 completes the proof. ∎

As in Proposition 5.15, one can also show that the k largest values of Λ n converge in law to the k largest values of the limit point process.

Exercise 5.7. Under the assumptions of Proposition 5.13, show that the point process (5.41)

Λ n ∶= +∞ k=1 δ (k n, n -1 ζ X k ) ( 5
The goal of this exercise is to show that for each fixed k, after normalization by ∑ n =1 X , the ordered k largest values among (X ) ⩽n converge to (v 1 ,...,v k ). Although we will not show it, we mention that this is equivalent to the statement that the point process

n =1 δ X (∑ n i=1 X i )
converges in law on R >0 to ∑ +∞ n=1 δ vn = ´δx ´y dΛ(y) dΛ(x).

(ii) Show that, for every a > 0 sufficiently large, we have

E X 1 1 {X 1 ∈[a,2a]} ⩽ 2a 1-ζ .
(5.42)

(iii) Deduce that lim ε→0 limsup n→+∞ 1 n 1 ζ n k=1 E X k 1 {X k ⩽εn 1 ζ } = 0.
(

(iv) Use this to show that for every ε > 0, (v) Denoting by X 1,n ⩾ X 2,n ⩾ ⋯ ⩾ X n,n the decreasing ordering of the sequence (X k ) 1⩽k⩽n , conclude that for every integer k ⩾ 1, the vector

X 1,n ∑ n =1 X ,..., X k,n ∑ n =1 X (5.45)
converges in law to (v 1 ,...,v k ). It may be helpful to refer to Exercises A.10 and A.11.

Poisson-Dirichlet processes

In the context of mean-field spin glasses, the energies attributed to different configurations of the system are not independent random variables. However, one might imagine that, possibly after clumping together neighbouring configurations in a suitable way, we ultimately end up with Gibbs weights whose extreme values resemble the sequence (v n ) n⩾1 appearing in Exercise 5.7. This intuition turns out to be valid. For a toy model of spin glasses called the random energy model, in which we assume that the energy values of different configurations are independent, one can more readily justify this intuition, as will be explained in more detail in Section 6.3.

In fact, we will ultimately aim to capture not only the asymptotic behaviour of the largest weights of the Gibbs measure, but also how the configurations associated with these extreme values are organized in space. Remarkably, this richer structure is still described by a canonical object called a Poisson-Dirichlet cascade (or also a Ruelle probability cascade, after [START_REF] Ruelle | A mathematical reformulation of Derrida's REM and GREM[END_REF]). This object is built from a hierarchy of Poisson point processes as those appearing in Proposition 5.13. In this section, in order to prepare for the study of Poisson-Dirichlet cascades and their universality, we delve more deeply into the analysis of this Poisson point process.

The Poisson-Dirichlet point process with parameter ζ > 0 is the Poisson point process Λ on R >0 with intensity measure

dµ(x) ∶= ζ x ζ +1 dx.
(5.46)

For each ε > 0, the Poisson random variable Λ[ε,+∞) has mean µ[ε,+∞), so the support of Λ has finitely many points in the interval [ε,+∞). Since µ(R >0 ) is infinite, the support of Λ has infinitely many points. We can thus order the random support of Λ into an infinite decreasing sequence (u n ) n⩾1 , so that this Poisson point process can be represented as

Λ ∶= +∞ n=1 δ un .
(5.47)

We will refer to the process (u n ) n⩾1 as the ordered Poisson-Dirichlet point process with parameter ζ > 0. In order to be able to normalize this sequence of positive weights into a probability measure, we need to restrict ourselves to the range ζ ∈ (0,1) for which the sequence (u n ) n⩾1 is summable, as shown in the next proposition. For future reference, it will also be convenient to record that for ζ in this range, the moments of order less than ζ of the sum are finite.

+∞ n=1 u n = ˆR>0 xdΛ(x) = ˆ(0,1] xdΛ(x) + ˆ(1,+∞) xdΛ(x).
The second of these integrals is finite since the support of Λ only has finitely many points in the interval (1,+∞). If ζ < 1, then the second integral is almost surely finite since, by Proposition 5.12, its expectation is finite, as

E ˆ(0,1] xdΛ(x) = ˆ(0,1] xdµ(x) = ˆ1 0 ζ x ζ dx. For ζ ⩾ 1, we write ˆ(0,1] xdΛ(x) = +∞ n=0 ˆ(2 -(n+1) ,2 -n ] xdΛ(x) ⩾ +∞ n=0 2 -(n+1) Λ(2 -(n+1) ,2 -n ].
(5.49)

The random variables Λ(2 -(n+1) ,2 -n ] n⩾0 are independent Poisson random variables, and the mean of

Λ(2 -(n+1) ,2 -n ] is ˆ2-n 2 -(n+1) ζ x ζ +1 dx = 2 ζ n (2 ζ -1).
In particular, when ζ ⩾ 1, we can find a constant c > 0 such that for every n ⩾ 0,

P 2 -(n+1) Λ(2 -(n+1) ,2 -n ] ⩾ c ⩾ c.
A Borel-Cantelli argument thus ensures that the integral on the left side of (5.49) diverges almost surely in this case.

Assuming that ζ ∈ (0,1), a similar argument can be used to bound the moments (5.48). Indeed, using that x a ⩽ 1 + x and (x + y) a ⩽ x a + y a for x,y ⩾ 0, we see that

E +∞ n=1 u n a ⩽ E ˆ(0,1] xdΛ(x) a + E ˆ(1,+∞) xdΛ(x) a ⩽ 1 + E ˆ(0,1] xdΛ(x) + E ˆ(1,+∞) x a dΛ(x) ⩽ 1 + ˆ1 0 ζ x ζ dx + ˆ+∞ 1 ζ x 1+ζ -a dx.
Leveraging the assumption that a < ζ completes the proof. ∎ Henceforth, unless otherwise stated, whenever we speak of a Poisson-Dirichlet point process, we implicitly assume that ζ ∈ (0,1). Relying on the summability of (u n ) n⩾1 , we can define the Poisson-Dirichlet process (v n ) n⩾1 with parameter ζ ∈ (0,1) by normalizing the sequence

(u n ) n⩾1 , v n ∶= u n ∑ +∞ k=1 u k .
(5.50)

We think of this process as giving the weights of a Gibbs measure supported on the natural numbers, and we write ⟨⋅⟩ for its associated average, with α denoting the canonical random variable on this space. We also denote by (α ) ⩾1 independent copies, or replicas, of the random variable α under this measure. This means that for any k ⩾ 1 and any bounded function

f ∶ N k → R, ⟨ f (α 1 ,...,α k )⟩ ∶= j 1 ,..., j k ⩾1 f ( j 1 ,..., j k )v j 1 ⋯v j k .
(5.51)

We now prove a fundamental invariance property of the ordered Poisson-Dirichlet point process that will allow us to explicitly compute free energy functionals associated with the Gibbs average (5.51). This invariance property will also lead to deep distributional identities that characterize the Gibbs measures whose ordered weights are given by a Poisson-Dirichlet process. (5.53)

have the same law.

Proof. Let µ be as in (5.46), and let ν be the law of

(X 1 ,Y 1 ) on R >0 × R d . Since (X n ,Y n ) n⩾1 are i.i.
(µ ⊗ ν) ○ f -1 (A × B) = ˆR>0 ×R d ˆ+∞ 0 1 {ux∈A} 1 {y∈B} ζ u -1-ζ dudν(x,y) = ˆR>0 ×R d x ζ 1 {y∈B} ˆ+∞ 0 1 {v∈A} ζ v -1-ζ dvdν(x,y) = µ(A) ˆR>0 ×R d x ζ 1 {y∈B} dν(x,y) = EX ζ 1 (µ ⊗ ν ζ )(A × B).
Similarly, the marking theorem and the mapping theorem applied with the function g(u) ∶= (EX

ζ 1 ) 1 ζ u imply that +∞ n=1 δ ((EX ζ 1 ) 1 ζ un,Y ′ n )
is a Poisson point process with intensity measure (µ ○g -1 )⊗ν ζ . Given a measurable set A ⊆ R >0 , a change of variables reveals that

µ ○ g -1 (A) = ˆ+∞ 0 1 {(EX ζ 1 ) 1 ζ u∈A} ζ u -1-ζ du = EX ζ 1 ˆ+∞ 0 1 {v∈A} ζ v -1-ζ dv = EX ζ 1 µ(A).
Since the Poisson point processes in (5.53) have the same intensity measure, they have the same law by Proposition 5.9. This completes the proof. ∎ Proposition 5.20. Let (X n ) n⩾1 be i.i.d. random variables on R >0 that are independent of the ordered Poisson-Dirichlet point process (u n ) n⩾1 with parameter

ζ ∈ (0,1). If EX ζ 1 < +∞, then Elog⟨X α ⟩ = 1 ζ logEX ζ 1 .
(5.54)

Proof. Theorem 5.19 implies that

Elog +∞ n=1 u n X n = Elog +∞ n=1 EX ζ 1 1 ζ u n = 1 ζ logEX ζ 1 + Elog +∞ n=1 u n . (5.55)
Remembering the definition of the Poisson-Dirichlet process in (5.50) and of the Gibbs measure in (5.51) completes the proof. ∎

This result is slightly generalized in Exercise 5.9. Its counterpart for the Poisson-Dirichlet cascades discussed in the next section will be used to determine the initial condition for the infinite-dimensional Hamilton-Jacobi equation arising in the context of the spin-glass models studied in the next chapter. Another consequence of the invariance property in Theorem 5.19 is the Bolthausen-Sznitman invariance for the ordered Poisson-Dirichlet point process.

Lemma 5.21 (Bolthausen-Sznitman invariance for PDP [START_REF] Bolthausen | On Ruelle's probability cascades and an abstract cavity method[END_REF]). Let (g n ) n⩾1 be a sequence of independent centred Gaussian random variables with variance v > 0, independent of the ordered Poisson-Dirichlet point process (u n ) n⩾1 with parameter ζ ∈ (0,1). For every t ∈ R, the Poisson point processes In the context of mean-field spin glasses, we think of each term u n as representing the asymptotic value of the exponential of the energy of a particular configuration; and since this energy is a Gaussian random variable, it is relatively natural to perturb u n by multiplying it with the exponential of a Gaussian random variable as done in Lemma 5.21.

Proof of Lemma 5.21. Consider the sequence of random variables (X n ,Y n ) n⩾1 with values in R >0 × R defined by

(X n ,Y n ) ∶= exp(t(g n -tvζ 2)),g n -tvζ .
Using the formula for the moment generating function of a standard Gaussian random variable reveals that

EX ζ 1 = exp(-t 2 ζ 2 v 2)Eexp(tζ g 1 ) = 1.
It follows by a change of variables that the measure ν ζ defined by (5.52) is given by

ν ζ (B) = 1 √ 2πv ˆR e tζ (x-tvζ 2) 1 {x-tvζ ∈B} e -x 2 2v dx = 1 √ 2πv ˆR 1 {x-tvζ ∈B} e -(x-tζ v) 2 2v dx = P{g 1 ∈ B}.
This means that ν ζ is the Gaussian distribution on R with variance v. Invoking Theorem 5.19 completes the proof. ∎

The Bolthausen-Sznitman invariance for the ordered Poisson-Dirichlet point process can be used to establish the Ghirlanda-Guerra identities for the Gibbs measure (5.51). These identities were introduced in [START_REF] Ghirlanda | General properties of overlap probability distributions in disordered spin systems. Towards Parisi ultrametricity[END_REF] (see also [START_REF] Aizenman | On the stability of the quenched state in mean-field spin-glass models[END_REF][START_REF] Guerra | About the overlap distribution in mean field spin glass models[END_REF]) in the context of mean-field spin glasses. It turns out that these identities characterize the law of the weights of this Gibbs measure, as shown in Exercise 5.10. To state them, it will be convenient to introduce the "overlap" R , ′ ∶= 1 {α =α ′ } (5.56) between two independent samples α ∈ N and α ′ ∈ N from the Gibbs measure (5.51).

At this stage it might sound a bit strange to call the indicator function in (5.56) an overlap, but the choice of terminology will become gradually clear. For now, we note that one can represent α and α ′ as elements of a Hilbert space and interpret the overlap R , ′ as a scalar product between these elements. To do so, we can choose the Hilbert space to be the space of square-integrable sequences, denote by (e n ) n∈N the canonical basis in this space, and identify each integer n with the vector e n , so that R , ′ = e α ⋅ e α ′ . The Ghirlanda-Guerra identities specify the law of the overlap R 1,n+1 given the overlap array R n ∶= (R , ′ ) , ′ ⩽n .

Theorem 5.22 (Ghirlanda-Guerra identities for PDP). Let ⟨⋅⟩ denote the Gibbs measure (5.51) associated with the Poisson-Dirichlet process (v n ) n⩾1 with parameter ζ ∈ (0,1). For every n ⩾ 1 and every bounded and measurable function f of the overlaps R n ∶= (R , ′ ) , ′ ⩽n as defined in (5.56), we have

E⟨ f (R n )R 1,n+1 ⟩ = 1 n E⟨ f (R n )⟩E⟨R 1,2 ⟩ + 1 n n =2 E⟨ f (R n )R 1, ⟩, (5.57)
and moreover,

E⟨R 1,2 ⟩ = E⟨1 {α 1 =α 2 } ⟩ = 1 -ζ . (5.58)
Proof. Let (g n ) n⩾1 be a sequence of independent standard Gaussian random variables, and for each t ∈ R, define the random weights

v t n ∶= u n expt g n -ζt 2 ∑ +∞ k=1 u k expt g k -ζt 2 = u n exp(tg n ) ∑ +∞ k=1 u k exp(tg k ) .
We denote by ⟨⋅⟩ t the average with respect to the random measure on N with weights (v t n ) n⩾1 . By the Bolthausen-Sznitman invariance (Lemma 5.21), the Poisson point processes have the same law. This implies that for any bounded measurable function

f of the overlaps R n , 0 = E⟨ f (R n )g α 1 ⟩ = E⟨ f (R n )(g α 1 -tζ )⟩ t .
The first equality is obtained by averaging with respect to the Gaussian randomness first. Invoking the Gibbs Gaussian integration by parts formula (Theorem 4.6) reveals that

tE⟨ f (R n ) n =1 1 {α 1 =α } -n1 {α 1 =α n+1 } -ζ ⟩ t = 0.
Using once again that the point processes in (5.59) have the same law, recalling (5.56) and rearranging yields

E⟨ f (R n )R 1,n+1 ⟩ = 1 n E⟨ f (R n ) n =1 R 1, -ζ ⟩ = 1 n n =2 E⟨ f (R n )R 1, ⟩ + 1 -ζ n E⟨ f (R n )⟩.
Taking f = n = 1 shows that 1ζ = E⟨R 1,2 ⟩, and completes the proof. ∎

Denoting by ν the law of a Bernoulli random variable with probability of success 1ζ , we deduce from Theorem 5.22 that conditionally on R n , the distribution of the overlap R 1,n+1 is given by the mixture

1 n ν + 1 n n =2 δ R 1, . (5.60)
This means that with probability 1 n , the overlap between α n+1 and α 1 is sampled independently according to ν, and with probability n-1 n it takes one of the existing values from R 1,2 ,...,R 1,n uniformly at random. It turns out that one can in fact infer the entire law of the weights of the Gibbs measure (5.51) from the Ghirlanda-Guerra identities, not just this conditional marginal distribution. This point is the subject of Exercise 5.10.

Exercise 5.9. Under the setting of Proposition 5.20, let (Z n ) n⩾1 be i.i.d. random variables on R >0 , independent of (X n ) n⩾1 and of the ordered Poisson-Dirichlet point process (u n ) n⩾1 with parameter ζ ∈ (0,1). Assume that EX

ζ 1 < +∞ and EZ ζ 1 < +∞. Show that Elog +∞ n=1 u n X n Z n = Elog +∞ n=1 u n Z n + 1 ζ logEX ζ 1 .
(5.61)

Exercise 5.10. Let (v j ) j⩾1 denote the decreasing enumeration of the weights of a random probability measure ⟨⋅⟩ on the natural numbers. We define the overlaps as in (5.56), suppose that the Ghirlanda-Guerra identities (5.57) hold, and define ζ ∈ (0,1) so that (5.58) holds. The main purpose of this exercise is to show that the law of (v j ) j⩾1 must be the law of the Poisson-Dirichlet process with parameter ζ .

(i) Consider the closed set

V ∶= x = (x j ) j⩾1 +∞ j=1
x j ⩽ 1 and (x j ) j⩾1 is non-increasing (5.62) of the product space [0,1] N equipped with the product topology, and for each integer n ⩾ 1, define the function p n ∶ V → R by p n (x) ∶= ∑ j⩾1 x n j . Prove that p n is continuous for n > 1. Is p 1 continuous? (ii) Prove that the set P of finite linear combinations of constants and products p n 1 ⋯ p n k for n 1 ,...,n k ⩾ 2 is dense in the space C(V ;R) of continuous functions from V to R endowed with the uniform norm.

(iii) For each k ⩾ 1, n 1 ,...,n k ⩾ 2, and

x ∈ V , define f (x;n 1 ,...,n k ) ∶= ∏ k =1 p n (x), let n ∶= n 1 + ⋯ + n k , and set S(n 1 ,...,n k ) ∶= E f (v j ) j⩾1 ,n 1 ,...,n k . Establish Talagrand's identities, S(n 1 + 1,n 2 ,...,n k ) = n 1 -ζ n S(n 1 ,...,n k ) + k j=2 n j n S(n 2 ,...,n j + n 1 ,...,n k ).
(iv) Deduce that the law of (v j ) j⩾1 is entirely determined by the parameter ζ , and conclude that the random probability measure ⟨⋅⟩ must be the Gibbs measure (5.51) associated with the Poisson-Dirichlet process with parameter ζ .

Poisson-Dirichlet cascades

Using Poisson-Dirichlet point processes, we now iteratively construct Poisson-Dirichlet cascades (also called Ruelle probability cascades [START_REF] Ruelle | A mathematical reformulation of Derrida's REM and GREM[END_REF]). As was discussed at the opening of the previous section, these richer objects will serve as "canonical models" for describing the geometry of the Gibbs measure of mean-field spin glasses, in the limit of large system size.

For each fixed choice of integer K ⩾ 1, we build Poisson-Dirichlet cascades as random probability measures on the index set N K . The case K = 1 will correspond to the Poisson-Dirichlet process. We think of N K as the set of leaves of a rooted tree with vertex set

A ∶= N 0 ∪ N ∪ N 2 ∪ ... ∪ N K , (5.63)
where N 0 ∶= {∅}. In this identification, the root of the tree is the element ∅, and

each vertex α = (n 1 ,...,n k ) ∈ N k for k ⩽ K -1 has children αn ∶= (n 1 ,...,n k ,n) ∈ N k+1 (5.64)
for all n ∈ N. For each vertex α = (n 1 ,...,n k ) ∈ A, we write α = k for its distance from the root ∅, or equivalently for the number of coordinates in α, and for each 0 ⩽ ⩽ k we denote by α ∶= (n 1 ,...,n ) (5.65) the vertex on the path from the root ∅ to α at distance from the root. We understand that α 0 = ∅. For any two leaves α,β ∈ N K we write

α ∧ β ∶= max ⩽ K α = β (5.66)
for the number of common vertices in the paths from the root ∅ to the leaves α and β .

∅ 1 1,1 1,2 1,3 2 2,1 2,2 2,3 3
3,1 3,2 3,3 N N 2 N K-1 N K α β K K -1 2 α ∧ β 1 0 Figure 5.1
The ancestry lines of the leaves α and β meet at the first level of this regular tree of infinite degree and depth K.

Although the Poisson-Dirichlet cascade will be indexed by α ∈ N K , its construction will involve random variables indexed by vertices of the entire tree A. Given a sequence of parameters

0 = ζ 0 < ζ 1 < ... < ζ K < ζ K+1 = 1, (5.67)
for each non-leaf vertex α ∈ A ∖ N K , let Λ α be the Poisson-Dirichlet point process with parameter ζ α +1 ; we construct these point processes independently from one another as α varies. We denote by (u αn ) n⩾1 the ordered Poisson-Dirichlet point process corresponding to Λ α . In this way, parent vertices α ∈ A ∖ N K enumerate independent Poisson point processes Λ α , while child vertices αn ∈ A∖N 0 enumerate individual points u αn . To each node α ∈ A, we associate the quantity

w α ∶= 1⩽ ⩽K u α .
(5.68)

We would now like to define the Poisson-Dirichlet cascade as the random measure on N K with weights

v α ∶= w α ∑ β ∈N K w β . (5.69)
This is possible due to an analogue of Proposition 5.18.

Lemma 5.23. The sum ∑ α∈N K w α is finite with probability one.

Proof. The proof proceeds by induction on K. The case K = 1 is the content of Proposition 5.18, so let us assume that the result holds when we sum over K -1 ⩾ 1 levels, and prove it for sums over K levels. Notice that for every α ∈ N K-1 and n ∈ N, we have w αn = w α u αn , and therefore

α∈N K w α = α∈N K-1 n∈N w αn = α∈N K-1 w α n∈N u αn .
For each α ∈ A ∖ N K , let U α ∶= ∑ n∈N u αn , so that

α∈N K w α = α∈N K-1 w α U α = α∈N K-2 n∈N w αn U αn = α∈N K-2 w α n∈N u αn U αn .
Observe that for each α ∈ N K-2 , the random variables (U αn ) n⩾1 are i.i.d. since the Poisson point processes (Λ αn ) n⩾1 are. Moreover, as ζ K-2 < ζ K-1 , Proposition 5.18 implies that

E(U αn ) ζ K-2 = E ˆ+∞ 0 xdΛ αn (x) ζ K-2 < +∞.
It follows by Theorem 5.19 that (u αn U αn ) n⩾1 and (cu αn ) n⩾1 for c ∶= (EU Since the quantities on the right side of (5.70) are i.i.d. over α ∈ N K-2 and independent of (w α ) α∈N K-2 , it follows that ∑ α∈N K w α is finite with probability one if and only if

ζ K-2 α1 ) 1 ζ K-2 are
α∈N K-2 w α U α = α∈N K-2 n∈N w α u αn = α∈N K-1 w α is.
α ∈ N K , v α ∶= w α ∑ β ∈N K w β . (5.71)
We stress that the Poisson-Dirichlet cascade is defined as the family of weights (v α ) α∈N K , as opposed to the set of these weights. While the precise labelling of the weights will not be important, we do care to keep track of the underlying tree structure. In other words, we are interested in the knowledge of (v α ) α∈N K up to the action of the bijections of N K that preserve the tree structure.

We think of this cascade as giving the weights of a Gibbs measure supported on the set N K , and we write ⟨⋅⟩ for its associated average, with α denoting the canonical random variable. We write (α ) ⩾1 to denote independent copies, or replicas, of the random variable α under this measure. This means that for any k ⩾ 1 and any bounded function

f ∶ (N K ) k → R, ⟨ f (α 1 ,...,α k )⟩ ∶= α 1 ,...,α k ∈N K f (α 1 ,...,α k )v α 1 ⋯v α k .
(5.72)

We now generalize the explicit formula obtained in Proposition 5.20 and the Ghirlanda-Guerra identities in Theorem 5.22 from the setting of the Poisson-Dirichlet processes to that of Poisson-Dirichlet cascades. It will be convenient to introduce a sequence (ω k ) 0⩽k⩽K of i.i.d. uniform random variables on [0,1]. These uniform random variables can be used to generate any random variable taking values in a complete separable metric space, as explained in Exercise 5.11 in the simple case of real-valued random variables. We give ourselves a measurable function

X K ∶= X K (ω 0 ,...,ω K ) (5.73) from [0,1] K to R.
Assuming that Eexpζ K X K < +∞, we now show that certain functionals associated with the Gibbs measure (5.72) can be computed recursively by considering the sequence

X k ∶= X k (ω 0 ,...,ω k ) ∶= 1 ζ k+1
logE ω k+1 expζ k+1 X k+1 (5.74) defined for 0 ⩽ k ⩽ K -1. By continuity, see Exercise 5.12, we take X -1 ∶= E ω 0 X 0 . These functionals will come up in the definition of enriched free energies in the next chapter.

Theorem 5.25. Let (ω β ) β ∈A be a family of independent uniform random variables on [0,1], independent of the Poisson-Dirichlet cascade (v α ) α∈N K , and let X -1 be defined recursively via (5.74). We have

Elog⟨expX K (ω ∅ ,ω α 1 ,...,ω α K-1 ,ω α )⟩ = X -1 .
(5.75)

In order to avoid ambiguities, we emphasize that on the left side of (5.75), the random variable α takes values in N K , is sampled according to the Gibbs measure defined in (5.72), and that ∅,α 1 ,...,α K-1 ,α denote the sequence of ancestors of α, starting from the root, as defined in (5.65). The expectation ⟨⋅⟩ only averages over the random variable α, while the expectation E averages the weights (5.71) of the Gibbs measure as well as the random variables (ω β ) β ∈A . On the other hand, the recursive calculation defining X -1 only involves much simpler recursive averages over the random variables ω K ,...,ω 0 .

Proof of Theorem 5.25. First, we observe that by Jensen's inequality and a simple induction, for -1 ⩽ k ⩽ K,

Eexpζ K X k = E(E ω k+1 expζ k+1 X k+1 ) ζ K ζ k+1 ⩽ Eexpζ K X k+1 ⩽ Eexpζ K X K < +∞,
so the sequence (X k ) -1⩽k⩽K is well-defined. To simplify notation, for each leaf α ∈ N K , write Ω α ∶= (ω ∅ ,ω α 1 ,...,ω α K-1 ,ω α ) for the sequence of uniform random variables along the path joining α to the root of the tree. In this notation, equality (5.75) may be written as

Elog α∈N K w α expX K (Ω α ) = Elog α∈N K w α + X -1 .
It turns out to be more convenient to establish a slightly more general equality. Let Z > 0 be a random variable with EZ ζ K < +∞ and let (Z α ) α∈N K be a sequence of i.i.d. copies of Z independent of all other random variables. We will prove that

Elog α∈N K w α Z α expX K (Ω α ) = Elog α∈N K w α Z α + X -1 , (5.76)
and that both sides are well-defined by induction on K. The case K = 1 is a consequence of Proposition 5.20 in the form (5.55), or more directly, of the rephrasing given in Exercise 5.9. Indeed, conditionally on the randomness of ω ∅ ,

Elog n∈N u n Z n expX 1 (ω ∅ ,ω n ) = Elog n∈N u n Z n + 1 ζ 1 logEexpX 1 (ω ∅ ,ω 1 ) = Elog α∈N w α Z α + X 0 .
Averaging with respect to the randomness of ω ∅ establishes (5.76) for K = 1. Let us therefore suppose that the result holds for K -1 ⩾ 1 and prove it for K. For each α ∈ N K-1 we write Ω αn ∶= (Ω α ,ω αn ), so that

α∈N K w α Z α expX K (Ω α ) = α∈N K-1 n∈N w αn Z αn expX K (Ω αn ) = α∈N K-1 w α n∈N u αn Z αn expX K (Ω α ,ω αn ).
(5.77)

Denote by F K-1 the σ -algebra generated by (w α ) α∈N K-1 and (Ω α ) α∈N K-1 , and observe that for each α ∈ N K-1 , the random variables (u αn ) n⩾1 , (Z αn ) n⩾1 and (ω αn ) n⩾1 are independent of F K-1 . It follows that

E[(Z αn expX K (Ω α ,ω αn )) ζ K F K-1 ] 1 ζ K = c(E ωαn expζ K X K (Ω α ,ω αn )) 1 ζ K = cexpX K-1 (Ω α ) for c ∶= (EZ ζ K ) 1 ζ K . Theorem 5.19 now implies that conditionally on F K-1 , n∈N u αn Z αn expX K (Ω α ,ω αn ) d = cU α expX K-1 (Ω α )
for U α ∶= ∑ n∈N u αn , and where we recall that d = denotes equality in distribution. Conditionally on F K-1 , both sides of this distributional equality are independent over α ∈ N K-1 , so substituting into (5.77) reveals that conditionally on

F K-1 , α∈N K w α Z α expX K (Ω α ) d = c α∈N K-1 w α U α expX K-1 (Ω α ).
Averaging over the randomness of F K-1 gives this equality in distribution unconditionally. The particular choice of X K = 0 yields the distributional equality

α∈N K w α Z α d = c α∈N K-1 w α U α ,
so proving (5.76) is reduced to establishing the equality

Elog α∈N K-1 w α U α expX K-1 (Ω α ) = Elog α∈N K-1 w α U α + X -1 .
(5.78) By Proposition 5.18, the random variables

(U α ) α∈N K-1 satisfy EU ζ K-1 α = E ˆxdΛ α (x) ζ K-1
< +∞ and are i.i.d. and independent of all other random variables in (5.78). Since the equation (5.78) is of the same type as (5.76) with K replaced by K -1 and (Z α ) α∈N K replaced by (U α ) α∈N K-1 , invoking the induction hypothesis completes the proof. ∎

In Chapter 6, we will use this result to compute the initial condition for the Hamilton-Jacobi equation associated with spin-glass models. For this purpose, it will be convenient to establish a slightly more general version. Instead of just considering K + 1 sources of randomness (ω k ) 0⩽k⩽K , we generate N(K + 1) independent random variables (ω k,i ) 0⩽k⩽K,1⩽i⩽N sampled from the uniform distribution on [0,1], and instead of just considering one function X K , we give ourselves N measurable functions X K,i ∶= X K,i (ω 0,i ,...,ω K,i ) (5.79)

for i ∈ {1,...,N}. Assuming that Eexpζ K X K,i < +∞, we recursively define, for every k ∈ {-1,...,K -1} and i ∈ {1,...,N},

X k,i ∶= X k,i (ω 0,i ,...,ω k,i ) ∶= 1 ζ k+1 logE ω k+1,i expζ k+1 X k+1,i . (5.80)
Once again, by continuity, we understand that X -1,i ∶= E ω 0,i X 0,i . The next result states that this more general iterative procedure can be used to calculate more complicated averages with respect to the Poisson-Dirichlet cascade ⟨⋅⟩.

Corollary 5.26. Let N ⩾ 1 be an integer, let (ω β ,i ) β ∈A,1⩽i⩽N be a family of independent uniform random variables on [0,1] independent of the Poisson-Dirichlet cascade (v α ) α∈N K , and for each i ∈ {1,...,N}, let X -1,i be defined recursively via (5.80). We have

1 N Elog⟨exp N i=1 X K,i (ω ∅,i ,ω α 1 ,i ,...,ω α K-1 ,i ,ω α,i )⟩ = 1 N N i=1
X -1,i .

(5.81)

Proof. The main point is to realize that we can apply Theorem 5.25 to compute the term on the left side of (5.81). Indeed, we can find a measurable mapping

f = ( f 1 ,..., f N ) ∶ [0,1] → [0,1]
N that sends Lebesgue measure on [0,1] to Lebesgue measure on [0,1] N (a simple splitting of the binary expansion of a real number will do). Letting (ω k ) 0⩽k⩽K be independent uniform random variables on [0,1], the random variables ( f i (ω k )) 0⩽k⩽K,1⩽i⩽N have the same law as the (ω k,i ) 0⩽k⩽K,1⩽i⩽N in the definition of the variables X k,i ; and integrating each ω k from k = K downwards to k = 0 corresponds to integrating (ω i,k ) 1⩽i⩽N in the exact same way.

Let Y K ∶= ∑ N i=1 X K,i . By independence, Eexpζ K Y K = N i=1 Eexpζ K X K,i < +∞.
Using again independence and an induction from k = K -1 to k = 0, we verify that

Y k ∶= 1 ζ k+1 logE ω k+1 expζ k+1 Y k+1 = N i=1 X k,i ,
and similarly for k = -1. The result is therefore a consequence of Theorem 5.25. ∎

We now turn our attention to extending the Ghirlanda-Guerra identities (Theorem 5.22) to the setting of Poisson-Dirichlet cascades. To define the analogue of the overlap (5.56), we fix a sequence 0 = q -1 ⩽ q 0 < q 1 < ... < q K < q K+1 = +∞, (5.82) and define the overlap between two independent samples α and α ′ from the Poisson-Dirichlet cascade ⟨⋅⟩ by R , ′ ∶= q α ∧α ′ .

(5.83)

In the context of mean-field spin glasses, we will think of this overlap R , ′ as representing the scalar product between the configurations whose Gibbs weights are represented by v α and v α ′ .

In order to interpret the overlap (5.83) as a scalar product, we fix a separable Hilbert space H and let (e α ) α∈A be an orthonormal family in H. For each α ∈ N K , we define h α ∈ H by

h α ∶= K k=0 q k -q k-1 1 2 e α k , (5.84)
and observe that for every α 1 ,α 2 ∈ N K ,

h α 1 ⋅ h α 2 = α 1 ∧α 2 k=0 q k -q k-1 = q α 1 ∧α 2 = R 1,2 .
(5.85)

In other words, if we identify each leaf α ∈ N K with the element h α ∈ H, then the overlap (5.83) between two leaves α 1 and α 2 becomes the scalar product between their representatives h α 1 and h α 2 in H.

To prove the Ghirlanda-Guerra identities (5.57) for the Poisson-Dirichlet cascades, we will rely on a generalization of the Bolthausen-Sznitman invariance property (Lemma 5.21). For technical reasons, to extend the Bolthausen-Sznitman invariance property to the setting of the Poisson-Dirichlet cascade, it will be useful to first assume that q 0 = 0 in the sequence (5.82). This assumption will be lifted with ease when we prove the Ghirlanda-Guerra identities, since these identities possess a natural invariance property as we vary the sequence (q k ) 0⩽k⩽K .

Given a family (z α ) α∈A∖N 0 of independent standard Gaussian random variables, we introduce the family (Z q (α)) α∈N K of Gaussian random variables defined by

Z q (α) ∶= K k=0 q k -q k-1 1 2 z α k = K k=1 q k -q k-1 1 2 z α k .
(5.86)

Notice that the covariance structure

EZ q (α 1 )Z q (α 2 ) = q α 1 ∧α 2 = R 1,2 (5.87)
of these random variables matches the definition of the overlaps of the Poisson-Dirichlet cascade introduced in (5.83). As a side remark, we also point out that if we view the Z q (α)'s as elements of the Hilbert space of square-integrable random variables on the underlying probability space, then the mapping α ↦ Z q (α) provides us with a second way to realize the overlap (5.83) as a scalar product.

As was mentioned below Definition 5.24, we want to keep track of the information on the weights (v α ) α∈N K up to relabellings of N K that preserve the tree structure. It would therefore not suffice for our purposes to obtain a version of the Bolthausen-Sznitman invariance that only concerns the set of weights (w α ) α∈N K , or equivalently the point process ∑ α∈N K δ wα (whether or not each w α is paired with some Gaussian random variable is not the point we want to stress here). To make this precise, we say that a bijection π ∶ N K → N K preserves the tree structure if for every α,β ∈ N K , we have that π(α) ∧ π(β ) = α ∧ β . Lemma 5.27 (Bolthausen-Sznitman invariance for PDC [START_REF] Bolthausen | On Ruelle's probability cascades and an abstract cavity method[END_REF]). Suppose that q 0 = 0 in the sequence (5.82), let (Z q (α)) α∈N K denote the family of Gaussian random variables defined by (5.86) and taken independently of the random weights (w α ) α∈N K defined in (5.68), and let b ∶= ∑ K k=1 (q kq k-1 )ζ k . For each t ∈ R, there exists a random permutation π ∶ N K → N K that preserves the tree structure such that w π(α) exp t Z q (π(α)) -tb 2 ,Z q (π(α)) -tb α∈N K

(5.88)

has the same law as w α , Z q (α) α∈N K . Proof. Given a family (z α ) α∈A∖N 0 of independent standard Gaussian random variables, we define the family (g α ) α∈A∖N 0 of independent Gaussian random variables with variances v α ∶= q αq α -1 by

g α ∶= (q α -q α -1 ) 1 2 z α .
For each α ∈ A ∖ N K , we recall that Λ α = (u αn ) n⩾1 is a Poisson-Dirichlet point process with parameter ζ α +1 ∈ (0,1), and that these processes are independent as α varies in A ∖ N K . It follows by the Bolthausen-Sznitman invariance in Lemma 5.21 that the Poisson point processes

Λ + α ∶= +∞ n=1 δ (uαn,gαn) and Λ ′ α ∶= +∞ n=1 δ (uαn exp(t(gαn-tv α +1 ζ α +1 2)),gαn-tv α +1 ζ α +1 )
have the same law. We also observe that the random variables (Λ + α ) α∈A∖N K are independent, and this also holds for (Λ ′ α ) α∈A∖N K . We denote by π α ∶ N → N the permutation that sorts the random variables (u αn exp(t(g αn -tv α +1 ζ α +1 2))) n⩾1 decreasingly, and set

U + α ∶= (u αn ,g αn ) n⩾1 , U ′ α ∶= u απα (n) exp t g απα (n) - tv α +1 ζ α +1 2 ,g απα (n) -tv α +1 ζ α +1 n⩾1 .
Since (Λ + α ) α∈A∖N K and (Λ ′ α ) α∈A∖N K have the same law, we have that (U + α ) α∈A∖N K and (U ′ α ) α∈A∖N K have the same law as well. We next define recursively the permutation π ∶ A → A such that π(∅) = ∅ and, for every α ∈ A ∖ N K and n ∈ N, π(αn) ∶= π(α)π π(α) (n).

(5.89)

The fact that π is indeed a permutation can be shown by induction on the depth of the tree. Notice also that π preserves the parent-child relationship: by (5.89), for every α ∈ A ∖ N K and n ∈ N, the permutation π sends αn to a child of π(α). In particular, for each k ∈ {0,...,K}, the restriction of π to N k is a permutation that preserves the tree structure. We now show by induction that, for every k ∈ {0,...,K -1},

(U + α ) α ⩽k d = (U ′ π(α) ) α ⩽k , (5.90)
where we recall that d = denotes equality in distribution. We have already observed that the identity (5.90) is valid when k = 0. Proceeding inductively, suppose that the identity is valid with k ∈ {0,...,K -2}. Recall that the two families (U + α ) α ⩽k and (U + α ) α =k+1 are independent, and that the random variables (U + α ) α =k+1 are independent as α varies. We denote by F k the σ -algebra generated by (U + α ) α ⩽k . The family (U ′ π(α) ) α ⩽k is F k -measurable, and so is (π(α)) α ⩽k+1 . Conditionally on F k , the random variables (U ′ α ) α =k+1 are independent, and the family has the same law as (U + α ) α =k+1 . Since (π(α)) α =k+1 is F k -measurable, this remains valid for the family (U ′ π(α) ) α =k+1 . Combining these observations yields (5.90) with k replaced by k + 1. We can now deduce from (5.90) that

K k=1 u α k , K k=1 g α k α∈N K has the same law as K k=1 u π(α) k exp t g π(α) k - tv k ζ k 2 , K k=1 (g π(α) k -tv k ζ k ) α∈N K .
Recalling the definition of the family of Gaussian random variables (Z q (α)) α∈N K , of the weights (w α ) α∈N K and of the constant b = ∑ K k=1 v k ζ k completes the proof. ∎ Theorem 5.28 (Ghirlanda-Guerra identities for PDC [START_REF] Bovier | Derrida's generalised random energy models. I. Models with finitely many hierarchies[END_REF]). For every n ⩾ 1, every bounded and measurable function f of the overlaps R n = (R , ′ ) , ′ ⩽n , and every bounded and measurable function ψ ∶ R → R, we have

E⟨ f (R n )ψ(R 1,n+1 )⟩ = 1 n E⟨ f (R n )⟩E⟨ψ(R 1,2 )⟩ + 1 n n =2 E⟨ f ψ(R 1, )⟩ (5.91)
and

E⟨ψ(R 1,2 )⟩ = K k=0 ψ(q k )(ζ k+1 -ζ k ).
(5.92)

In particular, for every k ∈ {0,...,K},

E⟨1 {α 1 ∧α 2 =k} ⟩ = ζ k+1 -ζ k . (5.93)
Proof. We first explain why we can assume that q 0 = 0 without loss of generality. Notice that the overlap R , ′ is obtained by first computing α ∧ α ′ , and then mapping the result through the function

{0,...,K} → R k ↦ q k .
Defining R , ′ ∶= α ∧ α ′ , it therefore follows that (5.91)-(5.92) can be rephrased as the statement that, for every

f ∶ R n×n → R and ψ ∶ R → R, E⟨ f ( Rn )ψ( R1,n+1 )⟩ = 1 n E⟨ f ( Rn )⟩E⟨ψ( R1,2 )⟩ + 1 n n =2 E⟨ f ψ( R1, )⟩, and
E⟨ψ( R1,2 )⟩ = K k=0 ψ(k)(ζ k+1 -ζ k ).
Notice that these two statements do not depend on the choice of the values of (q 0 ,...,q K ). In other words, as soon as we can prove Theorem 5.28 for some choice of (q 0 ,...,q K ) satisfying (5.82), we can deduce its validity for all such choices. In particular, there is no loss of generality in imposing that q 0 = 0. (This invariance is not valid for Lemma 5.27, since the law of Z q depends on the choice of (q 0 ,...,q K ).) So we assume from now on that q 0 = 0. Moreover, as the overlap takes only finitely many values q 0 ,...,q K , by polynomial interpolation we may assume without loss of generality that ψ is a polynomial. By linearity it therefore suffices to prove (5.91) and (5.92) when ψ(x) = x m for some integer m ∈ N. Since (5.91) and (5.92) are clear for m = 0, let us assume that m ⩾ 1. Given a family (z α ) α∈A∖N 0 of independent standard Gaussian random variables, we introduce the family (Z q m (α)) α∈N K of Gaussian random variables defined by

Z q m (α) ∶= K k=1 q m k -q m k-1 1 2 z α k .
We write (w α ) α∈N K for the random weights in (5.68), and let

b m ∶= K k=1 (q m k -q m k-1 )ζ k .
For each t ∈ R and α ∈ N K , we define the random weight

v t,m α ∶= w α expt Z q m (α) -tbm 2 ∑ α∈N K w α expt Z q m (α) -tbm 2 = w α exp(tZ q m (α)) ∑ α∈N K w α exp(tZ q m (α)) ,
and denote by ⟨⋅⟩ t,m the Gibbs measure (5.72) with the weights (v α ) α∈N K replaced by the weights (v t,m α ) α∈N K . By the Bolthausen-Sznitman invariance (Lemma 5.27), there exists a random permutation π ∶ N K → N K that preserves the tree structure such that the families v t,m π(α) ,Z q m (π(α)) -tb m α∈N K and v 0,m α ,Z q m (α)

α∈N K

(5.94)

have the same law. This implies that for any f ∶ R n×n → R,

0 = E⟨ f (R n )Z q m (α 1 )⟩ = E α 1 ,...,α n ∈N K f (R n )Z q m (α 1 )v 0,m α 1 ⋯v 0,m α n = E α 1 ,...,α n ∈N K f (R n ) Z q m (π(α 1 )) -tb m v t,m π(α 1 ) ⋯v t,m π(α n ) = E α 1 ,...,α n ∈N K f (R n ) Z q m (α 1 ) -tb m v t,m α 1 ⋯v t,m α n = E⟨ f (R n )(Z q m (α 1 ) -tb m )⟩ t,m ,
where in the fourth equality, we used that π is a permutation that preserves the tree structure to ensure that the action of π on R n is trivial. Invoking the Gibbs Gaussian integration by parts formula (Theorem 4.6) yields that

tE⟨ f (R n ) n =1 R m 1, -nR m 1,n+1 -b m ⟩ t,m = 0.
Using again that the families in (5.94) have the same law with π preserving the tree structure, recalling that R 1,1 = q K by (5.83), and rearranging, we obtain that

E⟨ f (R n )R m 1,n+1 ⟩ = 1 n n =2 E⟨ f (R n )R m 1, ⟩ + q m K -b m n E⟨ f (R n )⟩.
(5.95)

Taking f = n = 1 reveals that E⟨R m 1,2 ⟩ = q m K -b m = K k=0 q m k (ζ k+1 -ζ k ),
where we have used that ζ 0 = 0 and ζ K+1 = 1. This establishes (5.92) and, together with (5.95), also gives (5.91). Taking ψ(x) ∶= 1 {x=q k } for 0 ⩽ k ⩽ K completes the proof. ∎

As was already mentioned several times, in the notion of Poisson-Dirichlet cascade, we pay attention to keeping track of the tree structure of N K , as opposed to simply studying the set of weights {w α α ∈ N K }, or equivalently the point process ∑ α∈N K δ wα . In fact, this point process has the same law as the point process

∑ +∞ n=1 δ vn , if (v n ) n⩾1 is a Poisson-Dirichlet process of parameter ζ K .
This can be shown by induction on the number of layers of the cascade. Alternatively, one can use Theorem 5.28 to observe that the overlap defined by R , ′ ∶= 1 {α =α ′ } satisfies the Ghirlanda-Guerra identities with E⟨1 {α 1 =α 2 } ⟩ = 1ζ K , and then use Exercise 5.10 to conclude. But the cascade (w α ) α∈N K itself is richer as it keeps track of how the indices are organized in the tree structure, an aspect which cannot be recovered from ∑ α∈N K δ wα .

Exercise 5.11. Given a right-continuous and non-decreasing function F ∶ R → [0,1], let F -1 ∶ (0,1) → R denote its quantile transform,

F -1 (x) ∶= inf{s ∈ R F(s) ⩾ x}.
(5.96) (i) Show that for all t ∈ R and x ∈ (0,1), we have F(t) < x if and only if t < F -1 (x).

(ii) Let X be a real-valued random variable, and denote by F its distribution function, F(x) ∶= P{X ⩽ x}. Letting U be a uniform random variable on the interval [0,1], show that F -1 (U) and X have the same law.

Exercise 5.12. Fix ε > 0 and a random variable X such that Eexpε X < +∞. Prove that

lim ζ ↘0 1 ζ logEexpζ X = EX.
(5.97)

Ultrametricity as a universal property

We have already discussed informally that Poisson-Dirichlet cascades provide us with "canonical models" for the asymptotic behaviour of Gibbs measures of meanfield spin glasses. The goal of this section is to give some substance to this claim. We start by discussing background motivation in more detail.

In the context of mean-field spin glasses, as in statistical inference, we will be concerned with the study of a random Gibbs measure on R N , and we would like to find ways to capture its asymptotic behaviour. To fix notation, we denote by ⟨⋅⟩ N the expectation with respect to this random Gibbs measure over R N , with canonical random variable σ , and by (σ ) ⩾1 independent copies of the random variable σ under ⟨⋅⟩ N . Recalling that ⟨⋅⟩ N is itself random, we denote by E the expectation with respect to this additional source of randomness.

It is not at all clear how to capture asymptotic properties of a sequence of probability measures defined on spaces of increasingly large dimension. Besides, the spin-glass models we aim to consider have a very high degree of symmetry, and there is no axis or other subspace of small dimension that we would particularly want to concentrate upon. A very fruitful approach is to keep track of the array of overlaps R ∶= (R , ′ ) , ′ ⩾1 between the different replicas. For one thing, it is shown in Lemma 1.7 of [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF] that the law of this overlap array characterizes the underlying Gibbs measure up to a random orthogonal transformation. Since the models we will study have a very large group of symmetries, due to their mean-field character, there is hopefully not much loss of information if one only understands the Gibbs measure up to an orthogonal transformation. Moreover, under modest assumptions on the model, we know a priori that each coordinate of R takes values in a bounded set, say [-1,1] if the support of the Gibbs measure on R N is contained in B √ N (0).

So we can at least extract convergent subsequences, and wonder about properties of the limit overlap array. From now on we denote by R any such limit overlap array.

The array R must be symmetric positive semi-definite. Notice also that if π is a permutation of finitely many indices in N, then the array (R π( ),π( ′ ) ) , ′ ⩾1 has the same law as the array R. Under these assumptions, the Dovbysh-Sudakov theorem (Theorem 1.7 in [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF]) asserts that the off-diagonal part of the array R can still be represented as an array of overlaps. More precisely, there exists a random probability measure G on some Hilbert space, with associated expectation ⟨⋅⟩, such that, using again the notation (σ ) ⩾1 for the replicas under this new probability measure, the off-diagonal elements (R , ′ ) ≠ ′ of the limit array have the same joint distribution as (σ ⋅ σ ′ ) ≠ ′ . This result can be thought of as a variant of the de Finetti representation theorem for exchangeable sequences.

Notice that in the statement of the Dovbysh-Sudakov theorem, we excluded the diagonal elements of the array R from the representation. The result does not extend to on-diagonal elements as stated, as their might be some "loss of norm" as we pass to the limit. We illustrate that this phenomenon of loss of norm can indeed happen on a simple example. We take the uniform measure on the sphere of radius √ N in R N as the sequence of Gibbs measures (so these are non-random in this example). In this case, the limit overlap array is R , ′ = 1 { = ′ } . The only choice then to represent the off-diagonal elements of this array is to take the limit Gibbs measure to be concentrated at the origin, so σ ⋅ σ = 0, and this does not match the on-diagonal elements of the limit overlap array that are equal to 1.

For convenience, until the end of this section, we redefine R to be (R , ′ ) ≠ ′ , i.e. we leave aside the on-diagonal elements.

A crucial observation for mean-field spin-glass models is that, up to the addition of a small perturbation to the Hamiltonian, we can ensure that any possible limit overlap array R must satisfy the Ghirlanda-Guerra identities. The perturbation added to the Hamiltonian is small in the sense that it does not alter the value of the limit free energy. We have seen in Exercise 5.10 that when the overlap can only takes values in {0,1}, an overlap array that satisfies the Ghirlanda-Guerra identities must have the same law as that of a Poisson-Dirichlet process. It turns out that this observation admits a powerful generalization: if an overlap array satisfies the Ghirlanda-Guerra identities, then it must have the same law as that of a Poisson-Dirichlet cascade. This is the operationalization of the idea that Poisson-Dirichlet cascades provide us with "canonical models" for what the Gibbs measure of a mean-field spin glass asymptotically looks like.

To state this characterization of Poisson-Dirichlet cascades precisely, let us start by recapitulating its context. We study a random probability measure G supported on the unit ball of a Hilbert space H, whose expectation we denote by ⟨⋅⟩, and we denote by E the expectation with respect to the randomness of the random measure G itself; we write (σ ) ⩾1 for the replicas under G, and R = (σ ⋅ σ ′ ) ≠ ′ stands for their overlap array. We denote by ζ the law of one overlap under E⟨⋅⟩, so that for every measurable set A ⊆ R,

ζ (A) = E⟨1 {R 1,2 ∈A} ⟩.
(5.98)

We say that the measure G satisfies the Ghirlanda-Guerra identities if for every n ⩾ 1, every bounded measurable function f of the overlaps R n = (R , ′ ) ≠ ′ ⩽n , and every bounded measurable function ψ ∶ R → R, we have

E⟨ f (R n )ψ(R 1,n+1 )⟩ = 1 n E⟨ f (R n )⟩E⟨ψ(R 1,2 )⟩ + 1 n n =2 E⟨ f (R n )ψ(R 1, )⟩. (5.99)
Equivalently, the measure G satisfies the Ghirlanda-Guerra identities if, for every n ⩾ 1, the law of R 1,n+1 under E⟨⋅⟩ conditionally on R n is given by

1 n ζ + 1 n n =2 δ R 1, .
(5.100)

Theorem 5.29 (Characterization of PDC [START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF]). If the random probability measure G satisfies the Ghirlanda-Guerra identities (5.99), then the law of the infinite overlap array R = (R , ′ ) ≠ ′ under E⟨⋅⟩ is uniquely determined by the probability measure ζ in (5.98). Moreover, the measure ζ is supported on R ⩾0 .

Concerning the on-diagonal elements (σ ⋅ σ ) ⩾1 , one can show that the Ghirlanda-Guerra identities impose that G be supported on the sphere in H whose radius is the square root of the top of the support of ζ . In other words, denoting by q * the top of the support of ζ , we have, with probability one under E⟨⋅⟩, that

σ 2 = q * .
(5.101)

We refer to Theorem 2.15 of [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF] for a (relatively simple) proof of this fact. When the support of ζ is finite, we already know from Theorem 5.28 that the overlap array associated with a Poisson-Dirichlet cascade satisfies the Ghirlanda-Guerra identities, with ζ being the measure given by

ζ = K k=0 (ζ k+1 -ζ k )δ q k .
(5.102)

Recall also from the discussion around (5.84)-(5.85) that the overlap array associated with a Poisson-Dirichlet cascade can indeed be interpreted as a matrix of scalar products for a suitable choice of random probability measure. Theorem 5.29 thus tells us that, when the support of ζ is finite, the law of the overlap array coming from G is the same as that coming from the Poisson-Dirichlet cascade with parameters (ζ k ) 1⩽k⩽K and (q k ) 0⩽k⩽K chosen so that (5.102) holds.

For more general ζ , we have not yet built a canonical example of an overlap array satisfying (5.98) and the Ghirlanda-Guerra identities. This can be achieved by approximation, as will be explained in Corollary 5. [START_REF] Barbier | Mutual information for symmetric rank-one matrix estimation: a proof of the replica formula[END_REF].

In short, up to extending the notion of Poisson-Dirichlet cascade and its associated overlap array by continuity, we see that if a random probability measure G satisfies the Ghirlanda-Guerra identities, then the law of its overlap array must be that of a Poisson-Dirichlet cascade.

The fundamental step in the proof of Theorem 5.29 involves the concept of ultrametricity. We say that the support of the measure G is ultrametric if, with probability one over E⟨⋅⟩, we have

R 1,3 ⩾ min(R 1,2 ,R 2,3 ).
(5.103)

In such a circumstance, we may also say that the overlap array R is ultrametric.

Recalling from (5.101) that G is supported on a sphere, we can rephrase the ultrametricity property as the statement that, with probability one over E⟨⋅⟩, we have

σ 1 -σ 3 ⩽ max σ 1 -σ 2 , σ 2 -σ 3 . (5.104)
Equivalently, this says that with probability one over E, every choice of three points σ 1 , σ 2 , and σ 3 in the support of G must be such that (5.104) holds. In words, this stronger form of the triangle inequality means that every triangle has its two longest sides of equal length. Indeed, for any three points σ 1 , σ 2 , and σ 3 in the support of G, if the segment joining σ 1 to σ 2 happens to be the shortest of the three sides of the triangle, then (5.104) ensures that the segment joining σ 1 to σ 3 is at most as long as that joining σ 2 to σ 3 ; and by symmetry, we conclude that these two segments have the same length. In terms of overlaps, this means that among R 1,2 , R 1,3 , and R 2,3 , the two smallest overlaps must be equal. The notion of ultrametricity is intimately tied with tree structures. For convenience we will assume that the support of the overlap distribution ζ is finite, and argue that the ultrametricity property allows us to organize the support of G into the leaves of a tree, so that the distance (or equivalently for us: the overlap) between any two leaves α and β is only a function of α ∧ β , the depth of their most recent common ancestor. Indeed, the ultrametricity property implies that, for every p in the support of ζ , the relation

σ 1 ∼ p σ 2 ⇐⇒ R 1,2 ⩾ p (5.105)
defines an equivalence relation. We then draw the tree as in Figure 5.1, so that the nodes at depth k represent the equivalence classes of the relation ∼ p for p the k th atom in the support of ζ , i.e. for p = q k if ζ is as in (5.102). We refer to Lemma 4.2 of [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] for a more precise construction.

In particular, one can easily see that Poisson-Dirichlet cascades satisfy the ultrametricity property (5.103), using that for any three leaves α, β , γ we have

α ∧ γ ⩾ min α ∧ β ,β ∧ γ .
(5.106)

The crucial step towards establishing Theorem 5.29 is to show that the Ghirlanda-Guerra identities imply ultrametricity. The setting for this result is as discussed in the paragraph above (5.98) and we do not restate it here.

Theorem 5.30 (Panchenko ultrametricity [START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF]). If the random probability measure G satisfies the Ghirlanda-Guerra identities (5.99), then its support is ultrametric.

We refer the interested reader to Theorem 2.14 in [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF] for a proof of this result, and to [START_REF] Panchenko | Ghirlanda-Guerra identities and ultrametricity: an elementary proof in the discrete case[END_REF] for a simpler argument in the special case when the support of ζ is finite.

Once Theorem 5.30 is established, the proof of Theorem 5.29 is comparatively less demanding. We first illustrate its core mechanics by deriving the joint law of the three overlaps (R 1,2 ,R 1,3 ,R 2,3 ), assuming the validity of the Ghirlanda-Guerra identities and the ultrametricity property. We will then generalize this argument and thereby deduce Theorem 5.29 from Theorem 5.30. Theorem 5.30 states that the joint law of the triple of overlaps (R 1,2 ,R 1,3 ,R 2,3 ) is supported on the set of points with two equal coordinates smaller than the third. This set is displayed in Figure 5.2. We first write down the law of this triple away from the diagonal {x = y = z}. Starting with the off-diagonal part of the red isosceles triangle in Figure 5.2, which corresponds to the set {(x,y,z) ∈ [0,1] 3 z = y < x}, we have that for every bounded measurable function Indeed, recall that ultrametricity forces the two smallest overlaps among (R 1,2 , R 1,3 , R 2,3 ) to always be equal, so

ψ ∶ R 3 → R, E⟨ψ(R 1,2 ,R 1,3 ,R 2,3 )1 {R 1,2 >R 1,3 =R 2,3 } ⟩ =
E⟨ψ(R 1,2 ,R 1,3 ,R 2,3 )1 {R 1,2 >R 1,3 =R 2,3 } ⟩ = E⟨ψ(R 1,2 ,R 1,3 ,R 1,3 )1 {R 1,2 >R 1,3 } ⟩, (5.108)
and then the Ghirlanda-Guerra identities and (5.98) yield (5.107). By symmetry, we can also write down the joint law of the three overlaps on the off-diagonal parts of the blue and green triangles in Figure 5.2.

To determine the joint law on the diagonal {(x,y,z) ∈ [0,1] 3 x = y = z}, we can use ultrametricity to write

E⟨ψ(R 1,2 ,R 1,3 ,R 2,3 )1 {R 1,2 =R 1,3 =R 2,3 } ⟩ = E⟨ψ(R 1,2 ,R 1,2 ,R 1,2 )1 {R 1,2 =R 1,3 } ⟩ -E⟨ψ(R 1,2 ,R 1,2 ,R 1,2 )1 {R 1,2 =R 1,3 <R 2,3 } ⟩.
Using ultrametricity once again, we can drop the condition R 1,2 = R 1,3 in the second term since R 1,2 < R 2,3 , and then use the Ghirlanda-Guerra identities to find that

E⟨ψ(R 1,2 ,R 1,3 ,R 2,3 )1 {R 1,2 =R 1,3 =R 2,3 } ⟩ = 1 2 ˆψ(x,x,x)dζ (x) + 1 2 ˆ{x=y} ψ(x,x,x)dζ (x)dζ (y) - 1 2 ˆ{x<z} ψ(x,x,x)dζ (x)dζ (z) = 1 2 ˆ{x>y} ψ(x,x,x)dζ (x)dζ (y) + ˆ{x=y} ψ(x,x,x)dζ (x)dζ (y).
Bringing all terms together, we conclude that for any bounded measurable function

ψ ∶ R 3 → R, we have E⟨ψ(R 1,2 ,R 1,3 ,R 2,3 )⟩ = 1 2 ˆ{x>y} ψ(x,y,y) + ψ(y,x,y) + ψ(y,y,x) + ψ(x,x,x) dζ (x)dζ (y) + ˆ{x=y} ψ(x,x,x)dζ (x)dζ (y). (5.109)
This shows in particular that the law of the triple (R 1,2 ,R 1,3 ,R 2,3 ) is uniquely determined by the probability measure ζ , as stated in Theorem 5.29. We now generalize this argument and show that, under the assumptions of ultrametricity and of the validity of the Ghirlanda-Guerra identities, the law of the full overlap array is completely determined in terms of the law of R 1,2 . We recall that the setting is as explained in the paragraph above (5.98).

Proposition 5.31. If the random probability measure G satisfies the Ghirlanda-Guerra identities and its support is ultrametric, then the law of the infinite overlap array R = (R , ′ ) ≠ ′ under E⟨⋅⟩ is uniquely determined by the probability measure ζ in (5.98).

Proof. Let n ⩾ 1 be an integer. We show the following statement by induction on n. Letting ( 1 , ′ 1 ),...,( n , ′ n ) be n distinct pairs of integers such that k < ′ k for every k ⩽ n, letting ≼ 1 ,...,≼ n-1 ∈ {=,<}, and letting ψ ∶ R n → R be a bounded and measurable function, the quantity

E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n } ⟩
is uniquely determined from the knowledge of the law ζ of the overlap R 1,2 .

The claim is immediate for n = 1. We now assume that the claim is valid for some integer n ⩾ 1, and give ourselves ( 1 , ′ 1 ),...,( n+1 , ′ n+1 ) as above, order relations ≼ 1 ,...,≼ n ∈ {=,<}, and a bounded and measurable function ψ ∶ R n+1 → R. By symmetry between the replicas, we may assume that n+1 = 1 and ′ n+1 = L, where L is the largest element of { ′ 1 ,..., ′ n+1 }. We distinguish two cases. Case 1. Suppose that the order relation ≼ n is <.

If L ∉ { ′ 1 ,..., ′ n }, (5.110)
then we can readily apply the Ghirlanda-Guerra identities to simplify the expectation we aim to compute, since we recall that conditionally on R L-1 , the law of R 1,L under E⟨⋅⟩ is given by (5.100) (substituting n by L -1 there). Otherwise, let k ⩽ n be such that ′ k = L. By ultrametricity, the events R k,L < R 1,L and R 1,k < R 1,L are identical, and on this event, we have that R 1,k = R k,L . In the expression

E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R 1,n+1 )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n <R 1,n+1 } ⟩,
we can therefore substitute every occurrence of R k,L among R 1 , ′ 1 ,...,R n, ′ n by R 1,k . We can do so for every k such that ′ k = L, until we arrive at an expression that involves no term of the form R k,L for any k among the overlaps R 1 , ′ 1 ,...,R n, ′ n . Once this is done, we can simplify the expression by applying the Ghirlanda-Guerra identities, as already observed.

Case 2. We now assume that the order relation ≼ n is the equality =. There might be several consecutive occurrences of the equality sign suffixing the sequence ≼ 1 ,...,≼ n , and our goal is to reduce this number of suffixing equality signs by at least one, so that by induction, we can ultimately reduce all computations to occurrences of Case 1. We write

E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R 1,L )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n =R 1,L } ⟩ = E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R n, ′ n )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n } ⟩ -E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R n, ′ n )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n <R 1,L } ⟩ -E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R n, ′ n )1 {R 1 , ′ 1 ≼ 1 ⋯≼ n-1 R n, ′ n } 1 {R n, ′ n >R 1,L } ⟩.
The first term on the right side of this equality can be computed by the induction hypothesis, while the second one is an instance of Case 1. The last term can be decomposed into a sum, by partitioning the event according to the position of R 1,L relatively to the overlaps R 1 , ′ 1 ,...,R n, ′ n . That is, we rewrite the last term of the previous display as a sum of terms of the form

E⟨ψ(R 1 , ′ 1 ,...,R n, ′ n ,R n, ′ n )1 {R 1 , ′ 1 ≼ 1 ⋯≼ k-1 R k , ′ k ≼ ′ k R 1,L ≼ ′′ k R k+1 , ′ k+1 ≼ k+1 ⋯≼ n-1 R n, ′ n } ⟩,
for various choices of k and of the order relations ≼ ′ k ,≼ ′′ k ∈ {=,<}. Since this decomposition comes from a partitioning of the event R 1,L < R n, ′ n , the number of equality signs suffixing the sequence

≼ 1 ,...,≼ k-1 ,≼ ′ k ,≼ ′′ k ,≼ k+1 ,...,≼ n-1
is the same as the number of equality signs suffixing the sequence ≼ 1 ,...,≼ n-1 , which is indeed one less than in the sequence ≼ 1 ,...,≼ n we started with. This shows that we can progressively bring this number to zero, while otherwise only summing other terms that we can already compute. This completes the induction argument. ∎

The fact that when G satisfies the Ghirlanda-Guerra identities, the law of R 1,2 must be supported on R ⩾0 is shown in Exercise 5.13. Theorem 5.29 is now an immediate consequence of Theorem 5.30 and Proposition 5.31.

As was discussed below the statement of Theorem 5.29, when the measure ζ in (5.98) has finite support, the overlap arrays associated with Poisson-Dirichlet cascades are the canonical representatives of random overlap arrays that satisfy the Ghirlanda-Guerra identities. It is desirable to extend this construction and be able to also discuss such canonical representatives for more general choices of the measure ζ . This can easily be achieved by approximation, up to the extraction of a subsequence. In fact, it follows from Proposition 5.31 that no subsequence extraction is necessary. In particular, the law of the overlap array associated with a Poisson-Dirichlet cascade is a continuous function of ζ . Corollary 5.32. Let M > 0, let (ζ n ) n⩾1 be a sequence of probability measures with finite support in [0,M] that converge weakly to a probability measure ζ . For each

n ⩾ 1, let R (n) = (R (n)
, ′ ) ≠ ′ ⩾1 denote the overlap array associated with a Poisson-Dirichlet cascade such that the law of of R

(n) 1,2 is ζ n . Then (R (n) ) n⩾1 converges in law in the sense of finite-dimensional distributions.
Proof. Since each entry of R (n) takes values in [0,M], it is clear that we can extract subsequentially convergent limits. Let R denote one such limit array. By construction, the overlap R satisfies the Ghirlanda-Guerra identities (5.99) for continuous functions f and ψ. Using for instance Exercise A.8, we see that this property guarantees that the conditional law of R 1,n+1 given (R , ′ ) ≠ ′ ⩽n is the law displayed in (5.100), and so R satisfies the Ghirlanda-Guerra identities (5.99) for arbitrary bounded measurable functions f and ψ. By the Portmanteau theorem (Theorem A.17), we also see that R is ultrametric, and that the law of R 1,2 is ζ . By Proposition 5.31, the law of R is uniquely determined by these properties, so the proof is complete.

∎

If desired, we can apply the Dovbysh-Sudakov theorem (Theorem 1.7 in [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF]) to obtain that the limit overlap array in Corollary 5.32 can be represented in the form of (σ ⋅ σ ′ ) ≠ ′ , where (σ ) ⩾1 are independent samples from a random probability measure G on some Hilbert space.

Exercise 5.13 (Talagrand's positivity principle). Under the setting explained in the paragraph above (5.98), we assume that the random probability measure G satisfies the Ghirlanda-Guerra identities. Our goal is to show that the law ζ of one overlap, see (5.98), must be supported in R ⩾0 .

(i) Let A be a measurable subset of R, let n ⩾ 1 be an integer, and let A n denote the event that for every ≠ ′ ⩽ n, we have R , ′ ∈ A. Show that

1 A n+1 ⩾ 1 An 1 - n =1 1 {R ,n+1 ∉A} . (5.111)
(ii) Show that for every n ⩾ 1, we have

E⟨1 An ⟩ ⩾ (ζ (A)) n-1 . (5.112)
(iii) Conclude that the support of ζ must be a subset of R ⩾0 .

Chapter 6 Mean-field spin glasses

In this chapter, we enter more firmly into the realm of mean-field spin glasses. One of the most basic models in this class is the Sherrington-Kirkpatrick (SK) model. Like the Curie-Weiss model and the models of statistical inference studied in previous chapters, but unlike the Ising model on Z d , it is a mean-field model, in the sense that the variables that compose it are mutually exchangeable. And like the models of statistical inference, but unlike the Ising and Curie-Weiss models, the Hamiltonian defining the Gibbs measure of the SK model is random; in other words, the interactions between the variables are disordered. In the late 1970's, Giorgio Parisi proposed a self-contained description for the limit free energy of the SK model, now known as the Parisi formula, using sophisticated non-rigorous arguments [START_REF] Parisi | Infinite number of order parameters for spin-glasses[END_REF][START_REF] Parisi | The order parameter for spin glasses: a function on the interval 0-1[END_REF][START_REF] Parisi | A sequence of approximated solutions to the S-K model for spin glasses[END_REF]. This progressively led to important progress on the theoretical understanding of these models [START_REF] Mézard | Replica symmetry breaking and the nature of the spin glass phase[END_REF][START_REF] Mézard | Nature of the spin-glass phase[END_REF][START_REF] Mézard | Spin glass theory and beyond[END_REF][START_REF] Parisi | Order parameter for spin-glasses[END_REF], at the physics level of rigour. A mathematical proof of the Parisi formula was then given in the early 2000's in [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Talagrand | The Parisi formula[END_REF]. A more robust proof was later found in [START_REF] Panchenko | The Parisi ultrametricity conjecture[END_REF][START_REF] Panchenko | The Parisi formula for mixed p-spin models[END_REF] using the idea of ultrametricity discussed in the previous section, and also inspired by [START_REF] Aizenman | On the stability of the quenched state in mean-field spin-glass models[END_REF][START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF][START_REF] Arguin | On the structure of quasi-stationary competing particle systems[END_REF][START_REF] Bolthausen | On Ruelle's probability cascades and an abstract cavity method[END_REF][START_REF] Ghirlanda | General properties of overlap probability distributions in disordered spin systems. Towards Parisi ultrametricity[END_REF][START_REF] Panchenko | A connection between the Ghirlanda-Guerra identities and ultrametricity[END_REF][START_REF] Panchenko | Ghirlanda-Guerra identities and ultrametricity: an elementary proof in the discrete case[END_REF][START_REF] Panchenko | A unified stability property in spin glasses[END_REF].

As was already touched upon briefly in the overview to this book, there are many seemingly modest generalizations of the SK model, such as the bipartite model described in (0.4) and represented in Figure 0.2, that remain poorly understood mathematically. An important motivation for writing this book is to lay some groundwork for making progress on this problem.

We will not provide a proof of the Parisi formula in this book. Rather, our goal is to explain how the Parisi formula relates to the Hamilton-Jacobi approach, and more specifically, how it can be interpreted as the solution to an infinite-dimensional Hamilton-Jacobi equation evaluated at a point. In Section 6.1, we describe the SK model and state the Parisi formula. In Section 6.2, we explore a naive attempt at enriching the free energy in the SK model by mimicking what we did in the context of statistical inference. This enriched free energy satisfies an approximate Chapter 6 Mean-field spin glasses Hamilton-Jacobi equation, with an error term that is the variance of the overlap between two replicas. We encountered a similar situation in statistical inference, and in this context, we could leverage the Nishimori identity to ensure that the error term vanishes in the limit of large system size. Unfortunately, this is not the case at low temperature in spin-glass models such as the SK model, and we will need to refine our approach. In Section 6.3, we briefly discuss a simplification of the SK model known as the random energy model, in the hope of gaining some insight into the appropriate way of enriching the free energy. This will hint at a potential role for the Poisson-Dirichlet cascades already discussed in Section 5.6. In Section 6.4, we use this object to define a more sophisticated enriched free energy, and argue informally that this object should satisfy an infinite-dimensional Hamilton-Jacobi equation, up to an error term that is expected to vanish in the limit of large system size. Moreover, the non-linearity in the equation associated with the SK model is convex, so the Hopf-Lax formula suggests a variational representation for the limit free energy. In Section 6.5, we connect this Hopf-Lax formula with the Parisi formula. Taking the validity of the Parisi formula for granted, this gives some substance to the heuristic derivations performed earlier. One of the advantages of the Hamilton-Jacobi approach is that it leads to the formulation of a natural conjecture for the limit free energy of more difficult models such as the bipartite model, for which variational formulas break down. In Section 6.6, we conclude this chapter by discussing the state of current research on this point.

The Sherrington-Kirkpatrick model and the Parisi formula

The SK model is often motivated as a pure optimization problem known as the Dean's problem. We imagine that the Dean of a university is tasked with dividing a group of N students indexed by the elements of {1,...,N} into two dormitories indexed by -1 and +1. An allocation of the N students into the two dormitories can be identified with a configuration vector

σ = (σ 1 ,...,σ N ) ∈ Σ N ∶= {-1,+1} N . (6.1)
To decide on dormitory allocations, the Dean is provided with a collection of parameters (g i j ) i, j⩽N called interaction parameters which describe how much student i likes or dislikes student j. A positive parameter means that student i likes student j, while a negative parameter means that student i dislikes student j. The Dean strives to maximize the "comfort function"

c N (σ ) ∶= N i, j=1 g i j σ i σ j . (6.2)
We define the comfort function in this way so that it most resembles an Ising-type model; the possibly more natural choice of ∑ i, j g i j

1 {σ i =σ j } = 1 2 c N (σ ) + 1 2 ∑ i, j g i j
would not change the analysis much. The story of the Dean is a simple illustration that "complex" optimization problems of this form arise quite naturally. For arbitrary choices of the coefficients (g i j ) i, j⩽N , finding near-optimizers to the function c N is known to be NP-hard [START_REF] Arora | On nonapproximability for quadratic programs[END_REF]. Instead of this worst-case analysis, we would like to consider the problem of optimizing the function c N for a "typical" instantiation. We encode this by positing that the interaction parameters (g i j ) i, j⩽N are independent standard Gaussian random variables. The key assumption here is that the random variables are independent and identically distributed; arguing as in the proof of Theorem 4.24 allows us to extend many results to a much broader class of distributions, see for instance Exercise 6.4. The Hamiltonian of the SK model is defined as the comfort function c N (σ ), suitably normalized, and for this choice of independent standard Gaussian interaction parameters (g i j ) i, j⩽N . Precisely, for every σ ∈ Σ N , we set

H N (σ ) ∶= 1 √ N N i, j=1 g i j σ i σ j . (6.3)
We may as well think of H N as a function defined on R N . The family (H N (σ )) σ ∈R N is a centred Gaussian process with covariance structure

EH N (σ 1 )H N (σ 2 ) = 1 N N i, j=1 σ 1 i σ 1 j σ 2 i σ 2 j = N 1 N N i=1 σ 1 i σ 2 i 2 = NR 2 1,2 (6.4)
that depends on the spin configurations σ 1 and σ 2 only through their normalized scalar product, or overlap,

R 1,2 ∶= 1 N σ 1 ⋅ σ 2 = 1 N N i=1 σ 1 i σ 2 i . (6.5)
Notice that we could also have directly defined (H N (σ )) σ ∈R N by specifying that it is a centred Gaussian process whose covariance is given by (6.4); the main utility of the explicit formula (6.3), besides its possibly more intuitive appeal, is that it guarantees the existence of such a process. The law of (H N (σ )) σ ∈R N is invariant under permutations of the coordinates of σ . This is why the SK model is said to be mean-field. Instead of focusing on analyzing the maximum

1 N max σ ∈Σ N H N (σ ), (6.6)
and consistently with the rest of the book, we prefer to analyze the free energy

F N (β ) ∶= 1 N Elog σ ∈Σ N exp β H N (σ ) (6.7)
at each inverse temperature β > 0. The reason for the normalization by √ N in (6.3) is that the configurations σ that contribute meaningfully to the sum in (6.7) are such that H N (σ ) is of order N (see for instance Exercise 6.1), so that the exponential can be in balance with the cardinality of Σ N . As we vary the parameter β , the sum will therefore interpolate between a small-β situation in which the main contribution consists of the very numerous terms that have H N (σ ) not very large, and a large-β situation where the sum is dominated by the few terms for which H N (σ ) is very close to the maximum. For large β , this is made more precise in Exercise 6.3.

A first formula for the limit of the free energy F N (β ) was proposed by Sherrington and Kirkpatrick in [START_REF] Sherrington | Solvable model of a spin-glass[END_REF]. However, in this same work, they observed that their formula predicted that the entropy of the limit Gibbs measure would become negative for large β , but we recall from (1.7) that the entropy is always non-negative, so their proposed solution could not be valid for large β . A few years later, Giorgio Parisi [START_REF] Parisi | Infinite number of order parameters for spin-glasses[END_REF][START_REF] Parisi | The order parameter for spin glasses: a function on the interval 0-1[END_REF][START_REF] Parisi | A sequence of approximated solutions to the S-K model for spin glasses[END_REF] proposed a more sophisticated formula now known as the Parisi formula, which was then established rigorously in [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Talagrand | The Parisi formula[END_REF] (see also [START_REF] Panchenko | The Sherrington-Kirkpatrick model[END_REF]). It states that lim

N→+∞ F N (β ) = inf ζ ∈D[0,1] Φ ζ (0,0) -β 2 ˆ1 0 tζ (t)dt + log(2) , (6.8)
where

Φ ζ ∶ [0,1] × R → R is the solution to the parabolic equation ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -∂ t Φ ζ (t,x) = β 2 ∂ 2 x Φ ζ (t,x) + ζ (t) ∂ x Φ ζ (t,x) 2 on [0,1] × R d Φ ζ (1,x) = logcosh(x) for x ∈ R (6.9)
solved backwards in time from t = 1 to t = 0, and where the infimum in (6.8) is taken over the space

D[0,1] ∶= ζ ∶ [0,1] → [0,1] ζ is right-continuous and
non-decreasing with ζ (1) = 1 (6.10) of probability distribution functions on [0,1]. The equation (6.9) is sometimes called the Parisi PDE. One of the main goals of this chapter is to recast this statement using the point of view provided by the Hamilton-Jacobi approach. Before doing this, we mention two directions in which the SK model can be generalized. On the one hand, the covariance structure (6.4) can be generalized in the following way. Given a sequence (β p ) p⩾1 of non-negative numbers that decrease sufficiently rapidly, the Hamiltonian

H N (σ ) ∶= p⩾1 β 1 2 p H N,p (σ ) (6.11)
of the mixed p-spin model on Σ N is given by a linear combination of pure p-spin Hamiltonians on Σ N ,

H N,p (σ ) ∶= 1 N (p-1) 2 N i 1 ,...,ip=1 g i 1 ...ip σ i 1 ⋯σ ip , (6.12)
where (g i 1 ...ip ) i 1 ,...,ip⩽N are independent standard Gaussian random variables, independent of each other as we vary p. The Hamiltonian (6.11) is a centred Gaussian process with covariance structure

EH N (σ 1 )H N (σ 2 ) = Nξ (R 1,2 ) (6.13)
that depends on the spin configurations σ 1 and σ 2 only through their overlap R 1,2 , and where we have set ξ (x) ∶=

p⩾1 β p x p . (6.14)
The SK model corresponds to the choice β p = 1 {p=2} , and so ξ (x) = x 2 . More than the explicit form of H N (σ ) given in (6.11) and (6.12), it is the function ξ characterizing the covariance structure of the Hamiltonian that will play a central role. In particular, this function is essentially the non-linearity in the Hamilton-Jacobi equation we will need to consider. We refer to the function ξ as the covariance function. The limit of the free energy

F N (β ) ∶= 1 N Elog σ ∈Σ N exp β H N (σ ) (6.15)
in the mixed p-spin models admits a variational representation analogous to (6.8).

The connection between the SK model and the Hamilton-Jacobi approach that will be presented in this chapter extends to this more general setting [START_REF] Mourrat | The Parisi formula is a Hamilton-Jacobi equation in Wasserstein space[END_REF].

Another possible way to generalize the SK model consists in considering that each coordinate of the configuration vector (6.1) can itself be a vector; the bipartite model is an example of such a setup. For illustration, we briefly motivate the bipartite model through a slightly modified version of the Dean's problem.

Suppose the Dean of a university actually needs to split students and tutors into two groups. We assume that there is an equal number N of students and tutors for convenience, but this is inessential. An allocation of the 2N students and tutors into the two groups +1 and -1 can be identified with a vector

σ = (σ 1 ,σ 2 ) = (σ 1,1 ,σ 1,2 ,...,σ 1,N ),(σ 2,1 ,σ 2,2 ,...,σ 2,N ) ∈ Σ 2 N , (6.16)
where σ 1 ∈ Σ N encodes the group allocations for the students and σ 2 ∈ Σ N the group allocations for the tutors. We postulate that the Dean now only has access to a collection of parameters (g i j ) i, j⩽N , where g i j encodes the quality of the interaction between student i and tutor j. The Dean aims to maximize the comfort function

c N (σ ) ∶= N i, j=1 g i j σ 1,i σ 2, j (6.17)
over all σ ∈ Σ 2 N . Choosing the interaction parameters (g i j ) i, j⩽N to be independent standard Gaussian random variables and re-scaling c N , we define the bipartite model [START_REF] Fyodorov | Antiferromagnetic ising spin glass[END_REF][START_REF] Fyodorov | Phase transitions in frustrated metamagnets[END_REF][START_REF] Kincaid | Phase diagrams of liquid helium mixtures and metamagnets: experiment and mean field theory[END_REF][START_REF] Ya | Spin glass in an lsing two-sublattice magnet[END_REF] whose Hamiltonian on Σ 2 N is given by

H N (σ ) ∶= 1 √ N N i, j=1 g i j σ 1,i σ 2, j . (6.18)
The Hamiltonian (6.18) is a centred Gaussian process with covariance structure

EH N (σ 1 )H N (σ 2 ) = N σ 1 1 ⋅ σ 2 1 N σ 1 2 ⋅ σ 2 2 N . (6.19)
So in this model, the relevant covariance function takes the two "intra-community" overlaps as arguments, and is defined on R 2 by ξ (x,y) ∶= xy. (

The fundamental difference between the bipartite model and the mixed p-spin models is that the covariance function ξ is always convex for the latter, but not for the former. To be precise, for scalar (as opposed to vector) models, for which the function ξ is defined on R, the relevant question is whether the function is convex on R ⩾0 , and the function in (6.14) satisfies this convexity property for every choice of the non-negative parameters (β p ) p⩾1 . Due to the absence of a comparable convexity property for the bipartite model, the limit of the free energy

1 N Elog σ ∈Σ 2 N exp β H N (σ ) (6.21)
with H N as in (6.18) has not yet been identified. As will be explained below, the Hamilton-Jacobi approach suggests a natural conjecture for what this limit could be. Similar considerations apply to essentially arbitrary vector spin-glass models, provided that the covariance of the Hamiltonian can be expressed as a function of the matrix of overlaps between the different types of coordinates. The dividing line between the models for which the limit free energy has or has not been identified is exactly whether or not the underlying covariance function ξ is convex. In the most general models in which each spin configuration σ can be decomposed into a fixed number D of different components σ = (σ 1 ,...,σ D ), the covariance function takes the matrix of all possible overlaps N -1 (σ

1 d ⋅ σ 2 d ′ ) d,d ′ ⩽D
as input, and the precise question is whether the function ξ is convex over the space of D-by-D positive semi-definite matrices. Exercise 6.1. For the Hamiltonian H N defined in (6.3), show that there exists a constant C < +∞ such that

C -1 N ⩽ E max σ ∈Σ N H N (σ ) ⩽ CN. (6.22)
Exercise 6.2. For the Hamiltonian H N defined in (6.3), prove that almost surely,

lim N→+∞ 1 N max σ ∈Σ N H N (σ ) - 1 N E max σ ∈Σ N H N (σ ) = 0. (6.23)
Exercise 6.3. For the Hamiltonian H N defined in (6.3) and the free energy F N (β ) defined in (6.7), show that

1 N E max σ ∈Σ N H N (σ ) ⩽ F N (β ) ⩽ 1 N E max σ ∈Σ N H N (σ ) + log(2) β . (6.24)
Exercise 6.4. Fix a collection (x i j ) i, j⩽N of i.i.d. centred random variables with variance one, and write

H x N (σ ) ∶= 1 √ N i, j⩽N x i j σ i σ j and F x N (β ) ∶= 1 N Elog σ ∈Σ N exp β H x N (σ ) (6.25)
for their associated SK Hamiltonian and free energy. Denote by F N (β ) the Gaussian SK free energy (6.7). Assuming that E x 11 3 < +∞, show that for every inverse temperature parameter β > 0,

limsup N→+∞ F x N (β ) -F N (β ) = 0. (6.26)

A first attempt at a Hamilton-Jacobi approach to the SK model

We now return to the setting of the SK model. We prefer to slightly change notation and thereby slightly generalize the model in the following way. We fix a probability measure P 1 on Σ 1 , and for each integer N ⩾ 1, let

P N ∶= (P 1 ) ⊗N (6.27)
denote its N-fold tensor product. Given a "time" parameter t ⩾ 0, we introduce the free energy

F ○ N (t) ∶= - 1 N Elog ˆΣN exp √ 2tH N (σ ) -Nt dP N (σ ), (6.28)
where the Hamiltonian H N is that of the SK model, see (6.3). When P 1 is the uniform measure on Σ 1 , the free energy (6.7) we had defined earlier is related to this new quantity according to

F N (β ) = -F ○ N β 2 2 + log(2) + β 2 2 . (6.29)
So we may as well focus on trying to compute F ○ N (t) for each value of t ⩾ 0. By (6.4), for every σ ∈ Σ N , the random variable H N (σ ) is a centred Gaussian with variance N, so Eexp

√ 2tH N (σ ) -Nt = 1. (6.30)
In the language of statistical physics, one could say that we have normalized the Hamiltonian so that the annealed free energy is always zero. Combining (6.30) with Jensen's inequality shows that F ○ N (t) ⩾ 0; this is why we chose to include a negative sign in front of the free energy (6.28). Choosing to parameterize the inverse temperature by √ 2t in place of β is convenient because then the variance of the Gaussian field √ 2tH N (σ ) scales linearly, as a Brownian motion would. As was the case in Chapter 4, see in particular the discussion around (4.93), this parametrization then satisfies a certain semigroup property. While we will not dwell much on this point, this choice of parametrization will allow us in particular to obtain expressions for the derivatives of the free energy that do not explicitly depend on t.

In the expression (6.28) for the free energy, we chose to use an integral symbol for what turns out to be just a sum over Σ N , possibly with some suitable weights if the measure P 1 is not centred. The main reason we chose to denote it in this way is that the results we will discuss concerning F ○ N (t) can be generalized straightforwardly to the case where the measure P 1 is arbitrary with compact support, provided that we replace the term -Nt in (6.28) by -t σ 4 N, or more generally by -Ntξ (N -1 σ 2 ) for the mixed p-spin models in (6.13). In other words, this term should be half the variance of √ 2tH N (σ ), so that the identity in (6.30) generalizes nicely. If the limit of F ○ N (t) can be identified, then the possibly unwelcome extra term -Ntξ (N -1 σ 2 ) can later be removed by essentially quoting Proposition 3.20 on the generalized Curie-Weiss model. This is explained using a slightly different argument in Section 5 of [START_REF] Mourrat | Extending the Parisi formula along a Hamilton-Jacobi equation[END_REF]. Here we will stick to the case in which P 1 is supported on Σ 1 , in which case this additional term is constant and therefore does not cause any difficulty.

As in Section 4.3 on statistical inference, there is no hope of finding a partial differential equation solved by the function F ○ N (t) in (6.28) on its own. Rather, we need to enrich the Hamiltonian to contain more terms, so that the free energy then depends on additional parameters. The goal then is to find additional terms that are sufficiently simple that we can analyze them when setting t = 0; but that are sufficiently expressive that we can compensate any small variation in t by small variations in the other parameters, thereby obtaining a Hamilton-Jacobi equation. To get a first rough intuition for what one could do, we start by rewriting the Hamiltonian H N (σ ) for the SK model as

N i=1 1 √ N N j=1 g i, j σ j σ i . (6.31)
Inspired by the analysis of the previous chapters, we would like to try to only add terms to the Hamiltonian that do not encode any interaction between the variables; in other words, linear functions of σ would be best. We would like to add a term that would at least somewhat resemble (6.31); since the term 1 √ N ∑ N j=1 g i, j σ j is random, we may try adding a new term in the Hamiltonian proportional to ∑ N i=1 z i σ i , where z = (z 1 ,...,z N ) ∈ R N is a standard Gaussian vector.

The brief informal argument we just gave is at best a first hint for what one could try to do. We will not attempt to make it more precise here, and instead just proceed with the suggested idea and see how well it fares. So we fix a standard Gaussian vector z = (z 1 ,...,z N ) ∈ R N independent of (H N (σ )) σ ∈R N , and for each h ⩾ 0, we consider the enriched free energy

F N (t,h) ∶= - 1 N Elog ˆΣN exp √ 2tH N (σ ) -Nt + √ 2hz ⋅ σ -Nh dP N (σ ). (6.32)
We have kept the practice of adding the compensating term -Nh, i.e. half the variance of √ 2hz ⋅ σ , in the exponential. For measures P N that are not necessarily supported on Σ N , replacing this term by -h σ 2 (and -Nt by -Ntξ (N -1 σ 2 )) ensures that all the calculations below remain valid. As usual, we write ⟨⋅⟩ for the Gibbs average associated with the Hamiltonian

H N (t,h,σ ) ∶= √ 2tH N (σ ) -Nt + √ 2hz ⋅ σ -Nh, (6.33)
we denote by σ the canonical random variable under ⟨⋅⟩, and by (σ ) ⩾1 independent copies of σ under ⟨⋅⟩. We stress that the probability measure ⟨⋅⟩ is random, since it depends on the realization of (H N (σ )) σ ∈R N (or equivalently, of (g i, j ) 1⩽i, j⩽N ) and z.

Leveraging the Gibbs Gaussian integration by parts formula (Theorem 4.6), it is readily verified that

∂ h F N (t,h) = - 1 N E⟨ 1 √ 2h z ⋅ σ -N⟩ = E⟨R 1,2 ⟩, (6.34)
while

∂ t F N (t,h) = - 1 N E⟨ 1 √ 2t H N (σ ) -N⟩ = E⟨R 2 1,2 ⟩. (6.35)
This implies that

∂ t F N (t,h) -∂ h F N (t,h) 2 = Var(R 1,2 ), (6.36)
where we write Var to denote the variance over all sources of randomness, i.e. with respect to the measure E⟨⋅⟩. The formula (6.36) looks very similar to what we found in (3.5) for the Curie-Weiss model and in (4.83) for symmetric rank-one matrix estimation, and suggests that the limit f of the enriched free energy F N should satisfy the Hamilton-Jacobi equation

∂ t f -∂ h f 2 = 0 on R >0 × R >0 . (6.37)
Unfortunately, unlike previous models, in the spin-glass setting this guess is too naive. More precisely, this guess can be shown to be correct for small values of t ⩾ 0, where it can be used to recover the original formula of Sherrington and Kirkpatrick; but it fails for large values of t ⩾ 0, where the variance of the overlap R 1,2 no longer vanishes in the large N limit. In the language of statistical physics, the SK model exhibits "replica symmetry breaking" at low temperature. Physicists use the phrase "replica symmetry breaking" because if we sample four replicas σ 1 ,...,σ 4 from the Gibbs measure, one could expect by symmetry to witness the same overlap value between σ 1 and σ 2 and between σ 3 and σ 4 . The symmetry between the replicas is not literally broken, but indeed these overlap values will in general be different because the overlap between two replicas is random, and remains so even in the limit of large system size.

To sum up, the consideration of the enriched free energy in (6.32) seems to be pointing in the right direction, but is insufficient, and we therefore need a more refined way of enriching the free energy.

Some insight from the random energy model

To gain a deeper understanding of the structure of the Gibbs measure and determine how to appropriately enrich the free energy (6.28), we take a small detour and analyze a simplified model called the random energy model, or REM [START_REF] Derrida | Random-energy model: an exactly solvable model of disordered systems[END_REF]. The Hamiltonian √ 2tH N (σ ) appearing in the free energy (6.28) is a family of Gaussian random variables indexed by σ ∈ Σ N with variance 2tN. The main difficulty in understanding the behaviour of this Hamiltonian is that these Gaussian random variables are correlated, by (6.4). If we consider pure p-spin models instead, then the covariance between H N (σ 1 ) and H N (σ 2 ) becomes NR p 1,2 . For σ 1 ,σ 2 ∈ Σ N and in the limit of large p, this would simplify to N1 {R 1,2 =1} . That is to say, in this limit, the random variables (H N (σ )) σ ∈Σ N have become independent. The random energy model corresponds to setting exactly this as the definition: for a family (E σ ) σ ∈Σ N of independent standard Gaussian random variables, t ⩾ 0 and σ ∈ Σ N , we set

H REM N (t,σ ) ∶= √ 2tNE σ . (6.38)
Our goal in this section is to inquire about the structure of the associated Gibbs measure,

G REM N (t,σ ) = expH REM N (t,σ ) ∑ τ∈Σ N expH REM N (t,τ) , (6.39)
especially for large values of t. The hope is that the Gibbs measure of "true" spin glasses behaves similarly, possibly after clumping together neighbouring configurations in some way so as to tame the effect of correlations. Since we are dealing with a family of independent standard Gaussian random variables, a natural starting point is to appeal to Proposition 5.16. Substituting n by 2 N there, we introduce the constant

a N ∶= 2N log(2) -log(N) -loglog(2) -log(4π) 1 2 , (6.40)
so that Proposition 5.16 ensures that the point process

σ ∈Σ N δ a N (Eσ -a N ) = σ ∈Σ N δ a N √ 2tN (H REM N (t,σ )- √ 2tNa N ) (6.41)
converges in law on R to a Poisson point process with intensity dν(x) ∶= e -x dx. If we introduce the parameter

ζ ∶= log (2) t (6.42)
and observe that a N √ 2tN

= ζ + o(1), this suggests that the point process

Λ N ∶= σ ∈Σ N δ ζ H REM N (t,σ )-a 2 N (6.43)
should also converge in law on R to a Poisson point process with intensity ν. Indeed, the lower-order terms should not impact the convergence. Perhaps the simplest way to justify this is to essentially repeat the proof of Proposition 5.16.

Proposition 6.1. The point process Λ N in (6.43) converges in law on R to a Poisson point process with intensity dν(x) ∶= e -x dx.

Proof. Fix a function f ∈ C c (R;R ⩾0 ) of compact support, and observe that

Eexp -ˆR f dΛ N = exp 2 N log 1 - 1 4πtNζ 2 ˆR 1 -e -f (x) exp - (x + a 2 N ) 2 4tNζ 2 dx .
Using the definition of the constant a N in (6.40), the asymptotic expansion

a 2 N 2tNζ 2 = 1 + o(1),
and the fact that f is of compact support reveals that uniformly over x in the support of f ,

2 N 4πtNζ 2 exp - (x + a 2 N ) 2 4tNζ 2 = 2 N 4πtNζ 2 exp -x - 1 2 a 2 N + o(1) = e -x + o(1).

It follows by a Taylor expansion of the logarithm that

Eexp -ˆR f dΛ N = exp - ˆR 1 -e -f (x) e -x dx + o(1).
Invoking Propositions 5.4 and 5.9 completes the proof. ∎

Together with the mapping theorem (Proposition 5.10) applied to the function f (x) ∶= e x ζ , this result implies that the point process

σ ∈Σ N δ exp(H REM N (t,σ )-a 2 N ζ ) (6.44)
converges in law on R >0 to the Poisson point process with intensity measure

dµ(x) ∶= ζ x ζ +1 dx. ( 6.45)
Notice that the Gibbs measure (6.39) is invariant under translations of the Hamiltonian (6.38) by a constant, since

G REM N (t,σ ) = expH REM N (t,σ ) ∑ τ∈Σ N expH REM N (t,τ) = exp H REM N (t,σ ) -a 2 N ζ ∑ τ∈Σ N exp H REM N (t,τ) -a 2 N ζ . (6.46)
In view also of Exercise 5.8, this suggests that the point process supported on the Gibbs weights of the REM should converge in law to the re-normalization of the Poisson point process with intensity measure (6.45). More precisely, if we assume that ζ ∈ (0,1) so that the Poisson-Dirichlet process (v n ) n⩾1 is well-defined, then as N tends to infinity we should expect that

σ ∈Σ N δ G REM N (t,σ ) converges in law to +∞ n=1 δ vn . (6.47)
Observe that ζ ∈ (0,1) if and only if t > log [START_REF] Abbe | Exact recovery in the stochastic block model[END_REF]. It is possible to show that in this regime, the convergence in law (6.47) does indeed take place. We therefore expect the Poisson-Dirichlet process to play an important role in the enrichment of the free energy (6.28).

A Hamilton-Jacobi approach to the SK model

In the previous section, we suggested that the weights of the Gibbs measure at low temperature, perhaps after some suitable grouping-together of neighbouring sites, might resemble the weights of the Poisson-Dirichlet process introduced in Section 5.5. Although we will not aim to justify this precisely, this intuition turns out to be valid. Recall that our ultimate goal is to find a more refined random magnetic field to replace √ 2hz in (6.32) so that we can find relationships between derivatives with respect to the various parameters. Recall also from (6.35) that the t-derivative of the free energy is the second moment of the overlap R 1,2 , and that using the naive random magnetic field √ 2hz, we could create the relation (6.36) which replaces the second moment of R 1,2 by its variance. In other words, in this case we would be done if R 1,2 was constant. Now, if we were to enrich the free energy with a random magnetic field that has the overlap of a Poisson-Dirichlet process, then this would still not suffice in general. Indeed, we see from (5.56) that the overlap of a Poisson-Dirichlet process can only take two values, so armed with this, all we could possibly hope for would be to compensate for circumstances in which R 1,2 itself is known to take only two values. Compared with the naive attempt in (6.32), this seems to mark some further progress, but it is still insufficient in general. Aspects of the geometry of the Gibbs measure are important, beyond the simple recording of the statistics of the Gibbs weights.

Our work in Sections 5.6 and 5.7 suggests a clear way forward: Poisson-Dirichlet cascades will provide us with the richest possible class of overlap structures we can expect to encounter. This is admittedly still only a relatively vague intuition. We will not push much towards giving it more substance, and mostly stick again to the attitude of just trying it out and seeing that things now actually work well. For the curious and motivated reader, we point out however the following possibly interesting route to explore. Below (6.31), we gestured at the idea that, since the term 1 √ N ∑ N j=1 g i, j σ j is random, it might make sense to add a new term in the Hamiltonian proportional to ∑ N i=1 z i σ i , where z = (z 1 ,...,z N ) ∈ R N is a standard Gaussian vector. To be more precise, what we are really concerned about is the law of 1 √ N ∑ N j=1 g i, j σ j under the Gibbs measure for a "typical" but fixed realization of the random coefficients (g i, j ) i, j⩽N . Exercise 6.5 explores this point in a simple setting, and suggests that aspects of the structure of the Gibbs measure such as whether the overlap is concentrated must be taken into consideration here.

We now proceed to construct the refined magnetic field that will be used as a replacement for the simple field √ 2hz appearing in (6.32). Given an integer K ⩾ 1 and parameters

0 = ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, (6.48) 0 = q -1 ⩽ q 0 < q 1 < ⋯ < q K < q K+1 = +∞, (6.49)
as in (5.67) and (5.82), we define the right-continuous non-decreasing path q ∶ [0,1) → R ⩾0 by

q ∶= K k=0 q k 1 [ζ k ,ζ k+1 ) . (6.50)
The reason for storing the families of parameters (ζ k ) 1⩽k⩽K and (q k ) 0⩽k⩽K in the form of this path will become clear as we proceed. We recall that we denote by A the tree in (5.63), whose set of leaves is N K , that α denotes the ancestor of α at depth as in (5.65), and that α ∧ β denotes the depth of the most common ancestor of the leaves α and β as in (5.66). We give ourselves a family (z α ) α∈A of independent standard Gaussian random vectors in R N independent of all other sources of randomness, and set, for every α ∈ N K ,

Z q (α) ∶= K k=0 (2q k -2q k-1 ) 1 2 z α k . (6.51)
The family of random variables (Z q (α)) α∈N K is Gaussian, and its covariance structure is given, for every α 1 ,α 2 ∈ N K , by

EZ q (α 1 )Z q (α 2 ) = q α 1 ∧α 2 . (6.52)
This will be our new random external field. We let (v α ) α∈N K be a Poisson-Dirichlet cascade with parameters (ζ k ) 1⩽k⩽K , independent of all other sources of randomness, and we pair these two objects in our definition of the enriched free energy as

F N (t,q) ∶= - 1 N Elog ˆΣN α∈N K exp H N (t,q,σ ,α) v α dP N (σ ), (6.53)
for the enriched Hamiltonian

H N (t,q,σ ,α) ∶= √ 2tH N (σ ) -Nt + Z q (α) ⋅ σ -Nq K . (6.54)
We denote by ⟨⋅⟩ the associated Gibbs measure, with canonical random variable (σ ,α). Explicitly, this means that for every bounded measurable function

f ∶ R N × N K → R, we have ⟨ f (σ ,α)⟩ ∶= ´ΣN ∑ α∈N K f (σ ,α)exp H N (t,q,σ ,α) v α dP N (σ ) ´ΣN ∑ α∈N K exp H N (t,q,σ ,α) v α dP N (σ ) . (6.55)
We denote by (σ ,α ) ⩾1 independent copies of (σ ,α) under ⟨⋅⟩, also called replicas. We may now discuss two types of overlaps: those that involve the σ variables, which we may call the σ -overlaps and keep writing as

R 1,2 ∶= 1 N i⩽N σ 1 i σ 2 i ; (6.56)
and those that involve the α variables, which we may call the α-overlaps and will simply denote by α 1 ∧ α 2 .

Choosing P 1 to be the uniform measure on Σ 1 , the free energy (6.7) in the SK model is related to this enriched free energy (6.53) by

F N (β ) = -F N β 2 2 ,0 + log(2) + β 2 2 . ( 6.57)
Using the Gaussian integration by parts formula, we now compute the derivatives of the enriched free energy with respect to t and (q k ) 0⩽k⩽K . We will find a relationship between these quantities, up to an error expressed in terms of the variance of the σ -overlap given the α-overlap. The conditional expectation at play here is with respect to the measure E⟨⋅⟩. For instance, we may write, for any function

f ∶ Σ 2 N → R, E⟨ f (σ 1 ,σ 2 ) α 1 ∧ α 2 ⟩ ∶= K k=0 1 {α 1 ∧α 2 =k} E⟨1 {α 1 ∧α 2 =k} f (σ 1 ,σ 2 )⟩ E⟨1 {α 1 ∧α 2 =k} ⟩ . (6.58)
Before we proceed, we recall that in Theorem 5.28, specifically in (5.93), we managed to identify the law of the α-overlap when the underlying Gibbs measure is simply the measure with weights (v α ) α∈N K on the leaves of the tree. One key observation is that the proof of this result carries verbatim to the present setting with a more complicated Gibbs measure. The point is that the Bolthausen-Sznitman invariance property of Poisson-Dirichlet cascades stated in Lemma 5.27 remains valid even if we multiply each w α by some fixed function of Z q (α), since the family (Z q (α)) α∈N K is independent of (w α ) and its law is invariant under permutations that preserve the tree structure. The other random field denoted by (Z q (α)) α∈N K there should be taken as independent from the random field (Z q (α)) α∈N K we have defined here. In particular, we may replace each w α by

w α ˆΣN expH N (t,q,σ ,α)dP N (σ ), (6.59)
and then proceed through the proof of Theorem 5.28 to obtain that, with the Gibbs measure as in (6.55), we still have for every k ∈ {0,...,K} that

E⟨1 {α 1 ∧α 2 =k} ⟩ = ζ k+1 -ζ k . (6.60)
Lemma 6.2. For every t ⩾ 0 and every path q ∶ [0,1] → R of the form (6.50), we have

∂ t F N (t,q) = E⟨R 2 1,2 ⟩, (6.61)
and for every k ∈ {0,...,K},

∂ q k F N (t,q) = E⟨1 {α 1 ∧α 2 =k} R 1,2 ⟩. (6.62)
In particular,

∂ t F N (t,q) - K k=0 (ζ k+1 -ζ k ) ∂ q k F N (t,q) ζ k+1 -ζ k 2 = E⟨ R 1,2 -E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩. (6.63)
Proof. The same calculation as for (6.35) still gives

∂ t F N (t,q) = - 1 N E⟨ 1 √ 2t H N (σ ) -N⟩ = E⟨R 2 1,2 ⟩.
Concerning the derivative with respect to q k , the Gibbs Gaussian integration by parts formula (Theorem 4.6) implies that for every k ∈ {0,...,K -1},

∂ q k F N (t,q) = - 1 N E⟨(2q k -2q k-1 ) -1 2 z α k ⋅ σ -(2q k+1 -2q k ) -1 2 z α k+1 ⋅ σ ⟩ = 1 N E⟨ 1 {α 1 k =α 2 k } -1 {α 1 k+1 =α 2 k+1 } σ 1 ⋅ σ 2 ⟩ = E⟨1 {α 1 ∧α 2 =k} R 1,2 ⟩.
For k = K, the last term on the first two lines above disappears, and we have

∂ q K F N (t,q) = 1 N E⟨1 {α 1 K =α 2 K } σ 1 ⋅ σ 2 ⟩ = E⟨1 {α 1 ∧α 2 =K} R 1,2 ⟩.
So the relation (6.62) is indeed valid for every k ∈ {0,...,K}. Combining this with (6.60) shows that

∂ q k F N (t,q) ζ k+1 -ζ k = E⟨1 {α 1 ∧α 2 =k} R 1,2 ⟩ E⟨1 {α 1 ∧α 2 =k} ⟩ = E⟨R 1,2 α 1 ∧ α 2 = k⟩. (6.64)
We next observe that the right side of (6.63) can be rewritten as

E⟨R 2 1,2 ⟩ -E⟨ E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩ = ∂ t F N (t,q) - K k=0 E⟨1 {α 1 ∧α 2 =k} E⟨R 1,2 α 1 ∧ α 2 = k⟩ 2 ⟩ = ∂ t F N (t,q) - K k=0 (ζ k+1 -ζ k ) ∂ q k F N (t,q) ζ k+1 -ζ k 2 .
This completes the proof. ∎ By Jensen's inequality, we have

E⟨ E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩ ⩾ E⟨R 2 1,2 ⟩, (6.65)
and thus

E⟨ R 1,2 -E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩ ⩽ E⟨ R 1,2 -E⟨R 2 1,2 ⟩ 2 ⟩. (6.66)
So the right side of (6.63), which we think of as an error term, is smaller than or equal to the error term we found in (6.36) in our naive attempt at enriching the free energy.

The right side of (6.63) measures the size of the conditional variance of the σ -overlap given the α-overlap. Can we ensure that this is small? In other words, can we assert that we can essentially infer the σ -overlap from the observation of the α-overlap? Recall also that, as for the Curie-Weiss model and the models of statistical inference, we do not aim for the error term to be small for absolutely all choices of the parameters, but only for "most" such choices.

As was discovered in [START_REF] Panchenko | The free energy in a multi-species Sherrington-Kirkpatrick model[END_REF][START_REF] Panchenko | Free energy in the Potts spin glass[END_REF][START_REF] Panchenko | Free energy in the mixed p-spin models with vector spins[END_REF], the Panchenko ultrametricity theorem (Theorem 5.30) implies the following synchronization phenomenon: up to a small perturbation of the Hamiltonian (to ensure a family of Ghirlanda-Guerra identities), the σ -and the α-overlaps must be asymptotically monotonically coupled. A pair of real random variables (X,Y ) is said to be monotonically coupled if, denoting by (X ′ ,Y ′ ) an independent copy of (X,Y ), we have

P{X < X ′ and Y ′ < Y } = 0. ( 6.67)
Denoting by F -1 X and F -1 Y the inverse cumulative distribution functions of X and Y respectively, one can show (see Proposition 2.1 in [START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF]) that the pair (X,Y ) is monotonically coupled if and only if its law is that of (F -1

X (U),F -1 Y (U))
, where U is a uniform random variable on the interval [0,1].

In the monotonically coupled pair (X,Y ), we think of X as the σ -overlap and of Y as the α-overlap, and wonder whether we can recover X from the observation of Y . Naturally, if Y is deterministic, then the observation of Y cannot help much in predicting X. On the other hand, if the law of Y is nicely spread out, for instance if it is a uniform random variable on [0,1], then the observation of Y allows us to recover the "secret" random variable U in the representation above, and thus to perfectly infer the value of X.

Coming back to the analysis of the overlaps, we thus see that with the synchronization phenomenon in place, the error term on the right side of (6.63) will be small provided that the law of the α-overlap is sufficiently spread out. Recall also from (6.60) that we know the law of the α-overlap exactly. Combining all this, it was shown in Proposition 5.5 of [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF] that when a family of Ghirlanda-Guerra identities all hold asymptotically as N tends to infinity, and when ζ k+1ζ k = 1 (K + 1) for every k ∈ {0,...,K}, we have that

limsup N→+∞ E⟨ R 1,2 -E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩ ⩽ 12 K . (6.68)
This informal discussion motivates us to give more substance to the idea that the free energy F N (t,q) should really be thought of as a function of the path q, as was already suggested by the notation, and to seek to make sense of it for more general paths than the piecewise-constant paths we have so far considered. To start with, we give an informal justification of the fact that we decided to bundle together the parameters (ζ k ) 1⩽k⩽K and (q k ) 0⩽k⩽K into the path q in (6.50). We have already discussed extensively, in particular in Section 5.7, that it makes sense to focus on understanding the behaviour of overlap arrays. While we have chosen to define the α-overlap simply as α 1 ∧ α 2 , in view of (6.52), it would have been more faithful to the definition of our random field Z q to have defined this same quantity as seen through the sequence of parameters (q k ) 0⩽k⩽K , namely q α 1 ∧α 2 . And we recall from (6.60) that for every k ∈ {0,...,K}, we have

E⟨1 {q α 1 ∧α 2 =q k } ⟩ = ζ k+1 -ζ k . (6.69)
Recalling the definition of the path q in (6.50), we thus see that the law of q α 1 ∧α 2 , which one might argue is the "true" α-overlap, is the law of q(U), where U is a uniform random variable over [0,1]. With this in mind, it is perhaps not surprising that the function F N (t,q) satisfies a natural continuity property as a function of the path q. We write

Q(R ⩾0 ) ∶= q ∶ [0,1) → R ⩾0
q is right-continuous and non-decreasing , (

and for every p

∈ [1,+∞], we let Q p (R ⩾0 ) ∶= Q(R ⩾0 ) ∩ L p ([0,1];R).
We also note that when we say that a path q is piecewise-constant, we mean that we can partition the interval [0,1) into a finite number of intervals in the interior of which q is constant. In other words, a piecewise-constant path q ∈ Q(R ⩾0 ) is any path that can be represented in the form of (6.50) for a suitable choice of the parameters. Proposition 6.3. For any pair of piecewise-constant paths q,q ′ ∈ Q(R ⩾0 ), F N (t,q) -F N (t,q ′ ) ⩽ ˆ1 0 q(u) -q ′ (u) du. (6.71)

In particular, the enriched free energy (6.53) admits a unique extension to the space R ⩾0 × Q 1 (R ⩾0 ) satisfying (6.71) for all times t ⩾ 0 and paths q,q ′ ∈ Q 1 (R ⩾0 ).

Proof. The key step of the proof is to realize that we can allow for repetitions in the parameters (q k ) 0⩽k⩽K without encountering contradictions. The bound (6.71) then follows by simple estimates on the derivatives obtained in Lemma 6.2. We decompose the proof into three steps.

Step 1: integrating the Poisson-Dirichlet cascade. We use Theorem 5.25 to describe an explicit procedure for integrating out the Poisson-Dirichlet cascade. We fix parameters (ζ k ) 1⩽k⩽K and (q k ) 0⩽k⩽K as in (6.48)-(6.49), and let q be as in (6.50).

For every y 0 ,...,y K ∈ R N , we define

X K (y 0 ,...,y K ) ∶= log ˆΣN exp √ 2tH N (σ ) -Nt + K k=0 (2q k -2q k-1 ) 1 2 y k ⋅ σ -Nq K dP N (σ ), (6.72)
and then recursively, for every k ∈ {0,...,K -1},

X k = X k (y 0 ,...,y k ) ∶= 1 ζ k+1 logE y k+1 exp(ζ k+1 X k+1 ),
where we denote by E y k+1 the integration of the variable y k+1 according to the N-dimensional standard Gaussian probability measure. We also define X -1 ∶= E y 0 X 0 , with the same interpretation. Using a measurable mapping from [0,1] to R N that sends the Lebesgue measure on [0,1] to the standard Gaussian probability measure on R N (which can be constructed by first mapping the Lebesgue measure on [0,1] to the Lebesgue measure on [0,1] N by splitting the binary expansion of the argument, and then using Exercise 5.11 coordinate by coordinate), we see that we can apply Theorem 5.25 and obtain that

-NE F N (t,q) (H N (σ )) σ ∈R N = X -1 , (6.73)
where we write

F N (t,q) ∶= - 1 N log ˆΣN α∈N K exp H N (t,q,σ ,α) v α dP N (σ ).
In other words, F N (t,q) is the quantity on the right side of (6.53) before taking the expectation, and we have F N (t,q) = EF N (t,q).

In the derivation of (6.73), we used that that the conditional expectation appearing there consists in averaging over the randomness coming from the Poisson-Dirichlet cascade and the random field Z q , since these are the only other sources of randomness and they have been chosen independently of (H N (σ )) σ ∈R N .

Step 2: allowing for possible repetitions of parameters. We fix an integer K ⩾ 1 as well as sequences of parameters

0 = ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, 0 = q -1 ⩽ q 0 ⩽ q 1 ⩽ ⋯ ⩽ q K < q K+1 = +∞,
and consider the piecewise-constant path

q ∶= K k=0 q k 1 [ζ k ,ζ k+1 ) .
Since q is piecewise-constant, we have already given a definition of F N (t,q), but this definition would require that we rewrite q into a form that suppresses possible repetitions in the parameters. We claim that if we instead naively follow the procedure with the repeated parameters, we obtain the same value for F N (t,q).

If there is a repetition in the parameters (q k ) 0⩽k⩽K , say q k-1 = q k for some k ∈ {0,...,K}, then the term indexed by k in the sum on the right side of (6.72) vanishes, and thus for every y 0 ,...,y k ∈ R N , we have that X k-1 (y 0 ,...,y k-1 ) = X k (y 0 ,...,y k ).

It is therefore clear that removing this repetition in the sequence (q k ) 0⩽k⩽K does not change the value of the result.

Step 3: extending Lemma 6.2 to the case of repeated indices. We now justify that the identity (6.62) from Lemma 6.2 remains valid even if we allow for repetitions in the parameters (q k ) 0⩽k⩽K . We know that (6.62) holds when the (q k ) 0⩽k⩽K satisfy (6.49). Moreover, the free energy is clearly a continuous function of the parameters (q k ) 0⩽k⩽K , and the right side of (6.62) is also continuous with respect to these parameters. A classical analysis argument thus suffices to conclude that the free energy F N is differentiable also at the boundary of the set of non-decreasing sequences (q k ) 0⩽k⩽K , and that the relation (6.62) still holds there.

Step 4: extending to Q 1 (R ⩾0 ). Now that we can allow for repetitions in the parameters, for any two piecewise-constant paths q and q ′ , we can always find an integer K ⩾ 1 and sequences of parameters 0

= ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, 0 = q -1 ⩽ q 0 ⩽ ⋯ ⩽ q K and 0 = p -1 ⩽ p 0 ⩽ ⋯ ⩽ p K with q = K k=0 q k 1 [ζ k ,ζ k+1 ) and q ′ = K k=0 p k 1 [ζ k ,ζ k+1 ) .
For each s ∈ [0,1], we introduce the piecewise-constant path

q s ∶= K k=0 sq k + (1 -s)p k 1 [ζ k ,ζ k+1 ) ,
and define the interpolating free energy ϕ(s) ∶= F N (t,q s ). The fundamental theorem of calculus and the chain rule imply that

F N (t,q) -F N (t,q ′ ) ⩽ sup s∈[0,1] ϕ ′ (s) ⩽ K k=0 sup s∈[0,1] ∂ q k F N (t,q s ) q k -p k .
It follows from the previous step and (6.60) that

F N (t,q) -F N (t,q ′ ) ⩽ K k=0 ζ k+1 -ζ k q k -p k = ˆ1 0 q(u) -q ′ (u) du.
A density argument then allows us to uniquely extend the enriched free energy to the space Q 1 (R ⩾0 ) while preserving the bound (6.71). This completes the proof. ∎

To write equation (6.63) in Lemma 6.2 in a way that depends directly on the path q ∈ Q(R ⩾0 ) as opposed to the parameters (ζ k ) 1⩽k⩽K and (q k ) 0⩽k⩽K , we introduce additional notation. Given a function h ∶ R ⩾0 × Q 2 (R ⩾0 ) → R, a time t ⩾ 0 and a path q ∈ Q 2 (R ⩾0 ), we say that h admits a Gateaux derivative at (t,q) ∈ R ⩾0 × Q 2 (R ⩾0 ) if there exists a function ∂ q h(t,q,⋅) ∈ L 2 ([0,1];R) such that for every q ′ ∈ L 2 ([0,1];R) with q + εq ′ ∈ Q 2 (R ⩾0 ) for ε > 0 sufficiently small, we have as ε > 0 tends to zero that

h(t,q + εq ′ ) -h(t,q) = ε ˆ1 0 ∂ q h(t,q,u)q ′ (u)du + o(ε). (6.74)
Let q be as in (6.50), and assume that the enriched free energy F N admits a Gateaux derivative at the pair (t,q). For every other path q ′ = ∑

K k=0 p k 1 [ζ k ,ζ k+1
) associated with a non-decreasing sequence 0 = p -1 ⩽ p 0 ⩽ ⋯ ⩽ p K , we have

F N (t,q + εq ′ ) -F N (t,q) = ε K k=0 p k ˆζk+1 ζ k ∂ q F N (t,q,u)du + o(ε). (6.75)
On the other hand, a Taylor expansion reveals that

F N (t,q + εq ′ ) -F N (t,q) = ε K k=0 p k ∂ q k F N (t,q)1 [ζ k ,ζ k+1 ) + o(ε). (6.76)
Since these equalities must hold for all piecewise-constant paths q ′ , we must have

∂ q k F N (t,q) = ˆζk+1 ζ k ∂ q F N (t,q,u)du. (6.77)
Assuming that max k (ζ k+1ζ k ) is small, we have the approximate equality

ˆ1 0 ∂ q F N (t,q,u) 2 du ≃ K k=0 (ζ k+1 -ζ k ) 1 ζ k+1 -ζ k ˆζk+1 ζ k ∂ q F N (t,q,u)du 2 , (6.78)
see also Exercise 6.6 for a more precise argument. Up to an error that tends to zero as max k (ζ k+1ζ k ) tends to zero, we can therefore rewrite the identity (6.63) from Lemma 6.2 as

∂ t F N (t,q) - ˆ1 0 ∂ q F N (t,q,u) 2 du ≃ Var R 1,2 α 1 ∧ α 2 , (6.79)
where we used the notation

Var R 1,2 α 1 ∧ α 2 ∶= E⟨ R 1,2 -E⟨R 1,2 α 1 ∧ α 2 ⟩ 2 ⟩. (6.80)
Combined with the synchronization mechanism discussed earlier, see in particular (6.68), this suggests that the limit free energy

f ∶ R ⩾0 × Q 2 (R ⩾0 ) → R should satisfy the infinite-dimensional Hamilton-Jacobi equation ∂ t f (t,q) - ˆ1 0 ∂ q f (t,q,u) 2 du = 0 on R >0 × Q 2 (R ⩾0 ). (6.81)
Our next goal will be to explain that this guess allows us to correctly predict the Parisi formula presented in (6.8).

Exercise 6.5. We consider the SK model on Σ N with P N ∶= (P 1 ) ⊗N for a possibly non-centred probability measure P 1 on Σ 1 , and we assume that there exists q ∈ R with lim

N→+∞ E⟨ R 1,2 -q 2 ⟩ = 0.
Let g = (g 1 ,...,g N ) be a standard Gaussian vector and Z be a standard Gaussian random variable independent of all other sources of randomness and of each other. We understand that the probability P and associated expectation E also takes the average over g, and we denote by E Z the average over the randomness of Z. Show that for every F ∈ C c (R;R) and ε > 0, we have

lim N→+∞ P ⟨F g ⋅ σ √ N ⟩ -E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ > ε = 0. (6.82)
In words, for most realizations of g, the law of the random variable N -1 2 g ⋅ σ under ⟨⋅⟩ is essentially Gaussian, with mean N -1 2 g ⋅ ⟨σ ⟩ and variance 1q.

Exercise 6.6. Let f ∈ L 2 ([0,1];R). Show that for every ε > 0, there exists δ > 0 such that for every 0

= ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, if max k (ζ k+1 -ζ k ) ⩽ δ , then ˆ1 0 f 2 - K k=0 (ζ k+1 -ζ k ) 1 ζ k+1 -ζ k ˆζk+1 ζ k f 2 ⩽ ε.
(6.83)

Connecting the Hamilton-Jacobi approach with the Parisi formula

The goal of this section is to demonstrate that the Hamilton-Jacobi equation (6.81) that we derived heuristically allows us to correctly predict the Parisi formula (6.8).

Our first task will be to identify the initial condition to this Hamilton-Jacobi equation. Just as in the Curie-Weiss model and in the models of statistical inference we considered in Chapter 4, we will be able to leverage the product structure of the reference measure P N to show that the quantity F N (0,q) in fact does not depend on N. In order to connect to the Parisi formula as presented in (6.8), we will also relate this quantity to the second-order partial differential equation appearing in (6.9).

Once this is done, we need to make sense of the infinite-dimensional Hamilton-Jacobi equation (6.81). There are at least three possible ways to approach this problem. A first possibility would be to seek to adapt the notion of viscosity solution to infinite-dimensional spaces. Alternatively, we could devise finite-dimensional approximations of the equation, and then try to show the convergence of these approximations as the dimension is sent to infinity. Finally, we could decide that since the non-linearity in (6.104) is convex, whichever notion of solution we come up with will admit a variational representation analogous to the Hopf-Lax formula from Theorem 3.8, and so we may as well define the solution according to this variational formula directly. We will start by exploring the last of these options, since it is the easiest. As will be stressed in Section 6.6, this approach is no longer applicable for non-convex models such as the bipartite model, and we will therefore also outline the workings of the second of these three options there. That all these approaches yield the same function when they are applicable is shown in [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF].

We start by studying the initial condition associated with the Hamilton-Jacobi equation (6.81). In the next lemma, we will show in particular that ψ(q) ∶= lim N→+∞ F N (0,q) = F 1 (0,q). (

For each piecewise-constant path q, Theorem 5.25 describes an explicit procedure for computing this quantity in terms of recursive averages over the Gaussian measure of quantities of the form 1 ζ k logEexpζ k X k .

In order to encode this recursive procedure even for general paths, one can view this calculation in terms of a second-order partial differential equation. To illustrate the idea, let us observe that if (B t ) t⩾0 is a standard Brownian motion and f ∈ C c (R;R ⩾0 ), then the function

u(t,x) ∶= E f (x + B 2t ) (6.85)
solves the heat equation ∂ t u = ∂ 2 x u, and a simple calculation yields that the function

ũ ∶= ζ -1 k logu is then a solution to ∂ t ũ = ∂ 2 x ũ + ζ k (∂ x ũ) 2 .
The representation of the initial condition in the next lemma will be an iteration of this observation.

To state this precisely, we start by recalling that we denote by Q ∞ (R ⩾0 ) the space of paths in Q(R ⩾0 ) that remain bounded. For every such path q ∈ Q ∞ (R ⩾0 ), we define q(1) ∶= lim u↗1 q(u). (6.86)

In particular, for q as in (6.50), we have q(1) = q K . For every q ∈ Q ∞ (R ⩾0 ), we also define its right-continuous inverse q -1 ∶ R → [0,1] according to q -1 (t) ∶= sup{u ∈ [0,1] q(u) ⩽ t}. (6.87)

We understand the supremum above to be zero in case the set is empty. Finally, for every t ⩾ 0 and x ∈ R, we write φ (t,x) ∶= log ˆΣ1 exp(xσ -t)dP 1 (σ ). (6.88)

In case P 1 were not supported on Σ 1 , we would replace the term t in the exponential by tσ 2 .

Lemma 6.4 (Parisi PDE). For every piecewise-constant path q ∈ Q(R ⩾0 ) and integer N ⩾ 1, we have F N (0,q) = F 1 (0,q) = -Φ q (0,0), (6.89)

where the function

Φ q ∶ [0,q(1)] × R → R satisfies ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -∂ t Φ q (t,x) = ∂ 2 x Φ q (t,x) + q -1 (t)(∂ x Φ q (t,x)) 2 on [0,q (1
)] × R, Φ q (q(1),x) = φ (q(1),x) for x ∈ R. (6.90)

Before proceeding to the proof, we discuss the nature of the partial differential equation (6.90) and the associated notion of solution. At small scales, the first-order term (∂ x Φ q ) 2 becomes less and less important compared with the second-order term ∂ 2

x Φ q , which is regularizing. As a consequence, this equation is much easier to study than the first-order Hamilton-Jacobi equations that have accompanied us throughout the book, and the only slight difficulty is that the function q -1 has some jump discontinuities. For each q ∈ Q ∞ (R ⩾0 ), we can choose among several equivalent definitions of the notion of being a solution to (6.90).

Perhaps the most intuitive notion of solution, which we could call the notion of classical solution, would be to ask that Φ q be a continuous function such that, at every (t,x) ∈ [0,q(1)]×R except those t's that are points of discontinuity of q -1 , the function Φ q is continuously differentiable in t and twice continuously differentiable in x, and the first line of (6.90) holds for each such (t,x). To guarantee uniqueness, this must be supplemented with a mild growth condition, in our case we can simply require Φ q to have at most linear growth in x at infinity, uniformly over t.

An alternative and analytically very convenient notion of solution is that of mild solution. We could for instance look for a function in C([0,q(1)];C 2 (R;R)) with at most linear growth and such that, for every t ∈ (0,q(1)), we have Φ q (t,x) = e (q(1)-t)∆ φ (q(1),⋅)(x) + ˆq(1) t q -1 (t)e (s-t)∆ (∂ x Φ q (s,⋅)) 2 (x)ds, (6.91) where we denote by (e t∆ ) t⩾0 the heat semigroup; explicitly, the heat semigroup is such that for every measurable function f ∶ R → R for which the integral is well-defined,

e t∆ f (x) ∶= ˆR 1 √ 4πt exp - (x -y) 2 4t f (y)dy.
This formulation allows one to show the well-posedness of equation (6.90) over a short time interval using a fixed-point argument, and this can then be iterated using estimates from the maximum principle; see also [START_REF] Jagannath | A dynamic programming approach to the Parisi functional[END_REF]. If desired, one can then show the equivalence between the notions of classical and mild solutions.

We will not go into details on these points. At least, the arguments we now present yield in particular that a solution to (6.90) indeed exists, starting with the proof below covering the case of piecewise-constant q.

Proof of Lemma 6.4. We fix a piecewise-constant path q ∈ Q(R ⩾0 ) of the form (6.50). Applying Corollary 5.26 to the random variables

X K,i ∶= log ˆΣ1 exp(Z i q (α)σ -q K dP 1 (σ )
shows that F N (0,q) = F 1 (0,q), so the initial condition (6.84) is well-defined and is given by

ψ(q) = F 1 (0,q) = -logE ˆΣ1 α∈N K exp(Z q (α)σ -q K )v α dP 1 (σ ). (6.92)
To express ψ(q) as the value at (0,0) of the solution to (6.90), we denote by (B t ) t⩾0 a standard Brownian motion, and recursively define the function Φ q ∶ [0,q(1)]×R → R by Φ q (q(1),x) ∶= φ (q K ,x) and Φ q (t,x) ∶= ζ -1 k+1 logEexpζ k+1 Φ q (q k+1 ,x + B 2q k+1 -2t ) (6.93) for t ∈ [q k ,q k+1 ) and -1 ⩽ k ⩽ K -1. It then follows from Theorem 5.25 that ψ(q) = -Φ q (0,0).

We now show that Φ q satisfies the equation (6.90). It will be important to observe that

q -1 = K+1 k=0 ζ k 1 [q k-1 ,q k ) . (6.94)
For each k ∈ {-1,...,K -1}, we consider the function defined for t ∈ [q k ,q k+1 ) by Φq (t,x) ∶= expζ k+1 Φ q (t,x).

As for (6.85), we see that the function Φq satisfies

-∂ t Φq = ∂ 2 x
Φq on [q k ,q k+1 ) × R.

Taking the logarithm, we obtain that

-∂ t Φ q = ∂ 2 x Φ q + ζ k+1 (∂ x Φ q ) 2 on [q k ,q k+1 ) × R.
Remembering from (6.94) that q -1 = ζ k+1 on [q k ,q k+1 ) completes the proof. ∎

In order to extend Lemma 6.4 to arbitrary paths q ∈ Q ∞ (R ⩾0 ), one possible route would be to show the continuity of Φ q (0,0) as a function of q, since we already know the continuity of F 1 (0,⋅) from Proposition 6.3. Since we have not been very precise in our discussion on the well-posedness of (6.90), we prefer to proceed in a slightly different way: by leveraging Lemma 6.4 and slightly upgrading Proposition 6.3, we will justify that taking solutions to (6.90) for a sequence of piecewise-constant paths that converge to some arbitrary q ∈ Q ∞ (R ⩾0 ), we can pass to the limit and obtain a solution to (6.90) for this limit path. In particular, this shows that a solution to (6.90) exists for arbitrary q ∈ Q ∞ (R ⩾0 ), and the only point of interest that we did not justify with full precision is that this solution is unique; this uniqueness property can be obtained by a classical Picard fixed-point argument.

Before starting the proof, we make a simple observation that is convenient when wanting to compare Φ q for different choices of q. The observation is that, if we were to define Φ q over a larger time interval than [0,q(1)], then we would end up with a function that coincides with Φ q on [0,q(1)]. In other words, for each T ⩾ q(1) and for u the solution to

⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -∂ t u(t,x) = ∂ 2 x u(t,x) + q -1 (t)(∂ x u(t,x)) 2 on [0,T ] × R, u(T,x) = φ (T,x) for x ∈ R, (6.95)
we have that u and Φ q coincide on [0,q(1)] × R. This comes from the fact that the function φ in (6.88) is such that

-∂ t φ = ∂ 2 x φ + (∂ x φ ) 2 .
Indeed, for the natural Gibbs measure associated with (6.88), the derivative -∂ t φ computes the second moment of σ , while ∂ 2 x φ computes its variance and ∂ x φ its first moment. With this observation in mind, we can without loss of generality think of the function Φ q as defined on R ⩾0 × R, by simply extending it to be equal to φ outside of [0,q(1)] × R. For every T ⩾ q(1), this extension satisfies the partial differential equation in (6.95). Lemma 6.5. (i) For every piecewise-constant path q ∈ Q(R ⩾0 ), and for Φ q the solution to (6.90), we have for every t ⩾ 0 and x ∈ R that ∂ x Φ q (t,x) ⩽ 1 and 0 ⩽ ∂ 2 x Φ q (t,x) ⩽ 1. (6.96)

(ii) For every piecewise-constant q,q ′ ∈ Q(R ⩾0 ), t ⩾ 0 and x ∈ R, Φ q (t,x) -Φ q ′ (t,x) ⩽ ˆ1 0 q(u) -q ′ (u) du. (6.97)

(iii) For every piecewise-constant q,q ′ ∈ Q(R ⩾0 ), t ⩾ 0 and x ∈ R,

∂ x Φ q (t,x) -∂ x Φ q ′ (t,x) ⩽ 4 ˆ1 0 q(u) -q ′ (u) du. (6.98)
Proof. We decompose the proof into four steps.

Step 1: reduction to t = 0. We first show that it suffices to prove the estimates in the statement of Lemma 6.5 for t = 0. Recall from (6.94) that for q as in (6.50), we have

q -1 = K+1 k=0 ζ k 1 [q k-1 ,q k ) .
We claim that for each fixed t 0 ∈ [0,q(1)], say with t 0 ∈ [q k 0 -1 ,q k 0 ) for some k 0 ∈ {0,...,K}, there is a simple relationship between Φ q (t 0 +t,x) and Φ q(t, x), where q ∈ Q(R ⩾0 ) is such that

q-1 = ζ k 0 1 [0,q k 0 -t 0 ) + K+1 k=k 0 +1 ζ k 1 [q k-1 -t 0 ,q k -t 0 ) .
Since when a constant is added to the initial condition in (6.90), this only changes the solution by this same constant, we see indeed that for every t ∈ [0,q(1) -t 0 ] and x ∈ R, we have Φ q (t 0 +t,x) +t 0 = Φ q(t, x).

This justifies that we may as well focus on the case t = 0. If P 1 were not supported on Σ 1 , we would need to argue slightly differently, seeking instead to interpret Φ q as a version of Φ q for a modified reference measure P 1 .

Step 2: proof of (i). The representation (6.93) of the function Φ q for a piecewiseconstant path q ∈ Q(R ⩾0 ) together with Theorem 5.25 imply that

Φ q (0,x) = Elog ˆΣ1 α∈N K exp(σ (x + Z q (α)) -q K )v α dP 1 (σ ). (6.99)
Denoting by ⟨⋅⟩ the Gibbs average for N = 1 associated with the Hamiltonian (σ ,α) ↦ σ (x + Z q (α)), we have

∂ x Φ q (0,x) = E⟨σ ⟩ and ∂ 2 x Φ q (0,x) = E⟨1 -σ 1 σ 2 ⟩ = 1 -E⟨σ ⟩ 2 .
From these equalities, it is immediate that ∂ x Φ q (0,x) ⩽ 1 and 0 ⩽ ∂ 2 x Φ q (0,x) ⩽ 1.

Step 3: proof of (ii). The result follows from the observation that using (6.99), the proof of Proposition 6.3 applies verbatim.

Step 4: proof of (iii). We fix two piecewise-constant paths q,q ′ ∈ Q(R ⩾0 ). We can assume without loss of generality that there exist an integer K ⩾ 1 and sequences of parameters 0

= ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, 0 = q -1 ⩽ q 0 ⩽ ⋯ ⩽ q K and 0 = p -1 ⩽ p 0 ⩽ ⋯ ⩽ p K such that q = K k=0 q k 1 [ζ k ,ζ k+1 ) and q ′ = K k=0 p k 1 [ζ k ,ζ k+1 ) .
For each s ∈ [0,1] and k ∈ {0,...,K}, we define q s k ∶= sq k + (1s)p k , and the associated piecewise-constant path

q s ∶= K k=0 q s k 1 [ζ k ,ζ k+1 ) .
Notice that we can choose the parameters (q k ) 0⩽k⩽K and (p k ) 0⩽k⩽K in such a way that for every k ∈ {0,...,K}, we have q k-1 ≠ q k or p k-1 ≠ p k . This ensures that for every s ∈ (0,1), the sequence (q s k ) 0⩽k⩽K is strictly increasing. We set

ϕ(s) ∶= ∂ x Φ q s (0,x) = E⟨σ ⟩ s ,
where ⟨⋅⟩ s denotes the Gibbs average associated with the Hamiltonian (σ ,α) ↦ σ (x + Z q s (α)). A direct computation and the chain rule give

ϕ ′ (s) = E⟨∂ s Z q s (α 1 ) -σ 1 σ 2 ∂ s Z q s (α 2 )⟩ s = K k=0 (2q s k -2q s k-1 ) -1 2 (q k -p k ) -(q k-1 -p k-1 ) E⟨z α 1 k -σ 1 σ 2 z α 2 k ⟩ s .
The Gibbs Gaussian integration by parts formula (Theorem 4.6) and the symmetry between replicas reveal that

E⟨z α 1 k ⟩ s = (2q s k -2q s k-1 ) 1 2 E⟨σ 1 -σ 2 1 {α 1 k =α 2 k } ⟩ s E⟨σ 1 σ 2 z α 2 k ⟩ s = (2q s k -2q s k-1 ) 1 2 E⟨σ 1 + σ 2 1 {α 1 k =α 2 k } -2σ 1 σ 2 σ 3 1 {α 1 k =α 2 k } ⟩ s . It follows that ϕ ′ (s) = 2 K k=0 (q k -p k ) -(q k-1 -p k-1 ) E⟨(σ 1 σ 2 σ 3 -σ 1 )1 {α 1 k =α 2 k } ⟩ s = 2 K-1 k=0 (q k -p k )E⟨(σ 1 σ 2 σ 3 -σ 1 )(1 {α 1 k =α 2 k } -1 {α 1 k+1 =α 2 k+1 } )⟩ s + 2(q K -p K )E⟨(σ 1 σ 2 σ 3 -σ 1 )1 {α 1 K =α 2 K } ⟩ s = 2 K k=0 (q k -p k )E⟨(σ 1 σ 2 σ 3 -σ 1 )1 {α 1 ∧α 2 =k} ⟩ s ,
where we used that q -1 = p -1 = 0 in the second equality. Remembering that the property (6.60) is valid under very general conditions including our present one, we can rewrite this as

ϕ ′ (s) = 2 K k=0 (q k -p k )(ζ k+1 -ζ k )E⟨σ 1 σ 2 σ 3 -σ 1 α 1 ∧ α 2 = k⟩ s .
We can finally use the fundamental theorem of calculus to conclude that

∂ x Φ q (0,x) -∂ x Φ q ′ (0,x) ⩽ 4 K k=0 q k -p k ζ k+1 -ζ k = 4 ˆ1 0 q(u) -q ′ (u) du.
This completes the proof of (iii). ∎ Although we will not make use of this fact, we note in passing that Lemma 6.5 can be generalized, with essentially no change to the proof, to yield that for every integer k ⩾ 1, the derivative ∂ k

x Φ q (t,x) is bounded uniformly over the choice of piecewise-constant q ∈ Q(R ⩾0 ), t ⩾ 0 and x ∈ R, and the mapping of q ↦ ∂ k

x Φ q (t,x) is Lipschitz continuous with respect to the L 1 norm, uniformly over t ⩾ 0 and x ∈ R.

We can now extend the identity (6.89) from Lemma 6.4 to arbitrary bounded paths. Proposition 6.6. For every path q ∈ Q ∞ (R ⩾0 ), the initial condition (6.84) is given by ψ(q) = F 1 (0,q) = -Φ q (0,0), (6.100)

where the function Φ q ∶ [0,q(1)] × R → R satisfies the equation (6.90).

Proof. We recall that, as explained around (6.95), for any q ∈ Q ∞ (R ⩾0 ), we may as well think of Φ q as being defined on R ⩾0 × R. We fix an arbitrary path q ∈ Q ∞ (R ⩾0 ), and take a sequence of piecewise-constant paths (q n ) n⩾1 that converge to q in the L 1 norm. For convenience we may also assume that for every n ⩾ 1, we have q n (1) = q(1). By (ii) in Lemma 6.5, the functions Φ qn converge pointwise to some limit; we define Φ q to be this limit, and proceed to show that this limit solves (6.90) in the sense of satisfying (6.91). Since q n (1) = q(1), we have for every n ⩾ 1 and t ∈ [0,q(1)] that Φ qn (t,x) = e (q(1)-t)∆ φ (q(1),⋅)(x) + ˆq(1) t q -1 n (s)e (s-t)∆ (∂ x Φ qn (s,⋅)) 2 (x)ds. (6.101) By (iii) in Lemma 6.5, the sequence (∂ x Φ qn ) n⩾1 converges to some limit, uniformly over R ⩾0 × R. Using the uniform bound on ∂ 2 x Φ qn from (i) of Lemma 6.5, we see that the functions ∂ x Φ qn are also Lipschitz continuous in x, uniformly over R ⩾0 × R and over n ⩾ 1. It is therefore clear that Φ q is differentiable in x, that the limit of ∂ x Φ qn as n tends to infinity is ∂ x Φ q , and since q n converges to q in the L 1 norm, that

lim n→+∞ ∂ x Φ qn -∂ x Φ q L ∞ (R ⩾0 ×R;R) = 0. (6.102)
Observing also that, by Fubini's theorem (see e.g. Proposition 2.17 of [START_REF] Santambrogio | Optimal transport for applied mathematicians[END_REF] for a detailed argument), ˆ1 0 q n (u) -q(u) du = ˆ+∞ 0 q -1 n (s) -q -1 (s) ds, (6.103) we also have that the right side of (6.103) converges to 0 as n tends to infinity. Combining this with (6.102), we conclude from (6.101) that Φ q is indeed a mild solution to (6.90). Recalling from Lemma 6.4 that for each n ⩾ 1, we have

F 1 (0,q n ) = -Φ qn (0,0),
and that F 1 (0,⋅) is Lipschitz continuous by Proposition 6.3, this completes the proof. ∎

We recall from (6.81) that, through informal arguments, we suggested that the limit f of the free energy F N might satisfy the Hamilton-Jacobi equation

∂ t f (t,q) - ˆ1 0 ∂ q f (t,q,u) 2 du = 0 on R >0 × Q 2 (R ⩾0 ), (6.104)
with an initial condition which we have now firmly identified as f (0,q) = ψ(q) = -Φ q (0,0). (6.105)

While we have only really made sense of Φ q (0,0) for q ∈ Q ∞ (R ⩾0 ), via the partial differential equation in (6.90), we recall that -Φ q (0,0) = F 1 (0,q) and that F 1 (0,⋅) is Lipschitz continuous with respect to the L 1 norm, by Proposition 6.3, so we can extend Φ q (0,0) to every q ∈ Q 1 (R ⩾0 ) by continuity. The main reason we prefer to think of (6.104) as being posed on R >0 × Q 2 (R ⩾0 ) as opposed to, for instance, being posed on

R >0 × Q ∞ (R ⩾0 ), is that the space Q 2 (R ⩾0
) is naturally embedded in the Hilbert space L 2 ([0,1];R), and that the Hilbert-space geometry of L 2 ([0,1];R) is directly tied with our notion of derivative in (6.74), and is also closely related to the Euclidean-space geometry of our finite-dimensional approximations to (6.104). This is not fundamental however, and we could as well decide to try to make sense of (6.104) as being posed on R >0 × Q ∞ (R ⩾0 ), which we might still think of as a subset of L 2 ([0,1];R) if we want to.

As was discussed at the opening of this section, there are several different options for us to try to make sense of the equation (6.104). Since the non-linearity in the equation is convex, we may expect that whichever notion of solution we come up with will also admit a variational representation analogous to the Hopf-Lax formula. There are however two potential sources of problem for the validity of such a variational representation. The first one is the infinite-dimensional nature of the equation. The second one is that the space Q 2 (R ⩾0 ) of non-decreasing paths from [0,1) to R ⩾0 has a boundary: if we restrict to piecewise-constant paths of the form (6.50), we see that we must impose that the sequence (q k ) 0⩽k⩽K be a sequence of non-negative numbers, and be non-decreasing.

If we simplify this further and only consider the space of constant paths, which corresponds to the function introduced in (6.32), then this reduces to the constraint that the constant path must be non-negative, that is, h ⩾ 0 in the notation from (6.32). In this simplest case of a domain with a boundary, we have seen in Chapter 4 that, under some assumptions on the initial condition ψ 1 ∶ R → R, we can make sense of the solution to

∂ t f 1 -(∂ h f 1 ) 2 = 0 on R >0 × R ⩾0 (6.106)
subject to the initial condition f 1 (0,⋅) = ψ 1 , and that the solution is given, for every t,h ⩾ 0, by

f 1 (t,h) = sup h ′ ⩾0 ψ 1 (h + h ′ ) - (h ′ ) 2 4t ; (6.107)
see in particular (4.84) and (4.90). While we exploited the convexity of ψ 1 to make use of the convex selection principle there, the fact that we could essentially disregard the contribution of the boundary was only really using that the function ψ 1 was non-decreasing. Using the language of characteristics discussed in Section 3.5, the point is to ensure that the characteristic lines always go from the inside of the domain towards the boundary, never the other way around.

In our present setting, a similar monotonicity property also holds. If we only consider constant paths, with the free energy as in (6.32), then we found in (6.34) that the derivative with respect to the value h of the constant path is given by

N -1 E⟨σ 1 ⋅ σ 2 ⟩ = N -1 E⟨σ ⟩ 2 ⩾ 0. ( 6.108)
We fix an integer K ⩾ 1, denote

U K ∶= q = (q 0 ,...,q K ) ∈ R K+1 ⩾0 0 ⩽ q 0 ⩽ q 1 ⩽ ⋯ ⩽ q K , (6.109)
and for each sequence q = (q k ) 0⩽k⩽K ∈ U K , we consider the piecewise-constant path

q q ∶= K k=0 q k 1 [ k K+1 , k+1 K+1
) .

(6.110)

While so far we had kept the dependence of q on its parameters implicit, it will be useful to make this dependence explicit for a short while, as we do in the notation q q instead of just q. One can show that 0 ⩽ ∂ q 0 F N (t,q q ) ⩽ ⋯ ⩽ ∂ q K F N (t,q q ). (6.111)

The proof of this result combines the Gaussian integration by parts used for the proof of Lemma 6.2 with the explicit procedure for integrating the Poisson-Dirichlet cascade presented in Theorem 5.25 and Corollary 5.26. We will not give a detailed proof of (6.111), and only refer to Lemma 2.4 of [START_REF] Mourrat | The Parisi formula is a Hamilton-Jacobi equation in Wasserstein space[END_REF] or Proposition 14.3.2 of [START_REF] Talagrand | Mean field models for spin glasses[END_REF].

(An alternative proof can be devised by analyzing the properties of the equation in (6.90).) In analogy with (4.84) and (4.90), we can therefore hope that there is a nice variational formula for equations posed on U K , even though this space has a non-trivial boundary.

Recall from Lemma 6.2 and the discussion that followed that if K is chosen sufficiently large, then we expect that

∂ t F N (t,q q ) -(K + 1) K k=0 (∂ q k F N (t,q q )) 2 ≃ 0. (6.112)
Since the convex dual of the mapping (p k ) 0⩽k⩽K ↦ (K + 1)∑ K k=0 p 2 k is the mapping (q k ) 0⩽k⩽K ↦ 1 4(K+1) ∑q 2 k , and since we argued that boundary aspects should hopefully behave as nicely as they have in the context of (4.90), we are led to expect that, for K sufficiently large and q q as in (6.110), f (t,q q ) ≃ sup .113) Using also that q q+q ′ = q q + q q ′ , this can be rewritten as f (t,q q ) ≃ sup q ′ ∈U K ψ(q q + q q ′ ) -1 4t ˆ1 0 q q ′ (u) 2 du . (6.114)

q ′ ∈U K ψ(q q+q ′ ) - 1 4t(K + 1) K k=0 (q ′ k ) 2 . ( 6
Sending K to infinity, we see that the natural candidate for a Hopf-Lax variational representation of the solution to (6.104)-(6.105) is, for every t ⩾ 0 and q ∈ Q 2 (R ⩾0 ), .115) We note that for the more general mixed p-spin models defined in (6.13)-(6.14), the relation (6.61) would be replaced by

f (t,q) = sup q ′ ∈Q 2 (R ⩾0 ) ψ(q + q ′ ) - 1 4t ˆ1 0 (q ′ (u)) 2 du . ( 6
∂ t F N (t,q) = E⟨ξ (R 1,2 )⟩, (6.116)
while the derivatives with respect to each q k are still as in (6.62). In this context, we would therefore be led to suspect that the limit free energy f might satisfy1

∂ t f (t,q) - ˆ1 0 ξ (∂ q f (t,q,u))du = 0 on R >0 × Q 2 (R ⩾0 ), (6.117)
with the same initial condition (6.105). The inequality (6.111) remains valid in this case, and in particular, the argument of the function ξ in (6.117) is always non-negative. This is why the convexity of ξ on R ⩾0 is sufficient for the validity of a Hopf-Lax variational representation for f . Following along the heuristic argument above would thus suggest that

f (t,q) = sup q ′ ∈Q 2 (R ⩾0 ) ψ(q + q ′ ) -t ˆ1 0 ξ * (q ′ (u))du , (6.118)
where ξ * ∶ R → R is the convex dual of ξ ∶ R ⩾0 → R, so that for every s ∈ R, ξ * (s) ∶= sup r⩾0 (rsξ (r)). (6.119) In this section we take the (rather convenient!) point of view of simply defining the solution to (6.104)-(6.105) as being the function given in (6.115). The main goal of this section is to show that our guessed formula (6.115) is indeed consistent with the Parisi formula for the free energy of the SK model that was presented in (6.8)-(6.9). We recall from (6.57) that the free energy (6.7) is equal to

-F N β 2 2 ,0 + log(2) + β 2 2 , (6.120)
with F N (t,q) being the enriched free energy defined in (6.53) with the choice of P N = ( 1 2 δ -1 + 1 2 δ 1 ) ⊗N . If we believe that F N (t,q) converges to f (t,q) as defined in (6.115) for arbitrary choices of t ⩾ 0 and q ∈ Q 2 (R ⩾0 ), then in particular, it should be that the limit of the free energy in (6.7) is

-f β 2 2 ,0 + log(2) + β 2 2 . ( 6.121)
We now show that this guess indeed coincides with the Parisi formula (6.8).

Theorem 6.7 ( [START_REF] Mourrat | The Parisi formula is a Hamilton-Jacobi equation in Wasserstein space[END_REF]). Let f ∶ R ⩾0 × Q 2 (R ⩾0 ) → R be the function defined by (6.115), which we interpret as the solution to the Hamilton-Jacobi equation (6.104)-(6.105).

For every β ⩾ 0, we have

-f β 2 2 ,0 + log(2) + β 2 2 = inf ζ ∈D[0,1] Φ ζ (0,0) -β 2 ˆ1 0 tζ (t)dt + log(2) , (6.122)
where the space D[0,1] is defined in (6.10) and the function Φ ζ ∶ [0,1] × R → R denotes the solution to the Parisi PDE (6.9). In particular, the limit of the free energy (6.7) in the SK model is given by

lim N→+∞ F N (β ) = -f β 2 2 ,0 + log(2) + β 2 2 . (6.123)
Proof. The Hopf-Lax representation (6.115) implies that

-f β 2 2 ,0 + β 2 2 = inf q∈Q 2 (R ⩾0 ) Φ q (0,0) + 1 2β 2 ˆ1 0 q(u) 2 du + β 2 2 . ( 6.124)
To establish (6.122), for each a > 0 it will be convenient to write

Q ⩽a ∶= q ∈ Q ∞ (R ⩾0 ) q(1) ⩽ a (6.125)
for the paths in Q ∞ (R ⩾0 ) that are bounded by a. The proof now proceeds in three steps. First we show that the infimum on the right side of (6.124) can be restricted to paths in Q ⩽β 2 , then we use a change of variables to normalize to paths in Q ⩽1 , and finally we perform another change of variables to replace a path q ∈ Q ⩽1 by its right-continuous inverse q -1 ∈ D[0,1].

Step 1: restricting to Q ⩽β 2 . We fix a path q ∈ Q 2 (R ⩾0 ), and define the path q ∈ Q ⩽β 2 by q ∶= min(q,β 2 ). We denote by u * ∶= inf{u ∈ [0,1] q(u) ⩾ β 2 } the first point at which q exceeds β 2 , and observe that by the Lipschitz property of Φ q in Proposition 6.3,

Φ q(0, 0) + 1 2β 2 ˆ1 0 q(u) 2 du -Φ q (0,0) + 1 2β 2 ˆ1 0 q(u) 2 du ⩽ ˆ1 0 q(u) -q(u) du - 1 2β 2 ˆ1 0 q(u) 2 -q(u) 2 du = ˆ1 u * q(u) -β 2 du - ˆ1 u * q(u) -β 2 q(u) + β 2 2β 2 du ⩽ 0,
where we used that q(u) ⩾ β 2 for u ⩾ u * in the final inequality. Together with (6.124), this implies that

-f β 2 2 ,0 + β 2 2 = inf q∈Q ⩽β 2 Φ q (0,0) + 1 2β 2 ˆ1 0 q(u) 2 du + β 2 2 . (6.126)
Step 2: re-normalizing to Q ⩽1 . Substituting q with β 2 q in the previous display, we obtain that

-f β 2 2 ,0 + β 2 2 = inf q∈Q ⩽1 Φ β 2 q (0,0) + β 2 2 ˆ1 0 q(u) 2 du + β 2 2 . (6.127)
Our goal now is to relate, for each fixed q ∈ Q ⩽1 , the function Φ β 2 q from (6.90) with the function Φ ζ solution to (6.9) for a suitable choice of ζ ∈ D[0,1]. We choose ζ ∶= q -1 , the right-continuous inverse of q. By definition, the function

Φ q -1 ∶ [0,1] × R → R solves ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -∂ t Φ q -1 (t,x) = β 2 ∂ 2 x Φ q -1 (t,x) + q -1 (t) ∂ x Φ q -1 (t,x) 2 on [0,1] × R d Φ q -1 (1,x) = logcosh(x) for x ∈ R.
We write q ∶= β 2 q, and let Φ ∶ [0,β 2 ] × R → R be defined by Φ(t,x) ∶= Φ q -1 (t β 2 ,x).

Using that q-1 (t) = q -1 (t β 2 ), we obtain that Φ satisfies

⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -∂ t Φ(t,x) = ∂ 2 x Φ(t,x) +q -1 (t) ∂ x Φ(t,x) 2 on [0,β 2 ] × R d Φ(β 2 ,x) = logcosh(x) for x ∈ R.
We recall from the discussion around (6.95) that we may as well think of the function Φ β 2 q = Φ q as being defined on [0,β 2 ] × R, since β 2 ⩾ β 2 q(1). We also recall that the addition of a constant to the initial condition in (6.90) simply changes the solution by the addition of this constant. Using also (6.88), we therefore obtain that, for every t ∈ [0,β 2 ] and x ∈ R,

Φ β 2 q (t,x) = Φ(t,x) -β 2 .
Combining this with (6.127) yields that

-f β 2 2 ,0 + β 2 2 = inf q∈Q ⩽1 Φ q -1 (0,0) + β 2 2 ˆ1 0 q(u) 2 du - β 2 2 . ( 6

.128)

Step 3: re-parametrizing by D[0,1]. Our next step consists in replacing the optimization variable q ∈ Q ⩽1 in (6.128) by its right-continuous inverse. So we fix q ∈ Q ⩽1 and denote by ζ ∶ [0,1] → [0,1] its right-continuous inverse, that is, for every

t ∈ [0,1], ζ (t) ∶= sup{u ∈ [0,1] q(u) ⩽ t}. (6.129)
Since q is non-decreasing, we have that for every u,t ∈ [0,1],

u < ζ (t) ⇒ q(u) ⩽ t and u > ζ (t) ⇒ q(u) > t.
Since ζ ∈ D[0,1], we can write this function as t ↦ µ[0,t] for some probability measure µ on [0,1]. We infer from the previous display that, for every t ∈ [0,1],

µ[0,t] = ˆ1 0 1 {q(u)⩽t} du. (6.130)

The mapping A ↦ ´1 0 1 {q(u)∈A} du, where A is any measurable subset of [0,1], defines a probability measure on [0,1]; it is the image of the Lebesgue measure on [0,1] through the mapping q. By (6.130), this probability measure has the same cumulative distribution function as µ. By Dynkin's π-λ theorem (see Theorem A.5 and Exercise A.3), we deduce that this measure is equal to the measure µ. That is, for every bounded measurable function g ∶ [0,1] → R, ˆ1 0 g(q(u))du = ˆ1 0 g(t)dµ(t). (6.131) This corresponds to the fact that if we sample a uniform random variable U on [0,1], then the law of q(U) is µ. We use (6.131) with the function g replaced by the square function to obtain that ˆ1 0 q(u) 2 du = ˆ1 0 t 2 dµ(t).

We then re-express the latter integral in terms of ζ by integrating by parts, that is,

ˆ1 0 q(u) 2 du = 1 -2 ˆ1 0 ˆ1 0 s1 {t⩽s} dsdµ(t) = 1 -2 ˆ1 0 s ˆ1 0 1 {t⩽s} dµ(t)ds = 1 -2 ˆ1 0 sζ (s)ds.
Since the correspondence between q ∈ Q ⩽1 and ζ ∈ D[0,1] is bijective, we can perform the change of variables q → ζ in (6.128). Using also the identity in the previous display, we obtain that

-f β 2 2 ,0 + β 2 2 = inf ζ ∈D[0,1] Φ ζ (0,0) -β 2 ˆ1 0 tζ (t)dt , (6.132)
as announced. ∎

Towards non-convex models

Let us start by summarizing what we have done so far. For the SK model, we argued for the introduction of an enriched free energy F N (t,q) which depends on a variable t ⩾ 0 and a path q ∈ Q 1 (R ⩾0 ). This free energy is defined precisely in (6.53) for piecewise-constant paths, and then extended by continuity. The quantity we primarily want to compute is the large-N limit of F N (t,0), but the introduction of the additional variable is likely to help us to discover this limit. Indeed, we argued informally that if f = f (t,q) ∶ R ⩾0 × Q 2 (R ⩾0 ) → R denotes the limit of the free energy, we expect f to be a solution to

∂ t f (t,q) - ˆ1 0 ∂ q f (t,q,u) 2 du = 0 on R >0 × Q 2 (R ⩾0 ). (6.133)
Moreover, thanks to the fact that the reference measure P N is a product measure, we have that F N (0,q) = F 1 (0,q) for every N, so the initial condition for f is f (0,q) = ψ(q) ∶= F 1 (0,q). (6.134)

This object is computable relatively explicitly, since it only involves averages with respect to a single spin variable. For piecewise-constant paths q, we can compute this quantity recursively using Theorem 5.25, and then use the continuity of ψ for more general paths. Alternatively, we can also express this calculation in the form of the parabolic equation in (6.90).

Since the non-linearity in (6.133) is convex, we expect that the solution to this equation admits a Hopf-Lax variational representation, which should take the form

f (t,q) = sup q ′ ∈Q 2 (R ⩾0 ) ψ(q + q ′ ) - 1 4t ˆ1 0 (q ′ (u)) 2 du . (6.135)
And indeed, when setting q = 0 in this formula, we verified that this guess yields the answer predicted by Parisi in [START_REF] Parisi | Infinite number of order parameters for spin-glasses[END_REF][START_REF] Parisi | The order parameter for spin glasses: a function on the interval 0-1[END_REF][START_REF] Parisi | A sequence of approximated solutions to the S-K model for spin glasses[END_REF] and proved rigorously in [START_REF] Guerra | Broken replica symmetry bounds in the mean field spin glass model[END_REF][START_REF] Talagrand | The Parisi formula[END_REF].

In fact, the convergence of the enriched free energy F N (t,q) to the function f (t,q) defined in (6.133) can be shown rigorously for arbitrary choices of q ∈ Q 2 (R ⩾0 ).

Theorem 6.8 ( [START_REF] Mourrat | Extending the Parisi formula along a Hamilton-Jacobi equation[END_REF]). Let F N (t,q) be the enriched free energy defined in (6.53) and extended by continuity in Proposition 6.3, and let f (t,q) be as in (6.135). For every t ⩾ 0 and q ∈ Q 2 (R ⩾0 ), we have lim N→+∞ F N (t,q) = f (t,q). (6.136)

In fact, this result was shown to be valid for arbitrary choices of the measure P 1 of compact support and for general mixed p-spin models, provided that we replace the definition of the Hamiltonian in (6.54) by

H N (t,q,σ ,α) ∶= √ 2tH N (σ ) -Ntξ σ 2 N + Z q (α) ⋅ σ -q K σ 2 , (6.137)
and with f as in (6.118).

The main motivation for exploring this chain of ideas is to make progress towards the understanding of non-convex models. A typical representative in this class is the bipartite model defined in (6.18). We recall that for this model, the configuration vector σ is split into two parts, σ = (σ 1 ,σ 2 ) ∈ Σ 2 N . While we keep the length of each of these two components the same for convenience of notation, we point out that we do not plan to exploit any symmetry between these two components; to make this clearest, one can think that under the reference measure P N , the vectors (σ 1,i ) 1⩽i⩽N and (σ 2,i ) 1⩽i⩽N are independent with i.i.d. coordinates, but that the mean of σ 1,i may be different from the mean of σ 2,i .

For this model, the naive extension of the free energy we explored in Section 6.2 would already involve the choice of two additional parameters instead of one, so that we can act on each of the two coordinates of σ separately. Explicitly, the analogue of the free energy in (6.32) would be, for every t ⩾ 0 and h

= (h 1 ,h 2 ) ∈ R 2 ⩾0 , F N (t,h) ∶= - 1 N Elog ˆΣ2 N exp √ 2tH N (σ ) -Nt + 2h 1 z 1 ⋅ σ 1 -Nh 1 + 2h 2 z 2 ⋅ σ 2 -Nh 2 dP N (σ ), (6.138)
where z 1 and z 2 are independent standard N-dimensional Gaussian vectors, independent from (H N (σ )) σ ∈Σ 2

N

. The same Gaussian integration by parts calculations as those in (6.34) and (6.35) yield that, for each a ∈ {1,2},

∂ ha F N (t,h) = N -1 E⟨σ 1 a ⋅ σ 2 a ⟩, (6.139)
while ∂ t F N (t,h) = N -2 E⟨(σ 1 1 ⋅ σ 2 1 )(σ 1 2 ⋅ σ 2
2 )⟩. (6.140) This suggests that we should consider the relation

∂ t F N -∂ h 1 F N ∂ h 2 F N (t,h) = N -2 E⟨ σ 1 1 ⋅ σ 2 1 -E⟨σ 1 1 ⋅ σ 2 1 ⟩ σ 1 2 ⋅ σ 2 2 -E⟨σ 1 2 ⋅ σ 2 2 ⟩ ⟩. (6.141)
This term would be small if the overlaps were concentrated. Indeed, using that xy ⩽ (x 2 + y 2 ) 2, we see that

∂ t F N -∂ h 1 F N ∂ h 2 F N (t,h) ⩽ 1 2N 2 2 a=1 E⟨ σ 1 a ⋅ σ 2 a -E⟨σ 1 a ⋅ σ 2 a ⟩ 2 . (6.142)
However, as for the SK model, these overlaps will typically not have a small variance when t is large. As in Section 6.4, we therefore need to define a more sophisticated enriched free energy. To do so, we give ourselves an integer K ⩾ 1 and parameters

0 = ζ 0 < ζ 1 < ⋯ < ζ K < ζ K+1 = 1, (6.143) 0 = q -1,a ⩽ q 0,a ⩽ q 1,a ⩽ ⋯ ⩽ q K,a , (6.144)
for a ∈ {1,2}. For each k ∈ {-1,...,K}, we write q k ∶= (q k,1 ,q k,2 ) ∈ R 2 ⩾0 , and we also impose that q k ≠ q k+1 . Notice that the sequence (q k ) 0⩽k⩽K is increasing for the partial order induced by R 2 ⩾0 . We encode all these parameters into the path q ∶ [0,1) → R 2 ⩾0 defined by

q ∶= K k=0 q k 1 [ζ k ,ζ k+1 ) . (6.145)
We write

Q(R 2 ⩾0 ) ∶= q ∶ [0,1) → R 2 ⩾0
q is right-continuous and non-decreasing , (6.146)

where we understand that the notion of non-decreasingness is for the partial order induced by R 2 ⩾0 , that is, a path q ∶ [0,1) → R 2 ⩾0 is non-decreasing if for every u ⩽ v ∈ [0,1), we have q(v) -q(u) ∈ R 2 ⩾0 . We give ourselves a Poisson-Dirichlet cascade (v α ) α∈N K built over the tree A with the parameters in (6.143), and two families (z α,a ) α∈A,a∈{1,2} of independent standard N-dimensional random variables, independent of each other and of every other source of randomness. We then set, for each a ∈ {1,2},

Z q,a (α) ∶= K k=0 (2q k,a -2q k-1,a ) 1 2 z α k ,a . (6.147)
Finally, we define the enriched free energy

F N (t,q) ∶= - 1 N Elog ˆΣ2 N α∈N K
exp H N (t,q,σ ,α) v α dP N (σ ) (6.148) associated with the Hamiltonian

H N (t,q,σ ,α) ∶= √ 2tH N (σ ) -Nt + 2 a=1 Z q,a (α) ⋅ σ a -Nq K,a . (6.149)
The proof of Proposition 6.3 applies to this setting as well and allows us to extend

F N to R ⩾0 × Q 1 (R 2 ⩾0
), where we write

Q p (R 2 ⩾0 ) ∶= Q(R 2 ⩾0 ) ∩ L p ([0,1];R 2 ) for p ∈ [1,+∞].
The derivative calculations from Lemma 6.2 can be performed similarly here: the relation (6.140) is still valid, while for every k ∈ {0,...,K} and a ∈ {1,2},

∂ q k,a F N (t,q) = N -1 E⟨1 {α 1 ∧α 2 =k} σ 1 a ⋅ σ 2 a ⟩. (6.150)
In particular,

∂ t F N (t,q) - K k=0 (ζ k+1 -ζ k ) ∂ q k,1 F N (t,q) ζ k+1 -ζ k ∂ q k,2 F N (t,q) ζ k+1 -ζ k = N -2 E⟨ σ 1 1 ⋅ σ 2 1 -E⟨σ 1 1 ⋅ σ 2 1 α 1 ∧ α 2 ⟩ σ 1 2 ⋅ σ 2 2 -E⟨σ 1 2 ⋅ σ 2 2 α 1 ∧ α 2 ⟩ ⟩. (6.151)
As for (6.142), this quantity can be bounded in absolute value by

1 2N 2 2 a=1 E⟨ σ 1 a ⋅ σ 2 a -E⟨σ 1 a ⋅ σ 2 a α 1 ∧ α 2 ⟩ 2 ⟩. (6.152)
Up to the addition of a small perturbation to the Hamiltonian, one can enforce the asymptotic validity of a family of Ghirlanda-Guerra identities involving different linear combinations of all the overlaps at play. This implies the ultrametricity of these overlaps, which in turn implies that they must be synchronized. In short, we can make sure that the quantity in (6.152) is small "most of the time", provided that the integer K is sufficiently large. This suggests that the limit f ∶ R ⩾0 ×Q 2 (R 2 ⩾0 ) → R of the free energy should satisfy the infinite-dimensional Hamilton-Jacobi equation

∂ t f (t,q) - ˆ1 0 ∂ q 1 f (t,q,u)∂ q 2 f (t,q,u)du = 0 on R >0 × Q 2 (R 2 ⩾0 ), (6.153)
where we write q = (q 1 ,q 2 ) to denote the two components of the path q, or equivalently, we write ∂ q f (t,q,u) = (∂ q 1 f (t,q,u),∂ q 2 f (t,q,u)). The initial condition, which we still denote by ψ ∶ Q 1 (R 2 ⩾0 ) → R, is such that ψ(q) = F 1 (0,q), and it can be determined more explicitly using the same procedure as for the SK model.

It is at this stage that a new problem occurs. Indeed, we can no longer expect a Hopf-Lax variational representation to be valid for the solution to (6.153), since the non-linearity, which in essence is the mapping ξ (x,y) ∶= xy featuring in the covariance calculation (6.19), is neither convex nor concave. If we were to ignore this and blindly try to write down a Hopf-Lax variational formula anyway, we would need to consider the convex dual of ξ , which would be defined, for a,b ∈ R, analogously to (6.119), as ξ * (a,b) ∶= sup

x,y⩾0 ax + byxy). (6.154) Since this function takes the value +∞ on R 2 ⩾0 ∖ {0}, pursuing this direction does not look promising. A more refined attempt in the same direction is also shown to be invalid in Subsection 6.2 of [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF].

In the context of the models of statistical inference explored in Chapter 4, we could alternatively leverage the fact that the initial condition is always convex. This convexity property is valid even for the very general models discussed at the end of Section 4.3, and some of those models would be associated with a non-convex non-linearity in the equation. Also in view of the convex selection principle, the next question we need to ask is therefore whether the initial condition ψ in our present context is convex or concave, or whether it is neither of those. Unfortunately, the answer is negative in general. In Exercise 6.7, a simple example with only one type of spin taking values in Σ 1 is given for which the initial condition is neither convex nor concave, even if we restrict it to the space of constant paths 2 . We also recall that, in order for a Hopf variational representation to be valid, the convexity or concavity of the initial condition is not only sufficient, but also necessary, as is clear from the formula (3.68) with t = 0.

In short, our usual candidate variational representations for the solution to (6.153), namely the Hopf-Lax and the Hopf formulas, are invalid in general. This motivates to try to make sense of the solution to the Hamilton-Jacobi equation (6.153) directly. Perhaps the easiest way to proceed is to construct solutions to finite-dimensional approximations to (6.153), and then pass to the limit. That is, in the context of the SK model and inspired by (6.112), we would take a large integer K, define U K as in (6.109), and seek a function f (K) ∶ R ⩾0 ×U K → R that solves the finite-dimensional Hamilton-Jacobi equation

∂ t f (K) -(K + 1) K k=0 ∂ q k f (K) 2 = 0 on R >0 ×U K (6.155)
subject to the initial condition f (K) (0,q) = ψ(q q ), with q q as in (6.50). For the bipartite model, we would set

U 2 K ∶= q = (q 0 ,...,q K ) ∈ (R 2 ⩾0 ) K+1 0 ⩽ q 0 ⩽ q 1 ⩽ ⋯ ⩽ q K , (6.156)
where we understand that q k ⩽ q k+1 stands for q k+1q k ∈ R 2 ⩾0 , and we would aim to find a function f (K) ∶ R ⩾0 ×U 2 K → R that solves the finite-dimensional Hamilton-Jacobi equation

∂ t f (K) -(K + 1) K k=0 ∂ q k,1 f (K) ∂ q k,2 f (K) = 0 on R ⩾0 ×U 2 K , (6.157)
subject to the appropriate initial condition. Continuing with the bipartite model, for each q ∈ Q 2 (R 2 ⩾0 ), we would then want to use piecewise-constant approximations to q such as

q ≃ K k=0 1 [ k K+1 , k+1 K+1 ) 1 K + 1 ˆk+1 K+1 k K+1 q(u)du. (6.158)
Together with (6.110), this motivates the introduction of the sequence q (K) (q) = (q (K) k (q)) 0⩽k⩽K defined, for each k ∈ {0,...,K}, by

q (K) k (q) ∶= 1 K + 1 ˆk+1 K+1 k K+1
q(u)du.

(6.159) 2 For the expert reader who may think that this contradicts the main result of [START_REF] Auffinger | The Parisi formula has a unique minimizer[END_REF], we point out that the convexity shown there is with respect to the inverse function to our paths q. It is the convexity or concavity with respect to q that is relevant in the study of the variational representations to (6.153). We also add that Exercise 6.7 shows the invalidity of a Hopf variational representation to (6.153) for some choices of the reference measure, but we do not exclude the possibility that it is valid for other choices.

It is shown in [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF] that the functions f (K) are well-defined, and that for each q ∈ Q 2 (R 2 ⩾0 ), the limit

f (t,q) ∶= lim K→+∞ f (K) (t,q (K) (q)) (6.160)
exists and is finite. One may then decide that this limit object f is what we understand to be the solution to (6.153) with initial condition ψ. A more intrinsic notion, directly making sense of what it means to be a viscosity solution to (6.153) in infinite dimensions, was introduced in [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF] and shown to be equivalent to this definition by successive approximations. In the context of the SK model, one can write Hopf-Lax formulas for each of the finite-dimensional approximations and then pass to the limit in the resulting expressions, thereby verifying that the notion of solution just introduced here coincides with the variational definition in (6.135) (see also [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF][START_REF] Chen | Hamilton-Jacobi equations with monotone nonlinearities on convex cones[END_REF]). From now on, whenever we refer to the solution to (6.153), we refer to the function defined through the procedure ending with (6.160).

We are thus led to the following question. Question 6.9. Let F N (t,q) be the enriched free energy for the bipartite model, as defined in (6.148), and let f (t,q) be the solution to (6.153) with initial condition ψ.

Is it true that for every t ⩾ 0 and q ∈ Q 2 (R 2 ⩾0 ), we have lim N→+∞ F N (t,q) = f (t,q) ? (6.161)

We do not know the answer to this question. The following result gives an inequality between the two terms in (6.161). Theorem 6.10 ( [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF]). With the same notation as in Question 6.9, we have for every t ⩾ 0 and q ∈ Q 2 (R 2 ⩾0 ) that liminf N→+∞ F N (t,q) ⩾ f (t,q). (6.162)

One can also show that any subsequential limit of F N must satisfy the equation (6.153) "almost everywhere". This is shown in a sense involving finitedimensional approximations in [START_REF] Mourrat | Nonconvex interactions in mean-field spin glasses[END_REF], and also directly in infinite dimensions in [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] for a precise infinite-dimensional generalization of the notion of "almost everywhere". As was explained in Example 3.1, the verification of such a property does not suffice to identify the solution uniquely, even in finite dimensions.

We now describe how Question 6.9 would be phrased for more general meanfield spin glasses with a finite number of types. For a fixed integer D ⩾ 1, we take our configuration vector to be of the form σ = (σ 1 ,...,σ D ) ∈ (R N ) D . We take P N to be a probability measure on (R N ) D ≃ R D×N such that if σ is sampled according to P N , and if we think of σ as a D-by-N matrix, then the N columns of σ are independent and identically distributed with a fixed distribution P 1 of bounded support in R D . We give ourselves a centred Gaussian field (H N (σ )) σ ∈R D×N such that, for some smooth function ξ ∶ R D×D → R and for every σ 1 ,σ 2 ∈ (R N ) D ,

EH N (σ 1 )H N (σ 2 ) = Nξ σ 1 d ⋅ σ 2 d ′ N d,d ′ ⩽D . ( 6.163)
The mean-field character of the model is encoded by the fact that this covariance depends only on the D-by-D matrix of scalar products between the different "types" of spins. The set of functions ξ for which such a Gaussian random field exists is described precisely in Proposition 6.6 of [START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF]. Writing S D ⩾0 to denote the set of Dby-D positive semi-definite matrices, we give ourselves a family of real parameters (ζ k ) 1⩽k⩽K as in (6.143) and

0 = q -1 ⩽ q 0 < q 1 < ⋯ < q K ∈ S D ⩾0 , (6.164)
where we understand that q k ⩽ q k+1 means that q k+1q k ∈ S D ⩾0 , and q k < q k+1 means that q k ⩽ q k+1 and q k ≠ q k+1 . We encode this into a path q as in (6.145), an element of the space of paths Q(S D ⩾0 ) defined analogously to (6.146). We take (v α ) α∈N K to be the Poisson-Dirichlet cascade associated with the parameters (ζ k ) 1⩽k⩽K , and let (z α ) α∈A be independent D-by-N random matrices whose entries are independent standard Gaussians. We then define

Z q (α) ∶= K k=0 (2q k -2q k-1 ) 1 2 z α k , (6.165)
which is a random D-by-N matrix. Recall that for any two matrices a and b of the same size, we denote by a ⋅ b their entry-wise scalar product, that is, a ⋅ b ∶= tr(ab * ).

We define the enriched free energy as

F N (t,q) ∶= - 1 N Elog ˆΣ2 N α∈N K exp H N (t,q,σ ,α) v α dP N (σ ) (6.166)
for the Hamiltonian

H N (t,q,σ ,α) ∶= √ 2tH N (σ ) -Ntξ σ d ⋅ σ d ′ N d,d ′ ⩽D + Z q (α) ⋅ σ -σ ⋅ q K σ . (6.167)
Similar arguments as for the bipartite model lead us to expect that, as N tends to infinity, the free energy F N in (6.148) converges to the function

f ∶ R ⩾0 × Q 2 (S D ⩾0 ) → R that solves the infinite-dimensional Hamilton-Jacobi equation ∂ t f (t,q) - ˆ1 0 ξ ∂ q f (t,q,u) du = 0 on R >0 × Q 2 (S D ⩾0 ) (6.168)
subject to the initial condition f (0,q) = ψ(q) ∶= F 1 (0,q). One can construct a solution to this equation exactly as in the procedure leading to (6.160).

Question 6.11. Let F N (t,q) be the enriched free energy (6.166), and let f (t,q) be the solution to (6.168) with initial condition ψ. Is it true that for every t ⩾ 0 and q ∈ Q 2 (S D ⩾0 ), we have lim N→+∞ F N (t,q) = f (t,q) ? (6.169)

The results presented in Theorem 6.10 and in the paragraph below have been extended to this setting [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF][START_REF] Mourrat | Free energy upper bound for mean-field vector spin glasses[END_REF]. We also mention that, for certain problems of community detection which, unlike the situation explored in Section 4.5, involve random graphs whose average degree remains bounded, very similar difficulties emerge [START_REF] Dominguez | Mutual information for the sparse stochastic block model[END_REF][START_REF] Kireeva | Breakdown of a concavity property of mutual information for non-Gaussian channels[END_REF]. When the covariance function ξ is convex over S D ⩾0 , the function f in Question 6.11 admits a Hopf-Lax variational representation [START_REF] Chen | Hamilton-Jacobi equations from mean-field spin glasses[END_REF], and the identity (6.169) is indeed valid [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF].

In Theorem 3.19, we observed that whenever the non-linearity or the initial condition of the finite-dimensional Hamilton-Jacobi equation (3.20) is convex or concave, the solution to this equation must be a critical value of the function J t,x defined in (3.53). Remarkably, the limit of the free energy F N (t,q) defined in (6.166), assuming it exists, must also satisfy this property for the natural analogue of J t,x associated with the infinite-dimensional Hamilton-Jacobi equation (6.168). What is truly remarkable is that this property is valid even in situations in which neither the non-linearity nor the initial condition possess any global convexity or concavity property. To state this precisely, we define, for every t ⩾ 0 and p,q,q ′ ∈ Q 2 (S D ⩾0 ), J t,q (q ′ ,p) ∶= ψ(q ′ ) + ˆ1 0 p(u) ⋅ (q -q ′ )(u)du +t ˆ1 0 ξ (p(u))du. (6.170) Theorem 6.12 ( [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF]). Suppose that the free energy F N defined in (6.166) converges pointwise to a function f ∶ R ⩾0 × Q 2 (S D ⩾0 ) → R. For every t ⩾ 0 and q ∈ Q 2 (S D ⩾0 ), there exists q ′ ,p ∈ Q 2 (S D ⩾0 ) such that q = q ′ -t∇ξ (p), p = ∂ q ψ(q ′ ), (6.171) and f (t,q) = J t,q (q ′ ,p). (6.172)

The conditions in (6.171) correspond to the requirement that the pair (q ′ ,p) be a critical point of the functional J t,q . They can be written more explicitly as q(u) = q ′ (u) -t∇ξ (p(u)), p(u) = ∂ q ψ(q ′ ,u) du-a.e. in [0,1). (6.173)

As was observed in Section 3.5, these conditions can be interpreted as saying that the characteristic line starting at q ′ passes through the point (t,q). Roughly speaking, it is also shown in [START_REF] Chen | On the free energy of vector spin glasses with non-convex interactions[END_REF] that, up to a small perturbation of the parameters, one can choose the critical point (q ′ ,p) in Theorem 6.12 in such a way that the overlap matrix

N -1 (σ d ⋅ σ ′ d ′ ) d,d ′ ⩽D converges in law to p(U)
, where U is a uniform random variable on [0,1]. Exercise 6.7. Let P 1 be a probability measure on R with compact support, let z be a standard Gaussian random variable, and consider the free energy

f (t) ∶= -Elog ˆR exp √ 2tzσ -tσ 2 dP 1 (σ )
associated with the Hamiltonian H(t,σ ) ∶= √ 2tzσ -tσ 2 . Denoting by ⟨⋅⟩ its associated Gibbs measure, show that

∂ 2 t f = 2E ⟨σ 2 ⟩ -⟨σ ⟩ 2 ⟨σ 2 ⟩ -3⟨σ ⟩ 2 .
Deduce that for a suitable choice of measure P 1 , the free energy f is neither convex nor concave.

Appendix A Basic results in analysis and probability

In this appendix, we give detailed proofs of the basic results in analysis and probability theory that are used in the main text. The topics covered are classical, so readers already familiar with the material can safely skip it; we have merely included it to keep the book as self-contained as possible. The key results that we prove are the Caratheodory extension theorem, the Dynkin π-λ theorem, the Riesz representation theorem, the Stone-Weierstrass theorem, the Lebesgue differentiation theorem, the Portmanteau theorem, the Prokhorov theorem, and the injectivity of the Fourier and Laplace transforms. Although some of these results can be proved in greater generality, we will focus on the setting in which we give ourselves a metric space S, whose metric we denote by d.

A.1 Constructing measures

One of the most classical problems in geometry is to determine the length, area, or volume of a curve, surface or solid. This problem can be generalized to the setting of a metric space S as the task of assigning an adequate notion of size or measure to a subset of S. Intuitively, any reasonable notion of measure should be countably additive, monotone, and assign no mass to the empty set. Unfortunately, examples such as that due to Vitali [START_REF] Vitali | Sulla misura dei gruppi di punti di una retta[END_REF] show that it is not in general possible to assign such a notion of measure to all sets in S. To overcome this issue, we will only define the measure of a collection S of subsets of S which make the pair (S,S) a measurable space.

A pair (S,S) is a measurable space if S is a σ -algebra of subsets of S. A collection A of subsets of S is called an algebra if it contains S, it is closed under finite unions and it is closed under complements, (i) S ∈ A, (ii) for every A,B ∈ A, we have A ∪ B ∈ A, Proposition A.1. If E ⊆ 2 S is a collection of sets with ∅,S ∈ E and ρ ∶ E → [0,+∞] is a non-negative set function with ρ(∅) = 0, then (A.10) is an outer measure.

Proof. Taking E n = ∅ ∈ E for all n ⩾ 1 shows that µ * (∅) = ρ(∅) = 0. If A,B are subsets of S with A ⊆ B, then the infimum defining µ * (A) is taken over a larger set than that defining µ * (B) which means that µ * (A) ⩽ µ * (B). To establish countable sub-additivity, we fix a sequence (A n ) n⩾1 of subsets of S as well as ε > 0. For each n ⩾ 1, let (E n,i ) i⩾1 ⊆ E be such that for every n ⩾ 1,

A n ⊆ ∞ ⋃ j=1 E n,i and +∞ i=1 ρ(E n,i ) ⩽ µ * (A n ) + ε 2 n . Since (E n,i ) n,i⩾1 forms a cover of ⋃ ∞ n=1 A n , the definition of µ * implies that µ * ∞ ⋃ n=1 A n ⩽ +∞ n,i=1 ρ(E n,i ) ⩽ +∞ n=1 µ * (A n ) + ε.
Letting ε tend to zero completes the proof. ∎

The key insight behind the Caratheodory extension theorem is that an outer measure restricted to the σ -algebra of Caratheodory measurable sets becomes a measure. A subset A of S is Caratheodory measurable with respect to an outer measure µ * if, for every other subset E of S, µ * (E) = µ * (E ∩ A) + µ * (E ∩ A c ).

(A.11)

In other words, A is Caratheodory measurable if it can be used to partition any other set E in an additive fashion. We will write S * ∶= {A ⊆ S A is Caratheodory measurable} (A.12)

for the collection of Caratheodory measurable sets.

Theorem A.2 (Caratheodory extension). If µ * is an outer measure on S, then the collection S * of Caratheodory measurable sets is a σ -algebra, and the restriction of µ * to S * is a measure.

Proof. Since µ * (∅) = 0 and the definition of Caratheodory measurable is symmetric in A and A c , the set S * contains S and is closed under complements. To show that it is an algebra, fix two Caratheodory measurable sets A,B ∈ S * as well as an arbitrary subset E of S. By Caratheodory measurability of A and B,

µ * (E) = µ * (E ∩ A ∩ B) + µ * (E ∩ A ∩ B c ) + µ * (E ∩ A c ∩ B) + µ * (E ∩ A c ∩ B c ).
Combining the fact that A ∪ B = (A ∩ B) ∪ (A ∩ B c ) ∪ (A c ∩ B) with the sub-additivity of µ * reveals that

µ * (E ∩ (A ∪ B)) ⩽ µ * (E ∩ A ∩ B) + µ * (E ∩ A ∩ B c ) + µ * (E ∩ A c ∩ B).
It follows that

µ * (E ∩ (A ∪ B)) + µ * (E ∩ (A ∪ B) c ) ⩽ µ * (E),
where we have used the fact that (A ∪ B) c = A c ∩ B c . By sub-additivity of µ * this inequality is in fact an equality so A ∪ B ∈ S * and S * is an algebra. To prove that S * is a σ -algebra, it suffices to fix a sequence (A n ) n⩾1 of disjoint sets in S * and show that their union also lies in S * . Fix an integer n ⩾ 1, and to simplify notation, let B n ∶= ⋃ n i=1 A i ∈ S * and B ∶= ⋃ ∞ n=1 A n . For any subset E of S, the Caratheodory measurability of A n and the assumption that the sets A i are disjoint imply that

µ * (E ∩ B n ) = µ * (E ∩ B n ∩ A n ) + µ * (E ∩ B n ∩ A c n ) = µ * (E ∩ A n ) + µ * (E ∩ B n-1 ).
It follows by Caratheodory measurability of B n and an induction that

µ * (E) = µ * (E ∩ B n ) + µ * (E ∩ B c n ) = n i=1 µ * (E ∩ A i ) + µ * (E ∩ B c n ).
Noticing that B c ⊆ B c n , leveraging the monotonicity of µ * , and letting n tend to infinity shows that

µ * (E) ⩾ +∞ n=1 µ * (E ∩ A n ) + µ * (E ∩ B c ). (A.13)
Invoking the sub-additivity of µ * reveals that

µ * (E) ⩾ µ * ∞ ⋃ n=1 (E ∩ A n ) + µ * (E ∩ B c ) = µ * (E ∩ B) + µ * (E ∩ B c ) ⩾ µ * (E),
so in fact these inequalities are equalities and B ∈ S * as required for S * to be a σ -algebra. That µ * restricts to a measure on S * is immediate from (A. [START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF]) with E = B and the sub-additivity of outer measure. This completes the proof. ∎

The Caratheodory extension theorem is often stated as the possibility to extend a pre-measure from an algebra to the σ -algebra it generates. A pre-measure µ 0 on an algebra A is a non-negative set function µ 0 ∶ A → [0,+∞] which assigns no mass to the empty set and is countably additive whenever this makes sense,

(i) µ(∅) = 0, (ii) if (A n ) n⩾1 is a sequence of disjoint sets in A with ⋃ ∞ n=1 A n ∈ A, then µ 0 ∞ ⋃ n=1 A n = +∞ n=1 µ 0 (A n ). (A.14)
We have opted for the more general version of the Caratheodory extension theorem as we will prove the Riesz representation theorem by extending a set function from the collection of open sets, which does not form an algebra, to the Borel σ -algebra B(S). Nonetheless, let us establish this simplified version of the Caratheodory extension theorem and use it to extend the pre-measure m defined by (A.7) to a measure on the Borel σ -algebra B(R). The fact that (A.7) is a well-defined pre-measure on the algebra (A.6) is the content of Exercise A.2.

Corollary A.3 (Simplified Caratheodory extension). If µ 0 ∶ A → R is a pre-measure on an algebra A, then µ 0 can be extended to a measure µ on the σ -algebra σ (A).

Proof. Denote by µ * ∶ 2 S → [0,+∞] the set function defined by (A.10) with ρ = µ 0 , and recall that it is an outer measure by Proposition A.1. It follows by the Caratheodory extension theorem that the restriction µ of µ * to the σ -algebra S * of Caratheodory measurable sets is a measure. To see the µ is an extension of µ 0 , fix A ∈ A, and let (E n ) n⩾1 ⊆ A be a covering of A. Introduce the disjoint sets

F n ∶= A ∩ E n ∖ n-1 ⋃ i=1 E i ∈ A
whose union is A. The definition of a pre-measure implies that

µ 0 (A) = +∞ n=1 µ 0 (F n ) ⩽ +∞ n=1 µ 0 (E n ),
and taking the infimum over all coverings (E n ) n⩾1 shows that µ 0 (A) ⩽ µ * (A).

Taking the covering E 1 = A and E n = ∅ for n ⩾ 2 shows that this is in fact an equality, so µ 0 = µ * = µ on A. To show that the extension µ is defined on σ (A), it suffices to verify that A ⊆ S * . Fix a set A ∈ A as well as an arbitrary subset E of S and ε > 0.

Let (E n ) n⩾1 ⊆ A be a sequence of elementary sets with

E ⊆ ∞ ⋃ n=1 E n and µ * (E) ⩽ +∞ n=1 µ 0 (E n ) + ε. Since µ 0 is additive on A, µ * (E) + ε ⩾ +∞ n=1 µ 0 (E n ∩ A) + +∞ n=1 µ 0 (E n ∩ A c ) ⩾ µ * (E ∩ A) + µ * (E ∩ A c ).
Since ε is arbitrary, together with the sub-additivity of outer measure, this shows that A ∈ S * and completes the proof. ∎ Exercise A.4. Fix a finite measure µ on the Borel σ -algebra a metric space S. The purpose of this exercise is to show that µ is closed regular in the sense that (A.20) holds for any measurable set A ⊆ S.

(i) Show that the set

L ∶= A ∈ S µ(A) = sup F⊆A F closed µ(F) and µ(A c ) = sup F⊆A c F closed µ(F) (A.26)
is a λ -system.

(ii) Prove that L contains all open sets.

(iii) Conclude that µ is closed regular.

A.2 The Riesz representation theorem

The Riesz representation theorem allows us to identify a positive linear functional on the space of continuous functions on a compact metric space S with a unique measure on the Borel σ -algebra B(S). A positive linear functional on the space of continuous functions on S is a linear functional T ∶ C(S;R) → R that is non-negative on non-negative functions, (i) for all a,b ∈ R and f ,g ∈ C(S;R), we have T (a f + bg) = aT ( f ) + bT (g),

(ii) for all f ∈ C(S;R) with f ⩾ 0, we have T ( f ) ⩾ 0.

Notice that we do not require the continuity of the linear functional T ; however, as shown in Exercise A.5, this continuity is implied by the positivity of T . To prove the Riesz representation theorem, we will rely on two classical topological results whose proofs are particularly simple in the metric setting.

Lemma A.7 (Urysohn). If F is a closed subset of a metric space S that is contained in some open set U ⊇ F, then there exists a continuous function f ∶ S → [0,1] which is equal to one on F and vanishes outside U,

1 F ⩽ f ⩽ 1 U . (A.27)
Proof. Denote by d the metric on S, and define the function

f ∶ S → [0,1] by f (x) ∶= d(x,U c ) d(x,F) + d(x,U c ) .
Since the distance to a closed set is continuous and F ∩U c = ∅, the function f is continuous. Moreover, we have f (x) = 0 if and only if x ∈ U c and f (x) = 1 if and only if x ∈ F. This completes the proof. ∎ an open set U ⊆ S. For each integer n ⩾ 1, define the closed and therefore compact set

K n ∶= x ∈ S d(x,U c ) ⩾ 1 n in such a way that U = ⋃ ∞ n=1 K n .
Given an integer n ⩾ 1, invoke Urysohn's lemma to find a continuous function f n with 1 Kn ⩽ f n ⩽ 1 U . Since µ satisfies (A.29), we have that µ(K n ) ⩽ T ( f n ). It follows by the continuity of measure in Exercise A.1 that

µ(U) = lim n→+∞ µ(K n ) ⩽ limsup n→+∞ T ( f n ) ⩽ sup T ( f ) f ∈ C(S;R) and 0 ⩽ f ⩽ 1 U .
On the other hand, if f is a continuous function with 0 ⩽ f ⩽ 1 U , then (A.29) implies that T ( f ) ⩽ µ(U). Taking the supremum over all such functions f shows that

µ(U) = sup T ( f ) f ∈ C(S;R) and 0 ⩽ f ⩽ 1 U . (A.30)
This determines the measure µ on the π-system of open sets. Since µ is a finite measure, this in fact determines µ on the Borel σ -algebra by Exercise A.3.

Step To prove that this defines an outer measure, it suffices to show that ρ is sub-additive on the collection of open sets. Indeed, if this is the case, then

µ * (A) = inf +∞ n=1 ρ(U n ) A ⊆ ∞ ⋃ n=1 U n and U n is open for every n ⩾ 1
which is an outer measure by Proposition A.1. Given a sequence (U n ) n⩾1 of open sets, let U ∶= ⋃ ∞ n=1 U n , and fix a continuous function f with 0 ⩽ f ⩽ 1 U . Denote by K the closed support of f . Since K is compact as a closed subset of the compact space S and it is contained in U, it is possible to extract a finite sub-cover (U i ) i⩽n of K. By Lemma A.8 there exists a partition of unity

(h i ) i⩽n on K subordinate to the open cover (U i ) i⩽n . Since ∑ n i=1 h i = 1 on the support of K, we have that f = ∑ n i=1 gh i . It follows by linearity of T that T ( f ) = n i=1 T ( f h i ) ⩽ n i=1 ρ(U i ) ⩽ +∞ n=1 µ(U n ),
where we have used that 0 ⩽ f h i ⩽ 1 U i . Taking the supremum over all continuous functions f with 0

⩽ f ⩽ 1 U shows that ρ is sub-additive on the collection of open [a,b] ⊆ R. Given ε > 0, let a = y 0 < y 1 < ... < y n-1 < y n = b be a partition of the interval [a,b] with y i+1 -y i < ε for 0 ⩽ i ⩽ n -1. Introduce the collection of disjoint Borel sets E i ∶= f -1 ((y i-1 ,y i ]) ∩ K which partition K = ⋃ n i=1 E i . For each 1 ⩽ i ⩽ n,
T ( f ) = n i=1 T ( f h i ) ⩽ n i=1 (y i + ε)T (h i ) ⩽ n i=1 (y i + ε)ρ(U i ).

It follows by the properties (A.34) of the open sets

(U i ) i⩽n that T ( f ) ⩽ n i=1 (y i + ε) µ(E i ) + ε n ⩽ n i=1 (y i -ε)µ(E i ) + ε(2µ(K) + f ∞ + ε).
Combining this with the fact that f (x) > y i-1 > y iε on E i and letting ε tend to zero establishes (A.33) and completes the proof. ∎ Exercise A.5. Show that any positive linear functional T ∶ C(S;R) → R on the space of continuous functions on a compact metric space S is continuous with respect to the uniform norm.

A.3 The Stone-Weierstrass theorem

In this section, we prove a generalization of the well-known theorem of Weierstrass [START_REF] Weierstrass | Über die analytische Darstellbarkeit sogenannter willkürlicher Functionen einer reellen Veränderlichen[END_REF] which states that any continuous function on a compact interval [a,b] is the uniform limit of polynomials on [a,b]. A probabilistic proof of this classical result leveraging Bernstein polynomials is outlined in Exercise A.6. Following the work of Stone [START_REF] Marshall | The generalized Weierstrass approximation theorem[END_REF], we will generalize this result in two main directions. We will consider continuous functions on a compact metric space S as opposed to compact sub-intervals of the real line, and we will be concerned with the density of collections of functions which form an algebra in the space of continuous functions on S as opposed to focusing on polynomials. We will prove a version of the Stone-Weierstrass theorem for real-valued functions and another for complex-valued functions. The Stone-Weierstrass theorem is a very helpful tool as it often allows us to deduce an identity for all continuous functions once it has been proved for a smaller family of functions such as monomials. In Section A.6, we will use the real version of the Stone-Weierstrass theorem to show that a non-negative random variable is entirely determined by its Laplace transform, and we will leverage the complex version of the Stone-Weierstrass theorem to prove that a random variable is entirely determined by its characteristic function.

The real version of the Stone-Weierstrass theorem states that a collection of functions F which forms an algebra in the space of continuous real-valued functions on a compact metric space S is dense in C(S;R) endowed with the uniform norm,

f -g ∞ = sup x∈S f (x) -g(x) , (A.35)
provided it contains constants and separates points. An algebra in C(S;R) is a real vector subspace F of C(S;R) which is closed under multiplication, (i) for all λ , µ ∈ R and f ,g ∈ F, we have λ f + µg ∈ F, (ii) for all f ,g ∈ F, we have f g ∈ F.

A collection F of real-valued continuous functions on S separates points if, for all distinct x ≠ y ∈ S, there exists a function f ∈ F with f (x) ≠ f (y).

Theorem A.10 (Stone-Weierstrass real version). If (S,d) is a compact metric space and F is an algebra in C(S;R) which separates points and contains constants, then F is dense in (C(S;R), ⋅ ∞ ).

Proof. The proof proceeds in two steps. First, we will show that F is a lattice, in the sense that for all f ,g ∈ F, we have min( f ,g) ∈ F and max( f ,g) ∈ F. We will then fix f ∈ C(S;R) as well as ε > 0, and we will use the lattice property of F to construct an explicit function g ∈ F with fg ∞ ⩽ ε . This will establish the density of F in C(S;R). The density of F in C(S;R) is then immediate from its density in F.

Step 1: F is a lattice. To show that F is a lattice, it suffices to show that for any f ∈ F, we have f ∈ F. Indeed, this implies that for any f ,g ∈ F, we have

min( f ,g) = f + g -f -g 2 ∈ F and max( f ,g) = f + g + f -g 2 ∈ F.
Fix f ∈ F, and let M > 0 be large enough so the range of f is contained in the interval [-M,M]. By the Weierstrass approximation theorem in Exercise A.6, the absolute value function x ↦ x defined on the compact interval [-M,M] can be approximated by polynomials of x. This means that f (x) can be approximated by polynomials of f (x). Since F is an algebra which contains constants, any polynomial of f (x) belongs to F, so f ∈ F.

Step 2: constructing an approximating function. Observe that for any pair of distinct points s ≠ t ∈ S and all values c,d ∈ R, there exists g ∈ F with g(s) = c and g(t) = d.

Indeed, as F separates points, there exists h ∈ F with h(s) ≠ h(t), and the function g ∈ F defined by

g(x) ∶= c + (d -c) h(x) -h(s) h(t) -h(s)
satisfies the desired properties. We now fix a continuous function f ∈ C(S;R) as well as ε > 0. For all pairs of distinct points s ≠ t ∈ S, we write g s,t ∈ F for a function with g s,t (s) = f (s) and g s,t (t) = f (t). By continuity, the function g s,t approximates f in small enough neighbourhoods of both s and t. To construct a function g ∈ F which approximates f everywhere, fix s ∈ S, and consider the open neighbourhood of t, U t ∶= {x ∈ S g s,t (x) < f (x) + ε .

Since (U t ) t∈S defines an open cover of the compact set S, there exists a finite sub-cover (U t i ) i⩽n . The function g s ∶ S → R defined by

g s (x) ∶= min 1⩽i⩽n g s,t (x)
is such that g s (s) = f (s) and g s (x) < f (x) + ε for all x ∈ S. Moreover, we have that g s ∈ F as F is a lattice. Similarly, fix t ∈ S and consider the open neighbourhood of s, V s ∶= {x ∈ S g s (x) > f (x) -ε}.

Since (V s ) s∈S defines an open cover of the compact set S, there exists a finite sub-cover (U s j ) j⩽m . The function g ∶ S → R defined by

g(x) ∶= max 1⩽ j⩽m g s (x)
belongs to the lattice F and is such that fg ∞ ⩽ ε. This completes the proof. ∎

As shown in Exercise A.7, the real version of the Stone-Weierstrass theorem does not generalize to the setting of complex-valued continuous functions on S. It turns out that in addition to containing constants and separating points, for an algebra F in C(S;C) to be dense with respect to the uniform norm,

f -g ∞ = Re( f ) -Re(g) ∞ + Im( f ) -Im(g) ∞ , (A.36)
it must also be closed under conjugation. This means that for all f ∈ F, we need to have f ∈ F. It is worth emphasizing that an algebra in C(S;C) is a complex vector subspace of C(S;C) which is closed under multiplication. In particular, the first condition (i) in the definition of an algebra must hold for all λ , µ ∈ C.

Theorem A.11 (Stone-Weierstrass complex version). If (S,d) is a compact metric space and F is an algebra in C(S;C) which separates points, contains constants and is closed under complex conjugation, then F is dense in (C(S;C), ⋅ ∞ ).

Proof. Fix x ∈ R as well as ε > 0. The continuity of f gives δ > 0 with f

(t)-f (x) < ε whenever x -t < δ . If h ∈ R is such that h < δ , then F(x + h) -F(x) h -f (x) ⩽ 1 h ˆx+h x f (t) -f (x) dt ⩽ ε.
This completes the proof. ∎

The Lebesgue differentiation theorem will replace the continuity assumption of the integrand by local integrability and will weaken the conclusion to almost everywhere differentiability of the integral function. In higher dimensions, we will show in Theorem A. [START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF] Although the basic result that we will use from the theory of maximal operators holds in far greater generality, we will only state it on Euclidean space R d endowed with d-dimensional Lebesgue measure m in the context of operators on L 1 (R d ;R). The maximal operator associated with a family (T t ) t>0 of linear operators from L 1 (R d ;R) to the space of measurable functions on R d is the operator T * from L 1 (R d ;R) to the space of measurable functions on R d defined by

that if f ∈ L 1 loc (R d ;R), then for almost every x ∈ R d , lim r↘0 1 m(B r (x)) ˆBr(x) f (y) -f (
T * f (x) = sup t>0 T t f (x) (A.41)
The Hardy-Littlewood maximal operator (A.40) is the maximal operator associated with the family (H r ) r>0 of linear operators defined on L 1 (R d ;R) by Since C is an open cover of the compact set K, it is possible to extract a finite sub-cover (C j ) j⩽m ⊆ C. We now define the sequence (B i ) i⩽n iteratively by taking B 1 to be the ball C j of largest radius, and by taking B i to be the ball C j of largest radius disjoint from ⋃ i-1 k=1 B k . This construction ensures that if a ball C j is not selected, then it must intersect one of the selected balls B i , and if i is the smallest index such that this intersection takes place, then the radius of C j is at most the radius of B i . This means that K ⊆ ⋃ n i=1 3B i , where 3B i denotes the ball with the same centre as B i but with three times the radius. It follows by sub-additivity of measure that

H r f (x) ∶= 1 m(B r (x)) ˆBr(x) f (
c < m(K) ⩽ n i=1 m(3B i ) = 3 d n i=1 m(B i ).
The last equality uses the homogeneity of d-dimensional Lebesgue measure which is immediate from the Dynkin π-λ theorem and the fact that for every a > 0, we have m(aR) = a d m(R) for any finite union of disjoint rectangles R. This completes the proof. ∎ Proof. Fix λ > 0, and let

E λ ∶= {H * f (x) > λ }. Given x ∈ E λ , let r x > 0 be such that 1 m(B rx (x)) ˆBrx(x) f (y) dy ⩾ 1 m(B r (x)) ˆBr(x) f (y)dy > λ . The collection (B rx (x)) x∈E λ covers E λ , so, given any c < m(E λ ), the Vitali covering theorem gives a finite sub-collection (B rx i (x i )) i⩽n of disjoint balls with c ⩽ 3 d n i=1 m(B rx i (x i )) ⩽ 3 d λ n i=1 ˆBrx i (x i ) f (y) dy ⩽ 3 d λ ˆRd f (y) dy.
Letting c increase to m(E λ ) completes the proof.

∎ Theorem A.16 (Lebesgue differentiation). If f ∈ L 1 loc (R d ;R), then for almost every x ∈ R d , lim r→0 1 m(B r (x)) ˆBr(x) f (y) -f (x) dy = 0. (A.46)
of f . Since the sets (F y ) y∈R are disjoint, there exist at most countably many values of y for which P(F y ) > 0. Given ε > 0 it is therefore possible to find an integer N ⩾ 1 and a sequence a 0 ⩽ ⋯ ⩽ a N with max k (a ka k-1 ) ⩽ ε and P(F a k ) = 0 such that f maps into the interval (a 0 ,a N ). Introduce the function f ε ∶ S → R defined by (ii) Is restricting to points of continuity of F necessary?

f ε ∶= N-1 k=0 a k 1 B k , where B k ∶= {s ∈ S ∶ a k ⩽ f (s) < a k+1 }. Since f is continuous, for 0 ⩽ k ⩽ N -1, we have ∂ B k ⊆ F a k ∪ F a k+1 , and therefore P(∂ B k ) = 0. It follows by (v) that lim n→+∞ ˆS f ε dP n = N-1 k=0 a k P(B k ) = ˆS f ε dP. Using that f ε -f ∞ ⩽ ε to
Exercise A.10. The goal of this exercise is to show the continuous mapping theorem. Let S, S ′ be two metric spaces, let f ∶ S → S ′ be a measurable function, and let

C f ∶= x ∈ S f is continuous at x . (A.55)
Let (X n ) n⩾1 and X ∞ be random variables taking values in S with (X n ) n⩾1 converging in law to X ∞ as n tends to infinity.

(i) Show that the set C f is measurable.

(ii) Assuming that P{X ∞ ∈ C f } = 1, show that f (X n ) converges in law to f (X ∞ )
as n tends to infinity.

Exercise A.

A.6 Weak convergence through tightness and uniqueness

The topological characterization of weak convergence in the Portmanteau theorem is very useful to prove abstract results; however, it is difficult to use to establish the weak convergence of a specific sequence of probability measures. Typically, showing that a sequence (P n ) n⩾1 of probability measures converges weakly is a two-step process. First, we show that the sequence (P n ) n⩾1 is relatively compact, in the sense that any subsequence of (P n ) n⩾1 admits a weakly convergent subsequence, and then we show that there is only one possible subsequential limit.

Lemma A.19. Fix a sequence of probability measures (P n ) n⩾1 . If there exists a probability measure P with the property that any sequence (n(k)) k⩾1 admits a further subsequence (n(k(r))) r⩾1 such that (P n(k(r)) ) r⩾1 converges weakly to P, then (P n ) n⩾1 itself converges weakly to P.

Proof. Suppose for the sake of contradiction that (P n ) n⩾1 converges weakly to some probability measure P but that it is not uniformly tight. This means that there exist ε > 0 and r > 0 such that for any finite family of open balls B 1 ,...,B m of radius r > 0 there is some n ⩾ 1 with P n ⋃ m i=1 B i < 1ε. Indeed, if this were not the case, then given ε > 0 and k > 0 it would be possible to find a finite family of open balls B

(k) 1 ,...,B (k) n(k) of radius k -1 with P n n(k) ⋃ i=1 B (k) i ⩾ 1 -ε2 -k for every n ⩾ 1. The set K ∶= ∞ ⋂ k=1 n(k) ⋃ i=1 B (k) i
would then satisfy P n (K) ⩾ 1ε for every n ⩾ 1. Since S is complete, the closed and totally bounded set K is compact. This would contradict the assumption that (P n ) n⩾1 is not uniformly tight. With this r > 0 at hand, use the separability of S to find a collection

(B i ) i⩾1 of open balls of radius r > 0 with S = ⋃ ∞ i=1 B i . For each k ⩾ 1, let A k ∶= ⋃ k i=1 B i , and find n(k) ⩾ 1 with P n(k) (A k ) < 1 -ε. Since A m is open for each m ⩾ 1 and A m ⊆ A k for every k large enough, the Portmanteau theorem implies that P(A m ) ⩽ liminf k→+∞ P n(k) (A m ) ⩽ liminf k→+∞ P n(k) (A k ) < 1 -ε.
By the continuity of measure established in Exercise A.1 this contradicts the fact that S = ⋃ ∞ m=1 A m and completes the proof. ∎

In the context of our two-step strategy to establish weak convergence, we now turn our attention to the uniqueness of the limiting measure in the setting of Euclidean space R d . We will discuss two closely related transforms that entirely determine a probability measure, the characteristic function and the Laplace transform. The key difference between these two transforms is that the former is well-defined for any probability measure while the latter is slightly simpler but only well-defined for a smaller class of probability measures; here we will consider probability measures supported on the positive quadrant R d ⩾0 . The characteristic function of a probability measure P on R d is the complex-valued function ϕ ∶ R Fix a continuous function of compact support f ∈ C c (R d ;R) as well as ε > 0. Using Exercise A.12 on the tightness of probability measures on complete and separable metric spaces, find K large enough so that f vanishes outside the cube [-K 2,K 2] d , and so that P((-K,K)

d ) ⩾ 1 -ε and Q((-K,K) d ) ⩾ 1 -ε. For each m ∈ Z d denote by g m ∶ [-K,K] d → C the function g m (x) ∶= exp iπ 2K m ⋅ x , (A.64)
and write A for the algebra of finite linear combinations of the (g m ) m∈Z d . It is readily verified that A is an algebra in C(S;C) that separates points in the compact set [-K,K] d . Since A is closed under complex conjugation, the complex version of the Stone-Weierstrass theorem gives g ∈ A such that, for all

x ∈ [-K,K] d , f (x) -g(x) ⩽ ε 2 .
Remembering that f is supported on [-K 2,K 2] d , up to multiplying g by a nonnegative function which is equal to one on [-K 2,K 2] d , vanishes outside [-K,K] d and is bounded by one, assume without loss of generality that g is supported on

[-K,K] d . This ensures that f (x) = g(x) for x ∈ R d ∖ [-K,K] d .
Combining this with the fact that ´Rd gdP = ´Rd gdQ by the equality of characteristic functions reveals that ˆRd f dP -

ˆRd f dQ ⩽ ˆRd f -g dP + ˆRd f -g dQ ⩽ ε.
Letting ε tend to zero establishes (A.63) and completes the proof. ∎ Theorem A.23. If two probability measures P and Q on R d ⩾0 have the same Laplace transform, L P = L Q , then they must be equal, P = Q.

Proof. Up to replacing the function (A.64) by the function g m ∶ [-K,K] d → R defined by g m (x) = e m⋅x , the proof is identical to that of Theorem A. [START_REF] Arora | On nonapproximability for quadratic programs[END_REF].

∎ We now provide an example that shows how the ideas developed in this section can be used to establish the weak convergence of a sequence of probability measures on the positive half-space by looking only at their Laplace transforms. Proof. The direct implication is immediate. To show the converse implication, we proceed in two steps. First we argue that (P n ) n⩾1 is uniformly tight using the Prokhorov theorem, and then we prove that P is its only subsequential limit using Theorem A.23. The result then follows from Lemma A.19.

Step 1:

(P n ) n⩾1 is uniformly tight. Fix ε > 0 and M > 0. Denote by K M ∶= [0,M] d the box of side-length M in R d ⩾0
, and for a choice of λ * > 0 to be decided shortly, consider the constant vector λ ∶= (λ * ,...,λ * ) ∈ R d ⩾0 .

Since L P is continuous and L P (0) = 1, we can fix λ * > 0 small enough such that for every n sufficiently large, ˆRd

⩾0 e -λ ⋅x dP n (x) = L Pn (λ ) ⩾ 1 - ε 2 .
It follows that for every n sufficiently large,

1 - ε 2 ⩽ P n (K M ) + ˆRd ⩾0 ∖K M e -λ ⋅x dP n (x) ⩽ P n (K M ) + e -Mλ * P n (K c M ).
Rearranging reveals that

P n (K c M ) ⩽ ε 2(1 -e -Mλ * ) , so choosing M = log (2)
λ * ensures that P n (K c M ) ⩽ ε. This shows that (P n ) n⩾1 is uniformly tight, so any subsequence of (P n ) n⩾1 admits a weakly convergent subsequence by the Prokhorov theorem.

Step 2: (P n ) n⩾1 admits a unique subsequential limit. Let (P n(k) ) k⩾1 be a subsequence of (P n ) n⩾1 converging weakly to some probability measure Q on R d ⩾0 . On the one hand, the definition of weak convergence implies that for every λ ∈ R d ⩾0 , lim k→+∞ ˆS e -λ ⋅x dP n(k) = ˆS e -λ ⋅x dQ.

On the other hand, by assumption, for every λ ∈ R d ⩾0 , lim k→+∞ ˆS e -λ ⋅x dP n(k) = ˆS e -λ ⋅x dP. and denote by λ 1 ⩽ λ 2 its eigenvalues,

λ 1 = e β cosh(h) -e 2β cosh 2 (h) -2sinh(2β ) 1 2 , λ 2 = e β cosh(h) + e 2β cosh 2 (h) -2sinh(2β ) 1 2
.

Notice that e 2β cosh 2 (h) ⩾ e 2β ⩾ 2sinh(2β ), so the square root is well-defined. Given a spin configuration σ ∈ {±1} N , let σ N+1 = σ 1 in such a way that

F N (β ,h) = 1 N log σ ∈{±1} N N i=1 M σ i σ i+1 = 1 N log σ 1 ∈Σ 1 M N σ 1 σ 1 = 1 N logtr(M N ) = 1 N log λ N 1 + λ N 2 .
Since λ 2 > 0 and λ 2 > λ 1 , the limit free energy is given by

f (β ,h) ∶= lim N→+∞ F N (β ,h) = logλ 2 + lim N→+∞ 1 N log ⎛ ⎝ 1 + λ 1 λ 2 N ⎞ ⎠ = logλ 2 .
The quantity ∂ h F N (β ,h) is the mean magnetization (1.23), and is therefore bounded uniformly by one. It follows by smoothness of f in h, Propositions 2.11 and 2.15, and Exercise 2.6 that for every h ∈ R, the limit (1.25) of the mean magnetization is given by 1 2 , and is therefore a continuous function of h.

m(β ,h) ∶= lim N→+∞ ∂ h F N (β ,h) = ∂ h f (β ,h) = e β sinh(h) (e 2β cosh 2 (h) -2sinh(2β ))
Exercise 1.3. We treat each question separately.

(i) The Hamiltonian (1.28) may be written as

H N (σ ) = i, j∈B N+1 i∼ j 1 {σ i σ j =1} - i, j∈B N+1 i∼ j 1 {σ i σ j =-1} = {i, j} ∈ B 2 N+1 i ∼ j -2 i, j∈B N+1 i∼ j 1 {σ i σ j =-1} .
The final sum counts the number of pairs of nearest neighbours (i, j) ∈ B 2 N+1 with σ i σ j = -1. This corresponds to the number of edges Γ(σ ) in Γ(σ ). It follows that H N (σ ) + 2 Γ(σ ) gives the total number of nearest neighbour pairs in the lattice B 2 N+1 , so it depends only on the geometry of the lattice and not the specific configuration σ ∈ {±1} B N .

(ii) Given a configuration σ ∈ {±1} B N with σ 0 = -1, let j(σ ) ∶= min{ j ∈ {1,...,N + 1} σ (0, j) = 1} be the coordinate of the first spin to right of σ 0 that is unaligned with σ 0 . Notice that such a spin must exist as all spins in B N+1 ∖ B N are unaligned with σ 0 . Since the contours in Γ(σ ) delimit regions in which the sign of the spins is constant, there must be a contour which separates the origin from the point (0, j(σ )) and therefore surrounds the origin.

(iii) Given a spin configuration σ ∈ {±1} B N , consider a partition P(σ ) of the set of contours Γ(σ ),

Γ(σ ) = ⋃ η∈P(σ ) η.
Since H N (σ ) + 2 Γ(σ ) does not depend on the configuration σ by (i), we have

⟨1 {γ⊆Γ(σ )} ⟩ = ∑ σ ∈{±1} B N 1 {γ⊆Γ(σ )} exp(-2β Γ(σ ) ) ∑ σ ∈{±1} B N exp(-2β Γ(σ ) ) = exp(-2β γ ) ⋅ ∑ σ ∈{±1} B N 1 {γ⊆Γ(σ )} ∏ η∈P(σ )∖γ exp(-2β η ) ∑ σ ∈{±1} B N ∏ η∈P(σ ) exp(-2β η ) .
It therefore suffices to prove that

σ ∈{±1} B N 1 {γ⊆Γ(σ )} η∈P(σ )∖γ exp(-2β η ) ⩽ σ ∈{±1} B N η∈P(σ ) exp(-2β η ). (S.2)
Denote by Σ γ the set of configurations σ ∈ {±1} B N with γ ⊆ Γ(σ ), and define the flipping map F ∶ Σ γ → {±1} B N by

F(σ ) i ∶= ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ -σ i if i ∈ int(γ), σ i otherwise.
Observe that F is injective and satisfies Γ(F(σ )) = P(σ ) ∖ γ. It follows that the left side of (S.2) is given by

σ ∈{±1} B N 1 {γ∈Γ(σ )} η∈P(F(σ )) exp(-2β η )
which is in turn equal to

σ ∈{±1} B N 1 {σ ⊆F(Σγ )} η∈P(σ ) exp(-2β η ) ⩽ σ ∈{±1} B N η∈P(σ ) exp(-2β η ).
This establishes (S.2).

(iv) To bound ⟨σ 0 ⟩ from below away from zero we will use that

⟨σ 0 ⟩ = ⟨1 {σ 0 =1} ⟩ -⟨1 {σ 0 =-1} ⟩ = 1 -2⟨1 {σ 0 =-1} ⟩, (S.3)
and instead bound ⟨1 {σ 0 =-1} ⟩ from above away from 1 2. Denote by Γ 0 the set of all contours surrounding the origin, and write Γ k 0 for the set of contours in Γ 0 of length k. Combining (ii) with the union bound and (iii) reveals that

⟨1 {σ 0 =-1} ⟩ ⩽ γ∈Γ 0 ⟨1 {γ⊆Γ(σ )} ⟩ ⩽ γ∈Γ 0 exp(-2β γ ) = +∞ k=1 Γ k 0 exp(-2β k).
To bound the size of Γ k 0 , observe that any path enclosing the origin must have length at least 4. Moreover, it must cross the point (0, j) for some j ∈ {1,...,k 2}, and at each of its k nodes it can go in at most 4 directions. This means that k must be larger than 4 for Γ k 0 not to be empty, and that Γ k 0 can contain at most k4 k edges. It follows that

⟨1 {σ 0 =-1} ⟩ ⩽ +∞ k=4 k 4e -2β k .
Taking β > 3 2 log2 and invoking the dominated convergence theorem shows that the right side of this expression tends to zero as β → +∞. In particular, for any δ > 0 there exists β large enough with limsup N→+∞ ⟨1 {σ 0 =-1} ⟩ ⩽ δ 2 . Taking δ < 1 and substituting this into (S.3) gives β large enough with liminf N→+∞ ⟨σ 0 ⟩ > 0 as required.

(v) The symmetry between spin sites when h = 0 implies that the mean magnetization (1.23) simplifies to

m N (β ,0) = ⟨ 1 B 2 i∈B 2 σ i ⟩ = ⟨σ 0 ⟩.
It follows by (iv) that for β large enough, liminf N→+∞ m N (β ,0) > 0 which means that the two-dimensional Ising model with no magnetic field and with +1 boundary condition has a non-zero asymptotic mean magnetization. This suggests that it has a "memory" of the slight tilt it has been exposed to via the boundary condition, and therefore that it exhibits ferromagnetic behaviour. Although we will not pursue this further, one can show by subadditivity arguments that the limit free energy associated with this model is well-defined and does not depend on the boundary condition we choose; and use Proposition 2.15 to assert that any subsequential limit of m N (β ,0) must belong to the subdifferential (in h) of this limit free energy. By symmetry, we conclude that for β > 0 large enough, the limit free energy of the two-dimensional Ising model is not differentiable in h at h = 0.

S.2 Convex analysis and large deviation principles

Exercise 2.1. Fix α ∈ (0,1) and x,y

∈ C 2 -C 1 . Let x 1 ,y 1 ∈ C 1 and x 2 ,y 2 ∈ C 2 be such that x = x 2 -x 1 and y = y 2 -y 1 . Observe that αx + (1 -α)y = αx 2 + (1 -α)y 2 -αx 1 + (1 -α)y 1 ∈ C 2 -C 1 . (S.4)
This establishes the convexity of C 2 -C 1 .

Exercise 2.2. It is clear that int(C) ⊆ int(C). To prove the converse containment, fix z ∈ int(C) and x ∈ int(C). Let γ > 0 be small enough so that y ∶= z + γ(zx) ∈ C, and observe that z = (1α)x + αy

for α = 1 γ+1 ∈ (0,1). If ε > 0 is small enough so B ε (x) ⊆ C, then B (1-α)ε (z) ⊆ C
. This shows that z ∈ int(C), and therefore int(C) = int(C). The equality of the boundaries follows from the fact that ∂ This completes the proof.

A = A ∖ int(A) for every set A ⊆ R d . Exercise 2.3. Since int(C) ⊆ C and C is closed, we have int(C) ⊆ C. To prove the converse inclusion, fix x ∈ C and x ′ ∈ int(C). Let ε > 0 be small enough so B ε (x ′ ) ⊆ C. By convexity of C, for all t ∈ [0,1] and y ∈ B ε (x ′ ), we have (1 -t)x + ty ∈ C. This means that, for all t ∈ [0,1], B tε (1 -t)x +tx ′ ⊆ C,
Exercise 2.4. We say that points x 1 ,..

.,x k ∈ R d are affinely independent if k i=1 α i x i = 0 with α i ∈ R and k i=1 α i = 0 implies that α 1 = ⋯ = α k = 0.
This is equivalent to the linear independence of the vectors x 2x 1 ,...,x kx 1 . In particular, if x 1 ,...,x k are affinely independent in R d , then k ⩽ d + 1. It therefore suffices to prove that any point x ∈ conv(A) can be written as a convex combination of affinely independent points in A. Since x ∈ conv(A), it may be written as

x = k i=1 λ i x i for some x i ∈ A, λ i ⩾ 0 with k i=1 λ i = 1 and k ⩾ 1.
(S.5)

We take k minimal among all such representations. Suppose for the sake of contradiction that x 1 ,...,x k are not affinely independent. This means that there exist α 1 ,...,α k ∈ R, not all zero, with k i=1 α i x i = 0 and

k i=1 α i = 0.
Denote by I the set of indices i ∈ {1,...,k} such that α i > 0. Since the set I is not empty, we can choose m ∈ I to be such that

λ m α m = inf i∈I λ i α i .
In the representation

x = k i=1 λ i - λ m α m α i x i ,
the coefficients sum to one and are all non-negative by choice of m. Since the m'th coefficient is zero, this contradicts the minimality of k among all representations of the form (S.5). This shows that x 1 ,...,x k are affinely independent, and completes the proof.

Exercise 2.5. Let (x k ) k⩾1 ⊆ conv(A) be a sequence in the convex hull of A. By Exercise 2.4, for each k ⩾ 1, there exist x k 1 ,...,x k d+1 ∈ A and α k 1 ,...,α k d+1 ∈ [0,1] with

x k = d+1 i=1 α k i x k i and d+1 i=1 α k i = 1.
Since A and [0,1] are compact, there are subsequences of (x k i ) k∈K and (α k i ) k∈K with the same index set K that converge to some x i and α i , respectively, for every i ∈ {1,...,d + 1}. Since A is compact, each x i belongs to A. It follows that the subsequence (x k ) k∈K converges to the point

x = d+1 i=1 α i x i ∈ conv(A).
This completes the proof.

Exercise 2.6. Fix λ , µ ∈ R d and α ∈ (0,1). By Hölder's inequality with conjugate exponents p = α -1 and q = (1α) -1 , we have

expψ αλ + (1 -α)µ = Eexp(αλ ⋅ X)exp (1 -α)µ ⋅ X ⩽ Eexp(λ ⋅ X) α Eexp(µ ⋅ X) 1-α = expψ(λ ) α expψ(µ) 1-α .
Taking logarithms completes the proof.

Exercise 2.7. First suppose that f α is convex for each α ∈ I, and fix x,y ∈ R d as well as λ ∈ (0,1). For every α ∈ I, we have

f α λ x + (1 -λ )y ⩽ λ f α (x) + (1 -λ ) f α (y).
Taking the supremum over α ∈ I gives sup α∈I

f α λ x + (1 -λ )y ⩽ λ sup α∈I f α (x) + (1 -λ )sup α∈I f α (y)
which establishes the convexity of sup α∈I f α . Now, suppose instead that f α is lower semi-continuous for every α ∈ I. We observe that

(x,λ ) ∈ R d × R sup α∈I f α (x) ⩽ λ = ⋂ α∈I (x,λ ) ∈ R d × R f α (x) ⩽ λ .
Since the intersection of closed sets is closed, this shows that the epigraph of sup α∈I f α is closed, and thus that sup α∈I f α is lower semi-continuous. We note in passing that the first part of the exercise can also be proved by reasoning about the convexity of the epigraphs.

Exercise 2.8. Let x,x ′ ∈ R d and α ∈ (0,1). We want to verify that

g(αx + (1 -α)x ′ ) ⩽ αg(x) + (1 -α)g(x ′ ). (S.6)
Without loss of generality, we may assume that g(x) and g(x ′ ) are finite. For any given ε > 0, one can find y,y ′ ∈ R k such that f (x,y) ⩽ g(x)+ε and f (x ′ ,y ′ ) ⩽ g(x ′ )+ε.

By the definition of g as an infimum and the convexity of f , we have

g(αx + (1 -α)x ′ ) ⩽ f (αx + (1 -α)x ′ ,αy + (1 -α)y ′ ) ⩽ α f (x,y) + (1 -α) f (x ′ ,y ′ ).
Letting ε tend to zero, we obtain the result.

Exercise 2.9. We treat each question separately.

(i) Since f is a supremum of lower semi-continuous functions, it is lower semicontinuous, by Exercise 2.7.

(ii) Denote h(x) ∶= liminf y→x f (y), and let g be lower semi-continuous with g ⩾ f . By the lower semi-continuity of g and the inequality g ⩽ f , we have

g(x) ⩽ liminf y→x g(y) ⩽ liminf y→x f (x) = h(x).
Taking the supremum over g, we obtain that f ⩽ h. To show the converse inequality, it suffices to verify that h is lower semi-continuous, since we clearly have h ⩽ f . Let (x n ) n⩾1 be a sequence converging to x ∈ R d , and ε > 0. For each n ⩾ 1, we can find a point x ′ n with x ′ nx n ⩽ 1 n and

f (x ′ n ) ⩽ h(x n ) + ε.
In particular, the sequence (x ′ n ) n⩾1 converges to x, and

h(x) = liminf y→x f (y) ⩽ liminf n→+∞ f (x ′ n ) ⩽ liminf n→+∞ h(x n ) + ε.
Since ε > 0 is arbitrary, this shows that h is lower semi-continuous, and thus completes the argument.

(iii) Let x,y ∈ R d and α ∈ (0,1). We aim to show that

f (αx + (1 -α)x ′ ) ⩽ α f (x) + (1 -α) f (x ′ ).
Without loss of generality, we may assume that f (x) and f (x ′ ) are finite. We fix ε > 0. For each r > 0, one can find

x r ,x ′ r ∈ R d with x r -x + x ′ r -x ′ ⩽ r such that f (x r ) ⩽ f (x) + ε and f (x ′ r ) ⩽ f (x ′ ) + ε.
In particular, we have

αx r + (1 -α)x ′ r -αx -(1 -α)x ′ r ⩽ r and f (αx r + (1 -α)x ′ r ) ⩽ α f (x) + (1 -α) f (x ′ ) + ε.
Since this is valid for every r > 0, we deduce that

f (αx + (1 -α)x ′ ) ⩽ α f (x) + (1 -α) f (x ′ ) + ε.
Since ε > 0 was arbitrary, the proof is complete.

Exercise 2.10. We treat each question separately.

(i) Since f * is the supremum of affine functions, this is immediate from Exercise 2.7.

(ii) The definition of the convex dual implies that f (x)+ f * (x) ⩾ λ ⋅x for every x,λ ∈ R d .

It follows that for every

x,λ ∈ R d , x ⋅ λ -f * (λ ) ⩽ f (x) + f * (λ ) -f * (λ ) = f (x).
Taking the supremum over

x ∈ R d shows that f * * ⩽ f . (iii) If f ⩽ g and x,λ ∈ R d , then x ⋅ λ -g(x) ⩽ x ⋅ λ -f (x) ⩽ f * (λ ).
Taking the supremum over x ∈ R d shows that g * ⩽ f * and completes the proof.

Exercise 2.11. Fix λ ∈ R d , and observe that by the Cauchy-Schwarz inequality,

f * (λ ) = 1 2 sup x∈R d 2λ ⋅ x -x 2 ⩽ 1 2 sup x∈R d 2 λ x -x 2 .
The parabola y ↦ 2 λ yy 2 has a unique maximum at y = λ , so f * (λ ) ⩽ f (λ ). The matching lower bound is obtained by choosing x = λ .

Exercise 2.12. Fix x ∈ R d with x > L. By definition of the convex dual and Lipschitz continuity of f ,

f * (x) ⩾ sup p∈R d p ⋅ x -L p -f (0) ⩾ limsup α→+∞ α x x -L -f (0) = +∞.
This completes the proof.

Exercise 2.13. We use the notation f * ∶= ( f ) * . Since f ⩽ f , part (iii) of Exercise 2.10 yields that f * ⩾ f . To show the converse bound, we fix λ ∈ R d , ε > 0, and

x ∈ R d such that f * (λ ) ⩽ λ ⋅ x -f (x) -ε.
By Exercise 2.9, we can find x ′ ∈ R d such that x ′x ⩽ ε and f (x) ⩾ f (x)ε.

Combining this with the previous display, we obtain that

f * (λ ) ⩾ λ ⋅ x ′ -f ′ (x) ⩾ λ ⋅ x -f (x) -( λ + 1)ε ⩾ f * (λ ) -( λ + 2)ε.
Since ε > 0 is arbitrary, this completes the proof that f * = f * . By Exercise 2.9, the function f is convex and lower semi-continuous, so the Fenchel-Moreau theorem ensures that f * * = f . Conjugating the relation f * = f * therefore yields that f * * = f , as announced.

Exercise 2.14. The definition of the polar K ○ ensures that for each fixed x ∈ K and all v ∈ K ○ , we have x ⋅ v ⩽ 0. This means that K ⊆ K ○○ . To prove the converse inclusion, suppose for the sake of contradiction that there exists x ∈ K ○○ ∖ K. Since K is a closed convex set, the supporting hyperplane theorem (Theorem 2.2) gives a non-zero v ∈ R d with sup{v ⋅ y y ∈ K} < v ⋅ x.

If there were y ∈ K with v ⋅ y > 0, then we would have v ⋅ λ y < v ⋅ x for all λ > 0 which would lead to a contradiction upon letting λ tend to infinity. This means that v ∈ K ○ . Observe also that 0 = lim λ ↘0 λ y ∈ K, where y is any point in K, so v ⋅ x > 0. Since v ∈ K ○ , this contradicts the fact the x ∈ K ○○ and completes the proof.

Exercise 2.15. We treat each question separately.

(i) Fix x ∈ int(C) and n ∈ n C (x). For every ε > 0 small enough, we have x + εn ∈ C, and therefore

ε n 2 = n ⋅ (x + εn -x) ⩽ 0.
This implies that n C (x) ⊆ {0}. The converse inclusion is trivial. To show that T C (x) = R d , fix v ∈ R d , and let ε > 0 be small enough so

x + εv ∈ C. Observe that v = ε -1 x + εv -x) ∈ T C (x),
as required.

(ii) By Exercise 2.14, it suffices to show that n C (x) = T C (x) ○ . On the one hand, if v ∈ n C (x), then for all x ′ ∈ C and λ ⩾ 0, we have v⋅λ (x ′ -x) ⩽ 0. Since this inequality is preserved under limits, it follows that v ⋅ y ⩽ 0 for all y ∈ T C (x), and therefore that n

C (x) ⊆ T C (x) ○ . On the other hand, if v ∈ T C (x) ○ , then for all x ′ ∈ C, we have v ⋅ (x ′ -x) ⩽ 0 since x ′ -x ∈ T C (x). This shows that T C (x) ○ ⊆ n C (x). (iii) If v ∈ T C (x), there exist sequences (x ′ i ) i⩾1 ⊆ C and (λ i ) i⩾1 ⊆ R ⩾0 with λ i (x ′ i -x) → v. Letting t i ∶= min(2 -i ,λ -1 i ) and x i ∶= x +t i λ i (x ′ i -x) gives a sequence (t i ) i⩾1 ⊆ R >0 decreasing to 0 and a sequence (x i ) i⩾1 ⊆ R d con- verging to x with t -1 i (x i -x) → v. Moreover, since t i λ i ∈ [0,1],
we have that x i ∈ C for every i ⩾ 1. This completes the proof of the direct implication. The converse implication is immediate. We would like to show that (y n ) n⩾1 is Cauchy. Fix n,m ⩾ 1, and observe that

y n -y m 2 = y n -x 2 + y m -x 2 -2(y n -x) ⋅ (y m -x).
Exercise 2.18. Fix a step function of the form φ ∶= ∑

n i=1 φ i 1 [a i-1 ,a i ) for some (φ i ) i⩽n ⊆ R and non-decreasing (a i ) 0⩽i⩽n ⊆ [a,b]. If we denote by L the Lipschitz constant of F, then T (φ ) ⩽ n i=1 φ i F(a i ) -F(a i-1 ) ⩽ L n i=1 φ i (a i -a i-1 ) = L ˆb a φ (
(x) -F(a) = T (g) = ´x a f (t)dt. It follows that for any h ∈ R ∖ {0} with x + h ∈ [a,b], F(x + h) -F(x) h -f (x) ⩽ 1 h ˆx+ h x-h f (t) -f (x) dt.
Invoking the Lebesgue differentiation theorem established in Theorem A.16 completes the proof.

Exercise 2.19. We treat each question separately.

(i) The convex dual of x ↦ I {x⩾0} is the mapping

z ↦ sup x∈R d x ⋅ z -I {x⩾0} = I {z⩽0} .
(ii) We start by observing that

I {x⩾0} = sup z∈R d x ⋅ z -I {z⩽0} , (S.7)
as can be checked directly or by appealing to the previous step and the Fenchel-Moreau theorem. We can also write

I {Ax⩾c} = sup y∈R k (Ax -c) ⋅ y -I {y⩽0} = sup y∈R k x ⋅ A * y -c ⋅ y -I {y⩽0} .
Summing the two previous displays yields that

φ (x) = sup z∈R d , y∈R k x ⋅ z -I {z⩽0} + x ⋅ A * y -c ⋅ y -I {y⩽0} = sup z∈R d , y∈R k x ⋅ z -c ⋅ y -I {z⩽A * y} -I {y⩽0} = sup z∈R d (x ⋅ z -ψ(z)).
To go from the first to the second line, we took the supremum over y first and then replaced z by z -A * y. We obtained the announced identity.

(iii) The infimum on the left side of (2.37) can be rewritten as

-sup x∈R d -b ⋅ x -φ (x) = -φ * (-b),
and by the previous step, this is -ψ * * (-b), provided that ψ * * is well-defined. We also observe that

-ψ(-b) = -inf y∈R k c ⋅ y + I {y⩽0} + I {A * y⩾-b} = sup y∈R k -c ⋅ y -I {-y⩾0} -I {-A * y⩽b} = sup y∈R k c ⋅ y -I {y⩾0} -I {A * y⩽b} ,
using the change of variables y ↦ -y. This is the optimization problem on the right side of (2.37), so our goal is to show that ψ

* * (-b) = ψ(-b). Since ψ(z) = inf y∈R k -c ⋅ y + I {-y∈Kz} ,
the assumption on K z ensures that ψ is finite in a neighbourhood of -b. Arguing as in Exercise 2.8, we see that ψ must be convex, with the caveat that it might a priori take the value -∞. But by Remark 2.7, the fact that ψ is finite on some open set rules out this possibility, so ψ takes values in R ∪ {+∞}. Let ψ be the lower semi-continuous envelope of ψ, as defined in Exercise 2.9. By Exercise 2.13, we have that ψ * * = ψ. By Proposition 2.9, the function ψ is continuous at -b, and thus part (ii) of Exercise 2.9 ensures that ψ(-b) = ψ(-b). This completes the proof.

Exercise 2.20. To alleviate notation, we introduce the set

C ∶= n dom f (x) + conv S(x) .
The proof proceeds in three steps. First, we show that C ⊆ ∂ f (x), then we prove that C is closed and convex, and finally we show that ∂ f (x) ⊆ C.

Step 1: C ⊆ ∂ f (x). By Theorem 2.13, Proposition 2.14 and the continuity of f on dom f , we have S(x) ⊆ ∂ f (x). Together with the convexity of ∂ f (x), this shows that conv(S(x)) ⊆ ∂ f (x). We now prove that ∂

f (x) + n dom f (x) ⊆ ∂ f (x). Fix p ∈ ∂ f (x)
and n ∈ n dom f (x). For every x ′ ∈ dom f , we have

f (x ′ ) ⩾ f (x) + p ⋅ (x ′ -x) ⩾ f (x) + (p + n) ⋅ (x ′ -x),
where the first inequality uses that p ∈ ∂ f (x), and the second uses that n ∈ n dom f (x). This shows that p + n ∈ ∂ f (x), and therefore that ∂

f (x) + n dom f (x) ⊆ ∂ f (x). In particular, we have C ⊆ ∂ f (x).
Step 2: C is closed and convex. By Exercise 2.1, the set C is convex. To show that it is also closed, fix a sequence (p i ) i⩾1 ⊆ C converging to some point p ∈ R d .

For each i ⩾ 1, let n i ∈ n dom f (x) and s i ∈ conv(S(x)) be such that p i = n i + s i . The Lipschitz continuity of f on dom f ensures that the sequence (s i ) i⩾1 is bounded. Up to passing to a subsequence, we may therefore suppose that it converges to some point s ∈ conv(S(x)) = conv(S(x)). We have implicitly used Exercise 2.5 to say that conv(S(x)) is compact, and therefore closed, as the convex hull of the compact set S(x). The sequence (n i ) i⩾1 also converges along this subsequence to some point n ∈ n dom f (x). We have implicitly used that the normal cone is closed. Taking the limit along this subsequence shows that p = n + s ∈ C, and therefore that C is closed.

Step 3: ∂ f (x) ⊆ C. Suppose for the sake of contradiction that there exists p ∈ ∂ f (x) ∖C. The supporting hyperplane theorem (Theorem 2.2) then gives a non-zero vector

v ∈ R d with sup v ⋅ (n + s) n ∈ n dom f (x) and s ∈ conv(S(x)) < v ⋅ p. (S.8)
Recall the definition of the polar in Exercise 2.14, and observe that v ∈ n dom f (x) ○ . Indeed, if this were not the case, there would exist n ∈ n dom f (x) with v ⋅ n > 0. Given y 0 ∈ conv(S(x)), which exists by Rademacher's theorem (Theorem 2.10), we would then have, for all λ > 0,

λ v ⋅ n + v ⋅ y 0 = (λ n + y 0 ) ⋅ v < v ⋅ p.
Letting λ tend to infinity would give a contradiction. It follows by Exercise 2.15 that v ∈ n dom f (x) ○ = T dom f (x), where T dom f (x) denotes the tangent cone to dom f at x. Combining Exercises 2.15 and 2.3 with Rademacher's theorem gives a sequence (x i ) i⩾1 ⊆ int(dom f ) converging to x and a sequence (t i ) i⩾1 ⊆ R >0 decreasing to 0 such that ∇ f (x i ) exists for all i ⩾ 1, and the sequence (v i ) i⩾1 defined by v i ∶= t -1 i (x ix) converges to v. By Lipschitz continuity of f on dom f , the sequence (∇ f (x i )) i⩾1 is bounded, so, up to passing to a subsequence, assume without loss of generality that it converges to some c ∈ C. Since ∇ f (x + t i v i ) ∈ ∂ f (x + t i v i ) by Theorem 2.13 and p ∈ ∂ f (x) by assumption, for every i ⩾ 1, we have

-∇ f (x +t i v i ) ⋅ v i ⩽ f (x) -f (x +t i v i ) t i ⩽ -v i ⋅ p.
Letting i tend to infinity shows that -c ⋅ v ⩽ -v ⋅ p, and therefore c ⋅ v ⩾ v ⋅ p. This contradicts (S.

1 N logP S N ∈ A ∩ [p,+∞) ⩽ limsup N→+∞ logP{S N ⩾ x} = -I(x) ⩽ -inf x∈A I(x).
A similar argument shows that limsup

N→+∞ 1 N P S N ∈ A ∩ (-∞, p] ⩽ -inf x∈A I(x).
Combining these two bounds establishes the upper bound in (2.56). To prove the lower bound, fix x ∈ int(A), and let ε > 0 be such that [xε,x + ε) ⊆ A. Observe that

P{S N ∈ A} ⩾ P{S N ∈ [x -ε,x + ε)} = P{S N ⩾ x -ε} -P{S N ⩾ x + ε} If x > p and ε > 0 is small enough so x -ε > p, then (2.54) implies that lim N→+∞ 1 N logP{S N ⩾ x-ε} = -I(x-ε) and lim N→+∞ 1 N logP{S N ⩾ x+ε} = -I(x+ε).
Since I is increasing on the interval (p,1), we have -I(xε) ⩾ -I(x + ε), and therefore liminf

N→+∞ 1 N logP{S N ∈ A} ⩾ -I(x -ε).
Letting ε tend to zero and using the continuity of I gives the lower bound

liminf N→+∞ 1 N logP{S N ∈ A} ⩾ -I(x).
An identical bound holds if x ⩽ p. Taking the supremum over all x ∈ int(A) establishes the lower bound in the string of inequalities (2.56) and completes the proof.

Exercise 2.22. The log-Laplace transform of a Bernoulli random variable X is

ψ(λ ) = logEexp(λ X) = log pe λ + (1 -p) .
It follows by definition of the convex dual that

ψ * (x) = sup λ ∈R λ x -log pe λ + (1 -p) . (S.9)
Elementary calculus reveals that this supremum is achieved for λ ∈ R with

x = pe λ pe λ + (1 -p)
.

Rearranging shows that

e λ = x p ⋅ 1 -p 1 -x .
Substituting this into (S.9) gives

ψ * (x) = xlog x p -xlog 1 -x 1 -p -log x ⋅ 1 -p 1 -x + (1 -p) = xlog x p -xlog 1 -x 1 -p -log 1 -p 1 -x = xlog x p + (1 -x)log 1 -x 1 -p as required.
Exercise 2.23. We recall that by Proposition 2.9, the function f is continuous; we also recall that

f * (λ ) = sup x∈R d (λ ⋅ x -f (x)). (S.10) We fix λ ∈ R d .
The assumption of (2.119) and the continuity of f guarantee that for some constant C < +∞, we have for every

x ∈ R d that f (x) ⩾ ( λ + 2) x -C.
Plugging this estimate into (S.10) and using the Cauchy-Schwarz inequality yields that, for every λ

′ ∈ R d , λ ′ ⋅ x -f (x) ⩽ C -( λ + 2 -λ ′ ) x .
In particular, we have that f * (λ ) is finite. The bound above also implies that for every λ ′ ∈ R d with λ ′ ⩽ λ + 1, we may as well restrict the supremum in the definition of f * (λ ′ ) in (S.10) to values of x that range in a fixed compact set K. Since f is also continuous, the supremum in (S.10) is achieved for those values of λ ′ . Since the mapping (λ ,x) ↦ λ ⋅xf (x) is continuous and continuously differentiable with respect to its first variable, we are thus in position to appeal to the envelope theorem (Theorem 2.21). The claim will follow once we show that the supremum in (S.10) is achieved at a unique point x * (λ ), and that λ ↦ x * (λ ) is continuous. Suppose that there are two distinct optimizers a ≠ b ∈ R d for (S.10). We have in particular that λ

⋅ a -f (a) = λ ⋅ b -f (b).
The strict convexity assumption then implies that

1 2 λ ⋅ a -f (a) + λ ⋅ b -f (b) < λ ⋅ a + b 2 -f a + b 2 .
Since this contradicts the optimality assumption on a and b, this completes the proof of uniqueness. Denoting by x * (λ ) the optimum, we now show that the mapping λ ↦ x * (λ ) is continuous. Let (λ n ) n⩾1 be a sequence of points in R d converging to λ . Up to extraction of a subsequence, we may assume that x * (λ n ) converges to some a ∈ R d . Passing to the limit in the identity

f * (λ n ) = λ n ⋅ x * (λ n ) -f (x * (λ n )),
we conclude that a must be an optimizer in (S.10). By uniqueness of the optimizer, we have that a = x * (λ ), as desired.

Exercise 2.24. We treat each question separately.

(i) The envelope theorem (Theorem 2.21) implies that ∂ t f (t,0) = m 0 (t)

∂ t f (t n ,0) = lim n→+∞ m 0 (t n ) 2 = m 0 (t) 2 = ∂ t f (t,0).
This establishes the continuity of t ↦ ∂ t f (t,0).

(ii) Fix h > 0, and observe that by Proposition 2.22,

f (t c ,h) = t c m + h (t c ) 2 + hm + h (t c )- 1 + m + h (t c ) 2 log(1 + m + h (t c )) - 1 -m + h (t c ) 2 log(1 -m + h (t c )).
Remembering that t c = 1 2 and Taylor expanding the logarithm reveals that

f (t c ,h) = 1 2 m + h (t c ) 2 + hm + h (t c ) - 1 2 m + h (t c ) 2 + O m + h (t c ) 3 = (1 + o(1))hm + h (t c ),
where the last equality uses the fact that m h (t c ) → m 0 (t c ) = 0 as h decreases to zero. On the other hand, Taylor expanding the hyperbolic tangent and using the fixed point equation (2.115) implies that

m h (t c ) = m h (t c ) + h - (m h (t c ) + h) 3 3 + O (m h (t c ) + h) 5 = m h (t c ) - m h (t c ) 3 3 + h(1 + o(1)). Rearranging gives m h (t c ) = h 1 3 (1+o(1)). It follows that f (t c ,h) = h 1+ 1 3 (1+o(1)), and therefore lim h↘0 log f (t c ,h) logh = 1 + 1 3 .
An identical argument for h < 0 allows us to conclude that δ = 3.

(iii) We can and do restrict our attention to the case of t > t c . Taylor expanding the hyperbolic tangent and using the fixed point equation (2.115) implies that, as t tends to t c ,

m 0 (t) = 2tm 0 (t) - 8t 3 m 0 (t) 3 3 + O (tm 0 (t)) 3 = 2tm 0 (t) -(1 + o(1)) 8t 3 m 0 (t) 3 3
where the last equality uses the fact that m 0 (t) → m 0 (t c ) = 0 as t tends to t c . Rearranging shows that

S.3 Hamilton-Jacobi equations

Exercise 3.1. Fix r > 0 sufficiently small that x is a strict maximum of f in B r (x). By continuity of f N and compactness of B r (x), let x N ∈ B r (x) be such that f N (x N ) ⩾ f N (y) (S.11)

for every y ∈ B r (x). Since (x N ) N⩾1 stays in a compact set, it admits a subsequence converging to some x * ∈ B r (x). Letting N tend to infinity in (S.11) and leveraging the local uniform convergence of f N to f reveals that f (x * ) ⩾ f (y) for every y ∈ B r (x). Since x is a strict maximum we must have x * = x, so the only limit point of (x N ) N⩾1 is x. This implies that (x N ) N⩾1 converges to x as required. In particular, for N sufficiently large x N is in the open ball B r (x), and is thus a local maximum of f N .

Exercise 3.2. It suffices to prove that the definition of subsolution with "strict local maximum" implies the one with "local maximum". Let f ∶ R ⩾0 × R d → R be a subsolution for the definition with "strict local maximum", and fix a smooth function φ ∈ C ∞ (R >0 × R d ;R) with the property that fφ has a local maximum at

(t * ,x * ) ∈ R >0 × R d . Consider the smooth function η ∶ R >0 × R d → R defined by η(t,x) ∶= φ (t,x) + ε (t -t * ) 2 + x -x * 2 .
Observe that in a small enough neighbourhood of (t * ,x * ), we have We will now verify that (t * ,x * ) is a global maximum of ( fη). On the one hand, for any (t,x) ∈ B r 2 (t * ,x * ), ( fη)(t,x) ⩽ ( fφ )(t,x) ⩽ ( fφ )(t * ,x * ) = ( fη)(t * ,x * ).

( f -η)(t,
On the other hand, for any (t,x) ∈ B r (t This shows that (t * ,x * ) is indeed a global maximum of fη. Since f is a subsolution for the definition with "global maximum", it follows that ∂ t φ -H(∇φ ) (t * ,x * ) = ∂ t η -H(∇η) (t * ,x * ) ⩽ 0 which is the required subsolution criterion for the definition with "local maximum". Exercise 3.4. Given a function φ ∈ C 1 (R >0 × R d ;R), denote by P(φ ) the property that for every (t * ,x * ) ∈ R >0 × R d for which fφ has a local maximum at (t * ,x * ), we have ∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0.

It is clear that whenever P(φ ) holds for every φ ∈ C 1 (R >0 × R d ;R), it also holds for every smooth function φ ∈ C ∞ (R >0 ×R d ;R). Let us now suppose that P(φ ) holds for every φ ∈ C ∞ (R >0 ×R d ;R), and fix φ ∈ C 1 (R >0 ×R d ;R) and (t * ,x * ) ∈ R >0 ×R d with the property that fφ has a strict local maximum at (t * ,x * ). For each ε > 0, denote by φ ε ∈ C ∞ (R >0 × R d ;R) the mollification of φ . Since (φ ε ) ε>0 converges locally uniformly to φ , Exercise 3.1 gives a sequence (t ε ,x ε ) ε>0 ⊆ R >0 × R d converging to (t * ,x * ) with the property that (t ε ,x ε ) is a local maximum of f -φ ε for each ε > 0.

The property P(φ ε ) implies that for every ε > 0, ∂ t φ ε -H(∇φ ε ) (t ε ,x ε ) ⩽ 0. (S.12)

Since (∂ t φ ε ) ε>0 and (∇φ ε ) ε>0 converge to ∂ t φ and ∇φ locally uniformly by the properties of mollifiers, letting ε tend to zero in (S.12) and using the local Lipschitz continuity of H shows that ∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0.

Invoking Exercise 3.2 establishes P(φ ) and completes the proof. Combining this with the assumption that f satisfies the equation at (t * ,x * ) shows that f is a viscosity subsolution. An identical argument shows that it is also a viscosity supersolution. This completes the proof. Remembering that ∂ t Φ ⩾ V ∇Φ shows that ∂ t φ -H(∇φ ) (t * ,x * ) ⩽ 0 as required.

Exercise 3.7. Fix x * ∈ R d , and observe that g(t,x) ∶= f (t,x * + x) is a viscosity solution to the Hamilton-Jacobi equation (3.20). It follows by the comparison principle that sup

R ⩾0 ×R d ( f -g) = sup {0}×R d ( f -g) ⩽ L x * .
An identical argument can be used to control gf and show that for every x ∈ R d ,

f (t,x) -f (t,x * + x) ⩽ sup R ⩾0 ×R d f -g ⩽ L x *
as desired.

Exercise 3.8. Replacing f (t,⋅) and g(t,⋅) by f (t,⋅ + x) and g(t,⋅ + x) respectively, assume without loss of generality that x = 0. Up to renaming f and g if necessary, (i) The fact that φ t ∈ C 1 (R d ;R d ) is clear. Since ∇ψ is Lipschitz continuous and takes values in a bounded set, and ∇H is locally Lipschitz continuous, we also see that the mapping x ↦ ∇H(∇ψ(x))

is Lipschitz continuous. We denote its Lipschitz constant by L, and set T ∶= L -1 . We first show that φ t is bijective for every t < T . We fix t < T and y ∈ R d , and for every x ∈ R d , we set ρ(x) ∶= y +t∇H(∇ψ(x)). By the definition of T , we have for every x,x ′ ∈ R d that ρ(x ′ )ρ(x) ⩽ t T

x ′x .

The mapping ρ is therefore a contraction. By the Banach fixed-point theorem, there exists a unique x ∈ R d such that x = ρ(x). Recalling the definition of ρ, we see that the property x = ρ(x) is equivalent to y = φ t (x). We have thus shown that φ t is bijective. One can then use the implicit function theorem to guarantee that the inverse of φ t also belongs to C 1 (R d ;R d ).

(ii) We denote the inverse mapping to φ t by φ -1 t . Arguing as in the previous step, one can show that the mapping (t,x) ↦ φ - This shows that u ∈ C 1 ([0,T ) × R d ;R). To compute its derivatives, let us fix some notation. We write I d ∈ R d×d for the identity matrix on R d , and given a continuously differentiable function v ∶ R d → R k , we denote by ∇v ∶ R d → R d×k its Jacobian,

∇v = ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ∂ x 1 v 1 ⋯ ∂ x 1 v k ⋮ ⋱ ⋮ ∂ x d v 1 ⋯ ∂ x d v k ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ .
(S. [START_REF] Aldous | The ζ (2) limit in the random assignment problem[END_REF] In this notation, given another continuously differentiable function w ∶ R m → R d , the chain rule reads ∇ v(w(x)) = ∇w(x)∇v(w(x)).

Taking the gradient on both sides of the identity φ t φ -1 t (x) = x (S.16) reveals that ∇φ -1 t (x) I d -t∇ 2 ψ φ -1 t (x) ∇ 2 H ∇ψ φ -1 t (x) = I d . (S.17)

S.4 Statistical inference

Exercise 4.1. Since the matrix A has rank one, its image is a one-dimensional subspace of R d . It is therefore generated by some non-zero vector u ∈ R d . In particular, for every w ∈ R d , there exists λ (w) ∈ R with Aw = λ (w)u. Defining the vector v ∈ R d by v i = λ (e i ) for 1 ⩽ i ⩽ d, where e i denotes the canonical basis vector in R d , gives the decomposition A = uv * . Normalizing u and v so that they become unit vectors gives the decomposition A = λ uv * for some scalar λ ∈ R and some unit vectors u,v ∈ R d . Notice that v is non-zero since A would otherwise map R d onto zero and therefore be of rank zero. The symmetry of A implies that uv * = vu * . It follows that

(u ⋅ v) 2 = u * vu * v = u * uv * v = u 2 v 2 .
This is the case of equality in the Cauchy-Schwarz inequality, so u and v are equal up to a multiplicative constant. Since u and v are both unit vectors, they are in fact equal up to a sign. This means that, up to changing the sign of λ , we have A = λ uu * . The uniqueness of u comes from the fact that the image of A is spanned by the unit vector u, and it is spanned by a unique unit vector up to a sign. To establish the uniqueness of λ observe that λ = tr(A). This completes the proof. for the function F ∶ R d → R defined by F(z) ∶= max 1⩽i⩽d a i ⋅ z. Observe that F is Lipschitz continuous with F Lip ⩽ √ a. Indeed, fix z 1 ,z 2 ∈ R d with F(z 1 ) ⩾ F(z 2 ) and let 1 ⩽ j ⩽ d be such that a j ⋅ z 1 = F(z 1 ). The Cauchy-Schwarz inequality and the fact that a j 2 = Eg 2 j ⩽ a imply that

F(z 1 ) -F(z 2 ) = a j ⋅ z 1 -max 1⩽i⩽d a i ⋅ z 2 ⩽ a j ⋅ (z 1 -z 2 ) ⩽ √ a z 1 -z 2 .
Invoking the Gaussian concentration inequality (Theorem 4.7) completes the proof. To bound this further, observe that the random variables ((W x) i ) i⩽N are independent centred Gaussian with variance

E(W x) 2 i = E N j,k=1
W i j W ik x j x k = N j=1

x 2 j = 1.

It follows by independence that

Eexp λ W x 2 = E N i=1 exp λ (W x) 2 i = Eexp λ Z 2 N
for a standard Gaussian Z. A change of variables shows that for λ < 1 2, Taking the supremum in x and rearranging gives the required bound.

Eexp(λ Z 2 ) = 1 √ 2π ˆR e -x 2 ( 1 2 -λ ) dx = 1 √ 1 -2λ 1 √ 2π ˆR e -
(iii) To define the set A we will leverage a standard upper bound (see [START_REF] Vershynin | High-dimensional probability: an introduction with applications in data science[END_REF]Corollary 4.2.13]) on the smallest number of balls of radius ε > 0 required to cover the unit ball. In general, the smallest number of balls of radius ε > 0 required to cover a compact set K is known as the covering number of the set K, and it is denoted by N (K,ε). A related quantity is the packing number of a compact set K. A subset N of K is ε-separated if xy > ε for all distinct points x,y ∈ N . The packing number of K is the size of the largest possible ε-separated subset of K, and it is denoted by P(K,ε). By maximality, it is clear that the balls of radius ε > 0 centred at the points in an ε-separated subset of K cover the set K. In particular, the covering number is bounded by the packing number, N (K,ε) ⩽ P(K,ε).

We now use this observation to find an upper bound on the covering number of the unit ball B 1 (0) ⊆ R N . Denote by m Lebesgue measure on R N , fix ε > 0 and let P be a maximal ε-separated subset of B 1 (0). Since xy > ε for all distinct points x,y ∈ P, the balls (B Redefining the constant C establishes (4.61).

(iv) The layer-cake representation and the change of variables λ = λ ′ N q 2 give E W q * = ˆ+∞ 0 P W 2 * > λ 2 q dλ = N q 2 ˆ+∞ 0 P W 2 * ⩾ (λ ′ ) 2 q N dλ ′ .

Applying (4.61) and redefining the constant C completes the proof.

Exercise 4.6. Taylor's theorem, the assumption that EX = 0, and the mean value theorem imply that EXF(X) -EX 2 EF ′ (X) ⩽ EX(F(0)

+ F ′ (0)X) -EX 2 EF ′ (X) + 1 2 F ′′ ∞ E X 3 ⩽ EX 2 F ′ (X) -F ′ (0) + 1 2 F ′′ ∞ E X 3 ⩽ E X 3 F ′′ ∞ + 1 2 F ′′ ∞ E X 3 .
This completes the proof.

Exercise 4.7. Denote by E i the average with respect to the randomness of X i+1 ,...,X d and by E i the average with respect to the randomness of X i . Let f i ∶= E i f -E i-1 f with the convention that E 0 f = E f and E d f = f . Observe that for 1 ⩽ i < j ⩽ d,

E f i f j = E f i E j f j = 0.
Combining this with the fact that f -E f = ∑ d i=1 f i and Jensen's inequality reveals that

E( f -E f ) 2 = d i=1 E f 2 i = d i=1 E E i ( f -E i f ) 2 ⩽ d i=1 E( f -E i f ) 2 . (S.26)
At this point, introduce the function

f ′ i ∶= f (X 1 ,...,X i-1 ,X ′ i ,X i+1 ,...,X d ).
Since X ′ i is an independent copy of X i ,

E i ( f -E i f ) 2 = E i f 2 -(E i f ) 2 = E i f 2 -E i f E i f ′ i = E i f 2 -E i f f ′ i = 1 2 E i ( f -f ′ i ) 2 .
Substituting this into (S.26) completes the proof.

Exercise 4.8. We first give a proof that leverages the Efron-Stein inequality obtained in the previous exercise, and then sketch an alternative argument that is more direct.

For each integer N ⩾ 1, let ε 1 ,...,ε N be i.i.d. Rademacher random variables. Consider the normalized partial sum

S N ∶= 1 √ N N i=1 ε i ,
and recall that (S N ) N⩾1 converges to Z in distribution by the central limit theorem. Denote by ε ′ 1 ,...,ε ′ N another set of i.i.d. Rademacher random variables, also independent of ε 1 ,...,ε N , and for each 1 ⩽ i ⩽ N consider the normalized partial sum

S i N ∶= 1 √ N j≠i ε j + ε ′ i = S N + ε ′ i -ε i √ N .
If we denote by E i the average with respect to the randomness of the vector (ε i ,ε ′ i ), then the Efron-Stein inequality implies that Var f (S N ) ⩽ 1 2

N i=1 E( f (S N ) -f (S i N )) 2 = 1 2 N i=1 EE i ( f (S N ) -f (S i N )) 2 .
Notice that f (S N )f (S i N ) is non-zero if and only if ε i ≠ ε ′ i , so

E i ( f (S N ) -f (S i N )) 2 = 1 2 f S N + 1 -ε i √ N -f S N + -1 -ε i √ N 2 .
It follows by Taylor's theorem that Var f (S N ) ⩽ 1 4

N i=1 E f S N + 1 -ε i √ N -f S N + -1 -ε i √ N 2 = 1 4 N i=1 E 2 √ N f ′ (S N ) + O f ′′ ∞ N 2 = 1 4 N i=1 4 N E f ′ (S N ) 2 + O f ′′ ∞ f ′ ∞ N 3 2 = E f ′ (S N ) 2 + O f ′′ ∞ f ′ ∞ N 1 2 .
Letting N tend to infinity and using the fact that (S N ) N⩾1 converges to Z in distribution completes the proof. We now briefly sketch an alternative proof of the Gaussian Poincaré inequality. For every x ∈ R and t > 0, we denote the heat kernel evaluated at (t,x) by

Φ t (x) ∶= 1 √ 2πt exp - x 2 2t .
We recall that ∂ t Φ = 1 2 ∂ 2 x Φ, and we write ⋆ for the convolution operator (acting only on the space variable). A direct calculation and an application of Jensen's inequality give that, for every t ∈ (0,1],

∂ t Φ t ⋆ (Φ 1-t ⋆ f ) 2 = Φ t ⋆ Φ 1-t ⋆ f ′ 2 ⩽ Φ 1 ⋆ ( f ′ ) 2 .
Integrating this inequality over t ∈ (0,1] and evaluating the result at x = 0, we obtain another proof of the Gaussian Poincaré inequality. Exercise 4.9. Let f be the function defined in (4.110), and denote by t c ∶= sup{t ⩾ 0 f (t) = 0} ∈ R ⩾0 ∪ {+∞} its largest zero. Notice that ψ(0) = 0, so f is non-negative and f (0) = 0. By Proposition 4.10 and Theorem 4.9, the function f is convex, so we have f (t) > 0 for every t > t c . By Proposition 4.15, and since f is differentiable almost everywhere, it suffices to verify that t c ∈ R >0 . From the definition of ψ in (4.85) or (4.89), we see that ψ is a smooth function and satisfies ψ(0) = 0, while it follows from (4.82) and the Nishimori identity that ∂ h ψ(0) = E⟨x ⋅ x ′ ⟩ = Ex 1 2 = 0. (S.27)

The Gibbs measure above is with t = 0, h = 0, and N = 1; in this case, this Gibbs measure is simply the measure P 1 , and x ′ is an independent sample from P 1 . Recalling also from (4.82) that ψ is Lipschitz continuous, we deduce that there exists C < +∞ such that for every h ∈ R,

ψ(h) ⩽ Ch 2 .
For every t < C 4, the supremum in (4.110) is therefore achieved at h = 0, and thus t c is strictly positive. Since we assume that P 1 is not the Dirac mass at 0, we see from (4.95) that ∂ 2 h ψ(0) > 0, and in particular there exists h > 0 such that ψ(h) > 0. It is therefore clear from (4.110) that f (t) > 0 if t is sufficiently large, and thus t c is finite.

Exercise 4.10. A direct computation together with (4.73), the Nishimori identity and the derivative computation (4.82) imply that

E x -E[x Y] 2 = E x 2 -2Ex ⋅ E[x Y] + E E[x Y] 2 = NEx 2 1 -2E⟨x ⋅ x⟩ + E⟨x ⋅ x ′ ⟩ = NEx 2 1 -N∂ h F N (t,h).
It follows by Propositions 4.10, 2.11 and 2.15 that at the interior point of differentiability (t,h) ∈ R >0 × R >0 , lim

N→+∞ 1 N E x -E[x Y] 2 = Ex 2 1 -∂ h f (t,h).
Invoking the envelope theorem (Theorem 2.21) completes the proof.

Exercise 4.11. In this context, the prior P 1 is the law of a Bernoulli random variable with probability of success p ∈ (0,1), so it will be convenient to write σ ∈ Σ N and σ ∈ Σ N in place of x and x respectively. In this notation, for any t ⩾ 0,

H ○ N (t,σ ) = i< j 2t N W i j +W ji σ i σ j + 4t N σ i σ j σ i σ j + N i=1 2t N W ii + 2t -tN.
Noticing that

W i j +W ji √ 2
is again a standard Gaussian, we have the equality in distribution jointly over σ , Remembering that EW ii = 0 and rearranging completes the proof. Since P 1 is the law of a symmetric Bernoulli random variable, the initial condition simplifies to

ψ(h) = Elog 1 2 exp √ 2hz 1 + 2hσ 1 -exp -( √ 2hz 1 + 2hσ 1 ) -h = Elogcosh √ 2hz 1 + 2hσ 1 -h.
Substituting this into (S.28) completes the proof.

S.5 Poisson point processes and extreme values

Exercise 5.1. By Step 2 of the proof of Lemma 5.3, there exists a sequence (K n ) n⩾1 of compact subsets of S that covers S. The assumption that λ is locally finite implies that for each n ⩾ 1, the set of indices {i ∈ I x i ∈ K n } is finite. The index set I is the union of these sets as n ranges over the natural numbers, and is therefore countable. Each of the mappings λ ↦ ´S f n dλ is continuous, and therefore measurable with respect to the Borel σ -algebra B(M δ (S)). As a limit of measurable functions, the mapping λ ↦ λ (U) is itself measurable with respect to the Borel σ -algebra B(M δ (S)). This means that the λ -system of measurable sets A ⊆ S such that λ ↦ λ (A) is measurable contains the π-system of open sets. It follows by the Dynkin π-λ theorem that each of the mappings λ ↦ λ (A) is measurable with respect to the Borel σ -algebra B(M δ (S)). To see that this is the smallest σ -algebra for which this property holds, it suffices to approximate any function f ∈ C c (S;R) by a sequence of measurable functions that take a finite number of values.

Exercise 5.3. For a direct argument, we can reproduce the proof of the converse implication in Lemma 5.3 -alternatively, we can also appeal to this lemma with the set M = {λ n n ∈ N}.

Exercise 5.4. It is clear that ρ is non-negative and symmetric with ρ(λ ,λ ) = 0 for all λ ∈ M δ (S). It therefore suffices to verify that ρ satisfies the triangle inequality. Fix λ ,λ ′ ,λ ′′ ∈ M δ (S), and define the function φ ∶ R ⩾0 → R ⩾0 by φ (x) ∶= x 1 + x .

Observe that φ is non-decreasing and concave on R ⩾0 , and that it is such that ρ(λ ,λ ′′ ) = This completes the proof.

Exercise 5.5. Let (λ n ) n⩾1 be a Cauchy sequence in M δ (S). Given ε ∈ (0,1) and k ⩾ 1, it is possible to find n,m ⩾ 1 large enough so that ´S f k d(λ nλ m )

1 + ´S f k d(λ n -λ m ) ⩽ ε. Rearranging reveals that ˆS f k d(λ n -λ m ) ⩽ ε 1 -ε .
This implies that for every f ∈ F, the sequence ´S f dλ n n⩾1 is Cauchy in R. We now use a density argument to show that this in fact true for every f ∈ C c (S;R). Fix f ∈ C c (S;R) as well as ε > 0, and denote by K the compact support of f . Invoke Lemma 5.1 to find g ∈ F with f (x)g(x) ⩽ ε for all x ∈ S and suppg ⊆ K ε . Since the sequence ´S gdλ n n⩾1 is Cauchy, there exist n,m ⩾ 1 large enough with ˆS f d(λ nλ m ) = ˆS gd(λ nλ m ) + ˆS( fg)d(λ nλ m ) ⩽ ε + 2ε sup k⩾1 λ k (K ε ).

Arguing as in the part of the proof of Theorem 5.2 leading to (5.8) reveals that for ε small enough, we have sup k⩾1 λ k (K ε ) < +∞. This shows that for each f ∈ C c (S;R), the sequence ´S f dλ n n⩾1 is Cauchy in R, and therefore convergent and uniformly bounded. Invoking Lemma 5.3 completes the proof. To obtain a matching lower bound, define the sequence (g m ) m⩾1 of bounded and continuous functions on S by g m (s) ∶= max 0,1md(x,F) . Since F is closed, the sequence (g m ) m⩾1 decreases to 1 F . It follows that for every n,m ⩾ 1,

Eexp -tΛ n (F) ⩾ Eexp -t ˆS g m dΛ n .

Letting n tend to infinity and then m tend to infinity shows that Recall that Λ(F) is a Poisson random variable with mean µ(F) while Λ(U) is a Poisson random variable with mean µ(U). Since µ(∂ A) = 0, we have that µ(F) = µ(A) = µ(U), so these two random variables are in fact equal in distribution. The above string of inequalities is therefore a string of equalities. This establishes (S.29) and completes the proof.

Exercise 5.7. By Proposition 5.7, it suffices to investigate the convergence of the Laplace transform of Λ n , and by Remark 5.8 it suffices to verify (5.17 for i * = ⌈log 2 (εn 1 ζ a)⌉. It follows that

1 n 1 ζ n k=1 EX k 1 {X k ⩽εn 1 ζ } ⩽ n 1-1 ζ EX 1 1 {X 1 ⩽a} + i * -1 i=0 EX 1 1 {X 1 ∈[2 i a,2 i+1 a]} ⩽ n 1-1 ζ a + 2a 1-ζ i * -1 i=0 2 i(1-ζ ) ⩽ n 1-1 ζ a + 2ε 1-ζ n 1 ζ -1 2 1-ζ -1 ,
and therefore,

limsup n→+∞ 1 n 1 ζ n k=1 EX k 1 {X k ⩽εn 1 ζ } ⩽ 2ε 1-ζ 2 1-ζ -1 .
Letting ε tend to zero gives the desired asymptotic.

(iv) Fix ε,δ > 0, and let K be large enough so

P{u K > δ } ⩽ δ 2 .
This is possible since Recalling that X k,n ⩾ 0 for 1 ⩽ k ⩽ n, and using the previous part to let n tend to infinity and then δ tend to zero establishes the claim.

P{u K > δ } =
(v) Introduce the sequences (Y k ′ ,n ) k ′ ,n⩾k and (Y n ) n⩾1 of random elements defined by

Y k ′ ,n ∶= X 1,n ∑ k ′ =1 X ,n ,..., X k,n ∑ k ′ =1 X ,n
and Y n ∶= X 1,n ∑ n =1 X ,n ,..., X k,n ∑ n =1 X ,n respectively. Proposition 5.15 and the continuous mapping theorem (Exercise A.10) imply that, for each fixed k ′ ⩾ k, as n tends to infinity, the sequence (Y k ′ ,n ) n⩾1 converges in law to the random element

Z k ′ ∶= u 1 ∑ k ′ =1 u ,..., u k ∑ k ′ =1 u .
In turn, as k ′ tends to infinity, the sequence (Z k ′ ) k ′ ⩾k converges almost surely, and therefore in law, to the random element Y ∶= (v 1 ,...,v k ).

By the union bound, for every ε > 0,

P Y k ′ ,n -Y n 1 ⩾ ε ⩽ k j=1 P X j,n ∑ k ′ =1 X ,n - X j,n ∑ n =1 X ,n ⩾ ε k ⩽ k j=1 P n -1 ζ X j,n n -1 ζ ∑ k ′ =1 X ,n ⋅ n -1 ζ ∑ n =1 X ,n ⋅ n -1 ζ n =k ′ X ,n ⩾ ε k ⩽ k j=1 P n -1 ζ n =k ′ X ,n ⩾ ε k ⋅ n -1 ζ X 1,n ,
where the second inequality uses that 1 a -1 a+b = b a(a+b) for every a,b > 0. To bound this further, fix δ > 0, and let δ ′ > 0 be small enough so

P{u 1 ⩾ δ ′ } = P{Λ[δ ′ ,+∞) > 1} = P{Π > 1} ⩾ 1 -δ 2,
where Π denotes a Poisson random variable with mean µ[δ ′ ,+∞) = (δ ′ ) -ζ . Using the convergence in law established in Proposition 5.15, find n large enough such that P{n -1 ζ X 1,n ⩾ δ ′ } ⩾ 1δ , and observe that

P Y k ′ ,n -Y n 1 ⩾ ε ⩽ k j=1 P n -1 ζ n =k ′ X ,n ⩾ εδ ′ k + δ .
It follows by (iv) that for every ε > 0, lim

k ′ →+∞ limsup n→+∞ P Y k ′ ,n -Y n 1 ⩾ ε ⩽ δ .
Letting δ tend to zero and invoking Exercise A.11 completes the proof. Applying this equality again with X n Z n replaced by Z n completes the proof.

Exercise 5.10. We treat each question separately. This shows that p n is continuous for n ⩾ 2 as the uniform limit of continuous functions. To see that p 1 is not continuous, consider the sequence (x n ) n⩾1 ⊆ V defined by x n j ∶= (log(n)) -1 j -1 1 {2⩽ j⩽n} . Notice that x n ∈ V as

1 log(n) n j=2 1 j ⩽ 1 log(n) ˆn 1 1 x dx = 1.
Moreover, (x n ) n⩾1 converges to the zero sequence but

p 1 (x n ) = 1 log(n) n j=2 1 j ⩾ 1 log(n) ˆn 2 1 x dx = log(n) -log(2) log(n)
is asymptotically lower bounded by 1 and can therefore not converge to 0. This shows that p 1 is not continuous.

(ii) The set P is closed under multiplication and linear combinations so it forms an algebra of continuous functions on the compact set V . By the real version of the Stone-Weierstrass theorem (Theorem A.10), it therefore suffices to show that P separates points. Fix distinct elements x ≠ y ∈ V , and introduce the measures µ x ∶= j⩾1 x 2 j δ x j and µ y ∶= j⩾1 y 2 j δ y j on [0,1]. Since the coordinates of x are decreasing and sum to less than one, each non-zero x j has finite multiplicity in the sequence (x j ) j⩾1 , and this multiplicity is Indeed, multiplying g by 1 {σ 1 =σ l } has the effect of merging the set I with the set I 1 in the sense that g(R n )1 {σ 1 =σ l } is defined just as g with the only difference that these two subsets are merged into one to form a new partition of {1,. (iv) The sum of the coordinates in each term on the right side of the Talagrand identities has been reduced by one compared to the left side. A recursive application of the Talagrand identities therefore allows us to express all the quantities S(n 1 ,...,n k ) in terms of S(2), which is equal to 1ζ by (5.58). Together with the density of P in the space of continuous functions on V , this means that for any continuous function f ∶ V → R, the expectation E f ((v j ) j⩾1 ) is entirely determined by the Ghirlanda-Guerra identities (5.57) and the parameter ζ ∈ (0,1) determining the law (5.58) of the overlap R 1,2 . Since the Poisson-Dirichlet process satisfies the same properties, the sequence (v j ) j⩾1 must have the same law as a Poisson-Dirichlet process with parameter ζ . This completes the proof.

Exercise 5.11. We treat each question separately.

(i) To begin with, observe that for any x ∈ (0,1),

F F -1 (x) = lim n→+∞ F F -1 (x) + 1 n ⩾ x,
where we have used the right-continuity of F. Now, suppose for the sake of contradiction that there exist t ∈ R and x ∈ (0,1) with F(t) < x but t ⩾ F -1 (x). By non-decreasingness of F, we have

F(t) ⩾ F F -1 (x) ⩾ x > F(t)
which is a contradiction. Conversely, suppose that t < F -1 (x) but F(t) ⩾ x. By definition of F -1 we have F -1 (x) ⩽ t which is a contradiction.

(iii) Arguing by contradiction, let us assume that the support of ζ is not contained in R ⩾0 . This means that we can find ε > 0 such that ζ (-∞,-ε] > 0. By the result of the previous step applied to the set A ∶= (-∞,-ε], the event A n ∶= {R , ′ ⩽ -ε for all ≠ ′ ⩽ n} has positive probability. On this event, we have

n =1 σ 2 = n , ′ =1 R , ′ ⩽ n -n(n -1)ε,
which can be made negative by taking n sufficiently large. Since this is absurd, the proof is complete.

Exercise 6.2. Combining the covariance structure (6.4) of the SK Hamiltonian with Exercise 4.4 reveals that for any ε > 0,

P 1 N max σ ∈Σ N H N (σ ) - 1 N E max σ ∈Σ N H N (σ ) ⩾ ε ⩽ 2exp - Nε 2 2 .
Since the right side of this equation is summable in N, the result follows by a simple application of the Borel-Cantelli lemma.

Exercise 6.3. Bounding the maximum over all configurations by the sum over all configurations reveals that

1 N E max σ ∈Σ N H N (σ ) = 1 Nβ Elogexpβ max σ ∈Σ N H N (σ ) ⩽ F N (β )
β .

On the other hand, bounding the Hamiltonian at each configuration by the maximum of the Hamiltonian over all configurations shows that

F N (β ) β ⩽ 1 Nβ Elog σ ∈Σ N expβ max σ ∈Σ N H N (σ ) ⩽ log(2) β + 1 N E max σ ∈Σ N H N (σ ).
This completes the proof. √ tx i j + √ 1 -tg i j σ i σ j in such a way that H N,0 (σ ) = H N (σ ) and H N,1 (σ ) = H x N (σ ). As usual, denote by F N (t) its corresponding interpolating free energy and by ⟨⋅⟩ t the average with respect to its associated Gibbs measure. Fix t ∈ (0,1), and observe that

F ′ N (t) = β N E⟨∂ t H ′ N,t (σ )⟩ t = β 2N 3 2 √ t N i, j=1
Ex i j ⟨σ i σ j ⟩ t -β 2N 3 We will now bound the terms in this expression using the Gaussian integration by parts formula (Theorem 4.5) and the approximate Gaussian integration by parts formula (Exercise 4.6). Fix indices 1 ⩽ i, j ⩽ N and consider F(g) ∶= ⟨σ i σ j ⟩ t as a function of the Gaussian vector g ∶= (g i j ) i, j⩽N . A direct computation reveals that ∂ g i j F = β ⟨σ i σ j ∂ g i j H N,t (σ )⟩ t -⟨σ i σ j ⟩ t ⟨β ∂ g i j H N,t (σ

)⟩ t = β √ 1 -t √ N ⟨σ i σ j σ i σ j ⟩ t - β √ 1 -t √ N ⟨σ i σ j ⟩ t ⟨σ i σ j ⟩ t = β √ 1 -t √ N 1 -⟨σ i σ j ⟩ 2 t ,
so the Gaussian integration by parts formula yields

β 2N 3 2 √ 1 -t Eg i j ⟨σ i σ j ⟩ t = β 2N 3 2 √ 1 -t E∂ g i j F = β 2 2N 2 (1 -E⟨σ i σ j ⟩ 2 t ).
(S.31)

If instead we consider F(x) ∶= ⟨σ i σ j ⟩ t as a function of the random vector x ∶= (x i j ), then

∂ x i j F = β ⟨σ i σ j ∂ x i j H N,t (σ )⟩ t -⟨σ i σ j ⟩ t ⟨β ∂ x i j H N,t (σ )⟩ t = β √ t √ N 1 -⟨σ i σ j ⟩ 2 t ∂ 2 x i j F = -2 β √ t √ N ⟨σ i σ j ⟩ t ∂ x i j F = 2β 2 t N ⟨σ i σ j ⟩ 3 t -⟨σ i σ j ⟩ t .
In particular, ∂ 2 x i j F ⩽ 4β 2 t N , so the approximate Gaussian integration by parts formula and the assumption that Ex 2 i j = 1 imply that

β 2N 3 2 √ t Ex i j ⟨σ i σ j ⟩ t - β 2 2N 2 (1-E⟨σ i σ j ⟩ 2 t ) ⩽ β 2N 3 2 √ t 6β 2 t N E x 11 3 = 3β 3 √ t N 5 2 E x 11 3 .
Combining this with (S.30), (S.31), and the fundamental theorem of calculus reveals that

F x N (β ) -F N (β ) = F N (1) -F N (0) ⩽ sup t∈[0,1] F ′ N (t) ⩽ 3β 3 √ N E x 11 3 .
Letting N tend to infinity completes the proof. This will follow from the fact that lim

N→+∞ E⟨F g ⋅ σ √ N ⟩ 2 = lim N→+∞ E ⟨F g ⋅ σ √ N ⟩E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ = lim N→+∞ E E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ 2 = E Z 1 ,Z 2 ,Z 3 F √ qZ 1 + 1 -qZ 2 F √ qZ 1 + 1 -qZ 3
for some independent standard Gaussian random variables Z 1 ,Z 2 ,Z 3 also independent of all other sources of randomness. We first prove rigorously that lim [START_REF] Barbier | The adaptive interpolation method: a simple scheme to prove replica formulas in bayesian inference[END_REF] and then describe how to adapt the argument for the computation of the two other limits. Since g is independent of all other sources of randomness,

N→+∞ E⟨F g ⋅ σ √ N ⟩ 2 = E Z 1 ,Z 2 ,Z 3 F √ qZ 1 + 1 -qZ 2 F √ qZ 1 + 1 -qZ 3 , (S.
E⟨F g ⋅ σ √ N ⟩ 2 = E⟨E g F g ⋅ σ 1 √ N F g ⋅ σ 2 √ N ⟩.
To simplify this quantity, observe that conditionally on the replicas σ 1 ,σ 2 ∈ Σ N , the average

E g F g ⋅ σ 1 √ N F g ⋅ σ 2
√ N can be expressed as some continuous function of the covariance matrix

C N ∶= 1 R 1,2 R 1,2 1
of the random vector G N ∶= N -1 2 (g ⋅ σ 1 ,g ⋅ σ 2 ). This means that there exists a bounded continuous function

Γ ∶ R 2×2 → R with E⟨F g ⋅ σ √ N ⟩ 2 = E⟨Γ(C N )⟩.
To determine the asymptotic behaviour of this quantity, denote by C ∶= 1 q q 1 the covariance matrix of G ∶= ( √ qZ 1 + √ 1 -qZ 2 , √ qZ 1 + √ 1 -qZ 3 ), and fix ε > 0 as well as δ > 0. Introduce the set Since C N -C ⩽ √ 2δ on the set A δ ,N , and Γ is continuous, by choosing δ small enough relative to ε, the first term in (S.34) can be made to vanish. It follows by Chebyshev's inequality that

A δ ,N ∶= σ 1 ,σ 2 ∈ Σ N R 1,2 -q ⩽ δ ,
E⟨1 { Γ(C N )-Γ(C) >ε} ⟩ ⩽ E⟨1 A c δ ,N ⟩ ⩽ E⟨(R 1,2 -q) 2 ⟩ δ 2 .
Letting N tend to infinity shows that (Γ(C N )) N⩾1 converges in probability to Γ(C).

Since this sequence of random variables is bounded as F is, the limit (S.33) holds by the dominated convergence theorem. We now briefly describe the main differences in the computation of the limits of

E ⟨F g ⋅ σ √ N ⟩E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ and E E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ 2 .
The first of these quantities can be expressed as

E⟨E g E Z F g ⋅ σ √ N F g ⋅ ⟨σ ⟩ √ N + 1 -qZ ⟩,
so its asymptotic behaviour is determined by that of the covariance matrix

C N ∶= ⎡ ⎢ ⎢ ⎢ ⎢ ⎣ 1 σ ⋅⟨σ ⟩ N σ ⋅⟨σ ⟩ N ⟨R 1,2 ⟩ + 1 -q ⎤ ⎥ ⎥ ⎥ ⎥ ⎦
which converges to C as the asymptotic overlap distribution concentrates on the singleton q. Similarly, the second of these quantities can be expressed as

EE g E Z 1 ,Z 2 F g ⋅ ⟨σ ⟩ √ N + 1 -qZ 1 F g ⋅ ⟨σ ⟩ √ N + 1 -qZ 2 ,
so its asymptotic behaviour is determined by that of the covariance matrix

C N ∶= ⟨R 1,2 ⟩ + 1 -q ⟨R 1,2 ⟩ ⟨R 1,2 ⟩
⟨R 1,2 ⟩ + 1q which again converges to C. This completes the proof.

Exercise 6.6. For every g ∈ L 1 ([0,1];R), we use the notation

ζ k+1 ζ k g ∶= 1 ζ k+1 -ζ k ˆζk+1 ζ k g.
We first observe that, by Jensen's inequality, we have for every g ∈ L 2 ([0,1];R) that 0 ⩽ We also recall that the set of continuous functions is dense in L 2 ([0,1];R), see for instance Theorem 2.19 of [START_REF] Adams | Sobolev spaces[END_REF]. Let φ ∈ C([0,1];R) be a continuous function such that fφ L 2 ⩽ ε. Appealing to (S.35) with g = fφ , we see that we make an error of at most ε if we substitute f by φ in the expression on the left side of (6.83). Since φ is uniformly continuous, there exists δ > 0 such that for every x,y ∈ [0,1] with xy ⩽ δ , we have φ (y)φ (x) ⩽ ε.

If max k (ζ k+1 -ζ k ) ⩽ δ , then K k=0 ˆζk+1 ζ k φ - ζ k+1 ζ k φ 2 ⩽ ε 2 .
Using also the identity in (S.35), we therefore obtain the desired result, up to a reparametrization of ε > 0.

Exercise 6.7. A direct computation reveals that -∂ 2 t f = E⟨∂ 2 t H(t,σ )⟩ + E⟨(∂ t H(t,σ )) 2 ⟩ -E⟨∂ t H(t,σ )⟩ 2 . We now use Gaussian integration by parts to simplify each of these terms. On the one hand, the Gibbs Gaussian integration by parts formula (Theorem 4.6) implies that E⟨∂ 2 t H(t,σ )⟩ = -1 (2t) 3 2 E⟨zσ ⟩ = -

1 2t
E⟨σ 2 ⟩ -E⟨σ ⟩ 2 .

On the other hand,

E⟨(∂ t H(t,σ )) 2 ⟩ -E⟨∂ t H(t,σ )⟩ 2 = 1 2t E⟨z 2 σ (σ -σ ′ )⟩ + 2 (2t) 1 2 E⟨zσ ′′ σ 2 ⟩ -E⟨zσ 3 ⟩ + E⟨σ 4 ⟩ -E⟨σ 2 ⟩ 2 ,
and the Gaussian integration by parts formula (Theorem 4.5) reveals that

E⟨z 2 σ (σ -σ ′ )⟩ = Ez ⋅ z ´σ (σ -σ ′ )exp H(t,σ ) + H(t,σ ′ ) dP(σ )dP(σ ′ ) ´exp H(t,σ ) + H(t,σ ′ ) dP(σ )dP(σ ′ ) = E⟨σ (σ -σ ′ )⟩ + √ 2tE⟨zσ (σ -σ ′ )(σ + σ ′ )⟩ -2 √ 2tE⟨zσ (σ -σ ′ )σ ′′ ⟩ = E⟨σ (σ -σ ′ )⟩ + 2tE⟨(σ -σ ′ )(σ + σ ′ )(σ 2 + σ σ ′ -2σ σ ′′ )⟩ -2tE⟨(σ -σ ′ )(σ ′′ + σ ′′′ )(σ 2 + σ σ ′ + σ σ ′′ + σ σ ′′′ -4σ σ ′′′′ )⟩ = E⟨σ 2 ⟩ -E⟨σ ⟩ 2 + 2h E⟨σ 4 ⟩ -4E⟨σ 3 ⟩⟨σ ⟩ -3E⟨σ 2 ⟩ 2 + 12E⟨σ 2 ⟩⟨σ ⟩ 2 -6E⟨σ ⟩ 4 .
Similarly, the Gibbs Gaussian integration by parts formula implies that E⟨zσ ′′ σ 2 ⟩ -E⟨zσ 3 ⟩ = (2t) 1 2 E⟨σ 2 (σ ′′2 + σ ′′ σ -2σ ′ σ ′′ )⟩ -E⟨σ 2 (σ 2σ σ ′ )⟩ = (2t) The Gibbs Gaussian integration by parts formula implies that ∂ t f (t) = E⟨σ 2 ⟩ -1 (2t) 1 2 E⟨zσ ⟩ = E⟨σ ⟩ 2 from which it is clear that ∂ t f (0) = (2p -1) 2 < 1. The dominated convergence theorem reveals that lim t→+∞ ∂ t f (t) = 1, so there exists t * ⩾ 0 with ∂ 2 t f (t * ) > 0. However, by the formula we obtained earlier, and observing that the Gibbs measure ⟨⋅⟩ is simply the measure P 1 when t = 0, we have

∂ 2 t f (0) = 2⟨ σ -⟨σ ⟩ 2 ⟩ 1 -3⟨σ ⟩ 2 .
This quantity is strictly negative whenever p ∈ (0,1) is such that ⟨σ ⟩ 2 > 1 3. Since ⟨σ ⟩ = 2p -1, an explicit calculation yields that this occurs if and only if

p ∈ 0, 3 - √ 3 6 ∪ 3 + √ 3 6 ,1 ⊇ 0, 1 5 ∪ 4 5 ,1 .
This shows that for this choice of measure P 1 , the free energy f is neither concave nor convex.

S.7 Basic results in analysis and probability

Exercise A.1. Fix measurable sets A,B ∈ S with A ⊆ B, and observe that by additivity of measure, µ(B) = µ(A) + µ(B ∖ A) ⩾ µ(A).

This establishes monotonicity. To prove continuity from below, fix an increasing sequence of measurable sets (A n ) n⩾1 ⊆ S, and observe that by countable additivity of measure, This completes the proof.

Exercise A.2. To see that the function m F is well-defined, observe that m F is finitely additive. Indeed, if ((a i ,b i ]) 1⩽i⩽n is a finite sequence of disjoint intervals and Exercise A.9. We treat each question separately.

(i) To begin with, suppose that (P n ) n⩾1 converges weakly to P, and let t ∈ R be a point of continuity of F. Fix ε > 0 and let φ 1 and φ 2 be piecewise linear functions with 1 {x⩽t-ε} ⩽ φ 1 (x) ⩽ 1 {x⩽t} ⩽ φ 2 (x) ⩽ 1 {x⩽t+ε} .

Integrating the outer inequalities with respect to P and the inner inequalities with respect to P n reveals that F(tε) ⩽ ˆR φ 1 dP ⩽ liminf n→+∞ F n (t) ⩽ limsup n→+∞ F n (t) ⩽ ˆR φ 2 dP ⩽ F(t + ε).

We have used the fact that φ 1 ,φ 2 ∈ C b (R;R) to let n tend to infinity. Letting ε tend to zero and remembering that t is a point of continuity of F shows that (F n (t)) n⩾1 converges to F(t). Conversely, suppose that for any point t of continuity of F, the sequence (F n (t)) n⩾1 converges to F(t). Fix f ∈ C b (R;R) as well as ε > 0 and k ⩾ 1. Denote by C F the set of points of continuity of F. The set C F is countable by monotonicity of F, so there exists M ∈ C F with P((-M,M] c ) ⩽ ε. Let (x k i ) 0⩽i⩽k ⊆ C F be a partition of [-M,M] with max 1⩽i⩽k-1 x k ix k i-1 ⩽ k -1 . Using the fact that (F n ) n⩾1 converges to F at M and -M, find n large enough so P n ((-M,M] c ) ⩽ 2ε. We introduce the approximating function

f k (x) ∶= 1⩽i⩽k f (x k i )1 (x k i-1 ,x k i ] (x),
and we combine the uniform continuity of f on [-M,M] with the fact that max 1⩽i⩽k-1 x k i+1x k i ⩽ k -1 to find k large enough so f k (x)f (x) ⩽ ε whenever x ⩽ M. Since (x k i ) 0⩽i⩽k ⊆ C F , we have

lim n→+∞ ˆR f k dP n = 1⩽i⩽k f k (x k i )(F(x k i ) -F(x k i-1 )) = ˆR f k dP.
It follows that for n large enough, ˆR f dP -ˆR f dP n ⩽ 3ε f ∞ + 3ε.

Letting n tend to infinity and then ε tend to zero completes the proof.

(ii) The assumption is necessary. Consider the sequence (P n ) n⩾1 of probability measures with P n ({n -1 }) = 1. This sequence converges weakly to the probability measure P defined by P({0}) = 1. However, the sequence of distribution functions of P n fails to converge to the distribution function of P at the point of discontinuity t = 0.

Exercise A.10. We treat each question separately.

(i) For every ε > 0 and δ > 0, introduce the set U ε,δ ∶= x ∈ S there exist y,z ∈ S such that d(x,y) < δ , d(x,z) < δ , and d ′ ( f (y), f (z)) ⩾ ε , where d ′ denotes the metric on S ′ . The space U ε,δ is open. Moreover, the complement of C f is given by

⋃ ε>0 ⋂ δ >0 U ε,δ ,
where the ranges for ε and δ can be restricted to cover only rational values. This shows that C f is measurable.

(ii) Let F be a closed subset of S ′ . We have that

P{ f (X n ) ∈ F} = P{X n ∈ f -1 (F)} ⩽ P X n ∈ f -1 (F) .
By the Portmanteau theorem, we deduce that

limsup n→+∞ P{ f (X n ) ∈ F} ⩽ P X ∞ ∈ f -1 (F) .
Since X ∞ takes values in C f with probability one, the last quantity can be rewritten as P X ∞ ∈ C f ∩ f -1 (F) .

To show that C f ∩ f -1 (F) is a subset of F, fix x ∈ C f ∩ f -1 (F), and let (x n ) n⩾1 be a sequence of elements of f -1 (F) that converges to x. Since f (x n ) ∈ F, the sequence (x n ) n⩾1 converges to x ∈ C f . Combined with the fact that F is a closed set, this implies that f (x) ∈ F, and thus x ∈ f -1 (F). It follows that limsup

n→+∞ P{ f (X n ) ∈ F} ⩽ P X ∞ ∈ C f ∩ f -1 (F) ⩽ P f (X ∞ ) ∈ F .
Invoking the Portmanteau theorem again completes the proof.

Exercise A.

P{Y n ∈ F} ⩽ P{Z k ∈ F ε } + limsup n→+∞ P{d(Y k,n ,Y n ) ⩽ ε}.
Another application of the Portmanteau theorem to the convergence in law of (Z k ) k⩾1 to Y reveals that limsup n→+∞ P{Y n ∈ F} ⩽ P{Y ∈ F ε }. Leveraging the fact that F is closed shows that F ε ↘ F, so letting ε tend to zero completes the proof.

Exercise A.12. This is immediate from Theorem A.21 upon realizing that the sequence (P) n⩾1 converges weakly to P. It is also possible to give a direct proof. Since S is separable there exists a countable set (x i ) i⩾1 such that for every m ⩾ 1, we have S = ⋃ i⩾1 B 1 m (x i ). Given ε > 0, the continuity of measure established in Exercise A.1 implies the existence of n(m) ⩾ 1 with

P S ∖ n(m) ⋃ i=1 B 1 m (x i ) ⩽ ε 2 m . It follows that the set K ∶= ⋂ ∞ m=1 ⋃ n(m) i=1 B 1 m (x i ) is such that P(S ∖ K) = P ∞ ⋃ m=1 S ∖ n(m) ⋃ i=1 B 1 m (x i ) ⩽ +∞ m=1 ε 2 m = ε.
Since S is complete and K is closed and totally bounded, K is compact. This completes the proof.

Figure 0 . 1 A

 01 Figure 0.1 A simple example of frustration.

Figure 0 . 2

 02 Figure 0.2 Illustration of the bipartite model for N = 6. Elementary units are organized in two layers and only interact across layers.

N n=1 1

 1 {xn=k} . (1.1) the number of units in state k. The constraint on the total energy can be encoded as K k=1 N k (x)e k = Ne.(1.2)

. 5 )

 5 Using that N! = exp N logN -N + O(logN) , we can rewrite this asexp N logN -N + K k=1 (N p k )log(N p k ) -N p k + O(logN)

  (i) Show that, for each K ⩾ 1, we have lim

Figure 1 . 1

 11 Figure 1.1 Dependence of the magnetization (1.23) on the magnetic intensity h in the two-dimensional Ising model for N large and β > 0 small (on the left) and large (on the right).

Exercise 2 . 3 .. 8 ) 2 . 4 .

 23824 Let C be a closed convex set whose interior is not empty. Prove that C is the closure of its interior,C = int(C) (2Exercise Let A be a subset of R d , and denote byconv(A) ∶= n i=1 α i x i x i ∈ A,λ i ∈ [0,1] with n i=1

Exercise 2 . 6 .Exercise 2 . 7 .Exercise 2 . 8 .

 262728 Let X be a bounded random variable taking values in R d . Show that the log-Laplace transform ψ ∶ R d → R defined by ψ(λ ) ∶= logEexp(λ ⋅ X) Fix an index set I, and for each α ∈ I, let f α ∶ R d → R ∪ {+∞} be a convex function. Show that the supremum function sup α∈I f α (2.11) is convex. Prove the same statement with convex replaced by lower semi-continuous. Let f = f (x,y) ∶ R d × R k → R ∪ {+∞} be a convex function (of the pair (x,y)). For every x ∈ R d , we define g(x) ∶= inf y∈R k f (x,y).

Lemma 2 . 8 .

 28 Let f ∶ R d → R ∪ {+∞} be a proper and convex function. If there exist δ > 0 and x * ∈ R d such that f is bounded from above by M > 0 on the open ball B 2δ (x * ), then the following assertions hold.

Exercise 2 . 18 .

 218 The purpose of this exercise is to prove the one-dimensional Rademacher theorem for a Lipschitz function F ∶ [a,b] → R.

. 36 )

 36 (iii) Using the Lebesgue differentiation theorem in Theorem A.16, conclude that the function F is almost everywhere differentiable on [a,b]. Exercise 2.19. Let A be a d-by-k matrix, b ∈ R d , and c ∈ R k . We denote by A * the transpose of the matrix A, and by ⩽ the partial order on R d given by its product structure; in other words, for every x,y ∈ R d , we write x ⩽ y to mean that the inequality holds coordinate by coordinate. The goal of this exercise is to identify a general condition under which the "linear programs" inf b ⋅ x x ∈ R d , x ⩾ 0, Ax ⩾ c and sup c ⋅ y y ∈ R k , y ⩾ 0, A * y ⩽ b (2.37) are equal.

Figure 2 . 1

 21 Figure 2.1 Graphs of m ↦ tm 2ψ * (m) for t = 0.1 and t = 0.6 in the Curie-Weiss model.

Corollary 3 . 6 .

 36 If u,v ∈ L are a viscosity subsolution and a viscosity supersolution to the Hamilton-Jacobi equation (3.20) respectively, then sup

Corollary 3 . 7 .

 37 If u,v ∈ L are two viscosity solutions to the Hamilton-Jacobi equation (3.20) with the same initial condition u(0,⋅) = v(0,⋅), then u = v.

Lemma 3 . 9 .

 39 Under the assumptions of Theorem 3.8, the Hopf-Lax function (3.54) satisfies the right initial condition, f (0,⋅) = ψ.

. 68 ) 3 . 14 .Lemma 3 . 15 .Lemma 3 . 16 .

 68314315316 Lemma Under the assumptions of Theorem 3.13, the Hopf function (3.66) satisfies the right initial condition, f (0,⋅) = ψ.(3.69)Proof. Taking t = 0 in the representation (3.68) and combining the convexity of ψ with the Fenchel-Moreau theorem (Theorem 2.5) completes the proof. ∎ Under the assumptions of Theorem 3.13, for every(t,x) ∈ R ⩾0 × R d , there exists p ∈ R d with p ⩽ ψ Lip such that f (t,x) = p ⋅ x +tH(p)ψ * (p).(3.70)Proof. Observe that ψ * (p) = +∞ whenever p > ψ Lip by Exercise 2.12. This means that the supremum in (3.68) may be restricted to the compact ball of radius ψ Lip about the origin. Using also that the function ψ * is lower semicontinuous, by part (i) of Exercise 2.10, this implies that the supremum in (3.68) is achieved, as announced.∎ Under the assumptions of Theorem 3.13, the Hopf function f is jointly convex and belongs to the space L with sup t⩾0 f (t,⋅) Lip = ψ Lip .(3.71)

Proposition 3 . 18 .

 318 If f ,g ∶ R d → R are two convex functions, then sup x∈R d inf y∈R d f (x) + g(y)x ⋅ y = sup y∈R d inf x∈R d f (x) + g(y)x ⋅ y . (3.77) In particular, when ψ ∶ R d → R and H ∶ R d → R are both convex, the Hopf-Lax function (3.54) and the Hopf function (3.66) coincide.

  is a jointly convex and jointly Lipschitz continuous function that satisfies the Hamilton-Jacobi equation (3.20) on a dense subset of R ⩾0 × R d and f (0,⋅) ∈ C 1 (R d ;R), then f is a viscosity solution to the Hamilton-Jacobi equation (3.20).

  is a jointly convex and jointly Lipschitz continuous function that satisfies the Hamilton-Jacobi equation (3.20) on a dense subset of R ⩾0 × R d , then it satisfies the Hamilton-Jacobi equation (3.20) at all its points of differentiability in

  be a jointly convex and jointly Lipschitz continuous function that satisfies the Hamilton-Jacobi equation (3.20) on a dense subset of R ⩾0 × R d . Suppose that the initial condition ψ ∶= f (0,⋅) is such that, for every x ∈ R d and p ∈ ∂ ψ(x), there exists b ∈ R with (b, p) ∈ ∂ f (0,x) and b -H(p) ⩾ 0. Then f is a viscosity solution to the Hamilton-Jacobi equation(3.20).

  where we have used the smoothness of φ . Invoking the convex selection principle in Corollary 3.24 shows that f is a viscosity solution to the Hamilton-Jacobi equation(3.19). Together with the uniqueness result in Corollary 3.7, this implies that (F N ) N⩾1 converges to the unique viscosity solution f to the Hamilton-Jacobi equation(3.19), and arguing as in Proposition 3.20 gives the Hopf variational representation (3.121) of f . This completes the proof. ∎

.117) 4 . 4

 44 Comparison with a concrete algorithmIn this section, we discuss a classical and computationally efficient method of constructing an estimator for the symmetric rank-one matrix xx * from its noisy observation Y known as principal component analysis (PCA). The PCA estimator xx * of the rank-one matrix xx * is constructed by finding an eigenvector v ∈ R N with v 2 = N associated with the top eigenvalue of the symmetric matrix Y +Y * ∈ R N×N , and setting xx * ∶= λ 0 vv * (4.118)

and let a ∶= √ 2hz 1 +

 1 2hx 1 and b ∶= 1 + 2h, in such a way that the innermost integral becomes last equality we have identified the density of a Gaussian random variable with mean a b and variance 1 b. It follows by independence of z 1 and x 1 that

Figure 4 . 1

 41 Figure 4.1 The dotted line is the minimal mean-square error for the Rademacher prior (4.124) while the solid line is the mean-square error achieved using PCA.

Figure 4 . 3

 43 Figure 4.3 The two figures display the same graph of interactions with 100 red nodes and 100 blue nodes, and with each node having on average 10 links with nodes of the same color and 1 link with a node of a different color (all edges are independent). On the left figure, the blue and the red nodes have been placed uniformly at random over the entire area, and the colors have been concealed. On the right figure, the blue and the red nodes have been classified by placing them randomly to the left and to the right of the area respectively. In the community detection problem, the left figure is shown to the statistician, whose goal is to infer the colouring of the nodes.

Theorem 4 . 24 .

 424 Under assumptions (A1) and (A2), the free energy (4.142) in the stochastic block model is asymptotically equivalent to the free energy (4.9) in the symmetric rank-one matrix estimation problem with Bernoulli prior, By the asymptotic equivalences (4.154) and (4.160), it suffices to show that FSBM

+∞

  n=1 δ yn , and observe that every function of the form f (y) ∶= min((a(yb)) + ,1), for arbitrary a,b > 0, belongs to C c ((0,+∞];R). Choosing a = k and b = y 1 ± k -1

  d. and independent of (u n ) n⩾1 , the marking theorem (Proposition 5.11) and the mapping theorem (Proposition 5.10) applied with the function f (u,x,y) ∶= (ux,y) imply that +∞ n=1 δ (unXn,Yn) is a Poisson point process on R >0 × R d with intensity measure (µ ⊗ ν) ○ f -1 . Given two measurable sets A ⊆ R >0 and B ⊆ R d , Fubini's theorem and a change of variables reveal that

δ

  (un exp(t(gn-tvζ 2)), gn-tvζ ) and +∞ n=1 δ (un, gn) have the same law.

=

  Poisson point processes with the same intensity measure. In particular, denotes equality in distribution.

Figure 5 . 2

 52 Figure 5.2 When ultrametricity holds, the law of the triple of overlaps(R 1,2 ,R 1,3 ,R 2,3) is supported on the set of points (x,y,z) ∈ [0,1] 3 with two equal coordinates smaller than the third.

1 2

 2 ˆ{x>y} ψ(x,y,y)dζ (x)dζ (y).(5.107)

2 :

 2 defining the outer measure µ * . Inspired by the value (A.30) that µ must take on open sets, given an open set U ⊆ S, let ρ(U) ∶= sup T ( f ) f ∈ C(S;R) and 0 ⩽ f ⩽ 1 U , and define the function µ * ∶ 2 S → [0,+∞] by µ * (A) ∶= inf ρ(U) A ⊆ U and U is open . (A.31)

  x) dy = 0, (A.38) where we denote by B r (x) the open Euclidean ball of radius r > 0 centred at x ∈ R d , and by m(B r (x)) its Lebesgue measure. When d = 1, this readily implies that, for almost every x ∈ R, dt = f (x). (A.39) To prove (A.38), we will apply the theory of maximal operators to the Hardy-Littlewood maximal operator defined on the space L 1 loc (R d ;R) of locally integrable functions on R d by H * f (x) ∶= sup r>0 1 m(B r (x)) ˆBr(x) f (y)dy , (A.40)

  y)dy. (A.42) Lemma A.14 (Vitali covering). If C is a collection of open balls in R d and U denotes their union, then for any c < m(U), there exist disjoint balls (B i ) i⩽n ⊆ C with c Proof. Invoking Exercise A.4 allows us to find a compact set K ⊆ U with m(K) > c.

Lemma A. 15 (

 15 Hardy-Littlewood). The Hardy-Littlewood maximal operator (A.40) is weak-(1,1).

  d → C defined by ϕ P (t) ∶= ˆRd e it⋅x dP(x). (A.61) In the case when P admits a density with respect to Lebesgue measure, the characteristic function is the Fourier transform of the density. The Laplace transform of a probability measure P on R d ⩾0 is the real-valued function L P ∶ R d ⩾0 → R defined by L P (λ ) ∶= ˆRd ⩾0 e -λ ⋅x dP(x). (A.62) Theorem A.22. If two probability measures P and Q on R d have the same characteristic function, ϕ P = ϕ Q , then they must be equal, P = Q. Proof. By a straightforward approximation argument, it suffices to show that for every continuous function of compact support f ∈ C c (R d ;R), we have ˆRd f dP = ˆRd f dQ. (A.63)

  and thus (1 -t)x +tx ′ ∈ int(C). It follows thatx = lim t↘0 (1 -t)x +tx ′ ∈ int(C).

Exercise 2 . 16 .

 216 It suffices to show that for every x ∈ H ∖C the infimum d(x,C) ∶= inf xy 2 y ∈ C is achieved. Indeed, all other arguments in the proof of Lemma 2.1 extend without change to the Hilbert space setting. Fix x ∈ H ∖C, and let (y n ) n⩾1 be a minimizing sequence with lim n→+∞ xy n 2 = d(x,C).

Exercise 3 . 5 .

 35 Fix φ ∈ C ∞ (R ⩾0 × R d ;R) with the property that fφ has a local maximum at the point (t * ,x * ) ∈ R >0 × R d . Since f ∈ C 1 (R ⩾0 × R d ;R) and t > 0, we have ∂ t f -∂ t φ (t * ,x * ) = 0 and ∇ f -∇φ (t * ,x * ) = 0.

Exercise 3 . 6 .

 36 Let φ ∈ C ∞ (R >0 × R d ;R) be such that ( f -Φ)φ = f -(φ + Φ) has a local maximum at (t * ,x * ) ∈ R >0 × R d . Sincef is a subsolution to the Hamilton-Jacobi equation (3.20) and φ + Φ ∈ C ∞ (R >0 × R d ;R), we have ∂ t (φ + Φ)(t * ,x * ) ⩽ H(∇φ + ∇Φ)(t * ,x * ) ⩽ H ∇φ (t * ,x * ) +V ∇Φ(t * ,x * ) .

1 t

 1 (x) is in C 1 ([0,T ) × R d ;R). Introducing the function K ∶ R d → R defined by K(p) ∶= H(p) -∇H(p) ⋅ p,we can represent the function u as u(t,x) = ψ φ -1 t (x) +tK ∇ψ φ -1 t (x) .

Exercise 4 . 2 .

 42 To show that the conditional law of x given Y is the Gibbs measure (4.7), we need to prove that for all bounded and measurable functionsf ∶ R N → R and g ∶ R N×N → R, E f (x)g(Y ) = E ´RN f (x)expH

((

  iv) Combining the three previous parts shows thatI N (t) = 1 N I(x;Y ) = 1 N H(P x,Y P x,W ) -1 N H(P Y P W ) = t N 2 E x 4 -F ○ N (t)which agrees with(4.18). This completes the proof.Exercise 4.4. Let A ∈ R d×d be the positive semi-definite matrix such thatA 2 = (Eg i g j ) 1⩽i, j⩽d .We denote the rows of A by a 1 ,...,a d ∈ R d . Letting z be a standard Gaussian vector on R d , we have g d = Az, where d = denotes equality in distribution. In particular, Az) i = max 1⩽i⩽d a i ⋅ z = F(z)

Exercise 4 . 5 .

 45 We treat each question separately.(i) Fix λ > 0 and x ∈ R N of unit norm. Chebyshev's inequality reveals thatP W x 2 ⩾ aN ⩽ exp(-λ aN)Eexp λ W x 2 .(S.24)

.

  This means that Eexp(λ W x 2 ) ⩽ exp(CN) for some C > 2. Substituting this into (S.24) establishes (4.62).(ii) Given x with x ⩽ 1, let y ∈ A be such that xy ⩽ 1 2. By the triangle inequality,W x ⩽ W * xy + Wy ⩽ 1 2 W * + sup y∈A Wy .

2 N

 2 ε 2 (x)) x∈P are disjoint and contained in the expanded unit ball B 1+ε 2 (0). It follows by the additivity and scaling properties of Lebesgue measure thatP(B 1 (0),ε) ε N (B 1 (0),ε) ⩽ P(B 1 (0),ε) ⩽ 1 + 2 ε N . (S.25) Applying (S.25) with ε = 1 2 and invoking (ii) allows us to choose a set A of size 5 N with the property that W * ⩽ 2sup y∈A Wy Together with (i) and the union bound this implies that P W 2 * ⩾ aN ⩽ P 4sup y∈A Wy 2 ⩾ aN ⩽ 5 N exp C -a C N .

=

  σ i σ j + 4t N σ i σ j σ i σ j + N i=1 2t N W ii + 2t -tN,where d denotes equality in distribution. Applying this with t = λ

Exercise 4 . 12 .I N = inf h⩾0 λ 4 + h 2 λ

 41242 By Theorem 4.25, lim N→+∞ ψ(λ ) (S.28)for the initial condition ψ ∶ R ⩾0 → R defined byψ(h) ∶= Elog ˆΣ1 exp √ 2hσ z 1 + 2hσ σ 1h dP 1 (σ ).

Exercise 5 . 2 .

 52 We start by showing that for every open set U ⊆ S, the mapping λ ↦ λ (U) is measurable. Since U is open, the sequence ( f n ) n⩾1 of continuous functions defined by f n (s) ∶= min(1,ndist(s,U c )) increases to 1 U . It follows by the monotone convergence theorem that λ (U) = ˆS 1 U dλ = lim n→+∞ ˆS f n dλ .

+∞ n=0 2 -

 2 n φ ˆS f n d(λλ ′′ ) .Since φ is non-decreasing on R ⩾0 , we haveρ(λ ,λ ′′ ) ⩽ +∞ n=0 2 -n φ ˆS f n d(λλ ′ ) + ˆS f n d(λ ′λ ′′ ) .To bound this further, observe that by concavity of φ , for any a > 0 and b ⩾ 0, we haveφ (a + b)φ (b) a ⩽ φ (a)φ (0) a ,and therefore φ (a + b) ⩽ φ (a) + φ (b). This inequality holds trivially for a = 0. It follows thatρ(λ ,λ ′′ ) ⩽ +∞ n=0 2 -n φ ˆS f n d(λλ ′ ) + +∞ n=0 2 -n φ ˆS f n d(λ ′λ ′′ )= ρ(λ ,λ ′ ) + ρ(λ ′ ,λ ′′ ).

Exercise 5 . 6 .

 56 By Proposition A.24 it suffices to show that, given a measurable set A ⊆ S with µ(∂ A) = 0 and t ⩾ 0, we have lim n→+∞ Eexp -tΛ n (A) = Eexp -tΛ(A) . (S.29) The proof very closely resembles that of the Portmanteau theorem. To alleviate notation, let U ∶= int(A) and F ∶= A. Define the sequence ( f m ) m⩾1 of bounded and continuous functions on S by f m (s) ∶= min 1,md(s,U c ) . Since U c is closed, the sequence ( f m ) m⩾1 increases to 1 U . It follows that for every n,m ⩾ 1, Eexp -tΛ n (U) ⩽ Eexp -t ˆS f m dΛ n . Letting n tend to infinity and then m tend to infinity shows that limsup n→+∞ Eexp -tΛ n (U) ⩽ liminf m→+∞ Eexp -t ˆS f m dΛ = Eexp -tΛ(U) .

  liminf n→+∞ Eexp -tΛ n (F) ⩾ limsup m→+∞ Eexp -t ˆS g m dΛ = Eexp -tΛ(F) .Combining these two asymptotic bounds reveals that Eexp -tΛ(F) ⩽ liminf n→+∞ Eexp -tΛ n (A) ⩽ limsup n→+∞ Eexp -tΛ n (A) ⩽ Eexp -tΛ(U) .

logEexp

  ) for a given smooth and compactly supported functiong ∈ C ∞ c (R ⩾0 × R >0 ;R ⩾0 ). The independence of (X k ) 1⩽k⩽n implies that Eexp -ˆR⩾0 ×R >0 gdΛ n = Eexp -g(k n,n -1 ζ X 1 ) .

1

 1 P{Λ[δ ,+∞) > K} = P{Π > K} for a Poisson random variable Π with mean µ[δ ,+∞) = δ -ζ . Using the convergence in law established in Proposition 5.15, find n ⩾ K such that P{X K,n > δ } ⩽ δ . To alleviate notation introduce the sets A ∶= n k=K X k,n ⩾ εn 1 ζ and B ∶= {X K,n ⩽ δ n 1 ζ }, and observe that P(A) = P(A ∩ B) + P(A ∩ B c ) ⩽ P(A ∩ B) + P(B c ) ⩽ P(A ∩ B) + δ . To bound this further we use a generalized version of Chebyshev's inequality,εn 1 ζ P(A ∩ B) = εn 1 ζ E1 A 1 B ⩽ n k=K EX k,n 1 A 1 B ⩽ n k=K EX k,n 1 B , {X k,n ⩽δ n 1 ζ } + δ .

Exercise 5 . 9 .

 59 Applying equality (5.55) with X n replaced by X n Z n reveals that Elog

( i )

 i Fix x = (x j ) j⩾1 ∈ V . Since ∑ +∞ j=1 x j ⩽ 1, there must exist m ⩾ 1 with x m ⩽ m -1 . It follows by non-decreasingness of the coordinates of x that p n ((x j ))x n-1 m ⩽ m 1-n .

Exercise 6 . 4 .

 64 For each t ∈ [0,1], define the interpolating Hamiltonian H N,t (σ ) ∶= 1

Exercise 6 . 5 .

 65 By Chebyshev's inequality, it suffices to show that limN→+∞ E ⟨F g ⋅ σ √ N ⟩ -E Z F g ⋅ ⟨σ ⟩ √ N + 1 -qZ 2 = 0. (S.[START_REF] Barbier | Mutual information for symmetric rank-one matrix estimation: a proof of the replica formula[END_REF]

  and observe thatE⟨1 { Γ(C N )-Γ(C) >ε} ⟩ ⩽ E⟨1 { Γ(C N )-Γ(C) >ε} 1 A δ ,N ⟩ + E⟨1 A c δ ,N ⟩. (S.[START_REF] Barbier | The layered structure of tensor estimation and its mutual information[END_REF]

  i ∖ A i-1 ) = lim n→+∞ µ(A n ).

FExercise A. 7 . 0 f

 70 ,b i ],then, up to relabelling, we may assume that a = a1 < b 1 ⩽ a 2 < ... < b n-1 ⩽ a n < b n = b, ,b i ] = F(b) -F(a) = n i=1 (b i ) -F(a i ) = n i=1 m F ((a i ,b i ]).It follows that for any two finite sequences ((a i ,b i ]) 1⩽i⩽n and ((c j ,d j ]) 1⩽ j⩽m of disjoint intervals with the same union,a i ,b i ] ∩ (c j ,d j ] = m F m ⋃ j=1 (c j ,d j ] .This shows that m F is well-defined. To prove that it is a pre-measure, fix a sequence (I n ) n⩾1 of disjoint intervals with I n = (a n ,b n ] whose union I belongs to A. Since the union of (I n ) n⩾1 is a finite union of intervals of the form (a,b], the sequence (I n ) n⩾1 can be partitioned into finitely many subsequences such that the union of the intervals in each subsequence is a single interval of the form (a,b]. Considering each subsequence separately and using the finite additivity of m F , assume without loss of generality that I = (a,b]. On the one hand, by finite additivity, m F (I) = m F n ⋃ i=1 I i + m F I ∖ Denote by ⟨⋅,⋅⟩ ∶ L 2 (C;C) × L 2 (C;C) → C the canonical inner product on L 2 (C;C), ⟨ f ,g⟩ ∶= ˆ2π (e it )g(e it )dt.Fix a polynomial P ∶ C → C of the form P(z) ∶= ∑ n j=0 a j z j for some complex coefficients (a j ) j⩽n ⊆ C. Observe that the functions f and P are orthogonal, ˆ2π0 f (e it )P(e it )dt = j+1)t dt = 0. It follows that 2π = ⟨ f , f ⟩ = ⟨ f , f -P⟩ ⩽ 2π f ∞ f -P ∞ ⩽ 2π f -P ∞ ,so f -P ∞ ⩾ 1 for any polynomial P. This completes the proof. Exercise A.8. Fix an open set U ⊆ S and let F = U c . Since d(x,F) = 0 if and only if x ∈ F, the sequence ( f m ) m⩾1 ⊆ C b (S) defined by f m (x) ∶= min(1,md(x,F)) increases pointwise to 1 U as m tends to infinity. It follows by the monotone convergence theorem that P(U) = Q(U). Invoking the Dynkin π-λ theorem completes the proof.

  + . Since d -⩽ d + are the endpoints of the interval spanned by ψ ′ , and since ψ ′ (0) = m, we must have that m ⩽ d + . Since ψ * is non-decreasing on [m,+∞), it follows that ψ The same argument also applies to the case when d -< +∞ and x = d -. We now pick an open set A ⊆ R. For every x ∈ A, we can find ε > 0 sufficiently small that [xε,x + ε] ⊆ A. We deduce from (2.75) that

			x→d+	ψ * (x),
			x<d+
	so again (2.75) holds. liminf N→+∞	1 N	logP S

If x ∉ [d -,d + ] this is immediate since in this case ψ * (x) = +∞, by a small variant of Exercise 2.12. Suppose that d + < +∞ and that x = d * (d + ) ⩾ lim N ∈ A ⩾ -ψ * (x). Taking the supremum over x ∈ A establishes (2.79) and completes the proof. ∎ Theorem 2.17 (Cramér). If (X n ) n⩾1 is a sequence of i.i.d. random variables with finite log-Laplace transform ψ ∶ R → R, then the sequence (S N ) N⩾1 of sample averages (2.61) satisfies a large deviation principle with rate function ψ * . Proof. Using that ψ takes only finite values, one can check that ψ must be continuously differentiable. The claim is then an immediate consequence of the general large deviation result in Theorem 2.16 and the equality (2.63) which relates the log-Laplace transform of S N to that of X 1 .

  ].Exercise 2.22. Let X be a Bernoulli random variable with parameter p ∈ [0,1], and denote by ψ its log-Laplace transform. Show that the convex dual of ψ is the rate function (2.51),

	Exercise 2.21. Show that (2.54) and (2.55) are equivalent to the string of inequalities
	(2.56).

  .112) For every n ⩾ 1 sufficiently large, we can pick y n ∈ O xn . Since O xn ⊆ K for every n sufficiently large, we can, after the extraction of a subsequence, assume that (y n ) n⩾1 converges to some point y ∈ R d . In fact, it suffices that we show (2.112) for every sequence (x n ,y n ) n⩾1 with x n ≠ x converging to x, with y n ∈ O xn , and with y n converging to some y ∈ R d . This can be seen by arguing by contradiction: if (2.112) fails to hold for some sequence (x n ) n⩾1 , then we can construct another sequence (x n ,y n ) n⩾1 with y n ∈ O xn , with (2.112) still failing, and with y n converging to some y. So we assume that these conditions hold from now on. For every y ′ ∈ R d , we have

	g(x n ,y ′ ) ⩽ sup	g(x n ,y) = g(x n ,y n ),
	y∈R d

  Proof. Fix t ∈ R ⩾0 and h ∈ R. A direct computation gives that for every m ∈ [-1,1],

	Proposition 2.22. If t ⩽ 1 2 , then for every h ∈ R, the function (2.114) has a unique maximizer m h (t) ∈ [-1,1]. On the other hand, if t > 1 2 , then for all h small enough, the function (2.114) has exactly two local maximizers m -h (t) ⩽ m + h (t) ∈ [-1,1]. In
	all cases, these local maximizers are solutions to the fixed point equation
	m = tanh(h + 2tm).	(2.115)
	For t > 1 2 , there exists m * (t) > 0 (not depending on h) such that for all h small
	enough,
	m -h (t) < -m * (t) < m * (t) < m + h (t).	(2.116)

  This means that ∂ m g t,h is a decreasing function that tends to +∞ as m approaches -1 and tends to -∞ as m approaches 1. It follows by the intermediate value theorem that g t,h has a unique critical point m h (t), and the sign of ∂ m g t,h on each side of this critical point reveals that it is a maximizer.

	where the last equality uses the symmetry of the fixed point equation (2.115) when
	h = 0. The qualitative behaviour of the mapping h ↦ m(t,h) is therefore similar to
	that in the two-dimensional Ising model depicted in Figure 1.1. The phase transition
	at the critical temperature t c = 1 2 is explored further in Exercise 2.24. By varying
	the choice of the function ξ and the sequence of measures P N in the generalized
	Curie-Weiss model, one can devise models with a wide variety of behaviours, such
	as multiple phase transitions [107].
	Exercise 2.23. Let f ∶ R d → R be a function such that
		lim x →+∞	f (x) x	= +∞.	(2.119)
				1 2 , then ∂ 2 m g t,h (m) ⩽ 0, with the inequality
	being strict for every non-zero m ∈ [-1,1]. In the case t > 1 2 , the function ∂ 2 m g t,h has exactly two zeros,
		±m * (t) ∶= ± 1 -	1 2t	,
	and it is positive on the interval (-m * (t),m * (t)) and negative on its complement.
	Since ∂ m g t,0 (0) = 0, the continuity of h ↦ ∂ m g t,h implies that for h small enough, the
	function ∂ m g t,h must have a zero on the interval (-m * (t),m * (t)). Since ∂ m g t,h tends
	to +∞ as m approaches -1, decreases for m ⩽ -m * (t), increases on (-m * (t),m * (t))
	and has a zero in this interval before decreasing to -∞ as m approaches 1, it must
	have two zeros m ± h (t) ∈ [-1,1] outside the interval (-m * (t),m * (t)). Analyzing the
	sign of ∂ m g t,h reveals that these are the only local maximizers of g t,h ; the critical
	point in the interval (-m * (t),m * (t)) is a local minimizer. This completes the
	proof.				∎
	If t > 1 2 and h > 0, then g t,h (m + h (t)) > g t,h (m -h (t)), so the envelope theorem and Proposition 2.22 imply that m(t,h) = ∂ h f (t,h) = m + h (t). The sequence (m + h (t)) h>0
	is uniformly bounded by one, and by (2.115) any of its subsequential limits as h ↘ 0
	must satisfy the fixed point equation m = tanh(2tm). Together with the separation
	bound (2.116), this means that it must be m + 0 (t). That is,
		lim h↘0	m(t,h) = m + 0 (t) > 0.	(2.117)
	A similar argument shows that
	lim h↗0	m(t,h) = m -0 (t) = -m + 0 (t) < 0,	(2.118)

  Let ( f N ) N⩾1 be a sequence of continuous functions on R d converging locally uniformly to a function f ∶ R d → R. If f has a strict local maximum at x ∈ R d , show that there exists r > 0 and a sequence of points (x N ) N⩾1 ⊆ R d converging to x such that, for every N sufficiently large, we have f N (x N ) = sup Br(x) f N .

	Exercise 3.1. Exercise 3.2. Show that in the definition of a viscosity subsolution, replacing "local
	maximum" by "strict local maximum" yields an equivalent definition.
	Exercise 3.3. Show that in the definition of a Lipschitz viscosity subsolution,
	replacing "local maximum" by "global maximum" or "strict global maximum"
	yields an equivalent definition.
	Exercise 3.4. Show that in the definition of a viscosity subsolution, replacing the
	regularity of test functions
	, the initial condition is satisfied.
	This completes the proof.	∎

  Fix z ∈ R d and ε > 0. Since uφ achieves a local maximum at (t α ,x α ) ∈ R >0 × R d , and u is uniformly Lipschitz continuous with Lipschitz constant L,

				.49)
	To compare these two quantities, we would like to use the local Lipschitz continuity
	of the non-linearity H. With the definition of V in mind, we must therefore verify
	that
	∇φ (t α ,x α ) ⩽ L	and	∇φ ′ (t ′ α ,x ′ α ) ⩽ L.	(3.50)

  .[START_REF] Bertsekas | Convex optimization theory[END_REF] More precisely, the Hopf-Lax and Hopf formulas are representations of the value of the solution to the Hamilton-Jacobi equation(3.20) at the point (t,x) ∈ R ⩾0 × R d as saddle-point problems over the variables (y, p) ∈ R d × R d . Notice that for each fixed (y, p) ∈ R d × R d , the mapping (t,x) ↦ J t,x (y, p) is a solution to (3.20), so we can think of the Hopf-Lax and Hopf formulas as representations of the solution with initial condition ψ as envelopes of this family of solutions. At a point of differentiability of this envelope function, the function will be tangent to one particular solution in this set indexed by (y, p) ∈ R d × R d , and since the equation is of first order, a function that is tangent to a smooth solution at a point must solve the equation at that point. Of course, this is only an informal discussion, and the purpose of the next two sections is to prove the Hopf-Lax and Hopf formulas rigorously.

  to f with respect to the topology of local uniform convergence, Exercise 3.1 allows us to find a sequence (t N ,h N ) N⩾1 converging to (t * ,h * ) such that F N -φ has a local maximum at (t N ,h N ) for each integer N ⩾ 1. The bound (3.17) gives a constant C < +∞ with

  the enriched free energy (4.75) in the symmetric rank-one matrix estimation problem. For every t,h ⩾ 0, we have that F N (t,h) converges to f (t,h) as N tends to infinity, where f ∶ R ⩾0 × R ⩾0 → R is the unique viscosity solution to the Hamilton-Jacobi equation (4.84) subject to the initial condition

  .111) By Rademacher's theorem (Theorem 2.10), the set D is dense in R ⩾0 , and by the envelope theorem (Theorem 2.21), if h * (t) ⩾ 0 is any maximizer of the right side of (4.110), then we have for every t ∈ D that

	∂ t f (t) =	h 2 * (t) 4t 2 .	(4.112)

  .114)

	Proof. Proposition 4.4 and the asymptotic expansion (4.19) imply that

  Proposition 4.[START_REF] Daniel | Spin-glass models of neural networks[END_REF]. Fix the Rademacher prior (4.124). If t <1 4 , then mmse(t) = 1. If t is large enough and is a point of differentiability t ∈ D of the limit free energy (4.110), then >1 4 , and then we prove (4.125) by showing that h * (t) is essentially 2t for t large enough.Step 1: h * (t) = 0 for t ⩽ 1 4 while h * (t) > 0 for t > 1 4 . The derivative computation (4.82) for N = 1, the Nishimori identity, and the explicit form of the Rademacher prior (4.124) imply that

	mmse(t) <	1 4t	2 -	1 4t	.	(4.125)
	Remark 4.20. For t sufficiently large, a minor modification of the proof of Proposi-
	tion 4.19 also shows that
	limsup N→+∞	mmse N (t) <	1 4t	2 -	1 4t	,
	without having to impose that t ∈ D.

Proof of Proposition 4.19. The proof proceeds in two steps. Writing h * (t) for any maximizer of (4.110), first we show that h * (t) = 0 for t ⩽ 1 4 while h * (t) > 0 for t

  c = One can easily check that any choice of maximizer h * (t) of (4.110) must diverge to infinity as t tends to infinity. Using that h * (t) exceeds a large constant for t sufficiently large, we deduce from the previous display that h * (t) in fact grows linearly with t, and appealing again to the previous display, we obtain that for t sufficiently large, we have h

		h * (t) 2 to bound P A h * (t) reveals that
	h * (t) -2t ⩽	16t πh * (t)	exp -	h * (t) 4	+ 8t exp -2h

* (t) . * (t) -2t ⩽ exp(-t 4). It follows by Proposition 4.15 that for t large enough,

  1 4 such that for every point of differentiability t ∈ D of the limit free energy (4.110) with t > t * (p), we have Remark 4.22. One can also show that for every t > t The dotted line is the minimal mean-square error for the sparse prior (4.127) with p = 0.05 while the solid line is the mean-square error achieved using PCA.Proof of Proposition 4.21. A direct computation using the explicit form of the sparse prior(4.124) shows that

	mmse(t)
	1
	t
	1
	4
	Figure 4.2
	mmse(t) < 1.	(4.128)

* (p), we have limsup N→+∞ mmse N (t) < 1, without having to impose that t ∈ D.

  completes the proof. ∎ Remark 5.5. If S is an open subset of R d equipped with the Euclidean topology, then Proposition 5.4 is also valid if we only assume that (5.16) holds for every f ∈

	C ∞ c (S;R ⩾0

  .[START_REF] Barra | On the equivalence of Hopfield networks and Boltzmann machines[END_REF] converges in law on R ⩾0 ×R >0 to the Poisson point process with intensity dµ(t,x) ∶= Let Λ be a Poisson point process with intensity dµ(x) ∶= ζx ζ +1 dx, and recall that there exist u 1 ⩾ u 2 ⩾ ⋯ such that Λ = ∑

	(i) Using that ζ < 1, show that ´xdΛ(x) = ∑ +∞ n=1 u n is finite almost surely. For
	every n ⩾ 1, we set	v n ∶=	u n +∞ ∑ k=1 u k	.
	cζ x ζ +1 dt dx.
	Exercise 5.8. Fix ζ ∈ (0,1), and let (X n ) n⩾1 be i.i.d. random variables taking values
	in R ⩾0 such that, as x tends to infinity,
		P{X 1 ⩾ x} ∼ x -ζ .
		+∞ n=1 δ un .

  Proposition 5.18. The ordered Poisson-Dirichlet point process (u n ) n⩾1 with parameter ζ > 0 is almost surely summable if and only if ζ ∈ (0,1). Moreover, if ζ ∈ (0,1), then for every 0 < a < ζ ,

	+∞	a
	E	u n	< +∞.	(5.48)
	n=1
	Proof. To see that (u n ) n⩾1 is summable if and only if ζ ∈ (0,1), recall the repre-
	sentation (5.47) of the Poisson point process Λ with intensity measure (5.46), and
	observe that

  Theorem 5.[START_REF] Daniel | Spin-glass models of neural networks[END_REF]. Let (X n ,Y n ) n⩾1 be i.i.d. pairs of random variables on R >0 × R d independent of the ordered Poisson-Dirichlet point process (u n ) n⩾1 , with the property that EX

				ζ	,	(5.52)
				1
	and let (Y ′ n ) n⩾1 be a sequence of i.i.d. random variables with law ν ζ independent of
	(u n ) n⩾1 . The Poisson point processes
	+∞		+∞
	n=1	δ (unXn,Yn) and	n=1	δ ((EX ζ

ζ 1 < +∞. Let ν ζ be the probability measure on R d such that ν ζ (B) ∶= EX ζ 1 1 {Y 1 ∈B} EX 1 ) 1 ζ un,Y ′ n )

  Invoking the induction hypothesis completes the proof. ∎ Definition 5.24 (Poisson-Dirichlet cascade). The Poisson-Dirichlet cascade with parameters (5.67) is the family (v α ) α∈N K obtained by normalizing the (w α ) α∈N K , that is, for every

  leveraging the continuity of f and the definition (A.31) of the outer measure µ * , find an open setU i containing E i with ρ(U i ) ⩽ µ(E i ) + ε n and f (x) < y i + ε on U i . (A.34) Invoke Lemma A.8 to find a partition of unity (h i ) i⩽n on K subordinate to the open cover (U i ) i⩽n . Since f = ∑ n i=1 f h i and f h i ⩽ (y i + ε)h i , the linearity of T implies that

  let ε tend to zero completes the proof. Let (P n ) n⩾1 be a sequence of probability measures on R, and let P be a probability measure on R. Denote by F n ∶ R → [0,1] and F ∶ R → [0,1] the distribution functions associated with P n and P, F Prove that (P n ) n⩾1 converges weakly to P if and only if at every point t of continuity of F, the numerical sequence (F n (t)) n⩾1 converges to F(t).

	(i)
	(A.53)
	Show that P = Q.
	Exercise A.9. (A.54)

∎

Remark A.

[START_REF] Daniel | Modeling brain function[END_REF]

. A similar argument as that used to show that (ii) ⇒ (iii) can be used to prove that (ii) ⇒ (iv). For future reference, let us present this argument in the more general setting where we have a sequence of finite measures (µ n ) n⩾1 and a finite measure µ with the property that for every f ∈ C b (S;R),

lim n→+∞ ˆS f dµ n = ˆS f dµ. (A.51)

Given a closed set F ⊆ S, define the sequence of bounded and Lipschitz functions (g m ) m⩾1 on S by g m (s) ∶= max(0,1md(x,F)). Since F is closed, the sequence (g m ) m⩾1 decreases to 1 F . It follows that ˆS g m dµ n ⩾ µ n (F).

(A.52)

Using (A.51) to let n tend to infinity and then the dominated convergence theorem to let m tend to infinity shows that limsup n→+∞ µ n (F) ⩽ µ(F) as required.

Exercise A.8. Consider two probability measures P and Q on S such that, for every f ∈ C b (S;R), we have ˆS f dP = ˆS f dQ. n (t) ∶= P((-∞,t]) and F(t) ∶= P((-∞,t]).

  Proposition A.[START_REF] Auffinger | Random matrices and complexity of spin glasses[END_REF]. A sequence (P n ) n⩾1 of probability measures on R d ⩾0 converges weakly to a probability measure P on R d ⩾0 if and only if, for every λ ∈ R d ⩾0 ,

	lim n→+∞ ˆRd ⩾0	e -λ ⋅x dP n (x) =	ˆRd ⩾0	e -λ ⋅x dP(x).	(A.65)

  x) dx It follows by the Cauchy-Schwarz inequality that T defines a continuous linear functional on the space of step functions on L 2 ([a,b];R). By density, it extends to a unique continuous linear functional on the Hilbert space L 2 ([a,b];R). Invoking the Riesz representation theorem on Hilbert spaces established in Exercise 2.17 gives a unique function f ∈ L 2 ([a,b];R) with

	T (g) =	ˆb a	f (t)g(t)dt

for all g ∈ L 2 ([a,b];R). If we now fix x ∈ [a,b],

and apply this representation to the step function g ∶= 1 [a,x) we find that F

  8) and completes the proof. Exercise 2.21. A direct computation shows that the rate function I in (2.51) is decreasing on (0, p) and increasing on (p,1). Using this, it is readily verified that the string of inequalities (2.56) implies (2.54) and (2.55). Conversely, let us assume (2.54) and (2.55), and deduce (2.56). Fix a Borel set A ⊆ R, and observe that P{S

N ∈ A} ⩽ P S N ∈ A ∩ (-∞, p] + P S N ∈ A ∩ [p,+∞) .

Denote by x ∈ A the infimum of A ∩ [p,+∞). Since x ⩾ p, applying (2.54) reveals that limsup N→+∞

  Exercise 3.3. Arguing as in Exercise 3.2, it suffices to prove that the definition of subsolution with "global maximum" implies the one with "local maximum". Let f ∶ R ⩾0 × R d → R be a subsolution for the definition with "global maximum", and letφ ∈ C ∞ (R >0 × R d ;R) be such that fφ has a local maximum at (t * ,x * ) ∈ R >0 × R d . Denote by B r (t * ,x * ) an open ball centred at (t * ,x * ) on which fφ attains its maximum at (t * ,x * ). Let θ ∶ R ⩾0 × R d →R be a non-negative smooth function with θ = 1 on B r 2 (t * ,x * ) and θ = 0 outside B 2r 3 (t * ,x * ), and define the parabolaχ(t,x) ∶= (t -t * ) 2 + xx * 2 .and define the smooth function η ∶ R >0 × R d → R by

	η(t,x) ∶= φ (t,x)θ (t,x) +	L r	+	M r 2 +	2MM ′ r	χ(t,x).
	Introduce the constants
	L ∶= sup

x) ⩽ ( fφ )(t * ,x * )ε t -t * 2 + xx * 2 < ( fη)(t * ,x * ), so (t * ,x * ) is a strict local maximum of fη. It follows that ∂ t φ -H(∇φ ) (t * ,x * ) = ∂ t η -H(∇η) (t * ,x * ) ⩽ 0

which is the required subsolution criterion for the definition with "local maximum".

t>0 f (t,⋅) Lip , M ∶= sup Br(t * ,x * ) φ , M ′ ∶= sup Br(t * ,x * ) ∂ t θ + ∇θ 2 ,

  ,x * ) ∖ B r 2 (t * ,x * ), ( fη)(t,x) ⩽ ( fφ )(t,x) + φ (t,x) θ (t,x)θ (t * ,x * ) -

							2MM ′ r	χ(t,x)
	MM ′ r 2		t -t * r 2	+	x -x * r 2	-	2MM ′ r	χ(t,x)
								L r	+	M r 2 χ(t,x)
	r	+	x -x * r		+ M -	L r	+	M r 2 χ(t,x)

* ⩽ ( fφ )(t * ,x * ) + ⩽ ( fη)(t * ,x * ),

where we used the mean value theorem in the second inequality. Finally, for any

(t,x) ∈ R d ∖ B r (t * ,x * ), ( fη)(t,x) ⩽ ( fφ )(t * ,x * ) + f (t,x)f (t * ,x * ) + φ (t * ,x * ) -⩽ ( fη)(t * ,x * ) + Lr t -t * ⩽ ( fη)(t * ,x * ).

  It will be convenient to define the functionh ∶ R N × R N 2 byConditionally on the randomness of x, the random variable Y is Gaussian with density It follows that the right side of (S.21) is given byˆRN ˆRN ˆRN 2 f (x ′ )g(y)exp(h(x ′ ,y) + h(x,y)) (2π) N 2 2 ´RN exph(x ′′ ,y)dP N (x ′′ ) e -1 2 y 2 dydP N (x ′ )dP N (x).The change of variables x = x ′ reveals that this is equal toˆRN ˆRN ˆRN 2 f (x)g(y)exp(h(x ′ ,y))exp(h(x,y)) (2π) N 2 2 ´RN exph(x,y)dP N (x)Remembering (S.22) establishes (S.21) and completes the proof.Exercise 4.3. A direct computation leveraging the equality (S.22) reveals that for any two Borel setsA ⊆ R N 2 and B ⊆ R N , ˆB ˆA dP x Y dP Y = ˆB ˆA dP x Y dP x dP x dP Y . P x )dP Y = EH(P x Y P x ) = NI N (t) + H(P x,Y P x ⊗ P Y ).Remembering the definition of the mutual information in terms of the relative entropy and observing that the term dP Y dP W in the first integral is independent of the coordinate associated with the law of x simplifies this toH(P x,Y P x,W ) = H(P Y P W ) + I(x;Y )where the second equality uses (S.22). Since the conditional law of Y given x is Gaussian with mean 2tN xx * , conditionally on the randomness of x, we have

	(ii) By independence of x and W , so we also have
	○ N (t,x)dP N (x) N (t,x)dP N (x) ´RN expH ○ h(x,y) ∶= 2t N x ⋅ yx -t N x 4 . ⎛ ⎝ -1 2 y-2t N xx * 2 ⎞ ⎠ = 1 dP x,Y exp dP x,W = exp 2t N x ⋅ yx -t N x 4 , H(P x Y as desired. 1 (2π) N 2 2 I(x;Y ) = Similarly, (S.23) implies that ˆRN 2 (2π) N 2 2 exp h(x,y) -g(Y ). 1 2 y 2 . e -1 2 y 2 dydP N (x ′ )dP N (x) (S.21) (S.22) = 1 (2π) N 2 2 ˆRN ˆRN 2 f (x)g(y)exp h(x,y) -1 2 dP x,Y dP x,W = dP x ⊗ P Y dP x ⊗ P W dP x,Y dP x ⊗ P Y = dP Y dP W H(P x,Y P x,W ) dP x,Y dP x ⊗ P Y . = ˆRN 2 ˆRN exp 2t N x ⋅ yx -t N x 4 2t N x ⋅ yx -t x 4 dP x (x)dP W (y) N y 2 dP N (x)dy. P{Y ∈ A,x ∈ B} = 1 (2π) N 2 2 ˆB ˆA exp -1 2 y -2t N xx * 2 dydP x (x) = 1 (2π) N 2 2 ˆB ˆA exp 2t N x ⋅ yx -t N x 4 e -1 2 y 2 dydP x (x) = ˆA ˆB exp 2t N x ⋅ yx -t N dP x,Y dP x ⊗ P Y = dP x , and therefore I(x;Y ) = ˆRN ×R N 2 log dP x Y dP x dP x,Y = ˆRN 2 ˆRN log dP x Y Remembering the definition of the mutual information (4.4) shows that = N H(P Y P W ). 1 dP x dP x Y dP Y . = 1 N ˆRN 2 log exp 2t N x ⋅ yx -= 1 log dP Y dP W (y) dP Y (y) N ˆRN 2 N x 4 dP x (x)dP Y (y) t ˆRN dP x Y (iii) Applying (S.23) with B = R N shows that dP Y dP W (y) = ˆRN exp 2t N x ⋅ yx -t N x 4 dP It follows that F ○ N (t) = 1 N Elog ˆRN exp 2t N x ⋅Y x -t N x 4 dP x (x) x 4 dP This means that It follows that H(P x,Y P x,W ) = ˆlog dP Y dP W dP x,Y + ˆlog dP x,Y = ˆRN ˆRN 2 2t N x ⋅ yx -t x 4 dP Y x (y)dP x (x) N dP x,Y dP x ⊗ P Y = ˆlog dP Y dP W = E x E Y x 2t N x ⋅Y x -t x 4 , N E Y x 2t N x ⋅Y x = 2t N x ⋅ xx * x = 2t N x 4 , and therefore H(P x,Y P x,W ) = t N E x 4 dP x,Y as required. as desired.

x (x)dP W (y). (S.23) We now treat each question separately. (i) Given two Borel sets A ⊆ R N 2 and B ⊆ R N , the definition of the conditional expectation implies that P{(x,Y ) ∈ A × B} = x (x).

  ..,n} consisting of k -1 sets. Putting this all together reveals thatS(n 1 + 1,n 2 ,...,n k ) = 1 n S(n 1 ,...,n k )E⟨R 1,2 ⟩ + 1 n ∈I 1 ∖{1} S(n 1 ,...,n k ) ,...,n j + n 1 ,...,n k ).Combining like terms and remembering that R 1,2 is a Bernoulli random variable with probability of success 1ζ by (5.58) yields S(n 1 + 1,n 2 ,...,n k ) = n 1 -ζ n S(n 1 ,...,n k ) + ,...,n j + n 1 ,...,n k )

	+ S(n 2 as required. k 1 n j=2 ∈I j S(n 2 k j=2 n j n

  Eg i j ⟨σ i σ j ⟩ t Eg i j ⟨σ i σ j ⟩ t .(S.30)

						N
					2	√ 1 -t	i, j=1
	⩽	N i, j=1	β 2N 3 2 √ t	Ex i j ⟨σ i σ j ⟩ t -	β 2N 3 2 √ 1 -t

  1 2 E⟨σ 2 ⟩ 2 + 2E⟨σ 3 ⟩⟨σ ⟩ -2E⟨σ 2 ⟩⟨σ ⟩ 2 -E⟨σ 4 ⟩ . = 6E⟨σ ⟩ 4 -E⟨σ 4 ⟩ + 4E⟨σ 3 ⟩⟨σ ⟩ + 3E⟨σ 2 ⟩ 2 -12E⟨σ 2 ⟩⟨σ ⟩ 2 -2 E⟨σ 2 ⟩ 2 + 2E⟨σ 3 ⟩⟨σ ⟩ -2E⟨σ 2 ⟩⟨σ ⟩ 2 -E⟨σ 4 ⟩ + E⟨σ 2 ⟩ 2 -E⟨σ 4 ⟩ = 6E⟨σ ⟩ 4 + 2E⟨σ 2 ⟩ 2 -8E⟨σ 2 ⟩⟨σ ⟩ 2 = 2E ⟨σ 2 ⟩ -⟨σ ⟩ 2 ⟨σ 2 ⟩ -3⟨σ ⟩ 2 .as required. We now consider the probability measure P 1 ∶= pδ 1 + (1p)δ -1 for a choice of p to be determined. It will be important to observe that

	It follows that
	∂ 2 t f ⟨σ ⟩ =	pexp( pexp(	√ 2tz) -(1 -p)exp(-√ 2tz) √ 2tz) + (1 -p)exp(-√ 2tz)	.

  [START_REF] Aizenman | On the stability of the quenched state in mean-field spin-glass models[END_REF]. By the Portmanteau theorem, it suffices to show that for any closed set F ⊆ S, we have limsup n→+∞ P{Y n ∈ F} ⩽ P{Y ∈ F}. Fix a closed set F ⊆ S as well as ε > 0, and letF ε ∶= {x ∈ S d(x,F) ⩽ ε} be the ε-neighbourhood of F. Observe that P{Y n ∈ F} ⩽ P{Y k,n ∈ F ε } + P{d(Y k,n ,Y n ) ⩽ ε}.Since (Y k,n ) n⩾1 converges in law to Z k and F ε is closed, the Portmanteau theorem implies that limsup n→+∞

The reader who wonders whether the integral in (6.117) is finite can derive from (6.62) that, assuming ∂ q f (t,q,⋅) is well-defined, it must be uniformly bounded.

Acknowledgements

This book grew out of lectures given by JCM at the 2021 CRM-PIMS summer school in Montreal, at ETH Zurich in 2022, and to graduate students at New York University and ENS Lyon. The two of us (TD and JCM) met on the occasion of the summer school in Montreal and have been working together since then. We are extremely grateful to the organizers of these events for their support, including financial support to TD from ETH Zurich to take part in the writing of this book. We also very warmly thank the participants of these two events and of the graduate courses for their interest and very useful feedback.

(iii) if A ∈ A, then A c ∈ A.

A collection S of subsets of S is called a σ -algebra if it is an algebra which is closed under countable unions, (iv) if (A n ) n⩾1 is a sequence of sets in S, then ⋃ n⩾1 A n ∈ S.

To verify that an algebra A is a σ -algebra, it suffices to verify that it is countably additive on disjoint sets, (iv') if (A n ) n⩾1 is a sequence of disjoint sets in S, then ⋃ n⩾1 A n ∈ S. Indeed, if (A n ) n⩾1 is a sequence of sets in S, then the sequence

of disjoint sets is also in S and has the same union as (A n ) n⩾1 . The simplest example of a σ -algebra is the σ -algebra generated by a collection E of subsets of S,

This is the smallest σ -algebra containing E and it is well-defined as the intersection of a collection of σ -algebras is again a σ -algebra. We will typically endow a metric space S with its Borel σ -algebra. The Borel σ -algebra on a metric space S is the σ -algebra generated by all open sets,

B(S) ∶= σ ({U U is open in S}). (A.3)

A measure µ on a measurable space (S,S) is a non-negative set function µ ∶ S → [0,+∞] which assigns no mass to the empty set and is countably additive, (i) µ(∅) = 0, (ii) for every sequence

A measure µ is said to be a finite measure if µ(S) < +∞, and a finite measure P is said to be a probability measure if P(S) = 1. The simplest example of a measure is the Dirac measure at a point x ∈ S defined on the measurable space (S,2 S ) by δ x (A) ∶= 1 A (x).

(A.5)

Thinking back to the geometric problem of determining length, we would like our first non-trivial example of a measure to be the measure m on the real line which gives the length of a set. Although there are many sets whose length we do not know a priori, we know the value of the measure m on the algebra (A.7)

We will first construct a measure on the Borel σ -algebra B(R) which satisfies (A.7) using the Caratheodory extension theorem, and then we will prove that such a measure must be unique using the Dynkin π-λ theorem. This strategy of constructing a measure by first specifying its value on a large enough but simple collection of sets such as an algebra, then establishing its existence on the σ -algebra generated by this collection of sets using the Caratheodory extension theorem, and finally obtaining its uniqueness using the Dynkin π-λ theorem is very general. The idea behind the Caratheodory extension theorem is to first define an outer measure on the collection 2 S of all subsets of S, and then restrict this outer measure to the σ -algebra of Caratheodory measurable sets where it becomes a measure. An outer measure µ * on S is a non-negative set function µ * ∶ 2 S → [0,+∞] which assigns no mass to the empty set, is monotone and countably sub-additive, (i) µ * (∅) = 0, (ii) for all subsets A,B of S with A ⊆ B, µ * (A) ⩽ µ * (B), (A.8) (iii) for every sequence (A n ) n⩾1 of subsets of S,

The most common way to define an outer measure is to start with a collection E ⊆ 2 S of elementary sets and a non-negative set function ρ ∶ E → [0,+∞], and define the set function µ * ∶ 2 S → [0,+∞] by

Provided that ∅,S ∈ E and ρ(∅) = 0, this set function µ * is an outer measure. The assumption S ∈ E ensures that the infimum in (A.10) is never taken over the empty set, so µ * is well-defined.

A direct application of this result gives a measure m on the Borel σ -algebra B(R) which satisfies (A.7) on the algebra (A.6). We would like to say that this is the measure on the Borel σ -algebra B(R) which gives the length of a set. We therefore need to establish the uniqueness of such a measure. This will be done using the Dynkin π-λ theorem. A π-system is a collection P of sets that is closed under intersections,

A λ -system is a collection L of sets that contains the set S, is closed under complements and is closed under increasing unions,

If L is both a π-system and a λ -system, then it is a σ -algebra.

Proof. Since a λ -system is closed under complements, it suffices to show that L is closed under countable unions. Fix a sequence (A n ) n⩾1 of sets in L, and consider the increasing sequence

An induction relying on the fact that

2 ) c and the assumption that L is a π-system shows that (B n ) n⩾1 ⊆ L. Observing that (B n ) n⩾1 increases to ⋃ ∞ n=1 A n and remembering that L is a λ -system completes the proof. ∎ Theorem A.5 (Dynkin π-λ ). If P is a π-system and L is a λ -system with P ⊆ L, then σ (P) ⊆ L.

(A.15)

Proof. Denote by (P) the smallest λ -system containing P. By minimality of (P) and σ (P), it suffices to prove that (P) is a σ -algebra. Indeed, if this is the case, then σ (P) ⊆ (P) ⊆ L. By Lemma A.4 it is in fact sufficient to show that (P) is a π-system. For each A ∈ (P) define the λ -system

Since P is a π-system, for every A ∈ P, we have P ⊆ G A . It follows by minimality of (P) that (P) ⊆ G A . This means that for every A ∈ P and B ∈ (P), we have A ∩ B ∈ (P). In other words, for every B ∈ (P), we have P ⊆ G B . It follows by minimality of (P) that for every B ∈ (P), we have (P) ⊆ G B . This shows that (P) is a π-system and completes the proof. ∎

As a direct application of the Dynkin π-λ theorem, we can now show that there is a unique σ -finite measure on the Borel σ -algebra B(R) which corresponds to our intuition about the length of a set. A measure µ on a measurable space (S,S) is σ -finite if there exists a sequence of sets (A n ) n⩾1 ⊆ S which cover S and have finite measure,

A n and µ(A n ) < +∞ for all n ⩾ 1.

(A.16)

Corollary A.6. There is a unique measure m on the Borel σ -algebra

on the algebra (A.6) of finite unions of disjoint intervals. This measure is the Lebesgue measure.

Proof. Notice that any measure satisfying (A.17) must be σ -finite. The existence of a measure m satisfying (A.17

forms a λ -system. Since this λ -system contains the π-system (A.6) and this πsystem generates the Borel σ -algebra, the Dynkin π-λ theorem implies that L n = B(R). It follows that for every n ⩾ 1 and Borel set B ∈ B(R),

Using the continuity from below of measure established in Exercise A.1 shows that µ = ν on B(R) and completes the proof. ∎

The Caratheodory extension theorem and the Dynkin π-λ theorem can be combined to uniquely define many measures by first defining them on a simpler class of sets. For instance, product measures can be constructed by defining them on the algebra of finite disjoint unions of rectangles. In the next section, we will combine these two results to establish the Riesz representation theorem by extending a set function from the collection of open sets to the Borel σ -algebra. Another important consequence of the Dynkin π-λ theorem is that every finite measure on the Borel σ -algebra of a metric space is closed regular. A measure µ is closed regular if for every measurable set A ⊆ S,

For a finite measure, taking complements shows that being closed regular is equivalent to being open regular. A measure µ is open regular if for every measurable set

The fact that every finite measure on the Borel σ -algebra of a metric space is closed regular is the content of Exercise A.4.

Exercise A.1. Fix a measure µ on a measurable space (S,S).

(i) Show that µ is monotone in the sense that for all measurable sets A,B ∈ S with A ⊆ B, we have µ(A) ⩽ µ(B).

(ii) Show that µ is continuous from below in the sense that for any increasing sequence of measurable sets

Exercise A.2. Let F ∶ R → R be an increasing and right-continuous function, and write A for the algebra (A.6) of finite unions of disjoint intervals on R. Show that the non-negative set function m F ∶ A → [0,+∞] defined by

is a well-defined pre-measure.

Exercise A.3. Let P be a π-system of sets in S and let µ and ν be measures on the measurable space (S,σ (P)) which coincide on P. Show that for any A ∈ P with µ(A) < +∞, the collection of sets

forms a λ -system. Deduce that a finite measure on (S,σ (P)) is determined by its values on P.

Lemma A.8 (Partition of unity). If K is a compact subset of a metric space S and (U i ) i⩽n is an open cover of K, then there exist continuous functions (h i ) i⩽n with n i=1 h i = 1 on K and 0

The collection of functions (h i ) i⩽n is called a partition of unity on K subordinate to the open cover (U i ) i⩽n .

Proof. Fix x ∈ K and let 1 ⩽ i ⩽ n be such that x ∈ U i . Denote by B r(x) (x) an open ball centred at x of small enough radius r(x) > 0 so that x ∈ B r(x) (x) ⊆ U i . The collection of sets (B r(x) (x)) x∈K forms an open cover of the compact set K from which it is possible to extract a finite sub-cover (B r(x j ) (x j )) j⩽m . Let V i be the union of the closed balls B r(x j ) (x j ) which lie in the open set U i , and invoke Urysohn's lemma to find a collection of continuous functions

To modify the functions (g i ) i⩽n so their sum becomes one on K, apply Urysohn's lemma once again to find a continuous functions g with

.

Define g n+1 ∶= 1g so that ∑ n+1 i=1 g i > 0 on S, and define the family of functions

Since the support of h i coincides with that of g i , we have that 0 ⩽ h i ⩽ 1 U i . Moreover, as g n+1 = 0 on K, we have that ∑ n i=1 h i = 1 on K. This completes the proof. ∎ Theorem A.9 (Riesz representation). If S is a compact metric space and T ∶ C(S;R) → R is a positive linear functional on the space of continuous functions on S, then there exists a unique measure µ on the Borel σ -algebra B(S) such that for all continuous functions f ∈ C(S;R),

Proof. We proceed in four steps. First, we will establish the uniqueness of µ by showing that it is entirely determined by T on open sets. We will then define an outer measure µ * on 2 S by leveraging the value that µ should attribute to open sets. The Caratheodory extension theorem will then guarantee that the restriction µ of this outer measure µ * to the Borel σ -algebra is a measure. Finally we will show that T can be identified with µ through equation (A.29).

Step 1: uniqueness of µ. Fix a measure µ on B(S) for which (A.29) holds, and fix sets and that µ * is therefore an outer measure. The monotonicity of ρ ensures that this outer measures agrees with ρ on the collection of open sets.

Step 3: µ * restricts to a measure µ on B(S). Since open sets generate the Borel σ -algebra, by the Caratheodory extension theorem, to show that the restriction µ of the outer measure µ * to the Borel σ -algebra is a measure, it suffices to prove that every open set is Caratheodory measurable. By sub-additivity of outer measure this comes down to showing that for every open set U and every subset E of S with µ * (E) < +∞, we have

Given ε > 0, let V be an open set with E ⊆ V and ρ(V ) ⩽ µ * (E)+ε. By monotonicity of outer measure and the fact that µ * coincides with ρ on the collection of open sets,

To bound this further, find

Denote by K ⊆ U the compact support of f , and observe that by monotonicity of outer measure,

where we have used that

As the supports of f and g are disjoint and both contained in V , we have that

Letting ε tend to zero establishes (A.32) and shows that µ ∶= µ * B(S) defines a measure on the Borel σ -algebra B(S).

Step 4: T can be identified with µ. To show that T can be identified with the measure µ through equation (A.29), up to replacing f byf , it suffices to prove that for every f ∈ C(S;R),

Moreover, up to replacing f by f + f ∞ , it suffices to establish (A.33) for nonnegative functions f ∈ C(S;R). Fix a non-negative f ∈ C(S;R), and write K for its compact support. Since f is bounded, its range belongs to some bounded interval Proof. For each f ∈ F, we have

so the set F R of real and imaginary parts of functions in F is contained in F, and it is an algebra of real-valued functions which separates points and contains constants. It follows by the real version of the Stone-Weierstrass theorem that F R is dense in (C(S;R), ⋅ ∞ ). Since every function f ∈ C(S;C) may be written as Re( f ) + iIm( f ) for the real-valued functions Re( f ),Im( f ) ∈ C(S;R), and a sequence of complex-valued functions converges with respect to ⋅ ∞ if and only if its associated sequences of real and imaginary parts converge with respect to ⋅ ∞ , we have that F is dense in (C(S;C), ⋅ ∞ ). This completes the proof. ∎ (i) Argue that there is no loss in generality in considering the interval [0,1].

be a sequence of independent and identically distributed random variables with X 1 ∼ Ber(x).

For each n ⩾ 1, consider the sample average S n ∶= 1 n ∑ n i=1 X i , and define the function P n (x) ∶= E f (S n ). Prove that P n is a polynomial and that for n large enough, we have f -P ∞ ⩽ ε.

Exercise A.7. Denote by C ∶= {e it t ∈ [0,2π]} ⊆ C the unit circle, and consider the complex-valued function f ∶ C → C defined by f (z) ∶= z. Prove that f cannot be approximated in (C(C,C), ⋅ ∞ ) by polynomials (of the z variable).

A.4 The Lebesgue differentiation theorem

Before moving from the realm of analysis to that of probability theory, we prove the Lebesgue differentiation theorem on Euclidean space S = R d . In the one-dimensional setting, d = 1, this result generalizes the fundamental theorem of calculus.

is differentiable and F ′ = f .

To use the Hardy-Littlewood maximal operator to establish the almost everywhere convergence (A.38), we will leverage the fact that it is weak-(1,1). An operator

It turns out that whenever the maximal operator of a family of linear operators is weak-(1,1), the space of functions where the sequence of operators converges pointwise to the identity is closed in L 1 (R d ;R). In particular, if it contains a dense subspace such as the space of continuous functions, it must be all of L 1 (R d ;R).

Proposition A. [START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF]. If (T t ) t>0 is a family of linear operators from L 1 (R d ;R) to the space of measurable functions on R d whose maximal operator T * is weak-(1,1), then

with the property that lim t→0 T t f n (x) = f n (x) almost everywhere. The triangle inequality and Chebyshev's inequality imply that for any λ > 0,

Letting n tend to infinity and leveraging the sub-additivity of measure shows that the measure of the set where limsup t→0 T t f (x)f (x) > 0 is equal to zero. This is equivalent to the statement that lim t→0 T t f (x) = f (x) for almost every x ∈ R d , which means that the set (A.44) is closed as required. ∎

Proving the Lebesgue differentiation theorem therefore comes down to showing that the Hardy-Littlewood maximal operator (A.40) is weak-(1,1). This will follow from the Vitali covering lemma, which essentially says that the measure of an arbitrary union of balls is concentrated on a finite disjoint sub-collection of these balls.

Proof. The proof proceeds in two steps. First we show that for almost every

and then we leverage this to prove (A.46).

Step 1: proving (A.47). Combining Lemma A.15 with Proposition A. [START_REF] Michael Aizenman | Extended variational principle for the sherrington-kirkpatrick spin-glass model[END_REF] shows that the set of functions where (A.47) holds is closed in L 1 (R d ;R). Since it contains the dense set of continuous functions by the higher-dimensional version of Proposition A.12, it must be all of L 1 (R d ;R). Noticing that any locally integrable function on R d can be extended to an integrable function on R d that agrees with it on a small enough ball around a given point x ∈ R d establishes (A.47) for every locally integrable function.

Step 2: proving (A.46). Fix c ∈ R, and observe that (A.47) applied to the function

By additivity of measure and countability of the rationals, we can in fact make this convergence uniform for c ∈ Q. More precisely, it is possible to find a null set N ⊆ R d such that for every x ∈ R d ∖ N and c ∈ Q the convergence in (A.48) holds. If we now fix x ∈ R d ∖ N as well as ε > 0, then, by density of the rationals, it is possible

Taking the limsup as r tends to zero and then letting ε tend to zero completes the proof. ∎

A.5 A topological characterization of weak convergence

For the remainder of this chapter we will focus on the weak convergence of probability measures on a metric space S endowed with its Borel σ -algebra B(S). We write C b (S;R) for the space of bounded and continuous functions on S. A sequence (P n ) n⩾1 of probability measures on S converges weakly to a probability measure P on S if, for every bounded and continuous function f ∈ C b (S;R),

A sequence (X n ) n⩾1 of random elements on S converges in law to a random element X on S if the sequence of laws of the X n converges weakly to the law of X.

The law of the random element X on S is the probability measure P X on S defined by P X (A) ∶= P{X ∈ A}.

This notion of convergence is motivated by the fact that, as shown in Exercise A.8, a probability measure is determined by its action on the space C b (S;R) of continuous and bounded functions. In the setting of the real line, S = R, Exercise A.9 outlines a proof of the fact that convergence in law coincides with the familiar convergence in distribution seen in some introductory probability courses. In the setting of general metric spaces, the weak convergence of probability measures can be characterized topologically through the Portmanteau theorem.

Theorem A.17 (Portmanteau). For a sequence (P n ) n⩾1 of probability measures on S, the following statements are equivalent.

(i) (P n ) n⩾1 converges weakly to the probability measure P.

(ii) (A.49) holds for all bounded Lipschitz functions f ∶ S → R.

(iii) For any open set U ⊆ S, we have liminf n→+∞ P n (U) ⩾ P(U).

(iv) For any closed set F ⊆ S, we have limsup n→+∞ P n (F) ⩽ P(F). Proof. Suppose for the sake of contradiction that the sequence (P n ) n⩾1 does not converge weakly to P. This means that there exist a continuous and bounded function f ∈ C b (S;R), some ε > 0 and a sequence (n(k)) k⩾1 such that for all k ⩾ 1, ˆS f dP n(k) -ˆS f dP > ε.

The sequence (P n(k) ) k⩾1 cannot have a subsequence that converges weakly to P. This contradiction completes the proof. ∎

In general, relative compactness of the sequence (P n ) n⩾1 is obtained through the Prokhorov theorem, while uniqueness of the limiting measure is obtained through case-by-case considerations using tools such as characteristic functions or Laplace transforms. The Prokhorov theorem states that the relative compactness of the sequence (P n ) n⩾1 is implied by its uniform tightness. A sequence (P n ) n⩾1 of probability measures on S is uniformly tight if, for any ε > 0, there exists a compact set K ⊆ S such that, for every n ⩾ 1,

There are many different proofs of the Prokhorov theorem, some of which require heavy machinery from measure theory. We will present a relatively simple proof based on the Riesz representation theorem and the upshot of Exercise A.4 that every finite measure on a metric space is closed regular.

Theorem A.20 (Prokhorov). If the sequence (P n ) n⩾1 of probability measures is uniformly tight, then it admits a weakly convergent subsequence.

Proof. The proof proceeds in four steps. We start by extracting a subsequence (n(k)) k⩾1 along which the integral ´S f dP n(k) converges for every f ∈ C b (S;R), we then use the Riesz representation theorem to identify a candidate limit measure P for the sequence (P n(k) ) k⩾1 . Finally, we prove that P is a probability measure and that (P n(k) ) k⩾1 converges weakly to P.

Step 1: extracting the subsequence. Since (P n ) n⩾1 is uniformly tight, for every r ⩾ 1 there exists a compact set K r ⊆ S such that, for every n ⩾ 1,

Observe that each of the spaces C(K r ;R) is separable with respect to the uniform topology. Indeed, for each r ⩾ 1, the compactness of K r implies its separability, so, denoting by (x n ) n⩾1 a dense subset of K r , and applying the Stone-Weierstrass theorem to the algebra of rational linear combinations of products of the continuous functions 1 and d n (x) ∶= d(x,x n ) establishes the separability of C(K r ,R). It is therefore possible to find a countable set C r that is dense in C(K r ;R). A diagonalization argument yields a subsequence (n(k)) k⩾1 such that the functional T ∶ ⋃ r⩾1 C r → R defined by

is well-defined. By density of C r in C(K r ;R), this functional is actually welldefined on ⋃ r⩾1 C(K r ;R). In fact, it is also well-defined on C b (S;R). Indeed, for any f ∈ C b (S;R) and r ⩾ 1, we have f Kr ∈ C b (K r ;S) and

Step 2: identifying the candidate limit. For each r ⩾ 1 introduce the compact set Kr ∶= ⋃ r i=1 K i . Since T is a positive linear functional on C( Kr ;R), the Riesz representation theorem gives a finite measure µ r on Kr such that

Using Urysohn's lemma find a function f ∈ C( Kr 2 ;R) with 1 F ⩽ f ⩽ 1 U . Since F ⊆ Kr 1 ⊆ Kr 2 and T is given by the limit (A.58),

Together with the fact that f ⩽ 1 U , this implies that

Letting ε tend to zero shows that µ r 1 (A) ⩽ µ r 2 (A), and proves that the set function P ∶ B(S) → R ⩾0 in (A.60) is well-defined as a monotone limit.

Step 3: showing P is a probability measure. To show that P is a measure, fix a collection (A i ) i⩾1 of pairwise disjoint sets in B(S), and let A = ⋃ i⩾1 A i . On the one hand, for any n ⩾ 1,

so letting n tend to infinity gives the lower bound P(A) ⩾ ∑ +∞ i=1 P(A i ). On the other hand, given ε > 0 there is r large enough so

where the final inequality uses that P(A i ) is the monotonically increasing limit of µ r (A i ). Letting ε tend to zero establishes the matching upper bound and shows that P is a measure. To see that it is a probability measure, observe that for every r ⩾ 1,

Letting r tend to infinity shows that P(S) = 1 as required.

Step 4: establishing weak convergence. To conclude that the sequence (P n(k) ) k⩾1 of probability measures converges weakly to P, fix a closed set F ⊆ S as well as r ⩾ 1.

Letting k tend to infinity in the bound

The second inequality combines the fact that (P n(k) ) k⩾1 converges weakly to µ r on Kr by (A.58) and (A.59) with Remark A.18. Letting r tend to infinity and invoking the Portmanteau theorem completes the proof. ∎

It turns out that the converse of the Prokhorov theorem is also true for any complete and separable metric space.

Theorem A.21. Suppose S is a complete and separable metric space. If the sequence (P n ) n⩾1 of probability measures on S converges weakly, then it is uniformly tight.

Combining the uniqueness of weak limits with Theorem A.23 implies that P = Q. This completes the proof. ∎ Exercise A.12. Suppose S is a complete and separable metric space. Show that any probability measure P on S is tight in the sense that for every ε > 0 there exists a compact set K ⊆ S with P(K) ⩾ 1ε. (ii) If K = exp(o(N)), then (S.1) ensures that (1.15) remains valid. On the other hand, if log(K N ) grows faster than N, then there exists a constant c > 0 with log(K N ) ⩾ cN for all N large enough. The constant sequence with a k = 1 for every k ⩾ 1 is such that

Appendix S Solutions to exercises

so (1.15) cannot hold. This means that (1.15) holds for arbitrary choices of the sequence (a k ) k⩾1 if and only if K = exp(o(N)).

Exercise 1.2. Consider the two-by-two matrix M = (M i, j ) i, j∈{±1} defined by M i j ∶= e β i j+h j , Similarly,

Combining these two equalities reveals that

By convexity of C, we have yn+ym 2 ∈ C, and therefore

This shows that (y n ) n⩾1 is Cauchy. Since C is a closed subspace of the Hilbert space H, it is a complete metric space. The sequence (y n ) n⩾1 therefore converges to some y 0 ∈ C. By continuity of the norm, we have y 0x 2 = d(x,C) which completes the proof.

Exercise 2.17. We treat each question separately.

(i) To show that H = C ⊕C ⊥ we need to show that every x ∈ H may be written uniquely as x = y + z for some y ∈ C and z ∈ C ⊥ . Fix x ∈ H, and invoke Exercise 2.16 to define y ∶= P C (x) and z ∶= x -P C (x). It is clear that x = y + z and that y ∈ C. To see that z ∈ C ⊥ , fix w ∈ C. Since C is a vector space, applying the characterization (2.29) of the projection P C (x) to the vector w + P C (x) reveals that z ⋅ w = (x -P C (x)) ⋅ (w + P C (x) -P C (x)) ⩽ 0.

Since C is symmetric, this shows that z ∈ C ⊥ , so x may be written as x = y + z for some y ∈ C and z ∈ C ⊥ . To show that this decomposition is unique, suppose that we may also write x = w + v for some other w ∈ C and v ∈ C ⊥ . Since C and C ⊥ are both vector spaces, we have yw ∈ C and zv ∈ C ⊥ , so Pythagoras' theorem implies that 0

which implies that y = w and z = v as required.

(ii) The uniqueness of y ∈ H is clear. Indeed, if f (x) = x ⋅ y = x ⋅ y ′ for some y,y ′ ∈ H, then x ⋅ (yy ′ ) = 0 for all x ∈ H, and choosing x = yy ′ shows that y = y ′ . We now establish the existence of y. If f = 0, we can choose y = 0, so let us assume that f ≠ 0. Consider the closed subspace C ∶= ker f of H. Since f ≠ 0, the space C is not all of H, so Exercise 2.17 gives z ∈ C ⊥ with z = 1. For each x ∈ H, we have

Setting y ∶= z f (z) completes the proof.

suppose also that f (T,0) ⩾ g(T,0). Given M > 2L, the comparison principle with R = V T implies that

where we used that f (0,z) = g(0,z). To bound this further, notice that by the case of equality in the triangle inequality and the choice of z,

It follows that for all y ∈ R d with y > V T ,

Since f (0,⋅) and g(0,⋅) coincide on B V T , this inequality actually holds for every y ∈ R d . Recalling (S.13) completes the proof.

Exercise 3.9. The strategy will be to show that g is a viscosity solution to the Hamilton-Jacobi equation (3.20). Let φ ∈ C ∞ (R >0 ×R d ;R) be such that g-φ admits a local maximum at (t * ,x * ) ∈ R >0 × R d . Since g is a subsolution to ∂ t g -K(∇g) = 0, we have

Moreover, for any ε > 0 and z ∈ R d , we have

Dividing by ε > 0, letting ε tend to zero and choosing z = -∇φ (t * ,x * ) shows that ∇φ (t * ,x * ) ⩽ L. Since K and H coincide on the ball of radius L, it follows by (S.14) that

This shows that g is a subsolution to the Hamilton-Jacobi equation (3.20). An identical argument shows that it is also a supersolution to this Hamilton-Jacobi equation. Invoking the uniqueness result in Corollary 3.7 completes the proof.

Exercise 3.10. We treat each question separately.

Combining the fact that

with a direct computation shows that

where we have written z ∶= φ -1 t (x) to simplify notation. Together with the identity (S.17), this implies that

and therefore that the gradient of u is constant along the characteristic line t ↦ (t,φ t (x)), with ∇u(t,φ t (x)) = ∇ψ(x).

(iii) To deduce that u satisfies the Hamilton-Jacobi equation (3.20) at every point in [0,T ) × R d , it suffices to show that for every t < T and x ∈ R d , we have

Consistently with the notation in (S.15), we think of ∂ t φ t and ∂ t φ -1 t as "row vectors". In particular, we have that

where ∇ * H(p) is the row vector obtained as the transposition of the column vector ∇H(p). Taking the derivative on both sides of (S. [START_REF] Alvarez | Axioms and fundamental equations of image processing[END_REF]) and rearranging gives the identity

(S.20) To simplify notation, again let z ∶= φ -1 t (x). Recalling (S.18), we can write

Using also (S.20), we obtain the desired result (S. [START_REF] Daniel | Spin-glass models of neural networks[END_REF].

Applying the argument leading to (5.31) in the proof of Proposition 5.13 to the function f (x) ∶= g(k n,x) shows that for any 1 ⩽ k ⩽ n,

Moreover, the o(1 n) error term can be controlled uniformly over k. We thus have

which together with a Taylor expansion of the logarithm reveals that

Letting n tend to infinity, identifying a Riemann sum, and invoking Propositions 5.4 and 5.9 completes the proof.

Exercise 5.8. We treat each question separately.

(i) By Proposition 5.12, we have

which is finite since ζ < 1. In particular we have that ´(0,1] xdΛ(x) is finite almost surely. Since ´+∞

) dx < +∞, there are finitely many points in Λ that fall in the interval [1,+∞). Combining these two observations leads to the claim.

For a large enough we obtain the upper bound

(iii) Fix ε > 0, let a > 0 be large enough so the bound in (ii) holds, and fix n ⩾ 1 large enough so εn 1 ζ > a. The interval [0,εn 1 ζ ] can be partitioned as

given by µ x (x j ) x 2 j . It follows that the measures µ x and µ y are different, so there must exist a polynomial P ∶ [0,1] → R with j⩾1 x 2 j P(x j ) = ˆ1 0 Pdµ x ≠ ˆ1 0 Pdµ y = j⩾1 y 2 j P(y j ).

Noticing that the function z ↦ ∑ j⩾1 z 2 j P(z j ) belongs to the algebra P shows that P separates points as desired.

(iii) Fix k ⩾ 1 as well as n 1 ,...,n k ⩾ 2, and, through a slight abuse of notation, consider the function

To simplify this expression, we rewrite it as the average of some function g of the overlaps R n = (R , ′ ) , ′ ⩽n sampled from the average (5.51) associated with the weights (v j ) j⩾1 . Let n ∶= n 1 + ⋯ + n k , and partition the set {1,...,n} into k subsets I = (I j ) j⩽k with I j = n j and 1 ∈ I 1 . For , ′ ⩽ n write ∼ ′ if and only if and ′ belong to the same set in I, and introduce the function

By construction,

v n 1 j 1 ⋯v n k j k = S(n 1 ,...,n k ), so the Ghirlanda-Guerra identities (5.57) imply that

A direct computation shows that

Similarly, for any index 1 ⩽ ⩽ n with ∈ I 1 ,

while for any index in 1 ⩽ ⩽ n with ∉ I 1

(ii) Given t ∈ R and x ∈ (0,1), the conclusion of (i) may equivalently be expressed as the fact that F -1 (x) ⩽ t if and only if x ⩽ F(t). It follows that

Since the distribution function characterizes the law of a random variable, this completes the proof.

Exercise 5.12. By the dominated convergence theorem, the mapping

The mapping ζ ↦ logEexpζ X is therefore also differentiable on this interval, and its derivative at zero is EX, as announced.

Exercise 5.13. We treat each question separately.

(i) We write

and we clearly have

yielding the announced inequality (5.111).

(ii) Using the result of the previous question, we can write

The Ghirlanda-Guerra identities imply that the sum above is

An induction argument completes the proof of (5.112).

S.6 Mean-field spin glasses

Exercise 6.1. Fix β > 0. Jensen's inequality and the explicit formula for the moment generating function of a Gaussian random variable imply that

Optimizing over β > 0 to find β 2 = 2log2 yields

This establishes the upper bound. To prove the lower bound, we optimize one coordinate at a time (some would call it a "greedy algorithm"). For each integer 1 ⩽ i ⩽ N and spins σ 1 ,...,σ i-1 ∈ Σ 1 , define the random variable

in such a way that

Fix σ 1 ,...,σ N ∈ Σ 1 inductively so that, for every i ∈ {1,...,N}, the spin σ i has the same sign as X i (σ 1 ,...,σ i-1 ) (with some arbitrary decision rule in the case that X i (σ 1 ,...,σ i-1 ) = 0). With this choice of σ = (σ 1 ,...,σ N ) ∈ Σ N , the random variables (X i (σ 1 ,...,σ i-1 )) 1⩽i⩽N are independent, and X i (σ 1 ,...,σ i-1 ) is a Gaussian random variable with variance 2(i-1). These random variables are also independent of the family (g i,i ) 1⩽i⩽N . Since

and the absolute value of a standard Gaussian has mean 2 π , we obtain that

for some constant C > 0. This completes the proof. To obtain the matching upper bound, we will only consider the case when a and b are both finite; the general case can be deduced through a simple limiting argument. Fix ε > 0 as well as n ⩾ 0, and use the continuity of F to find δ > 0 and δ n > 0 with

, so it is possible to extract a finite subcover. Up to relabelling, denote by ((a i ,b i + δ i )) 1⩽i⩽n this finite subcover and assume that b i + δ i ∈ (a n+1 ,b n+1 + δ n+1 ) so that the subcover is ordered according to the right endpoint of each interval. The choice of δ and the non-decreasingness of F imply that

The choice b i + δ i ∈ (a n+1 ,b n+1 + δ n+1 ), the non-decreasingness of F and the choice of δ i reveal that this is bounded further by

Letting ε tend to zero and combining the resulting bound with (S.36) completes the proof.

Exercise A.3. Since µ and ν agree on P and A ∈ P, it is clear that S ∈ L. To verify the second property of a λ -system, fix B,C ∈ L with B ⊆ C. By additivity of measure,

where the assumption that µ(A) = ν(A) < +∞ ensures that the subtraction is welldefined. Finally, if (B n ) n⩾1 is an increasing sequence of sets in L and B ∶= ⋃ ∞ n=1 B n , then the continuity of measure established in Exercise A.1 implies that

which means that B ∈ L. This shows that L is a λ -system. The final claim follows from the Dynkin π-λ theorem by taking A = S. This completes the proof. (iii) Since L is a λ -system and the set of open sets forms a π-system that generates the Borel σ -algebra, the result follows by the Dynkin π-λ theorem.

Exercise A.5. Fix a positive linear functional T ∶C(S;R) → R as well as a continuous function f ∈ C(S;R). Since f ∞ ⋅ 1 ± f ⩾ 0, the positivity of T implies that

It follows by linearity of T that

Rearranging reveals that T ( f ) ⩽ f ∞ T (1) and completes the proof. It therefore suffices to prove the result on C([0,1];R). Fix ε > 0 and f ∈ C([0,1];R). Given x ∈ [0,1], let (X n ) n⩾1 be a sequence of independent and identically distributed random variables with X 1 ∼ Ber(x). For each n ⩾ 1, consider the sample average S n ∶= 1 n ∑ n i=1 X i , and define the polynomial

To express P n as a polynomial we have used the fact that ∑ n i=1 X i is a Binomial random variable with n trials and probability of success x. To show that for n large enough P n is within ε of f , use the uniform continuity of f on [0,1] to find δ > 0 so that f (y)f (x) ⩽ ε 2 whenever xy ⩽ δ , and observe that by Chebyshev's inequality and independence of the (X n ) n⩾1 , for any x ∈ [0,1],

Choosing n large enough ensures that f -P ∞ ⩽ ε and completes the proof.