Ada Altieri

Alessandro Manacorda

Alia Abbara

Beatriz Seoane

Chen Liu

Davide Facoetti

Elisabeth Agoritsas

Félix Roy

Misaki Ozawa

Valentina Ros

First of all, I would like to thank Guilhem Semerjian for

Introduction

In a Constraint Satisfaction Problem (CSP), N discrete valued variables are subject to M constraints. Each of the constraints enforces some requirements on a subset of the variables. A solution of the CSP is an assignment of the variables that satisfies simultaneously all the constraints. Famous examples of CSPs are the k-satisfiability (k-SAT) problem and the graph q-coloring one (q-COL). In the first one the variables are Boolean and each constraint is the disjunction (OR) of k literals (a variable or its negation). In the second one the variables are placed on the vertices of a graph, they can take q possible values, to be interpreted as colors, and each edge of the graph enforce the constraint that the two vertices at its ends take different colors. In this Ph.D. we will be also interested in the bicoloring of k-hypergraph problem, that is similarly defined on a hypergraph, with hyperedges linking subsets of k (instead of two for a graph) vertices; the variables on the vertices can take two colors, and the constraint associated to each hyperedge is that both colors are present among its k adjacent vertices.

CSPs can be studied from several different perspectives; computational complexity theory [START_REF] Garey | Computers and intractability: A guide to the theory of NP-completeness[END_REF][START_REF] Papadimitriou | Computational complexity[END_REF][START_REF] Moore | The Nature of Computation[END_REF] classifies them according to their worst-case difficulty, assessed by the existence or not of an efficient algorithm (running in a time polynomial in N, M ) able to solve (i.e. to determine the existence or not of a solution) all their possible instances. Another perspective [START_REF] Monasson | 2+p-sat: Relation of typical-case complexity to the nature of the phase transition[END_REF][START_REF] Biroli | A variational description of the ground state structure in random satisfiability problems[END_REF][START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Mertens | Threshold values of random k-sat from the cavity method[END_REF][START_REF] Krzakala | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF][START_REF] Achlioptas | On the solution-space geometry of random constraint satisfaction problems[END_REF][START_REF] Achlioptas | Algorithmic barriers from phase transitions[END_REF][START_REF] Molloy | The freezing threshold for k-colourings of a random graph[END_REF][START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF], that has been adopted in this Ph.D., consists of the characterization of the "typical" difficulty of CSPs, where typical is defined with respect to a random ensemble of instances. The most commonly studied random ensemble is obtained by drawing the M constraints uniformly at random. This corresponds to construct a G(N, M ) Erdős-Rényi random graph (or k-hypergraph). In this Ph.D., we will also consider a slightly different ensemble, the k-uniform l + 1-regular one, where the probability is uniform on the set of k-hypergraphs for which each vertex belongs to l + 1 hyperedges.

Random CSPs bear a formal similarity with models studied in statistical mechanics of disordered systems, and in particular with mean-field spin-glasses, the interactions induced by the constraints being of a frustrating nature while lacking a finite-dimensional structure thanks to the randomness in their construction. For instance, in the q-coloring problem on graphs, the variables can be considered as Potts spins, and the problem corresponds to find the ground state of an antiferromagnetic Potts model. If in the ground state all the links are bicolored, then all the constraints are satisfied simultaneously, and the problem admits a solution.

A particularly interesting regime is the thermodynamic limit where N, M → ∞ at a fixed ratio α = M/N , the density of constraints per variable. Random CSPs exhibit threshold phenomena in this limit, the probability of some properties jumping abruptly from 1 to 0 as a function of the control parameter α. The most prominent of these phase transitions occurs at the satisfiability threshold α sat , that depends on the parameter k, q of the problem. For α < α sat typical instances are satisfiable, i.e. admit configurations of variables that satisfy all constraints simultaneously, while for α > α sat a random instance is typically unsatisfiable. Thanks to the analogy between random CSPs and spin glasses, the application of the methods first developed in the context of statistical mechanics of disordered sytems, namely the replica and cavity method [START_REF] Monasson | 2+p-sat: Relation of typical-case complexity to the nature of the phase transition[END_REF][START_REF] Biroli | A variational description of the ground state structure in random satisfiability problems[END_REF][START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Mertens | Threshold values of random k-sat from the cavity method[END_REF][START_REF] Krzakala | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF], has provided predictions of α sat for many models, but also unveiled many other phase transitions for the structure of the set of solutions in the satisfiable phase. In addition, it has led to the proposal of new algorithms that exploit this detailed picture of the solution space. Many of these predictions have been confirmed rigorously later on [START_REF] Achlioptas | On the solution-space geometry of random constraint satisfaction problems[END_REF][START_REF] Achlioptas | Algorithmic barriers from phase transitions[END_REF][START_REF] Molloy | The freezing threshold for k-colourings of a random graph[END_REF][START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF].

In this Ph.D. we give a particular attention to one of the phase transitions that occurs in the satisfiable phase, namely the clustering (or dynamic) transition that occurs at a critical density denoted α d . This transition can be defined in various ways; the name clustering emphasizes the drastic change of the shape of the set of solutions, viewed as a subset of the whole configuration space. Below α d the set of solutions of typical instances is rather well-connected, any solution can be reached from any other one by a rearrangement of a nonextensive number of variables. Above α d the solution set splits in a large number of distinct groups of solutions, called clusters, that correspond to a pure state decomposition of the uniform measure over the set of solutions. The clusters are internally well-connected but well-separated one from the other. This transition marks also the birth of a specific type of long-range correlations between variables, known as point-to-set correlations, which implies the solvability of an information-theoretic problem called tree reconstruction [START_REF] Mossel | Information flow on trees[END_REF]. These correlations forbid in turn the rapid equilibration of the stochastic processes that respect the detailed balance condition [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF], hence the name dynamic given to α d . The static properties of the model are instead not affected by the clustering transition and are only sensitive to a further condensation transition α c that affects the number of dominant clusters [START_REF] Krzakala | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF]. In the cavity method [START_REF] Mézard | The bethe lattice spin glass revisited[END_REF] treatment of the random CSPs α d can also be defined as the appearance of a non trivial solution of the one step of Replica Symmetry Breaking (1RSB) equation with Parisi breaking parameter X = 1, see in particular [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF] for the connection between this formalism and the reconstruction problem. In the large k limit the dynamic transition happens at a much smaller constraint density than the satisfiablity one. For instance, the asymptotic expansion of these two thresholds for the bicoloring of k-hypergraphs is α d (k) ∼ 2 k-1 ln k/k and α sat (k) ∼ 2 k-1 ln 2.

Despite the rather detailed picture of the set of solutions of random CSPs, an important open problem remains to understand the behavior of algorithms that attempt to find a solution in the satisfiable regime, where typical instances admit such configurations. In particular one would like to determine the algorithmic threshold α alg (k) above which no algorithm is able to find a solution in polynomial time with high probability (assuming P =NP). For small values of k it is possible to design algorithms (see [START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Selman | Noise strategies for improving local search[END_REF][START_REF] Ardelius | Behavior of heuristics on large and hard satisfiability problems[END_REF][START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF][START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF]) that are found through numerical simulations to be efficient at densities very close to the satisfiability threshold. The situation is quite different in the large k limit, where these algorithms cannot be studied numerically. One has to resort to analytical studies in this case, which can only be performed on simpler heuristics. The best result in this direction is the one of [START_REF] Coja-Oghlan | A better algorithm for random k-sat[END_REF], which provides an algorithm that provably works in polynomial time up to densities of constraints coinciding at leading order with α d (k). This leaves a wide range of α where typical instances have a non-empty set of solutions, but no known algorithm is able to find them efficiently (and where some families of algorithms have been proven to fail [START_REF] Gamarnik | Performance of sequential local algorithms for the random nae-k-sat problem[END_REF][START_REF] Coja-Oghlan | Walksat stalls well below satisfiability[END_REF][START_REF] Hetterich | Analysing survey propagation guided decimation on random formulas[END_REF]).

One could hope to get some insight on the typical behavior of the algorithms and on the value of α alg from the several phase transitions undergone by the set of solutions in the satisfiable phase. However the connection between these two aspects is delicate, because the algorithms considered are out-of-equilibrium, either because they explicitly break the detailed balance condition, or because their relaxation time scale is larger than the time scales accessible for practical experiments. Even if one cannot understand precisely α alg (k) in terms of a structural phase transition one can reasonably state that the dynamic transition is a lower bound to the algorithmic one, α d (k) ≤ α alg (k). Indeed for α ≤ α d simulated annealing [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF] should be able to equilibrate in polynomial time down to arbitrarily small temperatures, and hence sample uniformly the solution set. For α slightly larger than α d one expects simulated annealing to fall out-ofequilibrium on polynomial timescales but in many cases it should still be able to find (non-uniformly) solutions, hence the bound α d (k) ≤ α alg (k) is not tight in general.

The study of the structural phase transitions in the satisfiable regime, and in particular the definition of α d in terms of long-range correlations, relies on the characterization of a specific probability law on the space of configurations, namely the uniform measure over solutions. In this Ph.D. we study probability measures over the set of solutions of random CSPs, for which not all solutions are equally probable. The same idea has been used in several articles, see in particular [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF][START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF], with slightly different perspectives and results. For instance in [START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF] solutions are weighted according to their local entropy, a quantity that counts the number of other solutions in their neighborhood. In [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF] the solutions are weighted according to their number of frozen variables taking the same value in the whole cluster. In [START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF] hard sphere particles are considered as a CSP, with a bias due to an additional pairwise interaction between particles. In this Ph.D. we will study the modifications of the clustering threshold α d induced by the non-uniformity between solutions. Focusing on the bicoloring on k-hypergraph problem, we will see that with an appropriate choice of the bias it is possible to delay the clustering threshold to higher densities of constraints, both for finite values of k and in the large k limit. For finite k values we will check that this strategy has a positive impact on the performances of simulated annealing algorithms, in agreement with the discussion above.

This manuscript is based on the three papers [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF][START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF][START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF] that have been produced during my Ph.D.. It is organized as follows. A survey describing the phase transitions in random CPSs and the search algorithms is presented in chapters 1-5, with a summary of some of the results obtained in the literature. In chapter 1 we start by defining the Constraint Satisfaction Problems and give some elements of the complexity theory. We also introduce the random ensembles of CSPs and explain their connection with statistical physics problems. Chapter 2 presents the several phase transitions undergone by the set of solutions in random CSPs in the thermodynamic limit. In chapter 3 we introduce the simulated annealing algorithm and other local search algorithms that perform a biased walk in the configuration space toward the set of solutions [START_REF] Selman | Noise strategies for improving local search[END_REF][START_REF] Ardelius | Behavior of heuristics on large and hard satisfiability problems[END_REF][START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF][START_REF] Semerjian | Relaxation and metastability in a local search procedure for the random satisfiability problem[END_REF]. In chapter 4 we introduce the message passing algorithms [START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF][START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF][START_REF] Montanari | Solving constraint satisfaction problems through belief propagation-guided decimation[END_REF][START_REF] Ricci | On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms[END_REF] inspired by statistical mechanics considerations. Chapter 5 describes the performances of the algorithms on random instances of CSPs. In particular it separates the regime of small values of k where some algorithms are efficient up to densities very close to the satisfiability threshold, and the large k limit, where there is a wide range of α in the satisfiable phase for which no provably efficient algorithm is known at present to find solutions.

In chapter 6 we introduce biased measures on the set of solutions. We give an explicit definition of the biased measures studied in this Ph.D., and describe the biased measures studied in the literature ( [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF][START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF]). Chapter 7 presents the cavity method that allows us to estimate the value of the thresholds for the several phase transitions. We describe the explicit derivation of these thresholds for the biased measures studied in the Ph.D..

In chapter 8 we present the results obtained for finite values of k for the bicoloring of k-hypergraphs. The sections 8.2-8.4 present the results obtained in the paper [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF], that has been written in collaboration with Federico Ricci-Tersenghi. In [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] we have studied a simple implementation of the bias in the measure on the set of solutions of a hypergraph bicoloring instance, where the interactions induced by the bias can be factorized over the bicoloring constraints. We showed, for k between 4 and 6, that with well-chosen parameters such a bias allows us to increase α d , and to improve the performances of simulated annealing. The section 8.5 presents part of the results obtained in [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF], in which we introduce a more generic way of weighting the different solutions, that extends the one presented in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] and incorporates interactions at larger range between variables. We show that for finite k this more generic biased measure allows a further increase of the dynamic threshold α d .

The last two chapters study the large k regime. For the uniform measure over the set of solutions, some rigorous bounds are known on the asymptotic behavior of α d [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF]. In chapter 9 we present the results obtained in the paper [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF], in which we provide an asymptotic expansion of the clustering threshold: we found that for the bicoloring on k-hypergraphs the clustering threshold occurs on the scale α ∼ 2 k-1 (ln k + ln ln k + γ)/k with γ constant. We obtain a similar scaling α ∼ (q/2)(ln q + ln ln q + γ) for the q-coloring problem in the large q limit, with exactly the same constant γ d . We estimate this constant for the uniform measure γ d,u ≈ 0.871, which falls into the range allowed by the previous bounds [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF].

chapter 10 presents the results obtained in the paper [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF]. In this paper we adapt the large k expansion of [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] to the biased measures studied in this Ph.D. and manage to assess the asymptotic effect of the bias on α d . We show that the clustering threshold for our implementation of the bias arises at the same scale as in the uniform case, but with a constant γ d that depends on the rescaled parameters describing the bias. We find that the factorized bias of [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] cannot improve on the constant γ d in the asymptotic expansion with respect to γ d,u , while the bias with larger interaction range allows us to increase its value up to γ d ≈ 0.977. Although this provides only a modest improvement, bearing on the third order of the asymptotic expansion of α d , it opens the possibility to study further generalizations of the bias and to bring some light on the nature of the algorithmic gap between α alg and α sat .

Definitions

In a constraint satisfaction problem (CSP), a set of N variables are submitted to a set of M constraints (also called clauses). The variables x i , i ∈ {1, . . . , N } take their values in a finite set χ. When χ is of size 2, the variables can either be viewed as Boolean variables: χ = {0, 1} or, to adopt the physics notation, as spins σ i ∈ {-1, 1}, using the change of variable σ i = 2x i -1. When the size of χ is an arbitrary integer q, the variables can be viewed as Potts spins, or colors: χ = {1, . . . , q}. We call x = {x 1 , . . . , x N } ∈ χ N a configuration of the N variables. For a subset S ⊂ {1, . . . , N } of variables, we call x S its configuration. The clauses c a , a ∈ {1, . . . , M } involves a subset ∂a ⊂ {1, . . . , N } of variables, and impose a constraint on the value of their configuration x ∂a . They are functions c a : χ |∂a| → {0, 1} that evaluate to 1 when the constraint is satisfied, and to 0 otherwise. We call ∂i the subset of clauses in which the variable i appears. There is a natural cost function E : χ N → R + to define on a CSP, that counts the number of constraints that are unsatisfied (violated) by a configuration x:

E(x) = M a=1 (1 -c a (x ∂a )) (1.1)
In the optimization version of the problem, one tries to find an optimal configuration that minimizes this cost function. In the decision version of the problem, one tries to answer the question: 'Given a threshold value E 0 , is there a configuration x * with a cost E(x * ) ≤ E 0 ?' One often takes the threshold to be 0, so that the question is 'Is there a configuration that satisfies all the constraints simultaneously ?'. Such a configuration is called a solution. Note that if the evaluation of the cost function is easy to perform, the decision version of the problem cannot be harder than the optimization version. Indeed once we have an optimal configuration, we only need to evaluate its cost and compare it to E 0 in order to answer the decision problem. We will adopt the following definitions. The constraint satisfaction problem is the general question to answer, defined by a set of parameters left unspecified. To be more precise, these parameters are for instance the set χ, the numbers N, M of variables and clauses, the function space in which the clauses c a can take their values, the specific form of the cost function. An instance of the problem is obtained by setting a particular value to all the problem parameters. One says that an algorithm solves a constraint satisfaction problem if it produces an answer to any instance of the problem. A certificate is an assignment x that allows to check the answer of the problem.

There is another version of the CSPs, called the counting problem, that aims at counting all the solutions of a given instance. Generally this version is more difficult than the decision and optimization versions.

The satisfiability problem

As a first example, consider the satisfiability (SAT) problem. It is defined on a set of N Boolean variables. The clauses {c 1 , . . . , c M } are logical constraints on the set of literals. For a variable x i , its literal l i is either equal to x i or to its negation x i = 1 -x i . Each clause is a disjunction (logical OR) on the literals formed from the variables in ∂a = {i(1), . . . , i(|∂a|)}: c a = (l i [START_REF] Garey | Computers and intractability: A guide to the theory of NP-completeness[END_REF]

∨ l i(2) ∨ • • • ∨ l i(|∂a|)
). The clause c a evaluates to 1 if and only if at least one literal evaluates to 1. Among the 2 |∂a| possible configurations that the variables in ∂a can take, there is only one that violates the clause, namely the one for which l i(1) = l i(2) = • • • = l i(|∂a|) = 0. The satisfiability formula F is the conjunction (logical AND) over the set of clauses:

F = c 1 ∧ c 2 ∧ • • • ∧ c M (1.2)
F evaluates to 1 if and only if all the clauses {c 1 , . . . c M } evaluate to 1. We can call F a SAT formula, or a CNF formula (where CNF stands for conjunctive normal form). In the decision version of the SAT problem, a satisfying assigmement, or solution, is then a configuration x such that F = 1. An instance of the SAT problem is thus defined by specifying the formula F , i.e. the value of N, M , then for each clause a the choice of the subset ∂a, and for each variable i in ∂a the choice of the literal l i that appears in c a . The k-SAT problem is the restriction of the SAT problem to the set of instances that have clauses of fixed length k. The optimization version of the SAT problem is called the MAX-SAT problem (or MAX-k-SAT when the clauses have length k). Using the spins (σ i = 2x i -1) the clauses can be re-written as follow:

c a (σ ∂a ) = 1 - i∈∂a I[σ i = -J a i ] (1.3)
where J a i takes its values in {-1, +1}, with J a i = 1 if the literal is x i , and J a i = -1 if the literal is x i , and I[A] is the indicator function of the event A.

x 1

x 2

x 3

x 4

x 5 x 6

x 7

x 8

x 9

F = (x 1 ∨ x 2 ∨ x 3 ∨ x 4 ) ∧ (x 4 ∨ x 5 ) ∧ (x 4 ∨ x 6 ∨ x 7 ) ∧ (x 3 ∨ x 7 ∨ x 8 ∨ x 9 )

A graphical representation

There is a natural graphical representation of the instances of a CSP. For any instance, we associate a bipartite graph G = (V, C, E). Figure 1.1 is an example of this representation for a SAT formula. The filled circles represent the set V of N vertices of the first type (called variable nodes), that is associated with the set of variables. The empty squares represent the set C of M vertices of the second type (called function nodes), that represent the set of clauses. E is the set of edges between variable nodes and function nodes. A link is drawn between the variable node i and the function node a if the variable i appears in the clause a: i ∈ ∂a. Note that for the SAT problem, an instance is not yet fully specified by this representation, since the information about the choice of the literal is missing. This can be specified by using two types of edges, for instance plain edges when l i = x i , and dashed edges when l i = x i . The bipartite graph G can be viewed equivalently as a hypergraph, where the variables are still represented as vertices, but the clauses are now hyper-edges, i.e. edges linking a subset of vertices |∂a| possibly greater than 2.

More examples of CSP

The k-XORSAT problem:

Compared to the k-SAT problem, the disjunction ∨ (OR) is replaced by the eXclusive OR (XOR) (denoted by ⊕):

c a = (l i(1) ⊕ • • • ⊕ l i(k) ) (1.4)
This is equivalent to asking that the sum k j=1 l i(j) of the literals is equal to 1 modulo 2. This problem has a simpler structure than the k-SAT problem. In particular, one can solve this problem in polynomial time using algorithms based on Gaussian elimination.

2. The q coloring (q-col) problem:

In this problem the variables can take q possible values. Each clause involves a pair of variables, and is satisfied when the two variables do not take the same color: c a (x i , x j ) = I[x i = x j ].

The bicoloring problem on k-hypergraphs

This problem is very similar to the q-coloring problem. Here q = 2: the variables are Booleans (or spins), and each constraint involve a k-tuple (with k ≥ 2) of variables. The constraints forbids the configurations where all the variables take the same value. It is also called the Not All Equal (NAE) k-SAT problem, but note that compared to the k-SAT problem, the constraints forbids 2 configurations (namely the configuration where x i = 1 for all i ∈ ∂a, and the opposite configuration where x i = 0 for all i ∈ ∂a).

Other combinatorial optimization problems

The CSPs belongs to the family of combinatorial optimization problems. These problems consist of finding a configuration (in a finite set of possible configurations) that minimizes a cost function. We give few examples:

1. the assignment problem: given N agents and N tasks, and an affinity matrix M ij representing the affinity of agent i for the task j, one looks for an assignment of each agent to a task in order to maximize the total affinity. The space of configurations is the space of permutations π ∈ S(N ). One defines the total cost function

E(π) = i C iπ(i) (1.5)
2. the Hamiltonian cycle. This is a decision problem. Given a graph G = (V, E), a Hamiltonian cycle is a path that visit each vertex of the graph exactly once, and gets back to this starting point. The question is then 'Does there exist a Hamiltonian cycle ?' The configurations are in the set of paths on G.

the traveling salesman problem: this is the optimization version of the

Hamiltonian problem on a weighted graph. The cost function is defined as the sum of weights of the edges visited by the path. The question is then to find a Hamiltonian cycle that minimize the cost

Worst-case complexity 1.2.1 Time complexity

From an algorithmic point of view, one would be interested in finding an algorithm that solves a CSP (i.e. solves all its possible instances) in a reasonable amount of time. Intuitively, an algorithm will require more time to provide an answer for large size instances. There are several ways to define the size of an instance. A natural definition is to take N the number of variables, but one could also have chosen N + M , or |E| the total number of edges in the graphical representation of a CSP. Most of the time one works with instances for which M = αN , with α a constant independent of N , therefore all these possible definitions are polynomially related. In computational complexity theory [START_REF] Garey | Computers and intractability: A guide to the theory of NP-completeness[END_REF] it is therefore equivalent to work with any of them, and we will fix the size of the problem to be N . One defines the time complexity function T (N ) of an algorithm as the largest amount of time needed for this algorithm to solve an instance of given size N . Note that this definition depends on the computer used to implement the algorithm. One will not enter into the details, and just state that one uses a 'realistic' computer model. Realistic means that such a model can run at most one (or a finite small number) of operations per unit time, where operations are for instance comparison, sum and multiplication. Some examples of realistic model are the one-tape (or multi-tape) Turing machines, and the random-access machines (RAMs). The algorithms can be split into classes according to the behavior of their time complexity function.

A polynomial time algorithm has a time complexity function T (N ) =

O(p(N )) for some polynomial function p.

2. An exponential time algorithm has a time complexity function that can be lower bounded by 2 bn for some constant 0 < b.

This distinction is useful because exponential time algorithm have a time complexity that explodes when the size increases, therefore they become inefficient for large size instances. One says that a problem is intractable when it is so hard that no polynomial algorithm can solve it. Note that the time-complexity function is a worst-case measure, in the sense that is corresponds to the time needed for the algorithm to solve the hardest instance of a given size. It could happen that the majority of other instances require much less time, and the algorithm would be 'efficient' in a large proportion of the instances it encouters.

As an example, the k-SAT problem is solved by an exponential time algorithm called DPLL, that one will describe further. As we shall see in the following, experimental observations and rigorous results show however that for instances such that M = αN , with α small enough, this algorithm finds solutions very quickly for the vast majority of instances, even when N gets large.

Complexity classes

The theory of computational complexity classifies the difficulty of the decision problems according to the existence of an algorithm that solve a problem in polynomial time. This classification uses the notion of polynomial reduction. One says that a problem π 1 can be reduced to another problem π 2 when there is a function that maps an instance of π 1 into an instance of π 2 . To be more precise, let D π be the set of instances of problem π. Let Y π ⊆ D π be the subset of instances for which the answer to the decision problem is 'yes'. Then a reduction is a function f : D π1 → D π2 such that for all I 1 ∈ D π1 , one has I 1 ∈ Y π1 if and only if f (I 1 ) ∈ Y π2 . A polynomial time reduction is a reduction f that is computable by a polynomial time algorithm in the size of the problem π 1 . One requires additionally that the size of f (I 1 ) is polynomial in the size of I 1 . The relation "π 1 can be reduced polynomially to π 2 " is denoted

π 1 ∝ π 2 .
This relation is transitive:

Lemma 1 (Transitivity) if π 1 ∝ π 2 and π 2 ∝ π 3 , then π 1 ∝ π 3 .
One defines the two following classes of problems:

1. The polynomial class P is the set of problems that can be solved by a polynomial time algorithm.

2. The non-deterministic polynomial class NP is the set of problems that have a polynomial time verifiability. This means when one provides a certificate that allows to check a 'yes' answer to the problem, the verification itself can be done in polynomial time.

The term non-deterministic comes from an alternative definition of the NP class, as the set of problems that can be solved by a polynomial time nondeterministic algorithm. A non-deterministic algorithm is composed of two stages. In the first stage, a certificate is proposed, and in the second stage, the certificate is verified. A non-deterministic algorithm is said to operate in polynomial time if the second stage is done in polynomial time, disregarding the first stage that might have required more time. Roughly speaking, a nondeterministic polynomial algorithm can pursue alternate paths or branches in parallel, and says 'yes' if any of these branches says 'yes'. Note that if one has a polynomial reduction from π 1 to π 2 , then π 1 cannot be harder than π 2 . Suppose indeed that one has a polynomial algorithm that solves π 2 . One can construct a polynomial algorithm to solve π 1 as follows. Given an instance I 1 ∈ D π1 , this algorithm just applies the transformation f to get an instance I 2 = f (I 1 ) of π 2 , and applies the polynomial algorithm to solve the instance I 2 . Therefore:

Lemma 2 If π 1 ∝ π 2 , then π 2 ∈P implies π 1 ∈P (and equivalently π 1 / ∈P implies π 2 / ∈P)
Note that a problem π in P is automatically in NP. If there is a polynomial time algorithm that solves π, this algorithm can be converted into a checking algorithm that ignores the certificate that is provided, and simply returns its answer. Therefore P⊆NP.

A natural question is then to ask if P=NP, or if there exist problems in NP that are not in P. This is an open question, but it is widely conjectured that P =NP. This conjecture is supported by the existence of another class of NP problems gathering the hardest problems in NP. It is called the NP-complete class (NP-c). A problem π is in NP-c if it satisfies the two conditions: If one problem in NP-c could be solved in polynomial time, then all the problems in NP could be solved in polynomial time thanks to the polynomial reduction, and one would have P=NP. On the other hand, if one problem in NP is intractable, then all the problems in NP-c are intractables. Some problems in NP-c, such as the satisfiability problem, have been studied for decades without any proof that they belong to P, that is why one believes that P =NP. Figure 1.2 illustrates the conjecture on the structure of the sets NP, P and NP-c.

From the definition of the NP-c class, proving that a problem π ∈NP belongs to NP-c requires to prove that any problem π ∈NP can be polynomially reduced to π. This seems difficult to achieve, but once one has proved that there is at least one problem in NP-c, the proof can be simplified thanks to this Lemma: Lemma 3 Let π 1 ∈NP-c. If π 2 ∈NP, and π 1 ∝ π 2 , then π 2 ∈NP-c.

Indeed since π 1 ∈NP-c then for any problem π ∈NP one has π ∝ π 1 , then by transitivity π ∝ π 2 . The first problem that has been proved to be NP-complete is the satisfiability problem, by Stephen Cook in 1971 [START_REF] Cook | The complexity of theorem-proving procedures[END_REF] Theorem 1 (Cook,1971) The satisfiability problem is NP-complete.

We will not give the proof, but solely remark that the SAT problem has a very universal structure, that allows to re-write any decision problem into the SAT problem. Once we know that the SAT problem is in NP-c, one can apply the lemma 3 to prove that another problem π is NP-c. The method should follow these steps:

1. Show that π ∈NP.

Select a known problem π ∈NP-c

3. Construct a reduction f from π to π 4. Prove that f is a polynomial reduction.

Using this method, hundreds of problems have been shown to belong to NP-c. The table 1.1 gathers the problems that one has introduced above:

An algorithm for the SAT problem 1.3.1 The Unit Clause Propagation procedure

We have seen that the 2-SAT problem is in P. We present here a polynomial algorithm that solves it. The algorithm is a sequential assignment procedure, which means that at each time step a variable is assigned to a given value. The algorithm ends either when all the variables are assigned, the assignment obtained being SAT, or when it has proven that the formula is UNSAT. Each time a variable is assigned, one can simplify the initial CNF formula according to the following reduction procedure. Suppose that one has assigned the variable i to x i = 1 (the case x i = 0 is symmetric). Each clause containing the literal x i is satisfied by this assignment, therefore can be removed from the formula. In each clause containing the opposite literal x i , one removes the literal x i since it cannot satisfy the clause. The length of these clauses is then reduced by 1. One denotes F |{x i = 1} the simplified formula obtained with this procedure. This reduction procedure might produce a 0-clause, namely if in the formula there a unit clause (a 1-clause) of the form c = (x i ), that is violated by the choice x i = 1. One calls this a contradiction, it means that the simplified formula F |{x i = 1} is UNSAT, therefore to construct a SAT assignment for F , the choice x i = 0 is necessary.

If at some step the formula obtained contains a unit clause, one is forced to satisfy it, and the simplified formula thus obtained might contain new unit clauses that one will have to satisfy in turn. This sequence of forced steps is called Unit Clause Propagation (UCP), and is done with the recursive UCP procedure (see algorithm 1). This procedure takes as input a CNF formula F (possibly containing unit clauses), a partial assignment of variables A, and a set of variables V . In the first call we set F to be the initial CNF formula, and A = ∅, V = {1, . . . , N } Algorithm 1 UCP (F ,A,V )

if there is a 0-clause in F then return UNSAT end if while there exist unit clauses do Pick a unit clause in F , say c = (l i ) and satisfy it Add the assignment of the variable x i in A if there is a 0-clause in F then return UNSAT end if end while return A

The UCP procedure (1) returns all the assignments that were forced due to the presence of unit clauses. There is three possible outputs for the UCP procedure:

1. the output is an assignment of all the variables in V , then F is SAT and the assignment produced is a solution to F .

2. the output is a partial assignment. This partial assignment can be extended to a complete SAT assignment if and only if the input formula F is SAT. The simplified formula obtained does not contain unit clauses.

3. the output is UNSAT, therefore the input formula F is UNSAT.

The algorithm to solve 2-SAT uses the UCP procedure. It works as follows: given a 2-SAT formula F over a set of variables V , choose a variable i ∈ V , and fix x i = 1. Then call the UCP procedure UCP(F |{x i = 1},{x i = 1},V \ i). If the output has assigned all the variables, declare F satisfiable, and return the SAT assignment found. If it is only a partial assignment, keep it in memory, and let F , V be the simplified formula and the set of not-yet assigned variables. Choose a new i variable in V , and restart the UCP procedure UCP(F

|{x i = 1},{x i = 1},V \ i ).
If the output is UNSAT, it means that the initial choice has to be changed. Thus set x i = 0 and restart the UCP procedure with UCP(F |{x i = 0},{x i = 0},V \ i). If once again the procedure outputs UNSAT, then declare that the initial formula F is UNSAT. If not one can keep the partial assignment found and repeat the procedure. The step in which one chooses arbitrarily the value of a variable in a 2-clause is called a free step. By opposition, the assignment of a variable in a unit clause is called a forced step. The step in which we came back to the arbitrary choice done at the free step x i = 1 to change it to x i = 0 is called a backtracking step.

One can measure the complexity of this algorithm by the number of variablefixing operations. For a 2-SAT formula, S. Cook showed that this algorithm has a polynomial complexity [START_REF] Cook | The complexity of theorem-proving procedures[END_REF]. Note however that the algorithm presented above does not provide a method for solving the optimization problem MAX-2-SAT. In fact this problem has been shown to be in NP-c by M.R Garey, D.S Johnson and L Stockmeyer in [START_REF] Garey | Some simplified npcomplete graph problems[END_REF] in 1976.

The DPLL algorithm for the SAT problem

The above algorithm can in fact be slightly modified to work on a CNF formula containing clauses of arbitrary length. In 2-SAT formulas, since any free choice produces unit clauses, one only needs to do backtracking steps on the last variable assigned on a free choice. All the other variables fixed at previous free steps are fixed, since they belong to a partial assignment compatible with at least one solution (if it exists). When there are clauses of length greater than 2 instead, one might have to do several free steps before having the possibility to apply the UCP procedure. If these choices lead to a contradiction, the backtracking then have to explore all the possible choices for these free variables before concluding that the formula is UNSAT. The number of backtracking steps might explode, leading to an exponential time complexity. This algorithm is called the Davis Putnam Logemann Loveland (DPLL) algorithm, from the authors names, and has been introduced in 1962 [START_REF] Davis | A machine program for theorem-proving[END_REF]. It is described by the algorithm 2 (see [START_REF] Moore | The Nature of Computation[END_REF] p.726).

Algorithm 2 DPLL (INPUT: a SAT formula F , OUTPUT: is F satisfiable ?)

If F is empty then return SAT If F contains and empty clause then return UNSAT Select an unset variable

x i If F |{x i = 1} = 1 then return SAT If F |{x i = 0} = 1 then return SAT Return UNSAT
The iterations of DPLL can be represented by a decision tree, as shown in figure 1.3 (the example is inspired from [START_REF] Mézard | Physics, Information, Computation[END_REF]).

It starts at the root of the tree, with the initial formula F containing all the variables to be assigned. A first variable is chosen and assigned to a value (x 1 = 1 in the example). The left-node of the first generation represent the simplified formula obtained with this choice. Since no unit clauses are produced, one needs to make another arbitrary choice: x 2 = 1 in the example. If there is a unit clause, the algorithm fixes the value of one variable present in a unit clause. When the algorithm finds a contradiction, it backtracks to the last variable assigned during a free step, and restarts the search from this node.

From this representation, one can measure the running time of DPLL on the formula F , by counting the number of nodes of the tree. It is intuitively clear why DPLL have an exponential complexity on SAT formulas with clauses of length greater than 2. If most of the variables have been fixed during a free

(1 ∨ 2 ∨ 3) ∧ (2 ∨ 3 ∨ 4) ∧ (1 ∨ 2 ∨ 4) ∧ (1 ∨ 3 ∨ 4) ∧ (1 ∨ 2 ∨ 4) ∧ (2 ∨ 3 ∨ 4) ∧ (1 ∨ 3 ∨ 4) ∧ (1 ∨ 2 ∨ 3) x 1 = 1 x 1 = 0 (2 ∨ 3 ∨ 4) ∧ (2 ∨ 4) ∧ (3 ∨ 4) ∧ (2 ∨ 4) ∧ (2 ∨ 3 ∨ 4) ∧ (3 ∨ 4) (2 ∨ 3) ∧ (2 ∨ 3 ∨ 4) ∧ (2 ∨ 3 ∨ 4) ∧ (2 ∨ 3) x 2 = 1 x 2 = 0 x 2 = 1 x 2 = 0 (3 ∨4) ∧(4) ∧(3 ∨4) ∧(4) (3 ∨4) ∧(3 ∨4) ∧(3 ∨4) ∧(3 ∨4) (3) ∧(3) (3 ∨4) ∧(3 ∨4) x 3 = 1 x 3 = 0 x 3 = 1 UNSAT (4) ∧ (4) (4) ∧ (4) UNSAT (4)
x 4 = 0 UNSAT UNSAT SAT

F = (x 1 ∨ x 2 ∨ x 3 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 4 ) ∧ (x 1 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 4 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 3 )
step, and all the free choices need to be backtracked, the number of steps is roughly the number of nodes of a binary tree of depth N , which is O(2 N ). On a 2-SAT formula instead, each free choice is followed by a cascade of forced steps (possibly O(N ) of them). Even if all the variables have been fixed during a free step, and all the free steps have been backtracked, the complexity is still polynomially bounded by N 2 . In the algorithm 2, one has not specified how to choose the variable during a free step. One can use heuristics that aim at increasing the performances of DPLL. For instance, one can decide to pick a variable that will generate many unit clauses, or a variable that will satisfy the largest number of clauses. Since there is no good criterion to decide which heuristic to use, one would have to experiment them on the formula one has to solve.

Performances of the algorithms and random

CSP ensembles

Complete and incomplete algorithms

An algorithm that solves a CSP, i.e. that provides an answer to all the possible instances of the CSP, is called a complete algorithm. DPLL is an example of complete algorithm for the SAT problem. By opposition, an incomplete algorithm is not guaranteed to provide an answer to any possible instances. In a decision problem, proving that a instance has a solution is often easier than proving that there is no solution. In the SAT problem for instance, a certificate for a 'yes' answer (to the question 'Is there a solution ?') is provided by a solution to the problem, that is an assignment of the N variable, therefore having size N . A certificate for the answer 'no' instead might have an exponential length, if for instance one tries to exhibit all the possible assignments, showing that they all are UNSAT. The display of this certificate by itself have an exponential time complexity. An incomplete algorithm only tries to find a solution, and answers 'I don't know' whenever it is unable to provide one, or to prove that the problem has no solution. We will present in the following chapters several incomplete algorithms for CSPs. Although they do not solve all the possible instances of the problem, it is observed experimentally that they are more efficient than the complete algorithms, on typical instances. To characterize their performances, one needs to precise what we mean by typical instances. This can be done by introducing random ensembles of instances. One then study the performances of algorithms on instances drawn from this ensemble.

Random CSP ensembles

We have seen that an instance of CSP can be represented with a hypergraph. Therefore one can use random graph ensembles to define the random CSP ensembles. The Erdős Rényi (ER) ensemble G N (k, M ) is an example of random hypergraph ensemble. An instance of G N (k, M ) is drawn by choosing independently for each clause a k-tuple of variables uniformly at random among the N k possible ones. Note that while the hyperedges a ∈ C have a fixed degree k, the degree of the vertices i ∈ V is not fixed.

We will also use another random hypergraph ensemble, called the k-uniform (l + 1)-regular random hypergraph ensemble (the choice (l + 1) is made here for convenience when using the cavity method, see chapter 7). In this ensemble both the hyperedges and the vertices have a fixed degree k and l + 1. All the hypergraphs with this property are equiprobable in this ensemble.

To define a random ensemble for the SAT problem, we need to specify how to draw the signs J a i . The random k-SAT ensemble is one of the most studied random ensembles for the k-SAT problem. An instance is obtained by drawing a hypergraph from G N (k, M ), then independently for each variable in each ktuple, one chooses a sign J a i with probability 1/2. It is useful to introduce the density of constraints α = M/N , and to work with the ensemble G N (k, αN ). For the k-uniform (l + 1)-regular random hypergraph ensemble, since N and M must satisfy the relation N (l + 1) = M k, the density of constraints is related to the degrees as l + 1 = αk. Intuitively, increasing the density of constraints, the number of solutions should shrink, since it is harder to satisfy an overconstrained instance. At high density, we will see that most of the instances are in fact UNSAT.

We define the thermodynamic (large size) limit as N, M → ∞ with a fixed ratio α = M/N . In this limit, several properties of the random k-SAT ensemble concentrate around their mean value. This is called the self-averaging phenomenon. In particular, in this limit many random CSPs (including the random k-SAT ensemble) exhibit threshold phenomena. The probability of some property jumps abruptly from 1 to 0 when varying the control parameter α. We say that a property of the instances is typical when its probability goes to 1 in the thermodynamic limit.

The most prominent threshold is the satisfiability threshold α sat (k). Below α sat (k), typical instances are SAT, while above α sat (k), typical instances are UNSAT. In the satisfiability phase α < α sat (k), many other phase transitions concerning the properties of the set of solutions of a random instance are predicted by the so-called cavity method, that we shall describe in the next chapters.

Among them, the clustering transition describes a drastic change in the structure of the set of solutions in the space of configuration. Below the clustering threshold, the set of solutions is rather well-connected, any solution can be reached from any other one by nearby intermediate solutions, while above the clustering threshold the solution set splits in a large number of distinct groups of solutions, called clusters, which are internally well-connected but well separated one from the other.

Performances of DPLL on the random k-SAT ensemble

Our aim is to study and compare the performances of algorithms on a random ensemble of instances. We start by giving the experimental study of the DPLL algorithm on the random k-SAT ensemble.

For a fixed value k, one can plot the fraction of UNSAT instances found by DPLL as a function of α for several sizes N . The result for k = 3 is shown in figure 1.4 taken from the study of S. Kirkpatrick an B. Selman in 1994 [START_REF] Kirkpatrick | Critical behavior in the satisfiability of random boolean expression[END_REF] One can see that it is a increasing function of α, that goes from 0 to 1. As N increases, the drop becomes sharper around a threshold value. This is a numerical evidence of the satisfiability threshold.

One is interested in the running time of DPLL on random instances of k-SAT. On figure 1.5 taken from the study of D. Mitchell, B. Selman, H. Levesque in 1992 [START_REF] Mitchell | Hard and easy distributions for SAT problems[END_REF], the median running time is plotted against α. One can see that the running time has a peak around the satisfiability threshold. The height of this peak increases rapidly with N . At smaller and larger values of α instead, the running time is much smaller, and grows with N at a smaller rate. These observations indicate that the instances drawn from the random k-SAT ensemble are harder when α is close to α sat (k). In the region of smaller α, the instances are underconstrained and therefore easy to solve. In the region of larger α, the instances are so overconstrained that DPLL finds quickly a contradiction showing that the instance is UNSAT.

In [START_REF] Beame | On the complexity of unsatisfiability proofs for random k-cnf formulas[END_REF], P. Beame, R.Karp, T. Pitassi, and M. Saks show that the size of a certificate for an unsatisfiable instance of random 3-SAT is with high probability (w.h.p.) bounded from above by 2 cN/α , with c some contant, where "with high probability" means with probability going to one in the large size limit N, M → ∞ at a fixed ratio α = M/N . This results confirms the decreasing complexity observed in the unsatisfiable phase when α increases above the satisfiability threshold α sat . In [START_REF] Frieze | Analysis of two simple heuristics on a random instance of k-sat[END_REF], A. Frieze and S. Suen show that in the satisfiable phase a modified version of DPLL without backtracking (with the unit clause branching rule) can find solutions efficiently for small enough densities: up to α ≤ 3.003 for k = 3, which is strictly smaller than the satisfiability threshold prediction α sat (k = 3) = 4.267 ([7]). This confirms the experimental observation that DPLL works in polynomial time for small enough densities. In [START_REF] Cocco | Trajectories in phase diagrams, growth processes, and computational complexity: How search algorithms solve the 3-satisfiability problem[END_REF], S. Cocco and R. Monasson provide a theoretical estimation of the algorithmic threshold for DPLL on random 3-SAT, introducing the random (2 + p)-SAT to study the evolution of the formula under the sequential assignments, with p the fraction of 3-clauses in the formula. They predict that above the density α 3.003, the running time of DPLL with the unit clause branching rule becomes typically exponential.

The algorithmic barrier

The numerical study of the performances of DPLL on the random k-SAT ensemble indicates that in the region of α close to the satisfiability threshold the typical instances are hard to solve. In the next chapter we will introduce several incomplete algorithm that are designed to search for solutions, and we will compare their performances on the random ensembles. In particular, one is interested in determining the interval of α where it is possible to find an algorithm that is successful on typical instances. As we have seen this interval has to be in the satisfiable region α < α sat (k), but one could ask whether there exists a range in which typical instances have solutions, but no algorithm is able to find one. More precisely, one is interested in the putative algorithmic barrier α alg (k), above which no algorithm is able to find a solution in polynomial time, assuming P =NP, and for typical instances. Is it possible to show that α alg (k) coincides with α sat (k), by exhibiting an algorithm efficient in the entire satisfiable region? Or are there limitations that force the strict inequality α alg (k) < α sat (k)? The structure of the set of solutions of typical instances undergoes a series of phase transitions in the satisfiable phase, that are predicted by the cavity method. It is therefore interesting to know if some of these phase transitions affect the performances of algorithms. For instance, algorithms such as Monte Carlo Markov Chains for the resolution of CSPs are affected by the clustering transition.

Some properties of the random hypergraph ensembles

In the ER ensemble G N (k, M ) the degree of a uniformly chosen vertex i is a binomial random variable B(M, k/N ). In the thermodynamic limit therefore the degree of i obeys a Poisson law of mean αk.

A crucial property shared by the ER ensemble G N (k, αN ) and the k-uniform (l +1)-regular random hypergraph ensemble is their locally tree-like structure in the thermodynamic limit. Given a uniformly chosen vertex, the neighborhood within a finite distance is, with probability going to 1 as N → ∞ (one says with high probability) acyclic. Consider a random instance of hyperhraph F , and a uniformly chosen vertex i of F , and let B i,r (F ) be the subgraph containing the vertices at distance smaller or equal than r from i. Then it is shown in [START_REF] Aldous | The objective method: Probabilistic combinatorial optimization and local weak convergence[END_REF] that B i,r (F ) converges in distribution to the random tree ensemble T r (k, α).

An instance of T r (k, α) is constructed from the Galton Watson branching process: starting from the root, one draws its degree d from the distribution p d . For the G N (k, αk) ensemble one take the Poisson law p d = e -αk (αk) d /d!, while for the regular ensemble one has p d = I[d = l + 1]. We attach to the root d function nodes, and each of them are then connected to k -1 variable nodes, that constitute the first generation of the tree. Then for each variable node of the first generation, one draws an independent integer d with probability (d +1) . The distribution r d is called the residual degree distribution. It is the probability, given that the variable node i is connected to a given function node a, that i is connected to d other function nodes. For the G N (k, αN ) ensemble r d is also distributed according to a Poisson law of mean αk, while for the k-uniform (l+1)-regular random hypergraph ensemble one has

r d = p d (d+1) d ≥0 p d
r d = I[d = l].
One reiterates the same process for each generation of the tree, until the r-th generation.

Since in the large size limit a random graph converges locally to a tree, one is then interested in the typical length of loops in these ensembles. It can be showed that for a random hypergraph F and a randomly chosen vertex i, the length l i of the shortest loop going through i in F is O(log N ), with high probability (w.h.p.). This scaling can be understood by the following heuristic argument. In G N (k, αN ), the average number of variable nodes in B i,r (F ) is (αk(k -1)) r . If one considers that loops appear when this number becomes comparable to the total number of nodes N (actually when it becomes comparable to √ N by the Birthday Paradox), then one obtains r ∝ ln N/ ln(αk(k -1)).

Statistical physics and constraint satisfaction problems 1.5.1 Boltzmann probability measure

There is a natural mapping between optimization problems and statistical physics problems. In optimization problems, the goal is to minimize a cost function E defined on the set of possible configurations. For a CSP, there is a natural cost function E : χ N → R + that counts the number of clauses violated by a given assignment x ∈ χ N of the N variables:

E(x) = M a=1 (1 -c a (x ∂a )) (equation (1.1)).
For each instance F of a CSP, one introduces a Boltzmann probability distribution on the set of configurations:

µ β (x) = 1 Z(β) e -βE(x) (1.6)
the parameter β plays the role of an inverse temperature. In this setting the cost function E is also called the Hamiltonian, or energy function. The normalization Z(β) is called the partition function. In the β → ∞ limit, the probability distribution µ β concentrates on the configurations that minimize the energy. These configurations are called ground states in statistical physics, and corresponds to the optimal configurations of the optimization problem. In the decision version of the problem (with a threshold E 0 = 0), a solution is a zero-energy configuration. If the instance F admits solutions, then in the large β limit the probability distribution (1.6) converges to the uniform measure on the set of solutions, and the partition function counts the number of solutions.

Using the expression (1.1) of the energy-cost function E, one can re-write the probability distribution µ β :

µ β (x) = 1 Z(β) e -β M a=1 (1-ca(x ∂a ))
(1.7)

Statistical physics models

From the above expression, one sees the mapping from a CSP to a statistical physics problem. In this problem, the variables x i ∈ χ interact through the local couplings ψ a (x ∂a ) = 1-c a (x ∂a ). The graphical representation of the instance F can be viewed as the underlying graph representing the local interaction between variables.

Spin glass model are a generalization of the Ising model, in which the energy function reads

E(σ) = - M a=1 J a i∈∂a σ i (1.8)
the variables σ i are spins σ i ∈ {+1, -1}, and the coupling J a is either in R or in {-1, +1}. The p-spin model is is the restriction of the general spin glass model to p-body interaction. The 2-spin model is also called the Edward-Anderson model.

In the zero temperature limit, the spin glass model can be seen as a CSP: the Boltzmann measure µ β is concentrated on the configurations which minimize the energy (1.8). A solution of the spin-glass problem should satisfy the following constraints. If J a > 0 (resp. J a < 0), then the product i∈∂a σ i should be equal to 1 (resp. -1). For a two body interaction, it means that the variables should be aligned (resp. anti-aligned), therefore it is called a ferromagnetic (resp. antiferromagnetic) interaction.

Note that an instance of XORSAT can be transformed into an instance of the spin glass CSP with couplings

J a living in {-1, +1}. Recall that a XORSAT clause c a = (l i(1) ⊕ • • • ⊕ l i(|∂a|)
) equals 1 when the sum of literals is equal to 1 modulo 2. Let n a be the number modulo 2 of variables that appear negated (the variables such that l i = x i ) in the clause a. Then the constraint

l i(1) ⊕• • •⊕l i(|∂a|) = 1 can be rewriten x i(1) ⊕• • •⊕x i(|∂a|) = b a , with b a = 1-n a .
Setting J a = 1 -2b a the above constraint is equivalent to ask for the constraint J a σ i(1) . . . σ i(|∂a|) = 1 to be satisfied.

Conversely, each instance of the spin glass problem (with couplings J a ∈ {-1, +1}) can be transformed into a XORSAT instance. We conclude that the decision version of the CSP associated with spin glasses at zero temperature is in P.

Graphical models

The measure (1.6) belongs to a family of measures described by graphical models, that we define here. Let G = (V, C, E) be a bipartite graph, called a factor graph. V is called the set of variable nodes, C is the set of factor nodes, and E is the set of edges between variables and factors. For each variable node i ∈ V we denote by ∂i = {a ∈ C : (ia) ∈ E} the set of factor nodes connected to i, and for each factor node a ∈ C we denote by ∂a = {i ∈ V : (ia) ∈ E} the set of variable nodes connected to a. One defines also a space of configurations χ N , with N = |V |, and χ a finite alphabet. Finally let {ω a } a∈C be a set of non-negative weights ω a : χ |∂a| → R + . A measure µ over χ N described by the graphical model G, χ, {ω a } is defined as follows:

µ(x) = 1 Z a∈C ω a (x ∂a ); Z = x a∈C ω a (x ∂a ) (1.9)
The measure is well defined only if there exists at least one configuration for which all the weights ω a are strictly positive. Note that in the case of the measure (1.6) defined for a given instance F of CSP, the factor graph coincides with the graph representing F , and the weights {ω a } are written: ω a (x ∂a ) = e -β(1-ca(x ∂a )) . The uniform measure on the set of solutions of F (assuming it is non-empty) is obtained by taking the limit β → ∞ in µ β . It can be written

µ u (x) = 1 Z u a∈C (1 -c a (x ∂a )) = 1 Z u a∈C I[x ∂a satisfies the clause a] (1.10)
where the subscript 'u' stands for 'uniform'.

We will have to define other measures associated with the instance F , for which the factor graph will be sometimes slightly different to the initial one representing F . Therefore it is more convenient to work with this general definition. We will generally denote by Θ the set of parameters needed to describe a given family of graphical models: i.e. the rules to construct the factor graph from the initial graph representing an instance F , and the function space in which the set of functions {ω a } a∈C belongs. For a given instance F , one defines Θ(F ) to be the specification of these parameters to the instance F , and let µ Θ(F ) be the measure associated with these parameters.

A measure µ described by a graphical model have the following decorrelation property called global Markov property. Let A, B, and S be three disjoint subset of V . One says that S separates A and B when there is no paths joining a node in A to a node in B without passing through S. Then (from [START_REF] Mézard | Physics, Information, Computation[END_REF] proposition 9.2): [START_REF] Molloy | The freezing threshold for k-colourings of a random graph[END_REF] one says that the variables x A , x B are conditionally independent. In the chapter 4 we will describe the message-passing algorithms, that are built using the Markov property, and that allow to compute the properties of the measure µ when the factor graph is a tree.

P µ [x A , x B |x S ] = P µ [x A |x S ]P µ [x B |x S ] (1.

Chapter 2

Phase transitions in random CSPs

We give now a more detailed picture of the phase transitions occuring in random CSPs when the density of constraints α is varied. We will take the particular example of the random k-SAT ensemble, but many other random CSP ensembles share the same qualitative picture.

The satisfiability transition

We say that an event occurs with high probability (w.h.p.) when its probability goes to 1 in the thermodynamic limit. We recall that the satisfiability threshold α sat (k) separates a phase α < α sat (k) where random instances are SAT w.h.p. to a phase where random instances are UNSAT w.h.p. The cavity method [START_REF] Monasson | 2+p-sat: Relation of typical-case complexity to the nature of the phase transition[END_REF][START_REF] Biroli | A variational description of the ground state structure in random satisfiability problems[END_REF][START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Mertens | Threshold values of random k-sat from the cavity method[END_REF][START_REF] Krzakala | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF] provides an estimate of the satisfiability threshold. However, the existence of the satisfiability transition is not yet proven for all values of k. It is summarized in the following conjecture:

Conjecture 1 Let F be a random CNF formula with N variables and M = αN clauses, drawn from the random k-SAT ensemble. Let P N (k, α) be the probability that the formula F is SAT. For any k ≥ 2 there exists a constant α sat (k) such that for all > 0, lim

N →∞ P N (k, α sat (k) -) = 1
, and lim

N →∞ P N (k, α sat (k) + ) = 0 (2.1)
This conjecture has been proven for k = 2, with α sat (2) = 1 by V. Chvatal, B. Reed (1992) [START_REF] Chvatal | Mick gets some (the odds are on his side) (satisfiability)[END_REF], W. Fernandez de la Vega (1992) [START_REF] De | On random 2-sat[END_REF], and A. Goerdt (1996) [START_REF] Goerdt | A threshold for unsatisfiability[END_REF]. In [START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF], J. Ding, A. Sly and N. Sun prove the satistfiability conjecture for large but finite k, and show that the value of α sat (k) is given by the onestep symmetry breaking cavity method prediction. Proving the existence of the satisfiability threshold for all values of k ≥ 3 is an open problem, however the following theorem by E. Friedgut (1999) [START_REF] Friedgut | Sharp thresholds of graph proprties, and the k-sat problem[END_REF] provides a partial result in this sense:

Theorem 2 (Friedgut) For every k ≥ 2, there exists a sequence α k (N ) such that for all > 0, lim

N →∞ P N (k, α k (N ) -) = 1
, and lim

N →∞ P N (k, α k (N ) + ) = 0 (2.2)
The above result states that the transition from SAT to UNSAT takes place in a window smaller than any fixed for N large enough. However it remains to prove the convergence of the sequence α k (N ) to some value α sat (k) as N → ∞, to prevent from possible oscillations. Upper and lower bounds have been established rigorously on the sequence α k (N ). In the following we present some of the methods used to derive these bounds.

Upper bounds

Upper bounds can be obtained with the first moment method. We describe the general strategy (presented in [START_REF] Mézard | Physics, Information, Computation[END_REF]). Define a function on the set of instances U (F ) such that

U (F ) = 0 if F is UNSAT ≥ 1 otherwise (2.3)
Then one applies the Markov inequality to U (F ) to obtain:

P[F is SAT] ≤ E[U (F )] (2.4)
note that the equality is reached for U (F ) = I[F is SAT], but we do not know how to compute this quantity. One looks instead for a simpler function for which E[U (F )] can be computed, and has a vanishing limit when N → ∞ for large enough α. As a first choice one can use U (F ) = Z(F ), the number of solutions of F (see for instance [START_REF] Franco | Probabilistic analysis of the davis-putnam procedure for solving satisfiability[END_REF]). By linearity of the expectation and by uniformity in the generation of the clauses the expectation E[Z(F )] is equal to the total number of assignments 2 N times the probability that a given assignment σ 0 is SAT for F . This probability is the product over the clauses of the probability that each clauses is satisfied by σ 0 , therefore:

E[Z(F )] = 2 N (1 -2 -k ) M = exp[N (ln 2 + α ln(1 -2 -k ))] (2.5)
In the large N limit one gets:

E[Z(F )] → 0 if α > α U B (k) +∞ α < α U B (k) , α U B (k) = - ln 2 ln(1 -2 -k ) (2.6)
and we have obtained the upper bound α U B (k) on the conjectured satisfiability threshold α sat (k). Intuitively, one can expect that this bound is not tight, because Z(F ) can take exponentially large values in N , and moreover its fluctuations can be exponentially large as well. In order to improve on this bound a possible strategy is to choose a function U (F ) that counts the number of assignments in a small subclass of solutions. If the subclass is small enough one can hope to reduce the fluctuations of U (F ) and obtain a tighter upper bound. In [START_REF] Lefteris | Approximating the unsatisfiability threshold of random formulas[END_REF] (1998), L. Kirousis, E. Kranakis, D. Krizanc, and Y. Stamatiou (1998) use the subclass of locally maximal SAT assignments, where a locally maximal SAT assignment is a SAT assignment such that for each variable i such that x i = 0, flipping the variable to x i = 1 leads to an UNSAT assignment. They define U (F ) to be the number of locally maximal SAT assignment and apply the first moment method to this function. They obtain the upper bound α U B (k), that is solution of the equation

α ln(1 -2 k ) + ln 2 -exp -kα/(2 k -1) = 0 (2.7)

Lower bounds

To derive a lower bound on α k (N ) one can use the second-moment method. The method was introduced by D. Achlioptas and C. Moore in 2006 [START_REF] Achlioptas | Two moments suffice to cross a sharp threshold[END_REF]. Again the strategy is to use a function U (F ) that vanishes when the formula is UNSAT, and that is strictly positive otherwise. Applying the Cauchy-Schwarz inequality to U (F ) we get:

P[F is SAT] ≥ E[U (F )] 2 E[U (F ) 2 ] (2.8)
One needs to find a function U (F ) for which the above ratio

E[U (F )] 2 E[U (F ) 2
] is not too difficult to compute and has a non vanishing limit when N → ∞ limit for small enough α. One can show that applying this method to Z(F ) does not provide a useful bound, because the ratio

E[Z(F )] 2 E[Z(F ) 2 ]
vanishes for any non-zero value of α. It is however possible to apply this technique to another function U . In particular, one can choose U to be the size of a subset of the set of solutions. By choosing carefully this subset, one can hope to reduce the flutuations of U (F ) around its mean, so that the ratio

E[U (F )] 2 E[U (F ) 2 ]
is not vanishing in the large N limit. Using this approach, D. Achlioptas and C. Moore show in [START_REF] Achlioptas | Two moments suffice to cross a sharp threshold[END_REF] (Theorem 1.) that for k ≥ 3: lim

N →∞ P N (k, α) = 1 if α ≤ 2 k-1 ln 2 -2 (2.9)
Note that in the large k limit, this result combined with the upper bound α U B (k) provides the scaling

α sat (k) = O(2 k ).
Another approach consists of analysing the behavior of some explicit algorithm that search for SAT assignments. If one can show that a particular algorithm finds a SAT assignment with positive probability when α is smaller than some value, then applying the Friedgut theorem 2 one can deduce that this happens w.h.p., therefore this value is a lower bound on the satisfiable threshold. More details on this analysis are given in the section 5.2.2.

Quenched and annealed averages

Consider a measure µ Θ(F ) , defined with a graphical model Θ, whose support is the set of solutions of a given instance F :

µ Θ(F ) (σ) = 0 if σ is not a solution > 0 if σ is a solution (2.10)
One might sometimes relax the condition µ Θ(F ) (σ) = 0 when σ is not solution, and introduce a temperature β. The free entropy density is defined as follows:

Φ(Θ(F )) = 1 N ln Z(Θ(F )) (2.11)
When the instance F is randomly drawn (for instance from the random k-SAT ensemble), the measure µ Θ(F ) becomes itself random. The goal of the cavity method is to determine the typical properties of the measure µ Θ(F ) and of the free entropy density Φ(Θ(F )). To do so one defines the quenched free entropy density:

Φ q (Θ) = lim N →∞ 1 N E[ln Z(Θ(F ))] (2.12)
where the average E is taken on the random ensemble of instances. In many statistical physics systems, the free entropy density Φ(Θ(F )) concentrates thanks to the self-averaging phenomenon, therefore the quenched free-entropy density indeed contains information on the properties of the typical measures µ Θ(F ) drawn from the random ensemble. Usually, the quenched free energy cannot be evaluated exactly for random CSPs. Its value can however be estimated with the cavity method, that we will describe in the next chapters. There is a natural upper bound on the quenched free entropy density, provided by the annealed free entropy density:

Φ a (Θ) = lim N →∞ 1 N ln E[Z(Θ(F ))] (2.13)
From Jensen's inequality applied on the random variable Z(Θ(F )) one obtains

Φ q (Θ) ≤ Φ a (Θ) (2.14)

Overview of the phase transitions in random CSPs

The cavity method predicts several phase transitions affecting the structure of the set of solutions of typical instances, in the satisfiable phase. We give in this section a qualitative description of the phase transition, and will explain the cavity computation by itself in the next chapters.

The clustering transition

The clustering transition occurs in the satisfiable phase α < α sat (k). It is also known as the reconstruction or the dynamical transition. This transition can be described from various perspectives.

Looking at the set of solutions of typical instances, the clustering threshold separates a regime where the set of typical solutions is rather well connected, any solution can be reached from any other by via nearby intermediate solutions. Moreover, this rearrangement is only on a non-extensive number of variables. Above α d (k) the typical solutions breaks into an exponential number of clusters, or pure states, which are internally well connected, but separated one from each other by free-energy barriers.

Note that this definition concerns typical solutions with respect to the measure µ Θ(F ) chosen to describe the set of solutions. In fact, the set of solutions could be connected, but by narrow, winding paths that a Markov chain would take exponential time to find, and changing the measure could raise or lower the total probability of the paths from one region of state space to the other, changing the mixing time from exponential to polynomial. Therefore the value of the clustering threshold depends on this choice, and we denote explicitly α d (k, Θ) the clustering threshold. Usually, the uniform measure is chosen to describe the set of solutions, but it is possible to introduce non-uniform measures, and study how the clustering threshold is affected by this change.

As already mentioned, the satisfiability threshold α sat (k) separates the SAT phase where typical instances are satisfiables w.h.p., and the UNSAT phase where typical instances are unsatisfiables w.h.p.. This definition has to do with the whole set of solutions of the instance, not only typical solutions, therefore the value α sat (k) does not depend on the specific choice of the measure µ Θ .

The clustering transition can also be interpreted as the birth of the point-toset correlation function under the probability measure µ chosen to describe the set of solutions. It was introduced in [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF] by A. Montanari and G. Semerjian in 2006 (see also [START_REF] Bouchaud | On the Adam-Gibbs-Kirkpatrick-Thirumalai-Wolynes scenario for the viscosity increase of classes[END_REF]). Given a variable node i and a set of variable nodes B, the point-to-set correlation function is defined for spin variables as follows (we used the definition in [START_REF] Montanari | Clusters of solutions and replica symmetry breaking in random k-satisfiability[END_REF] equation [START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF]):

C(i, B) = σ B P µ (σ B ) σi P µ (σ i |σ B )σ i 2 - σi P µ (σ i )σ i 2 (2.15)
Let d(i, B) be the distance from i to B on the graph of interactions. This correlation measure how much information on the value of one spin variable i (the point) is provided by the observation of all spins in B (the set) at distance d(i, B) from it. In the unclustered regime the point-to-set correlation C(i, B) vanishes when d(i, B) grows, while it does not decay to 0 in the clustered regime. In [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF] A. Montanari and G. Semerjian show that this correlation implies the divergence of the relaxation time of local stochastic processes that respect the detailed balance condition (such as the Monte Carlo Markov chains), which justifies the terminology dynamic transition. One can understand this slow down in the dynamics from the appearance of energetic barriers separating the clusters.

This transition also implies the solvability of the tree reconstruction problem introduced in [START_REF] Mossel | Information flow on trees[END_REF] in 2003 by E. Mossel and Y. Peres. For the first discussion of the connection between 1RSB equations and the reconstruction problem, see the paper from M. Mézard and A. Montanari [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF]. We will present this problem and explain how to compute the clustering threshold in its formalism in the chapter 7 (section 7.3).

In the cavity method treatment of random CSPs the clustering threshold is defined as the appearance of a non-trivial solution of the one step of Replica Symmetry Breaking (1RSB) equation with Parisi parameter X = 1.

Cluster decomposition

In the unclustered phase, the typical solutions belongs to a single cluster. The thermodynamic properties of the measure µ are well described by the Replica Symmetric (RS) cavity method, that in particular provides an estimate Φ RS of the quenched free entropy density

Φ q = lim N →∞ (1/N )E[ln Z].
The RSB phenomenon described in the framework of spin glasses occurs in random CSPs precisely at the clustering transition. The assumption of the RS cavity method are not valid anymore, and one uses instead the one replicasymmetry-breaking (1RSB) cavity method. In this setting, one assumes that the solution set Ω splits into an exponential number of disjoint clusters (also called pure states) {Ω c }, and that the restriction µ c of the measure µ to one of these clusters is efficiently described by the RS cavity method. For instance, the point-to-set correlation C(i, B) associated with µ c is vanishing in the large d(i, B) limit, while µ has a non vanishing point-to-set correlation function. Let us write µ(σ) = ω(σ)/Z, with ω(σ) the weight of the configuration σ, and Z = σ ω(σ) the partition function. The partition function Z is decomposed into the contribution of each clusters:

Z = σ ω(σ) = c Z(c); Z(c) = σ∈Ωc ω(σ) (2.16)
where Ω c is the set of solutions belonging to the cluster c. One can decompose the measure µ as follows:

µ(σ) = c p(c)µ c (σ) (2.17)
with µ c the restriction of the measure µ to configurations in the cluster c:

µ c (σ) = I[σ ∈ Ω c ]ω(σ) Z(c) (2.

Complexity and the condensation transition

Let φ c = (1/N ) ln Z(c) be the free entropy density associated with the configurations of the cluster c. At the leading order in N , the number of clusters with a given free entropy density φ is assumed to be exponential in N : e N Σ(φ) , where Σ(φ) is called the complexity. Σ(φ) is assumed to be self-averaging, and plays the role of an entropy density for the set of clusters. One can re-write the partition function Z as an integral over φ:

Z = dφe N [φ+Σ(φ)] (2.19)
The total free entropy density Φ = (1/N ) ln Z can be evaluated via the Laplace method: Φ = sup

φ:Σ(φ)≥0 [Σ(φ) + φ] (2.20)
the condition Σ(φ) ≥ 0 ensures that the clusters exist in the thermodynamic (large N ) limit. The total free entropy thus results in a competition between clusters of large free entropy φ, and the most numerous clusters (with large Σ(φ)). Let φ * be the value achieving the supremum. Under the law µ, we say that a solution is typical if it belongs to a cluster of free entropy density φ * . If Σ(φ * ) > 0, there is an exponential number of clusters contributing to the total free entropy density. If Σ(φ * ) = 0 instead, there is only a sub-exponential number of dominant clusters.

Increasing the value of α, there is a transition from a phase where Σ(φ * ) > 0 toward a phase where Σ(φ * ) = 0. It is called the condensation transition, and denoted α c (k, Θ), where we have emphasized the dependence of this transition on the choice of the measure describing the set of solutions µ Θ .

Note that by construction one has α d (k, Θ) ≤ α c (k, Θ) ≤ α sat (k). Figure 2.1 shows a sketch of the dynamical, condensation and satisfiability transitions on the set of solutions.

Computing the complexity function

In order to compute the condensation threshold α c one needs to compute the complexity function Σ. To do so (see [START_REF] Monasson | Structural glass transition and the entropy of the metastable states[END_REF]), one introduces the following potential, called replicated entropy:

Φ 1 (X ) = 1 N ln Z 1 (X ); Z 1 (X ) = c (Z(c)) X (2.21)
where X is a real parameter called the Parisi parameter (the subscript '1' is just here to remind that this function is used in the 1RSB framework). It plays the role of an inverse temperature for the modified distribution over the set of clusters:

p X (c) = (Z(c)) X Z 1 (X ) = e X N φc Z 1 (X ) (2.22)
it allows to weight differently the clusters according to their free entropy. As we did before for the true partition function Z, one can re-write the modified partition function Z 1 (X ) as an integral over φ:

Z 1 (X ) = dφe N [X φ+Σ(φ)] (2.23)
Evaluating the integral via the Laplace method yields

Φ 1 (X ) = sup φ [Σ(φ) + X φ] = Σ(φ * 1 (X )) + X φ * 1 (X ) (2.24)
where we defined

φ * 1 (X ) = argmax φ [X φ + Σ(φ)] (2.25)
Thus the Parisi potential Φ 1 (X ) and the complexity Σ(φ) are Legendre transforms of each other. One can invert this relation to get

Σ(φ * 1 (X )) = Φ 1 (X ) -X dΦ 1 (X ) dX (2.26)
The potential Φ 1 (X ) and its derivative can be estimated with the 1RSB cavity method, leading to an estimation of the complexity function. In most of the cases, the complexity function is a concave function of φ, as represented in figure 2.2

In the uncondensed regime α d ≤ α ≤ α c , the expression (2.20) achieves its supremum for a value φ * which satisfies the constraint Σ(φ) ≥ 0, therefore φ * = φ * 1 (X ) and Φ = Φ 1 (X = 1). It can be checked that the prediction Φ 1 (X = 1) coincides with the RS prediction Φ RS of the free entropy denstity. Therefore in the clustered-uncondensed regime the thermodynamic prediction of the RS assumption is valid. The phase transition α d does not affect thermodynamic quantities. As we have seen, this transition can be detected by the slow down of the dynamics, therefore this phase is also called the dynamic 1RSB (d1RSB) Right: condensed regime, the supremum of (2.20) is achieved for φ * such that Σ(φ * ) = 0 and (dΣ(φ)/dφ) = -X s (green tangent), the value of φ such that (dΣ(φ)/dφ) = -1 has a negative complexity.

Σ Σ φ φ -1 -1 -X s
phase. The order parameter that detects this phase transition is the point-to-set correlation function.

In the condensed regime α c ≤ α ≤ α sat , the value of φ such that (dΣ(φ)/dφ) = -1 has a negative complexity, and the supremum of (2.20) is achieved for a smaller value X s < 1: Φ = Φ 1 (X s ). The free entropy density is not described anymore by the RS prediction Φ RS , and the system undergoes a first-order phase transition. The condensed phase is therefore also called the static 1RSB (s1RSB) phase. Note that values of the Parisi parameter such that X > 1 never contribute to the computation of the thermodynamic quantities (it can be however useful to look at these values if one is interested in atypical cluster sizes).

One can resume these observations with the following expression

Φ = inf X ∈[0,1] Φ 1 (X ) X (2.27)
which is valid in all phases. In the unclustered phase α < α d (k, Θ) there is only one cluster, one has Σ(φ) = 0 for all φ, therefore from the definition of Z 1 (X ) one gets Φ 1 (X ) = X φ = X Φ, and the equation (2.27) is correct. In the clustered phase, call X * the minimizer of the expression (2.27). In the uncondensed phase one has X * = 1. Indeed from the expression (2.24) of Φ 1 (X ) one sees that

d(Φ 1 (X )/X ) dX = - Σ(φ(X )) X 2 (2.28)
hence in the uncondensed phase since Σ(φ(X )) = Σ(φ * (X )) > 0 the function Φ 1 (X )/X is decreasing and reach its minimum at X = 1. Therefore the expression (2.27) indeed gives Φ = Φ 1 (1). In the condensed regime, one has Σ(φ(X = 1)) < 0, therefore the equation (2.27) selects the value X s such that Σ(φ(X s )) = 0. Since Φ 1 (X s ) = X s φ(X s ) one gets back to the correct result: Φ = Φ 1 (X s ).

Computing α c and α sat from the complexity

In practice, to compute the condensation threshold one needs to compute the value of the complexity function at X = 1: Σ(X = 1) = Σ(φ * 1 (X = 1) from the expression (2.26), and let α c be the value at which Σ(X = 1) vanishes.

The complexity function allows to predict the value of the satisfiability threshold. Note that at X = 0, the partition function Z 1 (X = 0) counts the number of clusters, independently of their size. Note also that for X = 0 one has Φ 1 (X = 0) = sup φ [Σ(φ)] = Σ(φ * 1 (X = 0)). Therefore when Σ(φ * 1 (X = 0)) becomes negative it means that for typical large instances all the clusters have disappeared, hence that there is no solution.

Rigidity and freezing transitions

From the notion of clusters, one defines the notion of frozen variable inside a cluster. They are variables that take the same value in all the solutions of the corresponding cluster. One defines the rigidity transition α r (k, Θ). Above the rigidity transition typical solutions (with respect to µ Θ ) contain an extensive number of frozen variables. One defines the freezing transition α f (k), above which all solutions contain an extensive number of frozen variable. The freezing transition concern all solutions, not only typical one, and thus is harder to compute than the rigidity transition. The freezing threshold has been introduced in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], its value has been computed using a large deviation derivation, for the bicoloring of hypergraphs. By definition the following inequality must hold:

α d (k, Θ) ≤ α r (k, Θ) ≤ α f (k) ≤ α sat (k).
One says that a solution is frozen when it contains an extensive number of frozen variables. Experimentally, it is found that several incomplete algorithms (such as simulated annealing or focused algorithms that we will present in the next chapter) only found unfrozen solutions. Even in the rigidity phase α r < α < α f where typical solutions are frozen, it seems that it is easier for these algorithms to find untypical unfrozen solutions. The freezing transition thus appears as a good candidate for an upper bound on the algorithmic barrier α alg (k). Indeed, one can hope that as long there is some untypical unfrozen solutions, an efficient algorithm should be able to find them. In [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], a biased measure on the set of solutions is introduced in order to turn to typical the unfrozen solutions that were untypical in the uniform measure µ u . In the new biased measure, the rigidity threshold is increased up to densities close to the satisfiability threshold.

Some values of the thresholds

We now give some values of the thresholds introduced above. In the table 2.1 the threshold values for the random k-SAT ensemble for the uniform measure over the set of solutions (see [START_REF] Montanari | Clusters of solutions and replica symmetry breaking in random k-satisfiability[END_REF] Table 1.). Note the qualitative change between k = 3 and k ≥ 4. At k = 3 the clustering threshold equals the condensation threshold. In that specific case, the phase transition separates the unclustered phase from a full-RSB phase, and the complexity computed under the 1RSB approximation (see section 2.3.4) is not valid. For k ≥ 4 instead there a clustered-uncondensed phase. The thresholds for the q coloring problem defined on the random (l + 1)regular graph ensemble (with k = 2) are shown in the table 2.2 (see [START_REF] Zdeborová | Phase transitions in the coloring of random graphs[END_REF]). The convention for l sat is that the random graphs are w.h.p. uncolorable if l ≥ l sat (q). Similarly, the clustered (resp. condensed) phase is located in the region l ≥ l d (resp. l ≥ l c ). In the table 2.3 we give some of the thresholds for the bicoloring problem on the random (k, l + 1)-regular ensemble. These results are taken from [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF]. Note that even if the model is only defined for integer values of l, the threshold values of the rigidity l r and the condensation l c are given as real numbers, since these two thresholds can be expressed as analytic functions of l. One has to round this real threshold to its two nearest integers, to find the largest (resp. smallest) l such that the property (the presence of frozen variables, or the condensation of the set of solutions) is true (resp. false) in the thermodynamic limit. Note that we have made the arbitrarily choice to show the thresholds of the k-SAT problem on the Erdős Rényi ensemble, and the thresholds of the q-coloring problem on the regular ensemble, but in practice there is no technical limitation to compute the thresholds of these two problems on any of these two ensembles. Chapter 3

k l d l r l c l sat 3 5

Local Search Algorithms

We will present now a family of incomplete algorithms that search for solutions of an instance of CSP. Starting from an initial configuration x 0 , they perform a walk in the configuration space, with a bias toward the set of solutions. The term local means that at each step, only one variable (or sometimes a small number with respect to N ) is flipped.

Simulated Annealing

We have seen that one can re-write a CSP into the form of a statistical physics problem. We can now apply the tools developed for physical problems to CSPs.

In particular, the Monte Carlo method allows to sample from a Boltzmann distribution µ β as defined in (1.6) with finite inverse temperature β. In the zerotemperature limit (β → ∞) the Monte Carlo method cannot be applied directly, because the convergence hypotheses are not fulfilled (specifically irreducibility).

To sample from the Boltzmann distribution with β → ∞, and thus extract the solution of the instance F , one can use the Simulated Annealing algorithm. This method performs several steps of Monte Carlo, decreasing the temperature at each step. The application of Simulated Annealing for optimization problem has been introduced in [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF] by S. Kirkpatrick, C. D. Gelatt Jr., M. P. Vecchi in 1983.

Monte Carlo method

This method is an simple way to sample from a Boltzmann distribution at finite temperature, when the number N of variables gets large. One constructs a Monte Carlo Markov Chain which is guaranteed to converge to the target distribution µ β . The transition rules ω(x → x ) of the Markov Chain must satisfy the following conditions in order to ensure the convergence:

1. irreducibility: for any pair of configuration x, x there exists a path (x 0 , . . . , x n ) of length n with x 0 = x, x n = x , and with non-zero transition probabilities: ω(x i → x i+1 ) > 0 for all i = 0, . . . , n -1 2. aperiodicity: for any pair of configurations x and x there exists a positive integer n(x, x ) such that for any n ≥ n(x, x ) there exists a path of length n connecting x to x with non-zero probability. (This is automatically verified for an irreducible chain if one enforces the additional condition ω(x → x) > 0)

3. stationarity: the distribution µ β (x) is stationary with respect to the transition probabilities ω(x → x ):

x µ β (x)ω(x → x ) = µ β (x ) (3.1)
The last condition is also called the global balance condition. A stronger condition is often enforced, called the detailed balance condition. For any pair x, x such that either ω(x → x ) > 0 or ω(x → x) > 0 one requires:

µ β (x)ω(x → x ) = µ β (x )ω(x → x) (3.2)
Note that the detailed balance condition implies the global balance condition. In many cases the detailed balance is used as an easy way to prove global balance. Under the above conditions on the transition probabilities, the MCMC converges to the target distribution µ (see [START_REF] Mézard | Physics, Information, Computation[END_REF] theorem 4.12): Theorem 3 Assume that the rates ω(x → x ) satisfy the three conditions mentionned above. Let X 0 , . . . , X t , . . . be random variables distributed according to the Markov Chain with rates ω(x → x ), and initial condition X 0 = x 0 . Let f : χ N → R be a real valued function, then:

1. The probability distribution of X t converges to the stationary distribution:

lim t→∞ P[X t = x] = µ β (x) (3.3)

Time averages converge to averages over the stationary distribution

lim t→∞ 1 t t s=1 f (X s ) = x µ β (x)f (x) almost surely (3.4)
The Monte Carlo method consists of defining the transition rules of a Monte Carlo Markov Chain that converges toward the desired distribution, and then simulating it on a computer until convergence. The Metropolis and the Glauber dynamics are two examples of MCMC adapted to a Boltzmann measure µ β (σ) = e -βE(σ) /Z(β) defined over a set of spin variables.

Metropolis algorithm

Let σ (i) be the configurations that coincides with σ except for the variable i: σ

(i) i = -σ i . Let ∆ i = E(σ (i)
) -E(σ). At each step, a site i is chosen uniformly at random, and its spin variable is flipped with probability

ω i (σ) = exp[-β max(∆ i , 0)] (3.5)
The transition probabilities can be expressed in terms of the ω i :

ω(σ → τ ) = 1 N N i=1 ω i (σ)I[τ = σ (i) ] + 1 - 1 N N i=1 ω i (σ) I[τ = σ] (3.6)
One can check that ω(σ → τ ) satisfy all the conditions that ensure the convergence of the Markov Chain towards the measure µ β . The Algorithm 3 shows an implementation of the Metropolis algorithm.

Heat bath algorithm

Another possibility is to use the Glauber dynamics. Compared to the Metropolis algorithm, the only change is in the expression of ω i :

ω i (σ) = µ β (σ (i) ) µ β (σ) + µ β (σ (i) ) = 1 2 1 -tanh β∆ i 2 (3.7)

Relaxation time scale of the Monte Carlo dynamics

The performance of the Monte Carlo method is affected by the phase transitions occuring in statistical physics models. In the low temperature (or high density) phase, the dynamics become very slow, and the time needed for the Markov process to converge increases. In order to quantify this slow down, it is useful to introduce a time scale for the equilibrium dynamics. Starting from a configuration σ(0) drawn from the stationary distribution µ β , one defines for each variable i the time correlation function:

D i (t) = σ i (0)σ i (t) -σ i 2 (3.8)
the average • is taken with respect to the realization of the Monte Carlo dynamics and to the choice of σ(0). For finite size systems, this time correlation function is expected to be positive and decreasing, with D i (0) = 1 and a vanishing limit when t → ∞. In the large t limit, the Monte-Carlo Markov chain has indeed converged to the stationary distribution µ β , and σ i (0) and σ i (t) are decorrelated.

From the time correlation function D i (t), one can extract the relaxation time scale, that is the smallest integer τ such that D i (t) decreases below a given threshold > 0:

τ i ( ) = min{τ > 0 such that D i (t) ≤ ∀t ≥ τ } (3.9)
In [START_REF] Montanari | On the dynamics of the glass transition on bethe lattices[END_REF], A. Montanari and G. Semerjian show that this time scale can be related to the point-to-set correlation function:

C i (r) = C(i, B(i, r)) = | σ i σ i Bi(r) -σ i 2 | (3.10)
where B(i, r) is the subset of variable nodes j at distance from i smaller than the integer r: d ij ≤ r. Using the notation σ ∼i,r = σ V \B(i,r) , the average • Bi(r) is taken with respect to the conditional distribution P µ β [•|σ ∼i,r ]. The correlation length is the smallest integer l such that C i (r) decreases below :

l i ( ) = min{l ≥ 0 : C i (r) ≤ , ∀r ≥ l} (3.11)
It is shown in [START_REF] Montanari | On the dynamics of the glass transition on bethe lattices[END_REF] that for the Glauber dynamics at finite β on a graphical model (with local interactions) as defined in the paragraph 1.5.3, the following inequality holds:

l i (2 √ 2 ) ≤ Cτ i ( ) (3.12)
with C = 2ke a constant.

In the unclustered regime α < α d , the point-to-set correlation function C i (r) for typical large instances decreases to 0 when r gets large. The length scale l i ( ) is thus finite in this phase. In the clustered regime α > α d , the point-to-set correlation function has instead a non vanishing limit. The length scale l i ( ) is set by convention to be equal to the maximum distance from i to a vertex in G, that scales as O(ln N ) for typical instances of the ER or the regular ensemble. Therefore the relaxation time diverges with N for typical instances. It is also shown in [START_REF] Montanari | On the dynamics of the glass transition on bethe lattices[END_REF] that for the p-spin glass model, in the low temperature phase, that the relaxation time has an exponential scaling in N . One can also expect a similar scaling for the random k-SAT.

Intuitively, the divergence of the relaxation time can be understood from the presence of free-energy barriers. In the clustered phase, the measure µ β decomposes into an exponential number of clusters. These clusters are well separated by free-energy barriers: the total weight of all configurations that are not in a cluster is exponentially small in N . Looking at the equilibrium dynamics, one starts from a configuration σ(0) inside a cluster c. Because of the presence of the exponentially large barriers separating the clusters, the dynamic will stay inside the cluster for a long time (assumed to be exponential), before exploring other clusters at larger time scales.

Cooling scheme

The global balance (stationarity) condition for the Boltzmann distribution at finite β writes:

x e -β(E(x)-E(x )) ω(x → x ) = 1 (3.13)
Since each of the terms are smaller or equal than 1, one has ω(x → x ) ≤ e -β(E(x )-E(x)) . Therefore in the β → ∞ limit, for any x, x such that E(x ) > E(x) one has ω(x → x ) → 0. This means that the paths are forced to be non-increasing in energy, i.e. the Markov Chain is not irreducible. Intuitively, this means that the algorithm can get trapped in a local minima and cannot escape from it since it cannot climb up in energy.

Since the Markov Chain is irreducible for any finite temperature, another idea would then be to work with a fixed finite temperature β which is small enough. One uses a MCMC to sample from it, and since the set of solutions is included in the support of the measure µ β , a strategy would be to wait until one generates a solution. However this strategy might also fail, because the probability under µ β of having a zero-energy configuration is exponentially small in N . Indeed one has

P µ β [E(σ) = 0] = exp[N (lim β →∞ φ N (β ) -φ N (β)],
with φ N (β) = (1/N ) ln Z(β) the free entropy density associated with µ β , which is a decreasing function of β.

To overcome these difficulties, the Simulated Annealing algorithm uses a cooling scheme. It performs n steps of Monte Carlo Markov Chain (MCMC), with temperatures

β 1 < β 2 < • • • < β n .
The final configuration of the step i is used as the initial configuration of step i + 1. At the beginning, small values of β (high temperature) allows the dynamics to escape from local minima and equilibrate on the space of configurations. At the end, the large value of β (small temperature) allows to find the minima (hoping it is a global one) inside the basin where the dynamics has falled.

Focused algorithms

In the Metropolis algorithm 3, the new variable to be flipped is chosen uniformly at random among the set of N variables. One can try to enhance the performances by choosing it according to some heuristics. In focused search algorithms, one chooses the new variable at random among the variables that appears in violated clauses.

RandomWalkSAT algorithm

We present first the RandomWalkSAT algorithm 4 that performs a simple random walk. This algorithm has been proposed by C.H. Papadimitriou in [START_REF] Papadimitriou | On selecting a satisfying truth assignment[END_REF]). It takes as input the CNF formula F and a maximum number T of steps and return either a SAT formula or 'not found'. The algorithm starts with a random initial configuration, for instance independently for each variable one can take x i uniformly in χ = {-1, +1}. In [START_REF] Papadimitriou | On selecting a satisfying truth assignment[END_REF], C.H. Papadimitriou showed that RandomWalkSAT solves w.h.p. any satisfiable instance of the random 2-SAT ensemble, in a number of steps T = O(N 2 ). In [START_REF] Schöning | A probabilistic algorithm for k-sat based on limited local search and restart[END_REF], U. Schöning showed the following result, true for any instance F of the 3-SAT problem. Let R be the number of restarts of the RandomWalkSAT algorithm, each time with an initial condition drawn uniformly at random. Let T = 3N be the number of flips of each restarts. If none of the R successive runs of the RandomWalkSAT on the instance F has provided a solution, then the probability that F is satisfiable is lower than e -(3/4) N R . Therefore after many restarts R (4/3) N , if there are solutions, the probability that RandomWalkSAT has not found one of them is very small.

Algorithm 4 RandomWalkSAT(F ,T )

Initialize σ to a random assignment for T times do if σ is SAT then return σ and stop else choose an unsatisfied clause a uniformly at random choose an index i uniformly at random in ∂a Set σ = σ (i) end if end for return 'not found'

WalkSAT algorithm

The algorithm RandomWalkSAT 4 makes essentially random steps although it focuses on unsatisfied clauses. It is possible instead to choose the new variable according to some specific properties. For instance one can use the energy-cost function E(x) = M a=1 (1 -c a (x ∂a )) defined in equation (1.1), and decide to flip the variable that lead to the largest decrease in energy. An algorithm based on this rule is called a greedy algorithm. However the risk with this greedy choice is to be trapped in local minima (similarly as the case of Metropolis algorithm with zero-temperature). A good strategy is to design a mixed algorithm that performs both random and greedy steps. The idea is that the greedy steps drive the assignments toward the configurations with low energy, while the random steps allow to escape from local minima. The algorithm WalkSAT has been proposed in 1996 by B. Selman, H. Kautz, B. Cohen [START_REF] Selman | Local search strategies for satisfiability testing[END_REF], and is presented in the algorithm 5. As for RandomWalkSAT, it takes as input a CNF formula F , a maximum number T of iterations, and additionally a probability p ∈ [0, 1] of performing a random step (with probability p) or a greedy step (with probability 1 -p). One can then optimize on p in order to increase the performances.

Focused Metropolis algorithm

Let us mention that the Metropolis algorithm 3 can be turned into a focused algorithm 6. The Focused Metropolis Algorithm (FMS) has been introduced in 2005 by S. Seitz, M. Alava and P. Orponen in [START_REF] Seitz | Focused local search for random 3-satisfiability[END_REF].

Algorithm 6 Focused Metropolis Search

S = random assignment of values to the variables while S is not a solution do C = a clause not satisfied by S selected uniformly at random

V = a variable in C selected uniformly at random ∆E = change in number of unsat clauses if V is flipped in S if ∆E ≤ 0 then flip V in S else Draw p ∈ [0, 1] uniformly at random if p < η ∆E then flip V in S end if end if end while
The heuristic for the choice of the new variable to be assigned, and the rule to decide if a flip is performed leads to several other variations around WalkSAT. It is hard to predict in advance which algorithm will be the most efficient. In the next chapters we will describe some of the numerical experiments that compare their performances on the random k-SAT ensemble. The focused algorithms are said to be out-of-equilibrim, because they do not satisfy the global balance condition with respect to the Boltzmann measure µ β , therefore are not guaranteed to converge to it. However, since our goal is to find only one solution to the instance F , it is not necessary to sample correctly the distribution defined on the set of solutions.

Chapter 4

Message Passing Algorithms

Belief propagation

Consider a measure µ defined with a graphical model, as presented in chapter 1 (equation (1.9)). When the associated factor graph is a tree, Belief Propagation (BP) is a method that allows to compute the partition function Z, and the marginal with respect to µ of a variable x i , or any subset of variables. It also provides an efficient way to sample a configuration x from µ. Moreover, all these tasks can be achived in polynomial time in the size N . In the Erdős Rényi hypergraph ensemble G N (k, M ) and in the (k, l)-regular hypergraph ensemble, typical large instances are locally tree-like. In practice, the BP method is also used as an approximation on instances randomly drawn from one of these ensembles.

BP messages

For each edge (i, a) ∈ E of the factor graph, define two messages η i→a , η a→i . These messages are marginal distributions on a modified factor graph in which some nodes have been removed. More precisely, η i→a is the marginal of the variable x i when the function node a has been removed, and η a→i is the marginal of the variable x i when all the function nodes in ∂i \ a have been removed. Using the Markov Property, it can be shown that on a tree, the set of messages {η i→a , η a→i } (i,a)∈E satisfies the following set of local equations: where the z i,a , z i,a are constants that ensure the normalization of the messages. These equations are called the Belief Propagation equations. Note that the message η i→a going from i → a depends only on the messages { η b→i } b∈∂i\a coming to the variable node i, as illustrated in figure 4.1. Similarly, the message η a→i going from a → i depends only on the messages {η j→a } j∈∂a\i coming to the variable node i. We will sometimes use the shorthand notation:

η i→a (x i ) = 1 z i,a b∈∂i\a η a→i (x i ) (4.1) η a→i (x i ) = 1 z i,a x ∂a\i ω a (x ∂a ) j∈∂a\i η j→a (x j ) (4.2) a i η a→i η i→a
η i→a = f ({ η b→i } b∈∂i\a ); η a→i = g({η j→a } j∈∂a\i ) (4.3)
to denote the BP equations. The BP equations have been introduced by J. Pearl in 1988 ( [START_REF] Pearl | Probabilistic reasoning in intelligent systems: networks of plausible inference[END_REF]), in the context of probabilistic inference, and by R.G. Gallager in 1962 ( [START_REF] Robert | Low-density parity check codes[END_REF]) for decoding low density parity check codes. See also [START_REF] Kschischang | Factor graphs and the sum-product algorithm[END_REF][START_REF] Yedidia | Understanding belief propagation and its generalizations[END_REF] for a presentation of the BP algorithm.

BP iterations

On a finite tree, the BP equations (4.3) admit a unique solution. One can solve them using a recursive procedure. Let {η

(t) i→a , η (t)
a→i } (i,a)∈E be the set of messages guessed at step t. One then considers the BP equations as the fixedpoint equations of the following recursive equations:

η (t+1) i→a = f ({ η (t) b→i } b∈∂i\a ); η (t) a→i = g({η (t) j→a } j∈∂a\i ) (4.4)
For the initial condition, a simple choice is to take uniform messages: η (0) i→a (x i ) = 1/|χ|. One can also define a probability measure P on the space of distribution over χ, and extract the initial messages i.i.d from P . The algorithm 7 (see [START_REF] Mézard | Physics, Information, Computation[END_REF]) describe the iterative procedure to find a solution to the BP equations (4.3). It takes as input a factor graph G, the set of function nodes {ω a } a∈C defined on G, a number t max maximum of iterations, and a precision threshold . It outputs either a set of messages, or state that is has not converged. On a tree of diameter t max the algorithm 7 is guaranteed to find the set of messages solution of (4.3), independently of the choice of the initialization and of the updating scheme (4.4). In this algorithm, we have not specified the ordering of the edges (i, a) for the update of the messages. When BP is used as a heuristic on factor graphs with loops (without guarantee of convergence), one usually reshuffles the ordering, taking a random permutation of the edges before each update.

Marginals

Once we have the solution of the BP equations, the marginal µ i (x i ) of a variable x i can be constructed from the messages { η a→i } a∈∂i . Using the tree structure of the factor graph and the Markov property one can write:

µ i (x i ) ∝ a∈∂i    σ σ ∂b\i ω b (σ ∂b ) j∈∂b\i η j→b (σ j )    (4.5)
using the BP equation (4.2) one gets:

µ i (x i ) = 1 z i a∈∂i η a→i (x i ) (4.6)
where z i is a normalization constant. One can write similarly the marginal of a subset of variables. Let C R be a subset of function nodes, and V R be the subset of variable nodes connected to any of the function nodes in C R , and let R be the induced subgraph. Suppose that R is connected. Finally let ∂R be the subset of function nodes that are not in C R but are connected to a variable node in

V R . Then the marginal µ R of the variables x R in V R is µ R (x R ) = 1 z R a∈C R ω a (x ∂a ) a∈∂R η a→i(a) (x i(a) ) (4.7)
where for each a ∈ ∂R, i(a) is the unique variable node i ∈ ∂a ∪ V R . For the particular case C R = {a}, using the BP equation (4.1) this expression reads:

µ a (x ∂a ) = 1 z a ω a (x ∂a ) i∈∂a η i→a (x i ) (4.8)

Conditional laws

Let x ∼i be the configuration of the variables in V \i. The conditional distribution with respect to µ of x ∼i given that x i = z takes the form:

P µ [x ∼i |x i = z] ∝ a∈C ω a (x ∂a )I[x i = z] (4.9)
Therefore the factor graph associated with this conditional measure is the initial factor graph, on which one has added an extra function node I[x i = z] of degree 1, attached to the variable node i. One can run the BP algorithm 7 on this modified factor graph to obtain the marginal law P µ [x j |x i = z], for a given variable node j = i. Consider the particular case where one wants to compute the marginal P µ [{x ∂a\i } a∈∂i |x i ]. On a tree factor graph, when the variable node i is removed, the graph splits into |∂i| disconnected subtrees {T a→i } a∈∂i , where T a→i is the subtree whose root is a. Since the value of x i is fixed, the subsets of variables {x Ta→i } a∈∂i become independent of each other:

P µ [{x ∂a\i } a∈∂i |x i = z] ∝ a∈∂i P µ [x ∂a\i |x i = z]
(4.10) and P µ [x ∂a\i |x i = z] can be expressed in terms of the BP equation written for P µ [x Ta→i ]:

P µ [x ∂a\i |x i = z] ∝ ω a (x ∂a ) j∈∂a\i η j→a (x j ) (4.11)

Bethe free entropy and Entropy

When the factor graph is a tree, one can express the free entropy density Φ = (1/N ) ln Z from the set of messages that is solution of the BP equation (4.3):

Φ Bethe = 1 N a∈C ln Z a ({η i→a } i∈∂a ) + 1 N i∈V ln Z i ({ η a→i } a∈∂i ) (4.12) - 1 N (i,a)∈E ln Z i,a (η i→a , η a→i ) (4.13)
where

Z a ({η i→a } i∈∂a ) = σ ∂a ω a (σ ∂a ) i∈∂a η i→a (σ i ) (4.14) Z i ({ η a→i } a∈∂i ) = σi a∈∂i η a→i (σ i ) (4.15) Z (i,a) (η i→a , η a→i ) = σi η i→a (σ i ) η a→i (σ i ) (4.16)
this expression is called the Bethe free entropy density. It has been introduced by H. A. Bethe in 1935 in [START_REF] Bethe | Statistical physics of superlattices[END_REF] for the ferromagnetic Ising model. Note that one can alternatively consider the Bethe free entropy density as a functional on the set of messages Φ Bethe [{η i→a , η a→i } (i,a)∈E ]. It can be shown (see [START_REF] Yedidia | Constructing free energy approximations and generalized belief propagation algorithms[END_REF]) that the stationary points of the Bethe free entropy are in one-to-one correspondence with the solutions of the BP equation.

From the expression of the free entropy, one can derive the expression of the entropy in terms of the set of messages. Define the Shannon entropy associated to the measure µ:

S[µ] = - x µ(x) ln µ(x) (4.17)
The following general relation holds between S[µ] and the free entropy density Φ:

S[µ] = N Φ + U (4.18)
where U is the internal energy:

U = - x µ(
S[µ] = N Φ Bethe - a∈C x ∂a ω a (x ∂a ) ln(ω a (x ∂a )) i∈∂a η i→a (x i ) x ∂a ω a (x ∂a ) i∈∂a η i→a (x i ) (4.21)

Hard-fields

One says that a BP message η i→a is a hard field when it forces the variable x i to take a particular value z ∈ χ:

η i→a (x i ) = η hard z (x i ), with η hard z (x i ) = I[x i = z]. A similar definition holds for the message η a→i : η a→i (x i ) = η hard z (x i ) = I[x i = z].
When the measure µ is supported on the set of solutions of a CSP instance F (assuming that F is satisfiable), and for tree factor graphs, the presence of hard fields in the set of BP messages solving the BP equations (4.3) indicates that some variables should take one particular value in order to construct a SAT assignment (i.e. in all the solutions to F , these variables take the same value). Indeed from the expression of the marginal µ i (equation (4.6)), we see that whenever there is a hard fields η a→i :

η a→i = η hard z , then we get µ i (x i ) = I[x i = z].

Warning Propagation

One defines the Warning Propagation (WP) messages to be the projection of the BP messages on their hard part:

ν i→a (x i ) = I[η i→a (x i ) > 0]; ν a→i (x i ) = I[ η a→i (x i ) > 0] (4.22)
these messages have the following interpretation. When ν i→a (z) = 1, then according to the set of constraints in ∂i \ a, the variable x i is allowed to take the value z. When ν i→a (z) = 0 instead, the set of constraints in ∂i \ a, forbid the variable x i to take the value z. Similarly, when ν a→i (z) = 1, then the constraint a allows the variable x i to take the value z, and forbid it when ν a→i (z) = 0. The Warning Propagation equations are the projection of the BP equations (4.3) on the WP messages. We will use the shorthand notation:

ν i→a = f WP ({ ν b→i } b∈∂i\a ); ν a→i = g WP ({ν j→a } j∈∂a\i ) (4.23)
The first BP equation (4.1) has the same form for all CSP. Projected on the WP messages this gives:

ν i→a (σ i ) = 0 if ∃a ∈ ∂i \ a : ν a→i (σ i ) = 0 1 otherwise (4.24)
The second BP equation depends on the choice of the function nodes {ω a } a∈C . Take for instance the k-SAT clauses ω a (x ∂a ) = c a (x ∂a ) = 1-i∈∂a I[σ i = -J a i ]. Then the second WP equation states:

ν a→i (-J a i ) = 0 if ∀j ∈ ∂a \ i : ν a→j (J a j ) = 0 1 otherwise (4.25) ν a→i (J a i ) = 1 (4.26)
Another example is the bicoloring on k-hypergraphs (NAE-SAT) for which ω a (σ ∂a ) = I[σ i(1) , . . . , σ i(|∂a|) n.a.e], where 'n.a.e' stands for 'not-all-equal'. In that case the second WP equation states:

ν a→i (σ i ) = 0 if ∀i ∈ ∂a \ i : ν a→i (-σ i ) = 0 1 otherwise (4.27)
In order to solve the WP equations, one can run an algorithm similar to the algorithm 7, replacing the BP iterations (4.4) by the WP iterations obtained by introducing an uptdate scheme on the WP equations (4.23):

ν (t+1) i→a = f WP ({ ν (t) b→i } b∈∂i\a ); ν (t) a→i = g WP ({ν (t) j→a } j∈∂a\i ) (4.28)
When the WP messages are initialized with the permissive value ν i→a (x i ) = 1 on all the edges, these iterations are guaranteed to converge. On a tree, the set of WP messages obtained is the unique solution to the WP equations. From the WP solution, one can construct the equivalent of the marginal µ i , but for WP messages:

ν i (x i ) = a∈∂i ν a→i (x i ) (4.29)
One calls ν i a local field. When ν i (z) = 1, this means that the value x i = z is allowed by the set of messages { η a→i } a∈∂i , while it is forbidden when ν i (z) = 0. For a measure whose support is the set of solution of a given CSP instance F , the set of local fields provides useful information about the satisfiability of this instance. One says that a contradiction occurs for the variable x i when ν i (x) = 0 for all x ∈ χ. If such a contradiction occurs, it means that for any value that the variable x i can take, there will be clauses that will be violated. Therefore the instance F has no solutions. If no contradictions occur, then the set of variables for which there is a unique value x i ∈ χ such that ν i (x i ) = 1 corresponds to the set of variables that are forced to take this specific value in all the solutions of the instance F (if there is solutions). For the SAT problem, it is shown in [START_REF] Montanari | Solving constraint satisfaction problems through belief propagation-guided decimation[END_REF] that the WP equations correspond to the UCP rule. In UCP, a contradiction corresponds to the appearance of a zero-length clause, indicating that two unitclauses are contradicting each other. A contradiction in UCP occurs if and only if there is a contradiction in the WP messages. If no contradiction has occured, then the partial assignment found by the UCP algorithm corresponds to the set of variables with ν i (x i ) = 1 for some x i ∈ χ.

Note that if no contradiction has occured, it does not mean that the instance F admits a solution (otherwise UCP/WP would solve the decision problem on k ≥ 3-SAT and since it works in polynomial time one would have incidentally P=NP). Moreover, when F admits solutions, it can happen that a variable that is forced to take a given value in all solutions is not unveiled by the WP iterations. This may cause the failure of the message-passing algorithms that we will present below.

In the case where the factor graph is a tree, however, it can be shown that if no contradiction has occured, then the problem admits solutions (see [START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF] Theorem 1 for the SAT problem). In that case, the information contained in the local fields can be used to construct a solution.

Survey Propagation

When a factor graph contains loops, the BP equation may admit more than one solution. In the RSB framework described in chapter 2, above the clustering threshold, for large typical instances, the measure µ is assumed to split into an exponential number of states {µ c }, each of these states being the restriction of the measure µ to one cluster c:

µ(x) = c p(c)µ c (x) (4.30)
where p(c) is the distribution over the set of clusters. In the 1RSB cavity method, one assumes that each state µ c is associated with one solution of the BP equation, to be denoted {η c i→a , η c a→i } (i,a)∈E . The 1RSB cavity method then introduces a set of probability distributions {P i→a , P a→i } (i,a)∈E . The probability distribution P i→a is the probability to observe the BP message η c i→a when the cluster c is chosen randomly according to the distribution p(c). A similar definition holds for P a→i . These probability distributions satisfy a set of equations, called the 1RSB cavity equations, that we denote using the shorthand:

P i→a (η i→a ) = F ({ P b→i } b∈∂i\a ), P a→i ( η a→i ) = G({P j→a } j∈∂a\i ). (4.31)
We will postpone the derivation of these equations to chapter 7 dedicated to the cavity method (see section 7.6). Note that the {P i→a , P a→i } (i,a)∈E are probability distributions over the space of messages. For instance, a message η(σ) over a spin variable can be parametrized by its mean value h: η(σ) = (1 + hσ)/2. Therefore the space of this message is already [0, 1]. It is more tractable to work with the projection of the distributions {P i→a , P a→i } (i,a)∈E on their contribution to hard fields. One uses the following decomposition:

P i→a (η i→a ) = x∈χ h i→a,x δ[η i→a -η hard x ] + Q i→a (η i→a ) (4.32) P a→i ( η a→i ) = x∈χ h a→i,x δ[ η a→i -η hard x ] + Q a→i ( η a→i ) (4.33)
When the Parisi parameter X is equal to the special value 0 or 1, one can obtain closed equations on the weight of hard fields {h i→a,x , h a→i,x } (i,a)∈E,x∈χ . For the special case X = 0 the equations obtained are called (SP) Survey Propagation equations:

h i→a,x = F SP ({ h b→i,x } b∈∂i\a,x ∈χ ), h a→i,x = G SP ({h j→a,x } j∈∂a\i,x ∈χ ) (4.34)
We recall that setting the Parisi parameter X = 0 corresponds to putting equal probability on all clusters, regardless of their size, while X = 1 weights them proportional to their size, i.e., corresponding to uniformly random solutions. The SP message h i→a,x can be interpreted as the probability (over the choice of cluster) that the set of clauses in ∂i \ a send a hard field η i→a = η hard x forcing x i to take the value x. Similarly, the SP message h a→i can be interpreted as the probability that the clause a send a hard field η a→i = η hard x . Note that on tree problems, since there is only one solution to the BP equations, the probability distributions {P i→a , P a→i } (i,a)∈E are Dirac delta's on the unique solution to the BP equations. The SP messages are either equal to 0 or to 1, and thus reduce to the WP messages, with the following correspondence:

• If there is a x ∈ χ such that h i→a,x = 1 then it translates to ν i→a (x) = 1 and ν i→a (x ) = 0 ∀x = x

• if for all values x ∈ χ one has h i→a,x = 0, then ν i→a (x) = 1 for all x.

From the SP messages, one can construct a local field ν SP i (z), that can be interpreted as the probability that the variable i receives a warning that forces it to take the value x i = z. Let P i (µ i ) be the probability distribution of the marginal µ i in the 1RSB framework. One can express P i as a function of the { P a→i } a∈∂i :

P i (µ i ) = a∈∂i d P a→i ( η a→i )δ[µ i -f marg ({ η a→i })] (4.35)
where

µ i = f marg ({ η a→i }
) is a shorthand for the equation (4.6). We then project on the decomposition

P i (µ i ) = x∈χ ν SP i (x)δ[µ i -I[• -x]] + Q i (µ i ) (4.36)
to get an expression for ν SP i :

1 -ν SP i (z) = a∈∂i (1 -h a→i,z ) (4.37)

Algorithms

Sampling procedure

The resolution of the BP equations on a tree allows to compute the marginals and conditional probability laws of µ. From this knowledge, one can sample from µ using a sequential assignment procedure. This procedure assign a value to each variable recursively. Suppose that at time t, one has assigned the values of a subset D of variables according to their marginal law P µ [x D ], leading to a partial assignment x D = z D . One defines the conditional probability P µ [x V \D |z D ] on the subset of variables that are still not assigned. Then one chooses a new variable i ∈ V \ D, computes its marginal from the conditional measure, and draws its value from this marginal. If at each step of the algorithm the marginal thus computed is exact, then this procedure leads to an exact sampling from µ. However, computing marginals of a measure µ over a set in N dimensions is computationnaly intractable. One can use instead a heuristic to estimate the marginals. The BP-guided decimation is a sampling procedure that uses the BP estimation of the marginals.

BP-guided decimation

The BP-guided decimation is described in the algorithm 8. It takes as input the graphical model that describe the measure µ (the factor graph G, and the set of functions {ω a } a∈C ). It outputs either a configuration z (that is a solution to the associated instance F ), or returns "not-found".

Algorithm 8 BP-guided sampling(G, {ω a } a∈C ) initialize D = ∅ for t = 1 . . . , N do Run BP for P µ [x|z D ]
if BP does not converge then return "not found" and exit else if there is a contradiction then return "not found" and exit

else choose i ∈ V \ D compute the BP marginal P µ (x i |z D ) draw x i from P µ (x i |z D ) Set x i = z i and D → D ∪ {i} Add a factor I[x i = z i ] to the graphical model end if end for return z D
When the CSP instance F does not admit a solution, the measure whose support is the set of solution is ill-defined, but one can still define the BP messages, and run the BP-decimation algorithm 8. The algorithm will either find a contradiction, concluding that the instance is not satisfiable, or the BP iterations will not converge. As already mentioned, it can happen that a variable x i which is forced to take one particular value (either in the initial instance F , or in the simplified formula obtained after t steps) is not detected by the BP iterations. In that case the marginal µ i can have non-zero weights on other values, leading to a wrong assignment of the variable x i . The resulting simplified instance therefore does not admit solutions, and the algorithm fails. It is not specified in the algorithm 8 how the new variable to be assigned is chosen. One possibility is to take the variable which is the most biased toward one value, i.e. the variable i which has the largest argmax z {µ i (z)}.

From this algorithm, it is possible to construct a similar version in which the BP messages are replaced by the WP messages. At each step, the WP iterations are run in order to determine the local fields of the not-yet assigned variables. Each variable that is forced to take one value is assigned to this value. The formula is simplified, and if there is no contradiction, the WP iterations are run for the next step. If there is no local field forcing a variable, then one picks at random a variable and fix it to an arbitrary value. This algorithm is equivalent to the UCP algorithm presented in chapter 1, with an extra free step (or to the DPLL without backtracking). Note that since in the WP messages one has discarded some information about the tendency of a variable to take a particular value, the BP-guided decimation should be in principle more efficient than the WP-guided decimation, because the BP-guided decimation algorithm might be closer to choosing a uniformly random solution, by giving variables values with the right marginal probability.

SP-guided decimation

The WP and BP-guided decimation algorithms are guaranteed to work on satisfiable instance whose underlying graph is a tree. When the factor graph has loops, they can be used as heuristics, but without a guarantee of convergence. The SP-guided decimation is a similar procedure, that uses the SP messages {h i→a,x , h a→i,x } (i,a)∈E,x∈χ defined in (4.32). It is also a heuristic, there is no guarantee of convergence, but it aims at taking into account the presence of loops in the factor graph. The SP-guided decimation is described in the algorithm 9. It has been introduced in 2002 by M. Mézard and R. Zecchina, in [START_REF] Mézard | Random k-satisfiability problem: From an analytic solution to an efficient algorithm[END_REF]. An algorithmic description is given in [START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF]. It takes as input the graphical model (G, {ω a } a∈C ) and a threshold > 0. The new variable to be assigned is the one with a largest bias π i toward a given value, with π i = max x∈χ {ν SP i (x)}. The value is assigned according to π i :

x i = argmax x {ν SP i }.
If at some point all the variables have a bias close to 0, it means that there is no variables that is forced to take a particular value. In this case the problem is simpler, and the algorithm calls WalkSAT to solve the instance.

Algorithm 9 SP-guided decimation(F ,{ω a } a∈C , ) Set D = ∅ for t = 1, . . . , N do Run SP if SP does not converge then return "not found" else For each i ∈ V \ D, compute the bias π i = max x∈χ {ν SP i (x)} Let j ∈ V \ D be the variable with the largest π i if π i ≤ then
Call WalkSAT else Fix x j according to π j Simplify the formula according to this choice end if end if end for return the assignment found We will see in the next chapter that this algorithm outperforms the BP-guided decimation and the local search algorithms defined in the previous chapter, on random k-SAT instances, especially in the clustered phase. There is several possibilities to improve this algorithm. One can introduce backtracking: if the algorithm fail at some point, one releases the value of some variables that were fixed. The performances of SP-guided decimation with backtracking have been studied in [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF]. Moreover, the 1RSB cavity method provides a way to study the cluster decomposition. In particular, the complexity Σ(X ) at X = 0 is an estimation of the logarithm of the number of clusters, for typical large instances. One can use this quantity to choose the next variable to be assigned. If a choice lead to a huge decrease in Σ(X = 0), this would mean that a lot of clusters have been killed by this choice because they are not compatible with it. One can for instance choose the variable that maximizes the number of clusters still present at the next step.

Chapter 5

Performance of the algorithms on random Constraint Satisfaction Problems

In this chapter we compare the performances of the algorithms that search for solutions on the random CSP ensembles. We will present some of the numerical experiments and theoretical results that have been obtained in the literature. As we have seen, these algorithms can be divided in two broad families. In the first one, the algorithms perform a biased random walk in the space of configurations towards the solutions. In the second one, the algorithms assign sequentially the value of the variables: at a given step, only a subset of variables has been assigned, the others are free. The algorithm then chooses a new variable and its value according to some heuristic. These heuristics can be based on simple properties of the not-yet assigned variables, for instance the number of clauses that will be satisfied by this choice. Or they can use the information provided by message passing algorithms. In the first chapter, we have seen that the performances of DPLL on the random k-SAT ensemble depends on the density of constraints α. Close to the satisfiability transition α sat (k), the typical instances are hard to solve, and the running time of DPLL grows exponentially in N . We would like to study the performances of the other algorithms, and determine the range of α for wich their running time grows polynomially in N . In particular, it would be interesting to characterize the putative algorithmic barrier α alg (k) ≤ α sat (k) above which no algorithm is able to find a solution in polynomial time, for a typical random instance (assuming P =NP). If it is possible to design an algorithm able to run polynomially in the whole satisfiable phase, then α alg (k) = α sat (k), but it is also possible that some intrinsic properties of the set of solutions implies a strict inequality. Up to now, the precise location of α alg (k) is still unknown, especially for large values of k. For small values of the connectivity k, numerical experiments can be performed in reasonable time. They suggest that some local search algorithms and message-passing inspired algorithms can work up to densities very close to the satisfiability threshold, thus setting lower bounds on α alg (k) almost coinciding with the upper bound α sat (k). When k increases, the numerical experiments become heavier, but simple enough algorithms can be studied analytically for all k. In this regime there is a wide range of α for which instances admit solutions w.h.p., yet no known algorithm is able to find one in polynomial time. The cavity method applied to random k-SAT provides a very detailed description of the structure of the set of solutions. It is an interesting open problem to determine whether some of the phase transitions undergone by the set of solutions affect the perfomances of the algorithms, and therefore could be related to the algorithmic barrier α alg (k).

Small connectivity k

Two numerical experiments on local search algorithms at small k

In [START_REF] Selman | A new method for solving hard satisfiability problems[END_REF], B. Selman, H. Levesque and D. Mitchell compare the performances of the complete algorithm DPLL (with the unit clause branching rule) with a variation of the WalkSAT algorithm, called GSAT, in which there is only greedy steps (setting p = 0 in the WalkSAT algorithm). They run these two algorithms on random formulas drawn from the random 3-SAT ensemble in the hard regime α 4.3, with increasing number of variables N . They show that the GSAT algorithm is faster and therefore able to run on larger instances than the DPLL algorithm with the unit clause branching rule. This study provides evidence that the local search algorithm GSAT performs better than the complete algorithm DPLL on random 3-SAT in the hard regime. In [START_REF] Selman | Noise strategies for improving local search[END_REF], B. Selman, H.A. Kautz and B. Cohen compare the performances of four local search algorithms: simulated annealing, GSAT, WalkSAT with a non-zero probability p of random steps, and an algorithm called random noise. This algorithm is a variation of the WalkSAT algorithm, in which the random step is not focused: the variable to be flipped is chosen uniformly at random among the N variables. While GSAT performs only greedy steps, the three other algorithms can escape from local minima by making uphill moves. They show that GSAT performs less well than algorithms with escape strategies, and that WalkSAT outperforms the other algorithms.

Overview of the algorithmic performances for small values of k

In order to compare the performances of the algorithms, it is convenient to introduce the algorithmic threshold for each algorithm α ALG a (k) (using the notations of [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF]) above which the probability of finding a solution in polynomial time vanishes. More precisely, the performance of an algorithm depends on the density of constraints α, on the size N of the problem, and on the cutoff T on the time we let it run. Let p ALG succ (α, T, N ) be the probability (with respect to the random instance generation and the stochasticity of the algorithm) that the algorithm ALG has found a solution after T steps on a random instance of size N with density α. Following the definition given in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF], section V.A, the algorithmic threshold α ALG a (k) is the smallest α such that: lim

N →∞ p ALG succ (α, T = N c , N ) = 0 (5.1)
for all c > 0. In practice the large N limit is inaccessible, and one has to extrapolate from the results obtained at finite N to estimate the algorithmic threshold. Note however that as long as N is finite, any running time T can be considered polynomial if the parameter c is arbitrary. Moreover as N increases the only interesting regime is the linear regime. One then defines the algorithmic threshold as the smallest value of α such that lim N →∞ p ALG succ (α, T = κN, N ), with κ a constant that can be sent to infinity after the large N limit.

For the RandomWalkSAT algorithm on the random 3-SAT ensemble, the algorithmic threshold is located at 2.7. This value has been determined both numerically and analytically in [START_REF] Semerjian | Relaxation and metastability in a local search procedure for the random satisfiability problem[END_REF] by R. Monasson and G. Semerjian and in [START_REF] Barthel | Solving satisfiability problems by fluctuations: The dynamics of stochastic local search algorithms[END_REF] by W. Barthel, A. K. Hartmann and M. Weigt.

In [START_REF] Seitz | Focused local search for random 3-satisfiability[END_REF] S. Seitz, M. Alava and P. Orponen show that WalkSAT with an optimized parameter p has a linear regime up to α ≈ 4.20 on random 3-SAT. They also found that the FMS algorithm (see algorithm 6 presented in chapter 3) works in linear time up to α ≈ 4.20. We recall that the clustering threshold for the uniform measure is α d (3) = 3.86. Therefore it does not seems that the clustering threshold is relevant to understand the limitations of the local search algorithms for k = 3. Moreover, the result obtained for FMS and WalkSAT is very close to the satisfiability threshold α sat (3) = 4.267. (Note that a similar observation can be made for the 3-coloring on graph: in [START_REF] Achlioptas | Almost all graphs with average degree 4 are 3-colorable[END_REF], D. Achlioptas and C. Moore show that a simple algorithm works up to 4.03, while the clustering threshold is at 4.00.) Above k = 3, the set of solution undergoes a qualitative change: the condensation threshold separates from the clustering threshold, α d < α c . It is not clear however if this affects the behavior of the local search algorithms: in [START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF], the authors show that α F M S a (4) = 9.6 for FMS, which is above the clustering and the condensation threshold α d (4) = 9.38, α c (4) = 9.547. In [START_REF] Orponen | An efficient local search method for random 3satisfiability[END_REF], S. S. Seitz and P. Orponen study experimentally the focused record-to-record travel (FRRT) algorithm. It is a local search algorithm that flips at each step a variable involved in an unsatisfied clause. The flip is accepted if the increase in the number of unsatisfied clauses is smaller than a threshold integer d. They show that this algorithm works in lineal time up to α 4.23 on the random 3-SAT ensemble.

These results have to be compared with the performances of message-passing algorithms. For the BP-guided decimation (BPD) one gets α BP D a [START_REF] Moore | The Nature of Computation[END_REF] 3.8, and α BP D a (4) = 9.05 ( [START_REF] Ricci | On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms[END_REF]), while SP-guided decimation (SPD) provides better results : α SP D a (3) 4.252 ([20], [START_REF] Parisi | Some remarks on the survey decimation algorithm for ksatisfiability[END_REF]), and α SP D a (4) 9.73 ([20]). Although very close to the satisfiability threshold, the algorithmic threshold for SPD is still smaller. As expected, SPD performs better than BPD. This can be explained from the fact that the SP messages take into account the clustering phenomenon. Indeed above the condensation threshold, even if the BP iterations converge to a fixed point, the BP marginals thus obtained are likely to provide a bad estimate of the exact marginals, thus leading to wrong assignments in the decimation. The running time of BP and SP-guided decimation is determined by the number of iterations t max needed for the Message-Passing iterations to converge. Each Message-Passing iteration requires O(N ) operations, therefore the convergence is achieved in O(t max N ) operations. For the whole decimation algorithm, one obtains a complexity O(N 2 t max ). One can estimate the order of magnitude of t max as the typical diameter of the graphs: t max = O(ln N ). In practice it is observed ( [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF], [START_REF] Ricci | On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms[END_REF], [START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF]) that the number of iterations t max is growing slowly in N . In [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF] All these results can be compared with the performances of the algorithms using simpler heuristics. For some of them, the rate of convergence can be proven rigorously. The Generalized Unit Clause (GUC) algorithm has been proven to work up to α GU C a (3) = 3.003 on random 3-SAT, and to α GU C a (4) = 5.54 on random 4-SAT (see [START_REF] Frieze | Analysis of two simple heuristics on a random instance of k-sat[END_REF]). GUC is a sequential algorithm: at each step, the algorithm classifies the clauses of the simplified formula according to their length. Let 1 ≤ q ≤ k be the size of the smallest clauses. The algorithm chooses uniformly at random a clause of length q, then chooses uniformly at random a literal in this clause and assign the value of the underlying variable so that the clause is satisfied. The behavior of this algorithm can be analyzed for any k, and in particular on can study its performances in the large k limit (see part 5.2.2). In [START_REF] Kaporis | Selecting complementary pairs of literals[END_REF] and [START_REF] Hajiaghayi | The Satisfiability Threshold of Random 3-SAT Is at Least 3[END_REF], the authors independently proved that this bound can be rigorously improved for random 3-SAT, using an algorithm that sets variables with high degree and a large imbalance between positive and negative appearances. They prove that this algorithm succeeds up to densities of constraints α ≤ 3.52. From these comparisons one can conclude that using the more refined heuristics presented above helps to increase the algorithmic performances. It is also interesting to compare these results with the performances of DPLL, i.e. when one allows backtracking. As already mentioned in the section 1.4.3, the algorithmic threshold for DPLL is estimated at 3.003 for the random 3-SAT ensemble (see [START_REF] Cocco | Trajectories in phase diagrams, growth processes, and computational complexity: How search algorithms solve the 3-satisfiability problem[END_REF]), thus it seems that allowing backtracking does not improve on the algorithmic threshold in this case.

Frozen variables and whitening dynamics

It is interesting to study the properties of the solutions found by the local search algorithms. In particular, one would like to know whether these solutions contain frozen variables. Recall that frozen variables are variables that take the same value in all the solutions of a cluster. One can use the correspondence between clusters and solutions of the BP equations, and say that a variable is frozen in the cluster c if in the corresponding BP fixed point the variable receives a hard message forcing it to a particular value. This information is contained in the solutions of the WP equations, which are the projection of the BP equations on hard messages. Therefore to determine the set of frozen variables in a solution σ, one can run the WP iterations, starting from the initial condition ν

(t=0) i→a (τ i ) = σ i (5.2)
It can be shown that this dynamics is monotonic in time, and never produces contradictions, so that the WP iterations converge toward a unique fixed point in a finite number of iterations. A variable i is then declared frozen if it receives a hard message ν a→i from at least one clause a that forces it to take one particular value.

For the k-SAT problem and for the NAE-k-SAT problem, the WP iterations with σ as initial condition is equivalent to the whitening dynamics. Starting from the initial condition σ (t=0) = σ (with σ a solution of the instance), the variables are iteratively whitened (set to σ i = 0), if they are not constrained. A variable i such that σ (t) i = 0 is declared white at time t + 1 if the partial configuration obtained by flipping it and keeping the other non-white variables j (such that σ (t) j = 0) unchanged can be extended to a solution by an assignment of the white variables. In other words, one has to check if the variable belongs only to clauses which either involve a white variable, or are satisfied by another variable. The procedures stops when either all variables are white, or a fixed point is reached. The variables that are non-white at the end of the run are then declared frozen.

In [START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF], the authors checked that the solutions found by the FMS algorithm were completely white, i.e. there were no frozen variables in the solutions they found. It is also the case for the solutions of the BSP algorithm [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF], and more generally for both local search algorithm and message-passing guided decimation algorithms. This results provide the conjecture that the algorithmic barrier α alg (k) is upper bounded by the freezing threshold α f (k), above which all solution contain a finite fraction of frozen variables. The rigidity threshold is defined as the value of α above which the typical solutions (with respect to the measure µ defined to describe the set of solutions) contains a finite fraction of frozen variables. In the region α r ≤ α < α f the algorithms find untypical unfrozen solutions ( [START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF][START_REF] Elitza | A new look at survey propagation and its generalizations[END_REF][START_REF] Braunstein | Survey propagation as local equilibrium equations[END_REF][START_REF] Dall'asta | Entropy landscape and non-gibbs solutions in constraint satisfaction problems[END_REF]).

Analytical study of the BP-guided decimation

In [START_REF] Montanari | Solving constraint satisfaction problems through belief propagation-guided decimation[END_REF] and [START_REF] Ricci | On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms[END_REF], a theoretical framework is developed to study the running time of the BP-guided decimation algorithm. Using the cavity method, the authors analyse the behavior of an idealized algorithm where marginal are computed exactly. To do so, they introduce an ensemble of CSP parametrized by α, θ, that allows to study the statistics of the simplified formulas obtained after t = θN decimation steps. An instance of this ensemble is generated as follows:

1. draw a satisfiable instance with parameter α < α sat (k) 2. draw a uniform solution τ of this instance 3. construct a set D by including in D each variable independently with probability θ 4. consider the residual formula on the variables in V \ D, obtained from the initial formula by imposing

σ i = τ i for i ∈ D.
The initial ensemble is recovered with θ = 0. Note that the formula thus obtained cannot be seen as a formula drawn from the initial random ensemble, since there are non-trivial correlations between variables in the measure µ D induced by the choice of τ . In order to study the statistical properties of the formulas obtained from the ensemble defined above, they compute the quenched averaged residual entropy:

ω(θ) = lim N →∞ E F E τ E D [ln Z(τ D )] (5.3) with Z(τ D ) = σ a=1 I[σ ∂a satisfies the clause a]I[σ D = τ D ] (5.4)
being the number of solutions compatible with the partial assignment τ D . They also introduce the average fraction of variables which have been explicitly assigned or which can be logically deduced from these assignments

φ(θ) = θ + lim N →∞ E F E τ E D [nb. of directly implied variables] (5.5)
Note that (dφ/dθ) -1 is the average number of newly implied variables appearing during one step. If this number diverges (signaled by a discontinuity in φ(θ)), it might be the sign of potential errors that the decimation procedure will make if the marginals are not computed exactly. Both ω(θ) and φ(θ) are computed with the RS cavity method. They provide an estimation of the free entropy and fraction of variables at step t = θN , in an idealized algorithm in which the marginals are computed exactly. For k = 4, they obtain the following results. The quenched residual free entropy is a decreasing function of θ. Below a threshold value α * = 9.05 the function is smoothly decreasing, while above this value the function ω(θ) undergoes a discontinuity in its derivative at θ c (α), corresponding to a condensation transition. Above θ c (α) the total free entropy ω(θ) is dominated by a sub-exponential number of clusters, therefore it might be harder to find solutions. For φ(θ), below the threshold value α * = 8.05, φ(θ) is a smoothly increasing function of θ. Above α * , φ(θ) is discontinuous at some value of θ. This discontinuity is the sign of an avalanche of directly implied variables during the decimation procedure. These estimations are compared with numerical experiments. They compute the empirical probability of success for BPD, and see that it vanishes when α is close to α * = 9.05 (k = 4). The performances of the algorithm seems therefore to be related to the phase transition undergone by the residual free entropy ω(θ). While φ(θ) does not seem In [START_REF] Montanari | Solving constraint satisfaction problems through belief propagation-guided decimation[END_REF] A. Montanari, F. Ricci-Tersenghi and G. Semerjian provide the asymptotic expansion of the value α * (k) at which φ starts to be discontinuous:

α spin (k) = e 2 k k (1 + O(k -1 )) (5.6)
This provides an estimated lower bound on the algorithmic threshold for the BP-guided decimation algorithm. In [START_REF] Coja-Oghlan | On belief propagation guided decimation for random k-sat[END_REF], A. Coja-Oghlan proves that BPguided decimation algorithm fails to solve random k-SAT formulas already for

α = O(2 k /k).

Large connectivity k

Asymptotic expansion of the thresholds

In the large k limit the satisfiability threshold for the random k-SAT ensemble scales as:

α sat (k) ∼ 2 k ln 2.
(5.7)

This result has been obtained rigorously in [START_REF] Achlioptas | Rigorous location of phase transitions in hard optimization problems[END_REF] by D. Achlioptas, A. Naor and Y. Peres. Using the first and second moment method, they obtain that the satisfiability threshold is located in the interval (2 k ln 2 -k, 2 k ln 2). In [START_REF] Mertens | Threshold values of random k-sat from the cavity method[END_REF], S. Mertens, M. Mézard and R. Zecchina provide an asymptotic expansion in powers of 2 -k of the satisfiability threshold in the large k limit, using the cavity metod.

In [START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF] this prediction for the asymptotic expansion is proven to be exact for large but finite k. In [START_REF] Achlioptas | On the 2-colorability of random hypergraphs[END_REF], D. Achlioptas and C. Moore prove the scaling of the satisfiability threshold for the bicoloring on k-hypergraphs:

α sat (k) ∼ 2 k-1 ln 2 . (5.8)
For the q-coloring problem on graphs one has (see [START_REF] Achlioptas | The Chromatic Number of Random Regular Graphs[END_REF]):

c sat (q) ∼ 2q ln q (5.9)
where c = 2α is the average degree. The asymptotic expansions of the rigidity and clustering thresholds have also been established for several CSPs, for the uniform measure. It is shown in [START_REF] Achlioptas | Algorithmic barriers from phase transitions[END_REF] that the clustering threshold in the large k, q limit is upper-bounded by:

α d (k) ≤ 2 k (ln k)/k (5.10)
for the k-SAT problem, by:

α d (k) ≤ 2 k-1 (ln k)/k (5.11)
for the bicoloring on k-hypergraphs, and by: c d (q) ≤ q ln q (5.12)

for the q-coloring on graphs. Moreover, the authors show that the rigidity threshold arise at the same scale. It is possible to derive the following subdominant terms for the rigidity threshold, using the naive reconstruction procedure (see [START_REF] Semerjian | On the freezing of variables in random constraint satisfaction problems[END_REF] and [START_REF] Sly | Reconstruction of random colourings[END_REF]). This gives for the bicoloring on k-hypergraphs:

α r (k) = 2 k-1 k (ln k + ln ln k + γ r + o(1)); γ r = 1 (5.13)
and for the q-coloring on graphs (see also [START_REF] Molloy | The freezing threshold for k-colourings of a random graph[END_REF]):

c r (q) = q(ln q + ln ln q + γ r + o( 1)); γ r = 1 (5.14)

In general, the analytic determination of the rigidity threshold is simpler than for the clustering threshold, because it corresponds to the bifurcation of a scalar (instead of functional) fixed-point equation, as we will see in chapter devoted to the cavity method. For these two problems, it is conjectured that the clustering threshold arise at the same scale:

α d (k) = 2 k-1 k (ln k+ln ln k+γ d +o(1)); c d (q) = q(ln q+ln ln q+γ d +o(1)) (5.15)
In [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF], the leading term is proven rigorously for these two problems. For the coloring problem, the inequality 1 -ln 2 ≤ γ d ≤ γ r = 1 has been established rigorously in [START_REF] Sly | Reconstruction of random colourings[END_REF], and the strict inequality γ d < 1 was later obtained in [START_REF] Sly | Reconstruction of colourings without freezing[END_REF]. In [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] we characterize the value of γ d for the two models in terms of a functional equation, that is solved numerically, yielding to the estimate γ d 0.871 for both models. We also provide an analytic lower bound γ d ≥ 1+ln(2( √ 2-1)) 0.812. The freezing transition has been predicted in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF] to arise at

α f (k) ∼ α sat (k)/2 for the NAE-k-SAT problem.
The condensation threshold is also expected to arise close to the satisfiability threshold. In [START_REF] Montanari | Clusters of solutions and replica symmetry breaking in random k-satisfiability[END_REF] A. Montanari, F. Ricci-Tersenghi and G. Semerjian obtain the following asymptotic expansion for the random k-SAT problem, using the uniform measure:

α c (k) = 2 k ln 2 - 3 ln 2 2 + O(2 -k ) (5.16)

Algorithmic performances in the large k limit

Numerical experiments cannot access directly the large k limit, but simple enough algorithms can be studied analytically for all k; sequential assignment algorithms that use simple heuristics to guide their choices can be described in terms of differential equations. In [START_REF] Chao | Probabilistic analysis of a generalization of the unit-clause literal selection heuristics for the k satisfiability problem[END_REF], M-T. Chao and J. Franco study the unit clause algorithm defined in Algorithm 10 and proved that a satisfiable assignment can be found with positive probability when N → ∞ whenever α < α * (k), where

α * (k) = 1 2 k -1 k -2 k-2 2 k k .
(5.17)

Algorithm 10 Unit clause(F )

if there is any unit clause (forced step) then Pick a unit clause uniformly at random and satisfy it. else (free step) Pick a not-yet assigned variable uniformly at random and assign it to 0/1 uniformly at random end if Although we will not explain the proof, it is interesting to mention that it relies on the uniform randomness of the simplified formula over the set V (t) of the not-yet assigned variables obtained at step t: Lemma 4 (uniform randomness [START_REF] Achlioptas | Lower bounds for random 3-SAT via differential equations[END_REF]) For every 0 ≤ t ≤ N , and 1 ≤ s ≤ k, the set S s (t) of s-clauses form a random s-SAT formula drawn from the random s-SAT ensemble with |V (t)| variables and C s (t) clauses. Moreover these s-SAT formulas are independent of each other.

Note that this lemma does not hold for the BP-guided decimation: the decimation procedure has introduced non-trivial correlations between the variables in V (t). The proof then uses the fact that the formula composed of the 2-clauses at each step t is distributed from the random 2-SAT ensemble. Looking at its density, whenever it crosses the value α sat (2) = 1, then the algorithm will fail w.h.p., while it can be proven that if the density of 2-clauses is bounded away from 1, then the algorithm succeeds with positive probability. The evolution of the 2-clause density conditioned on the previous steps can be studied by means of differential equations on the mean value of the conditional s-clause densities.

This method has also been applied in [START_REF] Chvatal | Mick gets some (the odds are on his side) (satisfiability)[END_REF] to improve this result. They study the Shortest Clause algorithm. This algorithm is a variation of the Unit Clause algorithm 10. Instead of looking at unit-clause only, this algorithm also look at clauses of length 2 (2-clauses). At each step, if there are unit-clauses, then the algorithm performs a forced step. If there is no unit-clauses but there are 2clauses, then the algorithm picks one of them at random and satisfy at random a literal in it. Otherwise it performs a free step. [START_REF] Chvatal | Mick gets some (the odds are on his side) (satisfiability)[END_REF] proves that this algorithm finds a solution w.h.p. for α smaller than

1 8 k -1 k -3 k-3 2 k k (5.18)
In [START_REF] Frieze | Analysis of two simple heuristics on a random instance of k-sat[END_REF] this bound is improved by studying the Shortest Clause algorithm with limited backtracking. They show that the algorithm find a solution w.h.p. when α < α * (k), where α * (k) 1.817(2 k /k). In the large k limit, all these algorithms are performant for densities smaller than c2 k /k, with a constant c depending on the algorithm. In [START_REF] Montanari | Solving constraint satisfaction problems through belief propagation-guided decimation[END_REF], a similar scaling is obtained: the BP-guided decimation algorithm is conjectured to fail above the threshold e2 k /k corresponding to the appearance of cascades of logically implied variables during the decimation process, and in [START_REF] Coja-Oghlan | On belief propagation guided decimation for random k-sat[END_REF], A. Coja-Oghlan proves that BP-guided decimation fails to solve random k-SAT formulas already for α = O(2 k /k). This scaling is improved in [START_REF] Coja-Oghlan | A better algorithm for random k-sat[END_REF], where an algorithm is shown to work up to densities of constraints coinciding at leading order with α d (k) ∼ 2 k ln k/k. Up to now, there is no algorithm that has been proven to find solution in polynomial time above this scaling. This leaves a multiplicative gap of order k (neglecting the sub-dominant logarithmic correction) from the satisfiability transition, hence a wide range of parameters α for which typical instances are known to have solutions, yet no provably efficient algorithm is known at present to find them. Some negative results have also been obtained. In [START_REF] Coja-Oghlan | Walksat stalls well below satisfiability[END_REF], it is shown that the WalkSAT algorithm is ineffective w.h.p. for densities of constraints larger than c2 k (ln k) 2 /k, with c > 0 a constant, while it has been shown in [START_REF] Coja-Oghlan | Analyzing walksat on random formulas[END_REF] that this algorithm finds satisfying assignment in linear time w.h.p. if α < c 2 k /k, with c > 0 another constant. In [START_REF] Gamarnik | Performance of sequential local algorithms for the random nae-k-sat problem[END_REF], it is shown that for the NAE-k-SAT problem, a certain class of local algorithms fail to find solutions for α > 2 k-1 (ln k) 2 /k, for large k.

Chapter 6

Biased Measures for random Constraint Satisfaction Problems

In this chapter we introduce biased probability measures over the set of solutions of random instances of CSPs, i.e. probability measures for which not all solutions are equiprobable. This perspective has been used in previous works [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF][START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF][START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF] (in [START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF] the local entropy, or density of solutions in configuration space, is used to weight differently the solutions, in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF] this role is played by the number of frozen variables, while in [START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF] hard sphere particles are considered as a CSP, with a bias due to an additional pairwise interaction between particles). Indeed the structural phase transitions (clustering, condensation, rigidity) introduced in chapter 2 depend on the choice of the measure describing the set of solutions. In [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF] it has been demonstrated that the threshold for properties that are typical in the uniform ensemble (in particular the existence of frozen variables) can be significantly moved by an appropriate bias. This opens some hope to diminish the algorithmic gap, by giving more weight to solutions that are "easier" to find, for instance because they contain less frozen variables, and to turn atypical properties of the uniform measure into typical ones of the biased measure.

We will present the form of the bias used during this Ph.D., that was studied in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] and in [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF]). We will also present the biases introduced in the papers mentioned above. In this Ph.D., we have focused on the increase of the dynamic threshold α d that results from a well-chosen bias between solutions. The algorithmic motivation for the study of this threshold comes from the Simulated Annealing [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF] procedure, as explained in chapter 3. Below α d a Markov Chain reversible with respect to a finite-temperature probability distribution should be able to equilibrate in polynomial time, hence to find solutions once the temperature is lowered slowly enough (if there is no reentrance in temperature). Although it is true that many algorithm do not respect detailed balance (focused algorithm), therefore are not affected by this transition, and that Simulated Annealing algorithm can find solutions even out-of-equilibrium, we observed that the bias introduced in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] had positive impact on the performances of Simulated Annealing. We have focused on the bicoloring on k-hypergraph, that exhibits the same threshold phenomena as k-SAT, but for which the computations are a bit simpler.

Definitions

Biased measure over the set of solutions

Let F be an instance of CSP, with S(F ) the set of its solutions. Assuming that it is non-empty, one can define the uniform measure over the set of solutions:

µ u (σ) = 1 Z u × 1 if σ ∈ S(F ) 0 if σ ∈ S(F ) (6.1)
where the subscript u stands for "uniform". The variables σ i (i ∈ {1, . . . , N }) leave in a discrete space χ (χ = {-1, +1} for spin variables). In the most generic form, a biased measure over the set of solutions can be writen:

µ b (σ) = b(σ) σ b(σ ) (6.2)
where b :

χ N → R is a non-negative function: b(σ) > 0 if σ ∈ S(F ) = 0 if σ ∈ S(F ) (6.3)
There are infinitely many possible choices for the function b, we will present the choices that we made during the Ph.D., and the choices made in the papers [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF][START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF][START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF]. Note that we need to impose some locality requirements on µ, so that the Monte Carlo simulations are tractable. Indeed at each step one needs to compute the change in energy when a variable is flipped. For measures defined with graphical models (such as (1.7)), computing the change in energy requires only a small number of operations. In the next chapters, we will apply the cavity method to study the typical properties of the measure describing the solution set S(F ), when F is drawn from a random CSP ensemble. This method requires that the measure can be described as a graphical model, and that the underlying graph representing the measure is locally tree-like in the thermodynamic limit. As we have seen, the uniform distribution µ u can be described by a graphical model whose underlying graph is the initial graph representing the instance F . We recall here its formulation as a graphical model (equation (1.10))

µ u (σ) = 1 Z u a∈C (1 -c a (σ ∂a )) = 1 Z u a∈C I[σ ∂a satisfies the clause a] (6.4)
In the rest of this section we will introduce biased measures that introduce local interactions (with respect to the notion of distance induced by the hypergraph representing F ) between variables. For simplicity we will consider only instances defined on k-hypergraphs.

Intra-clause bias

We introduce a biased function b that factorizes on the clauses:

b(σ) = a∈C ω(σ ∂a ) (6.5)
The function ω : χ k → R vanishes when the subset of variables does not satisfy the constraint a, and is strictly positive when it does. In that way the set of solutions S(G) is not modified. Note that the underlying graph representing this graphical model coincides with the initial graph representing the instance F .

Application to the bicoloring on k-hypergraphs

We study this form of bias on the bicoloring of k-hypergraphs in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF]. We recall that in that case, the variables are spins σ ∈ {+1, -1}, and each constraint a forbids the variables σ ∂a to take the same value:

1 -c a (σ ∂a ) = I[σ ∂a n.a.e] (6.6)
where n.a.e stands for 'not all equal'. The biased measure reads:

µ(σ) = 1 Z(G) a∈C ω(σ ∂a ) (6.7)
We assume that ω is invariant under all permutations of its k elements. As the latter are binary variables, ω depends only on the number p of -1 among its arguments, and we will denote ω p ≥ 0 the value it then assumes. This translates into the formula:

ω(σ 1 , . . . , σ k ) = ω p if k i=1 σ i = k -2p . (6.8)
The uniform measure over the solutions of the bicoloring problem is recovered for the choice

ω 0 = ω k = 0, ω 1 = • • • = ω k-1 = 1.
If one chooses instead ω p to depend on p for p ∈ {1, . . . , k -1}, while keeping ω 0 = ω k = 0, one obtains a probability measure µ that is still supported solely on the proper bicolorings of G, but is not uniform anymore. We will sometimes relax the constraint ω 0 = ω k = 0, to model the effect of a positive temperature that allows some constraints to be violated, as it will be used in Simulated Annealing. We will in any case always assume that ω p = ω k-p : this ensures that the global spinflip symmetry µ(-σ) = µ(σ) (which is indeed a property of the set of proper bicolorings) is preserved.

We have worked in particular with the specific choice:

ω 0 = ω k = 0, ω 1 = ω k-1 = 1 -, ω 2 = • • • = ω k-2 = 1 (6.9)
i.e. we weight solutions according the their number of "almost monochromatic" clauses (clauses with only one variable that takes a different value from the others). These "almost monochromatic" clauses are responsible for the existence of frozen variables. We recall that the frozen variables are variables that takes the same values in all the solutions of a cluster. This is therefore one of the mechanism of the RSB phenomenon, although not the only one. Indeed for instance when k ≥ 4, the rigidity transition α r is strictly greater than the clustering transition α d < α r , hence there in the range α d ≤ α < α r the set of solution is clustered yet typical solutions do not contain a positive fraction of frozen variables.

Bias with interactions at distance 1

We consider the following measure that introduces the shortest non-trivial interactions that allows the coupling of variables from different hyper-edges:

µ(σ) = 1 Z a∈C ω(σ ∂a ) i∈V ϕ i (σ i , {σ ∂a\i } a∈∂i ) (6.10)
in that case, the function ω is just the indicator function of the event "σ 1 , . . . , σ k satisfy the constraint". The biasing function ϕ i > 0 couples the i-th variable with its |∂i|(k -1) neighbors at distance 1.

Application to the bicoloring on k-hypergraphs

We study this form of bias on the bicoloring problem of k-hypergraph in [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF].

For simplicity we restrict ourselves to regular hypergraphs, where every vertex has the same degree |∂i| = l + 1. We used the same function ϕ on all the vertices. There is still a vast freedom in the choice of ϕ. We restrict this choice by imposing its invariance under a global spin-flip of its arguments (that enforces the global spin-flip symmetry σ → -σ of the set of solution). We also impose its invariance under the permutations of the l + 1 hyperedges around i and of the k -1 neighboring variables inside each of these hyperedges. This amounts to take

ϕ(σ i , {σ ∂a\i } a∈∂i ) = ϕ({m a→i } a∈∂i ) with m a→i = j∈∂a\i 1 + σ i σ j 2 (6.11)
and ϕ invariant under the permutations of its l + 1 arguments. m a→i counts the number of variables in ∂a \ i that are of the same color as σ i . In [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF] we discard part of the information contained in m a→i and only distinguish between the cases m a→i = 0 and m a→i > 0. This is indeed a relevant information about the solution σ: in the former case σ i is the only variable of its color in the a-th hyperedge, hence σ i cannot be flipped without violating the a-th monochromatic constraint. One says that a forces i in such a situation. On the contrary when m a→i > 0 the variable σ i is not forced to its value by the a-th hyperedge. With this simplification, and because of the invariance by permutation of the arguments of ϕ, the weight of the variable i becomes a function of the number of constraints forcing it. As σ is a solution, the event m a→i = 0 is equivalent to "the variables in σ ∂a\i are all equal (a.e)", and the biased measure becomes thus:

µ(σ) = 1 Z a∈C ω(σ ∂a ) i∈V ψ a∈∂i I[σ ∂a\i a.e] (6.12)
where ψ(p) is the weight attributed to a variable contained in p ∈ {0, . . . , l + 1} forcing hyperedges. With this choice, there are l + 1 free parameters to describe the function ψ (its argument can take l + 2 values, but a global multiplicative constant gets absorbed in the normalization Z).

As mentioned, the presence of forcing clauses is one mechanism of the RSB phenomenon, but cannot explain it on its own. In [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF] we showed that bias defined in (equation 6.12) was already giving positive results on the clustering threshold, both for small connectivity k and in the large k limit. In the future it would be interesting to study more general form of biases, keeping all the information in m a→i .

In [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF] we give a particular attention to the specific form:

ψ(0) = 1, ψ(p) = b(1 -) p for 1 ≤ p ≤ l + 1 (6.13)
which contains the two parameters b > 0 and < 1. This form actually encompasses the two following cases:

• when = 0, in such a way that:

ψ(p) = 1 if p = 0 b if p > 0 (6.14)
one recovers a measure studied in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], that we will present in the next section. Indeed the weight of a solution σ is then b raised to a power equal to the number of variables that are forced by at least one constraint, i.e. those that are not "whitened" after T = 1 step of a coarsening algorithm used in [START_REF] Achlioptas | On the solution-space geometry of random constraint satisfaction problems[END_REF][START_REF] Ding | Satisfiability threshold for random regular nae-sat[END_REF][START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF][START_REF] Braunstein | Constraint satisfaction by survey propagation[END_REF][START_REF] Parisi | On local equilibrium equations for clustering states[END_REF][START_REF] Braunstein | Survey propagation as local equilibrium equations[END_REF][START_REF] Elitza | A new look at survey propagation and its generalizations[END_REF], whose large deviations on atypical solutions were studied in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF] for arbitrary values of T (with an unfortunate conflict of notation the parameter called b here is denoted e in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF]).

• when b = 1 one has

ψ(p) = (1 -) p (6.15)
the weight of a solution σ is thus (1 -) raised to the number of forcing constraints, hence we recover the bias introduced in the previous paragraph (6.1.2), and studied in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF]. Indeed in a solution every constraint Center panel: the introduction of an interaction, represented as a black square, between all the vertices at distance 1 from the central vertex i, generates short loops even if G is a tree. Right panel: the factor graph representation of the probability measure (6.16), the white circles stand for the variable nodes v (i,a) , the black circles (resp. white squares) are the interaction factors ψ (resp. ω).

a forces at most one of its variable i ∈ ∂a, when σ i is the unique representant of its color in σ ∂a , hence there is no double counting of forcing constraints in the product over variables in (6.12).

Obviously the uniform measure (1.10) is recovered when b = 1 and = 0.

Factor graph and auxiliary variables

The factor graph representing the measure (6.12) is not the underlying graph G representing the instance F . Instead the function ψ has introduced short loops, as is it represented in figure 6.1 (middle panel). This prevents from a direct application of the cavity method, and requires a preliminary step in order to get rid of these short loops. To achieve this we introduce some auxiliary, redundant variables in the following way: for each edge (i, a) ∈ E between a vertex i and one of its incident hyperedge a ∈ ∂i we introduce two variables, w a→i ∈ {0, 1} and σ a i ∈ {-1, +1} which are deterministic functions of the original configuration σ, according to w a→i = I[σ ∂a\i a.e] and σ a i = σ i . We call v (i,a) = (σ a i , w a→i ) the value of these two auxiliary variables, and v = {v (i,a) } (i,a)∈E their global configuration. Consider now the following probability law for v:

µ(v) = 1 Z i∈V ϕ({v (i,a) } a∈∂i ) a∈C ω({v (i,a) } i∈∂a ) (6.16)
where

ϕ(σ 1 , w 1 , . . . , σ l+1 , w l+1 ) = ψ l+1 i=1 w i I[σ 1 = • • • = σ l+1 ] (6.17) ω(σ 1 , w 1 , . . . , σ k , w k ) = ω(σ 1 , . . . , σ k ) k i=1 I[w i = I[{w i = I[{σ j } j =i a.e]}]] (6.18)
One realizes easily that for a configuration v in the support of (6.16) σ a i is independent of a, and the marginal law of σ is nothing but (6.16). The partition function Z is the same in the two expressions (6.12) and (6.16), and in the support of µ(v) the variables w a→i are the deterministic functions of σ defined above. This equivalent rewriting with redundant variables has an important advantage: as shown in the right panel of Fig. 6.1 the graphical model corresponding to (6.16), with variables v on the edges (i, a) of G and interaction nodes both on the original hyperedges ( ω) and on the original vertices ( ϕ), respects the topology of G, and is thus (locally) a tree if G is.

Bias with interactions at larger distance

We have presented in the two previous paragraphs (6.1.2 and 6.1.3) the form of the bias studied in this Ph.D.. Although we did not investigate it, it is natural to extend these definitions, introducing interactions between variables at larger distance. We show in this paragraph that for some specific form of the bias, it is possible to apply a similar transformation as the one described above that introduce auxiliary variables, to recover a locally tree-like factor graph. This approach has been studied in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], with a bias counting the number of frozen variables in a solution.

Let B(i, r) be the set of variables at distance ≤ r from i. One would like to study the following probability measure:

µ(σ) = 1 Z a∈C ω(σ ∂a ) i∈V ϕ i (σ i , σ B(i,r) ) (6.19)
This measure cannot in general be represented by a simple graphical model, but it is possible to apply a similar transformation as the one used in the paragraph 6.1.3, to recover a locally tree-like underlying graph, if we assume that the function ϕ i is constructed in the following way. One each edge (i, a), one introduces a set of auxiliary variables {g

(s) i→a , g (s)
a→i } s∈{1,...,r} that are determined uniquely from the solution σ:

g (1) i→a = σ i (6.20) g (s) a→i = G s ({g (
ϕ i (σ i , σ B(i,r) ) = φ i ({ g (s)
a→i } s∈{1,...,r} a∈∂i

) (6.23)
Introducing the auxiliary variable σ a i = σ i as before, and defining for each edge (i, a) the auxiliary variable v (i,a) = (σ a i , {g

(s) i→a , g (s)
a→i } s∈{1,...,r} ), one can consider the probability law for v:

µ(v) = 1 Z i∈V ϕ i ({v (i,a) } a∈∂i ) a∈C ω({v (i,a) } i∈∂a ) (6.24)
where

ϕ i ({v (i,a) } a∈∂i ) = σ∈χ φ i ({ g (s)
a→i } s∈{1,...,r} a∈∂i

) a∈∂i I[σ a i = g (1) i→a = σ] × r-1 s=1 I[g (s+1) i→a = G s ({ g (s) b→i } b∈∂i\a )] (6.25) ω({v (i,a) } i∈∂a ) = ω({σ a i } i∈∂a ) k i=1 r s=2 I[ g (s) a→i = G s ({g (s) j→a } j∈∂a\i )] (6.26)
As before, one can check that for a configuration v in the support of this measure, σ a i is independent of a, and the marginal of σ is the initial biased measure. The partition function is the same in the two expressions.

This transformation can be applied for any definition of the auxiliary variables {g

(s) i→a , g (s) a→i } s∈{1,..

.,r} a∈∂i

and update rules G s , G s . In the cavity method that we will present in the next chapter, we study the typical properties of the measure µ chosen to describe the set of solution, when the graph G is drawn from a given random graph ensemble. One needs to introduce several layers of probability measures over the set of variables, therefore the computations are in general not tractable if the variables do not live in a small space. In practice it is needed that the auxiliary variables {g

(s) i→a , g (s)
a→i } s∈{1,...,r} a∈∂i live in a discrete space.

In [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], A. Braunstein, L. Dall'Asta, G. Semerjian and L. Zdeborova have used the WP messages ν

(s) i→a , ν (s)
a→i defined in chapter 4, obtained after s iteration of the WP equations, with initial condition ν (1) i→a = δ •,σi . In this paper, the bias is giving different weights to the configurations according to the fraction of variables that are still frozen after s iterations of the WP/whitening procedure. We will give more details about this bias in the next section.

A natural choice would have been to take for {g

(s) i→a , g (s) a→i } s∈{1,...,r} a∈∂i the BP messages {η (s) i→a , η (s) a→i } s∈{1,..

.,r} a∈∂i

obtained after s iterations, similarly as it was done for the WP messages in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF]. Of course the space of the BP messages is too big to be handled by the cavity method, even for messages over spin variable that can be described from their mean value m = σ η ∈ [-1, 1]. One could have chosen the bias to depend on the marginal obtained after s iterations:

ϕ(σ i , σ Bi,r ) = φ i ({η (s) i } s∈{1,...,r} ) (6.27) with η (s) i (σ i ) = a∈∂i η (s) a→i (σ i ) σ a∈∂i η (s) a→i (σ) (6.28)
In particular, one can think of a bias that would favor configurations for which the point-to-set correlation

C (s) i = σ ση (s) i (σ) σ η (s) i (σ) (6.29)
is small. Note however that this is the point-to-set correlation function corresponding to the uniform measure, hence trying to minimize it is not guaranteed to increase the clustering threshold associated with the biased measure. With the WP messages this problem does not happen, since they are the same for both the uniform and the biased measure.

Biased measures studied in the literature

Bias counting the frozen variables

We now describe more precisely the bias introduced in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF], defined on the bicoloring of k-hypergraphs. The biased measure weights the solutions according to their fraction of frozen variables during the whitening procedure. One can find the frozen variables running the WP iterations defined in chapter 4 (in paragraph 4.1.6). For spin variables, it is suitable to use the notations, defined from the set of BP messages {η i→a , η a→i }:

h i→a =      +1 if η i→a (σ i ) = δ σi,+1 -1 if η i→a (σ i ) = δ σi,-1 0 otherwise (6.30)
and similarly:

u a→i =      +1 if η a→i (σ i ) = δ σi,+1 -1 if η a→i (σ i ) = δ σi,-1 0 otherwise (6.31)
For the bicoloring problem the WP equations reads:

u t+1 a→i =      +1 if ∀j ∈ ∂a \ i : h t j→a = -1 -1 if ∀j ∈ ∂a \ i : h t j→a = +1 0 otherwise (6.32) h t+1 i→a =      +1 if ∃b ∈ ∂i \ a : u t+1 b→i = +1 -1 if ∃b ∈ ∂i \ a : u t+1 b→i = -1 0 otherwise (6.33)
To find the frozen variables in a solution σ, one runs the WP iterations, starting from the initial condition:

h 0 i→a = σ i (6.34)
One can show that the dynamical evolution of the WP messages is monotonic, and therefore guaranteed to converge to a fixed point, independently of the update scheme. One can translate the WP iterations into an evolution of configurations: σ t,W P ∈ {-1, +1, 0} N :

σ 0,W P i = σ i , and σ t,W P i =      +1 if ∃b ∈ ∂i \ a : u t+1 b→a = +1 -1 if ∃b ∈ ∂i \ a : u t+1 j→a = -1 0 otherwise (6.35)
One says that the variable σ i is whitened after time t if σ t,W P i = 0. Let t i (σ) the whitening time of the variable σ i :

t i (σ) = inf{t : σ t,W P i = 0} (6.36)
if t i (σ) = ∞ then σ i is declared frozen. Define the empirical cumulative distribution of the whitening times:

P t (σ, G) = 1 N N i=1 I[t i (σ) ≥ t + 1] (6.37) P t (σ, G)
is the fraction of still frozen variables at time t in σ. The authors have studied in particular the following biased measure:

µ(σ) = 1 Z( , T, G) σ M a=1 ω a (σ ∂a ) N i=1 exp [N P T (σ, G)] (6.38)
with a fixed time T ∈ {1, . . . , ∞}. When > 0, this bias favors configurations with a large fraction P T (σ, G) of frozen variables at time T , while when < 0, this favors configurations with a small fraction of frozen variables after T iterations of the whitening procedure. This measure can also be writen in terms of the whitening times of each variables: . This bias introduce interactions between variables at distance T . Note that they also studied a biased measure that weight solutions according to their fraction of frozen variables for all times:

µ(σ) = 1 Z( , T, G) σ M a=1 ω a (σ ∂a ) N i=1 b(t i (σ)) (6.
µ(σ) 1 Z({ t }, G) σ M a=1 ω a (σ ∂a ) exp ∞ t=1 N P t (σ, G) , ( 6.40)
thus introducing interactions at all distances in the graph. As already mentioned, the case T = 1 in the probability measure (6.38) corresponds to the bias introduced in section 6.1.3, with the correspondence b = e . The effect of the biased measure (6.38) (with T = 1) on the clustering transition is studied in [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF]. For k = 6, they observed that is could not improve on the value of α d . In [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF], we showed that with another choice for the function ψ(p) defined in (6.12) we could improve on its value, for k = 4, 5.

Under the RS assumption, the authors have computed the entropy s(T, θ, k, l) counting the number of solutions that have a fraction θ of frozen variables at time T :

s(T, θ, k, l) = lim N →∞ E G [s(T, θ, G)], with (6.41
)

s(T, θ, G) = 1 N lim N →∞ ln   σ∈S(G) I[P T (σ, G) = θ]   (6.42)
They have also computed the dominant path P t (T, θ, k, l) which is the fraction of frozen variables at time t conditioned on P T = θ.

P t (T, θ, k, l) = lim N →∞ E G σ∈S(G) P t (σ, G)I[P T (σ, G) = θ] σ∈S(G) I[P T (σ, G) = θ] (6.43)
If it tends to 0 as t → ∞, then typical solutions under the constraint P T = θ are unfrozen. They observed the existence of a threshold value θ tip called the tipping point: for T, k, l fixed, below (resp. above) θ tip the constraint P T = θ selects unfrozen (resp. frozen) solutions. For fixed k, T , they define the threshold l T (k) above which the entropy of the tipping point s(T, θ tip (T, k, l), k, l) becomes negative. Within the RS framework, this corresponds to disappearance of solutions such that P T = θ tip (k, l). In fact above l T (k), all the solutions have a fraction θ > θ tip of frozen variables at time T , thus contain a positive fraction of frozen variables in the large t limit. The threshold l T (k) can be interpreted as follows: below l T (k), it is possible to find a value of for which the typical solutions under the measure (6.38) are unfrozen, while for l > l T (k) it is not possible, and the typical solutions of (6.38) are frozen for any value of . They showed that l T =1 (k) > l r (k), where l r (k) is the rigidity transition under the uniform measure. They also showed that l T is growing with T . l f is interpreted as the limit T → ∞ of l T . They obtain the following large k scalings for this set of thresholds:

l T (k) = 2 k-1 k ln 2 ln •T (k) 1 + O ln •(T +1) (k) ln •T (k) (6.44)
where ln •T (k) is the T -times iterated logarithmic function: ln

•(T +1) (k) = ln(ln •T (k)).
They estimated the scaling freezing transition to occur at l f (k) ∼ 2 k-1 k ln 2/2 ∼ l sat (k)/2. We recall that the scaling for l r (k) is

l r (k) = 2 k-1 (ln k + ln ln k + 1 + O(ln ln k/ ln k)) (6.45)
In conclusion, with the biased measure (6.38) at finite T , it is possible to obtain typical unfrozen solutions (tuning the value of the parameter ), for densities of constraint significantly larger than the rigidity threshold corresponding to the uniform measure In [START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF], H. Zhao and H. Zhou study a similar biased measure on the random k-SAT problem. They introduce a bias that weight the solutions according to the number of variables that are needed to satisfy the clauses. The others variables are white, in the sense that when they are flipped, the new configuration is still a solution. In this formalism, the variables σ i can take 3 values (the white state being σ i = 0). A configuration σ lives in {-1, +1, 0} N The partition function associated with the measure is

Z = σ M a=1 ω(σ ∂a ) N i=1 e -βδ 0 σ i (6.46)
where ω(σ ∂a ) ensure the constraint a to be satisfied:

ω(σ ∂a ) = 1 - i∈∂a 1 -δ J a i σi . ( 6.47)
The parameter β > 0 favors configurations σ ∈ {-1, +1, 0} N with a maximal number of variables that are equal to 0, therefore that do not contribute to the satisfaction of the clauses. They apply the BP-guided decimation procedure on this biased measure, and compute the probability of success for random 3-SAT. They observe a small positive probability of success for α 4.15, while for the BPD algorithm on uniform measure they observe null probability of success above 4.15, thus indicating that the biased measure allows for better performances. Note that increasing the value of β, one might enter in a condensed phase where the BP marginals do not provide a good estimation on the exact marginals. It would also be interesting to estimate numerically the algorithmic threshold for this algorithm, as defined in chapter 5.

Local entropy

In [START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF] the authors study a measure that favors solutions that are in regions with a high density of solutions. These regions are believed to be more accessible by the search algorithms, because they correspond to wide minima in the energy landscape. They weight each solution σ according to its local entropy, i.e. a measure of the number of solutions σ whithin a given distance from the reference solution σ. The local entropy is defined as follows:

F (σ, γ) = 1 N log N(σ, γ) (6.48)
where

N(σ, γ) = σ a∈C
ω(σ ∂a )e -γd(σ,σ ) (6.49)

N(σ, γ) is a weighted sum on the solutions σ ∈ S(G), with a weight that depends on the Hamming distance from the reference configuration σ. γ is a coupling strength, in the limit γ → ∞ this sum is dominated by the reference solution.

When γ is small the terms selected in the sum belongs to wider regions around σ. They show that this quantity can be used as a cost function to optimize on. More precisely, they introduce the following measure:

µ(σ) = 1 Z(y, γ)
e yF (σ,γ) (6.50)

Note that they do not impose the condition σ ∈ S(F ), and allow positive weights for configurations σ that are not solutions. In fact they expect that configurations that are surrounded by a dense region of solutions are themselves solutions.

In practice they observe that this is the case experimentally: approximate sampling procedures on this measure allow to find solutions. The local entropy plays the role of the energy, and the parameter y plays the role of the inverse temperature. When y is large this favors configurations σ with large local entropy, i.e. surrounded by a a dense region of solutions. In [START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF] they also consider an additional parameter β, looking at the following distribution:

µ(σ) = 1 Z(β, y, γ) e yφ(σ,β,γ) ; φ(σ, β, γ) = ln   σ e -βE(σ )-γd(σ,σ )   (6.51)
φ(σ, β, γ) is called the local free entropy, taking its β → ∞ limit, one recovers the local entropy F (σ, γ). With low value of γ, the entropy landscape is smoother than the energy landscape, its local minima corresponding to dense regions of local minima in the energy landscape. This corresponds to coarse-grain the energy landscape, with γ controlling the granularity. The goal with this approach is to avoid being trapped in the local minima of the energy landscape.

Increasing the value of γ, one focus on the configurations σ that are closer to the reference solution σ, therefore this allows to focus on configurations σ which are inside a narrow local minimum. If β is large enough this is likely to be a global minimum. When γ → ∞ the only configuration selected in φ(σ, β, γ) is σ = σ, and one recovers the initial Gibbs measure, with inverse temperature βy, and the entropy landscape tends to the initial energy landscape. They perform approximate sampling procedures, such as Simulated Annealing, and Message-Passing decimation procedures to sample from this distribution, with cooling schemes that progressively increase the value of the parameters β, γ and y. Note that the local free entropy and the local entropy are not easy to compute, compared to the energy function.

To obtain this quantity, a first approach developed in [START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF] is to study the system defined in N, in which each variable i is coupled to an external field e γσ i (using the expression of the Hamming distance:

d(σ, σ ) = N i=1 (1 -σ i σ i )/2)).
The parameter β is not considered in [START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF] (β = ∞). F (σ, γ) can be estimated with the Bethe Peirls approximation, running the BP equations for this modified system, and computing the Bethe free energy from the messages obtained. Note that this approach might fail in the condensed phase, because the BP iterations might not converge, or the Bethe estimation might not be correct. They apply a Simulated Annealing algorithm defined as follows: σ is initialized at random. At each step F (σ, γ) is computed. Then a random local flip is accepted or rejected using Metropolis rule at fixed temperature 1/y. More precisely, let ∆F be the difference in local entropy. If there is an increase in ∆F , then the flip is always accepted. If there is a decreese in ∆F , then the flip is accepted w.p e y∆F , and rejected otherwise. Applying this algorithm on the perceptron and on the k-SAT problem, they observe that this algorithm performs better than the usual Simulated Annealing algorithm that uses the energy function (as defined in chapter 3).

In [START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF] they use another approach to compute the local free entropy. Instead of using BP, consider that y is a non-negative integer, and rewrite the partition function

Z(β, y, γ) = σ e yF (σ,β,γ) = σ,σ 1 ,...,σ y e -β y a=1 E(σ a )-γ y a=1 d(σ,σ a ) = σ 1 ,...,σ y e -β( y a=1
E(σ a )+A({σ a ,β,γ})) (6.52) where in the last equation the reference configuration has been traced out

A({σ a , β, γ}) = - 1 β ln σ e -γ y a=1 d(σ,σ a ) (6.53)
They apply Simulated Annealing algorithm to sample from this measure, with increasing γ, decreasing β. At each step, a replica a ∈ {1, . . . , y} is randomly chosen, and the randomly chosen variable inside this replica is flipped according to the Metropolis rule computed from the change in the energy function defined from the probability law with partition function (6.52). The algorithm stops either when one solution is found in one of the replicas, or when the maximal number of steps is reached. They observe a better performance using this method on the perceptron, compared to the non interacting case (γ = 0), which correponds to running the energetic Simutaled Annealing procedure in parallel on the y independent copies.

Biased interactions in the hard sphere packing problem

In [START_REF] Sellitto | A thermodynamic description of colloidal glasses[END_REF] and [START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF] the authors use a biased measure to sample solutions for the problem of packing hard spheres. In this problem, a set of N spheres of fixed radius live in a d-dimensional volume V . The variables correspond to the position of each sphere. The constraint is that for each pair of spheres, the distance between their centers is larger than their diameter, so that they do not overlap, hence the name hard sphere. A solution is a assignment of the positions of the spheres so that there is no overlapping. The parameter controlling the hardness of this problem is the packing fraction ϕ = N v d /V , v d being the volume of a unit-diameter sphere. The thermodynamic limit is N, V → ∞ with ϕ constant. As ϕ increases, one encounters a dynamical transition ϕ d similar to the one found in random k-SAT. Below ϕ d , the solutions can be sampled uniformly in polynomial time in N , using Monte Carlo methods, or molecular dynamics, while above this requires an exponential time. This result is valid in the large d limit (taken after the thermodynamic limit).

The uniform measure over the set of solutions can be viewed as a Gibbs-Boltzmann distribution describing a set of particles that interact with each other through hard interactions. The authors introduce a bias in the interactions: keeping the hard constraint (i.e. forbidding the spheres to overlap), they add a soft interaction at larger distances. They look at the change of ϕ d under this bias, trying to maximize its value. The biased pair-wise interaction v(r) between two particles at distance r takes the following form:

v(r) = ∞, r < 1 v + (r), r > 1 (6.54)
where we have considered spheres of unit diameter, so that two spheres are overlapping when their distance r is smaller than 1. v + (r) is an arbitrary function that decrease to 0 as r → ∞. The uniform case is recovered for v + (r) = 0. More precisely, in [START_REF] Sellitto | A thermodynamic description of colloidal glasses[END_REF], they have shown that with this specific form:

v + (r) = -U 0 , r ≤ r 0 0, r > r 0 (6.55)
for some r 0 > 1 and U 0 > 0, it is possible to increase the dynamical transition ϕ d , thus to increase the range for which it is possible to sample solutions in polynomial time in N . This interaction introduces short range attractive interaction, just after the hard wall. Intuitively, this means that a measure that favor solutions that are packed more closely can be sampled in polynomial time for higher packing fractions than the uniform measure.

In [START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF] a more detailed soft interaction with more than one step is studied:

v + (r) =                -U 0 , r ≤ r 0 -U 1 , r 0 < r ≤ r 0 + r 1 . . . -U n-1 , n-2 i=0 r i < r ≤ n-1 i=0 r i 0, r > n-1 i=0 r i (6.56)
Optimizing on the several parameters, they could improve on the dynamical packing fraction ϕ d . They observe that the profile that maximises the value of ϕ d should have a short-range attractive part, then a weak repulsive tail at larger r.

Chapter 7

Cavity Method

In this chapter we present the cavity method treatment of random CSPs. We apply this method to the bicoloring on k-hypergraphs, and study the typical properties of the two biased measures (defined in (6.7) and in (6.12)) that describe the set of solutions. The replica and the cavity method have been first developed in the context of statistical mechanics of disordered systems. Random CSPs bear a formal similarity with mean-field spin glasses, the interactions induced by the constraints being of a frustrating nature while lacking a finitedimensional structure thanks to the randomness in their construction. This analogy has allowed to apply these techniques to random CSPs [START_REF] Monasson | 2+p-sat: Relation of typical-case complexity to the nature of the phase transition[END_REF][START_REF] Biroli | A variational description of the ground state structure in random satisfiability problems[END_REF][START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Mertens | Threshold values of random k-sat from the cavity method[END_REF][START_REF] Krzakala | Gibbs states and the set of solutions of random constraint satisfaction problems[END_REF]. This line of study has provided predictions of α sat (k) for many models [START_REF] Mézard | Spin-Glass Theory and Beyond[END_REF][START_REF] Mézard | Physics, Information, Computation[END_REF], but also unveiled many other phase transitions for the structure of the set of solutions in the satisfiable phase [START_REF] Zdeborová | Generalization of the cavity method for adiabatic evolution of gibbs states[END_REF] (see chapter 2 for the description of some of them). Moreover, it has lead to the proposal of new algorithms that exploit this detailed picture of the solution space (see chapter 5 for the presentation of some of them). Many of these (heuristic) predictions have been confirmed rigorously later on [START_REF] Achlioptas | On the solution-space geometry of random constraint satisfaction problems[END_REF][START_REF] Achlioptas | Algorithmic barriers from phase transitions[END_REF][START_REF] Molloy | The freezing threshold for k-colourings of a random graph[END_REF][START_REF] Ding | Proof of the satisfiability conjecture for large k[END_REF].

We will pay a particular attention to the clustering phase transition α d ≤ α sat , which is also known as the dynamical or reconstruction transition. This transition can be defined in several ways. Below α d the set of solutions of typical instances is rather well-connected, any solution can be reached from another through a path consituted of nearby solutions. Above α d the solution set splits into an exponential number of isolated subsets of solutions, called clusters, which are internally well-connected but separated from each other by regions without solutions. This is called the clustering phenomenon (see section 2.3.1). This transition is also defined as the appearance of long range correlations between variables, known as the point-to-set correlation [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF]. It was shown in [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF] that these correlations forbid the rapid equilibration of the stochastic processes that respect the detailed balance condition, such as Simulated Annealing algorithm (see section 3.1.4). This property motivates the name dynamic transition. In the cavity method [START_REF] Mézard | The bethe lattice spin glass revisited[END_REF] treatment of the random CSPs, we define α d as the appearance of a non trivial solution of the one step of Replica Symmetry Breaking (1RSB) equation with Parisi breaking parameter X = 1. The appearance of a non-zero point-to-set correlation also implies the solvability of an information theoretic problem called hypertree reconstruction [START_REF] Mossel | Information flow on trees[END_REF], see [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF] for the connection between 1RSB equations at X = 1 and the reconstruction problem, that will be discussed in Section 7.3. In this setting, α d corresponds to the hypertree reconstruction threshold, above which the problem becomes solvable.

The study of the structural phase transitions in the satisfiable regime, and in particular the definition of α d in terms of long-range correlations, relies on the characterization of a specific probability law on the space of configurations, namely the uniform measure over solutions. In this chapter we will compute the clustering threshold for the two biased measures defined in the previous chapter (in (6.7) and in (6.12)). In chapter 8 we will show that for finite values of k optimizing on the parameters describing the biases allows us to increase α d , and therefore to improve the performances of simulated annealing (we checked it for the biased measure (6.7)). In chapter 10 we will study the effect of the bias on the clustering threshold in the large k limit.

This chapter is organized as follows. In Section 7.1 we recall the definitions of the two biased measures, and write the BP equations (see section 4.1.2). In Section 7.2 we present the simplest version of the cavity method, called replica symmetric (RS) that provides predictions for the typical properties of the measures. In Section 7.3 we derive the formalism that allows us to compute the clustering threshold, exploiting its definition in framework of the reconstruction problem. In Section 7.4 we explain how to compute the rigidity threshold α r , that provides an upper bound on the clustering threshold α d ≤ α r . In section 7.5 we derive recursive distributional equations that will be used in the large k expansion of α d (see chapters 9 and 10). In Section 7.6 we present the 1RSB formalism, that is the first non-trivial level of the cavity method, which allows to deal with the RSB phenomenon, and provides estimates for several structural phase transitions (such as the clustering, condensation and satisfiability transitions). Finally in Section 7.7 we compute an analytical upper-bound on α d , the Kesten-Stigum threshold. This chapter summarizes part of the derivations performed in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF], [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] and [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF].

Definition of the model 7.1.1 Graphical model

We consider in this chapter the two graphical models representing the biased measures introduced in chapter 6. We will denote by Θ 0 the graphical model for the intra-clause measure (see equation (6.7)), that we recall here:

µ Θ0 (σ) = 1 Z Θ0 (G) M a=1 ω(σ ∂a ) (7.1)
Θ 0 gathers both the underlying factor graph G Θ0 (which in this case coincides with the initial factor graph G = (V, C, E) representing the instance F ), and the choice of the function ω (defined by the parameters {ω p } p=0,...,k as in equation (6.8)). Note that the uniform measure is recovered in this setting by choosing

ω 0 = ω k = 0 and ω 1 = • • • = ω k-1 = 1.
As explained in the section 6.1.3, the factor graph of the biased measure with interactions at distance 1 (see equation (6.12)) contains small loops compared to the initial factor graph representing F . The measure (6.12) has been modified, introducing auxiliary variables v (i,a) on the edges (i, a) ∈ E, where v = (σ, w) ∈ {-1, +1} × {0, 1}. We will denote by Θ 1 the graphical model for the measure over the set of variables v = {v (i,a) } (i,a)∈E (see equation (6.16)). Θ 1 gathers the modified factor graph G Θ1 and the choice of the function ψ (introduced right after the equation (6.12)) that counts the number of forcing clauses surrounding each vertex i ∈ V . In this setting, the uniform measure is recovered by choosing ψ(p) = 1 for all p. The new factor graph G Θ1 lies on the initial factor graph G, the new variables v i,a sit on the initial edges (i, a) ∈ E, and there are two types of function nodes, living on the vertices V and the k-hyperedges C respectively. We recall the expression of the measure µ Θ1 introduced in (6.16):

µ Θ1 (v) = 1 Z Θ1 (G) i∈V ϕ({v (i,a) } a∈∂i ) a∈C ω({v (i,a) } i∈∂a ) (7.2)
We recall the expression of the function nodes:

ϕ(σ 1 , w 1 , . . . , σ l+1 , w l+1 ) = ψ l+1 i=1 w i I[σ 1 = • • • = σ l+1 ] (7.3
)

ω(σ 1 , w 1 , . . . , σ k , w k ) = ω(σ 1 , . . . , σ k ) k i=1 I[w i = I[{w i = I[{σ j } j =i a.e]}]] (7.4)
Assume that the underlying factor graph G = (V, C, E) is distributed according to a random ensemble, such as the k-uniform l + 1-regular random graph ensemble, or the Erdős Rényi random graph ensemble G N (k, M ), that have been defined in chapter 1, part 1.4.2. As explained in chapter 1 (part 1.4.5), these structures are locally tree-like. The aim of the cavity method is to determine the typical properties of the graphical models µ Θ (with Θ ∈ {Θ 0 , Θ 1 }) defined on this graph, and of the free energy density ln Z Θ (G)/N in the thermodynamic limit. The first step of the cavity method amounts to study such models on finite trees. In that case we have seen in chapter 4 that one can exploit the recursive nature of trees to derive an exact description of µ Θ in terms of its marginals (see part 4.1.3), using the Belief Propagation messages (defined part 4.1.1). The free energy can be expressed in terms of these messages (see part 4.1.4).

BP equations and Bethe free-energy

We recall in this section the Belief Propagation equations introduced in chapter

Intra-clause bias

We introduce the messages η i→a and η a→i on each edge (i, a) of the factor graph G Θ0 = G, that are the marginal probability laws of σ i in amputated graphs where some interactions are discarded; η i→a is the marginal of σ i in the factor graph where one removes the hyperedge a, and η a→i is the marginal of σ i in the factor graph where one removes all the hyperedges in ∂i \ a. Removing an interaction in a tree breaks it into independent subtrees, which allows to write recursive equations between these messages:

η i→a (σ i ) = 1 z ia b∈∂i\a η b→i (σ i ) (7.5) η a→i (σ i ) = 1 z ai σ ∂a\i ω(σ ∂a ) j∈∂a\i η j→a (σ j ) , ( 7.6)
where the constants z ia and z ai are normalizing factors. These equations are valid for any (discrete) domain of the spins σ i ; as we are studying the case where σ i = ±1, we can parametrize the probability laws η i→a and η a→i by their mean values, defining

η i→a (σ i ) = (1 + h i→a σ i )/2 and η a→i (σ i ) = (1 + u a→i σ i )/2, with h i→a , u a→i ∈ [-1, 1].
The recursive equations can be rewritten with this parametrization as

h i→a = f Θ0 ({u b→i } b∈∂i\a ) , u a→i = f Θ0 ({h j→a } j∈∂a\i ) , (7.7)
where the functions f Θ0 and f Θ0 read explicitly

f Θ0 (u 1 , . . . , u d ) = d i=1 (1 + u i ) - d i=1 (1 -u i ) z 0,Θ0 (u 1 , . . . , u d ) (7.8) z 0,Θ0 (u 1 , . . . , u d ) = d i=1 (1 + u i ) + d i=1 (1 -u i ) (7.9) f Θ0 (h 1 , . . . , h k-1 ) = σ1,...,σ k ω(σ 1 , . . . , σ k )σ k k-1 i=1 (1 + h i σ i ) z 0,Θ0 (h 1 , . . . , h k-1 ) (7.10) z 0,Θ0 (h 1 , . . . , h k-1 ) = σ1,...,σ k ω(σ 1 , . . . , σ k ) k-1 i=1 (1 + h i σ i ) (7.11)
The function f Θ0 has been written here for a vertex of degree d + 1. On a tree the equations (7.7) admit a single solution, that can be found by iterating them from the leaves towards the interior of the graph. Once this solution is determined one can easily compute the marginal probability of σ i under µ Θ0 using the formula in (7.5) with all messages incoming onto i (see section 4.1.3), as well as the partition function Z Θ0 (G). The Bethe free entropy density introduced in section 4.1.4 provides an exact expression for the free entropy density:

(1/N ) ln Z Θ0 (G)) = Φ Bethe Θ0 where Φ Bethe Θ0 = 1 N N i=1 ln Z v 0,Θ0 ({u a→i } a∈∂i ) + 1 N M a=1 ln Z c 0,Θ0 ({h i→a } i∈∂a ) - 1 N (i,a)
ln Z e 0,Θ0 (h i→a , u a→i ) , (7.12) where the last sum runs over the edges of the factor graph, and the local partition functions are defined as:

Z v 0,Θ0 (u 1 , . . . , u d ) = σ d i=1 1 + σu i 2 , ( 7.13)
Z c 0,Θ0 (h 1 , . . . , h k ) = σ1,...,σ k ω(σ 1 , . . . , σ k ) k i=1 1 + σ i h i 2 , ( 7.14
)

Z e 0,Θ0 (h, u) = σ 1 + σh 2 1 + σu 2 . ( 7.15)
Finally, the Shannon entropy of the measure µ Θ0 can be expressed from the BP messages, see equation (4.21):

S[µ Θ0 ] = N Φ Bethe Θ0 - a∈C σ ∂a ω(σ ∂a ) ln(ω(σ ∂a )) i∈∂a η i→a (σ i ) σ ∂a ω(σ ∂a ) i∈∂a η i→a (σ i ) (7.16)
Bias with interactions at distance 1

We now specify the BP equations and the expression of the thermodynamic quantities for the measure µ Θ1 (equation (7.2)). The BP messages are η i→a and η a→i , the marginal laws of the variable v (i,a) that is placed on the edge (i, a) of G, in graphs where one has removed the interactions a and i, respectively. These definitions are illustrated in the right panel of figure 7.1. Note that in a literal application of the BP algorithm one would have introduced messages from every variable node to every interaction node, for instance η i→(i,a) and η (i,a)→a ; as the variable nodes are of degree two these two messages are actually equal, we denoted their common value η i→a to lighten the notations. The BP equations between these messages are of the form

η i→a = f Θ1 ({ η b→i } b∈∂i\a ), η a→i = f Θ1 ({η j→a } j∈∂a\i ) (7.17)
where the functions f Θ1 and f Θ1 derive from the interaction nodes ϕ and ω stated in equations (7.3,7.4). The relation η = f Θ1 ( η 1 , . . . , η l ) is thus found to mean Center panel: the introduction of an interaction, represented as a black square, between all the vertices at distance 1 from the central vertex i, generates short loops even if G is a tree. Right panel: the factor graph representation of the probability measure (7.2), the white circles stand for the variable nodes v (i,a) , the black circles (resp. white squares) are the interaction factors ψ (resp. ω). The messages η i→a and η a→i obey the Belief Propagation equations (7.17).

η(σ, w) = 1 z 0,Θ1 ( η 1 , . . . , η l ) w1,...,w l ψ w + l i=1 w i l i=1 η i (σ, w i ) (7.
where z 0,Θ1 ( η 1 , . . . , η l ) is a normalization constant. Similarly η = f Θ1 (η 1 , . . . , η k-1 ) stands for:

η(σ, w) = 1 z 0,Θ1 (η 1 , . . . , η k-1 ) σ1,...,σ k-1 w1,...,w k-1 ω(σ, σ 1 , . . . , σ k-1 )I[w = I[σ 1 , . . . , σ k-1 a.e]] k-1 i=1 η i (σ i , w i )I[w i = I[σ, σ 1 , . . . , σ i-1 , σ i+1 , . . . , σ k-1 a.e]] (7.19)
with z 0,Θ1 (η 1 , . . . , η k-1 ) a normalization constant. More explicitly one has:

η(σ, 1) = 1 z 0,Θ1 (η 1 , . . . , η k-1 ) k-1 i=1 η i (-σ, 0) (7.20) η(σ, 0) = 1 z 0,Θ1 (η 1 , . . . , η k-1 ) k-1 i=1 η i (-σ, 1) j =i η j (σ, 0) + I⊂{1,...,k-1} 2≤|I|≤k-2 i∈I η i (-σ, 0) i ∈I η i (σ, 0) (7.21)
as there is at most one variable which is the unique representant of its color in a set of k ≥ 3 binary variables that is not monochromatic.

The Bethe free energy takes the following form:

Φ Bethe Θ1 = 1 N N i=1 ln Z v 0,Θ1 ({ η a→i } a∈∂i ) + 1 N M a=1 ln Z c 0,Θ1 ({η i→a } i∈∂a ) - 1 N (i,a)
ln Z e 0,Θ1 (η i→a , η a→i ) , (7.22) where:

Z v 0,Θ1 ( η 1 , . . . , η d ) = σ w1,...,w d ψ d i=1 w i d i=1 η i (σ, w i ) , ( 7.23)
Z c 0,Θ1 (η 1 , . . . , η k ) = σ1,...,σ k w1,...,w k ω(σ 1 , . . . , σ k ) k i=1 η i (σ i , w i )I [w i = I[{σ j } j =i a.e]] , (7.24
)

Z e 0,Θ1 (η, η) = σ,w η(σ, w) η(σ, w) . ( 7.25)
For the Shannon entropy of the measure µ Θ1 we get the expression:

S[µ Θ1 ] = N Φ Bethe Θ1 - i∈V σ,{w ( i,a)} a∈∂i ψ a∈∂i w (ia) ln ψ a∈∂i w (ia) i∈∂a η i→a (σ, w (i,a) )
σ,{w (i,a) } a∈∂i ψ a∈∂i w (ia) i∈∂a η i→a (σ, w (i,a) ) (7.26)

Replica symmetric cavity method

The aim of the cavity method is to characterize the properties of the measure µ Θ for typical random graphs in the thermodynamic limit. In particular one would like to estimate the value of the quenched free-entropy density and of the averaged Shannon entropy density

Φ(k, α, Θ) = lim N →∞ 1 N E[ln Z Θ (G)], s(k, α, Θ) = lim N →∞ 1 N E[S[µ Θ ]] (7.27)
thanks to the self-averaging phenomenon, the free entropy density (ln Z Θ (G))/N and the entropy density are expected to concentrate on these values. There are different versions of the cavity method, that rely on self-consistent hypotheses of various complexity on the effect of long loops. In the simplest version, called replica symmetric (RS), one assumes a fast decay of the correlations between distant variables in the measure µ Θ , in such a way that the BP equations converge to a unique fixed point on a typical large instance, and that the measure is well decribed by the locally tree-like approximation. Increasing α above the clustering threshold α d (k, Θ) causes the appearance of long-range correlations between distant variables under the measure µ Θ . In such a case the RS hypothesis breaks down, we will present in the section 7.6 the one-step replica symmetry breaking formalism, that is able to deal with this RSB phenomenon [START_REF] Mézard | The bethe lattice spin glass revisited[END_REF].

RS cavity equations

Consider an uniformly chosen directed edge i → a, in a random hypergraph, and call P RS the probability law of the message η i→a solution of the BP equation.

Similarly let P RS be the probability law of the message η a→i . The RS cavity method assumes the decorrelation of the incoming messages on a uniformly chosen vertex i (resp hyperedge a). The incoming messages are i.i.d with the law P RS (resp. P RS ). The distributions obey the following self-consistent equations: with η drawn from P RS , d is a random variable drawn from r d , and the η i 's are i.i.d. copies of a random variable of law P RS . In general, there might be non-uniqueness of the solution of the RS equations (7.28,7.29), and the determination of the relevant solution (that describes properly the typical properties of the measure) is not obvious. The solution of these equations can be solved using population dynamics, that one will explain in the next chapter, section 8.1. We will see however that for the measure µ Θ0 the relevant solution can be found by symmetry considerations. For the measure µ Θ1 , it is also possible to guess the solution, when one focuses on the k-regular l + 1-uniform ensemble.

P RS (η) = ∞ d=0 r d d i=1 d P RS ( η i )δ[η -f Θ ( η 1 , . . . , η d )] (7.28) P RS ( η) = k-1 i=1 dP RS (η i )δ[ η -f Θ (η 1 , . . . , η k-1 )] ( 7

Intra-clause bias

As we assume that ω p = ω k-p , i.e. that the model is invariant under the global spin-flip symmetry, the RS equations admit as a solution the uniform distributions P RS (h) = δ(h), P RS (u) = δ(u). For frustrated models with an antiferromagnetic character this is the relevant solution (the local order that might emerge from a non-trivial solution is incompatible with the long loops of all sizes).

The RS cavity prediction for the free-entropy (7.27) is then obtained by averaging the Bethe expression (7.12) with respect to the message distributions P RS and P RS , which yields:

Φ RS (k, α, {ω p }) = ∞ d=0 p d d i=1 du i P RS (u i ) ln Z v 0 (u 1 , . . . , u d ) + α k i=1 dh i P RS (h i ) ln Z c 0 (h 1 , . . . , h k )
-αk dhduP RS (h) P RS (u) ln Z e 0 (h, u) . (7.31) One obtains explicity the value of the free-entropy by inserting the trivial solution of the RS equations into (7.31):

Φ RS (k, α, {ω p }) = ln 2 + α ln 1 2 k σ1,...,σ k ω(σ 1 , . . . , σ k ) = ln 2 + α ln 1 2 k k p=0 k p ω p . ( 7.32)
Note that this expression actually coincides with the annealed (first moment) computation lim N →∞ (ln E[Z(G)])/N . In the special case

ω 0 = ω k = 0, ω 1 = • • • = ω k-1 = 1
, for which µ Θ0 (σ) corresponds to the uniform measure over proper bicolorings, the partition function Z Θ0 (G) counts the number of solutions, hence the free-entropy ln Z Θ0 (G) is equal to the entropy of the uniform measure. The prediction of the RS cavity method for the entropy density is thus (using a subscript 'u' for uniform):

s RS u (k, α) = Φ RS u (α) = ln 2 + α ln 1 - 1 2 k-1 . (7.33)
For a generic choice of parameters {ω p } the free-entropy ln Z Θ0 differs from the (Shannon) entropy of the measure µ Θ0 (σ). Averaging over the expression (7.16), and using the expression of the joint marginal of the variables around a constraint (see equation (4.8)) one thus obtains the RS prediction for the entropy density

s RS (k, α, {ω p }) = ln 2 + α ln 1 2 k k p=0 k p ω p -α k p=0 k p ω p ln ω p k p=0 k p ω p . (7.34)
This quantity is a decreasing function of α and becomes negative for α > α s=0 , with

α s=0 (k, {ω p }) = ln 2 k p=0 ( k p )ωp ln ωp k p=0 ( k p )ωp -ln 1 2 k k p=0 k p ω p . (7.35)
The negativity of the entropy for α > α s=0 is a clear sign of the failure of the RS assumptions, the Shannon entropy of a discrete probability measure being non-negative. This is however not the only mechanism for the appearance of Replica Symmetry Breaking (RSB), as we shall see this phenomenon can occur in a phase with s > 0.

Bias with interaction at distance 1

In this case, it is a priori not obvious to guess the solution of the RS equation (7.28,7.29) from symmetry arguments. Indeed, the dependence in w of the BP messages is non-trivial, and may vary from one vertex to another. In the general case, one needs to solve numerically the equations (7.28,7.29). This can be done using population dynamics, a method that we will explain in the section 8.1. However one might find more than one solution, and it remains difficult to determine which solution is relevant to estimate the typical properties of the measure µ Θ1 . The situation gets simplified in the case of the k-uniform l + 1regular hypergraph. In this case, the local neighborhood of every vertex is the same, it is thus natural to look for a translationally invariant solution of the BP equations. Moreover the probability measure we are studying is invariant under the spin-flip symmetry σ → -σ, we can thus further restrict ourselves to a solution of the BP equation that respects this invariance. This amounts to take η i→a (σ, w) = η * (w), η a→i (σ, w) = η * (w) for all edges (i, a). Plugging this form into (7.17) yields the equations satisfied by η * and η * :

η * (w) = 1 z l p=0 l p ψ(p + w) η * (0) l-p η * (1) p , (7.36) η * (1) = 1 z η * (0) k-1 , (7.37) η * (0) = 1 z (k -1)η * (1)η * (0) k-2 + (2 k-1 -k -1)η * (0) k-1 . (7.38)
Introducing the ratio of probabilities

y = η * (0) η * (1) , y = η * (0) η * (1) , ( 7
, y = 2 k-1 -k -1 + k -1 y . ( 7.40)
It turns out that for any choice of the parameters k, l, and ψ there exists a unique solution (y, y) to the equations (7.40), which might not be obvious at first sight; a proof of this existence and uniqueness is provided in Appendix A.

One obtains the RS prediction for the free entropy density and the entropy density by averaging the expressions (7.22) and (7.26); on the translationally invariant solution this yields: One can see again for some choices of the parameters, in particular when l gets large enough, this expression of the entropy becomes negative. We denote l s=0 (k, ψ) the first integer value such that the entropy becomes negative.

Φ RS (k, l, ψ) = 1 - (l + 1)(k -1) k ln 2 -(l + 1) ln 1 + 1 y y + ln l+1 p=0 l + 1 p ψ(p) y -p + l + 1 k ln 2 k-1 -k -1 + k y , (7.41

The dynamic transition

We shall now present the formalism that allows to compute the location of the dynamic transition which, as explained in the introduction, manifests itself in different ways. Here we shall exploit its definition in terms of the existence of long-range point-to-set correlations in the probability measure µ Θ [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF][START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF], that are related to the solvability of a tree reconstruction problem [START_REF] Mossel | Information flow on trees[END_REF]. In the section 7.6 we will explain the connection with the 1RSB formalism.

The point-to-set correlation function, or overlap, has been introduced in chapter 2 (see equation (2.15)). The function C(i, B) measures the correlation between the variable at the point i and those in the set B. Let us define the point-to-set correlation function averaged over the graph ensemble, at distance n, as follows:

C n = lim N →∞ E[C(0, B n )] = lim N →∞ E[ σ 0 σ 0 σ Bn -σ 0 2 ] , (7.43)
where 0 is an arbitrary reference vertex and B n is the set of vertices at distance n from 0; • denotes the expectation with respect to µ Θ , while • σ Bn is the conditional average with the law µ Θ (•|σ Bn ). Note that the second term in C n actually vanishes thanks to the invariance of µ Θ under the spin-flip transformation. For the biased measure µ Θ1 with interactions at distance 1, because the interactions in the biased measure couple spins belonging to neighboring hyperedges it is not enough to take for B n the set of variables at distance exactly n from the root; it is however equivalent to include in B n all the variables are distance at least n, or at distances n and n + 1. The dynamic transition separates an underconstrained (Replica Symmetric, RS) regime in which C n → 0 as n → ∞, and an overconstrained (Replica Symmetry Breaking, RSB) one in which C n remains strictly positive at large distances. To compute C n we first remark that the local neighborhood of the vertex 0, up to any finite distance, converges when N → ∞ to a regular tree structure. Moreover the marginal law of µ on any finite neighborhood of G converges, within the hypothesis of the RS solution described in Sec. 7.2, to a measure that admits an explicit description in terms of a broadcast process that we describe below for the two measures µ Θ0 and µ Θ1 .

The broadcast process

Generating a configuration with the law of µ in a finite neighborhood of a root vertex 0 amounts indeed to do a sequential sampling, starting from the root variable σ 0 , which is drawn uniformly at random in {-1, +1}. The value of the variables in the first generation {σ ∂a\0 } ∂0 are then drawn from the conditional law µ({σ ∂a\0 } ∂0 |σ 0 ). In chapter 4 (section 4.1.3) we have seen how to compute the conditional law from the BP messages that are solutions of the BP equations (see equation (4.11)). Then the value of the variables of the next generations are chosen according to the conditional law knowing the value of the previous variables. We first specify this broadcast process for the measure µ Θ0 representing the intra-clause biased measure, then describe it for the measure µ Θ1 with interactions at distance 1, in the k-uniform l + 1-regular graph ensemble.

Broadcast process for the measure with intra-clause bias µ Θ0

In this case we have seen that the solution of the RS equation (7.28,7.29) is

P RS (h) = δ(h), P RS (u) = δ(u).
On a finite set of variables the measure µ Θ0 is then described (within the RS assumption) by the set of messages {h i→a = 0, u a→i = 0} for (i, a) in the subgraph containing the subset of variables. One can then use this solution to sample a configuration from µ Θ0 , using the following broadcast process. Choose σ 0 to ±1 with probability 1/2. Then, independently for each edge a adjacent to the root, draw the configuration of the k -1 other variables with the conditional probability p(σ ∂a\0 |σ 0 ), where Once the values of all the spins at distance 1 from the root have been set in this way the same process can be iterated, each of these vertices of the first generation being in turn considered as the root of the subtree lying below it. On figure 7.2 is represented a hypertree with a root on the top and two generations of vertices below it, on which the broadcast process is performed.

p(σ 1 , . . . , σ k-1 |σ) = ω(σ 1 , . . . , σ k-1 , σ) σ 1 ,...,σ k-1 ω(σ 1 , . . . , σ k-1 , σ) . ( 7

Broadcast process for the bias at distance 1

In this case we have seen that the solution of the RS equation (7.28,7.29) is

P RS (η) = δ[η -η * ], P( η) = δ[ η -η * ],
where η * (v), η * (v) are solution of the equation (7.36-7.38). The broadcast process takes the following form:

• choose σ 0 = ±1 with equal probability 1/2.

• draw the l + 1 variables v 1 = (σ 1 , w 1 ), . . . , v l+1 = (σ l+1 , w l+1 ) adjacent to the root with the probability

p 0 (v 1 , . . . , v l+1 |σ 0 ) = ψ l+1 i=1 w i l+1 i=1 η * (w i ) w 1 ,...w l+1 ψ l+1 i=1 w i l+1 i=1 η * (w i ) l+1 i=1 I[σ i = σ 0 ] . (7.45)
• consider each of the v 1 , . . . , v l+1 variables of the first generation as the root of the subtree lying below it, and draw the value of the descendents v 1 , . . . v k-1 of a variable equal to v from the conditional law

p(v 1 , . . . , v k-1 |v) = ω(σ, σ 1 . . . σ k-1 )I[w = I[{σ i } a.e.]] k-1 i=1 η * (w i )I[w i = I[σ, {σ j } j =i a.e.]] {σ i ,w i } ω(σ, σ 1 . . . σ k-1 )I[w = I[{σ i } a.e.]] k-1 i=1 η * (w i )I[w i = I[σ, {σ j } j =i a.e.]]
.

(7.46)

• consider again the variables of the second generation as roots, and extract the value of their descendents from the conditional law

p(v 1 , . . . , v l |v) = ψ w + l i=1 w i l i=1 η * (w i ) w 1 ,...w l ψ w + l i=1 w i l i=1 η * (w i ) l i=1 I[σ i = σ] . (7.47)
• iterate the last two steps until all the variables in the target neighborhood have been assigned.

On figure 7.3 is represented the hypertree with the root 0 at the top on which lives the root variable σ 0 , and three generation of variables below it. This broadcast procedure, that must be performed on the v variables and not only on the σ's to preserve the Markov structure of the tree, can be interpreted as the transmission of an information (the value σ 0 at the root) through noisy channels (the conditional laws p 0 , p and p defined in (7.45,7.46,7.47)), that we will describe below.

The reconstruction problem and its recursive distributional equations

This description of the sampling from µ Θ as a broadcast process naturally calls for an interpretation as the transmission of an information, the value of the spin at the root, through noisy channels, towards a set of receivers, the vertices at a certain distance n from the root, to be denoted B n . For the measure µ Θ0 the noisy channels are represented by the hyperedges and correspond to the conditional probability expressed in (7.44). For the measure µ Θ1 the noisy channels are the conditional laws p 0 , p and p defined in (7.45,7.46,7.47) In this information theoretic perspective the tree reconstruction problem asks the following question: given only the values σ Bn of the spins at distance n from the root in a configuration σ generated as above, how much information is available on the value of the root σ 0 , in the sense that the observation of σ Bn allows to infer the value of the root σ 0 with a success probability larger than the one expected from a random guess ? Does a non-vanishing amount of information on σ 0 survives in the n → ∞ limit ? In this Bayesian setting the information-theoretical optimal strategy of an observer having to reconstruct σ 0 from σ Bn is to compute

σ 0 p 0 p p Figure 7.3:
The tree structure considered for the computation of C n , represented here for k = l + 1 = 3. The generation of a configuration from the law ρ is performed in a broadcast fashion, the root σ 0 being ±1 with probability 1/2, this information is then propagated down the tree with transmission channels p 0 , p and p.

the posterior probability µ Θ (σ 0 |σ Bn ), which for Ising variables is completely described by the magnetization σ 0 σ Bn . The correlation function C n is a possible way of quantifying this amount of information, the tree reconstruction problem being solvable if and only if C n remains strictly positive in the large n limit. Thanks to the tree structure of the interaction graph this strategy is actually computationally feasible and can be performed in a recursive way, that we now specify for the two measures µ Θ0 , µ Θ1 .

Reconstruction for the bias with intra-clause interactions

Once supplemented by the boundary conditions

h i→a = σ i for i ∈ B n , ( 7.48)
the BP equations (7.7) admit a unique solution for any finite tree, and the posterior probabilities µ Θ0 (σ i |σ Bn ) can be determined, for i / ∈ B n , from these messages according to

µ Θ0 (σ i |σ Bn ) = a∈∂i 1 + u a→i σ i σ a∈∂i 1 + u a→i σ , ( 7.49)
note that in order to lighten the notations we have kept implicit the dependency of the messages on the observed variables σ Bn . The average value of the spin at the root, in the posterior probability measure conditional on the observations of the spins at distance n, reads

σ 0 Bn = h 0 = σ0 σ 0 µ Θ0 (σ 0 |σ Bn ) = f Θ0 ({u a→0 } a∈∂0 ) . ( 7.50)
(with f Θ0 defined in the equation (7.8)). It is this value which has to be compared with the actual value of σ 0 in the broadcast process that led to σ Bn in order to assess the amount of information transmitted between 0 and B n .

Once averaged over µ Θ0 (σ) this yields the point-to-set correlation function C(0, B n ) = σ 0 σ 0 Bn . The computation descrived above was performed for a given tree. We now want to take the average E over the graph ensemble. We recall that the finite set B n converges in distribution to a tree drawn as an instance of a Galton-Watson process with n generations, in which every vertex has a random offspring (number of descendent hyperedges) distributed as a Poisson law of parameter αk (we consider here the ER ensemble G N (k, αN )). Let us call as above h ∈ [-1, 1] the posterior magnetization of the root given the observations of the spins on the vertices of the n-th generation (the root corresponding to the generation n = 0). h is a random variable because of the randomness in the generation of the Galton-Watson tree on the one hand, and in the broadcast process yielding σ Bn on the other hand. We shall denote P σ,n (h) its distribution when the broadcast is conditioned to the root being equal to σ. As both the broadcast process (from the root downwards) and the resolution of the BP equations (from the leaves upwards) decompose recursively along the branches of the tree, it is possible to obtain from the above analysis an inductive formula relating these distributions for trees of depth n and n + 1, namely

P σ,n+1 (h) = ∞ l=0 e -αk (αk) l l! l i=1 d P σ,n (u i ) δ(h -f Θ0 (u 1 , . . . , u l )) , ( 7.51
)

P σ,n (u) = σ1,...,σ k-1 p(σ 1 , . . . , σ k-1 |σ) k-1 i=1 dP σi,n (h i ) δ(u -f Θ0 (h 1 , . . . , h k-1 )) , (7.52)
with the initial condition P σ,0 (h) = δ(h-σ) that expresses the observation of the variables at the boundary of the tree. These are examples of Recursive Distributional Equations (RDEs), as they define by recursion a sequence of probability distributions. They can be equivalently written as equalities in distribution between random variables, for instance (7.51) means h d = f Θ0 (u 1 , . . . , u l ) with h drawn from P σ,n+1 , l is a Poisson distributed random variable with mean αk, and the u i 's are i.i.d. copies of a random variable of law P σ,n .

One can slightly simplify these equations by noticing that the invariance of the bicoloring problem under a global spin-flip of all its variables implies that P -σ,n (h) = P σ,n (-h) and P -σ,n (u) = P σ,n (-u); this can be checked by induction from (7.51,7.52), using p(σ 1 , . . . , σ k-1 |σ) = p(-σ 1 , . . . , -σ k-1 | -σ) and the fact that f Θ0 and f Θ0 change sign when all their arguments are multiplied by -1. It is thus redundant to track the evolution with n of the distributions with both σ = +1 and σ = -1, we shall instead define P n (h) = P +1,n (h) and P n (u) = P +1,n (u) and close the equations on these two sequences of distributions:

P n+1 (h) = ∞ l=0 e -αk (αk) l l! l i=1 d P n (u i ) δ(h -f Θ0 (u 1 , . . . , u l )) , (7.53
)

P n (u) = σ1,...,σ k-1 p(σ 1 , . . . , σ k-1 | + 1) k-1 i=1 dP n (h i ) δ(u -f Θ0 (h 1 σ 1 , . . . , h k-1 σ k-1 )) . (7.
54)

The answer to the reconstructibility question raised above can be read off from the behavior of P n (h) in the large n limit: if it tends to the trivial distribution δ(h) (which is always a stationary solution of (7.53,7.54)), then all the information on the value of the root has been washed out and the reconstruction problem is not solvable. On the contrary if the limit of P n (h) is non-trivial then the observation of σ Bn contains some information on the root and the problem is said to be solvable. The occurence of these two situations depend on the parameters k, α and {w p } of the model; increasing α gives rise to a larger number of spin variables to be observed, which makes the inference problem easier. Hence it is natural to expect the existence of a threshold α d (k, {w p }) such that the problem is unsolvable (resp. solvable) for α < α d (resp. α > α d ). We will see in the section 7.6.2 that this threshold coincides with the clustering threshold for the random bicoloring on k-hypergraphs. Indeed, the cavity formalism leads to the same distributional equations as equations (7.51,7.52). More precisely, in the cavity framework they corresponds to the 1RSB equations with Parisi parameter X = 1 [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF].

It is actually possible to describe this transition by the point-to-set correlation function, which constitute a scalar order parameter (instead of the functional one P n ). It can be expressed as follows

C n (α, k, {w p }) = σ 1 2 dP σ,n (h)σh . (7.55)
Using the symmetry properties described above this expression can be simplified:

C n (α, k, {w p }) = dP n (h) h . (7.56)
The solvability threshold is determined by the large n limit of the point-to-set correlation:

C(α, k, {w p }) = lim n→∞ C n (α, k, {w p }) . (7.57)
This is indeed an order parameter for the transition in the sense that C(α, k, {w p }) > 0 if and only if α > α d (k, {w p }). This equivalence is a consequence of some symmetry properties of the distributions P n that we will describe now.

Let us call T n (h) the distribution of the posterior magnetization h of the root, in a broadcast process which is not conditioned on the value of the root in σ, i.e. T n (h) = (P +,n (h) + P -,n (h))/2. A consequence of Bayes theorem applied to the joint law between the spins at the root and on the boundary vertices is the following converse relation between conditional and unconditional laws, P σ,n (h) = (1 + σh)T n (h) (see [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Richardson | Modern Coding Theory[END_REF] for more details on this property and its consequences). In addition the invariance of the problem under a global spin-flip implies that T n (-h) = T n (h), which yields a symmetry constraint on the distributions P n ,

P n (-h) = 1 -h 1 + h P n (h) , dP n (h) b(h) = dP n (h) b(-h) 1 -h 1 + h , (7.58)
the second equality being valid for any function b(h) such that the integrals exist. This symmetry (sometimes called Nishimori symmetry [START_REF] Nishimori | Statistical Physics of Spin Glasses and Information Processing: An Introduction[END_REF], and also fulfilled by P n (u)) implies several identities between the moments of h that can be derived by appropriate choices of the test function b [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Richardson | Modern Coding Theory[END_REF]; here we shall only state the simplest one, namely

C n (α, k, {w p }) = dP n (h) h = dP n (h) h 2 , ( 7.59)
that can be obtained from (7.58) with b(h) = h(1 -h). This is enough to justify our statement of the characterization of the reconstruction transition via the behavior of C n : if the latter vanishes in the n → ∞ limit this implies that both the average and the variance of P n go to zero, hence P n tends to the trivial distribution δ(h).

Reconstruction for the bias with interactions at distance 1

We now give the explicit derivation of the posterior probability µ Θ1 (σ 0 |σ Bn ), adapting the computations explained in the previous section. In this case we look at the k-uniform l + 1-regular graph ensemble. The conditional measure µ Θ1 (•|σ Bn ) is exactly described in terms of the solution of the BP equations (7.17), supplemented with the boundary condition η i→a (v) = δ v,v (i,a) on the edges at distance larger than n of the root, v (i,a) being the value taken by the variable during the broadcast. These BP messages, directed towards the root, are thus random variables because of the randomness in the boundary condition σ Bn . One can nevertheless write recursion equations on their distributions, their law depending only on their distance from the boundary, as it was done in the previous section. We shall denote P v,n (η) the law of the message η on an edge at distance n from the boundary, conditional on the value of the variable on this edge being v in the broadcast, and similarly P v,n ( η) for the law of the η messages. Putting together all the above observations leads to the following recursion equations:

P v,n+1 (η) = v1,...,v l p(v 1 , . . . , v l |v) l i=1 d P vi,n+1 ( η i ) δ(η -f Θ1 ( η 1 , . . . , η l )) , (7.60) P v,n+1 ( η) = v1,...,v k-1 p(v 1 , . . . , v k-1 |v) k-1 i=1 dP vi,n (η i ) δ( η -f Θ1 (η 1 , . . . , η k-1 )) , (7.61)
where f Θ1 , f Θ1 , p and p have been defined in (7.17,7.46,7.47), respectively, and with the initial condition for n = 0:

P v,0 ( η) = δ( η(•) -δ v,• ) . (7.62)
The point-to-set correlation function is then computed as for the conditional average magnetization of the root. The recursion equations (7.60,7.61), (which are equivalent to the 1RSB equations with Parisi breaking parameter equal to X = 1), always admit the trivial solution P v (η) = δ(η -η * ), P v ( η) = δ( η -η * ) as a stationary fixed point. In the non-reconstructible (RS) phase this is the limit reached by P v,n and P v,n in the large n limit, and then C n → 0. On the contrary in the reconstructible (RSB) phase the limit of P v,n and P v,n is a non-trivial fixed point, and C n remains strictly positive. For a given choice of the parameters k and ψ we define the dynamic transition l d as the threshold separating these two behaviors. As l is here an integer parameter we will say more precisely that l < l d is the RS phase, l ≥ l d the RSB phase, i.e. l d (k, ψ) is the smallest integer value of l such that RSB occurs.

C n = 1 2 σ0 v1,...,v l+1 p 0 (v 1 , . . . , v l+1 |σ 0 ) l+1 i=1 d P vi,n ( η i ) σ 0 m( η 1 , . . . , η l+1 ) , (7.63

Simplifications and symmetries

The recursion equations (7.60,7.61) bear, for each value of n, on eight distributions P v,n , P v,n , as the variable v = (σ, w) takes four different values. This 109 number can however be divided by two thanks to the invariance of the problem under the spin-flip symmetry σ → -σ. To state its consequences let us define the flip transformation η → η f between messages, according to η f (σ, w) = η(-σ, w) (and similarly η f (σ, w) = η(-σ, w)). The channels p and p being invariant under a global spin-flip, one can check that P (-,w),n (η) = P (+,w),n (η f ) , P (-,w),n ( η) = P (+,w),n ( η f ) , (7.66) which allows to close (7.60,7.61) on the four distributions {P (+,w),n , P (+,w),n } w=0,1 , that we shall denote for simplicity {P w,n , P w,n } w=0,1 . Using this property, as well as the invariance of f Θ1 , f Θ1 under a permutation of their arguments and a more explicit version of the expressions (7.46,7.47) of p and p, one can simplify (7.60,7.61) into:

P w,n+1 (η) = l p=0 l p ψ(p + w) y -p l p =0 l p ψ(p + w) y -p p i=1 d P 1,n+1 ( η i ) × l i=p+1 d P 0,n+1 ( η i ) δ(η -f Θ1 ( η 1 , . . . , η l )) , (7.67
)

P 1,n+1 ( η) = k-1 i=1 dP 0,n (η i ) δ( η -f Θ1 (η f 1 , . . . , η f k-1 )) , (7.68)
P 0,n+1 ( η) = k -1 y y dP 1,n (η 1 ) k-1 i=2 dP 0,n (η i ) δ( η -f Θ1 (η f 1 , η 2 , . . . , η k-1 )) (7.69) + 1 y k-2 t=2 k -1 t k-1 i=1 dP 0,n (η i ) δ( η -f Θ1 (η f 1 . . . η f t , η p+1 , . . . , η k-1 )) .
The expression (7.63) of the correlation function C n can similarly be rewritten as:

C n = l+1 p=0 l+1 p ψ(p) y -p l+1 p =0 l+1 p ψ(p ) y -p p i=1 d P 1,n ( η i ) l+1 i=p+1 d P 0,n ( η i ) m( η 1 , . . . , η l+1 ) .
(7.70) As already mentioned in the previous section, a further symmetry constrains the distributions P v,n ; let T n (η) the distribution of η in a broadcast process wich is not conditioned on the value of the root, i.e.:

P n (η) = v=(σ,w) η * (w)P v,n (η) , (7.71)
where η * is normalized in such a way that η * (0) + η * (1) = 1/2. Applying Bayes theorem to express the joint law of the variable at the root and those at the boundary one obtains [START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF] P v,n (η) = η(v) η * (w) P n (η) . (7.72) This yields a relation between P w,n = P (+,w),n for the two values of w, namely

P 1,n (η) = y η(+, 1) η(+, 0) P 0,n (η) , ( 7.73)
where we recall that y = η * (0)/η * (1) was defined in Eq. (7.39). Moreover the spin-flip symmetry implies the invariance of P n , i.e. P n (η) = P n (η f ). This property, combined with (7.72), allows to relate P w,n in η and η f through a change of density, namely

P 0,n (η f ) = η(-, 0) η(+, 0) P 0,n (η) , P 1,n (η f ) = η(-, 1) η(+, 1) P 1,n (η) . (7.74)
These symmetry relations, as well as the similar ones that hold for P w,n modulo the replacement of y by y in (7.73), will be particularly useful in the treatment of the large k limit presented in chapter 10. They imply a variety of identities between average observables, and in particular they can be used to rewrite the correlation function as

C n =
( η i ) l+1 i=p+1 d P 0,n ( η i ) A( η 1 , . . . , η l+1 ) (7.76) = l+1 p=0 l + 1 p ψ(p) y -p p i=1 d P 1,n ( η i ) l+1 i=p+1 d P 0,n ( η i ) m -( η 1 , . . . , η l+1 ) m + ( η 1 , . . . , η l+1 ) A( η f 1 , . . . , η f l+1 ) (7.77)
for an arbitrary function A which is invariant under the permutation of its arguments, and such that the integrals are well-defined, and then applying this identity with the test function A = m(1 -m).

Hard fields and the naive reconstruction

The tree reconstruction problem considered above asks whether the observation of σ Bn gives some information on the value of the root σ 0 , as quantified by the correlation function C n . Answering this question requires to solve the functional recursion relations (7.53,7.54 for µ Θ0 , and (7.67-7.68) for µ Θ1 ). The strategy is based on the optimal inference algorithm, namely the computation of the posterior probability µ Θ (σ i |σ Bn ) via the BP equations ((7.7) for µ Θ0 ,(7.17) for µ Θ1 ).

We shall now consider a more drastic question, namely whether σ Bn allows to infer σ 0 with perfect certainty, i.e. is σ i uniquely determined in the measure µ Θ (•|σ Bn ), or in other words is σ i constant in all the proper bicolorings of the graphs that take the values σ Bn on the boundary vertices? This corresponds to a projection of the BP algorithm towards its Warning Propagation [START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF] The computation of H n requires to project the BP equations on the WP equations, and to compute from the distribution over the BP messages (denoted P n (h) for µ Θ0 , and P w,n (η) for µ Θ1 ) the probability of having a hard message. We will precise the definition of hard message in the case µ Θ1 . To explain the computation of H n let us first remark that σ 0 is implied by σ Bn if and only if all the proper bicolorings of the tree that coincides with σ Bn on the boundary take the same value at the root. By definition we only consider biased measures that do not strictly forbid any solution (ω p > 0 for p ∈ {1, . . . , k -1} for µ Θ0 , and ψ(p) > 0 for all p for µ Θ1 ), hence the certain determination of σ 0 can only arise from the bicoloring constraints acting on the spin variables. This observation can be turned into an algorithm, called the naive reconstruction procedure: consider all the hyperedges at the boundary, and declare them "forcing to the value σ" if their k -1 variables at distance n from the root are all equal to -σ, and "not forcing" otherwise. Now the variables at distance n-1 are assigned the value σ if at least one of their incident boundary hyperedge is forcing to this value (by construction of the broadcast process there cannot be conflicting forcings to + andon the same variable), and a "white" value 0 if all the hyperedges are not forcing. This process can be iterated from the boundary towards the root, hyperedges being forcing if and only if k -1 among their variables have been assigned the same value +1 or -1. H n is thus the probability that this successive forcing mechanism percolates from the boundary to the root, with at least one of its incident hyperedge forcing it.

Intra-clause bias

To incorporate naturally this computation in the one presented in the section 7.3.2 we decompose the field distributions as follows:

P n (h) = H n δ(h-1)+(1-H n ) Q n (h) , P n (u) = H n δ(u-1)+(1-H n ) Q n (u) , ( 7
.78) where Q n and Q n are probability measures with no atom in 1. H n is the weight of the "hard field" h = 1, that constrain completely the variable at the root and makes it a frozen variable under the boundary condition σ Bn . One also says that h = 1 is forcing the value of the root to σ = 1. H n is precisely the probability of success of the naive reconstruction procedure. In the following we shall call Q n and Q n the distributions of soft fields; note that they do not contain atoms in -1 (because a variable σ i cannot be forced by σ Bn to another value that it had in the broadcast), and that they enjoy the same Bayes symmetry (7.58) as the complete distributions P n and P n . Inserting this decomposition in (7.53,7.54), and considering the possible combinations of arguments of the functions f Θ0 , f Θ0 in (7.7) that yields h, u = 1 one easily obtains the following evolution equations for H n and H n :

H n+1 = 1 -e -αk Hn , H n = p(-, . . . , -|+)H k-1 n . ( 7.79)
The first one expresses the fact that a spin is perfectly recovered as soon as one of its neighboring interactions forces it to its correct value, while the second one shows that this latter event happens when in the broadcast the k -1 variables adjacent to it have been given the same value, and that they all have been perfectly recovered.

Assembling these two equations we obtain a simple recursive equation on H n ,

H n+1 = 1 -e -Γ(α,k,{wp})H k-1 n , Γ(α, k, {w p }) = αk p(-, . . . , -|+) , (7.80)
with the initial condition H 0 = 1. By a numerical inspection of the shape of this recursion function one easily realizes that for k ≥ 3 the fixed point reached by H n for n → ∞ undergoes a discontinuous bifurcation from zero to a strictly positive value when Γ crosses a critical value Γ r = Γ r (k). The latter can be determined by noting that at such a bifurcation the derivative of the recursion function must be equal to 1, hence that Γ r and H r , the fixed point at the bifurcation, are solutions of

H r = 1 -e -ΓrH k-1 r , 1 = (k -1)H k-2 r Γ r e -ΓrH k-1 r . (7.81)
One can close these two equations on a single one that determines

H r : 1 = (k-1) ln(1-H r )(1-1/H r ), from which one obtain Γ r (k) = -ln(1-H r )/(H k-1 r ),
and in terms of the original parameter α this "rigidity" transition occurs at

α r (k, {w p }) = Γ r k p(-, . . . , -|+) = 1 k Γ r (k) k-1 p=1 k-1 p ω p ω 1 = 1 k Γ r (k) k-1 p=1 k p ω p 2ω 1 . (7.82)
We shall denote α r,u the value of this threshold for the uniform case

ω 1 = • • • = ω k-1 = 1, i.
e. when all proper bicolorings are weighted equally, in such a way that α r,u (k

) = 1 k Γ r (k)(2 k-1 -1) . (7.83)
From the intuitive interpretation of this computation as the analysis of a suboptimal reconstruction algorithm it is clear that α r should be an upper bound on α d , and indeed if H = lim n H n is strictly positive then P n cannot converge to δ(h). The inequality α d ≤ α r is in general strict, i.e. there can be non-trivial solution of the equations (7.53,7.54) that do not contain any hardfield; this has been seen numerically in many problems, and proven rigorously for the graph q-coloring problem in the large q limit in [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF]. Moreover this probability H of perfect reconstruction is a lower bound on the correlation function C, as for any value of n the fraction of hard fields H n is a lower bound on C n :

C n (α, k) = dP n (h)h = H n + (1 -H n ) dQ n (h)h (7.84)
and dQ n (h)h = dQ n (h)h 2 ≥ 0 , (7.85) because of the consequence stated in (7.59) of the Bayes symmetry, that is also enjoyed by Q n . An illustration of the bound H(α, k) ≤ C(α, k) can be found for k = 5 in the bottom panel of Fig. 9.2. We can see on this figure that for finite k this bound, as well as its consequence α d (k) ≤ α r (k), are not tight. There is an intermediate regime α d < α < α r where reconstruction is possible but naive reconstruction is not, all the relevant information on the value of the root is asymptotically contained in the soft fields distribution. Another view on this phenomenon is given in Fig. 7.4 where one sees the fraction H n of hard fields fall to zero as n grows while C n remains at a positive plateau value. Also when α > α r one can see on the bottom panel of Fig. 9.2 that C > H for k = 5, i.e. the soft fields do bring some additional information on the value of the root even in the presence of hard fields. Let us also underline that among all the parameters {ω p } that define the bias among proper bicolorings ω 1 = ω k-1 plays a special role in the expression (7.82) of the rigidity threshold. Indeed hard fields are propagated along constraints that are "almost violated", in the sense that they contain a single variable of a given color. Penalizing such "almost monochromatic" hyperedges tends thus to avoid the percolation of frozen variables.

Bias with interactions at distance 1

To embed the analysis of this naive reconstruction algorithm into the formalism defined above for the measure µ Θ1 we first introduce some terminology to classify the messages η, η; we will say that • η is forcing to σ = ±1 iff η(-σ, 0) = η(-σ, 1) = 0, η(σ, 0) > 0 and η(σ, 1) > 0.

• η is non-forcing iff η(σ, w) > 0 for all σ and w.

• η is forcing to σ = ±1 iff η(σ, 1) = 1, η(σ, 0) = η(-σ, 0) = η(-σ, 1) = 0; we write then η = η σ .

• η is non-forcing iff η(+, 0) + η(+, 1) > 0 and η(-, 0) + η(-, 1) > 0.

We will also use the term hard (resp. soft) field for the forcing (resp. nonforcing) BP messages. Inserting these definitions in the BP equations (7.17) one can check the combination rules argued for above: η = f ( η 1 , . . . , η l ) is forcing to σ iff at least one η i is forcing to σ and none forcing to -σ, η is nonforcing otherwise. Similarly η = g(η 1 , . . . , η k-1 ) is forcing to σ iff all the η i are forcing to -σ, and non-forcing otherwise.

We decompose now the distributions P w,n , P w,n between the contributions of the hard and of the soft fields, defining

P w,n (η) = h w,n R w,n (η) + (1 -h w,n ) Q w,n (η) , (7.86) P w,n ( η) = h w,n δ( η -η + ) + (1 -h w,n ) Q w,n ( η) , ( 7.87)
where h, h ∈ [0, 1] are the total weights of hard fields in the corresponding distributions, the R are normalized distributions on η's forcing to +, and Q and Q are probability laws supported on non-forcing messages. By construction there are no messages forcing toin P w,n = P (+,w),n . Inserting these decompositions in the recursion equations (7.67-7.69) we see that the evolution of the hard fields weights decouple; in particular from (7.68) we obtain h 1,n+1 = (h 0,n ) k-1 and from (7.69) h 0,n+1 = 0, we shall thus write more simply h n instead of h 1,n . The equation (7.67) yields

h w,n+1 = 1 - l p=0 l p ψ(p + w) y -p (1 -h n+1 ) p l p=0 l p ψ(p + w) y -p , ( 7.88)
the recursion can thus be closed on h 0,n and h n , and solved starting from the initial condition h 0,0 = 1. Finally H n can be read off from the expression (7.70) of C n by isolating the contribution with at least one forcing field η around the root, which gives

H n = 1 - l+1 p=0 l+1 p ψ(p) y -p (1 -h n ) p l+1 p=0 l+1 p ψ(p) y -p . ( 7.89)
Depending on the choice of the parameters (l, k, ψ) the sequence h 0,n (or equivalently H n ) either decays to 0 or to a strictly positive fixed point. The so-called rigidity threshold l r (k, ψ) introduced in the previous section separates these two behaviors: H n → 0 when l < l r whereas it remains strictly positive in the large n limit for l ≥ l r . In this latter case there is a positive probability for the observation of a far away boundary to completely determine the root (the naive reconstruction problem is solvable), hence there is certainly information about the value of the root (the usual reconstruction problem is also solvable).

The correlation function C n can also be decomposed from (7.70) as the sum of H n and a soft contribution:

C n = H n + l+1 p=0 l+1 p ψ(p) y -p (1 -h n ) p l+1 p =0 l+1 p ψ(p ) y -p p i=1 d Q 1,n ( η i ) l+1 i=p+1 d Q 0,n ( η i ) m( η 1 , . . . , η l+1 ) .
(7.90) Exploiting the symmetry relations (7.73,7.74) one can rewrite the second term in this equation with the integrand squared, exactly as we did in the expression (7.75) of C n , which proves the bound C n ≥ H n and confirms the intuition that the reconstruction problem is solvable if the naive reconstruction is.

The distribution of soft-fields

Uniform measure

We will now derive the recursion equations on the distribution of the soft fields Q n and Q n introduced in (7.78), that complete the equations (7.79) for the evolution of the weights H n and H n of hard fields. Both are obtained by plugging the decomposition (7.78) into the recursion equations (7.53,7.54) on P n and P n . We will focus on the uniform distriution (obtained by setting ω 0 = ω k = 0, and

ω 1 = • • • = ω k-1 = 1 in equation (6.8)).
The recursion equations that we will obtain in this section will be used in the large k analysis of the clustering threshold for the uniform measure, presented in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] (see chapter 9).

As f Θ0 (u 1 , . . . , u l ) = 1 as soon as one of the arguments is equal to 1, i.e. as soon as one of the neighboring interactions forces the root variable, it is easy to see from (7.53) that the soft part of P n arises from the combination of only soft u's, namely

Q n+1 (h) = ∞ l=0 e -αk(1-Hn) (αk(1 -H n )) l l! l i=1 d Q n (u i ) δ(h -f Θ0 (u 1 , . . . , u l )) .
(7.91) The treatment of (7.54) requires a little bit more work in order to put the resulting equation into the convenient form used in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF]. Denoting p the number of soft fields picked in the r.h.s. of (7.54), we first rewrite this equation as

P n (u) = k-1 p=0 k -1 p H k-1-p n (1 -H n ) p σ1...σ k-1 p(σ 1 . . . σ k-1 |+)
dQ n (h 1 ) . . . dQ n (h p ) δ(u -g(σ 1 h 1 . . . σ p h p , σ p+1 . . . , σ k-1 )) . (7.92)

As explained above the term δ(u -1) arises solely from the term p = 0, (σ 1 . . . σ k-1 ) = (-, . . . , -). Moreover one realizes by inspection of the expression of f Θ0 in (7.10) that u = 0 as soon as among the hard fields (σ p+1 . . . , σ k-1 ) at least one is positive and at least one is negative; this expresses the fact that in such a situation the bicoloring condition is satisfied whatever the value of the spin at the root. The initial condition for this evolution of the soft fields distribution turns out to be Q 1 (h) = δ(h), as can be easily realized by an explicit computation of P 0 (u) and P 1 (h) starting from P 0 (h) = δ(h -1).

Combining the expression (7.44) of the broadcasting probability p(σ 1 . . . σ k-1 |+) with the examination of the possibles outcomes of (7.10), one obtains after a short computation the following expression for the soft part of P n ,

(1 -H n ) Q n (u) = 2 k-1 2 k-1 -1 1 + (1 -H n ) k-1 -2 1 - H n 2 k-1 δ(u) + k-2 p=1 k -1 p (1 -H n ) p H k-1-p n p i=1 dQ n (h i ) 1 2 k-1 -1 σ1,...,σp δ(u -g p (σ 1 h 1 , . . . , σ p h p )) + k-2 p=1 k -1 p (1 -H n ) p H k-1-p n p i=1 dQ n (h i ) 1 2 k-1 -1 σ1,...,σp δ(u + g p (σ 1 h 1 , . . . , σ p h p )) + (1 -H n ) k-1 2 k-1 -1 σ1,...,σ k-1 k-1 i=1 dQ n (h i )δ(u -g(σ 1 h 1 , . . . , σ k-1 h k-1
)) , (7.93) where the primed sums exclude the configuration with all the spin arguments equal to +1, and where g p is defined as

g p (h 1 , . . . , h p ) = -g(h 1 , . . . , h p , +1, . . . , +1) = p i=1 1+hi 2 2 - p i=1 1+hi 2
.

(7.94)

Bias with interaction at distance 1

We also give here the recursion equation for the soft fields distributions, obtained by inserting the decompositions (7.86,7.87) into (7.67-7.69):

Q w,n+1 (η) = l p=0 l p ψ(p + w) y -p (1 -h n+1 ) p l p =0 l p ψ(p + w) y -p (1 -h n+1 ) p p i=1 d Q 1,n+1 ( η i ) × l i=p+1 d Q 0,n+1 ( η i ) δ(η -f Θ1 ( η 1 , . . . , η l )) , (7.95) Q 1,n+1 ( η) = k-1 u=1 k-1 u (h 0,n ) k-1-u (1 -h 0,n ) u 1 -(h 0,n ) k-1 u i=1 dQ 0,n (η i ) × k-1 i=u+1 dR 0,n (η i ) δ( η -f Θ1 (η f 1 , . . . , η f k-1 )) (7.96) Q 0,n+1 ( η) = P 0,n+1 ( η) . (7.97)
These recursive equations will be used in chapter 10, that deals with the asymptotic expansion of the clustering threshold for biased measures, and that gathers the results obtained in the paper [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF]. The last equation comes from the absence of hard fields in P 0 , one can thus take the expression (7.69) and insert in its right hand side the decomposition (7.86) for P 1 and P 0 to have an equation involving only the soft fields distributions; this is relatively cumbersome notationally in general, we shall only write the corresponding equation in a special case in chapter 10.

1RSB formalism

We now introduce the 1RSB formalism, and explain the connection between the reconstruction problem on trees and the 1RSB cavity equations at X = 1. The hypothesis underlying the RS cavity method must break down when the density of interactions per variable α becomes too large; a first hint of this phenomenon, called Replica Symmetry Breaking (RSB), is the negativity of the RS entropy at large enough α, which is impossible for a system with discrete degrees of freedom. In chapter 2, we have introduced the RSB phenomenon, and related it to the clustering of the solution set (see sections 2.3.1 and 2.3.2). As a matter of fact RSB can occur before α s=0 ; increasing α above a certain threshold causes the appearance of the long-range point-to-set correlation between distant variables under the measure µ, which contradicts the RS hypothesis. The point-to-set correlation function has been defined in equation (2.15), and has been estimated in the section 7.3.2 in the framework of the tree reconstruction problem. When this long-range correlation appears it becomes necessary to use more refined versions of the cavity method, that are able to deal with this RSB phenomenon [START_REF] Mézard | The bethe lattice spin glass revisited[END_REF]. At its first non-trivial level, called 1RSB for one step of RSB, the cavity method postulates the existence of a partition of the configuration space {-1, 1} N into "pure states", or clusters, such that the restriction of the measure µ to a pure state has good decorrelation properties. This restricted measure can then be treated as the full measure in the RS cavity method, i.e. with BP equations to describe its marginal probabilities, and the Bethe freeentropy to compute its partition function.

To be more quantitative let us index with c the partition of the configuration space into clusters, and denote Z(c) the contribution to the partition function of the c-th cluster, as it was done in the section 2.3.2 (see equation (2.16)). We also denote { η c a→i , η c i→a } the solution of the BP equations that describe it. The 1RSB cavity method aims at computing the potential (2.20) that we recall here:

Φ 1 (X ) = lim N →∞ 1 N ln c (Z(c)) X , ( 7.98)
where the Parisi parameter X allows to weight differently the various pure states, according to their relative weights. This quantity contains precious informations about the pure-state decomposition; suppose indeed that, at the leading exponential order, there are e N Σ(φ) pure states c with Z(c) = e N φ (again neglecting sub-exponential corrections). The complexity Σ(φ) plays thus the role of an entropy density, with pure states replacing usual configurations, and captures the RSB phenomenon quantitatively. The potential Φ 1 (X ) and the complexity Σ(φ) are Legendre transforms of each other [START_REF] Monasson | Structural glass transition and the entropy of the metastable states[END_REF]; evaluating (7.98) via the Laplace method yields indeed

Φ 1 (X ) = sup φ [Σ(φ) + X φ] , (7.99)
which can be inverted in terms of the conjugated parameter as

Σ(X ) = Φ 1 (X ) -X d dX Φ 1 (X ) .
(7.100)

1RSB cavity equations

In order to compute Φ 1 one introduces, for a given sample and a given edge (i, a) of the factor graph, two distributions P i→a and P a→i , that encode the laws of h c i→a and u c a→i when the pure state c is chosen randomly with a probability proportional to Z(c) X . These distributions are found to obey self-consistent equations of the form

P i→a = F Θ ({ P b→i } b∈∂i\a ) , P a→i = F Θ ({P j→a } j∈∂a\i ) , ( 7.101)
where P = F Θ ( P 1 , . . . , P d ) is a shorthand for

P (η) = 1 z 1,Θ ( P 1 , . . . , P d ) d i=1 d P i ( η i ) δ(η-f Θ ( η 1 , . . . , η d )) z 0,Θ ( η 1 , . . . , η d ) X ,
(7.102) and P = F Θ (P 1 , . . . , P k-1 ) means

P ( η) = 1 z 1,Θ (P 1 , . . . , P k-1 ) k-1 i=1 dP i (η i ) δ( η-f Θ (η 1 , . . . , η k-1 )) z 0,Θ (η 1 , . . . , η k-1 ) X ;
(7.103) the functions f Θ , z 0,Θ , and f Θ , z ,Θ were defined in Eqs. (7.8 -7.11) for the measure µ Θ0 , and (7.17) for the measure µ Θ1 . The factors z 1,Θ and z 1,Θ ensure the normalization of the distributions P (η) and P ( η).

In order to deal with random hypergraphs one introduces the probability distributions over the 1RSB messages P 1RSB (P ) and P 1RSB ( P ) that obeys the consistency relations similar to (7.28,7.29),

P 1RSB (P ) = ∞ d=0 p d d i=1 d P 1RSB ( P i ) δ[P -F Θ ( P 1 , . . . , P d )] , (7.104) P 1RSB ( P ) = k-1 i=1 dP 1RSB (P i ) δ[ P -F Θ (P 1 , . . . , P k-1 )] .
The 1RSB potential for typical random hypergraphs can then be computed from the solution of these equations as with:

Φ 1,Θ (X ) = ∞ d=0 p d d i=1 d P i P 1RSB ( P i ) ln Z v 1,
Z v 1,Θ ( P 1 , . . . , P d ) = d i=1 d P i ( η i ) (Z v 0,Θ ( η 1 , . . . , η d )) X , (7.107) Z c 1,Θ (P 1 , . . . , P k ) = k i=1 dP i (η i ) (Z c 0,Θ (η 1 , . . . , η k )) X , ( 7.108)
Z e 1,Θ (P, P ) = dP (η)d P ( η)(Z e 0,Θ (η, η)) X . (

The expressions of Z v 0,Θ , Z c 0,Θ , Z e 0,Θ have been specified in (7.13-7.15) for µ Θ0 , and in (7.23-7.25) for µ Θ1 . Finally the 1RSB prediction for the free-entropy is given in the equation (2.27) that we recall here:

Φ 1RSB Θ = inf X ∈[0,1] Φ 1,Θ (X ) X . (7.110)
Note that the 1RSB equations always admit the RS solution as a special case, when the distributions P in the support of P 1RSB are Dirac measures. In most models this trivial solution of the 1RSB equations is the only one at small values of α; then Φ 1 (X ) = X Φ RS , and the thermodynamic prediction of the RS and 1RSB versions of the cavity method coincides. Increasing the number of constraints of the system non-trivial solution of the 1RSB equations can appear; the dynamic (clustering) threshold α d (k, Θ) is defined as the smallest value of α for which the 1RSB equations with X = 1 admit a solution distinct from the RS one. A further distinction has then to be made: if the associated complexity Σ(X = 1) is positive the extremum in (7.110) is reached for X = 1 and Φ 1RSB = Φ RS . In such a "dynamic 1RSB" situation the typical configurations of the Gibbs measure are supported on an exponentially large number of pure states, in such a way that the total free-entropy (or any correlation functions between a finite number of variables) is unable to detect the difference with a RS situation. On the contrary when Σ(X = 1) < 0 the extremum in (7.110) selects a nontrivial value X s < 1 of the Parisi parameter, the Gibbs measure condenses on a sub-exponential number of clusters, and correlations between finite sets of variables unveil the RSB phenomenon. One calls condensation threhold α c the smallest value of α for which a solution of the 1RSB equations with Σ(X = 1) < 0 exists, which corresponds to a point of non-analyticity of the free-entropy density. We will give in section 7.6.3 an explicit expression of the complexity Σ(X = 1) for the biased measure with intra-clause bias µ Θ0 .

Simplifications for X = 1

The complete 1RSB equations have a rather intricate structure, as they are self-consistent equations for probability distributions over probability distributions, which make in particular their numerical resolution rather cumbersome. Fortunately for special values of the parameter X (i.e. X = 0 and X = 1) they can be largely simplified. We shall sketch here this simplification procedure for X = 1, which as explained above is the important one for the determination of the dynamic and condensation phase transitions; for further details the reader is referred to [START_REF] Montanari | Clusters of solutions and replica symmetry breaking in random k-satisfiability[END_REF] where the simplifications are explained in more details and in a general setting. We will see that the 1RSB equations at X = 1 coincide with the recursive distributional equations obtained for the tree-reconstruction problem, hence showing that the threshold for solvability and the clustering threshold are equal. We recall that in our settings the solution of the RS equations (7.28,7.29) is a trivial delta function

P RS (η) = δ[η-η * ], P RS ( η) = δ[ η-η * ].
For the measure with a bias that factorises on the clauses (see equation (7.1)), one has seen in section 7.2.2 that η * (σ) = 1/2, η * (σ) = 1/2. For the biased measure with interaction at distance 1, for the k-regular l + 1-uniform graph ensemble, η * (σ), η * (σ) is solution of the equations (7.36-7.38). This will allow for additional simplifications compared to the general setting described in [START_REF] Montanari | Clusters of solutions and replica symmetry breaking in random k-satisfiability[END_REF].

The crucial technical property of the equations that opens the door to simplifications at X = 1 is the fact that, for this value, the normalization constant z 1,Θ in (7.102) does not depend on the whole distributions P 1 , . . . , P d , but only on their average values d P i ( η i ) η i (a similar statement holds for z 1,Θ in (7.103)). More precisely, let

η(P ) = dP (η)η, η( P ) = d P ( η) η. ( 7
η d = f ( η 1 , . . . , η d ); η d = f (η 1 , . . . , η k-1 ) (7.114)
where d is drawn from the probability law r d , and η 1 , . . . , η d are obtained by independenlty drawing P i from P 1RSB , then constructing η i = η( P i ). Similarly to obtain η 1 , . . . , η k-1 , one draws P 1 , . . . , P k-1 i.i.d from P 1RSB , then one sets η i = η(P i ).

Conditional on this average values F is thus a multilinear function of its arguments, the equations (7.104) can then be averaged and closed on the mean distributions Q and Q defined as: T (η) = dP 1RSB (P )P (η) (7.115)

T ( η) = d P 1RSB ( P ) P ( η) , ( 7.116)
These two quantities are solutions of

T (η) = ∞ d=0 p d d i=1 d T ( η i ) δ(η -f ( η 1 , . . . , η d )) z 0,Θ ( η 1 , . . . , η d ) z 0,Θ ( η * , . . . , η * ) , (7.117) T ( η) = k-1 i=1 dT (η i ) δ( η -g(η 1 , . . . , η k-1 )) z 0,Θ (η 1 , . . . , η k-1 ) z 0,Θ (η * , . . . , η * ) . (7.118)
These equations are definitely simpler than the full 1RSB equations, as they bear on probability distributions instead of distributions of distributions; they have however one inconvenient feature, in particular for their numerical resolution, namely the reweighting terms z 0,Θ and z 0,Θ which prevents their direct interpretation as recursive distributional equations. To get around this difficulty we shall define the distributions

P x (η) = η(x) η * (x) T (η), P x ( η) = η(x) η * (x) T ( η) (7.119)
where x is a generic variable (x = σ for µ Θ0 , and x = v for µ Θ1 ). From the equality in distribution η(P ) Thus one can check that the distributions P x , P x are well-normalized, and that

T (η) = x η * (x)P x (η), T ( η) = x η * (x) P x ( η) (7.122)
Inserting the definition of P x , P x in (7.117,7.118) one obtain distributional equations for P x , P x . The particular form of these equation depends on the choice of the biased measure chosen. When one specifies to the biased measure µ Θ0 , one recovers the equations (7.53,7.54). When one specifies to the biased measure µ Θ1 , one recovers the equations (7.67-7.69). Therefore the 1RSB equations at X = 1 coincide with the recursive distributional equations found for the tree-reconstruction problem.

Complexity Σ(X = 1) and condensation threshold

In this section we give the expression of the complexity Σ(X = 1) for the measure µ Θ0 . First, one can check that Φ 1 (X = 1) = Φ RS as given in (7.32).

To compute the complexity at X = 1 from (7.100) we need to take the derivative with respect to X of Φ 1 from (7.98). Because of its variational character ((7.98) is stationary with respect to variations of P 1RSB and P 1RSB as long as the 1RSB cavity equations (7.104) are fulfilled) only the explicit dependency on X has to be differentiated. Doing the simplification at X = 1 yields then an expression in terms of P + and P + :

d dX Φ 1 (X ) X =1 = ∞ d=0 p d d i=1 du i P + (u i ) ln Z v 0 (u 1 , . . . , u d ) (7.123) + α σ1,...,σ k p(σ 1 , . . . , σ k ) k i=1 dh i P + (h i ) ln Z c 0 (σ 1 h 1 , . . . , σ k h k ) (7.124) -αk dhdu P + (h) P + (u) ln Z e 0 (h, u) , with p(σ 1 , . . . , σ k ) = ω(σ 1 , . . . , σ k ) σ 1 ,...,σ k ω(σ 1 , . . . , σ k ) . ( 7

.125)

We recall that the condensation threshold α c is the value above which the complexity at X = 1 becomes negative.

Kesten-Stigum bound

We shall discuss here a bound on α d , known as the Kesten-Stigum [START_REF] Kesten | Additional limit theorems for indecomposable multidimensional galton-watson processes[END_REF][START_REF] Mossel | Information flow on trees[END_REF] transition in the context of the tree reconstruction problem, or as the de Almeida-Thouless [START_REF] De Almeida | Stability of the Sherrington-Kirkpatrick solution of a spin-glass model[END_REF] transition for mean-field spin-glasses, that is tight for continuous bifurcations and that in any case provide an easy to compute analytical upper bound on α d (besides the bound α d < α s=0 and α d ≤ α r we already discussed). The computation we will present have been explained in details in [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] (in App. B) and in [START_REF] Ricci-Tersenghi | Typology of phase transitions in bayesian inference problems[END_REF].

Let us recall that the 1RSB equations (7.104) always admit as a solution the RS solution. This solution is P 1RSB (P ) = δ[P -P triv ], with P triv (h) = δ(h) for the uniform measure and for the measure µ Θ0 with intra-clause bias (and similarly for P 1RSB ( P )). For the biased measure µ Θ1 , one has P triv (η) = δ(ηη * ) with η * , η * are solutions of the equations (7.36-7.38). One way to test the existence of a non-trivial solution of the 1RSB equations is to investigate the local stability of the RS solution, that we describe for both cases.

Intra-clause bias

Suppose indeed that the distributions in the support of P 1RSB are close to P triv , i.e. that they are supported on small values of h. One can then expand (7.102,7.102) and study the evolution of their average moments under the iterations of (7.104). More precisely one looks at the mean and variance:

M = dP (h)h, V = dP (h)h 2 (7.126)
with P (h) a distribution in the support of P 1RSB that is close to δ(h). The global spin-flip symmetry imposes that the mean M remains zero. The first non-trivial moment is thus the variance. Expanding linearly the BP equations, one obtain at the first non-trivial order a relation between variances of the form

V = θ 2 d i=1 k-1 j=1 V ij (7.127)
that is obtained by innjecting the 1RSB equations (7.102,7.103) in the expression of the variance V . θ is the derivative of f Θ0 (u 1 , . . . , u k-1 ) with respect to one of its arguments, evaluated on the trivial fixed-point. This relation is linear in the variances, one can take its average with respect to P 1RSB , and obtain that the averaged variance will grow if and only if

∞ d=0 r d d (k -1)θ 2 > 1. (7.128)
with ∞ d=0 r d d = αk for the ER ensemble. We thus obtain the Kesten-Stigum threshold α KS above which the trival solution of the 1RSB equations is unstable (and there must then exist a non-trivial solution) as

α KS = 1 k(k -1)θ 2 with (7.129) θ = σ1,...,σ k ω(σ 1 , . . . , σ k )σ 1 σ 2 σ1,...,σ k ω(σ 1 , . . . , σ k ) = k-2 p=0 k-2 p (ω p -2ω p+1 + ω p+2 ) k p=0 k p ω p . (7.130)

Bias with interactions at distance 1

In this case, one defines the mean and variance as follows

M (v) = dP (η) (η(v) -η * (w)) (7.131) V (v, v ) = dP (η) (η(v) -η * (w)) (η(v ) -η * (w )) (7.132)
with P a distribution in the support of P 1RSB that is close to the RS distribution δ[η -η * ]. As before, under symmetry considerations the mean M (v) is zero. One can show that the variance obeys a similar linear relation

V (v 1 , v 2 ) = v 1 ,v 2 J v1,v 1 J v2,v 2 l i=1 k-1 j=1 V ij , (7.133)
where J is a 4 × 4 matrix that can be written as the product J = M M , with M and M the Jacobian matrices obtained from the relations (7.18) and (7.19) evaluated at the RS solution:

M v,v = ∂η(v) η 1 (v ) * , M v,v = ∂ η(v) η 1 (v ) * . (7.134)
The equation ( 7.133) can be averaged with respect to P 1RSB to obtain the stability criterion

l(k -1)θ 2 < 1 (7.135)
where θ is the largest eigenvalue of J v,v . We define l KS through the relation

l KS (k -1)θ 2 = 1. (7.136)
The matrices M and M take the explicit form:

M v,v = 1 η * (0)D   I[σ = σ ]A(w + w ) - A(w) + A(w+1) y (A(w ) + A(w + 1)) D  
where A(w) and D have the following expressions:

A(w) = l-1 p=0 l -1 p ψ(w + p) y -p D = 2 A(0) + A(1) + A(1) + A(2) y (7.137)
and

M (σ,0),(σ ,0) = 1 Dη * (0) AI[σ = σ ] + BI[σ = -σ ] - C y D M (σ,0),(σ ,1) = 1 Dη * (0) I[σ = -σ ] - 1 y D M (σ,1),(σ ,0) = 1 Dη * (0) I[σ = -σ ] - C D M (σ,1),(σ ,1) = - 1 D 2 η * (0) (7.138) 126
where A, B, C, D have the following expressions:

A = 2 k-2 -k + 1 + (k -2)y -1 B = 2 k-2 -2 C = 2 k-1 -k + (k -2)y -1 D = 2(2 k-1 -k + (k -1)y -1 ) (7.139)
Chapter 8

Finite k results

In this chapter we report the numerical study of the clustering transition for the bicoloring on k-hypergraphs, for small values of k (k = 4, 5, 6). In Section 8.1 we describe how to solve numerically the recursive distributional equations obtained in chapter 7 (see (7.53,7.54) for µ Θ0 , and (7.67-7.68) for µ Θ1 ) using population dynamics. Solving these equations allows us to determine the clustering transition. In the section 8.2 we give a review and extension of the numerical procedures to determine accurately the clustering threshold. In section 8.3 we present the phase diagrams predicted by the cavity method for the biased measure µ Θ0 that introduces a bias that factorises on the clauses. We show that this biased measure allows us to increase the dynamic threshold with respect to the uniform one. In section 8.4 we present numerical simulations on finite size samples via Simulated Annealing. We show that with well-chosen parameters the biased measure µ Θ0 allows us to improve the performances of Simulated Annealing. These three sections present the results obtained in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF].

In section 8.5 we present some numerical results obtained for the biased measure µ Θ1 that introduces interactions between variables at distance 1, we show that this biased measure allows us to increase further the dynamical threshold compared to the simpler biased measure µ Θ0 . This last section presents some of the results obtained in [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF].

Numerical resolution for finite k

We give an explicit description of the numerical resolution of the distributional equations found in the previous chapter (see (7.53,7.54) for µ Θ0 , and (7.67-7.69) for µ Θ1 ). For a given choice of the parameters (k, α) and of the bias function (ω or ψ), the model is either in a non-reconstructible, Replica Symmetric (RS) phase if the point-to-set correlation function C n decays to 0 as n → ∞, or in a reconstructible, Replica Symmetry Breaking (RSB) phase if C n remains strictly positive in this large distance limit. We defined α d as the threshold value above which one enters the RSB phase. For the k-uniform l + 1-regular graph ensemble l d is the smallest integer value of l which leads to RSB. We recall that in this ensemble the density of constraints α and the degree l satisfy the relation α = (l + 1)/k. There are several upper bounds on α d that can be computed analytically: if the entropy ((7.34) for µ Θ0 , (7.42) for µ Θ1 ) computed in the RS ansatz is negative this is certainly an evidence for the RSB phenomenon.

The existence of hard fields, i.e. the possibility of naive reconstruction (see section 7.4), implies the reconstructibility, hence the rigidity bound α d ≤ α r . Finally the Kesten-Stigum bound α d ≤ α KS follows from the instability of the trivial fixed point of the reconstruction recursive equations (see section 7.7).

Nevertheless there are no lower bounds on α d that are simple to compute, hence an explicit determination of this threshold requires a numerical resolution of the equations (7.53,7.54) and (7.67-7.69). This type of Recursive Distributional Equation (RDE) admits a natural numerical procedure to solve them, called population dynamics algorithm [START_REF] Abou-Chacra | A selfconsistent theory of localization[END_REF][START_REF] Mézard | The bethe lattice spin glass revisited[END_REF], in which a probability distribution is approximated by the empirical distribution of a large sample of representative elements.

We explain it for the set of equations (7.53,7.54), the procedure is the same for the equations (7.67-7.69).

Suppose that at some step n one has an approximation of P n (h) written as

P n (h) ≈ 1 N N i=1 δ(h -h (n) i ) , ( 8.1)
where N 1 is the size of the population, that controls the numerical accuracy of the procedure (in the limit N → ∞ empirical distributions converge to the exact ones). One can then insert this form in the r.h.s. of (7.54) to obtain an approximation of P n (u) as

P n (u) ≈ 1 N N i=1 δ(u -u (n) i ) , (8.2)
where each of the representants u (n) i has been constructed independently by drawing σ 1 , . . . , σ k-1 according to the law p(σ 1 , . . . , σ k-1 |+) (defined in (7.44)), then k -1 indices i 1 , . . . , i k-1 uniformly at random in {1, . . . , N }, and setting:

u (n) i = f Θ0 σ 1 h (n) i1 , . . . , σ k-1 h (n) i k-1 .
Similarly, P n+1 (h) can be approximated by an empirical distribution of the form (8.1), where according to (7.53) each of the representants h (n+1) i is obtained by drawing an integer l from the Poisson law r l = e -αk (αk) l l! , then l indices i 1 , . . . , i l uniformly at random in {1, . . . , N }, and taking

h (n+1) i = f Θ0 u (n) i1 , . . . , u (n) i l .
The initial condition P 0 (h) = δ(h -1) (corresponding to a perfect observation of the variables at distance n = 0 from the root) can obviously be represented by a sample with all representants h (0) i equal to 1, and at every iteration step observables can be estimated as empirical averages,

dP n (h)F (h) ≈ 1 N N i=1 F (h (n) i ) (8.3)
for an arbitrary function F . Iterating these two steps many times one converges to a fixed point solution of (7.53,7.54), which can either be the trivial one P (h) = δ(h), P (u) = δ(u), or a non-trivial solution. Figure 8.1 presents some numerical results obtained in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] with this procedure, for the measure µ Θ0 . The top panel displays the decay of the correlation function C n (α, k, {w p }) as a function of the distance n for different values of α, for k = 6, and with the choice of parameters

ω 0 = ω k = 0.005, ω 1 = ω k-1 = 0.92, w 2 = • • • = ω k-2 = 1.
When α is small this function decays to 0 as n → ∞, signalling the impossibility of reconstruction; when α increases the decay gets slower and proceeds in two steps, with a longer and longer plateau at a strictly positive value developping as α gets closer to the transition; finally for α > α d the plateau lasts forever, the large n limit C(α, k, {w p }) is strictly positive. This quantity, that jumps discontinuously from 0 to a strictly positive value when α crosses α d , is plotted as a function of α in the bottom panel of Fig. 8.1. To increase the numerical accuracy we estimated the limit by averaging the value of C n when n is large enough to have reached its plateau behavior.

On the numerical determination of the dynamic transition

As explained in the previous section, the dynamic threshold α d is the smallest value of α such that the 1RSB equations at X = 1 admit a non-trivial solution (besides the RS trivial one in which all fields are equal to zero). We have seen how to solve numerically these equations, we now explain how to improve the accuracy when measuring α d . Depending on the models the appearance of a non-trivial solution can occur either in a continuous or a discontinuous way. In the former case one has α d = α KS , the bifurcation occurs via the local instability of the trivial fixed point studied in Sec. 7.7, and α d is thus known analytically. In the latter case α d < α KS , the birth of the non-trivial solution occurs non-perturbatively and cannot be detected from the properties of the trivial fixed point. The accurate numerical determination of α d when the transition is discontinuous is a rather difficult task. It corresponds to study a bifurcation for a fixed-point equation of the form P = F (P, α), where P is a probability distribution and F a functional on this space, depending on the parameter α. We shall discuss in the parts 8.2.2 and 8.2.3 the different numerical strategies that can be followed to determine α d , in particular one that, to the best of our knowledge, is new in this context. To explain these different methods it is instructive to study first a much simpler case, in which the unknown P is replaced by a real number, that we shall instead denote x for clarity.

Scalar bifurcations

Let us consider a function f (x, α), smooth in its two real arguments, and the associated discrete dynamical system x n+1 = f (x n , α), parametrized by α. We recall some basic facts in this setting: the stationary configurations of the dynamical system are the solutions x * (α) of the fixed point equation x = f (x, α). Their (linear) stability is determined by the coefficient λ(α) = (∂ x f )(x * (α), α) (here and below we denote (∂ x f ), (∂ α f ) and so on the partial derivatives of the function f ); a fixed point x * (α) is indeed stable under iterations if |λ(α)| < 1, and unstable if |λ(α)| > 1. We also recall the implicit function theorem: if (x 0 , α 0 ) is a solution of f (x 0 , α 0 ) = x 0 , and if (∂ x f )(x 0 , α 0 ) = 1, then there is a unique smooth function x * (α) with f (x * (α), α) = x * (α) in a neighborhood of α 0 , with x * (α 0 ) = x 0 . Hence the bifurcations of the fixed point equation, i.e. the modifications in the number of solutions, or the singularities of these solutions, are associated to points where (∂ x f )(x(α 0 ), α 0 ) = 1, in order for the implicit function theorem to be unapplicable. At these points the stability parameter λ(α) reaches its critical value 1.

To be more concrete we shall make the additional hypotheses that the order parameter x is restricted to non-negative values (x ≥ 0), and that f (0, α) = 0 for all α. Let us assume that this trivial fixed point, that exists for all α, is the unique solution for small enough values of α, and becomes non-unique when α exceeds a threshold α d . The two simplest ways to implement these hypotheses are sketched in Fig. 8.2, corresponding to a continuous bifurcation on the top panel, a discontinuous one on the bottom. Let us state a series of simple facts on these two types of phase transitions, that will be enlightening when we turn to the functional case. For α > α d there exists a non-trivial branch of stable fixed points x * (α) > 0; in the neighborhood of the bifurcation the latter behaves as

x * (α) = K(α -α d ) + o((α -α d )) when α → α + d , (8.4) with K = -2(∂ xα f )/(∂ xx f
), the derivatives being computed in (0, α d ). The stability parameter of the non-trivial solution, λ(α) = (∂ x f )(x * (α), α), reaches its marginal value 1 at the bifurcation as

λ(α) = 1 -K (α -α d ) + o(α -α d ) when α → α + d , (8.5) with K = (∂ xα f ).
Let us now turn to the discontinuous case (cf. the bottom panel of Fig. 8.2), and emphasize the main properties of the critical behavior of the bifurcation. The trivial fixed point is stable for all values of the parameter α; the bifurcation occurs at α d with the abrupt appearance of a solution x d > 0. These two quantities can be determined by solving the system of equations

x d = f (x d , α d ) , 1 = (∂ x f )(x d , α d ) . (8.6)
For α > α d there are two branches of non-trivial solutions x -(α) < x d < x + (α) that emerge from x d (see the bottom panel of Fig. 8.3); in the neighborhood of α d they behave as

x ± (α) = x d ± K √ α -α d + o( √ α -α d ) when α → α + d , (8.7)
where the coefficient K can be computed from the expansion of f around the bifurcation point (explicitly

, K = -2(∂ α f )/(∂ xx f ), the derivatives being com- puted in (x d , α d )). For α > α d x + (α) (resp. x -(α)
) is linearly stable (resp. unstable); the stability parameter λ(α) = (∂ x f )(x + (α), α) of the stable non-trivial branch reaches its critical value 1 at the bifurcation, with a critical exponent 1/2:

λ(α) = 1 -K √ α -α d + o( √ α -α d ) when α → α + d , (8.8) with K = -2(∂ α f )(∂ xx f ).
We present in the top panel of Fig. 8.3 the iterates x n+1 = f (x n , α), for a few values of α < α d , starting from an initial condition x n=0 > x d . Their long time limit is of course 0, the only fixed point in this phase, but when α → α - d the decay is slower and slower, with a large number of iterations spent around a plateau value at x d . More quantitatively one can define n * (α) as the minimal n such that x n ≤ x d -, and obtain that

n * (α) ∼ K (α d -α) -1/2
when α → α - d , (8.9)

with K = 2π/K , independently of x (0) and (as long as 0 < < x d ). Actually a whole scaling function describing the evolution of x n around the plateau can be derived, see [START_REF] Montanari | On the dynamics of the glass transition on bethe lattices[END_REF] for more details.

Discontinuous functional bifurcations

Let us now come back to our original goal, namely the determination of the dynamic threshold α d above which appears a non-trivial solution of the 1RSB equations at X = 1. As in the scalar case this transition can occur either in a continuous or in a discontinuous way; the former case was analytically dealt with in Sec. 7.7, we shall hence concentrate now on the discontinuous transitions. The numerical methods that we describe in this section are applicable to the uniform measure and the biased measures µ Θ0 and µ Θ1 . In practice we only implemented them on the biased measure µ Θ0 during this Ph.D., but it could be interesting to adapt the method to the biased measure µ Θ1 in order to obtain a numerical prediction of the clustering threshold with better accuracy.

The 1RSB equations (7.53,7.54) can be written abstractly as a functional fixed point equation P = F (P, α); at variance with the scalar toy model discussed above they can only be solved approximately, for instance by the population dynamics numerical algorithm explained in Sec. It is not completely obvious how to extract a precise estimate of α d from this kind of data. The simplest approach amounts to determine the curves C n for several closely spaced values of α, and assess that α d ∈ [α < , α > ], where α < is the largest value for which C n drops to 0 at large n, α > the smallest value for which a stable plateau is encountered. This determination suffers however from inaccuracies due to the finite number of α values one can investigate, the finite number of iterations one can perform (leading to an underestimation of α > ) and to the finite size of the population that approximate the distribution P (α < can thus be overestimated, finite size fluctuations having a destabilizing effect).

One can try to circumvent these difficulties by getting some inspiration from the much simpler scalar bifurcation studied above. We recall that the criticality at α d showed up in three different ways: (i) x + (α) exhibits a square root singularity when α → α + d , see Eq. (8.7); (ii) the length of the plateau diverges when α → α - d with a critical exponent -1/2, cf. Eq. (8.9); (iii) the stability parameter λ(α) reaches 1 with a square root singularity when α → α + d , as written in Eq. (8.8).

Assuming the same critical behavior to occur in the discontinuous functional bifurcation case (even if much more complicated behaviors could occur in infinite dimensional spaces) one can try to exploit these scaling laws in order to obtain more precise estimates of α d . Point (i) translates into a square root singularity of the large n limit of C in the limit α → α + d ; this does not seem very useful to us, as it would involve a fit of C(α) in which both α d and C(α d ) are unknowns. On the contrary points (ii) and (iii) yield simpler fits for the determination of α d . The aspect (ii) is very easy to exploit: from the curves of the top panel of Fig. 8.1 one can deduce immediately a value n * (α) for the number of iterations necessary to fall below the plateau (as in the scalar case one can define n * (α) with any threshold strictly between 0 and the plateau value). According to (8.9) n * (α) -2 should vanish linearly at α d ; this is indeed what we obtain with a rather good accuracy, see the top panel of Fig. 8.4. However one cannot reach in this way a very large number of iterations, the numerical rounding errors and finite population size fluctuations having the tendency to accumulate over time; this cutoff on n thus limits the accuracy of this determination of α d . We have thus turned to the functional generalization of point (iii) above, namely the computation of a stability parameter λ(α) for the stable non-trivial branch α > α d , and the determination of α d as the parameter for which λ reaches 1. This extrapolation is done using the scaling anticipated in the scalar case in (8.8), and is illustrated in the bottom panel of Fig. 8.4. The functional nature of the unknown in the fixed point equation makes the definition of λ more complicated than in the scalar case, where it was simply ∂ x f ; we give detailed explainations on the numerical computation of λ(α) in the functional case in the next section. Before that let us emphasize that the square root behavior of λ around α d , guessed from the scalar bifurcation, is in very good agreement with the numerical results obtained in the functional case when the dynamic transition is discontinuous (see the bottom panel of Fig. 8 As a consistency check we also present in Fig. 8.5 a similar study in the case of a continuous transition. We see that the stability parameter λ computed on the non-trivial solution, i.e. for α > α d , reaches 1 with a linear behavior (as in the scalar case, see Eq. (8.5)), and that its extrapolation is in good agreement with the analytically computed value of α KS from Eq. (7.130). Moreover the numerical computation of the stability parameter of the trivial fixed-point coincides for α < α d with the analytical one, λ KS = αk(k -1)θ 2 .

1 -λ KS fit data α 1 -λ(α)
= ω k = 0, ω 1 = • • • = ω k-1 = 1)
. From the fit one obtains the result α = 4.088, wich is in good agreement with the analytical one α KS = 4.083.

The stability parameter λ in the functional case

As an intermediate step in the generalization from the scalar to the functional case let us consider a fixed point equation of the form x = f ( x, α), where the unknown x is a finite-dimensional real vector. The stability of a branch of solutions x(α) can be determined by considering the Jacobian matrix J of the first derivatives of f computed at the fixed point, that can be defined through the linearization

f ( x(α) + ε, α) = x(α) + J ε + o( ε ) , (8.10)
where ε is a small perturbation around the fixed point. The stability parameter λ(α) can then be defined as the spectral radius of J, i.e. the largest absolute value of the elements of its spectrum. This spectral radius can be expressed in terms of successive applications of J on a perturbation ε as

λ(α) = lim m→∞ J m ε ε 1 m , ( 8.11)
where we assume that ε has a non-vanishing projection on the eigenspace associated to the relevant eigenvalue, and where • can be any norm. For future use let us define ε m = J m ε and rewrite this expression as

λ(α) = lim m→∞ ε m ε m-1 ε m-1 ε m-2 . . . ε 1 ε 1 m (8.12)
We would like now to extend the computation of a stability parameter to the 1RSB equations (7.53,7.54) that can be rewritten as P = F (P, α) by grouping the two lines together. P being a probability distribution the Jacobian of F is now an infinite-dimensional operator, which makes the study of its spectrum rather difficult. Even worse, we do not have at our disposal an exact description of the fixed point P around which we would like to expand F : we only have a sequence of approximations of P n by the population representation written in Eq. (8.1). The individual elements of these representations still evolve at each iteration step, even when the observables computed as averages of P n have reached convergence (within the numerical accuracy fixed by the population size N ). To circumvent these difficulties we have followed a strategy inspired by the expression (8.12): we consider P n and a slight perturbation of it, P n + δP n , and assess the rate of growth of the perturbation along the iterations by the functional F . In order to implement this idea in practice one needs to choose a specific form for the perturbation; given that P n is represented as a sum of Dirac deltas we perturb it by giving an infinitesimal width to each of the peaks, that we replace by Gaussian distributions with a small variance. We thus define

P n (h) ≈ 1 N N i=1 δ(h -h (n) i ) , (P n + δP n )(h) ≈ 1 N N i=1 G(h; h (n) i , ε (n) i ) , (8.13
) As the ε are infinitesimally small one can linearize f Θ0 to compute the mean and variance of this random variable. In summary, the determination of λ(α) is done by tracking the evolution of P , P and their perturbed versions with populations of couples of real numbers, (h i , ε i ) and (u i , ε i ), that evolve in time according to the following generalization of the update rules given in Sec. 8.1. To obtain (h

P n (u) ≈ 1 N N i=1 δ(u -u (n) i ) , ( P n + δ P n )(u) ≈ 1 N N i=1 G(u; u (n) i , ε (n) i ) , ( 8
(n+1) i , ε (n+1) i
) one repeats, independently for i = 1, . . . , N , these steps:

• draw an integer d from the law p d

• draw d indices i 1 , . . . , i d uniformly at random in {1, . . . , N } • set h (n+1) i = f Θ0 (u (n) i1 , . . . , u (n) i d ) and ε (n+1) i = d j=1 (∂ j f Θ0 ) 2 ε (n) ij
, where ∂ j f Θ0 denotes the derivative of f Θ0 with respect to its j-th argument, computed in (u

(n) i1 , . . . , u (n) i d ) 139
Similarly the population (u

(n) i , ε (n) i
) is generated according to, again independently for i = 1, . . . , N :

• draw σ 1 , . . . , σ k-1 from the probability law p(σ 1 , . . . , σ k-1 |+)

• draw k -1 indices i 1 , . . . , i k-1 uniformly at random in {1, . . . , N } • set u (n) i = f Θ0 (σ 1 h (n) i1 , . . . , σ k-1 h (n) i k-1 ) and ε (n) i = k-1 j=1 (∂ j f Θ0 ) 2 ε (n) ij
The rate of growth of the perturbation during the iteration n → n + 1 is estimated as the ratio of the L 1 norms of the perturbation parameters,

λ n = N i=1 ε (n+1) i N i=1 ε (n) i , (8.15)
and the stability parameter is finally computed as

λ(α) = (λ n0 λ n0+1 . . . λ n0+m-1 ) 1 m . (8.16)
Indeed the first n 0 iterations are done with the usual population dynamics algorithm, evolving only the h i 's and u i 's, in order to reach an approximate convergence in distribution of the populations to their fixed points, and the perturbation is then initialized with ε

(n0) i = 1.
A large number m of additional iterations during which the growth rates are recorded are then performed, and averaged geometrically as in (8.12); in the large m limit the value of λ(α) should be independent of the norm used to define λ n . In practice we divide the ε (n+1) i by λ n after each iteration in order to keep the norm constant and avoid numerical underflows.

This method is similar to the one presented in [START_REF] Parisi | Diluted mean-field spin-glass models at criticality[END_REF][START_REF] Parisi | Erratum: Diluted mean-field spin-glass models at criticality[END_REF] to determine the location of a continuous RSB transition from a non-trivial RS solution.

Results of the cavity method 8.3.1 The existence of a RS phase for α > α d,u

We shall study the evolution of the dynamic phase transition when the measure µ Θ0 (σ) over the proper bicolorings of a typical Erdős-Rényi random hypergraph is not uniform anymore. In the setting considered in equation (6.7,6.8) this corresponds to take the parameters {ω p } of the interaction function different from the uniform choice

ω 0 = ω k = 0, ω 1 = • • • = ω k-1 = 1.
We will concentrate first on the "zero-temperature" case, i.e. on the measures that give a non-zero weight to proper bicolorings only, which implies ω 0 = ω k = 0. The choice of the other parameters is constrained by the global spin-flip symmetry that we want to preserve, hence ω p = ω k-p ; as it is obvious from (6.7), multiplying all the ω p by a common constant does not change the properties of the model. One realizes that for k = 3 there is no free parameter left, we will thus concentrate on the cases k ≥ 4 from now on. For arbitrary large values of k there will be of the order of k/2 free parameters in the ω p ; we will however make the following choice for the zero-temperature measure:

ω 0 = ω k = 0 , ω 1 = ω k-1 = 1 -, ω 2 = • • • = ω k-2 = 1 , (8.17)
where is the sole parameter that quantifies the deviation from the uniform measure (that is recovered for = 0). This slight loss in generality is made for the sake of simplicity, and motivated by considerations on the large k limit presented in chapter 10 (section 10.1). The parameter controls the relative weight given to the "almost monochromatic" constraints that contain a single vertex of one of the possible colors (positive values of disfavoring them); as discussed in Sec. 7.4 these are precisely those responsible for the existence of frozen variables, one of the mechanism of RSB. We present in Fig. 8.6 phase diagrams in the (α, ) plane for k = 4, k = 5 and k = 6. The three lines in these plots correspond to the thresholds defined in (7.35) from the vanishing of the RS entropy, in (7.130) from the instability of the RS solution (Kesten-Stigum threshold), and in (7.82) from the appearance of hard fields in the solution of the 1RSB equations at X = 1 (rigidity threshold); specializing these three expression with the choice of parameters (8.17) yields

α s=0 (k, ) = ln 2 k(1-) ln(1-) 2 k-1 -1-k -ln 1 -1+k 2 k-1 , (8.18) α KS (k, ) = 1 k(k -1) 2 k-1 -1 -k 1 + (k -4) 2 , (8.19) α r (k, ) = 1 k Γ r (k) 2 k-1 -1 -k 1 - . (8.20)
In addition the black squares in Fig. 8.6 signal a discontinuous appearance of a non-trivial solution of the 1RSB equations at X = 1 upon increasing α, that we located by a numerical resolution of these equations following the methods explained in Sec. 8.2. One can see on these plots that for all values of there is a critical density of constraints, α d ( ), such that a non-trivial solution of the 1RSB equations at X = 1 exist if and only if α > α d ( ). To make this separation more visible the area on the left of α d ( ), i.e. the RS phase of the model, has been painted in gray in Fig. 8.6. Let us call (α opt , opt ) the coordinates of the point on the line α d ( ) which maximizes the density α of constraints, α opt = max α d ( ), that corresponds to an optimal choice of the bias parameter. The numerical values of these optimal parameters can be found in Table 8.1 for k = 4, 5 and 6. By definition α opt ≥ α d,u = α d ( = 0), the dynamic transition of the usual model, with the uniform measure over the proper bicolorings; the nontrivial result here is that the inequality is strict, i.e. that a well chosen value of the biasing parameter is able to turn the clustered uniform measure into an unclustered biased one (for α ∈ [α d,u , α opt ]). ,α), at zero temperature ω 0 = 0. The RS phase, painted in gray, is on the left of α d ( ), the latter corresponds either to a continuous transition with α d ( ) = α KS ( ) (solid line, see (8.19)) or to a discontinous transition (black squares). The dashed horizontal line corresponds to = 0, the uniform measure, which intersects α d at α d,u . The arrow points to the optimal point that maximizes α d . The dotted line is the rigidity threshold α r from (8.20), the dot-dashed line marks the vanishing of the RS entropy (see Eq. (8.18)). 8.1: Dynamic threshold for the uniform measure (α d,u = α d ( = 0)), and largest α reachable in the RS phase, this optimal point having coordinates (α opt , opt ).

A further scrutiny of the phase diagrams reveals different scenarios depending on the value of k. For k = 4 the nature of the bifurcation on the line α d ( ) changes precisely at opt : for > opt the transition is continuous and thus α d coincides with the Kesten-Stigum line α KS , while it is discontinuous for < opt and there is a cusp at the optimal point (we shall come back on this point later on). It turns out that for k = 4, opt < 0: this is rather counterintuitive at first sight, as it means that favoring the almost violated configurations of variables actually makes the measure less frustrated. This peculiarity can be explained by noticing that for k = 4 the dynamic transition of the uniform measure ( = 0) is continuous and that α KS decreases with . As the dynamic transition of the uniform measure is discontinuous for k ≥ 5 [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] this peculiarity is restricted to k = 4, and one has opt (k ≥ 5) > 0. Turning now to the phase diagram for k = 5 in Fig. 8.6 one observes similarly a cusp in α d ( ) at opt , that separates a continuous and discontinuous branch of the dynamic transition line, but with now opt > 0. Finally for k = 6 the optimal point is on the discontinuous branch of α d ( ); increasing further one encounters a cusp for at some value of > opt and then a continuous branch α d ( ) = α KS ( ). The large k behavior of the model will be further discussed in chapter 10; we can nevertheless anticipate that for large enough k the Kesten-Stigum threshold becomes irrelevant, as it happens in the negative RS entropy region (compare the leading orders of Eqs. (8.18,8.19)). In this case the whole line α d ( ) corresponds to a discontinuous bifurcation. As a last remark on the phase diagrams of Fig. 8.6 let us emphasize that for all one has necessarily α d ( ) ≤ min(α KS ( ), α r ( ), α s=0 ( )), these three thresholds implying a mechanism of failure for the hypotheses underlying a purely RS phase. This should easily convince the reader of the necessity of discontinuous branch of α d ( ) in some parts of the phase diagrams. For instance when k = 4 and ≤ -0.3 the rigidity and negative entropy bounds imply α d ( ) < α KS ( ), in other words the dynamic transition must be discontinuous.

We have motivated earlier our study of the boundaries of the RS phase by algorithmic considerations, Monte Carlo Markov Chains being expected to equilibrate rapidly inside such a phase. However in a practical simulation one cannot assume that the initial configuration belongs to the support of a zerotemperature measure (otherwise the problem of finding a solution of the CSP would be already solved), it is thus necessary to make an annealing in temperature for a random initial condition to be allowed. For this reason we have also studied the evolution of the phase diagrams at positive temperature, modifying the parameters (8.17) with ω 0 = ω k > 0, see the results in Fig. 8.7. These plots show the absence of "reentrance" in temperature, in the sense that the lines α d ( ) move towards higher density of constraints when ω 0 is increased. Hence in principle a simulated annealing procedure with parameters (α, ) in the zero temperature RS domain, progressively decreasing ω 0 , should be able to remain equilibrated on polynomial time scales, hence finding solutions for α < α opt if the appropriate bias is used. A numerical test of this conjecture is presented in Sec. 8.4.

More detailed zero temperature phase diagrams

The extent of the RS domain in the (α, ) phase diagram presented in Fig. 8.6 was the most interesting information to extract from the cavity formalism in the perspective of this Ph.D.. For the sake of completeness we shall nevertheless discuss with slightly more details some properties of the RSB phase, and present another version of the phase diagrams for k = 4 and k = 5 in Fig. 8.8. transition where a non-trivial solution of the 1RSB equations emerge continuously from the trivial one upon increasing α, the filled (resp. empty) squares corresponds to the spinodal (sp) of the HO (resp. LO) branch that appears discontinuously when α is increased (resp. decreased). The filled circles are defined by the vanishing of the complexity of the HO branch.

The most important additional feature unveiled by these phase diagrams is that for some values of the parameters k, α, , there exits (at least) two different non-trivial solutions of the 1RSB equations at X = 1 (7.53,7.54). This type of behavior was described in [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] for a family of random CSPs generalizing the hypergraph bicoloring, and its consequences for inference problems (or planted CSPs) have been discussed in [START_REF] Ricci-Tersenghi | Typology of phase transitions in bayesian inference problems[END_REF]. In order to reach numerically these different solutions we used the population dynamics algorithm explained in Sec. 8.1 with an initial condition generalizing P n=0 (h) = δ(h -1) into

P n=0 (h) = (1 -ε) δ(h) + ε δ(h -1) . ( 8.21)
For each choice of the parameters we ran twice the population dynamics algorithm, once with ε = 1 and once with a small value of ε > 0 (in practice we used ε = 0.01); in the tree reconstruction interpretation the latter correspond to a variant known as robust tree reconstruction [START_REF] Janson | Robust reconstruction on trees is determined by the second eigenvalue[END_REF], in which only a fraction ε of the variables at large distance from the root are revealed to the observer. We will call HO, for high overlap, the initialization with ε = 1, and LO (low overlap) the small ε one. Depending on the parameters these two procedures can produce different solutions of the 1RSB equations, or not. More precisely, the different phases located in Fig. 8.8 are defined as follows:

• RS: both HO and LO initial conditions lead to the trivial solution.

• RS+HO: LO initial condition leads to the trivial solution, whereas HO initial condition leads to a non trivial solution.

• LO+HO: LO initial condition leads to a non trivial solution, HO initial condition leads to a non trivial solution with a higher overlap.

• HO/LO: both HO and LO initial conditions lead to the same non-trivial solution.

We present in the left panels of Fig. 8.9 the evolution of the overlap as a function of for different fixed α at k = 5, which should help to grasp the meaning and succession of the different phases. We recall that α KS is the limit of stability of the trivial fixed point, that undergoes a bifurcation at the Kesten-Stigum transition. The two lines denoted sp (for spinodal) in Fig. 8.8 correspond to the limit of existence of the two non-trivial branches of solution of the 1RSB equations. This more complete study of the number and domain of existence of solutions of the 1RSB equations should clarify the cusp at opt of the line α d ( ) found for k = 4 and k = 5 in Fig. 8.6: a first look at these figures could suggest that the two parts of the α d ( ) line join at a tri-critical point, in the sense that the discontinous transition becomes less and less discontinuous before crossing over to a continuous transition. However from Fig. 8.8 one sees that this is not the case, the discontinuous branch of α d ( ) extends to the RSB phase as a spinodal unrelated to the continuous transition, which can only make sense in the context of coexistence of two non-trivial solutions. A tri-critical point does exist in these phase diagrams, but it is located strictly inside the RSB phase, not at the cusp, and corresponds to the merging of the two spinodals.

Finally we have also indicated on Fig. 8.8 the threshold for the cancellation of the complexity of the HO solution (see in addition the right panels in Fig. 8.9); this corresponds to the condensation transition of the model in the α < α KS part of the phase diagram (for α > α KS the LO solution has a negative complexity hence the problem is condensed, see [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] for a discussion of this point).

Results of Simulated Annealing

In this Section we present the results of extensive simulations, where we have used the Simulated Annealing (SA) algorithm [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF] to find solutions of the hypergraph bicoloring problem. Our main aim is to show that SA finds solutions more easily if the biased measure µ Θ0 (see equations (6.7) and the specialization (8.17)) is used: although the uniform measure ( = 0) has a larger entropy, the biased one is more concentrated on solutions that can be reached in an easier way by SA and thus the SA algorithmic threshold improves if = 0 is used.

In this setting, we consider Erdős-Rényi random hypergraphs with k = 4, 5, 6 and sizes ranging from N = 10 4 to N = 10 6 . The parameters α and are taken in the relevant region where we expect an algorithmic phase transition to take place, that is around α d . Let us rewrite the biased measure that we are willing to sample via the SA at a generic finite temperature T = 1/β as

µ(σ) = 1 Z(G) e -βU (σ) (1 -) F (σ) (8.22)
where U (σ) is the number of unsatisfied constraints (i.e. monochromatic hyperedges) and F (σ) is the number of freezing clauses (i.e. hyperedges with exactly k -1 variables of the same color). This corresponds indeed to the measure (6.7) with parameters

ω 0 = ω k = e -β , ω 1 = ω k-1 = 1 -, ω 2 = • • • = ω k-2 = 1 . (8.23)
The solutions of the CSP have U = 0, and non-uniform weights if = 0. Our SA implementation uses the Metropolis algorithm with single-spin flip dynamics: at each time step one considers a configuration σ that differs from the current configuration σ by the reversal of an uniformly chosen spin. The move σ → σ is accepted with the probability min µ(σ ) µ(σ) , 1 = min e -β∆U (1 -) ∆F , 1 , (

where ∆U = U (σ ) -U (σ) and ∆F = F (σ ) -F (σ), in such a way that the detailed balance (reversibility) condition with respect to the measure (8.22) is ensured. We store the quantity k i=1 σ i for each clause, which allows a fast computation of the changes ∆U and ∆F when a spin is flipped.

We run SA with a very simple piecewise constant, uniformly spaced, temperature scheduling: the first Monte Carlo Sweep (MCS, i.e. N elementary steps described above) is performed with T fixed to T max (we used T max = 0.5 in all our simulations), then T is reduced by ∆T = T max /τ and a new MCS is performed, T is again reduced by ∆T , and so on and so forth. We perform in this way τ + 1 MCS, the last one being at zero temperature, the running time of the algorithm thus scales as N τ elementary steps.

The lowest value of U (σ) is always reached at the end of each run, when the annealing has reached zero temperature. So we present results only for the quantity U 0 = U (T = 0), that is the smallest number of violated clauses that the SA is able to reach in a running time of τ MCS. SA is successful as a solver if and only if U 0 = 0, but we will be interested in estimating the lowest energy reachable by SA even in the regime where it is not successful. In particular we are going to study the lowest intensive energy reached by SA, u 0 = U 0 /N , in the large size limit where it becomes independent on the problem size N .

Estimating the algorithmic threshold for Simulated Annealing

We shall first discuss the problem of the estimation of the algorithmic threshold for a stochastic algorithm like SA, concentrating for simplicity on the unbiased ( = 0) case, the extension to = 0 will be considered later on. The behavior of the algorithm depends on the density of constraints α, the annealing time τ , and the size of the problem N ; it can be described in terms of the average energy density u 0 (α, τ, N ) reached at the end of the run, or in terms of the probability (with respect to the random instance generation and the stochasticity of the algorithm) p succ (α, τ, N ) that the algorithm discovers a solution of the instance. It is clear that the energy (resp. success probability) reached by SA is a decreasing (resp. increasing) function of the running time τ . We are interested in the limit of large times but sub-exponential with respect to the problem size N (on exponentially large timescales any Monte Carlo simulation of a finite size system is ergodic and u 0 = 0 as long as α < α sat , but this is not the regime we are interested in). An idealized definition of the algorithmic threshold α algo would be the smallest density of constraints such that lim

N →∞ u 0 (α, τ = N c , N ) > 0 or lim N →∞ p succ (α, τ = N c , N ) = 0 , ( 8.25)
for any fixed exponent c, corresponding to polynomial time algorithms. Of course time and space requirements impose strong constraints on the values of τ and N that can be used in practice. The limit above must thus be performed by an extrapolation from finite N results, and if c is free any running time could be considered as "polynomial" as long as N is finite. To resolve this ambiguity we shall restrict our study to linear times (this time scale is the only one practically accessible on very large problems), i.e. consider τ fixed (but arbitrary large) in the thermodynamic limit N → ∞.

Even with this restriction the numerical extrapolation necessary to estimate α algo is far from being an easy task. The definition given above relies on the behavior of the asymptotic intensive energy u 0 as a function of α. We plot the corresponding data in Figure 8.10 for k = 4, 5: these data have been obtained for the unbiased measure ( = 0) and different problem sizes (10 4 ≤ N ≤ 10 6 ); note that data points with different N values are very close, i.e. the size dependence is very weak, and the values for N = 10 5 and N = 10 6 always coincide within errorebars. Unfortunately the asymptotic energy u 0 is strongly dependent on the running time τ and it is thus very difficult to extract from this figure the algorithmic threshold, i.e. the value of α where u 0 becomes positive in the large τ limit.

A much more convenient way of analyzing the same data is presented in Figure 8.11, where for each value of α we study the dependence of u 0 on τ . The α values shown are such that the relative difference between the smallest and the largest α values is around 5%. Again the size dependence is weak and we can mostly ignore it. Error bars have been computed only from sample to sample fluctuations. The main observation now -note the log-log scale in the plots Estimating the algorithmic threshold from these plots is very difficult due to the strong τ dependency.

-is that for the smallest α values shown in the plots the asymptotic energy is decreasing very fast with τ , faster than a power law (data not shown have u 0 0); on the contrary, for the largest α values, u 0 decreases slower than a power law. In the latter case we even observe an upwards curvature, suggesting a non-zero value for u 0 in the τ → ∞ limit.

In practice, our best estimate for the SA algorithmic threshold is given by the α value such that u 0 decays as an inverse power law of τ , thus separating the regimes where u 0 decays faster and slower than a power law in τ . For = 0, we find the following approximate values α alg (k = 4) ≈ 4.7, α alg (k = 5) ≈ 9.6 and α alg (k = 6) ≈ 18.5. We notice that all these algorithmic thresholds are larger than the threshold α d,u listed in Table 8.1, where the "dynamic" phase transition, defined as the appearance of a solution of the 1RSB equations, takes place. This observation is consistent with the idea that sampling solutions uniformly is more difficult than just finding one or few solutions. Indeed, while a MCMC is expected to sample uniformly the solutions efficiently only for α < α d,u , SA can find a solution in linear time until α alg , which is greater than α d,u .

In other words, in the range [α d,u , α alg ] the SA algorithm does not thermalize at the lowest temperatures explored during the annealing, but it can be seen as an efficient out of equilibrium process converging in linear time to a solution, as discussed for instance in [START_REF] Zdeborová | Generalization of the cavity method for adiabatic evolution of gibbs states[END_REF].

In particular for k = 4 the model has a continuous phase transition and the SA algorithm seems to be very efficient in this case: the algorithmic threshold α alg (k = 4) ≈ 4.7 is well beyond the dynamic threshold α d,u = 4.083 and not far from the 1RSB estimate of the satisfiability threshold α sat (k = 4) ≈ 4.9 [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] (this is only expected to be an upper bound on the true satisfiability threshold due to an instability towards higher levels of RSB). On the contrary for k ≥ 5 the phase transition taking place at α d,u is of the random first order type (discontinuous) and this seems to have a dramatic effect on the performance of SA, which is able to find solutions only slightly beyond α d,u , stopping far from the α sat threshold. For example, for k = 5 we have α d,u = 9.465, α alg ≈ 9.6 and α sat = 10.46 [START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF]. opt confirms that for these "optimal" parameters SA is effective in finding the ground state in linear time.

Performances of Simulated

As shown in Sec. 8.3.1 we can extend the RS phase in the region α > α d,u by tuning appropriately the bias . In Figure 8.12 we show the asymptotic energy as a function of the running time τ for the parameters α = α opt and = opt given in Table 8.1, that are optimal from the point of view of extending the RS phase to the largest α possible. For each value of τ we report the results obtained with 3 sizes N = 10 4 , 10 5 , 10 6 although the different data points are hardly visible due to their strong overlap (for τ = 10 6 fewer sizes are shown). Errors are computed from sample to sample fluctuations.

For all the values of k = 4, 5, 6 the behavior of the asymptotic energy is compatible with a power law decay or even faster than that (the straight line is just a guide to the eye with a slope -0.61). So, as expected, SA seems to be an efficient algorithm to find solutions in the RS phase, even when this phase extends beyond α d,u via the optimization of the bias .

Performances of Simulated Annealing with the biased measure

In Section 8.3 we have shown how the phase diagram and the corresponding thresholds change in presence of a non-zero bias ( = 0). The suggestion we get from this analytical study is that a non-zero bias should make easier for the SA algorithm to find solutions at large α values. However the connection between the phase diagram and the behavior of the SA is not obvious, as already shown in Section 8.4.1 for the = 0 case. The aim of the present section is to show the results of extensive numerical simulations running SA with the biased measure in order to gather evidence that a non-zero bias is in general beneficial for the performances of SA in finding a solution to the random hypergraph bicoloring problem.

We have already shown that finite size effects are very small and slightly visible only for N = 10 4 . So in the following we present uniquely data obtained with size N = 10 5 . We have checked these are practically indistinguishable from the results with N = 10 6 on the time scales reachable in the latter case.

In Figure 8.13 we show the data collected at the three largest α values for k = 4 (upper row), k = 5 (middle row) and k = 6 (lower row). In each panel we plot u 0 as a function of for three different cooling rates τ = 10 4 , 10 5 , 10 6 (from top to bottom in each panel). The plots provides a clear evidence that reweighting solutions with a bias > 0 enhances the probability that SA reaches lower energies.

Already a simple qualitative analysis reveals the advantage of using > 0. In every panel we see that u 0 reaches a minimum for a strictly positive value of . The value SA that minimizes u 0 is only weakly dependent on the SA cooling time τ , so it is likely to assume that lim τ →∞ SA > 0 and the bias is effective even in the limit of large times. The data in Figure 8.13 suggest that the SA algorithmic threshold may grow for moderately small values of with respect to its = 0 value. For example for k = 6 the SA algorithmic threshold for = 0 was estimated around α alg ≈ 18.5, but looking at the plots in the lower row it is evident that at least for α = 19 and 0.15 SA reaches the ground state u 0 = 0 and for α = 19.5 and 0.15 the convergence to u 0 = 0 is very fast in τ .

We have done a more quantitative analysis for the largest α values, shown in the three panels on the right. We have interpolated the data of u 0 with a quadratic function of , the interpolating parabolas are shown in the right panels in Figure 8.13. Fitting the minimum of the parabola u 0 ( SA ) as a power law in τ we find the results shown in Figure 8.14. For (k = 5, α = 9.9) the behavior is faster than a power law and thus we expect the SA algorithmic threshold with the biased measure to be slightly greater than α = 9.9. On the contrary for both (k = 4, α = 4.8) and (k = 6, α = 20) the behavior looks slightly slower than a power law and thus we are tempted to believe lim τ →∞ u 0 ( SA ) > 0 in these cases and the SA algorithmic threshold is slightly below.

Unfortunately the quantitative analysis cannot be made more robust, due to the strong τ dependence observed. Nevertheless we believe that the evidence that > 0 makes ground states more accessible to Simulated Annealing is strong enough, both for finite τ values and in the large τ limit.

Let us finally compare the optimal value SA of the bias that improves most the performances of SA with the optimal value opt found in Section 8. increases most the extent of the RS phase. We notice that SA is always larger than opt . Considering that, for the small values of k studied in the simulations, it is approximately true that for > opt the phase transition increasing α is continuous, while for < opt the model undergoes a random first order transition, we believe the most natural explanation for the observation SA > opt is the following. The ergodicity breaking taking place at a discontinuous (i.e. random first order) transition is much more severe than the one taking place at a continuous phase transition. In the case of a discontinuous phase transition, the SA algorithm can find solutions only slightly above the dynamic threshold α d , while in the continuous case SA remain efficient in finding solutions even well above α KS . The analysis supporting this scenario has been presented in Section 8.4.1. Thus it is natural to expect that SA presents its best performances for > opt where the phase transition is continuous and the ergodicity breaking not too severe. The finding that SA (k = 4) > 0 also resolves the rather counterintuitive result opt (k = 4) < 0. So even for k = 4 the SA algorithm finds the ground state more easily if frozen variables are avoided.

Comparison with the biased measure with interactions at distance 1

In this section we present the study of the clustering threshold for the biased measure µ Θ1 defined in equation (6.12), and compare it to the simpler measure µ Θ0 , in the k-uniform l + 1-regular random ensemble. We apply the numerical procedure described in Section 8.1 to solve the recursive distributional equations (7.67-7.69). The distributions P w,n are approximated by the empirical distribution over a sample of N representants as:

P w,n (η) ≈ 1 N N i=1 δ(η -η i ) . ( 8.26)
With 2N fields η i one can encode the distributions P w,n at distance n for w = 0, 1. The 2N fields representing the two distributions P w,n+1 are generated stochastically according to (7.68,7.69), and in turn the populations representing P w,n+1 can be obtained from (7.67). At each step of this iterative procedure one computes the correlation function C n from (7.70), interpreting the average over P w,n as an uniform sampling of an element of the corresponding population. An example of the results thus obtained is presented in Fig. 8.15, where one sees, depending on the choices of parameters, RS cases with C n vanishing at large n, and RSB situations where C n remains positive. The results presented in the rest of this section have been obtained with populations of size N = 10 6 ; we considered that C n → 0 whenever the average value of C n , for large enough values of n such that stationarity was reached within our numerical accuracy, dropped below a small threshold value (we used 0.005 in figures below). The function ψ(p) contains a large number (l + 1) of free parameters, some choices must hence be made on its specific form. We first considered the case where ψ(p) = (1 -) p , with a single free parameter < 1. As explained in Sec. 6.1.3 this corresponds the biased measure µ Θ0 with the choice of parameters (8.17), i.e. to a bias that factorizes over the hyperedges of the bicoloring problem. We have presented in the section 8.3 the study made in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] for Erdős-Rényi (ER) random hypergraphs in which the degree of a vertex has a Poisson distribution of average αk. The phase diagrams obtained for the regular case (see [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF]) are presented in Fig. 8.16. They show the clustering threshold l d (points), as well as the upper bounds corresponding to the rigidity, the Kesten-Stigum bound, and the vanishing of the RS entropy, in the plane (l, ), for k = 5 and k = 6. They are qualitatively similar to the results obtained in the section 8.3, figure 8.6 (see [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF]) for the ER case, and quantitatively close with the correspondence αk = l + 1 between the average degree of the ER ensemble and the one fixed here. The important point we want to emphasize here is the fact that a suitable choice of allows us to increase l d with respect to its value for the uniform measure ( = 0). For instance for k = 5 and l = 47, the RSB phase at = 0 is turned into a RS phase when = 0.04. Similarly for k = 6 the dynamic transition l d = 108 of the uniform measure can be pushed to l d = 113 for a well-chosen value of .

The natural question that arises at this point is whether the more generic bias introduced in this manuscript, i.e. the additional degrees of freedom in the choice of ψ(p), allows us to further increase the dynamic transition threshold l d . To investigate this point without introducing too large a space of parameters, that would be impossible to explore systematically, we considered the following function ψ:

ψ(0) = 1 , ψ(1) = b 1 , ψ(p) = b 2 (1 -) p for p ≥ 2 , ( 8.27)
with the three free parameters (b 1 , b 2 , ). We then solved numerically the RDEs with parameters close to the optimal values found previously in the restricted case with b 1 = 1 -and b 2 = 1. The results are shown in Fig. 8.17 in the parameter plane (b 1 , b 2 ), for fixed values of k, l and , with squares (resp. crosses) marking RS (resp. RSB) phases. The top panel shows the existence of a RS phase at k = 5, l = 48, whereas all values of led to RSB at this value of l for the factorized bias ψ(p) = (1 -) p . We did not find any choice of parameters (b 1 , b 2 , ) with a RS phase for l = 49. Similarly the bottom panel shows, for k = 6, the largest value of l, l = 114, for which we found a RS phase for well-chosen parameters (see also the drop of C n to 0 in Fig. 8.15 for = 0.095, b 1 = 1.6, b 2 = 1.2 and 1.3). We summarized the main results of this Section in the Table 8.2. The first column gives the value of l d of the uniform measure, the second column gives the result obtained with a bias of the form µ Θ0 (6.7) with a choice of parameter (8.17), when optimizing on the choice of to increase as much as possible l d . The third column gives the result obtained with a biased measure of the form µ Θ1 (6.12), for well-chosen values of (b 1 , b 2 , ). We can see that we were able to further improve the value of l d for k = 5 and k = 6 by using a bias that introduces interactions between variables of different hyperedges, with respect to the factorized bias. An even further improvement might be achieved by a more systematic exploration of the parameter space (b 1 , b 2 , ), or by using even more general bias functions ψ(p), at the price of a large computational cost due to the increased dimensionality of the parameter space. For comparison we give in the last column the satisfiability threshold, i.e. the smallest value of l such that the typical hypergraphs have no proper bicolorings, computed within the 1RSB ansatz (see [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF] for details), that is obviously an upper bound for l d , independently of the bias. Table 8.2: The values of l d for the uniform measure, for the bias of equation (6.7) that factorizes on the hyperedges, with the optimal value of , and for the bias of equation ( 6.12) for well-chosen parameters (b 1 , b 2 , ). The last column is the satisfiability threshold.

Chapter 9

The asymptotics of the clustering transition for the uniform measure

In the previous chapter we studied the possibility of tuning the value of the clustering threshold α d (k) by considering a non-uniform (biased) measure on the set of solutions of CSPs. For finite values of k we have seen that well-chosen biases allow indeed to increase α d (k), thus improving the range of α in which solutions of random CSPs can be found by efficient algorithms. It remains to see wether this improvement survives the large k limit, which requires being able to compute the asymptotics of α d (k) both in the unbiased and biased cases. The former is the object of this chapter, and the latter will be dealt in chapter 10.

The two examples of CSPs that will appear in this chapter are the graph coloring and the hypergraph bicoloring problems. Let us recall their definition (given in Section 1.1.4). In the first one the variables are placed on the vertices of a graph, they have q possible values, to be interpreted as colors, and each edge of the graph enforce the constraint that the two vertices at its ends take different colors. The hypergraph bicoloring problem (that was studied in the previous chapter) is similarly defined on a hypergraph, with hyperedges linking subsets of k (instead of two for a graph) vertices; the variables on the vertices can take two colors, and the constraint associated to each hyperedge is that both colors are present among its k adjacent vertices. The value of α d depends of course on the CSP under consideration and of its parameters (k, q in the two examples defined above). We have seen in chapter 7 that it can be estimated numerically by solving a functional equation (see equations (7.537.54) for the k-hypergraph bicoloring problem) that arises from the 1RSB formalism or its tree reconstruction interpretation (see for instance [START_REF] Zdeborová | Phase transitions in the coloring of random graphs[END_REF] for numerical values in the coloring case, and [START_REF] Dall'asta | Entropy landscape and non-gibbs solutions in constraint satisfaction problems[END_REF][START_REF] Gabrié | Phase transitions in the q-coloring of random hypergraphs[END_REF] for the hypergraph bicoloring). There is in general no explicit analytical expression for α d , but bounds on its value [START_REF] Mossel | Information flow on trees[END_REF][START_REF] Mézard | Reconstruction on trees and spin glass transition[END_REF] and asymptotic expansions for large k, q [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF] complement its numerical deter-mination (that becomes very difficult when k, q grow). In Section 7.4 we have presented a relatively simple upper bound on α d can be obtained by analyzing the so-called naive reconstruction procedure [START_REF] Semerjian | On the freezing of variables in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of random colourings[END_REF], where one asks whether the configuration of the variables at a large distance from a root variable implies unambiguously the value of the latter (instead of merely bringing some information on it). This property undergoes a phase transition at the "rigidity" threshold α r , with obviously α d ≤ α r , which can be alternatively interpreted as the appearance of "hard fields" in the solution of the 1RSB/reconstruction equations, or frozen variables taking the same value in all the solutions of a cluster. The analytic determination of α r is much simpler than the one of α d , because it corresponds to the bifurcation of a scalar (instead of functional) fixedpoint equation (see equation (7.79) for the k-hypergraph bicoloring problem). In particular, for the hypergraph bicoloring and coloring problems, its asymptotic expansion at large k, q shows that the relevant scale of constraint densities (for the graph coloring problem we use the more natural average degree c = 2α) is

α = α(k, γ) = 2 k-1 k (ln k + ln ln k + γ) , c = c(q, γ
) = q(ln q + ln ln q + γ) , (9.1) with γ a finite constant parameter, the rigidity transition α r occuring on this scale when γ crosses the critical value γ r = 1 (for both models). It turns out that the asymptotic behavior of α d is also on the scale of Eq. (9.1), with another constant γ d ≤ γ r . This statement follows from a series of rigorous works on this problem: for the coloring problem [START_REF] Sly | Reconstruction of random colourings[END_REF] proved that 1 -ln 2 ≤ γ d ≤ γ r = 1, and the strict inequality γ d < 1 was later obtained in [START_REF] Sly | Reconstruction of colourings without freezing[END_REF], implying the asymptotic existence of a regime [γ d , γ r ] where reconstruction is possible but naive reconstruction is not. A large family of models, including the two discussed here, was also adressed in [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF] which proved α d (k) ≥ 2 k-1 k ln k, c d (q) ≥ q ln q, confirming the leading term in the scaling (9.1).

We report in this chapter a study of the asymptotic expansion of α d for the graph coloring and hypergraph bicoloring problems that have been performed in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF], for the uniform measure. We will show that the transition indeed happens on the scale of constraint densities defined in (9.1), with the same value of γ d for both models, which is to some extent surprising given their different microscopic nature. We characterize γ d in terms of the behavior of a functional equation, whose numerical study yields the estimate γ d ≈ 0.871, while an analytical treatment provides the lower bound γ d ≥ 1+ln(2( √ 2-1)) ≈ 0.812. Even with the less demanding level of rigor of theoretical physics, to which we stick here, the computation is relatively involved because of the asymptotic proximity of the thresholds α d and α r , at the origin of a quite singular behavior of the probability distributions describing the intermediate regime of reconstruction without naive reconstruction.

Chapter is organized as follows. In Sec. 9.1 we perform the large k limit on the hypergraph bicoloring model and reduce the determination of γ d to the study of a reduced set of equations, in which k does not appear anymore. This reduced problem takes the form of recursive equations on a sequence of probability distributions, determining a correlation function at distance n. The study of the large n limit of this reduced problem, from which γ d can be finally deduced, is performed in Sec. 9.2; this limit requires some additional reparametrizations in the interesting regime. For simplicity in the main part of the text we concentrate on the hypergraph bicoloring case, and devote the Appendix B to the graph coloring problem: we show that the large q limit yields exactly the same reduced problem than the large k limit of the hypergraph bicoloring, hence the study of Sec. 9.2 and the determination of γ d is common to both. 9.1 The large k limit for a finite distance n

Evolution of the hard fields

Before studying the clustering threshold α d that corresponds to the bifurcation of the functional bifurcation equations (7.537.54) (with p(σ 1 , . . . ,

σ k-1 |+) = (1 -I[σ 1 = • • • = σ k-1 = +1])/(2 k-1 -1)
), it is much easier to consider the rigidity transition α r (k), that corresponds to the bifurcation of the scalar recursion equation (7.80). We have given in (7.81) the expressions that determine analytically α r (k) for all k. The value of Γ r (k) cannot be given explicitly when k is finite, but it is easy to perform its asymptotic expansion at large k, which yields

α r (k) = 2 k-1 k (ln k + ln ln k + γ r + o(1)) , with γ r = 1 . (9.2)
As explained above this is an upper bound on the sought for dynamic transition α d (k), and the rigorous results of [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF] suggest (or prove for the coloring problem [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF]) that the asymptotic expansion of α d (k) has the same form as the one of α r (k), with a different constant term γ d .

We shall thus study the large k limit taking α to diverge simultaneously with k according to the function α(k, γ) defined in Eq. (9.1), with γ a finite constant that becomes our control parameter for the density of constraints in this limit. It is instructive to investigate the behavior of the hard field weights H n , solution of the recursion equation (7.80), for a finite distance n. With this choice for the scaling of α with k one easily obtains by induction on n that the leading behavior of H n reads

H n = 1 - x n k ln k + o 1 k ln k , ( 9.3)
where x n is a γ-dependent sequence, defined recursively by a finite fixed point (depicted on the bottom panel of Fig. 9.3) (upper curve), while for γ < 1 the sequence x n diverges as n → ∞, very quickly (as iterated exponentials). The transition between these two behaviors thus occurs at the critical value of the parameter γ = γ r = 1, which corresponds to the expansion of α r (k) stated above. Note that this coincidence is not as trivial as it may look at first sight: the expansion of Eq. (9.2) was obtained by first taking the limit n → ∞, then k → ∞, while in (9.4) we have taken the limit k → ∞, and only later studied the behavior of x n for large n. The commutativity of these two limits can in this case be traced back to the simple shape of the recursion equation (7.80): either it has no non-trivial fixed point, or if it has one in the large k limit it must be of order 1 -O( 1)/(k ln k), and hence captured by the scaling (9.3).

x 0 = 0 , x n+1 = e -γ+xn . ( 9

The reduced order parameter

Our determination of the asymptotic behavior of α d (k) will be based on three hypotheses that we now spell out explicitly. First of all, we assume that this transition occurs on the scale (9.1), hence that the sought for constant γ d can be defined as

γ d = lim k→∞ kα d (k) 2 k-1 -ln k -ln ln k ; (9.5)
as mentioned above the rigorous results of [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF] support this hypothesis.

We shall furthermore assume that on this scale of α the order parameter C(α, k) is either equal to 0 (for α < α d (k)) or scales with k in the same way as its lower bound provided by the hard field weight H(α, k) (see Eq. (9.3)), hence define a reduced order parameter as

C(γ) = lim k→∞ (1 -C(α(k, γ), k))k ln k , ( 9.6)
that should be finite for γ > γ d , and diverge to +∞ for γ < γ d . Note that in order to have C(γ) ≥ 0 we introduced a minus sign in this definition, hence increasing values of C corresponds to smaller correlations in the original order parameter C. Finally we assume that C(γ) can be computed by reversing the order of the large n and large k limits, in other words we define for n finite C n (γ) as

C n (γ) = lim k→∞ (1 -C n (α(k, γ), k))k ln k , ( 9.7)
and our third hypothesis will consist of computing C(γ) as the large n limit of C n (γ).

To rephrase these last two hypotheses, we exclude the possibility that on the scale (9.1) the fixed point order parameter C(α, k) has distinct scalings with k depending on the value of γ, and similarly that the finite distance correlation function C n (α, k) decays with several plateaus at heights each scaling differently with k. These hypotheses are reasonable when confronted with the finite k numerical results (for instance those of Fig. 9.2), and are somehow corroborated by the scheme of proofs of [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF], that rely on the fact that once C n is "small enough", then it must decay to 0; we shall see in the rest of the paper that they are at least self-consistent. Note also that they hint at the intrisic difficulty of the computation: in the intermediate regime [α d (k), α r (k)] there are no hard fields in the fixed point distribution P (h), nevertheless the soft fields become asymptotically hard in the k → ∞ limit as the correlation function C tends to 1 according to (9.6).

Evolution of the soft fields distribution

In order to complete the computation of the reduced order parameter we need now to study the large k limit of the distributions Q n (h) of the soft fields, that obeys the recursion equations (7.91,7.93). Consider first the latter equation; the integer p that appears there is a random number drawn from the binomial distribution Bin(k -1, 1 -H n ). In the large k limit, because of the scaling (9.3) of H n , this distribution can be approximated by a Poisson distribution Po(x n / ln k), which hence concentrates on the smallest possible value that appears in the sum, namely p = 1. We thus obtain at the leading non-trivial order

Q n (u) = 1 - 3x n 2 k-1 ln k δ(u) + 3x n 2 k-1 ln k R n (u) , (9.8)
where R n (u) is a normalized distribution arising from the three contributions of the terms with p = 1 in (7.93),

R n (u) = 1 3 dQ n (h) [δ(u -g 1 (h)) + δ(u -g 1 (-h)) + δ(u + g 1 (-h))] , (9.9)
where g 1 (h) is obtained from (7.94) as

g 1 (h) = 1+h 2 2 -1+h 2 . ( 9.10)
Note that this function fulfills the identity 1-g1(h) 1+g1(h) = 1-h 2 ; as a consequence one can check that the Bayes symmetry is respected by R n , i.e. that R n (-u) =

1-u 1+u R n (u), if Q n verifies this symmetry.
Let us give an intuitive explanation of the equations (9.8,9.9): in the large k limit the weight of the hard fields H n is very close to 1, hence almost all of the k -1 fields h i acting on the variables of a hyperedge are forcing them. If they all force them to take the same color -1 then to satisfy the constraint the root must take the opposite color, which leads to an hard field u = 1 (the k -1 fields cannot be all forcing in the direction +1, this would force the root to be -1, in contradiction with the broadcast being conditional on the root equal to +1). If they are all hard but with the two colors represented, then the clause is satisfied for every choice of the root, and this produces a trivial soft field u = 0. The least unprobable situation that can produce a non-trivial soft field (u = 0, 1) is thus when among the k -1 entering fields h i exactly one, say h 1 , is soft, and all the k -2 other are hard. If the two colors appear among the k -2 hard fields, again the root is unbiased, this produces a trivial soft field u = 0. Assume thus that the k -2 hard fields are of the same color. If this color is +1, then necessarily σ 1 = -1 and this yields u = -g 1 (-h 1 ). If this color is -1, then both σ 1 = +1 (producing u = g 1 (-h 1 )) and σ 1 = -1 (corresponding to u = g 1 (h 1 )) are possible; this concludes the interpretation of (9.8,9.9).

We can now inject the simplified form (9.8) of Q n (u) in the other recursion equation (7.91); as f (u 1 , . . . , u l , 0, . . . , 0) = f (u 1 , . . . , u l ) the first term of Q n (u) does not contribute, and the number of R n (u i ) picked up in the r.h.s. of (7.91) is a Poisson random variable of parameter αk(1 -H n ) 3xn 2 k-1 ln k . This quantity converges to 3x n in the large k limit for the regime we are considering, hence (7.91) becomes:

Q n+1 (h) = ∞ l=0 e -3xn (3x n ) l l! l i=1 d R n (u i ) δ(h -f (u 1 , . . . , u l )) . (9.11)
We can finally express the reduced order parameter C n (γ) in the large k limit: combining its definition from (9.7), the expression of C n in terms of the soft fields distribution and of the hard fields weight given in Eq. (7.85), and the scaling of the latter quantity written in (9.3), we obtain C n (γ) = x n dQ n (h)(1 -h).

The limit of large distance n

Let us summarize what we have achieved up to now; starting from the recursion equations (7.53,7.54) for the distributions P n (h) we have taken the large k limit with the density of constraints scaling as α(k, γ) of Eq. (9.1), and argued for the existence of a reduced correlation function at finite distance, C n (γ) defined in (9.7). We have then shown that the latter could be computed by solving some recursion equations on both x n , a scalar quantity related to the weight of the hard fields in Eq. ( 9.3), and on Q n (h), the distribution of the soft fields (that is a distribution supported on [-1, 1], with no atoms in ±1). For the sake of readability we regroup now all the equations that will be necessary to proceed:

x 0 = 0 , x n+1 = e -γ+xn , (9.12)

Q n+1 (h) = ∞ l=0 e -3xn (3x n ) l l! l i=1 d R n (u i ) δ(h -f (u 1 , . . . , u l )) , (9.13) f (u 1 , . . . , u l ) = l i=1 1+ui 2 - l i=1 1-ui 2 l i=1 1+ui 2 + l i=1 1-ui 2 , Q 1 (h) = δ(h) , (9.14) R n (u) = 1 3 dQ n (h) [δ(u -g 1 (h)) + δ(u -g 1 (-h)) + δ(u + g 1 (-h))] , (9.15) g 1 (h) = 1+h 2 2 -1+h 2 , ( 9.16
)

C n (γ) = x n dQ n (h)(1 -h) . (9.17)
Let us emphasize that the parameter k has disappeared from these equations, that only depend on γ; moreover we show in the Appendix B that exactly the same equations describe the large q limit of the coloring problem. What remains to do now is to study the large n limit of these equations, as the threshold γ d will be determined according to the behavior of C n (γ) in this limit, namely C n (γ) → +∞ will signal that γ < γ d , while C n (γ) will remain finite for n → ∞ if γ > γ d .

The regime γ ≥ γ r = 1

Let us start by considering the case γ ≥ γ r = 1; then the sequence x n remains bounded for all n (its limit x(γ) being depicted as a function of γ on the bottom panel of Fig. 9.3), which translates the presence of hard fields in the fixed point solution of the corresponding finite k regime α ≥ α r . As a consequence the numerical resolution of (9.13) presents no difficulty and can be performed with the usual population dynamics algorithms explained in Sec. 8.1, Q n (h) being represented as an empirical distribution over a sample of h i 's. Figure 9.3 presents some numerical results obtained in this way; one can see on the top panel that C n (γ) converges to a finite limit as n grows, this limit value being drawn as a function of γ on the bottom panel. In this regime Q n converges (weakly) to a stationary distribution Q, solution of the fixed point equation obtained by replacing in (9.13) x n by its limit. The finite k inequalities

H n ≤ C n ≤ 1 becomes in these rescaled units 0 ≤ C n (γ) ≤ x n (γ); indeed, dQ n (h)(1 -h) = 1 -dQ n (h)h = 1 -dQ n (h)h 2 ∈ [0, 1] , ( 9.18)
because of the consequence stated in (7.59) of the Bayes symmetry enjoyed by Q n . In the large n limit we thus have C(γ) ≤ x(γ), as illustrated in the bottom panel of Fig. 9.3 which is the analog of the bottom panel of Fig. 9.2 for finite k.

Note that C(γ) seems to behave smoothly as γ → γ + r , as expected if the transition occurs at γ d < γ r . As x has a square root singularity this means that there is also a square root singularity in dQ(h)(1 -h) as a function of γ, in order for their product C(γ) to be regular.

The difficulties for γ < 1

The study of the large n limit is much more difficult when γ < 1: the sequence x n solution of (9.12) is then divergent (asymptotically very strongly, as iterated exponentials). As the direct implementation of the population dynamics procedure to solve (9.13) involves the manipulation of a number of fields u i 's that is distributed as a Poisson random variable of parameter x n , this becomes very quickly impossible to handle. A first hint of what happens for γ < 1 can nevertheless be unveiled by this numerical procedure: when γ is only slightly smaller than 1, x n grows rather slowly at first, the range of n that can be treated in this way is then sufficiently large to gain some useful informations. A typical result obtained in this way is plotted on Fig. 9.4 for γ = 0.96: even though x n is diverging it is tempting to conjecture from this plot that C n admits a limit when n → ∞. Looking at the expression (9.17) of C n one realizes that this is possible provided dQ n (h)(1 -h) tends to 0 as n grows, with an order of magnitude inversely proportional to the one of x n . This weak convergence of Q n (h) to δ(h -1) is compatible with the form of the recursion equation (9.13), the combination of a very large number of u i 's can indeed produce a field h very close to 1.

Lowering further the value of γ the speed of the divergence of x n increases so much that an extrapolation of C n to large n values becomes impossible with this numerical method, which as a consequence does not allow for an accurate estimation of γ d . We thus need to devise a formalism that incorporates the compensation mechanism between the divergence of x n and the concentration of Q n around δ(h -1) at the origin of the finite value of C. A natural idea would be to make the change of variable h = 1 -(h /x n ), with h a random variable that should remain finite and thus obey a simplified recursion equation. Unfortunately the Bayes symmetry enjoyed by Q n , and its consequences on the moments of h, imply that h has higher moments that diverge with n. This strategy seems thus rather unpractical. In fact it is more probable that the value of C n is determined by contributions of all values of h in (-1, 1), even very improbable ones where Q n is of order 1/x n , while the typical values around h = 1 do not contribute to C n because of the factor 1 -h in the integral in (9.17). In more technical terms we conjecture that x n Q n , which is a sequence of positive measures, even if not probability ones because their total mass is the diverging sequence x n , does converge to a positive measure η, in the sense of for all -1 < a < b < 1 which are continuity points of η. A direct numerical test of this conjecture, and the determination of η, seems rather difficult as η is not a probability measure and may have an infinite total mass. We have thus devised an alternative numerical procedure that allowed us to explore the regime γ < 1, as we shall now explain.

Reweighted probability distributions

The idea underlying this procedure is to study the evolution not of the probability measure Q n , but of a reweighted (or tilted) version of it, that give less importance to the typical (for Q n ) values of h around h = 1, which do not contribute to C n . Let us thus consider a positive function r n (h), and define the reweighted distribution ρ n (h) = x n r(h)Q n (h). In more precise mathematical terms ρ n is absolutely continuous with respect to Q n , with relative density x n r. For the reasons explained above our goal is to use a function r(h) that vanishes in h = 1; a convenient choice is to take r of the form r(h) = ((1 -h)/(1 + h)) t , with t an exponent that is arbitrary for the moment. Indeed with this choice the reweighting will factorize in (9.13), in the sense that r(f (u 1 , . . . , u l )) = r(u 1 )ו • •×r(u l ). We will actually further specify the function by taking the exponent t = 1/2, thus setting

ρ n (h) = x n Q n (h) 1 -h 1 + h 1 2 . ( 9.20)
The reasons for this choice of the exponent are two fold: the Bayes symmetry (7.58) enjoyed by Q n implies then that ρ n is symmetric in the usual sense, i.e. ρ n (-h) = ρ n (h). Furthermore (7.58) also implies the following identity (used in the proof of lemma 4.4 in [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF]),

dQ n (h) 1 -h 1 + h 1 2 = dQ n (h) 1 -h 2 , ( 9.21)
which can be obtained from (7.58) with the choice b(h

) = h((1 -h)/(1 + h)) 1/2 .
The right hand side of (9.21) is obviously between 0 and 1, thus for all finite n the measure ρ n has a finite total mass, smaller than x n . We shall therefore define a probability measure ν n by dividing ρ n by its mass m n :

m n = dρ n (h) = x n dQ n (h) 1 -h 1 + h 1 2 , ν n (h) = 1 m n ρ n (h) . (9.22)
Note that the three descriptions in terms of the (finite but not normalized to 1) positive measure ρ n , or the pair (m n , ν n ), or the original one in terms of (x n , Q n ), are strictly equivalent for all finite n; in particular one can compute the reduced correlation function as

C n (γ) = dρ n (h) 1 -h 2 = m n dν n (h) 1 -h 2 . (9.23)
Besides its equivalence with the original quantities at finite n the advantage of the reweighted description in terms of ρ n is a much nicer behavior when n grows, as we shall see.

Let us now derive the recursion equations that governs the evolution of ρ n (or equivalently of the pair (m n , ν n )). It will be more convenient to express some quantities in terms of h = arcth(h) and u = arcth(u), that corresponds to the effective magnetic fields associated to the magnetizations h and u. In particular the operation h = f (u 1 , . . . , u l ) translates in this domain to a simple addition, h = u 1 + • • • + u l , and the reweighting r(h) becomes e -h . For the sake of conciseness we shall take the liberty of using simultaneously both types of quantities in the following equations, keeping implicit the relation h = arcth(h) between them; we will also denote with the same symbol Q n the measure for the random variables h and h related in this way.

Let us first rewrite the equations (9.13,9.15) in terms of h: (9.25) where we introduced the function

Q n+1 ( h) = ∞ l=0 e -3xn (3x n ) l l! l i=1 d R n ( u i ) δ( h -u 1 -• • • -u l )) , (9.24) R n ( u) = 1 3 dQ n ( h) δ( u -a( h)) + δ( u -a(-h)) + δ( u + a(-h)) ,
a( h) = arcth(g 1 (tanh( h))) = 1 2 ln 1 + e 2 h . (9.26)
Under Q n+1 the random variable h has a compound Poisson distribution, it is thus more convenient to describe it in terms of its characteristic function (i.e. Fourier transform). We define the latter for the various measures as

Q n (z) = dQ n (h)e iz h , ρ n (z) = dρ n (h)e iz h , ν n (z) = dν n (h)e iz h = 1 m n ρ n (z) . (9.27)
With these conventions the two equations (9.24,9.25) become: h) + e -iza(-h) .

Q n+1 (z) = exp -3x n + x n dQ n (h) e iza( h) + e iza(-
(9.28) The effect of the reweighting is easily seen to translate in terms of the characteristic functions as a shift of the argument,

ρ n+1 (h) = x n+1 e -h Q n+1 (h) ⇔ ρ n+1 (z) = x n+1 Q n+1 (z + i) . ( 9.29)
Recalling that x n+1 = exp[-γ + x n ] we thus obtain ρ n+1 (z) = exp -γ + x n dQ n (h) e -a( h) e iza( h) + e -a(-h) e iza (-h) +e a(-h) e -iza(-h) -2 (9.30)

Replacing in this equation x n Q n by e h ρ n yields ρ n+1 (z) = exp -γ + dρ n (h) e h-a( h) e iza( h) + e h-a(-h) e iza (-h) +e h+a(-h) e -iza(-h) -2e h (9.31)

= exp -γ + dρ n (h) 1 + h 2 e iza( h) + 1 + h 1 -h 1 + h 2 e iza(-h) + 2 1 + h e -iza(-h) -2 (9.32)
where in the second line we expressed the weight factors in terms of h instead of h, using the expression of a( h) given in (9.26). As ρ n (h) = ρ n (-h) one can symmetrize the integrand to obtain

ρ n+1 (z) = exp -γ + dρ n (h) 1 2(1 + h) e iza( h) + e -iza( h) + 1 2(1 -h) e iza(-h) + e -iza(-h) - 2 √ 1 -h 2 = exp -γ + dρ n (h) 2 1 -h e iza(-h) + e -iza(-h) - 2 √ 1 -h 2 .
(9.33)

These two expressions for ρ n+1 (z) are obviously invariant under z → -z, which enforces the conservation of the symmetry ρ n+1 (h) = ρ n+1 (-h) along the iterations. The integrand of the first one is explicitly symmetric in h, while the more compact second one has been obtained by exploiting the symmetry of ρ n .

We have thus obtained a recursion equation for the reweighted measure ρ n , that is equivalent to the original system on (x n , Q n ). It will be useful to spell out its translation in terms of the mass m n of ρ n and its normalized version ν n . By setting z = 0 in (9.33) we obtain

m n+1 = exp -γ + dρ n (h)β(h) , ( 9.34)
with

β(h) = 2 1 + h + 2 1 -h - 2 √ 1 -h 2 . ( 9
ν n+1 (z) = exp m n dν n (h) 2 1 -h e iza(-h) + e -iza(-h) -2 , (9.37)
completed by the initial condition m 1 = e -γ , ν 1 (h) = δ(h), that derives directly from x 1 = e -γ , Q 1 (h) = δ(h).

Numerical resolution

There is one last difficulty to face in the derivation of a numerical procedure able to solve the equations (9.36,9.37), that we shall now explain. As ν n is a normalized probability measure we can represent it as a large sample of fields, according to the population dynamics strategy explained in Sec. 8.1. Suppose this has been done up to the n-th iteration, and that an estimate of the real number m n is also known. Then m n+1 can be easily computed from (9.36), the integration over ν n being estimated as an empirical average over the representative population. There remains to give an interpretation of the law ν n+1 whose characteristic function is given in (9.37), and to devise a sampling procedure from it in order to produce the population of i.i.d. samples that will represent ν n+1 . In order to do so let us introduce a positive (not normalized) measure π n and its total mass λ n according to

π n (y) = m n dν n (h) 2 1 -h (δ(y -a(-h)) + δ(y + a(-h))) (9.38) λ n = 2m n dν n (h) 2 1 -h , ( 9.39)
in such a way that (9.37) can be rewritten ν n+1 (z) = exp dπ n (y)(e izy -1) . (9.40)

In more technical terms π n is the Lévy measure associated to the infinitely divisible distribution ν n+1 [START_REF] Sato | Lévy Processes and Infinitely Divisible Distributions[END_REF], with λ n the total mass of π n , which is finite for a Poisson compound distribution. This form makes clear that under ν n+1 the random variable h can be described as the distributional equality h d = l i=1 y i , where in the right hand side l is a Poisson random variable of mean λ n , and the y i 's are i.i.d. copies drawn with the probability measure π n (y)/λ n . If ν n is known as a sample of fields it is possible to sample efficiently y from π n /λ n , by extracting a field h (n) i with a probability proportional to (1 -h (n) i ) -1/2 and then setting y = ±a(-h (n) i ), the two signs being chosen with probability 1/2. It could thus seem at first sight that the distributional interpretation given above leads to a valid numerical procedure to solve (9.36,9.37). A closer look reveals that this strategy cannot work as it is for γ < 1. Let us indeed rewrite the expression of λ n by exploiting the symmetry of ν n :

λ n = m n dν n (h) 2 1 -h + 2 1 + h = m n dν n (h) √ 2( √ 1 + h + √ 1 -h) √ 1 -h 2 . (9.41)
On the other hand the sequence x n can be expressed from the definition (9.20) of ρ n as

x n = dρ n (h) 1 + h 1 -h = m n dν n (h) 1 2 1 + h 1 -h + 1 -h 1 + h = m n dν n (h) 1 √ 1 -h 2 , ( 9.42)
where we used the normalization of Q n and the symmetry of ν n . Noting that the function [START_REF] Garey | Computers and intractability: A guide to the theory of NP-completeness[END_REF] we can conclude by comparing these two expressions of λ n and x n that 2x n ≤ λ n ≤ 2 √ 2x n . Hence for γ < 1 the lower bound implies that λ n diverges with n, which brings us back to the situation we are trying to avoid of a Poisson compound distribution with a diverging number of summands.

√ 2( √ 1 + h + √ 1 -h) is bounded between 2 and 2 √ 2 when h ∈ [-1,
Fortunately the reweighting performed above will allow us to circumvent this difficulty. One sees indeed on the expression (9.39) that the divergent contribution to λ n arises from the neighborhood of h = 1, i.e. of h = +∞. However the corresponding summands ±a(-h) are very small (recall the expression of a from Eq. (9.26)), it is thus conceivable that such a sum of a very large number of very small contributions can be handled in a simplified way. To formalize this intuition we rewrite (9.37) as

ν n+1 (z) = ν (≤) n+1 (z) ν (>) n+1 (z) , (9.43) ν (≤) n+1 (z) = exp dρ n (h) 2 1 -h e iza(-h) + e -iza(-h) -2 I(h ≤ 1 -ε n ) , (9.44)
ν (>) n+1 (z) = exp dρ n (h) 2 1 -h e iza(-h) + e -iza(-h) -2 I(h > 1 -ε n ) , (9.45)
where ε n is for the moment an arbitrary threshold; such a decomposition is loosely inspired by an analogy with the Lévy-Itô decomposition of Lévy processes [START_REF] Sato | Lévy Processes and Infinitely Divisible Distributions[END_REF]. These two contributions to ν n+1 (z) are characteristic functions of probability distributions, hence under the law ν n+1 the random variable h is the sum of independent draws from ν

(≤) n+1 and ν (>)
n+1 . The first one can be described as above, modulo the introduction of the cutoff ε n ; defining n . The law of ν (>) n+1 will instead be described via its cumulants, to be denoted c n+1,p for the p-th one. By taking derivatives with respect to z of ln ν

π (≤) n (y) = m n dν n (h) 2 1 -h (δ(y -a(-h)) + δ(y + a(-h)))I(h ≤ 1 -ε n ) , (9.46) λ (≤) n = 2m n dν n (h) 2 1 -h I(h ≤ 1 -ε n ) , ( 9
(>) n+1 (z) one finds easily that c n+1,p = m n dν n (h) 2 1 -h (1 + (-1) p ) a(-h) p I(h > 1 -ε n ) . (9.48)
As expected from the symmetry of the law ν

(>)
n+1 all its cumulants of odd order vanish and one can write for the even ones:

c n+1,2p = 2m n dν n (h) 2 1 -h a(-h) 2p I(h > 1 -ε n ) . (9.49)
The strategy we have followed in our numerical resolution is to choose ε n large enough such that λ (≤) n does not grow with n, in such a way that the contribution from ν (≤) n+1 can be generated with a finite Poissonian number of summands, but small enough so that ν (>) n+1 can be safely approximated as a Gaussian distribution, neglecting the cumulants c n+1,2p for p > 1 (there exist some ways to draw better approximations of infinitely divisible distributions by using more cumulants [START_REF] Asmussen | Approximations of small jumps of lévy processes with a view towards simulation[END_REF][START_REF] Zhiyi | Nonnormal small jump approximation of infinitely divisible distributions[END_REF], but we did not try to implement them). Let us give an explicit description of the algorithm. At the n-th iteration we assume to have an estimation of m n and of ν n as a population, namely

ν n (h) ≈ 1 N N i=1 δ( h -h (n) i ) . ( 9.50)
We will assume that the population has been sorted, h

(n) 1 ≤ h (n) 2 ≤ • • • ≤ h (n)
N , and translate the cutoff ε n by defining N n in such a way that h

(n) Nn ≤ 1 -ε n < h (n)
Nn+1 . We can thus estimate all integrals with respect to ν n according to

ν n (h)F ( h) ≈ 1 N N i=1 F ( h (n) i ) (9.51)
for all functions F , and in particular this gives us m n+1 from (9.36). We then compute

λ (≤) n ≈ 2m n 1 N Nn i=1 2 1 -h (n) i , c n+1,2 ≈ 2m n 1 N N i=Nn+1 2 1 -h (n) i a(-h (n) i ) 2 . (9.52)
We then construct the new population elements h (this can be done efficiently by precomputing a cumulative table);

• set h (n+1) i = δ 1 a(-h (n) i1 ) + • • • + δ l a(-h (n) i l ) +
√ c n+1,2 z where the δ i 's are ±1 with probability 1/2, independently from each other and from anything else, and z is a standard Gaussian random variable.

We can then compute the reduced correlation function C n+1 from (9.23), and sort the elements of the new population to be ready for the next step.

In our implementation we chose the threshold ε n , or equivalently N n , in an adaptive way: at each step we take the largest value of N n that maintains λ (≤) n computed from (9.52) smaller than a value λ fixed beforehand and kept constant along the iterations. The numerical accuracy of our procedure is thus limited by the finiteness of N (the representation (9.50) becomes exact only in the limit N → ∞) and of λ (because of the Gaussian approximation we perform for ν (>) n+1 ). The numerical results presented in figures have been obtained with N = 10 7 and λ = 20; we checked that varying λ between 10 and 30, or reducing N to 10 6 , did not affect our conclusions in a quantitative way.

On the top panel of Fig. 9.5 we show that this new method allows us to track the evolution of C n for values of γ deep in the regime γ < 1 that was unaccessible to the simplest strategy presented in Sec. 9.2.1. In particular for γ 0.88 there is a clear plateau in C n , the value of which was reported as a function of γ in the bottom panel of Fig. 9.3 along with the results previously obtained for γ ≥ 1. Note that the new algorithm is also valid for γ > 1 (and in this case the cutoff ε n can be safely put to 0), we found as expected a perfect agreement between the results of the two procedures in this case.

On the bottom panel of Fig. 9.5 we have plotted the evolution of (the inverse of) m n as a function of n, that presents the same kind of plateau behavior as C n . These numerical results suggest that C n diverges with n if and only if m n diverges, and hence that γ d can be defined as the smallest value of γ such that m n remains bounded when n → ∞. We have not been able to prove analytically this statement, that will be taken as an additional hypothesis in the rest of the paper. This conjecture could be wrong if there existed a regime of γ such that C n remains finite while m n diverges, because of the existence of another reweighting scheme that deals with the compensation between the divergence of x n and the convergence of Q n (h) to δ(h -1) in a more efficient way than the one we introduced in (9.20). We found numerically no trace of such a phenomenon, which if it existed would only modify our estimate of γ d by a quantitatively very small amount, as for γ = 0.87 the finite n study strongly suggests a common divergence of C n and m n (and for γ = 0.86 the divergences are obvious on Fig. The relationship between the parameter γ and the mass m of the reweighted measure ρ in the n → ∞ limit. The numerical results suggest that m reaches 1 with a square root behavior as γ tends to γ d .

The fixed point equation and the determination of γ d

The numerical results presented above suggest that for γ ≥ γ d both m n and C n converge when n → ∞. It is then tempting to conjecture that the probability distribution ν n itself converges to some ν, in other words that we can remove the indices n from (9.36,9.37) and get a system of self-consistent equations on the fixed-point (m, ν) as:

m = exp -γ + m dν(h)β(h) , ( 9.53)
ν(z) = exp dπ(y)(e izy -1) , (9.54)

π(y) = m dν(h) 2 1 -h (δ(y -a(-h)) + δ(y + a(-h))) . (9.55)
Let us make a few remarks on these equations; first, one can check that if m is finite and ν is a probability measure with no atoms in h = ±1 (i.e. in h = ±∞), then the measure π defined in (9.55) gives a finite mass to all sets that are bounded away from 0, does not have an atom in 0, and is such that dπ(y)|y|I(|y| ≤ 1) < ∞. This ensures that the integral in the definition of ν(z) converges, hence π is a valid Lévy measure for the infinitely divisible distribution ν. Moreover the assumption of convergence of ν n towards ν is self-consistent in the following sense: if ν n w → ν (i.e. in the sense of weak convergence of probability measures) then π n (f ) → π(f ) for all bounded continuous functions f that vanish in a neighborhood of 0, and lim →0 lim sup n→∞ dπ n (y)y 2 I(|y| ≤ ) = 0. These two properties allow to invoke theorem 8.7 of [START_REF] Sato | Lévy Processes and Infinitely Divisible Distributions[END_REF] to conclude that ν n w → ν, the infinitely divisible distribution with Lévy measure π. Of course this circular argument does not prove the convergence ν n w → ν, but at least shows its consistency.

Note that in (9.53-9.55) we have finally achieved the main goal of the article, namely getting rid of the parameters k and n that have been sent to infinity and obtaining a system of equations that only depend on the parameter γ. Moreover γ only appears in (9.53): one can thus view (9.54,9.55) as a self-consistent equation on ν, parametrized by m, from which one deduces γ according to (9.53). We have performed this numerical resolution at fixed m, relying again on the Gaussian approximation to deal with the divergence of the total mass of π, and checked that the results were in perfect agreement with the one obtained fixing γ in the n → ∞ limit. In Fig. 9.6 we have plotted the correspondance function between γ and m. The numerical resolution of the fixed point equation (9.55) led us to the following observations: (i) a solution of the equation only exists for m ≤ 1 (ii) the tail behavior of ν is very well described numerically by

ν({ h ≥ x}) ∼ S(x) e -r(m)x x ρ as x → +∞ , ( 9.56)
where S is a slowly varying function, ρ is an exponent very close to 1 for all the values of m we have considered, and r(m) ≥ 0 controls the dominant exponential decrease of the tail (iii) r(m) > 0 for m < 1, and r(m) → 0 as m → 1. Some of these numerical observations can be rationalized through the following analytical considerations. If one assumes the form (9.56), the exponential behavior of ν when h → +∞ (and also in -∞ thanks to the symmetry of ν) allows to continue ν(z) from the real to the imaginary axis, i.e. to define the Laplace transform of ν as

L(t) = ν(-it) = dν(h)e t h = dν(h) cosh(t h) . (9.57)
This function L is well-defined for t ∈ (-r(m), r(m)), formally infinite for |t| > r(m), with L(0) = 1 and L non-decreasing on [0, r(m)). Its behavior as t → r(m) depends on the exponent ρ that controls the algebraic decay in (9.56). If ρ ≤ 0 then L diverges in this limit, with a dominant behavior of (r(m) -t) ρ (and a prefactor that involves the slowly varying function S, see theorem XIII.5.1 in [START_REF] Feller | An introduction to probability theory and its applications[END_REF] for a rigorous statement of such a Tauberian argument), while if ρ > 0 the limit L(r(m)) is finite. We claim that for the probability measure ν solution of ( We must thus have ρ > 0, in such a way that L is finite in r(m); nevertheless some derivatives of L will be divergent in r(m), each derivative with respect to t in (9.57) multiplying the integrand by h, hence reducing the exponent ρ by 1. Suppose for simplicity that 0 < ρ ≤ 1, in such a way that L (t) is divergent (higher values of ρ would lead to the same conclusion by considering higher order derivatives). Because of the proportionality between the tails of ν and π the function

R (t) is also divergent, with R (t)/L (t) → m as t → r(m). Since L (t) = R (t) exp[R(t)] = R (t)L(t)
, we are led to the conclusion that 1 = m L(r(m)) . (9.58) This type of reasoning can actually be put on rigorous grounds, the statement (9.58) being essentially the content of corollary 2.1 in [START_REF] Watanabe | Ratio of the tail of an infinitely divisible distribution on the line to that of its lévy measure[END_REF]. As L(r(m)) ≥ L(0) = 1 we can conclude that a solution of (9.55) with a tail behavior of the form (9.56) can only exist for m ≤ 1, and that r(m) must vanish when m → 1, which confirms part of the numerical observations reported above. As m is a decreasing function of γ (see in particular Fig. 9.6) we are thus led by (9.53) to the following definition for our conjectured value of γ d :

γ d = dν(h)β(h) , (9.59)
where ν is the solution of (9.55) with m = 1 (and hence has a power law decay when h → ∞). A numerical evaluation of this quantity led us to the estimate γ d ≈ 0.871. The property m → 1 as γ → γ d was illustrated with the dotted horizontal line in the bottom panel of Fig. 9.5, that corresponds to the expected height of the plateau of the curves in the critical region.

The square root behavior of m as a function of γ when γ is close to γ d , that is clearly visible on Fig. 9.6 and confirmed by a numerical fit of the data, is in line with the bifurcation of the functional equation (9.55). It is more surprising to observe that C(γ) seems to behave linearly in the limit γ → γ d (see the bottom panel of Fig. 9.3), as it could suggest that C admits a continuation to smaller values of γ, in violation of the hypothesis made at the end of Sec. 9.2.4. There are alternative explanations to this observation: the square root contribution could be very small and not observable with our numerical accuracy, or the expected square root behavior of C(α, k) when α → α d at finite k could be washed out in the limit k → ∞, i.e. the limits α → α d and k → ∞ do not necessarily commute to correspond to the limit γ → γ d .

An analytic lower bound on γ d

As we have seen above our conjecture for γ d is given in terms of the solution of a functional equation that does not seem to admit a simple expression. One can however derive explicitly a quantitatively close lower bound, by observing that the function β(h) defined in (9.35)

is such that β(h) ≥ 2( √ 2 -

Chapter 10

The asymptotics of the clustering transition for biased measures

In this chapter, we investigate the large k asymptotics of the clustering transition undergone by the bicoloring of k-uniform random hypergraphs, when its solutions are weighted non-uniformly, with a soft interaction between variables belonging to distinct hyperedges. We find that α d (k) = 2 k-1 k (ln k + ln ln k + γ d + o(1)), where the constant γ d is strictly larger than for the uniform measure over solutions.

As already mentioned in chapter 5, an important problem in the field of random CSPs concerns the behavior of algorithms in the satisfiable regime, where the goal is to find a solution, as typical instances admit such configurations. In particular, one would like to determine the algorithmic thresold α alg (k) above which no algorithm is able to find a solution with high probability (assuming P =NP). We have seen (in section 5.1) that for small values of k it is possible to design algorithms (see [START_REF] Mézard | Analytic and algorithmic solution of random satisfiability problems[END_REF][START_REF] Selman | Noise strategies for improving local search[END_REF][START_REF] Ardelius | Behavior of heuristics on large and hard satisfiability problems[END_REF][START_REF] Alava | Circumspect descent prevails in solving random constraint satisfaction problems[END_REF][START_REF] Marino | The backtracking survey propagation algorithm for solving random k-sat problems[END_REF]) for which a numerical study shows that they are efficient up to densities very close to the satisfiability threshold. In the large k limit these algorithms cannot be studied numerically. It is however possible to perform the analytical study of some algorithms based on simpled heuristics. Up to know the best result is the one of [START_REF] Coja-Oghlan | A better algorithm for random k-sat[END_REF], which provides an algorithm that provably works in polynomial time up to densities of the order of 2 k-1 ln k/k, which coincides with the asymptotic scaling of α d . We recall that the satisfiability threshold occurs at the scaling α sat (k) ∼ 2 k-1 ln 2, therefore this leaves a wide range of α where typical instances do admit solutions, but no known algorithm is able to find them efficiently. Moreover some families of algorithms have been proven to fail [START_REF] Gamarnik | Performance of sequential local algorithms for the random nae-k-sat problem[END_REF][START_REF] Coja-Oghlan | Walksat stalls well below satisfiability[END_REF][START_REF] Hetterich | Analysing survey propagation guided decimation on random formulas[END_REF].

Although the connection between the algorithmic performances and the several structural phase transitions that occur in the satisfiable phase is very delicate, the strategy adopted in this Ph.D was to study the clustering threshold α d . One can reasonably state that the clustering transition is a lower bound on the algorithmic one, α d (k) ≤ α alg (k). Indeed for α < α d , simulated annealing is able to equilibrate in polynomial time down to arbitrary small temperatures [START_REF] Montanari | Rigorous inequalities between length and time scales in glassy systems[END_REF], and hence to sample the solution set (with a biased or uniform measure). For α slightly larger than α d one expects simulated annealing to fall out-ofequilibrium in polynomial time scales, but in many cases it should be able to find solutions, as we have seen in section 8.4 for the biased measure µ Θ0 . Hence the bound α d (k) ≤ α alg (k) is not tight in general.

In chapter 8 we have studied two implementations of the bias in the measure on the set of solution of a hypergraph bicoloring instance for small values of k. For the simplest biased measure µ Θ0 , the interactions can be factorized over the bicoloring constraints. The second biased measure µ Θ1 induces interactions between variables at distance 1. We studied the modifications of the clustering threshold α d induced by the non-uniformicity between the solutions. We showed, for k between 4 and 6, that with well-chosen parameters the bias allows to increase α d , and that it can be further increased with the biased measure µ Θ1 compared to the simpler implementation µ Θ0 . We checked for µ Θ0 that the bias could improve the performances of simulated annealing.

The large k behavior of α d is more involved. We presented some rigorous bounds on its asymptotic expansion in Section 5.2.1 (see [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF]). In chapter 9 (that report the results of [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF]) we found that the clustering threshold occurs on the scale α ∼ 2 k-1 (ln k + ln ln k + γ)/k, with γ constant, and more precisely for the uniform measure γ d,u ≈ 0.871, which falls into the range allowed by the previous bounds [START_REF] Sly | Reconstruction of random colourings[END_REF][START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF][START_REF] Sly | Reconstruction of colourings without freezing[END_REF]. In this chapter we adapt the large k expansion of chapter 9( [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF]) to the biased measures µ Θ0 , µ Θ1 and manage to assess the asymptotic effect of the bias on α d . We find that the factorized bias of µ Θ0 (see equation (6.7)) studied in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] cannot improve on the constant γ d in the asymptotic expansion with respect to γ d,u , while the biased measure µ Θ1 with larger interaction range allows to increase its value up to γ d ≈ 0.977. This is arguably a modest improvement, bearing on the third order of the asymptotic expansion of α d , nevertheless it opens the possibility to study further generalizations of the bias and to bring some light on the nature of the algorithmic gap between α alg and α sat .

This chapter is organized as follows. In Section 10.1 we derive a first upper bound on the possible increase of the dynamic threshold for the factorized measure µ Θ0 , that was obtained in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF]. Although we will see later that this bound is not tight, it already indicates that there is no possibility to improve on the first term of the asymptotic expansion of α d . Moreover, it motivates the choice (8.17) for the bias parameters ω p defining the bias. The analytical expansion of the dynamic transition threshold in the large k limit for the biased measure is presented in Sections 10.2-10.5. These sections present the results obtained in [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF]. We will focus on the (k, l + 1)-regular ensemble, where the probability is uniform on the set of hypergraphs for which each vertex belongs to l + 1 hyperedges. We have seen indeed in chapter 7 that the study of the biased measure µ Θ1 that introduces interaction between variables at distance 1 on the regular ensemble is simplified compared to the Erdős Rényi ensemble G N (k, M ). In Section 10.2 we give the main ideas of the computations made in the next sections, and simplify some of the equations that will be needed in the following. In Section 10.3 we perform the large k limit of the recursive distributional equations (7.67-7.69), and reduce the determination of γ d to a set recursive distributional equations that does not depends on k. The study of the large n limit of these equations is performed in Section 10.4. In section 10.5 we give the numerical results obtained fom the resolution of these distributional equations that allows to compute the asymptotic expansion of α d .

A first upper-bound for the intra-clause bias

The numerical resolution of the cavity equations presented in Sec. 8.3 shows that for small values of k one has α opt (k) > α d,u (k), in other words that distorting the measure over solutions can make it RS for larger densities of constraints than the uniform one. We want now to investigate the large k limit, for which the gap between the satisfiability threshold and the algorithmic one is most clearly demonstrated. In this section we will derive an asymptotic bound on the large k behavior of α opt (k), for the biased measure µ Θ0 with the specific choice (8.17).

As explained before for any one has α d ( ) ≤ min(α KS ( ), α r ( ), α s=0 ( )), these three upper bounds having simple expressions given in Eqs. (8.18-8.20). At large enough k it is easy to convince oneself that the Kesten-Stigum transition occurs after α s=0 , hence is completely irrelevant (the dominant term in the asymptotic expansion of α KS is of the order 2 2(k-1) instead of 2 k-1 for α s=0 and α r ). We show in Fig. 10.1 the lines α r ( ), α s=0 ( ) for a large value of k. We define (α * (k), * (k)) as the coordinates of the intersection of the rigidity and the zero-entropy line, in such a way that α opt (k) ≤ α * (k): for larger densities either the RS entropy is negative, or there exists a 1RSB solution with hard-fields (or both), in any case no RS phase can exist for α ≥ α * . We will now derive an asymptotic expansion at large k of this upper bound α * (k).

In the expression (8.20) of α r the coefficient Γ r (k) is a series depending solely on k, that was defined in Eq. (7.81); in order to obtain more easily its asymptotic equation we define a series w k by H r (k) = 1 -e -w k , in such a way that w k is solution of the implicit equation e w = 1 + (k -1)w. Taking the logarithm of this equation and iterating once yields

w k = ln k + ln ln k + O ln ln k ln k . (10.1) One can then compute Γ r (k) = w k 1 + 1 (k -1)w k k-1 = ln k + ln ln k + O(1) . (10.2)
This gives immediately the expansion of the rigidity threshold for the uniform that is confirmed by the asymptotic expansion (5.13) (see [START_REF] Semerjian | On the freezing of variables in random constraint satisfaction problems[END_REF] and [START_REF] Sly | Reconstruction of random colourings[END_REF] in which the third term (O(1)) is also derived: γ r = 1). We come back to the determination of (α * (k), * (k)); this intersection of the rigidity and zero-entropy line is solution of the two following equations, immediately obtained from (8.18,8.20):

α r (k, ǫ) α s=0 (k, ǫ) (α * (k), ǫ * (k)) α/2 (k-1)
α = ln 2 k(1-) ln(1-) 2 k-1 -1-k -ln 1 -1+k 2 k-1 = 1 k Γ r (k) 2 k-1 -1 -k 1 - . ( 10.4)
Neglecting only exponentially small corrections we can simplify (10.4) into

α 2 k-1 = ln 2 k(1 -) ln(1 -) + 1 + k = 1 k Γ r (k) 1 1 - . ( 10.5)
Without making additional approximations we see that * (k) is solution of

+ (1 -) ln(1 -) = ln 2 Γ r (k) - 1 k - ln 2 Γ r (k) . ( 10.6)
Given the asymptotic behavior of Γ r (k) stated in (10.2) it is easy to see that * (k) must vanish in the limit; one can thus expand the l.h.s. of (10.6) and

189 obtain * (k) = 2 ln 2 Γ r (k) + O 1 ln k . ( 10.7)
Reinserting in (10.5) we obtain

α * (k) = 2 k-1 1 k Γ r (k) 1 + 2 ln 2 Γ r (k) + O 1 ln k . ( 10.8)
Using the expansion (10.2) of Γ r (k) we have finally

α * (k) = 2 k-1 1 k ln k + √ 2 ln 2 √ ln k + O(ln ln k) .
(10.9)

The comparison with the expansion of α r,u (k) given in (10.3) shows that the leading order is not modified, the term ln ln k in the correction being replaced by a (larger) term of order √ ln k. In chapter 9 (corresponding to the results presented in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF]) we have found that α d,u has the same asymptotic expansion as α r,u (see equation (5.15)), with a strictly smaller constant γ d ≈ 0.871. We thus conclude that .11) hence that the best improvement of α opt (k) with respect to α d,u (k) that can be hoped for with the bias considered in this paper is a replacement of ln ln k by √ ln k in the second order term of their asymptotic expansions. We shall actually see in the next sections that this upper bound is too optimistic, and that with the bias µ Θ0 and the choice (8.17) we could not increase the asymptotic expansion of the clustering threshold.

α d,u (k) = 2 k-1 1 k [ln k + ln ln k + γ d ] ≤ α opt (k) ≤ α * (k) (10.10) α * (k) = 2 k-1 1 k ln k + √ 2 ln 2 √ ln k + O(ln ln k) , ( 10
We shall finally come back briefly on the choice of parameters we made in (8.17), where we used a single parameter for the bias instead of trying to exploit all the k/2 free values of ω p . Let us define α opt (k) = sup α d (k, {ω p }), where the maximization is now over all possible values of the ω p , under the conditions ω 0 = ω k = 0 and ω p = ω k-p . We have certainly α opt (k) ≥ α opt (k), and the inequality is probably strict at least for small enough values of k; however we shall now show that α opt (k) ≤ α * (k), hence that the larger freedom of choice of generic parameters does not allow to beat the upper bound derived and discussed in the special case (8.17). To see this more easily let us exploit the invariance of the measure (6.7) under a multiplication of all ω p by a common constant, and fix their normalizations in such a way that With this choice the expressions of the RS entropy (7.34) and rigidity threshold (7.82) become

s RS (α, {ω p }) = ln 2 + α k-1 p=1 k p ω p ln ω p -k ln 2 , α r (k, {ω p }) = 1 k Γ r (k) 1 2ω 1 . ( 10.13)
Consider now a given choice of the parameter ω 1 , and a value of α ≤ α r ; for these to allow a RS phase the corresponding entropy should be positive.

Maximizing the entropy in (10.13) with respect to {ω 2 , . . . , ω k-2 }, under the normalization condition (10.12) and for a fixed value of ω 1 is easily seen to yield

ω 2 = • • • = ω k-2 , i.e.
precisely the choice of parameters (8.17). In other words this bias is the one that allows to tune the fraction of frozen variables while keeping the measure as uniform as possible, in order to minimize the entropy cost it induces.

The asymptotics of the clustering threshold 10.2.1 Setting

The rest of chapter will be devoted to an asymptotic expansion of the clustering threshold when k → ∞. We will focus on the k-uniform l+1-regular hypergraph ensemble, and on the measure µ Θ1 defined in (6.12). We have seen in the section 8.5 that for finite k this biased measure has a larger l d with respect to the uniform one, and that the inclusion of interactions between variables at larger distance brings a further improvement compared to a biasing function factorized over the hyperedges (i.e. the measure µ Θ0 defined in (6.7)). It is thus natural to investigate this phenomenon in the large k limit, that allows for some analytical simplifications, and where the algorithmic gap discussed in the introduction is most clearly seen. One would like in particular to understand at which order of the asymptotic expansion of l d the effect of the bias does appear. We give here, for the convenience of the reader, the main ideas and explain the organization of the forecoming computation, which is the generalization of the one we presented in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] (see chapter 9) for the uniform measure. We will focus on the particular form of the function ψ(p) defined in (6.13), with the two parameters b and that we recall here: .14) Note that the specific choice of parameters (8.17) in the measure µ Θ0 corresponds in the above equation to set b = 1 (i.e. to take ψ(p) = (1 -) p ), as explained in the section 6.1.3. We start in the section 10.2.2 by summarizing the main equations derived in chapter 7, for arbitrary k, in this special case. In order to take the k → ∞ limit we must specify how the degree l and the parameters (b, ) behave with k; we will set

ψ(0) = 1, ψ(p) = b(1 -) p for 1 ≤ p ≤ l + 1 . ( 10
l = 2 k-1 (ln k + ln ln k + γ) , = 2 k ln k , ( 10.15)
where γ and are constants independent of k that parametrize the degree and the bias in this limit (the factor √ 2 being for later notational convenience), while b will be independent of k. We will find that both l d and the rigidity threshold l r have asymptotic expansions of the form (10.15), our goal being to determine the corresponding rescaled thresholds γ d and γ r , as a function of the parameters (b, ).

To do so we shall first expand the correlation function C n and its hard-fields contribution H n , for a finite distance n, and find that both go to their maximal value 1, with the correction term scaling as The tresholds l d and l r have been defined for finite k according to the positivity of the large n limit of the sequences C n and H n , respectively. Their asymptotic expansion should thus be performed by taking the large k limit after the large n one; however, under the natural hypothesis (that can be checked explicitly for H n ) that the large n limit of C n and H n is either strictly vanishing or scales with k as in (10.16), one can determine γ d and γ r by reversing the order of the limits and studying whether C n and H n remain bounded or not in the large n limit. The large n limit of C n is thus discussed in Sec. 10.4. Additional difficulties need to be overcome in the intermediate regime γ d < γ < γ r where reconstruction is possible but naive reconstruction is not: even if strictly hard fields are not present here the scaling (10.16) reveals that the soft fields are actually quasi-hard, the correlation function tending to one. We thus reformulate in Sec. 10.4.2 the recursion of 10.3.2 and put it in a form for which the large n limit can be performed in a numerically tractable way. Finally our explicit results for γ d are presented in 10.5.

C n = 1 - C n k ln k + o 1 k ln k , H n = 1 - H n k ln k + o 1 k ln k , ( 10

A specialization of some formulas

Let us first specialize some of the formulas we wrote previously for a generic ψ(p) to the case defined in equation (10.14) with the two parameters b and .

The BP equation (7.18) for the function η = f Θ1 ( η 1 , . . . , η l ) becomes

η(σ, 1) = 1 z b(1 -) l i=1 ( η i (σ, 0) + (1 -) η i (σ, 1
)) , (10.17)

η(σ, 0) = 1 z (1 -b) l i=1 η i (σ, 0) + b l i=1 ( η i (σ, 0) + (1 -) η i (σ, 1)) ,
for σ = ±1. The equation (7.40) for the factorized RS solution reads

y = 1 1 - 1 + 1 -b b (1 + (1 -) y -1 ) -l , y = 2 k-1 -k -1 + k -1 y . ( 10.18)
The evolution equations (7.88,7.89) for the hard fields become

h 0,n+1 = 1 - 1 -b + b(1 + (1 -h n+1 )(1 -) y -1 ) l 1 -b + b(1 + (1 -) y -1 ) l , ( 10.19
)

h 1,n+1 = 1 - (1 + (1 -h n+1 )(1 -) y -1 ) l (1 + (1 -) y -1 ) l , ( 10.20
)

H n = 1 - 1 -b + b(1 + (1 -h n )(1 -) y -1 ) l+1 1 -b + b(1 + (1 -) y -1 ) l+1 , ( 10.21)
where we recall the initial condition h 0,n=0 = 1 and the fact that h n+1 = (h 0,n ) k-1 . We can thus write a closed equation on h 0,n :

h 0,n+1 = 1 - 1 -b + b(1 + (1 -(h 0,n ) k-1 )(1 -) y -1 ) l 1 -b + b(1 + (1 -) y -1 ) l . ( 10.22)
One can check numerically that this equation undergoes a discontinuous bifurcation when l increases above the rigidity threshold l r . Here all the formulas depend analytically on l, we can thus consider it as a real parameter, even if the original model is only defined for integer l. The fixed point h 0 = lim n→∞ h 0,n jumps abruptly from 0 to a strictly positive value when l is increased above l r . We can determine the location of this threshold by noting that at such a bifurcation the function that maps h 0,n to h 0,n+1 is tangent with the diagonal, hence l r and the bifurcating fixed point h 0,r are solutions of

h 0,r = 1 - 1 -b + b(1 + (1 -(h 0,r ) k-1 )(1 -) y -1 ) lr 1 -b + b(1 + (1 -) y -1 ) lr , (10.23) 1 = l r (k -1)(h 0,r ) k-2 (1 -) y -1 b(1 + (1 -(h 0,r ) k-1 )(1 -) y -1 ) lr-1 1 -b + b(1 + (1 -) y -1 ) lr . (10.24)
For a generic bias ψ(p) the distribution R w,n of the hard fields introduced in (7.86) is a priori non-trivial, but for the particular choice of ψ defined in (10.14) it simplifies into

R w,n (η) = δ(η -η + ) , where η + (σ, 0) = 1 2 - δ σ,+ , η + (σ, 1) = 1 - 2 - δ σ,+ , (10.25
) for all w and n. We will also denote η -= (η + ) f the message forcing to -. This allows to simplify the equation (7.96) on the soft fields distribution, which reads now:

Q 1,n+1 ( η) = k-1 u=1 k-1 u (h 0,n ) k-1-u (1 -h 0,n ) u 1 -(h 0,n ) k-1 u i=1 dQ 0,n (η i ) × δ( η -f Θ1 (η f 1 , . . . , η f u , η -, . . . , η -)) . ( 10.26)
It will be useful in the following to encode in a compact way the value of f Θ1 (η 1 , . . . , η k-1 ) (defined in equation (7.19)) when all, or almost all, the arguments of f Θ1 are forcing messages. We shall hence define, for a real number α, the message η = g 0 (α) as

η(σ, w) = δ w,0 1 + σ tanh(α) 2 ; (10.27)
the value w is thus fixed to 0, while σ can be seen as an Ising spin submitted to an effective magnetic field α. One then founds that the values of f Θ1 when all its arguments are forcing are:

• f Θ1 (η -, . . . , η -) = η + and f Θ1 (η + , . . . , η + ) = η -, the usual combination rule to obtain a forcing message η;

• f Θ1 (η + , η -, . . . , η -) = g 0 ( ) and f Θ1 (η -, η + , . . . , η + ) = g 0 (-), with = -1 2 ln(1 -), when all the messages except one are forcing in the same direction, the last one in the opposite direction;

• f Θ1 (η + , . . . , η + , η -, . . . , η -) = g 0 (0) when there are at least two forcing fields in each direction.

We will also introduce two functions g + and g -that gives the value of f Θ1 when all its arguments are forcing in the same direction, except one which is arbitrary,

namely g + (η) = f Θ1 (η, η -, . . . , η -) and g -(η) = f Θ1 (η, η + , . . . , η + ). Explicitly, η = g σ (η) means η(σ, 1) = 1 z η(-σ, 0) , η(-σ, 1) = 0 , η(σ, 0) = 1 z η(σ, 0) , η(-σ, 0) = 1 z η(σ, 1) , ( 10.28)
with z normalizing this distribution. Note that the two functions g + and g -are linked by the spin-flip operation according to g + (η f ) = (g -(η)) f . 10.3 The large k limit for a finite distance n

Evolution of the hard fields

We start our large k asymptotic expansion, using the scaling of the parameters defined in (10.15), by considering the solution (10.18) of the translationally invariant RS equation; its leading order behavior is easily found to be

y = 1 + 2 k ln k + o 1 √ k ln k , y = 2 k-1 1 + O k 2 k-1 . (10.29)
Turning to the sequences h w,n for the weights of the hard fields, solutions of the recursion equations (10.19,10.20), one realizes easily that, for n finite in the large k limit with the scaling of the parameters stated above,

h 0,n = 1 - x 0,n k ln k +o 1 k ln k , h 1,n = 1 - x 1,n k ln k + o 1 k ln k , h n+1 = 1 - x 0,n ln k + o 1 ln k , ( 10.30)
where x 0,n and x 1,n are independent of k and solutions of the recursion relations:

x 0,n+1 = B e -γ + e -γ+x0,n , (10.31)

x 1,n+1 = e -γ+x0,n = x 0,n+1 -B e -γ . ( 10.32)
Here and sometimes in the following it is more convenient to use the notation

B = 1 -b b (10.33)
as a parameter equivalent to b. The recursion above is closed on x 0,n , and satisfies the initial condition x 0,n=0 = 0, that follows immediately from h 0,n=0 = 1. Note that for b = 1 (i.e. B = 0) one recovers the result of equation (9.4) for x n = x 0,n = x 1,n , as it should in the uniform case. One also finds by expanding (10.21) that H n , the hard fields contribution to the correlation function, is indeed given by the asymptotic expansion stated in (10.16), with H n = x 0,n . The behavior of the sequence x 0,n solution of (10.31) is easily determined by plotting the shape of the function x → B e -γ + e -γ+x , see the top panel of Fig. 10.2 for an example. For a given value of b (hence of B) there exists a critical value γ r (b) such that this function remains strictly above the diagonal when γ < γ r (b), while it intersects it for γ > γ r (b). As a consequence in the former case the sequence x 0,n diverges (very rapidly, as iterated exponentials) with n, whereas in the latter it converges to the smallest fixed point; these behaviors are illustrated in the bottom panel of Fig. 10.2. The divergence of x 0,n = H n corresponds, in the large k limit, to the vanishing of H n at finite k (recall the definition (10.16)), i.e. to the impossibility of naive reconstruction. The value of γ r (b) can be obtained by noticing that at this bifurcation the function x → B e -γ + e -γ+x is tangent with the diagonal at their unique intersection point x r (b), hence that (x r (b), γ r (b)) are solution of

x = B e -γ + e -γ+x 1 = e -γ+x ⇒ x = γ γ = 1 + B e -γ . ( 10.34)
As b > 0, B > -1, this equation admits a unique solution with γ > 0 (the sequence x 0,n being positive this is also the case for the fixed point x, and hence also of γ at the bifurcation), which can be expressed as

γ r (b) = 1 + W B e , ( 10

Evolution of the soft fields distribution

We shall now study the large k limit of the soft fields distributions Q w,n , Q w,n . The crucial point we shall exploit to simplify them is the fact that the hard fields weights h w,n are very close to 1 according to the scaling (10.30), hence the dominant contributions to Q w,n will arise when the incoming messages are almost all forcing. To put this remark on a quantitative ground we start with the equation (10.26) on the distribution Q 1,n+1 . The integer u that appears in this equation is a random number drawn from the binomial distribution Bin(k -1, 1-h 0,n ), conditioned to be strictly positive. In the large k limit, using the scaling behavior (10.30) of h 0,n , one sees that the average (k -1)(1 -h 0,n ) of the binomial distribution vanishes as O(1/ ln k), hence the main contribution in (10.26) arises from the smallest value u = 1 appearing in the sum. We thus obtain at the leading order:

Q 1,n+1 ( η) = dQ 0,n (η) δ( η -g + (η f )) , ( 10.36)
where the function g + was defined in (10.28). Consider now the equation (7.69) for Q 0,n+1 = P 0,n+1 . When all the η i 's are extracted from the hard part of P 0,n and P 1,n the arguments of f Θ1 are all forcing, with the two possible directions represented; in most of these terms there are at least two messages in each direction, except for the term in the first line of (7.69), and the term with t = k -2 in the second line. According to the discussion in Sec. 10.2.2 this will yield contributions of the form g 0 (0) and g 0 (± ), where the function g 0 was defined in (10.27). When exactly one of the η i is a soft field, we obtain a contribution g -(η f 1 ) from the first line if it is η 1 which is soft, a contribution g + (η k-1 ) from the term t = k -2 of the second line if η k-1 is the unique soft field, all other cases leading to subdominant contributions of the form g 0 (α) with α of order . Collecting these various contributions we thus obtain at the leading order:

Q 0,n+1 ( η) = x 1,n 2 k-1 ln k dQ 1,n (η) δ( η -g -(η f )) +
x 0,n 2 k-1 ln k dQ 0,n (η) δ( η -g + (η))

+ β + δ( η -g 0 ( )) + β -δ( η -g 0 (-))

+ 1 -x 0,n + x 1,n 2 k-1 ln k -β + -β -δ( η -g 0 (0)) , (10.37) with .38) We turn now our attention to the equation (7.95) for Q w,n+1 ; in the limit we are considering the non-vanishing contributions are found to arise only for values of p that remain finite, the law of p becoming P w,n (p), with p! for the Poisson law of parameter λ; the P w,n are indeed well-normalized probability distributions. In the right hand side of (7.95) a (finite) number p of messages η i are thus drawn from Q 1,n+1 , for which we can use the limit form (10.36), while the others l -p ∼ 2 k-1 ln k are drawn from Q 0,n+1 . Observing the form of Eq. (10.37) one realizes that the number of times the first two terms of Q 0,n+1 will be picked become Poissonian random variables of parameter x 1,n and x 0,n , respectively. All the other terms are of the form g 0 (α), which will be dealt with thanks to the simple exact identity: f Θ1 ( η 1 , . . . , η s , g 0 (α s+1 ), . . . , g 0 (α l )) = f Θ1 ( η 1 , . . . , η s , g 0 (α s+1 + • • • + α l )) .

β + = k -1 y (h 0,n ) k-1 , β -= k -1 y y h 1,n (h 0,n ) k-2 . ( 10
(10.40) The sum α of the arguments of g 0 is thus α = (a + -a -), where a + , a -are a pair of integers drawn from the multinomial distribution of parameters (l; β + , β -). One can thus compute the first two cumulants of α as

E[α] = l(β + -β -) , Var[α] = ( ) 2 l(β + + β --(β + -β -) 2
) . (10.41)

In the limit we are considering one finds that these two quantities converge to 2 , while the cumulants of higher order vanish, which show that α tends to a Gaussian distributed random variable with mean and variance both equal to 2 ; we × δ(η -f Θ1 (g 0 (α), g + (η f 1 ), . . . , g + (η f p ), g + (η p+1 ) f , . . . , g + (η p+q ) f , g + (η p+q+1 ), . . . , g + (η p+q+r ))) ,

where we used the identity g -(η f ) = g + (η) f to transform q of the arguments of f Θ1 . In this equation the function f Θ1 is the one defined in Eq. (10.17 The initial condition on Q w,n=1 can be deduced after a short computation from the one on P v,0 given in (7.62):

Q w,1 (η) = ∞ q,r=0
Po(q; e -γ )Po r; e -γ b D α δ η -f Θ1 g 0 α + q -r 2 ln b , (10.44) for both values w = 0, 1. The explicit value of η for a given choice of α, q and r reads η(+, 0) = 1 z b q e α , η(+, 1) = 1 z b q+1 e α , η(-, 0) = 1 z b r e -α , η(-, 1) = 1 z b r+1 e -α , (10.45) with z normalizing this distribution.

The recursion relation (10.42) bears on the two sequences of distributions Q 0,n and Q 1,n ; however the two sequences are not independent, and obey some symmetry properties, that follow from the equations (7.73,7.74). In the large k limit these relations translate into Q w,n (η f ) = η(-, w) η(+, w) Q w,n (η) , or equivalently (10.46) dQ w,n (η)A(η) = dQ w,n (η)A(η f ) η(-, w) η(+, w) , and x 0,n Q 0,n (η) = η(+, 0) η(+, 1)

x 1,n Q 1,n (η) , (10.47)

x 0,n dQ 0,n (η)A(η) = x 1,n dQ 1,n (η)A(η) η(+, 0) η(+, 1) , for any function A such that the integrals exist. One can check by induction on n that the sequences Q 0,n and Q 1,n solution of (10.42) with the initial condition (10.44) do indeed satisfy these identities. We can finally establish the scaling stated in (10.16) for the correlation function C n , by simplifying the expression (7.90) in the large k limit. Observing in particular that the probability law for p is the same in this equation and in (7.95) with w = 0 (modulo the shift l → l + 1 which is irrelevant in the limit), one finds after after a short computation the expression C n = x 0,n dQ 0,n (η)(1 -m(η)) , (10.48) for the reduced correlation function C n , where we defined m(η) = η(+, 0) -η(-, 0) η(+, 0) + η(-, 0) . (10.49)

Note that C n satisfies the inequalities 0 ≤ C n ≤ x 0,n = H n , which are immediate consequences of the bounds H n ≤ C n ≤ 1 we obtained at finite k and of the definitions in (10.16). As a consistency check one can also derive the bounds on C n directly in the large k formalism; one of them is obvious from the observation that m(η) ≤ 1 for all η, the other one follows from the identity dQ 0,n (η) m(η) = dQ 0,n (η) m(η) 2 ≥ 0 , (10.50) which can be proven from the Bayes symmetry expressed in (10.46), using the test function A(η) = m(η)(1 -m(η)).

The limit of large distance n

Let us summarize what we have just achieved and underline the main equations that will be used in the following. We have obtained recursive equations, in which the parameter k has disappeared, that allow to compute the reduced correlation function C n and its hard-fields contribution H n introduced in (10.16). The latter can be obtained from the scalar recursion (10.31), it depends on γ and b, and the asymptotic expansion of the rigidity threshold is of the form We shall now discuss the computation of C n in the large n limit, as the final step to complete the determination of γ d (b, ).

For γ > γ r (b)

The most natural way to solve numerically the functional recursion equation (10.42) on Q w,n is to use the population dynamics algorithm already explained in Sec. 8.1, that consists of approximating Q w,n by the empirical distribution over a sample of N representative elements {η 1 , . . . , η N }. An iteration step n → n + 1 amounts to update the populations by drawing the integers p, q, and r from their respective laws, extracting the η i 's from the current populations, and creating an η of the new population according to the argument of the Dirac delta in (10.42). When γ > γ r (b) this procedure can be performed without difficulty for arbitrarily large distances n, as the sequences x 0,n , x 1,n remain bounded for all n. Figure 10.3 presents numerical results obtained in this way for b = 0.4 and = 0. We have plotted on the top panel C n (γ, b, 0) as a function of n for some values of γ above the rigidity threshold γ r (b) ≈ 1.378. One can see that C n (γ, b, 0) converges at large n to a finite limit C(γ, b, 0), that we have plotted as a function of γ in the bottom panel, along with the limit H of H n = x 0,n . As we mentioned before the reduced overlap satisfy the bounds 0 ≤ C n ≤ x 0,n , hence in the large n limit one has 0 ≤ C ≤ H, that is indeed verified in the bottom panel of figure 10.3. This implies that C n remains bounded for γ > γ r (b), hence the expected inequality γ d (b, ) ≤ γ r (b). The observation of the bottom panel of figure 10.3 suggests the less obvious fact that this inequality is strict; indeed H has a square root singularity when γ → γ + r , as a consequence of the bifurcation it undergoes, while C seems pefectly smooth in this limit, suggesting that it remains finite down to a critical value γ d < γ r .

Unfortunately the most interesting regime γ d < γ < γ r cannot be studied with the simple numerical procedure we just described: when γ < γ r (b) the sequences x 0,n and x 1,n diverge, hence the random numbers p, q, r of fields η i that must be manipulated to implement (10.42) become very quickly too large for any practical purpose. We shall thus devise in the next subsection an alternative formulation to circumvent this difficulty, that was used in particular to obtain the points of the curve C below the rigidity threshold in the bottom panel of figure 10.3. In order to give an intuition on how this reformulation should be performed we first present in figure 10.4 the results of the simple procedure for γ slightly below γ r , and distances n not too large. One sees clearly in this plot that H n = x 0,n diverges, while C n seems to remain bounded; the expression (10.48) of C n reveals that such a situation is possible if Q 0,n concentrates on fields η with m(η) very close to 1. By inspection of the populations in our numerical simulations we have checked that this is indeed the case, and more precisely that both Q 0,n and Q 1,n tend to a Dirac peak on the hard-field η + . The finite value of C n in the large n limit of the intermediate regime γ d (b, ) < γ < γ r (b) which is reconstructible without strictly hard-fields arises thus from a delicate compensation in the multiplication of the diverging factor x 0,n and of the vanishing integral dQ 0,n (η)(1 -m(η)). The relevant contribution of the latter arises from atypical values of η for which Q 0,n is of order 1/x 0,n , the typical values of η ≈ η + having 1 -m(η) ≈ 0.

A reweighting scheme

To handle the difficulty that arises in the intermediate regime γ d < γ < γ r we will adapt the approach we developed in chapter 9 (see [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF]) for the uniform measure, introducing a reweighted probability distribution ρ n (η) that gives less importance to the typical quasi-hard fields that do not contribute to C n . We define it as

ρ n (η) = x 1,n Q 1,n (η)
η(-, 1) η(+, 1) , (10.51) the reweighting factor proportional to η(-,1) η(+,1) indeed vanishes when η is a hard-field η + . This choice also ensures the invariance of ρ n under a spin-flip transformation, ρ n (η f ) = ρ n (η), as can be easily seen from the first equality in (10.46) with w = 1. Note that ρ n is a positive measure, but not a normalized probability measure anymore. One can nevertheless check that its total mass, that we shall denote m n , is finite for all finite n. One has indeed, inverting the relation (10.51) and exploiting the normalization of Q 1,n , x 1,n = dρ n (η) η(+, 1) η(-, 1) = dρ n (η) 1 2 η(+, 1) η(-, 1) + η(-, 1) η(+, 1)

= dρ n (η) η(+, 1) + η(-, 1) 2 η(+, 1)η(-, 1) , (10.52) where we used the invariance ρ n (η f ) = ρ n (η) to symmetrize the integrand. Thanks to the inequality between arithmetic and geometric means of positive numbers the last integral is larger than m n , which implies finally m n ≤ x 1,n . We can thus define a probability distribution ν n by dividing ρ n by its total mass, ν n (η) = ρ n (η)/m n . The problem at hand is equivalently described in terms of the {x w,n , Q w,n }, of ρ n , or of the pair (ν n , m n ); for instance the reduced overlap can be expressed as where we used (10.47) to transform the integral over Q 0,n as one over Q 1,n . As we shall see the reweighted formulation is however much more convenient to study the large n limit, as it avoids the direct manipulation of the diverging quantities x w,n .

We will now derive recursion relations for (ν n , m n ); to do so it will be convenient to first introduce a different parametrization of the messages η. These are normalized probability distributions on a space of four states (σ, w), they can be thus encoded with three real numbers, that we shall choose as u (1) = η(+, 1) η(-, 1) , u (2) = η(+, 0) -η(+, 1) η(-, 0) -η(-, 1) , u (3) = (η(+, 0) -η(+, 1))(η(-, 0) -η(-, 1)) B 2 η(+, 1)η(-, 1) . (10.54)

We will group them as a row vector with three columns, u = (u (1) , u (2) , u (3) ), and define for later use the associated canonical basis e (1) = (1, 0, 0), e (2) = (0, 1, 0), e (3) = (0, 0, 1). Consider now the BP equation η = f Θ1 ( η 1 , η 2 , . . . ) defined in (10.43); it becomes in terms of this parametrization u (1) = i η i (+, 0) + η i (+, 1) η i (-, 0) + η i (-, 1) , u (2) = i η i (+, 0) η i (-, 0) , (10.55) u (3) = i η i (+, 0) η i (-, 0) ( η i (+, 0) + η i (+, 1))( η i (-, 0) + η i (-, 1)) .

Using this shift of argument in (10.66), and recalling from (10.32) that x 1,n+1 = e -γ+x0,n we obtain: We will now trade the integrations over Q 0,n and Q 1,n for integrals over ρ n , thanks to the change of densities expressed in (10.47) and (10.51). We will also write x 0,n + x 1,n = 2x 1,n + Be -γ according to (10.32), and write 2x 1,n as an integral over ρ n following (10.52). This yields The initial condition for the recursion on (ν n , m n ) is obtained from the one on Q 1,1 given in (10.44):

ρ n+1 (z) = exp -γ -
ρ n+1 (z) = exp -γ -B e -γ -
m 1 = exp -γ + e -γ 2 √ b - 1 b -1 - 2 2 , ( 10.76
)

ν 1 (η) = ∞ q,r=0 Po q; e -γ √ b Po r; e -γ √ b dα √ 2π 2 e -α 2
2 δ u -V 0 α + q -r 2 ln b .

(10.77)

For completeness we give here the expressions of the functions we introduced in terms of the u-parametrization: L(η) = 1 + B u (1) u (3) u (2) + (u (1) ) 2 + Bu (1) u (2) u (3) , (10.78) M (η) = 1 + B u (2) u (3) u (1) + 1 (u (1) ) 2 + B u (3) u (1) u (2) -1 u (1) (10.79) + 1 + B u (1) u (3) u (2) + (u (1) ) 2 + Bu (1) u (2) u (3) -u (1) , (10.80)

C(η) = 2 1 + B u (2) u (3)
u (1) 1 + B u (1) u (3) u (2) u (1) + Bu (2)

u (3) + 1 u (1) + B u (3)
u (2) . (10.81)

A Gaussian approximation for the quasi-hard fields

We have obtained above the recursion equations (10.72,10.73) for the scalar m n and the probability distribution ν n , complemented by the initial conditions (10.76,10.77). We will now discuss the possibility to solve numerically this recursion with a population representation of ν n , and its advantages with respect to the direct resolution in terms of Q w,n . To do so let us first rewrite the recursion equation (10.73) on ν n as ν n+1 (z) = exp - = m n dν n (η) η(+, 0) + η(-, 0) η(+, 1)η(-, 1) ( η(+, 1) + η(-, 1)) , (10.84) where the last expression of λ n has been obtained by symmetrizing the integrand.

According to the equation (10.82) a random variable u drawn from ν n+1 can be decomposed as the sum of two random variables, one Gaussian distributed and the other with a compound Poisson distribution. More explicitly one has the following equality in distribution, u d = α(e (1) + e (2) ) + p i=1 u i where α is a Gaussian with zero mean and variance 2 , p is extracted from a Poisson law of mean λ n , and the u i 's are i.i.d. copies extracted from π n /λ n . If ν n is known as an empirical distribution over a sample then it is possible to draw u from the probability law π n /λ n by extracting a field η in the population representing ν n with a probability proportional to L(η), and then set u = V δ (η) with δ = ± with equal probability 1/2. It seems then possible to use this distributional interpretation to solve numerically the recursion on (m n , ν n ). However this is doable in practice only if λ n remains bounded when n grows, otherwise one falls back on the problem we wanted to avoid of having to manipulate a diverging number of summands. As a matter of fact the reweighting has not offered a free lunch from this point of view: it turns out that λ n diverges if and only if x 1,n does, in other words if and only if γ < γ r (b). This statement is a consequence of the bounds c -(b)x 1,n ≤ λ n ≤ c + (b)x 1,n , where c ± (b) are positive constants, the proof of which we defer to the Appendix C for the sake of readability.

Fortunately the reweighting procedure we followed will help us to handle the divergence of λ n more easily than the one of x 1,n in the direct recursion. Indeed the divergence of λ n comes from the contributions of fields for which L(η) becomes very large; the crucial point is that these η yield very small values of V ± (η), we can thus make a Gaussian approximation for this sum of a very large number of very small random variables. To put this idea at work we rewrite (10.82) by decomposing it as ν n+1 (z) = exp - where ξ n is a threshold that is arbitrary for the moment, we shall specify it later on. The decomposition (10.85) means that under the law ν n+1 the random variable u is the sum of the Gaussian random variable described previously and of two random variables, one with the law ν (2),( 3)

n = 0. Replacing ν (>)
n+1 by a Gaussian is an approximation, that amounts to neglect the cumulants of order larger than 2, the accuracy of which is controlled by the cutoff ξ n . The larger is ξ n the better the truncation is, because a smaller part of the full law ν n+1 is treated approximatively, but the price to pay is a simultaneous increase of λ (≤) n , the average number of fields that must be summed in desribing ν (≤) n+1 . A compromise needs thus to be found between these two effects, we explain below how we fixed ξ n in practice.

Conclusion

In this Ph.D. we have studied the typical difficulty of Constraint Satisfaction Problems, via the introduction of random ensemble of instances, focusing on the bicoloring problem on k-hypergraphs. Following the studies [START_REF] Braunstein | The large deviations of the whitening process in random constraint satisfaction problems[END_REF][START_REF] Baldassi | Local entropy as a measure for sampling solutions in constraint satisfaction problems[END_REF][START_REF] Baldassi | Unreasonable effectiveness of learning neural networks: From accessible states and robust ensembles to basic algorithmic schemes[END_REF][START_REF] Maimbourg | Generating dense packings of hard spheres by soft interaction design[END_REF][START_REF] Zhao | Maximally flexible solutions of a random k-satisfiability formula[END_REF], we have introduced a biased probability measure over the set of solutions of a random CSP. Our goal was to study the modifications of the clustering threshold α d induced by the non-uniformity between solutions. We have focused on two particular implementations of the bias. In the first one, the interactions induced by the bias can be factorized over the bicoloring constraints. In the second biased measure we incorporate interactions between variables belonging to different clauses (in practice we have considered interactions between variables at distance 1 in the k-hypergraph).

Specifying further the choice of the bias, we have weighted the solutions according to the presence of forcing clauses, where one says that a clause is forcing a variable if the clause becomes violated when changing the value of the variable. Forcing clauses are related to the presence of frozen variables, which are one of the mechanism for the RSB phenomenon that arises at the clustering transition. This choice can be also motivated by the observation that algorithms usually reach solutions without frozen variables, hence favoring solutions without forcing clauses would bias the measure toward solutions that are easier to find for algorithms.

We have determined the clustering transition for the two different implementations of the bias mentioned above. For finite k values, α d can be computed via the numerical resolution of the 1RSB equations. In the paper [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF] written in collaboration with Federico Ricci-Tersenghi, that is presented in chapter 8 (sections 8.2-8.4), we have shown that for well-chosen parameters defining the biased measure factorizing on the clauses, it is possible to delay the clustering threshold to higher density of constraints. We have checked that this strategy has a positive impact on the performances of Simulated Annealing algorithms. Introducing a bias in the measure reduces its entropy, which is maximal for the uniform measure. Although this effect represents a limitation for the possible choices of the bias, our results show that reducing the weight of some of the solutions can help in the search of the remaining ones.

From a practical point of view, we have also developed a numerical method to determine accurately the dynamic threshold, that corresponds to discontinuous bifurcation in a functional equation, by studying a stability parameter close to the bifurcation (see section 8.2).

In the paper [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF], that is presented in section 8.5 (for the finite k results) and in chapter 10 (for the large k analysis), we have studied the biased measure with a larger range of interaction. We have shown that for finite k values, increasing the range of interaction allows us to delay further the clustering threshold.

In this Ph.D. we have also determined the dynamic transition in the large k limit through a partly analytic asymptotic expansion. In the paper [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF], that is presented in chapter 9, we have obtained for the bicoloring problem on khypergraph the following asymptotic expansion of the clustering threshold:

α d (k) = 2 k-1 k (ln k + ln ln k + γ d + o(1)) ,
for the uniform measure, with γ d,u ≈ 0.871. We have shown that the q-coloring problem admits a similar asymptotic expansion with the same constant γ d : α d (q) = (q/2)(ln q + ln ln q + γ d + o( 1)).

In the paper [START_REF] Budzynski | Biased measures for random constraint satisfaction problems: larger interaction range and asymptotic expansion[END_REF] we adapt this computation to the two biased measures studied in the Ph.D., for the bicoloring problem on k-hypergraphs. This large k limit computation is presented in chapter 10. We show that the clustering threshold for our implementation of the bias arises at the same scale as in the uniform case, but with a constant γ d that depends on the rescaled parameters describing the bias. We find that the factorized bias of [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] cannot improve on the constant γ d with respect to the value γ d,u ≈ 0.871 obtained in the uniform case. We find instead that the bias with larger interaction range allows us to increase its value up to γ d ≈ 0.977.

The improvement in the asymptotic behavior of the dynamic transition is modest, as it occurs in the third order of the asymptotic expansion. Note also that the constant γ d remains itself smaller than the rigidity threshold γ r,u = 1 of the uniform measure. However, the conceptual link between this transition and the important algorithmic gap observed in this limit justifies for us the efforts devoted to achieve this improvement, and calls for further investigations.

The natural direction to continue this work would be indeed to study a more general form of the bias, and to try to determine which choice would lead to the optimal scaling in the large k limit. In particular, is it possible to design a bias that improves on the second or first order in the asymptotic expansion compared to the asymptotic expansion of the uniform measure, or can we show that for some family of biased measures the only improvement bears on the third term of the asymptotic expansion ? The next step could be for instance to consider a more general form for the function ψ(p) defined in equation (10.14), retaining more information on the local configuration than just the number of forcing clauses around one variable. One could also increase the range of interaction to distance 2 or more. Both positive or negative results, i.e. the impossibility to increase α d beyond the third order term of the asymptotic expansion, would shed light on the intrisic difficulty of random CSPs.

Another interesting question to investigate in the future is the effect of the decimation on the biased measure, which can be analyzed along the lines of [START_REF] Ricci | On the cavity method for decimated random constraint satisfaction problems and the analysis of belief propagation guided decimation algorithms[END_REF]. Can we improve the performances of the Belief Propagation guided decimation (or the Survey Propagation guided decimation) algorithms, by using a biased measure on which one estimates the marginals ?

Finally it would also be interesting to extend this study to other CSPs, in particular to the k-satisfiability and the q-coloring problems. We have indeed focused our study on the k-hypergraph bicoloring problem because the computations are simpler for this model than for k-SAT or q-COL. Indeed the degree of freedom are binary, and the replica symmetric solution is trivial. At the same time this model exhibits the same phase transitions than more complicated random CSPs. We believe that the strategy developed in this Ph.D. can be extended to these models, and we expect a similar increase on the clustering threshold for an appropriate choice of the bias. The initial condition of this recursive computation is given by P σ,0 = δ(η -δ σ ), where δ σ is the measure concentrated on the color σ, i.e. δ σ (τ ) = I(σ = τ ), which corresponds to the colors being revealed on the leaves of the tree. These equations correspond to (7.7,7.51,7.52) (with ω(σ 1 , . . . , σ k ) = 1 -I[σ 1 , σ k a.e]) for the hypergraph bicoloring problem.

We separate now the contribution of "hard fields", or frozen variables, namely the configurations of boundary variables that determine unambiguously the root in the naive reconstruction procedure. This forced value can only be the correct one the root had in the broadcast process, hence we shall write :

P σ,n (η) = H n δ(η -δ σ ) + (1 -H n ) Q σ,n (η) , (B.3)
where Q σ,n has no atom on δ σ ; this mimicks the decomposition (7.78) of the main text. Plugging this decomposition in (B.1) yields the evolution equation for the weight of the hard fields:

H n+1 = 1 -e -cHn q-1 q-1
, with H 0 = 1 . (B.4) Indeed f c (η 1 , . . . , η l ) = δ σ if and only if for each color σ = σ at least one of the arguments η i is equal to δ σ , thus forbidding all colors except σ. The number of hard fields of the color σ = σ is easily seen from (B.1) to be Poisson distributed with average cHn q-1 , independently from one color σ to another, from which (B.4) follows.

The recursion (B.4) has a bifurcation at c r (q), in the sense that H n → 0 as n → ∞ if and only if c < c r (q). Writing down the equations fixing c r and H r at the bifurcation, which are similar to (7.81), and then expanding them for large q one finds the asymptotic expansion c r = q(ln q + ln ln q + 1 + o(1)). We shall thus study the large q limit with c getting also large, on the scale c = q(ln q + ln ln q + γ) with γ finite. In this limit one finds that for n finite

H n = 1 - x n ln q + o 1 ln q , (B.5)
where x n is of order 1 and obeys exactly the same recursion as in the bicoloring case, namely x 0 = 0 and x n+1 = e -γ+xn . Let us now simplify the evolution equation for the distributions Q σ,n of the soft fields. First of all we insert the decomposition (B.3) in the right hand side f c (σ, σ ; η 1 , . . . , η l ) correspond to the colors σ and σ . Using this notation, and the parametrization (B.8), one can deduce from (B.6) the evolution equation for Q σ,n (η) at lowest order:

Q σ,n+1 (η) = 1 q -1 σ =σ ∞ l=0 e -c(1-Hn) (c(1 -H n )) l l! l i=1     1 (q -1) 2 σi =σ σ i =σi dQ n (h i )     (B.9)
δ(η -f c (σ, σ ; s(σ 1 , σ 1 ; h 1 ), . . . , s(σ l , σ l ; h l )))

To put this expression under the form (B.8), and hence close the recursion on Q n (h), it remains to notice that f c (σ, σ ; η 1 , . . . , η l ) = s(σ, σ , h) with h = f (u 1 , . . . , u l ) , u i = u(σ, σ ; η i ) , (B.10) where f = f Θ0 is the function defined for Ising spins in (7.8), and

u(σ, σ ; η) = η(σ ) -η(σ) 2 -η(σ) -η(σ ) . (B.11)
For a fixed choice of σ and σ = σ, one can see that u(σ, σ ; s(σ i , σ i ; h i )) is a random variable with respect to the uniform choices of σ i = σ and σ i = σ i , that takes the following values (recall the definition of the function g 1 from (9.10)):

               g 1 (h i )
with probability q-2 (q-1) 2 , when σ i = σ , σ i = σ , g 1 (-h i ) with probability q-2 (q-1) 2 , when σ i = σ , σ i = σ , -g 1 (-h i ) with probability q-2 (q-1) 2 , when σ i = σ , σ i = σ , h i with probability 1 (q-1) 2 , when σ i = σ , σ i = σ , 0 otherwise .

(B.12)

As c(1 -H n ) q-2 (q-1) 2 → x n in the regime we are considering, the number of occurences of the first three cases in (B.9) is Poissonian of mean x n ; on the other hand the number of times the fourth case happens vanishes when q diverges (as 1/q), while the fifth does not contribute to (B.9), because f (u 1 , . . . , u l , 0, . . . , 0) = f (u 1 , . . . , u l ). We thus see by comparison with (9.13,9.15) that the probability distribution Q n (h) defined in (B.8) obeys exactly the same recursion equations as the one derived in the main text for the hypergraph bicoloring model, the initial condition Q 1 (h) = δ(h) being also valid here.

MOTS CLÉS

Barrières algorithmiques, Problèmes de satisfaction de contraintes, Aléatoire.

RÉSUMÉ

La complexité typique des Problèmes de Satisfaction de Contraintes (CSP) peut être étudiée à l'aide d'ensembles aléatoires de contraintes. On observe un phénomène de seuil quand la densité de contraintes augmente. En particulier à la transition de clustering, l'ensemble des solutions typiques se fracture en groupes de solutions séparés les uns des autres. Dans cette thèse nous introduisons un biais qui brise l'uniformité entre les solutions d'une instance de CSP, et nous étudions son effet sur la valeur du seuil de clustering. Nous étudions en particulier le problème de bicoloriage de k-hypergraphes. Pour de petites valeurs de k, nous montrons que ce biais peut augmenter la valeur du seuil de clustering, et que cela a un effet positif sur les performances de l'algorithme de Simulated Annealing pour la recherche de solutions d'une instance du problème de bicoloriage. Dans la limite où k tend vers l'infini, nous calculons le développement asymptotique du seuil de clustering pour la mesure uniforme et pour une mesure biasée. Nous évaluons le gain obtenu avec cette implémentation du biais.

ABSTRACT

The typical complexity of Constraint Satisfaction Problems (CSP) can be studied using random ensembles of instances. One observes threshold phenomena when the density of constraints increases, in particular a clustering phase transition at which typical solutions shatter into disconnected components. In this Ph.D., we introduce a bias that breaks the uniformity among solutions of a given instance of CSP, and look at the evolution of the clustering threshold under this bias, focusing on the bicoloring of k-uniform random hypergraphs. For small values of k, we show that this bias can delay the clustering transition to higher densities of constraints, and that it has a positive impact on the performances of Simulated Annealing algorithm to find a solution for a given instance of the bicoloring problem. In the large k limit, we compute the asymptotic expansion of the clustering threshold for the uniform and the biased measure, and characterize the gain obtained with our implementation of the bias.

KEYWORDS

Algorithmic barriers, Constraint satisfaction problems, Random.

Figure 1 . 1 :

 11 Figure 1.1: graphical representation of the SAT formula F= (x 1 ∨ x 2 ∨ x 3 ∨ x 4 ) ∧ (x 4 ∨ x 5 ) ∧ (x 4 ∨ x 6 ∨ x 7 ) ∧ (x 3 ∨ x 7 ∨ x 8 ∨ x 9 )

Figure 1 . 2 :

 12 Figure 1.2: sketch of the classification of the decision problems

Figure 1 . 3 :

 13 Figure 1.3: decision tree representing the DPLL algorithm solving the formulaF = (x 1 ∨ x 2 ∨ x 3 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 4 ) ∧ (x 1 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 4 ) ∧ (x 2 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 3 ∨ x 4 ) ∧ (x 1 ∨ x 2 ∨ x 3 )

Figure 1 . 4 :

 14 Figure 1.4: from [45] Fig 3.a, fraction of UNSAT formulas drawn from the random 3-SAT ensemble versus α for several sizes N .

Figure 1 . 5 :

 15 Figure 1.5: from [46], median running time of DPLL on random instances of 3-SAT ensemble as a function of α, for several sizes N

18 )

 18 and p(c) = Z(c)/Z is the distribution over the clusters.

Figure 2 . 1 :

 21 Figure 2.1: Sketch of the transitions undergone by the set of solutions. The set of solutions is in black, the rest of the configuration space is in white.

Figure 2 . 2 :

 22 Figure 2.2: Left: uncondensed regime the supremum of (2.20) is achieved for φ * such that Σ(φ * ) > 0 and (dΣ(φ)/dφ) = -1 (the red tangent has slope -1).Right: condensed regime, the supremum of (2.20) is achieved for φ * such that Σ(φ * ) = 0 and (dΣ(φ)/dφ) = -X s (green tangent), the value of φ such that (dΣ(φ)/dφ) = -1 has a negative complexity.

  k

Figure 4 . 1 :

 41 Figure 4.1: Messages η i→a , η a→i on a small factor graph with N = 7 vertices and M = 3 hyperedges with k = 3.

  R. Marino, G. Parisi and F. Ricci-Tersenghi study a version of the SP guided decimation algorithm including backtracking (BSP) on the random k-SAT ensemble with k = 3 and k = 4. They obtain the best algorithmic threshold for k = 3 and k = 4 :α BSP a (3) = 4.26, and α BSP a (4) = 9.9 which is very close to the satisfiability threshold.

Figure 5 . 1 :

 51 Figure 5.1: From [37]

Figure 6 . 1 :

 61 Figure 6.1: Left panel: an example of a hypergraph G with N = 7 vertices represented by black circles, and M = 3 hyperedges linking k = 3 vertices, drawn as white squares.Center panel: the introduction of an interaction, represented as a black square, between all the vertices at distance 1 from the central vertex i, generates short loops even if G is a tree. Right panel: the factor graph representation of the probability measure (6.16), the white circles stand for the variable nodes v (i,a) , the black circles (resp. white squares) are the interaction factors ψ (resp. ω).

  [START_REF] Montanari | Reconstruction and clustering in random constraint satisfaction problems[END_REF] with b(t) = eI[t≥T +1]

Figure 7 . 1 :

 71 Figure 7.1: Left panel: an example of a hypergraph G with N = 7 vertices represented by black circles, and M = 3 hyperedges linking k = 3 vertices, drawn as white squares.Center panel: the introduction of an interaction, represented as a black square, between all the vertices at distance 1 from the central vertex i, generates short loops even if G is a tree. Right panel: the factor graph representation of the probability measure (7.2), the white circles stand for the variable nodes v (i,a) , the black circles (resp. white squares) are the interaction factors ψ (resp. ω). The messages η i→a and η a→i obey the Belief Propagation equations (7.17).

. 29 )

 29 Where r d = I[d = l] for the random k-uniform, l + 1-regular graph, and r d = e -αk (αk) d /d! for the ER ensemble. These equations can be equivalently written as equalities in distribution between random variables (we use the symbol d =). For instance (7.28) means η d = f Θ ( η 1 , . . . , η d ) (7.30)

. 39 )

 39 one can get rid of the normalization factors z and z and rewrite (7.36,7.37,7p + 1) y -p

  )s RS (k, l, ψ) =Φ RS (l, ψ) -

. 44 )Figure 7 . 2 :

 4472 Figure 7.2: An example of a hypertree, vertices being represented by circles, while hyperedges are drawn as squares linked to k vertices, with k = 3 on figure.

  ) where p 0 is the law defined in(7.45), and with the expressionm( η 1 , . . . , η l+1 ) = m + ( η 1 , . . . , η l+1 ) -m -(η 1 , . . . , η l+1 ) m + ( η 1 , . . . , η l+1 ) + m -( η 1 , . . . , η l+1 ) , (7.64) m σ ( η 1 , . . . , η l+1 ) = w1,...,w l+1

d

  P 0,n ( η i ) m( η 1 , . . . , η l+1 ) 2 , (7.75) which obviously shows that C n ≥ 0. This alternative form of C n can be derived by first checking that

Figure 7 . 4 :

 74 Figure 7.4: The evolution of C n and H n for k = 5, α = 10.4, in the intermediate regime between α d and α r : the fraction H n of hard fields falls to zero (the vertical range has been reduced for the sake of readability) while the correlation function C n tends to a positive constant.

d=

  η where P is drawn from P 1RSB and η from P RS , and since P RS (η) = δ[η -η * ], we have actually the identity: dT (η)η = dP 1RSB (P )η(P ) = d P RS (η)η = η * (7.120) Similarly one has d T ( η) η = η * (7.121)

Figure 8 . 1 :

 81 Figure 8.1: Exemple of a discontinuous dynamic transition for k = 6, ω 0 = ω k = 0.005, ω 1 = ω k-1 = 0.92, and ω 2 = • • • = ω k-2 = 1. Top: C n (α, k, {ω p }) versus the number of iterations n for different values of α. Bottom: C(α, k, {ω p }) averaged over n after equilibration. The size of the population used is 10 6 .

Figure 8 . 2 :

 82 Figure 8.2: Top panel: example of a continuous bifurcation with f (x, α) = 1e -αx , for which α d = 1. Bottom panel: example of a discontinuous bifurcation with f (x, α) = 1 -e -αx 2 , for which α d = 2.45541.

Figure 8 . 3 :

 83 Figure 8.3: Properties of the iterates and fixed points for the discontinuous bifurcation undergone by the function f (x, α) = 1 -e -αx 2 . Top panel: x n as a function of n for a few values of α close but strictly less than α d . Bottom panel: the non-trivial solutions x ± (α) for α ≥ α d .

8 . 1 .

 81 Some examples of typical numerical results that can be obtained in this way are presented in Fig.8.1. On the top panel we plot the value of the point-to-set correlation C n as a function of the number n of iterations, for a few values of α. One sees on this plot, reminiscent of the top panel of Fig.8.3, the discontinuous birth of a non-trivial fixed point at α d , with a longer and longer plateau in the low α phase as a precursor of the transition. On the bottom panel of Fig.8.1 we present the asymptotic value of C reached for large n, for different values of α around the dynamic transition (corresponding to the bottom panel of Fig.8.3), that jumps discontinuously from 0 when α crosses α d .

Figure 8 . 4 :

 84 Figure 8.4: Study of the discontinuous dynamic transition encountered as a function of α for the choice of parameters k = 6, ω 0 = ω k = 0.005, ω 1 = ω k-1 = 0.92, ω 2 = • • • = ω k-2 = 1. Top panel: determination of α d from the study of the decorrelation time n * (α) for α < α d . The plot displays n * (α) -2 versus α, where one has defined n * (α) as the first iteration time for which the overlap (or point-to-set correlation) drops below the value C = 0.4. The line is a fit of the data of the form n * (α) -2 = A (α d -α), with fitting parameters A and α d . The linear behavior confirms the divergence of n * with a scaling exponent -1/2, as in the scalar case (8.9), the fit gives the estimation α d = 19.47. Bottom panel: determination of α d from the study of the stability parameter λ(α) for α > α d . The plot displays (1 -λ(α)) 2 versus α, the linear fits reproduces the scaling behavior (8.8) of the scalar case, and the fit yields α d = 19.467.

Figure 8 . 5 :

 85 Figure 8.5: The stability parameter λ(α) in the case of a continuous transition (here for k = 4, ω 0= ω k = 0, ω 1 = • • • = ω k-1 = 1). From the fit one obtains the result α = 4.088, wich is in good agreement with the analytical one α KS = 4.083.

Figure 8 . 6 :

 86 Figure 8.6: Phase diagram for k = 4, k = 5 and k = 6 (from top to bottom), in the plane (, α), at zero temperature ω 0 = 0. The RS phase, painted in gray, is on the left of α d ( ), the latter corresponds either to a continuous transition with α d ( ) = α KS ( ) (solid line, see(8.19)) or to a discontinous transition (black squares). The dashed horizontal line corresponds to = 0, the uniform measure, which intersects α d at α d,u . The arrow points to the optimal point that maximizes α d . The dotted line is the rigidity threshold α r from (8.20), the dot-dashed line marks the vanishing of the RS entropy (see Eq.(8.18)).

Figure 8 . 7 :

 87 Figure 8.7: Phase diagrams for k = 4, k = 5 and k = 6 (from top to bottom) in the plane (α, ), giving the RS phase delimitated by the KS bound and a dynamic line where the transition toward a non-trivial solution is discontinuous, for different temperatures. Top (k = 4): the dynamic line is given from left to right for ω = 0 (filled square and solid line), ω = 0.1 (empty triangle and dashed line), ω = 0.2 (filled triangle and dotted line). Middle (k = 5): from left to right ω = 0, ω = 0.002, ω = 0.005, ω = 0.01. Bottom (k = 6): from left to right ω = 0, ω = 0.005, ω = 0.01.

Figure 8 . 8 :

 88 Figure 8.8: Phase diagram for k = 4 (top), and k = 5 (bottom), in the plane (α, ), at zero temperature (ω 0 = 0): the solid line is the Kesten-Stigum (KS)transition where a non-trivial solution of the 1RSB equations emerge continuously from the trivial one upon increasing α, the filled (resp. empty) squares corresponds to the spinodal (sp) of the HO (resp. LO) branch that appears discontinuously when α is increased (resp. decreased). The filled circles are defined by the vanishing of the complexity of the HO branch.

Figure 8 . 9 :

 89 Figure 8.9: Overlaps C (left column) and complexities Σ (right column) versus at k = 5, for α = 9.5 (top), α = 9.6 (middle), and α = 9.7 (bottom).

Figure 8 . 10 :

 810 Figure 8.10: Lowest intensive energy reached by SA for k = 4 (top) and k = 5 (bottom) as a function of α for different cooling times τ . For each cooling time τ we show results for 3 problem sizes: N = 10 4 with filled circles, N = 10 5 with empty circles and N = 10 6 with triangles. The latter two values do always coincide (except for k = 5 and α = 9.8, where the N = 10 6 datapoint is missing).Estimating the algorithmic threshold from these plots is very difficult due to the strong τ dependency.

Figure 8 . 11 :

 811 Figure 8.11: Plotting the lowest intensive energy reached by SA with no bias ( = 0) as a function of τ provides a better way to estimate the algorithmic threshold α alg : the decay is faster (resp. slower) than a power law if α < α alg (resp. α > α alg ).

Figure 8 . 12 :

 812 Figure 8.12: The fast decays of u 0 as a function of τ for α = α opt and =

Figure 8 . 13 :

 813 Figure 8.13: The lowest intensive energy reached by SA reaches its minimum for a positive bias parameter .

9 Figure 8 . 14 :

 9814 Figure 8.14: The lowest energy reached with the optimal bias SA for the largest α value simulated as a function of τ .

b 2 = 1 . 4 b 2 = 1 . 3 b 2 = 1 Figure 8 . 15 :

 21421321815 Figure 8.15: An example of the shape of the correlation function C n as a function of n, here for k = 6, l = 114 and the bias function ψ(p) defined in Eq. (8.27), with b 1 = 1.6, = 0.095, and three values of b 2 .

Figure 8 . 16 :

 816 Figure 8.16: Phase diagrams in the (l, ) plane for the bias function ψ(p) = (1 -) p , for k = 5 (top panel) and k = 6 (bottom panel). The points are the clustering threshold l d ( ): for a given value of the RS phase corresponds to l < l d , the RSB phase to l ≥ l d . The three continuous lines are upper bounds of l d , the area on their right is necessarily in a RSB phase; the solid one is the Kesten-Stigum bound explained in Sec. 7.7, the dashed line marks the rigidity threshold l r defined in Sec. 7.4, and the dot-dashed line corresponds to the vanishing of the RS entropy (7.42).

Figure 8 . 17 :

 817 Figure 8.17: The phase diagrams in the (b 1 , b 2 ) parameter plane for the bias function of Eq. (8.27). The top panel is for k = 5, l = 48 and = 0.06, the bottom panel for k = 6, l = 114, = 0.095. The points marked with squares corresponds to a RS phase (l < l d (k, b 1 , b 2 , )), the crosses to a RSB phase (l ≥ l d (k, b 1 , b 2 , )). The line is the Kesten-Stigum bound, the area below it is RSB.

k uniform (b 1 , b 2 ,

 12

. 4 )Figure 9 . 1 :

 491 Figure 9.1: Top: the recursion function e -γ+x (solid lines) for three values of γ (from top to bottom γ = 0.9, γ = 1 = γ r , γ = 1.2) compared to x (dashed line). Bottom: the iterates x n defined by x 0 = 0, x n+1 = e -γ+xn for three values of γ (from top to bottom γ = 0.99, γ = 1 = γ r , γ = 1.2).

Figure 9 . 2 :

 92 Figure 9.2: Top: the correlation function C n (α, k) as a function of n for k = 5 and from left to right α = 9.2, α = 9.4, α = 9.6. Bottom: the large n limit C(α, k) = lim n C n (α, k) as a function of α for k = 5, along with the contribution of the hard fields H(α, k) = lim n H n (α, k) discussed in Sec. 7.4. The first curve is non zero for α ≥ α d (k = 5) ≈ 9.465, the second one for α ≥ α r (k = 5) ≈ 10.526.

Figure 9 . 3 :Figure 9 . 4 :

 9394 Figure 9.3: Top: the reduced correlation function C n (γ) as a function of n for a few values of γ ≥ γ r = 1. Bottom: the large n limit C(γ) = lim n C n (γ) as a function of γ, along with the contribution of the hard fields x(γ) = lim n x n (γ).The first curve exists for γ ≥ γ d ≈ 0.871, the second one for γ ≥ γ r = 1. The data for γ < 1 have been obtained through the reweighted algorithm expained in Sec. 9.2.3.

. 47 )

 47 one sees that under the law ν (≤) n+1 the random variable h satisfies the distributional equality h d = l i=1 y i , where in the right hand side l is a Poisson random variable of mean λ (≤) n and the y i are i.i.d. with the law π

  repeating N times, independently for i = 1, . . . , N :• draw a random number l from a Poisson distribution of parameter λ

Figure 9 . 5 :

 95 Figure 9.5: The results of the reweighted distribution method in the neighborhood of γ d : on the top panel the reduced order parameter C n (γ) as a function of n, on the bottom panel the inverse of the mass m n of the reweighted measure ρ n . On both panels the five curves correspond to γ = 0.85, γ = 0.86, γ = 0.87, γ = 0.88, γ = 0.89, from top to bottom on the top panel and from bottom to top on the bottom panel. These curves show that 0.87 < γ d < 0.88, C n and m n diverge for γ = 0.87, even if it is not completely visible on the displayed range of n.

Figure 9 . 6 :

 96 Figure 9.6: The relationship between the parameter γ and the mass m of the reweighted measure ρ in the n → ∞ limit. The numerical results suggest that m reaches 1 with a square root behavior as γ tends to γ d .

1 ) 2 - 1 )

 121 for all h ∈ [-1, 1]. As a consequence the sequences of masses m n obey the inductive bounds m n+1 ≥ exp[-γ +2( √ 2-1)m n ]. The function m → exp[-γ +2( √ 2-1)m] being increasing the sequence m n defined by m 1 = e -γ , m n+1 = exp[-γ + 2( √ m n ] can be shown by induction to lower bound the sequence m n , namely m n ≥ m n for all n. It is then simple to show that for γ < 1 + ln(2( √ 2 -1)) ≈ 0.812 the sequence m n diverges with n, and as a consequence so does m n . Under the assumption spelled out at the end of Sec. 9.2.4 on the equivalence of the divergence of m n and of C n this yields the lower bound γ d ≥ 1 + ln(2( √ 2 -1)).

Figure 10 . 1 :

 101 Figure 10.1: For k = 20, plot in the plane (α, ) of the rigidity line and zero RS entropy line, that intersect at the point (α * (20), * (20)).

  .[START_REF] Braunstein | Survey propagation: An algorithm for satisfiability[END_REF] where W (z) is the Lambert function, i.e. the principal solution of the equation z = W e W . Note that this result coincides with the asymptotic expansion of l r one obtains from(10.23), which shows the commutativity of the limits n → ∞ and k → ∞ for the determination of the rigidity transition.The function γ r (b) is plotted in figure10.6 (bottom panel, upper curve): it is a decreasing function of b, with γ r (1) = 1 for the uniform measure. An example for the values of the fixed point reached by x 0,n for γ > γ r (b) can be found in the bottom panel of Fig.10.3.

eFigure 10 . 2 :

 102 Figure 10.2: Top panel: the functions x and B e -γ + e -γ+x as a function of x, with b = 0.4 and from top to bottom γ = 1.25, 1.38, 1.50. The bifurcation occurs when these two functions cross, which happens here at γ r (0.4) ≈ 1.378. Bottom: x 0,n as a function of n, for b = 0.4 and from top to bottom γ = 1.25, 1.38, 1.50.

P 1 ,

 1 n (p) = Po(p; x 0,n ) , P 0,n (p) = 1 1 -b + b e x0,n × where we have introduced the notation Po(p; λ) = e -λ λ p

2 .

 2 will denote the corresponding probability density as D α = dα √ Collecting all these facts yieldsQ w,n+1 (η) = ∞ p,q,r=0P w,n (p)Po(q; x 1,n )Po(r; x 0,n ) D α

(

  ), in which one can take = 0 at this leading order; explicitly, η = f Θ1 ( η 1 , η 2 , . . . ) means η(σ, 1) = 1 z b i ( η i (σ, 0) + η i (σ, 1)) , (10.43) η i (σ, 0) + η i (σ, 1)) ,with σ = ±1.

( 10 .

 10 [START_REF] Mézard | The bethe lattice spin glass revisited[END_REF] with a constant γ r (b) easily determined from the large n behavior of H n : for γ < γ r (b) one has H n → ∞ as n → ∞, while H n remains bounded for γ ≥ γ r (b). The computation of the reduced correlation function C n requires instead the resolution of the functional recursion equation (10.42) on the distributions of the soft-fields Q w,n , supplemented by the initial condition (10.44), from which C n is computed using the equation(10.48). The sequence C n depends on the parameters γ, b and , and the constant γ d (b, ) in the asymptotic expansion of the dynamic threshold l d is deduced from the large n asymptotics of C n (if γ < γ d (b, ) then C n → ∞, while it remains bounded for γ > γ d (b, )).

C 1 ηFigure 10 . 3 :

 1103 Figure 10.3: Top: the reduced correlation function C n as a function of the distance n for b = 0.4, = 0, and several values of γ larger than the rigidity threshold γ r (0.4) ≈ 1.378. Bottom: the large distance limit C (points) and its hard-field contribution H (solid line) as a function of γ for b = 0.4 and = 0. The points of C below the rigidity threshold have been obtained with the reweighted algorithm presented in Sec. 10.4.2.

Figure 10 . 4 :

 104 Figure 10.4: The reduced correlation function C n and its hard-field contribution H n as a function of the distance n for b = 0.4, = 0 and γ = 1.35, slightly below the rigidity threshold.

( 1 )

 1 + z (2) ) 2 -x 0,n -x 1,n +x 1,n dQ 1,n (η) η(+, 0) + η(-, 0) η(+, 1) e iz•V-(η) +x 0,n dQ 0,n (η) η(+, 1) η(+, 0) + η(-, 0) e iz•V+(η) + η(-, 1) η(+, 0) + η(-, 0) e iz•V+(η f ) . (10.69)

( 1 ) 2 +( 1 ) 2 + 2 + 2 2 (z ( 1 )

 1212221 + z(2) ) dρ n (η) η(+, 0) + η(-, 0) η(-, 1)e iz•V-(η)+ dρ n (η) η(+, 0) η(-, 1)(η(+, 0) + η(-, 0))e iz•V+(η) + η(+, 0) η(+, 1)(η(+, 0) + η(-, 0)) e iz•V+(η f ) -dρ n (η) η(+, 1) + η(-, 1)η(+, 1)η(-, 1) . (10.70)Using the invariance under spin-flip of ρ n one can regroup the two terms in the second line of this equation; simplifying the prefactors one obtainsρ n+1 (z) = exp -γ -B e -γ -+ z (2) ) dρ n (η) η(+, 0) + η(-, 0) η(-, 1) e iz•V+(η) + e iz•V-(η) -η(+, 1) + η(-, 1) η(+, 1)η(-, 1) . (10.71)This is a recursion equation for the reweighted measure ρ n (and its characteristic function ρ n ). It will be more convenient in the following to work with the pair (ν n , m n ); the mass m n of ρ n can be expressed as ρ n (0), we thus obtainm n+1 = exp -γ -B e -γ -2 m n dν n (η)M (η) , (10.72)ν n+1 (z) = exp -+ z (2) ) 2 + m n dν n (η)L(η) e iz•V+(η) + e iz•V-(η) -2 , (10.73)where we introduced the functionsM (η) = η(+, 0) + η(-, 0)( η(+, 1) + η(-,1)) -η(+, 1) -η(-, 1) η(+, 1)η(-, 1) , (10.74) L(η) = η(+, 0) + η(-, 0) η(-, 1) . (10.75)

2 2 (z ( 1 )

 21 + z(2) ) 2 exp dπ n ( u)(e iz• u -1) ,(10.82) where we have introduced a measure π n of total mass we shall denote λ n , according toπ n ( u) = m n dν n (η)L(η) (δ( u -V + (η)) + δ( u -V -(η))) , (10.83)λ n = dπ n ( u) = 2m n dν n (η)L(η)

2 2 (z ( 1 )

 21 + z (2) ) 2 ν (≤) n+1 (z) ν

  n+1 (z) = exp m n dν n (η)L(η) e iz•V+(η) + e iz•V-(η) -2 I [L(η) ≤ ξ n ] ,(10.86)ν (>) n+1 (z) = exp m n dν n (η)L(η) e iz•V+(η) + e iz•V-(η) -2 I [L(η) > ξ n ] , (10.87)

  n+1 , the other with the law ν (>) n+1 . We describe the distribution ν (≤) n+1 using the distributional interpretation explained above, definingπ (≤) n ( u) = m n dν n (η)L(η) (δ( u -V + (η)) + δ( u -V -(η))) I [L(η) ≤ ξ n ] ,(10.88)λ (≤) n = dπ (≤) n ( u) = 2m n dν n (η)L(η)I [L(η) ≤ ξ n ] . (10.89) Under the law ν (≤) n+1 the variable u obeys the distributional equality u = p i=1 u i where p is a Poisson variable of mean λ (≤) n , and the u i 's are i.i.d copies extracted from π n+1 is instead approximated by a multivariate Gaussian Γ(V n , Σ n ) with V n and Σ n the mean and the covariance matrix of ν (>) n+1 , computed by taking derivatives of ln ν (>) n+1 with respect to z:V n = m n dν n (η)L(η)I [L(η) > ξ n ] (V + (η) + V -(η)) , (10.90)Σ (a),(b) n = m n dν n (η)L(η)I [L(η) > ξ n ] (V + (η) (a) V + (η) (b) + V -(η) (a) V -(η) (b) ) , (10.91)for a, b ∈ {1, 2, 3}. As V -(η) = (V + (η)) f several components of V n and Σ n vanish, namely V

Figure 10 . 6 :

 106 Figure 10.6: Top panel: γ d as a function of 2 for b = 1, the line being a guide to the eye. Bottom panel: γ d as a function of b for = 0, the solid line corresponding to the rigidity upper bound γ r (b).

( 1 -

 1 that η = f c (η 1 , . . . , η l ) means η(τ ) = l i=1 η i (τ ))

  being in the interval [1/ √ b, 1]. Combining these observations we obtain finally c -(b)x 1,n ≤ λ n ≤ c + (b)x 1,n , with for b ≤ 1 c -(b) = 2, c + (b) = 2 2/b, and for b ≥ 1 c -(b) = 2/ √ b, c + (b) = 2 √ 2.

Table 1 .

 1

	1: Some decision problems belonging to NP or NP-c
	P	NP-c
	2-SAT	SAT
	2-col	3-SAT
	XORSAT	3-col
	assignment NAE-k-SAT with k ≥ 3
		Hamiltonian circuit

end if end for return σ T

	Algorithm 3 Metropolis Algorithm
	Generate σ 0 uniformly at random from {+1, -1} N
	for t = 1 to t = T do
	Draw i uniformly at random in {1, . . . , N }
	Compute ω i (σ t-1 ) from (3.5)
	Draw a random number p uniformly in [0, 1]
	if p < ω i (σ t-1 ) then
	Set σ t = σ	(i) t-1
	else
	Set σ t = σ t-1

Algorithm 7

  Belief Propagation(G,{ω a } a∈C ,t max , )

	For each edge (a, i), initialize the message η i→a
	for t = 1 to t max do Compute the messages { η a→i } (i,a)∈E , then {η (t)	(t+1) i→a } (i,a)∈E from (4.4)
	Let ∆ be the maximum message change
	if ∆ < then return the set of messages {η	(t+1) i→a , η	(t) a→i }
	end if
	end for
	return "not converged"

  {G s , G s } are updating functions. One can then choose ϕ i to be a function of the auxiliary variables { g

	where the (s) a→i }	s∈{1,...,r} a∈∂i	surrounding it:
	s) j→a } i∈∂a\i ) where s ∈ {1, . . . , r}	(6.21)
	g j→a = G s+1 ({ g (s+1) b→j } b∈∂a\j ) where s ∈ {1, . . . , r -1} (s)	(6.22)

  version introduced in the section 4.1.6 that only keeps sure beliefs and discards the other ones. It is clear that if the answer to this new question is yes then the inferred value of σ i is the correct one that was used in the broadcast, hence if reconstruction is possible in this strong sense of certain inference it is also possible in the definition introduced in the section 7.3.2. We call H n the probability of this event. It turns out that H n is much simpler to compute than C n , with scalar recursions instead of functional ones, and that H n is a lower bound for C

n ; this last fact is quite intuitive, if σ Bn implies the value of σ 0 it certainly conveys information about it.

  .4).

		0.00025
		0.0002
		0.00015
	n * (α) -2	0.0001
		5e-05
			0
		-5e-05
			19.36	19.38	19.4	19.42	19.44	19.46	19.48
						α
			0.01
			0.008	data
				fit
			0.006
	(1 -λ(α)) 2	0.004
			0.002
			0
		19.46 -0.002	19.48	19.5	19.52	19.54	19.56	19.58
						α

  .35) Note that β(h) ≤ 1 for h ∈ [-1, 1], hence the integral in(9.34) is smaller than m n , which allows to check inductively that m n ≤ x n , in agreement with the direct derivation of this bound from (9.21). Dividing the expression (9.33) of ρ n+1 (z) by its total mass m n+1 we obtain the characteristic function ν n+1 (z) of ν n+1 , the normalized probability distribution. Replacing ρ n by m n ν n we thus have a recursion equation on the pair (m n , ν n ), namely

	m n+1 = exp -γ + m n dν n (h)β(h) ,	(9.36)

  [START_REF] Achlioptas | Rigorous location of phase transitions in hard optimization problems[END_REF], even if it is not completely visible on the displayed range of n.

	1
	0.95
	0.9
	0.85
	m
	0.8
	0.75
	0.7
	0.65
	0.86	0.87	0.88	0.89	0.9	0.91	0.92	0.93	0.94	0.95	0.96
						γ

  9.55) one should have ρ > 0. Let us indeed rewrite this equation in terms of the Laplace transforms as L(t) = exp[R(t)], with R(t) = dπ(y)(e ty -1).

	Consider now the tail behavior of the measure π related to ν by (9.55); a simple
	computation, based in particular on the fact that a( h) = h+O(e -2 h ) as h → ∞,
	shows that the asymptotic expansion (9.56) is also valid for the tail of π, up to
	the multiplicative coefficient m: π({y ≥ x}) ∼ m ν({ h ≥ x}) as x → ∞. As
	a consequence if ρ < 0 both L(t) and R(t) would diverge as (r(m) -t)

ρ when t → r(m), which is clearly incompatible with the equation L(t) = exp[R(t)].

(see equations (4.1) and 4.2)). We write the BP equations for the two graphical models Θ 0 and Θ 1 representing the biased measures.

Acknowledgments

This shows that the arguments of the square roots in (10.54) are non-negative numbers, as they should for the definition of u to be meaningful. Moreover this expression reveals the motivation for this peculiar choice of parametrization: the BP equation η = f Θ1 ( η 1 , η 2 , . . . ) becomes multiplicative with respect to its arguments when η is expressed in terms of u. We will also use the notation u = ( u (1) , u (2) , u (3) ) with u = (e u (1) , e u (2) , e u (3) ); as the components of u are positive those of u are real numbers, and the BP equation becomes additive in terms of u. It will also be useful to define the spin-flip operation on the triplets u and u; as η f (σ, w) = η(-σ, w) one deduces easily from (10.54) the corresponding transformations: (1) , 1 u (2) , u (3) , u f = (u (1) , -u (2) , u (3) ) . (10.56) In the following we will take the liberty to use the three equivalent parametrizations η, u and u according to which one is the most convenient, keeping implicit the relationships between them that we have just defined.

Let us now rewrite (10.42) by translating the image η of the function f Θ1 in the u parametrization:

where we defined V 0 (α) = (α, α, 0) , (10.58)

For completeness we also state the expression of V + with its argument translated 205 in the u parametrization, namely

u (1) u (2) , (10.61)

u (1) , (10.62)

Because of the additivity property of the parametrizations in terms of u it is easier to describe Q w,n in terms of its characteristic function, that we define as

where z = (z (1) , z (2) , z (3) ) and we denoted the standard scalar product z • u = z (1) u (1) + z (2) u (2) + z (3) u (3) . Indeed the equation (10.57) translates into

where the first factor comes from the Gaussian integration on α. For w = 1 the three integers p, q, r have Poisson distributions, the sums can then be easily performed to obtain

We can now come back to the reweighted measure ρ n we introduced in (10.51), and its normalized version ν n , for which we define the characteristic functions similarly

The reweighting factor η(-,1) η(+,1) between ρ n and Q 1,n can be expressed as e -u (1) , the characteristic functions of these two measures are thus linked by a simple shift of their arguments: (1) ⇐⇒ ρ n (z) = x 1,n Ξ 1,n (z + ie (1) ) . (10.68)

Algorithmic implementation

We now give an explicit description of the algorithm we implemented to solve the recursion equations (10.72,10.73) for m n and ν n . Suppose that at the n-th step of the iteration we have an estimation of m n and of ν n , with ν n represented as a population of fields: (10.91).

Each of the N elements of the new population representing ν n+1 is then generated independently of the others, by translating equation (10.85) as follows:

• draw an integer p from the Poisson law of mean λ (≤) n .

• extract i 1 , . . . , i p independently in {1, . . . , N n } with probability proportional to L(η i ) (this can be done efficiently by precomputing a cumulative table).

• insert in the new population (1) + e (2) ) + V n + g where the δ i 's are ± with equal probability, α is a centered Gaussian random variable of variance 2 , and g is a centered three-dimensional Gaussian vector with covariance matrix Σ n .

We can then compute the reduced overlap C n+1 from (10.53), and sort the elements of the new population according to their values of L(η).

In practice we chose the threshold ξ n (or equivalently N n ) in an adaptive way: for each iteration step we took the largest N n ≤ N that gave λ (≤) n ≤ λ, where λ is a parameter fixed beforehand. The accuracy of this numerical procedure is thus controlled by N , the approximation in (10.92) being better and better as N grows, and by λ, the Gaussian truncation being more precise when λ is larger. Obviously the memory and time requirements of the procedure also increase with N and λ; the numerical results presented below have been obtained with population sizes between 10 6 and 10 7 , and λ around 20, we checked that the conclusions were not modified, within our numerical accuracy, by modifying these values within a reasonable range.

Results

In Fig. 10.5 we complete our study of the case b = 0.4, = 0 that was started in Figs. 10.3 and 10.4. The reweighting procedure allows now to investigate the regime γ < γ r ≈ 1.378; as displayed on the top panel of Fig. 10.5 the large distance limit C remains bounded for values of γ down to 0.98, these results being reported as a function of γ in the bottom panel of Fig. 10.3. Further simulations allowed us to pinpoint more precisely γ d (b = 0.4, = 0) ≈ 0.977, as the largest value of γ for which C n diverges. As shown in the bottom panel of Fig. 10.5 the condition of divergence of C n coincides, within our numerical accuracy, with the divergence of m n . The value of the large n limit of m n is seen to be close to 1 when γ reaches γ d from above (see the plateau in the bottom panel of Fig. 10.5), an observation that we also made for the other values of b we investigated. We have given analytical arguments in [START_REF] Budzynski | The asymptotics of the clustering transition for random constraint satisfaction problems[END_REF] that indeed the plateau value of m n is exactly equal to 1 at γ d for the uniform measure, our numerical results suggest that this remains true when (b, ) = (1, 0), even if we do not have analytical support for this assumption in the general case.

We have repeated this procedure of determination of γ d (b, ) for various values of b and , and we present now the phase diagrams obtained in this way. Consider first the top panel of Fig. 10.6, which deals with the case b = 1, i.e. the bias factorized over the hyperedges considered in [START_REF] Budzynski | Biased landscapes for random constraint satisfaction problems[END_REF]. We see that for all values of = 0 one has γ d (1, ) < γ d (1, 0) = γ d,u , i.e. this bias has, in the large k limit, a detrimental effect on the dynamic phase transition that is pushed to lower values with respect to the one of the uniform measure. In the bottom panel of Fig. 10.6 we have plotted instead the threshold γ d as a function of b for = 0; one sees now that decreasing b below 1 (that corresponds to the uniform measure and is marked as an horizontal dashed line on figure) has a beneficial effect with an increase of γ d . The largest value we could reach was for b = 0.4, decreasing b further below reduces again γ d . The lowest value of b we could investigate was b = 0.3, for b < 0.3 we encountered numerical accuracy problems, the distribution ν n exhibiting strong fluctuations that prevented an accurate representation as a population. Finally in Fig. 10.7 we have checked that the parameter has a detrimental effect also for values of b = 1, we found indeed that γ d (b, ) < γ d (b, 0) when = 0, for all the values of b we considered. This leads us to the conclusion that, within the biasing strategy we considered in the large k limit, the optimal choice of parameters is = 0 and b ≈ 0.4, yielding a constant γ d ≈ 0.977, strictly larger than the one of the uniform case, γ d,u ≈ 0.871.

Appendix A

Existence and uniqueness of the RS solution

In this appendix we shall show that the translationally invariant RS equation (7.40) admits a unique solution for all choices of the bias function ψ that is strictly positive, ψ(p) > 0 ∀p ∈ {0, . . . l + 1}.

We first remark that in the uniform case, where ψ(p) is a positive constant independent of p, the equation (7.40) obviously admits a unique solution (with y = 1, y = 2 k-1 -2). We will now show that the number of solutions cannot change when ψ varies in its allowed domain. To achieve this we first rewrite (7.40) in the equivalent form

where for simplicity we denoted x = 1/ y and where the coefficients X p (x) are:

in the first line we used the convention l l+1 = l -1 = 0. The function G(x; ψ) introduced in (A.1) depends smoothly on its two arguments (polynomially in x, and linearly in ψ), the number of solutions x(ψ) of the equation G = 0 can thus only change at a bifurcation point, i.e. a pair (x; ψ) such that G(x; ψ) = ∂ x G(x; ψ) = 0, otherwise the implicit function theorem allows to smoothly continue any branch of solution. As we remarked above the solution is unique when ψ is independent of p, the uniqueness for all ψ will then follow if we show the absence of solution to the bifurcation equation:

where

(A.5) This equation being invariant under the multiplication of ψ by a positive constant we can further assume the normalization condition

For a given value of x, the existence of a ψ satisfying (A.4,A.6) is equivalent to the origin of R 2 being in the convex hull of the l + 2 points of coordinates

for p = 0, . . . , l + 1. We can then invoke the Caratheodory theorem that states that any point of the convex hull of a set A ⊂ R d can be written as the convex combination of d+1 points of A. Here d = 2, so the absence of solutions of (A.4,A.6) follows from the impossibility to satisfy, for any p, q, r ∈ {0, . . . , l + 1}, the system α p X p (x) + α q X q (x) + α r X r (x) = 0 , (A.7)

This is equivalent to the three quantities X p (x)Y q (x)-X q (x)Y p (x), X q (x)Y r (x)-X r (x)Y q (x) and X r (x)Y p (x) -X p (x)Y r (x) being of the same sign; using the expressions (A.3,A.5) of X and Y we have checked the impossibility of this condition, for all x > 0 and all triplets p, q, r, which concludes the reasoning.

Appendix B

The graph coloring case

This Appendix is devoted to the graph coloring problem. We shall present computations that are the counterparts of the ones explained in Sec. 7.3 and 9.1 of the main text for the hypergraph bicoloring problem, namely the definition of the reconstruction problem and its study in the limit where the number of colors q and the degree c diverge simultaneously according to (9.1), and show that the very same equations summarized at the beginning of Sec. 9.2 arise in this limit. As the computations are quite similar we will be more succint than in the main text and concentrate on the specificities of the coloring problem. Let us consider a rooted tree with spins σ i placed on its vertices. These spins can take q values, interpreted as colors, σ i ∈ {1, . . . , q}. A proper coloring of the tree is a configuration of the spins such that no edge is monochromatic (i.e. no pair of adjacent vertices are given the same color). A uniform proper coloring can be drawn in a broadcast fashion, by choosing the color σ of the root uniformly at random among the q possible ones, then each descendent of the root is assigned a color uniformly at random among the q -1 colors distinct from σ, and this is repeated recursively down to the n-th generation of the tree. In the reconstruction problem an observer is then provided with the colors on the vertices of the n-th generation of the tree only, and asked to guess the color of the root. The optimal strategy is to compute η, the posterior probability of the root given the observations, which is a distribution over {1, . . . , q}. A moment of thought reveals that the probability distribution of η, with respect to a broadcast process conditioned on the root value σ, and with respect to a random choice of the tree as a Galton-Watson branching process with Poisson offspring distribution of mean c, is a measure P σ,n (η) that can be determined recursively through the induction relation:

where the Belief Propagation recursion function f c is defined here in such a way of (B.1) and obtain, without any approximation,

where l is the number of neighbors of the root that receive a soft field, σ 1 , . . . , σ l the colors these vertices have in the broadcast, and the indicator variables p σ are equal to 1 if and only if no neighbor of the root assigned the color σ in the broadcast is perfectly recovered (i.e. receives a hard field). Hence the colors σ = σ with p σ = 0 are precisely the ones forbidden for the root, as at least one of its neighbors is forced to this value. The relation η = f c (σ, {p σ } σ =σ ; η 1 , . . . , η l ) is obtained by specializing f c of (B.2) to this pattern for the presence of hard fields in its arguments, and thus reads

with the convention p σ = 1. Let us call p = σ =σ p σ the number of colors that satisfy the condition explained above; in the equation (B.6) it corresponds to a random variable with a binomial distribution of parameters (q -1, e -cHn q-1 ). According to (B.5) the product of these parameters go to zero as 1/ ln q in the limit we are considering, we shall thus truncate (B.6) on the smallest possible values of p. As p = 0 yields a hard field in the left hand side of (B.6), the distribution of the soft fields is dominated in this limit by the case p = 1. As a consequence the fields η in the support of Q σ,n have non-zero values on two colors only, σ and another one σ uniformly distributed on the q -1 possibilities. Let us parametrize this type of distributions via a distribution

Let us also denote f c (σ, σ ; η 1 , . . . , η l ) the function f c (σ, {p σ } σ =σ ; η 1 , . . . , η l ) with p σ = δ σ ,σ , in such a way that the only two non-zero components of

Appendix C An Inequality

We provide in this Appendix a proof of the bounds c -(b)x 1,n ≤ λ n ≤ c + (b)x 1,n that we used in Sec. 10.4.3. We start by stating some inequalities that are fulfilled by the messages η in the support of ν n , and that are consequences of the BP equation (10.17). They are more compactly stated in terms of the u-parametrization; from (10.55) one obtains indeed u (2) u (3) u (1) = i η i (+, 0) η i (+, 0) + η i (+, 1) , u (1) u (3) u (2) = i η i (-, 0) η i (-, 0) + η i (-, 1)

, (C.1) which allows to conclude that u (2) u (3) u (1) ≤ 1 and u (1) u (3) u (2) ≤ 1, for all the η's in the support of ν n .

Consider now the expressions (10.52) for x 1,n , and (10.84) for λ n ; the ratio of the integrands in these two equations reads 2 η(+, 0) + η(-, 0) η(+, 1) + η(-, 1) η(+, 1) + η(-, 1) = η(+, 0) + η(-, 0) η(+, 1) + η(-, 1) 2 η(+, 1) η(+, 1) + η(-, 1) + 2 η(-, 1) η(+, 1) + η(-, 1)

The parenthesis in the right hand side of this eqation is of the form 2( √ x + √ 1 -x) for some x ∈ [0, 1], which is necessarily in the interval [2, 2 √ 2]. The prefactor in front of the parenthesis can be written, in terms of th u-parametrization, 1 1 + (u (1) ) 2 1 + Bu (1) u (2) u (3) + (u (1) ) 2 + B u (1) u (3) u (2) (C.2)

Consider first the case b ≤ 1, i.e. B ≥ 0; as the components of u are non-negative the expression in (C.2) is certainly lower bounded by 1. Moreover the bounds u (2) u (3) u (1) ≤ 1 and u (1) u (3) u (2) ≤ 1 imply that it is upper bounded by √ 1 + B = 1/ √ b. The case b ≥ 1, B ≤ 0 can be treated similarly, with now the expression in (C.2)