Keywords: polynomial optimization, Moment-SOS hierarchy, convergence rate, semidefinite programming, Putinar's Positivstellensatz, Putinar-Vasilescu's Positivstellensatz, Reznick

s Positivstellensatz, Pólya's Positivstellensatz, constant trace property, correlative and term sparsity Σ[x] cone of sums of squares of polynomials f 2 1 + . . . , f 2 r R[x] d space of polynomials of degree at most d Σ[x] d cone of SOS polynomials of degree at most 2d q = α q α x α polynomial in x with coefficients q α ∈ R deg(q) the degree of q ∈ R[x] defined by max{|α| : q α = 0} v d vector of monomials up to degree d, i.e., (x α ) α∈N n d b(n, d), b(d) n+d d Polynomial optimization k relaxation order t sparse order s the upper bound on the maximal block size p number of cliques R the positive parameter in the ball constraint f objective polynomial g set of polynomials {g 1 , . . . , g m } involved in the inequality constraints h set of polynomials {h 1 , . . . , h l } in the equality constraints S(g) basic closed semialgebraic set defined by {x ∈ R n : g

y) the moment matrix of order d defined by (y α+β ) α,β∈N n d M d (qy) the localizing matrix of order d defined by ( γ q γ y α+β+γ ) α,β∈N n d Q d (g) the truncated quadratic module of order d associated with g I d (h) the truncated ideal of order d associated with h Q d (g, h) Q d (g) + I d (h) Programming and softwares LP linear program SDP semidefinite program IP interior point method ADMM the alternating direction method of multipliers SBM spectral bundle methods SM spectral methods without bundle CGAL conditional gradient-based augmented Lagrangian yes no 1993 Putinar [168] If a polynomial f is positive on S(g) satisfying the Archimedean assumption, then f = σ0 + m i=1 σigi for some σi ∈ Σ[x]. yes no 1995 Reznick [175] If f is a positive definite form, then x 2k 2 f ∈ Σ[x] for some k ∈ N. no yes (prescribed) 1999 Putinar-Vasilescu [170]

If a polynomial f is nonnegative on S(g), then for every ε > 0, there exists k ∈ N such that

, where

m i=1 σigi for some σi being SOS of monomials, where d := 1 + deg(f )/2 and θ := x 2 2 + 1. no yes (prescribed)

, the authors combined CGAL with the Nyström sketch (named SketchyCGAL), which requires dramatically less storage than other methods and is very efficient for solving Shor's relaxation of large-scale Max-Cut instances.

SDP relaxations of non-convex quadratically constrained quadratic programs.

A nonconvex quadratically constrained quadratic (QCQP) program is a special instance of POP (1.2.1) for which the degree of the input polynomials is at most two. Famous instances of non-convex 1.7.1 Articles in peer-reviewed international journals

Résumé

Les certificats de positivité ou Positivstellensätze fournissent des représentations de polynômes positifs sur des ensembles semialgébriques de basiques, c'est-à-dire des ensembles définis par un nombre fini d'inégalités polynomiales. Le célèbre Positivstellensatz de Putinar stipule que tout polynôme positif sur un ensemble semialgébrique basique fermé S peut être écrit comme une combinaison pondérée linéaire des polynômes décrivant S, sous une certaine condition sur S légèrement plus forte que la compacité. Lorsqu'il est écrit comme ceci, il devient évident que le polynôme est positif sur S, et donc cette description alternative fournit un certificat de positivité sur S.

De plus, comme les poids polynomiaux impliqués dans le Positivstellensatz de Putinar sont des sommes de carrés (SOS), de tels certificats de positivité permettent de concevoir des relaxations convexes basées sur la programmation semidéfinie pour résoudre des problèmes d'optimisation polynomiale (POP) qui surviennent dans diverses applications réelles, par exemple dans la gestion des réseaux d'énergie et l'apprentissage automatique pour n'en citer que quelques unes. Développée à l'origine par Lasserre, la hiérarchie des relaxations semidéfinies basée sur le Positivstellensatz de Putinar est appelée la hiérarchie Moment-SOS.

Dans cette thèse, nous proposons des méthodes d'optimisation polynomiale basées sur des certificats de positivité impliquant des poids SOS spécifiques, sans ou avec dénominateurs.

La première partie de ce manuscrit se concentre sur les méthodes sans dénominateurs, et basées sur le Positivstellensatz de Putinar.

Nous proposons une nouvelle hiérarchie Moment-SOS pour résoudre des problèmes d'optimisation de polynômes creux à grande échelle. Sa nouveauté est d'exploiter simultanément la parcimonie corrélative et la parcimonie des termes en combinant les avantages de deux cadres existants pour l'optimisation des polynômes creux. Ce faisant, nous obtenons (i) une hiérarchie à deux niveaux de relaxations de programmation semidéfinies avec la propriété cruciale de comporter des blocs de matrices SDP (au lieu d'une seule grande matrice), et (ii) la garantie de convergence vers l'optimum global sous certaines conditions. Nous démontrons son efficacité et son évolutivité sur plusieurs instances à grande échelle du célèbre problème Max-Cut et sur certaines instances de l'important problème industriel de flux de puissance optimal (OPF), impliquant jusqu'à six mille variables et des dizaines de milliers de contraintes.

Ensuite, nous prouvons que chaque relaxation moment semidéfinie d'un POP contraint peut être reformulée comme un programme semi-défini faisant intervenir une matrice ayant la propriété de trace constante (CTP). En conséquence, de telles relaxations peuvent être résolues efficacement par des méthodes du premier ordre qui exploitent le CTP, par exemple, la méthode de Lagrangien augmenté basée sur le gradient conditionnel. Nous étendons également ce cadre d'exploitation de CTP aux POPs à grande échelle avec différentes structures de parcimonie. L'efficacité de cette méthode est illustrée sur des relaxations de moment de second ordre pour divers programmes quadratiques à contrainte quadratique générés aléatoirement.

La deuxième partie de ce manuscrit porte sur les méthodes avec dénominateurs, basées sur les certificats de positivité dus à Putinar et Vasilescu, Reznick, ainsi qu'à Pólya.

Nous revisitons deux certificats de positivité sur des ensembles semialgébriques basiques (éventuellement non compacts) dus à Putinar et Vasilescu. Nous utilisons la technique de Jacobi pour fournir une preuve alternative avec un degré effectif lié aux poids SOS dans de tels certificats. En conséquence, nous pouvons définir une hiérarchie de relaxations semidéfinies pour les POPs généraux. La convergence vers un voisinage de la valeur optimale et la dualité forte sont garanties. Dans une seconde contribution, nous introduisons une nouvelle méthode numérique pour résoudre des systèmes d'inégalités et d'égalités polynomiales avec éventuellement un nombre indénombrable de solutions. En prime, on peut appliquer cette méthode pour obtenir des optimiseurs globaux approchés en optimisation polynomiale.

Nous fournissons ensuite une nouvelle borne sur le degré des poids SOS dans le Positivstellensatz de Putinar-Vasilescu et obtenons le nouveau Positivstellensatz suivant:

Si f est un polynôme de degré au plus 2d f , positif sur l'ensemble semi-algébrique S := {x : g i (x) ≥ 0, i ∈ [m]} à intérieur non vide (et avec g 1 := R -x 2 2 avec R > 0), alors il existe des constantes positives c et c dépendantes de f, g i telles que pour tout ε > 0, pour tout k ≥ cε -c , (1 + x 2 2 Enfin, nous considérons la minimisation d'un polynôme sur un ensemble semialgébrique contenu dans l'orthant positif. Il peut être converti en un POP équivalent en élevant au carré chaque variable. En utilisant la parité et le concept de largeur de facteur, nous proposons une hiérarchie de relaxations semidéfinies basée sur l'extension du Positivstellensatz de Pólya par Dickinson-Povh. Comme caractéristique distinctive et cruciale, la taille maximale de la matrice de chaque relaxation semidéfinie résultante peut être choisie arbitrairement. De plus, la suite de valeurs renvoyée par la nouvelle hiérarchie converge vers la valeur optimale du POP d'origine au taux O(ε -c ) si l'ensemble semialgébrique a un intérieur non vide. Nous appliquons la même idée pour une extension du Positivstellensatz de Handelman pour obtenir une autre hiérarchie de relaxations semidéfinies avec une taille de matrice maximale prescrite. Lorsqu'elle est appliquée à la certification de la robustesse des réseaux de neurones multicouches et au calcul de valeurs singulières maximales positives, notre méthode basée sur le Positivstellensatz de Pólya fournit de meilleures bornes et s'exécute plusieurs centaines de fois plus rapidement que la hiérarchie Moment-SOS standard.

) k (f + ε) = σ 0 + m i=1 σ i g i ,
Mots-clés: optimisation polynomiale, hiérarchie Moment-SOS, taux de convergence, programmation semidéfinie, Positivstellensatz de Putinar, Positivstellensatz de Putinar-Vasilescu, Positivstellensatz de Reznick, Positivstellensatz de Pólya, propriété de trace constante, parcimonie des variables et des termes

Abstract

Positivity certificates or Positivstellensätze provide representations of polynomials positive on basic semialgebraic sets, i.e., sets defined by finitely many polynomial inequalities. The famous Putinar's Positivstellensatz states that every positive polynomial on a basic closed semialgebraic set S can be written as a linear weighted combination of the polynomials describing S, under a certain condition on S slightly stronger than compactness. When written in this it becomes obvious that the polynomial is positive on S, and therefore this alternative description provides a certificate of positivity on S.

Moreover, as the polynomial weights involved in Putinar's Positivstellensatz are sums of squares (SOS), such Positivity certificates enable to design convex relaxations based on semidefinite programming to solve polynomial optimization problems (POPs) that arise in various real-life applications, e.g., in management of energy networks and machine learning to cite a few. Originally developed by Lasserre, the hierarchy of semidefinite relaxations based on Putinar's Positivstellensatz is called the Moment-SOS hierarchy.

In this thesis, we provide polynomial optimization methods based on positivity certificates involving specific SOS weights, without or with denominators.

The first part of this manuscript focuses on methods without denominators, and based on Putinar's Positivstellensatz.

We propose a new Moment-SOS hierarchy for solving large-scale sparse polynomial optimization problems. Its novelty is to exploit simultaneously correlative sparsity and term sparsity by combining advantages of two existing frameworks for sparse polynomial optimization. In doing so we obtain (i) a two-level hierarchy of semidefinite programming relaxations with the crucial property to involve blocks of SDP matrices (instead of a single big matrix), and (ii) the guarantee of convergence to the global optimum under certain conditions. We demonstrate its efficiency and scalability on several large-scale instances of the celebrated Max-Cut problem and some instances of the important industrial optimal power flow problem (OPF), involving up to six thousand variables and tens of thousands of constraints.

Next, we prove that every semidefinite moment relaxation of a constrained POP can be reformulated as a semidefinite program involving a matrix with constant trace property (CTP). As a result, such relaxations can be solved efficiently by first-order methods that exploit CTP, e.g., the conditional gradient-based augmented Lagrangian method. We also extend this CTP-exploiting framework to large-scale POPs with different sparsity structures. Efficiency and scalability are illustrated on second-order moment relaxations for various randomly generated quadratically constrained quadratic programs.

The second part of this manuscript focuses on methods with denominators, based on positivity certificates due to Putinar and Vasilescu, Reznick, as well as Pólya.

We revisit two certificates of positivity on (possibly noncompact) basic semialgebraic sets due to Putinar and Vasilescu. We use Jacobi's technique to provide an alternative proof with an effective degree bound on the SOS weights in such certificates. As a consequence, we can define a hierarchy of semidefinite relaxations for general POPs. Convergence to a neighborhood of the optimal value as well as strong duality and analysis are guaranteed. In a second contribution, we introduce a new numerical method for solving systems of polynomial inequalities and equalities with possibly uncountably many solutions. As a bonus, one can apply this method to obtain approximate global optimizers in polynomial optimization.

We next provide a new degree bound on the SOS weights in Putinar-Vasilescu's Positivstellensatz and obtain the following new Positivstellensatz:

If f is a polynomial of degree at most 2d f , nonnegative on the semialgebraic set S := {x : g i (x) ≥ 0, i ∈ [m]} with nonempty interior (and with g 1 := R -x 2 2 for some R > 0), then there exist positive constants c and c depending on f, g i such that for any ε > 0, for all k ≥ cε -c ,

(1 + x 2 2 ) k (f + ε) = σ 0 + m i=1 σ i g i ,
for some SOS polynomials σ i with deg(σ 0 ) and deg(σ i g i ) at most 2(d f + k). Here • 2 denotes the 2 vector norm. As a consequence, we obtain a converging hierarchy of semidefinite relaxations for lower bounds in polynomial optimization on basic compact semialgebraic sets. The complexity of this hierarchy is O(ε -c ) for prescribed accuracy ε > 0. In particular, if m = L = 1 then c = 65, which yields a O(ε -65 ) computational complexity for minimizing a polynomial on the unit ball.

In another contribution we derive a sparse variant of Reznick's Positivstellensatz. If f is a positive definite form, Reznick's Positivstellensatz states that there exists k ∈ N such that x 2k 2 f is an SOS. Namely, assume that f = p c=1 f c , where each form f c depends on a subset of the initial variables, and assume that these subsets satisfy the so-called running intersection property (RIP). Then there exists k ∈ N such that f = p c=1 σ c /H k c , where σ c is a sum of squares of polynomials, H c is a uniform polynomial denominator, and both polynomials σ c , H c involve the same variables as f c , for each c ∈ [p]. In other words, the sparsity pattern of f is also reflected in this sparse version of Reznick's certificate of positivity. We then use this result to also obtain positivity certificates for (i) polynomials nonnegative on the whole space and (ii) polynomials nonnegative on a (possibly noncompact) basic semialgebraic set, assuming that the input data satisfy the RIP. Both are sparse versions of Putinar-Vasilescu's Positivstellensatz.

Finally, we consider the minimization of a polynomial on a semialgebraic set contained in the nonnegative orthant. It can be converted to an equivalent POP by squaring each variable. Using even symmetry and the concept of factor width, we propose a hierarchy of semidefinite relaxations based on the extension of Pólya's Positivstellensatz by Dickinson-Povh. As its distinguishing and crucial feature, the maximal matrix size of each resulting semidefinite relaxation can be chosen arbitrarily. Moreover, the sequence of values returned by the new hierarchy converges to the optimal value of the original POP at the rate O(ε -c ) if the semialgebraic set has nonempty interior. When applied to (i) robustness cerification of multi-layer neural networks and (ii) computing positive maximal singular values, our method based on Pólya's Positivstellensatz provides better bounds and runs several hundred times faster than the standard Moment-SOS hierarchy.

Acknowledgement

Firstly I would like to thank my supervisors for guiding and supporting me over the three years. I appreciate their constant enthusiasm and encouragement. They have set an example of excellence as a researcher, mentor, instructor, and role model. Secondly, I would also like to thank the Jury members for their guidance through the defense process. Their discussion, ideas, and feedback have been invaluable. Thirdly I thank my friends, lab mates, colleagues, and research team for a cherished time spent together in the lab and social settings. In addition, I am also grateful for my teachers' imparted knowledge and shared experiences with me. Finally, I would like to express my gratitude to my wife, parents, and brother. Without their tremendous understanding and encouragement over the past three years, it would be impossible for me to complete my study.

List of Figures

Efficiency and accuracy comparison for

5.7

Numerical results for randomly generated dense QCQPs with box constraints . . . . . . 90 5.8 Numerical results for minimizing a dense quadratic polynomials on a simplex . . . . . . 91 5.9 Numerical results for randomly generated dense QCQPs with simplex constraints . . . . 91 5.10 Numerical comparison with ADMM (COSMO) on randomly generated dense QCQPs with a ball constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.11 Numerical results for randomly generated dense POPs with a ball constraint . . . . . . . 92 5.12 Numerical results for minimizing a random quadratic polynomial with TS on the unit ball 97 5.13 Numerical results for randomly generated QCQPs with TS and a ball constraint . . . . . 97 5.14 Numerical results for minimizing a random quadratic polynomial with TS on a box . . . 98 5.15 Numerical results for randomly generated QCQPs with TS and box constraints . . . . . 98 5.16 Numerical results for minimizing a random quadratic polynomial with CS and ball constraints on each clique of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.17 5 Numerical results for stability number of some known graphs in [START_REF] Rossi | The network data repository with interactive graph analytics and visualization[END_REF]. . . . . . . . . . . 174 9.6 Numerical results for stability number of some known graphs in [START_REF] Rossi | The network data repository with interactive graph analytics and visualization[END_REF] x ∈ R n a real vector of n entries (x 1 , . . . , x n ) 0 zero finite dimensional vector x ≥ 0 x j ≥ 0 , ∀j ∈ [n] x y inner product of two finite dimensional real vectors x 2 Euclidean norm of a real vector x ∈ R n B n the closed unit ball of R n S n-1 the unit sphere of R n ∆ n the unit simplex {x ∈ R n + : 1 ≥ n j=1 x j } B(a, r) the closed ball of center a ∈ R n and radius r ≥ 0 ∂B(a, r) the sphere of center a ∈ R n and radius r ≥ 0 V vector space orthogonal to a given linear subspace V of R n

Analysis

M + (Ω) set of nonnegative Borel measures over Ω ⊂ R n C(Ω) space of continuous functions on Ω ⊂ R n

Ω f dx Lebesgue integral of f ∈ C(Ω) over Ω ⊂ R n f dλ integral of f ∈ C(Ω) w.r.t. λ ∈ M + (Ω) A f dλ integral of f ∈ C(A) over A ⊂ Ω ⊂ R n w.r.t. λ ∈ M + (Ω)
Algebraic geometry α ∈ N n a multi-index vector of nonnegative integers (α 1 , . . . , α n ) e j multi-index with j-th coordinate equal to 1, all others being 0 e multi-index with each coordinate equal to 1 |α| degree of α ∈ N n defined by α 1 + • • • + α n N n d index set with bounded degree {α ∈ N n : |α| ≤ d} x α power α ∈ N n of a vector x ∈ R n defined by

x α1 1 • • • x αn n R[x]
ring of polynomials in a vector of variables x = (x 1 , . . . , x n )

Chapter 1

Introduction 1.1 A brief history of positivity certificates

Deciding nonnegativity of a polynomial is an important and attractive problem throughout history of the development of real algebraic geometry. In his famous and seminal work [START_REF] Hilbert | Über die Darstellung Definiter Formen als Summe von Formenquadraten[END_REF], Hilbert characterized all cases where nonnegative polynomials are sums of squares (SOS) of polynomials. They are the first positivity certificates without denominators. Later Blekherman showed in [START_REF] Blekherman | There are significantly more nonegative polynomials than sums of squares[END_REF] that there are significantly more nonnegative polynomials than SOS. In 1927 Artin proved in [7] that every nonnegative polynomial can be decomposed as a sum of squares of rational functions, thereby solving Hilbert's 17th problem. Namely, f is nonnegative if and only if σ D f = σ N for some SOS polynomials σ N and σ D = 0. Accordingly, Hilbert-Artin's Positivstellensatz has a non-prescribed denominator. Nevertheless, in his celebrated work [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF] Reznick provides a representation that involves a uniform denominator for positive definite forms. Later on, positivity certificates on a general semialgebraic set, involving denominators, have been proposed by Stengle [START_REF] Stengle | A Nullstellensatz and a Positivstellensatz in semialgebraic geometry[END_REF] (see also Krivine [101]). A basic semialgebraic set S(g, h) can be written as S(g, h) := { x ∈ R n : g i (x) ≥ 0 , i ∈ [m]; h j (x) = 0 , j ∈ [l] } , (1.1.1) where n ∈ N is the dimension of the ambient space, m, l ∈ N are the number of inequality and equality constraints, and (g i , h j ) are polynomials, [m] := {1, . . . , m}. Stengle and Krivine rely on

P(g, h) :=    α∈{0,1} m σ α g α1 1 . . . g αm m + l j=1 φ j h j : σ α ∈ Σ[x], φ j ∈ R[x]    , ( 1.1.2)
a tool from real algebraic geometry called preordering, associated with the polynomials (g i , h j ).

Here R[x] denotes the ring of real polynomials and Σ[x] ⊂ R[x] stands for the set of SOS polynomials. Krivine-Stengle's Positivstellensatz (or representation of positive polynomials) states that f ≥ 0 on S(g, h) ⇔ ∃q 1 , q 2 ∈ P(g, h), s ∈ N :

q 1 f = f 2s + q 2 , (1.1.3)
f > 0 on S(g, h) ⇔ ∃q 1 , q 2 ∈ P(g, h) :

q 1 f = 1 + q 2 . (1.1.4)
Notice that the above representations involve a multiplier q 1 for f as well as cross-products of the g i 's in (1.1.2). When S(g, h) is compact, Schmüdgen proves in [START_REF] Schmüdgen | The K-moment problem for compact semi-algebraic sets[END_REF] that if f > 0 on S(g, h), then f ∈ P(g, h), yielding the first Positivstellensatz without denominators over a general compact semialgebraic set.

Let

Q(g, h) :=    σ 0 + m i=1 σ i g i + l j=1 φ j h j : σ i ∈ Σ[x], φ j ∈ R[x]    . (1.1.5)
The set Q(g, h) is called the quadratic module associated with the polynomials (g i , h j ). In 1993 Putinar [START_REF] Putinar | Positive polynomials on compact semi-algebraic sets[END_REF] refined Schmüdgen's Positivstelensatz [START_REF] Schmüdgen | The K-moment problem for compact semi-algebraic sets[END_REF] for compact basic semialgebraic sets (1.1.1) that satisfies an Archimedean condition. The latter states that R -x 2 2 belongs to Q(g, h)

Krivine-Stengle [START_REF] Krivine | Anneaux préordonnés[END_REF][START_REF] Stengle | A Nullstellensatz and a Positivstellensatz in semialgebraic geometry[END_REF] If a polynomial f is nonnegative on S(g), then σf = α∈{0,1} m (σα

m i=1 g α i i ) for some σ, σα ∈ Σ[x].
no yes (non-prescribed)

1974 Pólya [START_REF] Pólya | Über positive darstellung von polynomen[END_REF] If f is a form and f > 0 on R n + \{0}, then ( j xj) k f has nonnegative coefficients for some k ∈ N.

no yes (prescribed) 1991 Schmüdgen [START_REF] Schmüdgen | The K-moment problem for compact semi-algebraic sets[END_REF] If f is positive on S(g) and S(g) is compact, then f = α∈{0,1} m (σα

m i=1 g α i i ) for some σα ∈ Σ[x].
for some R > 0; this can be automatically ensured by including the additional redundant constraint g m+1 (x) := R -x 2 2 ≥ 0 in the definition of S(g, h). It avoids a multiplier for f and no crossproduct of the g i 's, a highly desirable feature for optimization purposes. Explicitly, Putinar's Positivstellensatz states the following result: Theorem 1.1. (Putinar [168]) Let f, g 1 , . . . , g m , h 1 , . . . , h l ∈ R[x]. Assume that Q(g, h) is Archimedean and f is positive on S(g, h). Then f belongs to Q(g, h).

In Table 1.1 we list some Positivstellensätze. SOS decompositions of nonnegative polynomials have a distinguishing feature with important practical implications: Indeed they are tractable because they can be obtained by solving a semidefinite program. Semidefinite programming (SDP) is an important class of convex conic optimization problems that can be solved efficiently, up to arbitrary precision, fixed in advance; the interested reader is referred to, e.g., [START_REF] Ben-Tal | Lectures on modern convex optimization: analysis, algorithms, and engineering applications[END_REF]Chapter 4]. Namely, writing a polynomial f ∈ R[x] 2d as an SOS boils down [START_REF] Parrilo | Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization[END_REF] to computing the entries of a symmetric (Gram) matrix G with only nonnegative eigenvalues (denoted by "G 0") such that f = v d Gv d , with v d being the vector of all monomials of degree at most d. This connection between SOS and SDP promotes many important applications of optimization, operations research, signal processing, computational geometry, probability and statistics, control, PDEs, let alone recent applications in quantum information and computer vision. For more details the interested reader is referred to, e.g., [START_REF] Wang | Certifying global optimality of AC-OPF solutions via the CS-TSSOS hierarchy[END_REF][START_REF] Vreman | Stability of control systems under extended weakly-hard constraints[END_REF][START_REF] Wang | Exploiting term sparsity in Moment-SOS hierarchy for dynamical systems[END_REF][START_REF] Schlosser | Converging outer approximations to global attractors using semidefinite programming[END_REF][START_REF] Schlosser | Sparse moment-sum-of-squares relaxations for nonlinear dynamical systems with guaranteed convergence[END_REF][START_REF] Chen | Semialgebraic representation of monotone deep equilibrium models and applications to certification[END_REF][START_REF] Chen | Semialgebraic optimization for lipschitz constants of relu networks[END_REF][START_REF] Tacchi | Exploiting sparsity for semi-algebraic set volume computation[END_REF][START_REF] Oustry | Inner approximations of the maximal positively invariant set for polynomial dynamical systems[END_REF][START_REF] Tacchi | Approximating regions of attraction of a sparse polynomial differential system[END_REF] and references therein.

A brief history on polynomial optimization

Optimization of polynomials on semialgebraic sets is an important area of applied mathematics, which initially motivated the introduction of the Moment-SOS hierarchy, based on positivity certificates with SOS weights. Indeed since the pioneer works of Lasserre [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF] and Parrilo [START_REF] Parrilo | Structured Semidefinite Programs and Semialgebraic Geometry Methods in Robustness and Optimization[END_REF], SOS have now become a powerful tool in polynomial optimization. Given polynomials

f ∈ R[x], g = {g i } m i=1 ⊂ R[x], h = {h j } l j=1 ⊂ R[x]
, consider the following polynomial optimization problem (POP):

f := inf x∈S(g,h) f (x) , (1.2.1)
with n variables, m inequality constraints, l equality constraints, and where S(g, h) is defined as in (1.1.1). In general POP (1.2.1) is non-convex and NP-hard (see Laurent [START_REF] Laurent | A comparison of the sherali-adams, lovász-schrijver, and lasserre relaxations for 0-1 programming[END_REF]). Consider first the unconstrained case of POP (1.2.1) corresponding to S(g, h) = R n . If f -f (≥ 0 on R n ) is an SOS polynomial then f can be obtained by solving a single semidefinite program (SDP). Obviously denominators are not required in this case. However in general f -f is an SOS of rational functions (not polynomials) due to Krivine-Stengle's Positivstellensatz, and therefore denominators are required, which yields:

f = sup λ,σ N ,σ D {λ : σ D (f -λ) = σ N ; σ N , σ D ∈ Σ[x] ; σ D (0) = 1 } . (1.2.2)
By fixing in advance a bound d on the degree of the denominator σ D , one may solve (1.2.2) by SDP combined with bisection search on λ, and let d increase if there is no solution. The normalization constraint σ D (0) = 1 ensures that neither the denominator σ D nor the numerator σ N is the zero polynomial.

In the constrained case, a basic idea is to rather consider

f = sup{λ ∈ R : f -λ ≥ 0 on S(g, h)} (1.2.3)
and replace the difficult constraint "f -λ ≥ 0 on S(g, h)" with the stronger but more tractable certificate of positivity on S(g, h) for f -λ. For instance, if S(g, h) in (1.1.1) is compact and assuming with no loss of generality that the Archimedean assumption holds, Putinar's Positivstellensatz (Theorem 1.1) provides the decomposition f -λ = σ 0 + m i=1 σ i g i , with σ i ∈ Σ[x]. Then one obtains the monotone non-decreasing sequence (ρ k ) k∈N of lower bounds on f defined by

ρ k := sup λ,σi { λ : f -λ = σ 0 + m i=1 σ i g i , σ i ∈ Σ[x] , deg(σ 0 ) ≤ 2k , deg(σ i g i ) ≤ 2k} , (1.2.4)
where denominators are not needed. Moreover, by invoking Putinar's Positivstellensatz, one obtains the convergence ρ k ↑ f as k increases. Introduced by Lasserre in [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF], for each fixed k, (1.2.4) is a semidefinite program and is an SOS strengthening of (1.2.3) (as we restrict the feasible set). The dual of (1.2.4) is also a semidefinite program which is a Moment relaxation of (1.2.1). The Moment-SOS hierarchy is the sequence (indexed by k) of semidefinite programs (1.2.4) and their associated duals. As proved by Nie [START_REF] Nie | Optimality conditions and finite convergence of Lasserre's hierarchy[END_REF], convergence of (ρ k ) k∈N to f is finite for generic constraints S(g, h), and with the numerical procedure of Henrion and Lasserre [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] one can extract global minimizers from an optimal solution of the (exact) semidefinite relaxation (dual to (1.2.4)) in the hierarchy. It relies on the flat extension condition of Curto and Fialkow [START_REF] Curto | Truncated K-moment problems in several variables[END_REF][START_REF] Laurent | Revisiting two theorems of Curto and Fialkow on moment matrices[END_REF]. In the above-mentioned frameworks, compactness of S(g, h) is crucial.

In this thesis, we treat the case where S(g, h) is not compact by an appropriate use of Putinar-Vasilescu's Positivstellensatz stated in Table 1.1 with the uniform denominator (1 + x 2 2 ) k . Furthermore, we discover and exploit that with this uniform denominator one may obtain different types of Moment-SOS hierarchies with smaller computational complexity than standard ones, hence with a better efficiency for solving POPs.

Overview of degree bound and convergence rate

The convergence rate of the Moment-SOS hierarchy to the optimal value of a POP inherently depends on the complexity of the representation of positive polynomials. Roughly speaking, obtaining a lower degree bound on the SOS polynomials involved in the positivity certificate allows one to improve the convergence rate of the corresponding Moment-SOS hierarchy. How to find such lower degree bound is an interesting question and goes hand in hand with the quest of improving the convergence analysis of the Moment-SOS hierarchy. Let us review some of the standard results on degree bounds of positivity certificates and the corresponding convergence rates of the associated Moment-SOS hierarchy.

We denote by S n-1 the unit sphere in R n . For each q ∈ R[x], let δ(q) := sup q(x) : x ∈ S n-1 inf {q(x) : x ∈ S n-1 } .

In [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF] Reznick provides a Positivstellensatz involving a uniform denominator for positive definite forms with an explicit degree bound:

Theorem 1.2. (Reznick [175,Theorem 3.12]) Suppose that q ∈ R[x] is a positive definite form of degree 2d, for some d ∈ N. Then for k ∈ N and

k ≥ 2nd(2d -1) 4 log 2 δ(q) - n + 2d 2 , (1.3.1)
x 2k 2 q is an SOS of polynomials.

In [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF]Theorem 3.12], Reznick guarantees that the SOS decomposition of x 2k 2 q is actually a sum of powers of linear forms. The degree bound of Reznick's Positivstellensatz yields a linear convergence rate of O(ε -1 ) for the minimization of a polynomial (see [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Theorem 6]).

Powers and Reznick [START_REF] Powers | A new bound for Pólya's Theorem with applications to polynomials positive on polyhedra[END_REF] improve the existing degree bound available for Pólya's Positivstellensatz [START_REF] Pólya | Über Positive Darstellung von Polynomen[END_REF] which is associated with another uniform denominator. Explicitly, if q is a homogeneous polynomial of degree d positive on the simplex

∆ n = {x ∈ R n : x j ≥ 0 , j ∈ [n] , j∈[n] x j = 1} , ( 1.3.2)
then for all k ∈ N satisfying

k ≥ d(d -1) q 2 min x∈∆n q(x) -d , (1.3.3)
( j∈ [n] x j ) k q has positive coefficients. Here for each h = α h α x α ∈ R[x], we note h := max α |hα| cα with c α := |α|! α1!...αn! for each α ∈ N n . This yields a linear convergence rate of O(ε -1 ) for the minimization of a homogeneous polynomial on the simplex.

Applying the result of Powers and Reznick, Schweighofer [START_REF] Schweighofer | On the complexity of Schmüdgen's Positivstellensatz[END_REF] obtains a degree bound for Schmüdgen's Positivstellensatz [START_REF] Schmüdgen | The K-moment problem for compact semi-algebraic sets[END_REF] claiming that given a semialgebraic set

S(g) = {x ∈ R n : g i (x) ≥ 0 , i ∈ [m]} , (1.3.4)
which is a subset of (-1, 1) n and a polynomial f having the minimal value f > 0 on S(g), then there exists a real c > 0 depending on g = {g i } i∈ [m] such that for all k ∈ N satisfying

k ≥ cd 2 f 1 + d 2 f n d f f f c , (1.3.5)
one has f ∈ P k (g), where P k (g) is the truncated preordering of order k ∈ N associated with S(g):

P k (g) := { α∈{0,1} m σ α g α1 1 . . . g αm m : σ α ∈ Σ[x] , deg(σ α g α1 1 . . . g αm m ) ≤ k} . (1.3.6)
Consequently, the corresponding SOS hierarchy of lower bounds (ρ pre k ) k∈N , with

ρ pre k := sup λ∈R {λ : f -λ ∈ P k (g)} , k ∈ N , (1.3.7)
converges to f with the rate O(ε -c ). Observe that no denominator is required in this case. Nevertheless, the representation of f -λ in P k (g) involves 2 m SOS polynomials. Recently, by relying on polynomial kernel methods, Fang and Fawzi [START_REF] Fang | The sum-of-squares hierarchy on the sphere and applications in quantum information theory[END_REF] have explicited the exponent c := 1 2 in (1.3.5) when S(g) is the unit sphere S n-1 . It yields the optimal convergence rate O(1/k 2 ) for the minimization of a polynomial on this set by using Schmüdgen's Positivstellensatz. Following Fang-Fawzi's method, Laurent and Slot obtain in [START_REF] Laurent | An effective version of Schmüdgen's Positivstellensatz for the hypercube[END_REF][START_REF] Slot | Sum-of-squares hierarchies for polynomial optimization and the Christoffel-Darboux kernel[END_REF] similar convergence rates when S(g) is one of the following sets: the standard hypercube [-1, 1] n , the unit ball B n and the unit simplex ∆ n . Let Q k (g) denote the truncated quadratic module of order k ∈ N associated with S(g), that is:

Q k (g) := {σ 0 + m i=1 σ i g i : σ i ∈ Σ[x] , deg(σ 0 ) ≤ k , deg(σ i g i ) ≤ k} . (1.3.8)
Notice that P k (g) = Q k (g) if S(g) is the unit ball or the unit sphere.

Applying the degree bound of Laurent and Slot [START_REF] Laurent | An effective version of Schmüdgen's Positivstellensatz for the hypercube[END_REF], Baldi and Mourrain obtain in [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF] polynomial degree bounds for Putinar's Positivstellensatz which improves the exponential bound given by Nie and Schweighofer [START_REF] Nie | On the complexity of Putinar's Positivstellensatz[END_REF]. [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF]) Let f, g 1 , . . . , g m be in R[x]. Assume that ∅ = S(g) ⊂ (-1, 1) n is Archimedean and that the minimal value f of f over S(g) is positive. Then there exist real numbers c 1 , c 2 , c 3 > 0 depending on g such that for all k ∈ N satisfying

Theorem 1.3. (Baldi-Mourrain

k ≥ c 1 d c2 f f f c3 , (1.3.9) one has f ∈ Q k (g).
Accordingly, the corresponding SOS hierarchy of lower bounds (ρ mod k ) k∈N , with

ρ mod k := sup λ∈R {λ : f -λ ∈ Q k (g)} , k ∈ N , (1.3.10)
converges to f with the rate O(ε -c ). Moreover, the representation of f -λ in Q k (g) involves only m + 1 SOS polynomials which is in deep contrast with the exponential number of SOS polynomials involved in the representation in P k (g). Moreover, no denominator is required in this case. Most of recent convergence rates obtained for polynomial optimization are summarized in Table 1.2. Recent work by Lombardi, Perucci and Roy [START_REF] Lombardi | An elementary recursive bound for effective Positivstellensatz and Hilbert's 17th problem[END_REF] provides degree bounds in a quite general situation. The best degree bounds for Hilbert's 17th problem in three homogeneous variables are actually due to Hilbert's original 1893 paper [START_REF] Hilbert | Über die Darstellung Definiter Formen als Summe von Formenquadraten[END_REF]. Some lower bounds for Hilbert's 17th problem (quite far away from the upper bounds) were proved in [START_REF] Blekherman | Sums of squares on the hypercube[END_REF] by Blekherman, Gouveia and Pfeiffer. Hilbert's 1893 result and sharp degree bounds for projective curves (i.e., curves defined by homogeneous polynomials) were proved in [START_REF] Blekherman | Sharp degree bounds for sum-of-squares certificates on projective curves[END_REF] by Blekherman, Smith and Velasco. 1] IP with nonsymmetric cone [START_REF] Papp | Sum-of-squares optimization without semidefinite programming[END_REF] (second-order) alfonso [START_REF] Papp | alfonso: Matlab package for nonsymmetric conic optimization[END_REF] arbitrary O(log(1/ε))

system of linear equations solving with complexity O(η 3 ) ADMM [START_REF] Boyd | Distributed optimization and statistical learning via the alternating direction method of multipliers[END_REF] (first-order)

SCS [156], COSMO [60] arbitrary O(ε -1 ) [81]
positive definite system of linear equations solving by LDL -decomposition with complexity O((s max ) 6 ) SBM [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF] (first-order) ConicBundle [START_REF] Helmberg | The spectral bundle method with second-order information[END_REF] with CTP O(log(1/ε)/ε) [START_REF] Ding | Revisit of spectral bundle methods: Primal-dual (sub) linear convergence rates[END_REF] positive definite linear system solving with complexity O((s max ) 6 )

CGAL [START_REF] Yurtsever | A conditional-gradient-based augmented lagrangian framework[END_REF] (first-order)

SketchyCGAL [219] with CTP O(ε -2 )
smallest eigenvalue computing by the Arnoldi iteration with complexity O(s max ) [START_REF] Lee | From O(k 2 N ) to O(N ): A fast complexvalued eigenvalue solver for large-scale on-chip interconnect analysis[END_REF] 1

.4 Computational complexity of the Moment-SOS relaxations

So far we have discussed the complexity of each representation in term of degree bounds, which basically translates to convergence rates for the values returned by the related relaxations. We next consider the cost of the numerical resolution of a given relaxation.

Computational cost of moment relaxations.

The k-th order moment relaxation for POP (1.2.1) can be rewritten in compact form as the following standard SDP:

τ = inf X∈S+ { C, X : A j , X = b j , j ∈ [ζ]} , (1.4.1)
where A, B := trace(AB) is the standard Frobenius inner product, S + is the set of positive semidefinite (psd) matrices written in a block diagonal form as follows: X = diag(X 1 , . . . , X ω ) with X j being a block of size s (j) , j ∈ [ω], and ζ is the number of affine constraints. We denote the largest block size by s max := max j∈[ω] s (j) . Note that SDP-relaxation (1.4.1) of POP (1.2.1) at step k of the Moment-SOS hierarchy has ω = m + 1 blocks whose largest size is s max = n+k n while the number of affine constraints is

ζ = O( n+k n 2
). Thus the computational cost for solving SDP (1.4.1) grows very rapidly with k.

We say that SDP (1.4.1) has constant trace property (CTP) if there exists a positive real number a such that trace(X) = a, for all feasible solution X of SDP (1.4.1). We also say that POP (1.2.1) has CTP when every moment relaxation of POP (1.2.1) has CTP.

In Table 1 QCQPs include the Max-Cut problem and the optimal power flow (OPF) problem [START_REF] Josz | Lasserre hierarchy for large scale polynomial optimization in real and complex variables[END_REF]; in addition we recall that linearly constrained quadratic programs have an equivalent Max-Cut formulation [START_REF] Lasserre | A max-cut formulation of 0/1 programs[END_REF]. They also have applications in deep learning, e.g., the computation of Lipschitz constants [START_REF] Chen | Semialgebraic optimization for lipschitz constants of relu networks[END_REF] and the stability analysis of recurrent neural networks [START_REF] Ebihara | l2 induced norm analysis of discrete-time LTI systems for nonnegative input signals and its application to stability analysis of recurrent neural networks[END_REF]. On the one hand, local optima of non-convex QCQPs can be obtained by using local solvers; see e.g., [START_REF] Carmon | First-order methods for nonconvex quadratic minimization[END_REF]. On the other hand, the global optimum of QCQPs can be approximated as closely as desired by using moment (SDP) relaxations. In practice, non-convex QCQPs usually involve a large number of variables (say, n ≥ 1000) and their associated SDP relaxations (1.4.1) can be classified in two groups as follows:

• The first order relaxation: k = 1 (also known as Shor's relaxation in the literature). In this case the number of affine constraints in SDP (1.4.1) is typically not exceeding the largest block size, i.e., ζ ≤ s max . It can be efficiently solved by most SDP solvers, in particular with SketchyCGAL [START_REF] Yurtsever | Scalable semidefinite programming[END_REF]. Nevertheless the first order relaxation may provide only a lower bound of the optimal value of POP (1.2.1). In this case one needs to solve the second and perhaps even higher order relaxations to obtain tighter bounds on the global optimum.

• The second and higher order relaxations: k ≥ 2. In this case the number of affine constraints in SDP (1.4.1) is typically much larger than the largest block size (ζ s max ). Then unfortunately most SDP solvers cannot handle large-scale SDPs of this form.

Common issues of solving large-scale SDP relaxations. When solving the second and higher order SDP relaxations, SDP solvers often encounter the following issues:

• Storage: Interior-point methods are often chosen by users because of their high accuracy.

These methods are efficient for solving medium-scale SDPs. However they frequently fail due to lack of memory when solving large-scale SDPs (say, s max > 500 and ζ > 2 × 10 5 on a standard laptop). First-order methods (e.g., ADMM, SBM, CGAL) provide an alternative to interior-point methods to avoid the memory issue. This is due to the fact that the cost per iteration of first-order methods is much lower than that of interior-point methods.

At the price of losing convexity one can also rely on heuristic methods and replace the full matrix X in SDP (1.4.1) by a simpler one, in order to save memory. For instance, the Burer-Monteiro method [START_REF] Burer | Local minima and convergence in low-rank semidefinite programming[END_REF] considers a low rank factorization of X. However, to get correct results the rank cannot be too low [START_REF] Waldspurger | Rank optimality for the burer-monteiro factorization[END_REF] and therefore this limitation makes it useless for the second and higher order relaxations of POPs. Not suffering from such a limitation, CGAL not only maintains the convexity of SDP (1.4.1) but also possibly runs with implicit matrix X.

• Accuracy: First-order methods have low convergence rates compared to the interior-point methods. Their performance depends heavily on the problem scaling and conditioning. As a result, in solving large-scale SDPs with first-order methods it is often difficult to obtain numerical results with high accuracy. Therefore, we do not expect the relative gap of the approximate value (val approx ) returned by first-order SDP solvers w.r.t. the exact value (val exact ), defined by

|val approx -val exact | |val exact | , (1.4.2)
to be smaller than 10 -8 (as for interior-point methods) but at least to be smaller than 1%.

Recent improvements on scalability and efficiency

Overcoming the scalability and efficiency issues mentioned in the previous section has become a major scientific challenge in polynomial optimization. Many recent efforts in this direction are mainly developed around the following ideas:

1. SDP-relaxations variants with small maximal matrix size solved efficiently by interior point methods. This includes correlative sparsity [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF][START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF], term sparsity [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF][START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF][START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF], symmetry exploitation [START_REF] Gatermann | Symmetry groups, semidefinite programs, and sums of squares[END_REF][START_REF] Riener | Exploiting symmetries in SDPrelaxations for polynomial optimization[END_REF], Jordan symmetry reduction [START_REF] Brosch | Jordan symmetry reduction for conic optimization over the doubly nonnegative cone: theory and software[END_REF], sublevel relaxations [START_REF] Chen | A sublevel Moment-SOS hierarchy for polynomial optimization[END_REF].

2. Exploit low-rank structures of SDP-relaxations; see, e.g., [START_REF] Weisser | Sparse-BSOS: a bounded degree SOS hierarchy for large scale polynomial optimization with sparsity[END_REF][START_REF] Yang | An inexact projected gradient method with rounding and lifting by nonlinear programming for solving rank-one semidefinite relaxation of polynomial optimization[END_REF].

3. First-order methods to solve SDP-relaxations involving matrix variables of potentially large size with constant trace [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF][START_REF] Mai | Exploiting constant trace property in large-scale polynomial optimization[END_REF].

4. Develop convex relaxations that are based on alternatives to semidefinite cones. For example this includes linear programming (LP) [START_REF] Lasserre | A bounded degree SOS hierarchy for polynomial optimization[END_REF]3], second-order conic programming (SOCP) [START_REF] Magron | SONC Optimization and Exact Nonnegativity Certificates via Second-Order Cone Programming[END_REF][START_REF] Wang | A second order cone characterization for sums of nonnegative circuits[END_REF]3], copositive programming [START_REF] Pena | Completely positive reformulations for polynomial optimization[END_REF], non-symmetric conic programming [START_REF] Papp | Sum-of-squares optimization without semidefinite programming[END_REF], relative entropy programming [START_REF] Dressler | A positivstellensatz for sums of nonnegative circuit polynomials[END_REF][START_REF] Murray | Signomial and polynomial optimization via relative entropy and partial dualization[END_REF], geometric programming [START_REF] Dressler | An approach to constrained polynomial optimization via nonnegative circuit polynomials and geometric programming[END_REF].

Sparsity exploitation is one of the notable methods without denominators for reducing the size of the Moment-SOS relaxations. For POPs in the form

f := min x∈S(g) f (x) , (1.5.1)
where S(g) is defined as in (1.3.4), Waki et al. [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] (resp. Wang et al. [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF]) have exploited correlative (resp. term) sparsity to define appropriate sparse-variants of the associated standard SOS-relaxations. Roughly speaking, in a given standard SOS-relaxation, they break each matrix variable into many blocks of smaller sizes and solve the new resulting SDP via an interior-point solver (e.g., Mosek [5] or SDPT3 [START_REF] Tütüncü | Solving semidefinite-quadratic-linear programs using SDPT3[END_REF]). It is due to the fact that the most expensive part of interior-point methods for a standard SDP:

min z,A (t) j c z s.t. z ∈ R w , A (t) j ∈ R q×q , A (t) 0 + w j=1 z j A (t) j 0 , t ∈ [u] ,
is solving a square linear system in every iteration. It has the complexity u(w 3 q 3 + w 3 q 2 ), which mainly depends on the matrix size q. Thus one can solve the above SDP efficiently by using interior-point methods if q, w are small and u is large.

Modern SDP solvers via the interior-point method (e.g. Mosek [4]) can solve an SDP problem involving matrices of moderate size (say, q ≤ 5, 000) and variables of moderate number (say, w ≤ 20, 000) in reasonable time on a standard laptop [START_REF] Toh | Some numerical issues in the development of SDP algorithms[END_REF]. The SDP relaxations arising from the Moment-SOS hierarchy typically involve matrices of size n+k k and variables of number n+2k 2k , where k is the relaxation order and n is the number of variables for a given POP. For problems with n 200, it is thus possible to compute the first-order SDP relaxation of a quadratically constrained quadratic problem (QCQP), as one can take k = 1, yielding n+k k 200 and n+2k 2k 20, 000 (in this case, this relaxation is also known as Shor's relaxation [START_REF] Shor | Quadratic optimization problems[END_REF]). However, the quality of the resulting approximation is often not satisfactory and it is then required to go beyond the first-order relaxation. But for solving the second-order relaxation (k = 2) one is limited to problems of small size, typically with n+4 4 ≤ 20, 000 (hence with n ≤ 23) on a standard laptop. Therefore, in view of the current state of SDP solvers, the dense Moment-SOS hierarchy does not scale well enough.

One possible remedy is to rely on alternative weaker positivity certificates, such as the hierarchy of linear programming (LP) relaxations based on Krivine-Stengle's certificates [START_REF] Krivine | Anneaux préordonnés[END_REF][START_REF] Stengle | A Nullstellensatz and a Positivstellensatz in semialgebraic geometry[END_REF][START_REF] Lasserre | A bounded degree SOS hierarchy for polynomial optimization[END_REF] or the second-order cone programming (SOCP) relaxation based on (scaled) diagonally dominant sums of squares (DSOS/SDSOS) [3] to bound from below the minimum of f . Even though modern LP/SOCP solvers can handle much larger size problems by comparison with SDP solvers, they have been shown to provide less accurate bounds, in particular for combinatorial problems [START_REF] Laurent | A comparison of the sherali-adams, lovász-schrijver, and lasserre relaxations for 0-1 programming[END_REF], and do not have the property of finite convergence for continuous problems (not even for convex QCQP problems [START_REF] Lasserre | An introduction to polynomial and semi-algebraic optimization[END_REF]Section 9.3]). Below we outline some existing remedies to exploit sparsity in different settings.

Unconstrained POPs.

A first option is to exploit term sparsity for sparse unconstrained problems, i.e., when S(g) = R n , and the objective f involves a few terms (monomials). The algorithm consists of automatically reducing the size of the corresponding SDP matrix by eliminating the monomial terms which never appear among the support of SOS decompositions [START_REF] Reznick | Extremal psd forms with few terms[END_REF]. Other classes of positivity certificates have been recently developed with a specific focus on sparse unconstrained problems. Instead of trying to decompose a positive polynomial as an SOS, one can try to decompose it as a sum of nonnegative circuits (SONC), by solving a geometric program [START_REF] Iliman | Amoebas, nonnegative polynomials and sums of squares supported on circuits[END_REF] or a second-order cone program [9,[START_REF] Wang | A second order cone characterization for sums of nonnegative circuits[END_REF], or alternatively as a sum of arithmetic-geometric-meanexponentials (SAGE) [START_REF] Chandrasekaran | Relative entropy relaxations for signomial optimization[END_REF] by solving a relative entropy program. Despite their potential efficiency on certain sub-classes of POPs (e.g., sparse POPs with a small number of variables and a high degree), these methods share the common drawback of not providing systematic practical modeling frameworks for constrained problems.

Correlative sparsity. In order to reduce the computational burden associated with the dense Moment-SOS hierarchy while keeping its nice convergence properties, one possibility is to take into account the sparsity pattern satisfied by the variables of the POP [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF][START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF]. The resulting algorithm has been implemented in the SparsePOP solver [START_REF] Waki | Algorithm 883: SparsePOP-A Sparse Semidefinite Programming Relaxation of Polynomial Optimization Problems[END_REF] and can handle sparse problems with up to several hundred variables. Many applications of interest have been successfully handled thanks to this framework, for instance certified roundoff error bounds in computer arithmetics [START_REF] Magron | Certified roundoff error bounds using semidefinite programming[END_REF][START_REF] Magron | Interval enclosures of upper bounds of roundoff errors using semidefinite programming[END_REF] with up to several hundred variables and constraints, optimal power flow problems [START_REF] Josz | Lasserre hierarchy for large scale polynomial optimization in real and complex variables[END_REF] (where a multi-ordered Lasserre hierarchy was proposed) with up to several thousand variables and constraints. More recent extensions have been developed for volume computation of sparse semialgebraic sets [START_REF] Tacchi | Exploiting sparsity for semi-algebraic set volume computation[END_REF], approximating regions of attraction of sparse polynomial systems [START_REF] Tacchi | Approximating regions of attraction of a sparse polynomial differential system[END_REF], noncommutative POPs [START_REF] Klep | Sparse noncommutative polynomial optimization[END_REF], Lipschitz constant estimation of deep networks [START_REF] Chen | Semialgebraic optimization for lipschitz constants of relu networks[END_REF] and for sparse positive definite functions [START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF]. In these applications, the cost polynomial and the constraint polynomials possess a specific correlative sparsity pattern. The resulting sparse Moment-SOS hierarchy is obtained by building blocks of SDP matrices with respect to some subsets or cliques of the input variables. When the sizes of these cliques are reasonably small, one can expect to handle problems with a large number of variables. For instance, the maximal size of cliques is less than 10 for some unconstrained problems in [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] or roundoff error problems in [START_REF] Magron | Certified roundoff error bounds using semidefinite programming[END_REF], and is less than 20 for the optimal power flow problems handled in [START_REF] Josz | Lasserre hierarchy for large scale polynomial optimization in real and complex variables[END_REF]. Even though correlative sparsity has been successfully used to tackle several interesting applications, there are still many POPs that cannot be handled by considering merely correlative sparsity. For instance, there are POPs for which the correlative sparsity pattern is (nearly) dense or which admits a correlative sparsity pattern with variable cliques of large cardinality (say, > 20), yielding untractable SDPs.

Term sparsity

To overcome these issues, one can exploit term sparsity as described in [START_REF] Wang | A new sparse sos decomposition algorithm based on term sparsity[END_REF][START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF][START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF]. The TSSOS hierarchy from [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF] as well as the complementary Chordal-TSSOS from [START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF] offers some alternative to problems for which the correlative sparsity pattern is dense or nearly dense. In both TSSOS and Chordal-TSSOS frameworks a so-called term sparsity pattern (tsp) graph is associated with the POP. The nodes of this tsp graph are monomials (from a monomial basis) needed to construct SOS strengthenings of the POP. Two nodes are connected via an edge whenever the product of the corresponding monomials appears in the supports of polynomials involved in the POP or is a monomial square. Note that this graph differs from the correlative sparsity pattern (csp) graph used in [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] where the nodes are the input variables and the edges connect two nodes whenever the corresponding variables appear in the same term of the objective function or in the same constraint. A two-step iterative algorithm takes as input the tsp graph and enlarges it to exploit the term sparsity in (1.5.1). Each iteration consists of two successive operations: (i) a support extension operation and (ii) either a block closure operation on adjacency matrices in the case of TSSOS [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF] or a chordal extension operation in the case of Chordal-TSSOS [START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF]. In doing so one obtains a two-level Moment-SOS hierarchy with blocks of SDP matrices. If the sizes of blocks are relatively small then the resulting SDP relaxations become more tractable as their computational cost is significantly reduced. Another interesting feature of TSSOS is that the block structure obtained at the end of the iterative algorithm automatically induces a partition of the monomial basis, which can be interpreted in terms of sign symmetries of the initial POP. TSSOS and Chordal-TSSOS allow one to solve POPs with several hundred variables for which there is no or little correlative sparsity to exploit; see [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF][START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF] for numerous numerical examples.

Organization and summary of contributions

The aim of this thesis is to address the above-mentioned complexity and efficiency issues. It consists of two parts related to positivity certificates without and with denominators, respectively:

Positivity certificates without denominators

• We provide in Chapter 2 an overview of the concepts and mandatory background related to the Moment-SOS hierarchy for polynomial optimization, which is of interest in this thesis.

• In Chapter 3, we exploit simultaneously correlative sparsity and term sparsity to improve scalability issues: (i) Correlative sparsity occurs when a polynomial is a sum of polynomials involving a subset of the initial variables. (ii) Term sparsity occurs when a polynomial involves only a few nontrivial terms. The idea is to exploit these different sparsity structures to derive specific relaxations, where one breaks each matrix variable of the standard (dense) SDP-relaxation into many blocks of smaller sizes and solve the new resulting SDP via an interior-point solver (e.g., Mosek, SDPT3). We apply this method to provide tight bounds for some large-scale optimal power flow problem involving up to six thousand variables and tens of thousands of constraints.

• In addition to size reduction of the SDP relaxations, as mentioned above, we introduce in Chapter 4 a new method that enables us to speed up the resolution of the SDP relaxations. Explicitly, if an equality constrained POP has a sphere constraint of the form R -x 2 2 = 0 for some R > 0, the matrix variables involved in the relaxations have a constant trace, a property that can be further exploited. In Chapter 5, we extend this property to the case of POPs that have ball constraint of the form R -x 2 2 ≥ 0 for some R > 0. The constant trace property boils down to SDPs where the feasible set of solutions is the intersection of the semidefinite cone (matrices with nonnegative eigenvalues) and the hyperplane of trace one matrices. By using first-order methods, this property allows one to solve SDP-relaxations involving matrix variables of potentially large size.

Positivity certificates with denominators

• Chapter 6 focuses on designing a hierarchy of semidefinite relaxations based on Putinar-Vasilescu's Positivstellensatz. It allows one to handle general polynomial optimization problems over possibly noncompact semialgebraic sets. This Positivstellensatz is obtained by combining homogenization techniques with Putinar's Positivstellensatz. Each SOS polynomial weight involved in the representation is replaced by an SOS of rational function with (prescribed) denominator. We also provide a new numerical method to find a point in a given basic semialgebraic set that possibly has positive dimension. This method is utilized to find an approximate optimal solution of the initial POP, yielding more expressiveness.

• We provide in Chapter 7 an improved degree bound for Putinar-Vasilescu's Positivstellensatz, namely a polynomial degree bound in the input degrees. As a result (under some mild conditions) we obtain an O(ε -c ) rate of convergence for the sequence of values returned by the corresponding hierarchy. The methodology consists of the following two steps: First, we provide a constructive proof of the Positivstellensatz. Then we provide degree bounds for its explicit SOS weights.

• In Chapter 8, we state a sparse version of Reznick's Positivstellensatz [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF]. The dense version states that any positive definite form can be decomposed as an SOS of rational functions with prescribed denominators. In the sparse setting, the form is a sum of forms, where each summand only depends on a subset of the initial variables, and we obtain a decomposition into a sum of sparse rational SOS.

• Chapter 9 is dedicated to the minimization of a polynomial over a semialgebraic set contained in the nonnegative orthant. We provide a hierarchy of semidefinite relaxations based on Pólya's Positivstellensatz and associated with sums of squares of s-nomials, i.e., linear combinations of s monomials with real coefficients, where s is prescribed in advance. The advantage of these SDP relaxations is that the maximal block size s is controllable so that we can solve them efficiently by using interior-point methods. We also obtain a convergence rate for this hierarchy which is similar to the one based on Putinar-Vasilescu's Positivstellensatz.

List of publications

We conclude this introductory chapter with a list of submitted, accepted and published contributions.

Part I

Positivity certificates without denominators

= (α 1 , . . . , α n ) ∈ N n , let |α| := α 1 + • • • + α n . For d ∈ N, let N n d := {α ∈ N n : |α| ≤ d} and N * d := {u ∈ N : u * d}, for * ∈ {≥, ≤, >, <}. For r ∈ N >0 ,
(x) = α∈N n d q α x α = q v d (x),
where q = (q α ) ∈ R b(d) is its vector of coefficients in the canonical monomial basis. For q ∈ R[x], let q := deg(q)/2 . The l 1 -norm of a polynomial q is given by the l 1 -norm of its vector of coefficients q, that is

q 1 := α |q α |. Given a ∈ R n , the l 2 -norm of a is a 2 := (a 2 1 + • • • + a 2 n ) 1/2 .

Riesz linear functional and moment/localizing matrices

Given a real-valued sequence y = (y α ) α∈N n , let L y : R[x] → R be the Riesz linear functional defined by q → L y (q) := α q α y α . Let d be a positive integer. A real infinite sequence (y α ) α∈N n has a representing measure if there exists a finite Borel measure µ such that y α = R n x α dµ(x) for every α ∈ N n . In this case, (y α ) α∈N n is called the moment sequence of µ. We denote by supp(µ) the support of a Borel measure µ.

The moment matrix of order d associated with a real-valued sequence y = (y α ) α∈N n and d ∈ N >0 , is the real symmetric matrix M d (y) of size b(d), with entries (y α+β ) α,β∈N n d . The localizing matrix of order d associated with y = (y α ) α∈N n and q = γ q γ x γ ∈ R[x], is the real symmetric matrix M d (q y) of size b(d) with entries ( γ q γ y γ+α+β ) α,β∈N n d .

Example 2.1. Consider the simple case where n = 1, q = 1 -x 2 and y = (y 0 , y 1 , y 2 , y 3 , y 4 ). Then L y (q) = y 0 -y 2 ,

M 2 (y) =   y 0 y 1 y 2 y 1 y 2 y 3 y 2 y 3 y 4   and M 1 (qy) = y 0 -y 2 y 1 -y 3 y 1 -y 3 y 2 -y 4 .
(2.2.1)

Quadratic module and ideal

Let g := {g i } i∈[m] ⊂ R[x] and h := {h j } j∈[l] ⊂ R[x].
Denote by S(g) and V (h) a basic semialgebraic set and a real variety defined respectively by

S(g) := { x ∈ R n : g i (x) ≥ 0 , i ∈ [m] } and V (h) := { x ∈ R n : h j (x) = 0 , i ∈ [l] } . (2.3.1)
Set S(g, h) := S(g) ∩ V (h). In other words, we have

S(g, h) = { x ∈ R n : g i (x) ≥ 0 , i ∈ [m] ; h j (x) = 0 , j ∈ [l] } . (2.3.2)
The quadratic module associated with g is defined by

Q(g) := {σ 0 + m i=1 σ i g i : σ 0 ∈ Σ[x] , σ i ∈ Σ[x]} (2.3.3)
and for a positive integer k, the set

Q k (g) := {σ 0 + m i=1 σ i g i : σ 0 ∈ Σ[x] k , σ i ∈ Σ[x] k-gi } (2.3.4) is the truncation of Q(g) of order k. Given h = {h j } j∈[l] ⊆ R[x], the set I(h) := { l j=1 ψ j h j : ψ j ∈ R[x]} (2.3.5)
is the ideal generated by h, and the set

I k (h) := { l j=1 ψ j h j : ψ j ∈ R[x] 2(k-hj ) } is the truncation of I(h) of order k. Set Q(g, h) = Q(g) + I(h) and Q k (g, h) = Q k (g) + I k (h) for k ∈ N.

The Moment-SOS hierarchy

A polynomial optimization problem (POP) is defined as

f := inf x∈S(g,h) f (x) , (2.4.1)
where S(g, h) are defined as in (2.3.1) for some polynomial f ∈ R[x], g and h defined as in the previous section. We will assume that POP (2.4.1) has at least one global minimizer. Set

k min := max i,j { f , g i , h j } . (2.4.2)
Given a POP of the form (2.4.1), consider the following associated hierarchy of SOS strengthenings indexed by k ∈ N ≥kmin :

ρ k := sup ξ∈R { ξ : f -ξ ∈ Q k (g, h)} . (2.4.3) For each σ ∈ Σ[x] d , there exists G 0 such that σ = v d Gv d . Thus for each k ∈ N ≥kmin , (2.4.
3) can be rewritten as an SDP:

ρ k = sup ξ,Gi,qj    ξ G i 0 , f -ξ = v k G 0 v k + i∈[m] g i v k-gi G i v k-gi + j∈[l] h j v 2(k-hj ) q j    . (2.4.4)
For every k ∈ N ≥kmin , the dual of (2.4.4) reads as

τ k := inf y∈R b(2k)    L y (f ) M k (y) 0 , y 0 = 1 M k-gi (g i y) 0 , i ∈ [m] M k-hj (h j y) = 0 , j ∈ [l]    . (2.4.5)
This primal-dual sequence of semidefinite programs (2.4.5)-(2.4.4) is the so-called Moment-SOS hierarchy for optimization (also known as "Lasserre's hierarchy"). If Q(g, h) is Archimedean, i.e., R -x 2 2 ∈ Q(g, h) for some R > 0, then (ρ k ) k∈N ≥k min converges to f by invoking Putinar's Positivstellensatz (Theorem 1.1).

Here we slightly abuse terminology and say that there is a "zero duality gap" between (2.4.3) and (2.4.5) if τ k = ρ k and τ k ∈ R (the abuse of terminology is due to the fact that zero duality gap can occur with both values being infinite). Slater's condition, i.e., there exists a feasible point at which inequality constrains are strict (see, e.g., [START_REF] Boyd | Convex optimization[END_REF]Section 5.2.3]), on either (2.4.3) or (2.4.5) is a well-known sufficient condition to ensure zero duality gap. However, in case of equality constraints in the description (2.3.2) of S(g, h), Slater's condition does not hold for (2.4.5). [START_REF] Josz | Strong duality in Lasserre's hierarchy for polynomial optimization[END_REF]) Let f be as in (2.4.1) with S(g, h) = ∅ as in (2.3.2). Assume that R -x 2 ∈ g for some real R > 0. Zero duality gap between the primal (2.4.3) and dual (2.4.5) holds for sufficiently large k ∈ N, i.e., ρ k = τ k and τ k ∈ R. Moreover, SDP (2.4.5) has an optimal solution.

Proposition 2.1. (Josz-Henrion

In [START_REF] Josz | Strong duality in Lasserre's hierarchy for polynomial optimization[END_REF] the authors prove that the set of optimal solutions of (2.4.5) is compact and therefore (2.4.5) has an optimal solution. Although there exist situations where SDP (2.4.3) has no optimal solution (see for instance the end of [147, Section 3]), the following proposition ensures the existence of an optimal solution under mild assumptions: Proposition 2.2. (Lasserre [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF]Theorem 3.4 (a)]) If S(g, h) has nonempty interior, then Slater's condition on the dual (2.4.5) holds for k ≥ k min , where k min is defined as in (2.4.2). In this case,

ρ k = τ k , τ k ∈ R and the primal (2.4.3) has an optimal solution.
The convergence rate of O(k -c ) for the sequence (ρ k ) k∈N follows from Theorem 1.3. For more details on the Moment-SOS hierarchy and its various applications, the interested reader is referred to [START_REF] Lasserre | Moments, positive polynomials and their applications[END_REF].

Extraction of global minimizers

Let δ a stand for the Dirac measure at point a ∈ R n . The following result is a consequence of Curto-Fialkow's Flat Extension Theorem [START_REF] Curto | Truncated K-moment problems in several variables[END_REF][START_REF] Laurent | Revisiting two theorems of Curto and Fialkow on moment matrices[END_REF]. Proposition 2.3. Let y be an optimal solution of the SDP (2.4.5) at some order k ∈ N, and assume that the flat extension condition holds, i.e., rank(M k-w (y )) = rank(M k (y )) =: r, with w := max i,j { g i , h j }. Then y has a representing r-atomic measure µ = r t=1 λ j δ a (t) , where (λ 1 , . . . , λ r ) belongs to the standard (r -1)-simplex and {a (1) , . . . , a (r) } ⊂ S(g, h). Moreover, τ k = f and a (1) , . . . , a (r) are all global minimizers of POP (2.4.1).

Henrion and Lasserre [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] provide a numerical algorithm to extract the r minimizers a (1) , . . . , a (r) from M k (y ) when the assumptions of Proposition 2.3 hold.

Finite convergence

Second-order sufficient condition. Given (λ i ) i∈ [m] and (γ j ) i∈[l] , let: [START_REF] Nocedal | Numerical optimization[END_REF]Chapter 2]) The second-order sufficient condition (S2) holds at x ∈ S(g, h) under the three following conditions.

x → L(x, λ, γ) := f (x) - i∈[m] λ i g i (x) - j∈[l] γ j h j (x), x ∈ R n . Given x ∈ S(g, h), let J(x) := { i ∈ [m] : g i (x) = 0 }. Definition 2.1. (see

• KKT-Lagrange multipliers:

There exist λ i ≥ 0, i ∈ [m], and γ j ∈ R, j ∈ [l], such that ∇ x L(x , λ , γ ) = 0 and λ i g i (x ) = 0 for all i ∈ [m].
• Linear independence constraint qualification: The family

{∇g i (x ), ∇h j (x )} i∈J(x ),j∈[l]
is linearly independent.

• Strict complementarity:

λ i + g i (x ) > 0, for all i ∈ [m]. • u ∇ 2 x L(x , λ , γ ) u > 0 for all u = 0 such that u ∇g i (x ) = 0 and u ∇h j (x ) = 0, i ∈ J(x ), for all j ∈ [l].
We say that I(h) is real radical if

I(h) = {f ∈ R[x] : ∃m ∈ N : -f 2m ∈ Σ[x] + I(h)} .
(2.6.1)

The following proposition provides a sufficient condition to ensure finite convergence of the sequence (τ k ) k∈N .

Proposition 2.4. The following statements are true:

1. (Nie [START_REF] Nie | Optimality conditions and finite convergence of Lasserre's hierarchy[END_REF]) The equality τ k = f occurs generically for some k ∈ N. [START_REF] Lasserre | An introduction to polynomial and semi-algebraic optimization[END_REF]Theorem 7.5]) The first statement of Proposition 2.4 means that with fixed f ∈ R[x], the equality τ k = f holds for k ∈ N sufficiently large on a Zariski open set (the complement of the zeros of a polynomial) in the space of the coefficients of g i , h j with given degrees. Note that the real radical property is not generic and so the condition "I(h) is real radical" must be checked case by case. On the other hand, if V (h) is the real zero set of a squared system of polynomial equations, i.e., l = n, then generically V (h) has a finite number of points.

(Lasserre

If (i) Q(g, h) is Archimedean, (ii) the ideal I(h)

Chapter 3

Exploiting correlative and term sparsity

Most of the content of this chapter is from [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]. This chapter is concerned with solving large-scale polynomial optimization problems. As is often the case, the polynomials in the problem description involve only a few monomials of low degree and the ultimate goal is to exploit this crucial feature to provide semidefinite relaxations that are computationally much cheaper than those of the standard SOS-based hierarchy [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF] or its sparse version [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF][START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] based on correlative sparsity.

Throughout this chapter, we consider large-scale instances of the following polynomial optimization problem (POP):

f = inf x { f (x) : x ∈ S(g) }, (3.0.1)
where the objective function f is assumed to be a polynomial in n variables x = (x 1 , . . . , x n ) and the feasible set S(g) ⊆ R n is assumed to be defined by a finite conjunction of m polynomial inequalities, namely S(g) := {x ∈ R n : g 1 (x) ≥ 0, . . . , g m (x) ≥ 0}, (3.0.2)

for some g = {g 1 , . . . , g m } ⊂ R[x].
Here "large-scale" means that the magnitude of the number of variables n and the number of inequalities m can be both proportional to several thousands. To tackle large-scale POPs, a natural idea is to simultaneously benefit from correlative and term sparsity patterns. This is the spirit of our contribution. Also in the same vein the work in [START_REF] Miller | Decomposed structured subsets for semidefinite and sum-of-squares optimization[END_REF] combines the (S)DSOS framework [3] with the TSSOS hierarchy but does not provide systematic convergence guarantees.

Contribution. Our main contribution in this chapter is as follows:

I. For large-scale POPs with a correlative sparsity pattern, we first apply the usual sparse polynomial optimization framework [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF][START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] to get a coarse decomposition in terms of cliques of variables. Next we apply the term sparsity strategy (either TSSOS or Chordal-TSSOS) to each subsystem (which involves only one clique of variables) to reduce the size of SDPs even further. While the overall strategy is quite clear and simple, its implementation is not trivial and needs some care. Indeed for its coherency one needs to take extra care of the monomials which involve variables that belong to intersections of variable cliques (those obtained from correlative sparsity). The resulting combination of correlative sparsity (CS for short) and term sparsity produces what we call the CS-TSSOS hierarchy -a two-level hierarchy of SDP relaxations with blocks of SDP matrices, which yields a converging sequence of certified approximations for POPs. Under certain conditions, we prove that the corresponding sequence of optimal values converges to the global optimum of the POP. II. Our algorithmic development of the CS-TSSOS hierarchy is fully implemented in the TSSOS tool [START_REF] Magron | TSSOS: a Julia library to exploit sparsity for large-scale polynomial optimization[END_REF]. The most recent version of TSSOS has been released within the Julia programming language, which is freely available online and documented. 1 With TSSOS, the accuracy and scalability of the CS-TSSOS hierarchy are evaluated on several large-scale benchmarks coming from the continuous and combinatorial optimization literature. In particular, numerical experiments demonstrate that the CS-TSSOS hierarchy is able to handle challenging Max-Cut instances and optimal power flow instances with several thousand ( 6, 000) variables on a laptop whenever appropriate sparsity patterns are accessible. We remark that the CS-TSSOS framework has been recently extended to handle noncommutative polynomial optimization [START_REF] Wang | Exploiting term sparsity in noncommutative polynomial optimization[END_REF] and complex polynomial optimization [START_REF] Wang | Exploiting sparsity in complex polynomial optimization[END_REF].

The rest of the chapter is organized as follows: in Section 3.1, we provide preliminary background on correlative and term sparsity. In Section 3.2, we explain how to combine them to obtain a two-level CS-TSSOS hierarchy. Its convergence is analyzed in Section 3.3. Eventually, we provide numerical experiments for large-scale POP instances in Section 6.3.

Preliminaries

A polynomial f ∈ R[x] can be written as f (x) = α∈A f α x α with A ⊆ N n and f α ∈ R, x α = x α1 1 • • • x αn n . The support of f is defined by supp(f ) := {α ∈ A | f α = 0}.
We use | • | to denote the cardinality of a set. For a finite set A ⊆ N n , let x A be the |A|-dimensional column vector consisting of elements x α , α ∈ A (fix any ordering on N n ). For a positive integer r, the set of r × r symmetric matrices is denoted by S r and the set of r × r positive semidefinite (PSD) matrices is denoted by S r + . A matrix A ∈ S r + is written as A 0. For matrices A, B ∈ S r , let A, B ∈ R denote the trace inner-product, defined by A, B = trace(A B), and let A • B ∈ S r denote the Hadamard product, defined by

[A • B] ij = A ij B ij . For d ∈ N, let For α ∈ N n , A, B ⊆ N n , let α + B := {α + β | β ∈ B} and A + B := {α + β | α ∈ A, β ∈ B}. For m ∈ N\{0}, let [m] := {1, 2, . . . , m}.

Chordal graphs and sparse matrices

In this subsection, we recall some basic results on chordal graphs and sparse matrices which are crucial for our subsequent development. Our notation and definitions here mostly follow from [START_REF] Vandenberghe | Chordal graphs and semidefinite optimization[END_REF].

An (undirected) graph G(V, E) or simply G consists of a set of nodes V and a set of edges

E ⊆ {{v i , v j } | v i = v j , (v i , v j ) ∈ V × V }.
For a graph G, we use V (G) and E(G) to indicate the node set of G and the edge set of G, respectively. The adjacency matrix of a graph G is denoted by B G for which we put ones on its diagonal. For two graphs G, H, we say that G is a subgraph of H, denoted by G ⊆ H, if both V (G) ⊆ V (H) and E(G) ⊆ E(H) hold. Definition 3.1. A graph is called a chordal graph if all its cycles of length at least four have a chord 2 .

The notion of chordal graphs plays an important role in sparse matrix theory. Any non-chordal graph G(V, E) can be always extended to a chordal graph G (V, E ) by adding appropriate edges to E, which is called a chordal extension of G(V, E). As an example, in Figure 3.1 the two dashed edges are added to obtain a chordal extension. The chordal extension of G is usually not unique and the symbol G is used to represent any specific chordal extension of G throughout the chapter. For graphs G ⊆ H, we assume that G ⊆ H always holds in this chapter.

A complete graph is a graph in which any two nodes have an edge. A clique of a graph is a subset of nodes that induces a complete subgraph. A maximal clique is a clique that is not contained in any other clique. It is known that for a chordal graph, its maximal cliques can be enumerated efficiently in linear time in terms of the number of nodes and edges. See e.g. [START_REF] Blair | An introduction to chordal graphs and clique trees[END_REF][START_REF] Fulkerson | Incidence matrices and interval graphs[END_REF][START_REF] Golumbic | Algorithmic graph theory and perfect graphs[END_REF] for the details.

From now on we consider graphs with the node set V ⊆ N n . Given a graph G(V, E), a symmetric matrix G with rows and columns indexed by V is said to have sparsity pattern G if G βγ = G γβ = 0 whenever β = γ and {β, γ} / ∈ E. Let S G be the set of symmetric matrices with Given a graph G(V, E), the PSD matrices with sparsity pattern G form a convex cone

S |V | + ∩ S G = {G ∈ S G | G 0}. (3.1.1)
Once the sparsity pattern graph G(V, E) is a chordal graph, the cone S

|V |

+ ∩ S G can be decomposed as a sum of simple convex cones thanks to the following theorem and hence the related optimization problem can be solved more efficiently. + ), it suffices to check the positive semidefiniteness of certain blocks of G. For more details on chordal graphs and sparse matrices, the reader may refer to [START_REF] Vandenberghe | Chordal graphs and semidefinite optimization[END_REF].

Correlative sparsity

To exploit correlative sparsity in the Moment-SOS hierarchy for POPs, one proceeds in two steps: 1) decompose the set of variables into cliques according to the links between variables emerging in the input polynomial system, and 2) construct a sparse Moment-SOS hierarchy with respect to the former decomposition of variables [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF].

More concretely, we define the correlative sparsity pattern (csp) graph associated with POP (3.0.1) to be the graph G csp with nodes V = [n] and edges E satisfying {i, j} ∈ E if one of following holds:

(i) there exists α ∈ supp(f ) s.t. α i > 0, α j > 0;

(ii) there exists k ∈ [m] such that x i , x j ∈ var(g k ), where var(g k ) is the set of variables involved in g k .

Let (G csp ) be a chordal extension of G csp and {I c } p c=1 be the list of maximal cliques of (G csp ) with

n c := |I c |. Let R[x(I c )] denote the ring of polynomials in the n c variables x(I c ) = {x j | j ∈ I c }.
We then partition the constraint polynomials g 1 , . . . , g m into groups Assume that f ∈ R[x] can be written as

{g i | i ∈ J c }, c ∈ [p] which satisfy (i) J 1 , . . . , J p ⊆ [m] are pairwise disjoint and ∪ p c=1 J c = [m]; (ii) for any i ∈ J c , var(g i ) ⊆ I c , c ∈ [p].
f = f 1 + • • • + f p , for some f c ∈ R[x(I c )].
We denote k min := max{ f , g i }. Then with k ≥ k min , the moment hierarchy based on correlative sparsity for POP (3.0.1) is defined as

ρ k := inf L y (f ) s.t. M k (y, I c ) 0, c ∈ [p], M k-gi (g i y, I c ) 0, i ∈ J c , c ∈ [p], y 0 = 1. (3.1.4)
In the following, we refer to (3.1.4) as the CSSOS hierarchy for POP (3.0.1). [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF] under some compactness assumption, the sequence (ρ k ) k≥kmin monotonically converges to the global optimum f of POP (3.0.1).

Remark 3.2. As shown in

Term sparsity

In contrast to the correlative sparsity pattern which focuses on links between variables, the term sparsity pattern focuses on links between monomials (or terms). To exploit term sparsity in the Moment-SOS hierarchy one also proceeds in two steps: 1) decompose each involved monomial basis into blocks according to the links between monomials emerging in the input polynomial system, and 2) construct a sparse Moment-SOS hierarchy with respect to the former decomposition of monomial bases [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF][START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF].

More concretely, let A = supp(f ) ∪ m i=1 supp(g i ) and N n k-gi be the standard monomial basis for i = 0, . . . , m. Fixing a relaxation order k ≥ k min , we define the term sparsity pattern (tsp) graph associated with POP (3.0.1) or the support set A, to be the graph

G tsp k with node set V k,0 := N n k and edge set E(G tsp k ) := {{β, γ} | β = γ ∈ V k,0 , β + γ ∈ A ∪ (2N) n }, ( 3
k,0 ) = A ∪ 2N n k and supp(G (0) k,i ) = ∅ for i ≥ 1. Now for each i ∈ {0} ∪ [m],
we iteratively define an ascending chain of graphs (G

(t) k,i (V k,i , E (t) k,i )) t≥1 .
To this end, we start with the initial graph G (0) k,i and each iteration consists of two successive operations: 1) support extension. Define F (t) k,i to be the graph with nodes V k,i and with (recall g 0 = 1)

E(F (t) k,i ) ={{β, γ} | β = γ ∈ V k,i , (3.1.6) (β + γ + supp(g i )) ∩ (∪ m j=0 supp(G (t-1) k,j )) = ∅}, i ∈ {0} ∪ [m]. 2) chordal extension. Let G (t) k,i := (F (t) k,i ) , i ∈ {0} ∪ [m]. (3.1.7) Note that F (1)
k,0 has edges {β, γ} with β + γ ∈ A ∪ (2N) n . To summarise, the iterative process is

G (0) k,i → • • • → G (t-1) k,i support extension -----------→ F (t) k,i chordal extension -----------→ G (t) k,i → • • • , for each i ∈ {0} ∪ [m].
G x 1 x 2 x 3 x 2 x 3 x 1 x 3 x 1 x 2 1 x 2 1 x 2 2 x 2 3 Let r i := |N n k-gi | = n+k-gi k-gi
, i = 0, . . . , m. Then with k ≥ k min and t ≥ 1, the moment hierarchy based on term sparsity for POP (3.0.1) is defined as

inf L y (f ) s.t. B G (t) k,0 • M k (y) ∈ Π G (t) k,0 (S r0 + ), B G (t) k,i • M k-gi (g i y) ∈ Π G (t) k,i (S ri + ), i ∈ [m], y 0 = 1. (3.1.8)
The notation B G • A in (3.1.8) refers to a matrix whose (β, γ)-entry is A βγ if β = γ or {β, γ} ∈ E(G), and 0 otherwise. We call t the sparse order and in the remainder of this chapter, the TSSOS hierarchy for POP (3.0.1) refers to the hierarchy (3.1.8). Remark 3.3. In (3.1.8), one has the freedom to choose a specific chordal extension for any involved graph G (t) k,j . As shown in [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF], if one chooses the maximal chordal extension then with k fixed, the resulting sequence of optimal values of the TSSOS hierarchy (as t increases) monotonically converges in finitely many steps to the optimal value of the corresponding dense moment relaxation for POP (3.0.1).

The CS-TSSOS Hierarchy

When applicable, one can significantly improve the scalability of the Moment-SOS hierarchy by exploiting correlative sparsity or term sparsity. For large-scale POPs, it is then natural to ask whether one can combine correlative sparsity and term sparsity to further reduce the size of SDPs involved in the Moment-SOS hierarchy and to improve its scalability even more. As we shall see in the following sections, the answer is affirmative.

The CS-TSSOS Hierarchy for general POPs

Let us continue considering POP (3.0.1) 3 . A first natural idea to combine correlative sparsity and term sparsity would be to apply the TSSOS hierarchy for each subsystem (involving one variable clique) separately, once the cliques have been obtained from the csp graph of POP (3.0.1). However, with this naive approach convergence may be lost and in the following we take extra care to avoid this annoying consequence.

Let G csp be the csp graph associated with POP (3.0.1), (G csp ) a chordal extension of G csp and {I c } p c=1 be the list of maximal cliques of (G csp ) with n c := |I c |. As in Section 3.1.2, the set of variables x is decomposed into x(I 1 ), x(I 2 ), . . . , x(I p ). Let J 1 , . . . , J p be defined as in Section 3.1.2. Now we apply the term sparsity pattern to each subsystem involving variables x(I c ), c ∈ [p] respectively as follows. Let

A := supp(f ) ∪ m i=1 supp(g i ) and A c := {α ∈ A | supp(α) ⊂ I c } (3.
= (α i ) ∈ N nc → α = (α i ) ∈ N n which satisfies α i = α i , if i ∈ I c , 0, otherwise. Let us define G (0) k,c,0 := G tsp k,c and G (0) k,c,i , i ∈ J c , c ∈ [p]
are all empty graphs with nodes N nc k-gi . Next, for an integer k ≥ 1, for each i ∈ {0} ∪ J c , c ∈ [p], we iteratively define an ascending chain of graphs (G

(t) k,c,i (V k,c,i , E (t) k,c,i )) t≥1 with V k,c,i := N nc k-gi via two successive operations: 1) support extension. Define F (t)
k,c,i to be the graph with nodes V k,c,i and with

E(F (t) k,c,i ) = {{β, γ} | β = γ ∈ V k,c,i , (β + γ + supp(g i )) ∩ C (t-1) k = ∅}, (3.2.2)
where

C (t-1) k := p c=1 (∪ i∈{0}∪Jc (supp(g i ) + supp(G (t-1) k,c,i ))). (3.2.3) 2) chordal extension. Let G (t) k,c,i := (F (t) k,c,i ) , i ∈ {0} ∪ J c , c ∈ [p]. (3.2.4) Example 3.2. Let f = 1 + x 2 1 + x 2 2 + x 2 3 + x 1 x 2 + x 2 x 3 + x 3
and consider the unconstrained POP: min{f (x) : x ∈ R n }. We have n = 3, m = 0 and take the relaxation order k = k min = 1. The variables are decomposed into two cliques: {x 1 , x 2 } and {x 2 , x 3 }. The tsp graphs with respect to these two cliques are illustrated in Figure 3.3. The left graph corresponds to the first clique: x 1 and x 2 are connected because of the term x 1 x 2 . The right graph corresponds to the second clique: 1 and x 3 are connected because of the term x 3 ; x 2 and x 3 are connected because of the term x 2 x 3 . If we apply the TSSOS hierarchy (using the maximal chordal extension in (3.2.4)) separately for each clique, then the graph sequences (G (t) 1,c ) t≥1 , c = 1, 2 (the subscript j is omitted here since there is no constraint) stabilize at t = 1. However, the added (dashed) edge in the right graph corresponds to the monomial x 2 , which only involves the variable x 2 belonging to the first clique. Hence we need to add the edge connecting 1 and x 2 to the left graph in order to get the guarantee of convergence as we shall see in Section 3.3.1. Consequently, the graph sequences (G Let r c,i := b(n c , k -g i ) for all c, i. Then with k ≥ 1, the moment hierarchy based on correlative-term sparsity for POP (3.0.1) is defined as

(t) 1,c ) t≥1 , c = 1, 2 stabilize at t = 2.
ρ (t) k := inf L y (f ) s.t. B G (t) k,c,0 • M k (y, I c ) ∈ Π G (t) k,c,0 (S rc,0 + ), c ∈ [p], B G (t) k,c,i • M k-gi (g i y, I c ) ∈ Π G (t) k,c,i (S rc,i + ), i ∈ J c , c ∈ [p], y 0 = 1. ( 3
G (0) k,c,0 = G tsp k,c ⊆ G tsp k+1,c = G (0) k+1,c,0 , which together with (3.2.2)-(3.2.3) implies that F (1) k,c,j ⊆ F (1) k+1,c,j for j ∈ {0} ∪ J c , c ∈ [p]. It then follows that G (1) k,c,j = (F (1) k,c,j ) ⊆ (F (1) k+1,c,j ) = G (1) k+1,c,j . Now assume that G (t) k,c,j ⊆ G (t) k+1,c,j , j ∈ {0} ∪ J c , c ∈ [p]
, holds for some t ≥ 1. Then by (3.2.2)-(3.2.3) and by the induction hypothesis, we have

F (t+1) k,c,j ⊆ F (t+1) k+1,c,j for j ∈ {0}∪J c , c ∈ [p]. Thus G (t+1) k,c,j = (F (t+1) k,c,j ) ⊆ (F (t+1) k+1,c,j ) = G (t+1) k+1,c,j
which completes the induction.

From Proposition 3.1 and Proposition 3.2, we deduce the following two-level hierarchy of lower bounds for the optimum f of (3.0.1) (3.0.1): , and consider the unconstrained POP: min{f (x) : x ∈ R n }. We have n = 6, m = 0. Let us apply the CS-TSSOS hierarchy (using the maximal chordal extension in (3.2.4)) to this problem by taking the relaxation order k = k min = 2 and the sparse order t = 1. First, according to the csp graph (see Figure 3.4), we decomposes the variables into two cliques: {x 1 , x 2 , x 3 } and {x 3 , x 4 , x 5 , x 6 }. Figure 3.5 and Figure 3.6 illustrate the tsp graphs for the first clique and the second clique, respectively. For the first clique one obtains four blocks of SDP matrices with respective sizes 4, 2, 2, 2. For the second clique one obtains two blocks of SDP matrices with respective sizes 5, 10. As a result, the original SDP matrix of size 28 has been reduced to six blocks of maximal size 10.

ρ (1) kmin ≤ ρ (2) kmin ≤ • • • ≤ ρ kmin ≥ ≥ ≥ ρ (1) kmin+1 ≤ ρ (2) kmin+1 ≤ • • • ≤ ρ kmin+1 ≥ ≥ ≥ . . . . . . . . . . . . ≥ ≥ ≥ ρ (1) k ≤ ρ (2) k ≤ • • • ≤ ρ k ≥ ≥ ≥ .
If one applies the TSSOS hierarchy (using the maximal chordal extension in (3.1.7)) directly to the problem by taking k = k min = 2, t = 1 (i.e., without decomposing variables), then the tsp graph is illustrated in Figure 3 x 2 5

1 x 2 3 x 2 2 x 2 1 x 1 x 2 x 3 x 2 x 3 x 1 x 3 x 1 x 2
x 2 4

x 2 3

x 3

x 5 x 6

x 4 x 6

x 4 x 5

x 3 x 6

x 3 x 5

x 3 x 4

x 6

x 4

x 5

The CS-TSSOS hierarchy entails a trade-off. Indeed, one has the freedom to choose a specific chordal extension for any graph involved in (3.2.5). This choice affects the resulting size of blocks of SDP matrices and the quality of optimal values of corresponding relaxations. Intuitively, chordal extensions with small clique numbers lead to blocks of small size and optimal values of (possibly) x 2 2

x 2 3 x 2 4

x 2 5

x 2 6

x 1 x 2 x 3

x 2 x 3 x 1 x 3 x 1 x 2 x 1 x 4 x 1 x 5 x 1 x 6 x 2 x 4 x 2 x 5 x 2 x 6
x 3

x 5 x 6

x 4 x 6

x 4 x 5

x 3 x 6

x 3 x 5

x 3 x 4

x 6

x 4

x 5 low quality while chordal extensions with large clique numbers lead to blocks of large size and optimal values of (possibly) high quality.

For all c, i, write M k-gi (g i y, I c ) = α D c,i α y α for appropriate symmetry matrices {D c,i α }. Then for each k ≥ 1, the dual of (3.2.5) reads as:

sup ρ s.t. p c=1 j∈{0}∪Jc G c,j , D c,j α + ρδ 0α = f α , ∀α ∈ C (t) k , G c,j ∈ S rc,j + ∩ S G (t) k,c,j , j ∈ {0} ∪ J c , c ∈ [p], (3.2.7)
where

C (t) k is defined in (3.2.3). Proposition 3.3. Let f ∈ R[x]
and S(g) be as in (3.0.2). Assume that S(g) has a nonempty interior. Then there is no duality gap between (3.2.5) and (3.2.7) for any k ≥ k min and t ≥ 1.

Proof. By the duality theory of convex programming, this easily follows from Theorem 3.6 of [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF] and Theorem 3.2.

Note that the number of equality constraints in (3.2.7) is equal to the cardinality of C (t) k . We next give a description of the elements in

C (t)
k in terms of sign symmetries.

Sign symmetries

Definition 3.2. Given a finite set A ⊆ N n , the sign symmetries of A are defined by all vectors r ∈ Z n 2 := {0, 1} n such that r α ≡ 0 (mod 2) for all α ∈ A. For any α ∈ N n , we define (α) 2 := (α 1 (mod 2), . . . , α n (mod 2)) ∈ Z n 2 . We also use the same notation for any subset A ⊆ N n , i.e., (A)

2 := {(α) 2 | α ∈ A} ⊆ Z n 2 . For a subset S ⊆ Z n 2 ,
i , υ i+1 } ∈ E(G), i ∈ [r -1]. From (β + υ 1 ) 2 , (υ 1 + υ 2 ) 2 ∈ (supp(G)) 2 , we deduce that (β + υ 2 ) 2 ∈ (supp(G)) 2 Z2 because (supp(G))
k = p c=1 supp(G (0) k,c,0 ) = p c=1 supp(G tsp k,c ) ⊆ p c=1 (A c ∪ (2N) nc ) ⊆ A ∪ (2N) n . Hence (C (0) k ) 2 ⊆ (A) 2 Z2 . Now assume that (C (t) k ) 2 ⊆ (A)
(t+1) k,c,j )) 2 ⊆ (A) 2 Z2 . Because G (t+1) k,c,j = (F (t+1)
k,c,j ) , by Lemma 3.2, we have (supp(G

(t+1) k,c,j )) 2 ⊆ (supp(F (t+1) k,c,j )) 2 Z2 ⊆ (A) 2 Z2 .
From this, (3.2.3) and the fact that (A) 2 Z2 is closed under addition modulo 2, we then deduce the inclusion (C

(t+1) k ) 2 ⊆ (A) 2 Z2
which completes the induction. Remark 3.5. Proposition 3.4 actually indicates that the block structure produced by the CS-TSSOS hierarchy is consistent with the sign symmetries of the POP.

Convergence analysis

Global convergence

We next prove that if for any graph involved in (3.2.5), the chordal extension is chosen to be maximal, then for any relaxation order k ≥ k min the sequence of optimal values (ρ (t) k ) t≥1 of the CS-TSSOS hierarchy converges to the optimal value ρ k of the corresponding CSSOS hierarchy (3.1.4). In turn, as the relaxation order k increases, the latter sequence converges to the global optimum f of the original POP (3.0.1) (after adding some redundant quadratic constraints) as shown in [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF].

Obviously, the sequences of graphs (G

(t) k,c,j (V k,c,j , E (t)
k,c,j )) t≥1 stabilize for all j ∈ {0} ∪ J c , c ∈ [p] after finitely many steps. We denote the resulting stabilized graphs by

G ( * ) k,c,j , j ∈ {0} ∪ J c , c
| α ∈ p c=1 (∪ i∈{0}∪Jc (supp(g i ) + supp(G ( * ) k,c,i )))} is the set of decision variables involved in (Q cs-ts k, *
). Let R be the set of decision variables involved in (3.1.4). We then define a vector y = (y α ) α∈R as follows:

y α = y α , if α ∈ p c=1 (∪ i∈{0}∪Jc (supp(g i ) + supp(G ( * ) k,c,i ))), 0, otherwise.
By construction and since G ( * ) k,c,i stabilizes under support extension for all c, i, we have M k-gi (g i y,

I c ) = B G ( * ) k,c,i
• M k-gi (g i y, I c ). As we use the maximal chordal extension in (3.2.4), the matrix

B G ( * ) k,c,i • M k-gi (g i y, I c ) is block diagonal up to permutation (see Remark 3.1). So from B G ( * ) k,c,i •M k-gi (g i y, I c ) ∈ Π G ( * ) k,c,i (S rc,i + ) it follows M k-gi (g i y, I c ) 0 for i ∈ {0} ∪ J c , c ∈ [p].
Therefore y is a feasible solution of (3.1.4) and so L y (f ) = L y (f ) ≥ ρ k . Hence ρ * k ≥ ρ k since y is an arbitrary feasible solution of (3.2.5) with value ρ * k . By Proposition 3.1, we already have ρ * k ≤ ρ k . Therefore, ρ * k = ρ k . To guarantee the global optimality, we need the following compactness assumption on the feasible set S(g). Assumption 1. Let S(g) be as in (3.0.2). There exists an M > 0 such that x ∞ < M for all x ∈ S(g).

Because of Assumption 1, one has x(I

c ) 2 2 ≤ n c M 2 , c ∈ [p]
. Therefore, we can add the p redundant quadratic constraints

g m+c (x) := n c M 2 -x(I c ) 2 2 ≥ 0, c ∈ [p] (3.3.1)
in the definition (3.0.2) of S(g) and set m = m + p, so that S(g) is now defined by

S(g) := {x ∈ R n | g i (x) ≥ 0, i ∈ [m ]}. (3.3.2) Note that g m+c ∈ R[x(I c )] for c ∈ [p].
Then by Theorem 3.6 in [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF], the sequence (ρ k ) k≥kmin converges to the globally optimal value f of POP (3.0.1). So this together with Theorem 3.3 gives the global convergence of the CS-TSSOS hierarchy.

A sparse representation theorem

Proceeding along Theorem 3.3, we are able to provide a sparse representation theorem for a polynomial positive on a compact basic semialgebraic set.

for some polynomials σ c,i ∈ Σ[x(I c )], i ∈ {0} ∪ J c , c ∈ [p],
satisfying R α ≡ 0 (mod 2) for any α ∈ supp(σ c,i ), i.e., (supp(σ c,i )) 2 ⊆ R ⊥ , where we regard R as a set of its column vectors. That is, (3.3.3) provides a certificate of positivity of f on S(g).

Proof. By Corollary 3.9 of [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF] (or Theorem 5 of [START_REF] Grimm | A note on the representation of positive polynomials with structured sparsity[END_REF]), there exist polynomials

σ c,i ∈ Σ[x(I c )], i ∈ {0} ∪ J c , c ∈ [p] such that f = p c=1 σ c,0 + i∈Jc σ c,i g i . (3.3.4) Let k = max{ deg(σ c,i g i )/2 : i ∈ {0} ∪ J c , c ∈ [p]}.
Let G c,i be a PSD Gram matrix associated with σ c,i and indexed by the monomial basis N nc k-gi . Then for all c, i, we define G c,i ∈ S rc,i with

r c,i = b(n c , k -g i ) (indexed by N nc k-gi ) by [G c,i ] βγ := [G c,i ] βγ , if R (β + γ) ≡ 0 (mod 2), 0, otherwise,
and let σ c,i = (x

N nc k-g i ) G c,i x N nc k-g i .
One can easily verify that G c,i is block diagonal up to permutation (see also [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF]) and each block is a principal submatrix of G c,i . Then the positive semidefiniteness of G c,i implies that G c,i is also positive semidefinite. Thus

σ c,i ∈ Σ[x(I c )].
By construction, substituting σ c,j with σ c,j in (3.3.4) boils down to removing the terms with exponents α that do not satisfy R α ≡ 0 (mod 2) from the right hand side of (3.3.4). Since any α ∈ supp(f ) satisfies R α ≡ 0 (mod 2), this does not change the match of coefficients on both sides of the equality. Thus we obtain

f = p c=1 σ c,0 + i∈Jc σ c,i g i
with the desired property.

Extracting a solution

In the case of dense Moment-SOS relaxations, there is a standard procedure described in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] to extract globally optimal solutions when the so-called flatness condition for the moment matrix is satisfied. This procedure was partially generalized to the correlative sparsity setting in [103, § 3.3]. However, in the combined sparsity setting, the corresponding procedure cannot be applied because we do not have complete information on the moment matrix associated with each clique.

In order to extract a solution in this case, we may add a dense moment matrix of order one for each clique in (3.2.5):

inf L y (f ) s.t. B G (t) k,c,0 • M k (y, I c ) ∈ Π G (t) k,c,0 (S rc,0 + ), c ∈ [p], M 1 (y, I c ) 0, c ∈ [p], B G (t) k,c,i • M k-gi (g i y, I c ) ∈ Π G (t) k,c,i (S rc,i + ), i ∈ J c , c ∈ [p], y 0 = 1. (3.3.5)
Let y be an optimal solution of (3.3.5). Typically, M 1 (y , I c ) (after identifying sufficiently small entries with zeros) is a block diagonal matrix (up to permutation). If for all c, every block of M 1 (y , I c )) is of rank one, then a globally optimal solution x to (3.0.1) which is unique up to sign symmetries can be extracted, and the global optimality is certified (see [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF]Theorem 3.2]). Otherwise, the relaxation might be not exact or yield multiple global solutions. Remark 3.6. Note that (3.3.5) is a tighter relaxation of (3.0.1) than (3.2.5) and so might provide a better lower bound for (3.0.1).

Applications and numerical experiments

In this section, we conduct numerical experiments for the proposed CS-TSSOS hierarchy and apply it to two important classes of POPs: Max-Cut problems and AC optimal power flow (AC-OPF) problems. Depending on specific problems, we consider two types of chordal extensions for the term sparsity pattern: maximal chordal extensions and approximately smallest chordal extensions 4 . The tool TSSOS which executes the CS-TSSOS hierarchy (as well as the CSSOS hierarchy and the TSSOS hierarchy) is implemented in Julia. For an introduction to TSSOS, one could refer to [START_REF] Magron | TSSOS: a Julia library to exploit sparsity for large-scale polynomial optimization[END_REF]. TSSOS is available on the website: https://github.com/wangjie212/TSSOS.

In the following subsections, we compare the performances of the CSSOS approach, the TSSOS approach, the CS-TSSOS approach and the SDSOS approach [3] (implemented in SPOT [START_REF] Megretski | Systems polynomial optimization tools (SPOT)[END_REF]). Mosek [6] is used as an SDP (in the CSSOS, TSSOS, CS-TSSOS cases) or SOCP (in the SDSOS case) solver. All numerical examples were computed on an Intel Core i5-8265U@1.60GHz CPU with 8GB RAM memory. The timing includes the time required to generate the SDP/SOCP and the time spent to solve it. The notations used in this section are listed in Table 3.1.

Benchmarks for unconstrained POPs

The Broyden banded function is defined as

f Bb (x) = n i=1 (x i (2 + 5x 2 i ) + 1 - j∈Ji (1 + x j )x j ) 2 ,
where

J i = {j | j = i, max(1, i -5) ≤ j ≤ min(n, i + 1)}.
The task is to minimize the Broyden banded function over R n which is formulated as an unconstrained POP. Using the relaxation order k = 3, we solve the CSSOS hierarchy (3.1.4), the TSSOS hierarchy (3.1.8) with t = 1 and the CS-TSSOS hierarchy (3.2.5) with t = 1. In the latter two cases, approximately smallest chordal extensions are used. We also solve the POP with the SDSOS approach. The results are displayed in Table 3.2.

It can be seen from the table that CS-TSSOS significantly reduces the maximal size of SDP blocks and is the most efficient approach. CSSOS, TSSOS and CS-TSSOS all give the exact minimum 0 while SDSOS only gives a very loose lower bound -13731 when n = 20. Due to the limitation of memory, CSSOS scales up to 180 variables; TSSOS scales up to 40 variables; SDSOS scales up to 20 variables. On the other hand, CS-TSSOS can easily handle instances with up to 500 variables.

Benchmarks for constrained POPs

• The generalized Rosenbrock function

f gR (x) = 1 + n i=2 (100(x i -x 2 i-1 ) 2 + (1 -x i ) 2 ).
• The Broyden tridiagonal function

f Bt (x) =((3 -2x 1 )x 1 -2x 2 + 1) 2 + n-1 i=2 ((3 -2x i )x i -x i-1 -2x i+1 + 1) 2 + ((3 -2x n )x n -x n-1 + 1) 2 .
• The chained Wood function

f cW (x) =1 + i∈J (100(x i+1 -x 2 i ) 2 + (1 -x i ) 2 + 90(x i+3 -x 2 i+2 ) 2 + (1 -x i+2 ) 2 + 10(x i+1 + x i+3 -2) 2 + 0.1(x i+1 -x i+3 ) 2 ),
where J = {1, 3, 5, . . . , n -3} and 4|n.

With the generalized Rosenbrock (resp. Broyden tridiagonal or chained Wood) function as the objective function, we consider the following constrained POP:

inf f gR (resp. f Bt or f cW ) s.t. 1 -( 20j i=20j-19 x 2 i ) ≥ 0, j ∈ [n/20], (3.4.1)
where 20|n. The generalized Rosenbrock function, the Broyden tridiagonal function and the chained Wood function involve cliques with 2 or 3 variables, which can be efficiently handled by the CSSOS hierarchy; see [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF]. For them, the CS-TSSOS hierarchy gives almost the same results with the CSSOS hierarchy. Hence we add the sphere constraints in (3.4.1) to increase the clique size and to show the difference. For these problems, the minimum relaxation order k = 2 is used. As in the unconstrained case, we solve the CSSOS hierarchy (3.1.4), the TSSOS hierarchy (3.1.8) with t = 1 and the CS-TSSOS hierarchy (3.2.5) with t = 1, and use approximately smallest chordal extensions. We also solve these POPs with the SDSOS approach. The results are displayed in Tables 3.3-3.5.

From these tables, one can see that CS-TSSOS significantly reduces the maximal size of SDP blocks and is again the most efficient approach. For the generalized Rosenbrock function, CSSOS, TSSOS and CS-TSSOS give almost the same optimum while SDSOS gives a slightly loose lower bound (only for n = 40); for the Broyden tridiagonal function, CSSOS, TSSOS and CS-TSSOS all give the same optimum while SDSOS gives a very loose lower bound (only for n = 40); for the chained Wood function, CSSOS, TSSOS and CS-TSSOS all give the same optimum while SDSOS gives a slightly loose lower bound (only for n = 40). Due to the limitation of memory, CSSOS scales up to 180 variables; TSSOS scales up to 180 or 200 variables; SDSOS scales up to 40 variables. On the other hand, CS-TSSOS can easily handle these instances with up to 1000 variables.

The Max-Cut problem

The Max-Cut problem is one of the basic combinatorial optimization problems, which is known to be NP-hard. Let G(V, E) be an undirected graph with V = [n] and with edge weights w ij for {i, j} ∈ E. Then the Max-Cut problem for G can be formulated as a QCQP in binary variables:

max 1 2 {i,j}∈E w ij (1 -x i x j ) s.t. 1 -x 2 i = 0, i ∈ [n]. (3.4.2)
The property of binary variables in (3.4.2) can be also exploited to reduce the size of SDPs arising from the Moment-SOS hierarchy, which has been implemented in TSSOS. For the numerical experiments, we construct random instances of Max-Cut problems with a block-band sparsity pattern (illustrated in Figure 3.8) which consists of l blocks of size b and two bands of width h. Here we select b = 25 and h = 5. For a given l, we generate a random sparse binary matrix A ∈ S lb+h according to the block-arrow sparsity pattern: the entries out of the blue area take zero; the entries in the block area take one with probability 0.16; the entries in the band area take one with probability 2/ √ l. Then we construct the graph G with A as its adjacency matrix. For each edge {i, j} ∈ E(G), the weight w ij randomly takes values 1 or -1 with equal probability. Doing so, we build 10 Max-Cut instances with l = 20, 40, 60, 80, 100, 120, 140, 160, 180, 200, respectively 5 . The largest number of nodes is 5005. For each instance, we solve the first-order Moment-SOS relaxation (Shor's relaxation), the CSSOS hierarchy with k = 2, and the CS-TSSOS hierarchy with k = 2, t = 1 for which the maximal chordal extension is used. The results are displayed in Table 3.6. From the table we can see that for each instance, both CSSOS and CS-TSSOS significantly improve the bound obtained by Shor's relaxation. Meanwhile, CS-TSSOS is several times faster than CSSOS at the cost of possibly providing a sightly weaker bound. In addition, CS-TSSOS yields smaller block sizes than CSSOS. In this table, only the integer part of optimal values is preserved.

The AC-OPF problem

The AC optimal power flow (AC-OPF) is a central problem in power systems. It can be formulated as the following POP in complex variables V i , S g q , S ij :

                                       inf Vi,S g q ,Sij q∈G (c 2q ( (S g q )) 2 + c 1q (S g q ) + c 0q ) s.t. ∠V r = 0, S gl q ≤ S g q ≤ S gu q , ∀q ∈ G, υ l i ≤ |V i | ≤ υ u i , ∀i ∈ N, q∈Gi S g q -S d i -Y s i |V i | 2 = (i,j)∈Ei∪E R i S ij , ∀i ∈ N, S ij = (Y * ij -i b c ij 2 ) |Vi| 2 |Tij | 2 -Y * ij ViV * j Tij , ∀(i, j) ∈ E, S ji = (Y * ij -i b c ij 2 )|V j | 2 -Y * ij V * i Vj T * ij , ∀(i, j) ∈ E, |S ij | ≤ s u ij , ∀(i, j) ∈ E ∪ E R , θ ∆l ij ≤ ∠(V i V * j ) ≤ θ ∆u ij , ∀(i, j) ∈ E. (3.4.3)
The meaning of the symbols in (3.4.3) is as follows: N -the set of buses, G -the set of generators, G i -the set of generators connected to bus i, E -the set of from branches, E R -the set of to branches, E i and E R i -the subsets of branches that are incident to bus i, i -imaginary unit, V i -the voltage at bus i, S g q -the power generation at generator q, S ij -the power flow from bus i to bus j, (•) -real part of a complex number, ∠(•) -angle of a complex number, | • | -magnitude of a complex number, (•) * -conjugate of a complex number, r -the voltage angle reference bus. All symbols in boldface are constants (c 0q , c 1q , c 2q ,

υ l i , υ u i , s u ij , θ ∆l ij , θ ∆u ij ∈ R,S gl q , S gu q , S d i , Y s i , Y ij , b c ij , T ij ∈ C).
For a full description on the AC-OPF problem, the reader may refer to [START_REF] Babaeinejadsarookolaee | The power grid library for benchmarking ac optimal power flow algorithms[END_REF]. By introducing real variables for both real and imaginary parts of each complex variable, we can convert the AC-OPF problem to a POP involving only real variables 6 .

To tackle an AC-OPF instance, we first compute a locally optimal solution with a local solver and then rely on an SDP relaxation to certify the global optimality. Suppose that the optimal value reported by the local solver is AC and the optimal value of the SDP relaxation is opt. The optimality gap between the locally optimal solution and the SDP relaxation is defined by

gap := AC -opt AC × 100%.
If the optimality gap is less than 1.00%, then we accept the locally optimal solution as globally optimal. For many AC-OPF instances, the first-order Moment-SOS relaxation (Shor's relaxation) is already able to certify the global optimality (with an optimality gap less than 1.00%). Therefore, we focus on the more challenging AC-OPF instances for which the optimality gap given by Shor's relaxation is greater than 1.00%. We select such instances from the AC-OPF library PGLiB [START_REF] Babaeinejadsarookolaee | The power grid library for benchmarking ac optimal power flow algorithms[END_REF].

Since we shall go to the second-order Moment-SOS relaxation, we can replace the variables S ij and S ji by their right-hand side expressions in (3.4.3) and then convert the resulting problem to a POP involving real variables. The data for these selected AC-OPF instances are displayed in Table 3.7, where the AC values are taken from PGLiB.

We solve Shor's relaxation, the CSSOS hierarchy with k = 2 and the CS-TSSOS hierarchy with k = 2, t = 1 for these AC-OPF instances and the results are displayed in Tables 3.7-3.8. For instances 162 ieee dtc, 162 ieee dtc api, 500 tamu, 1888 rte, with maximal chordal extensions Mosek ran out of memory and so we use approximately smallest chordal extensions. As the tables show, CS-TSSOS is more efficient and scales much better with the problem size than CSSOS. In particular, CS-TSSOS succeeds in reducing the optimality gap to less than 1.00% for all instances.

Exploiting the constant trace property: Equality constraints

Most of the content of this chapter is from [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF].

In the previous chapter, we have combined correlative and term sparsity exploitation to improve the scalability of the Moment-SOS hierarchy for POP.

Another complementary workaround to improve further the scalability is by exploiting a Constant Trace Property (CTP) of semidefinite relaxations associated with POPs coming from combinatorial optimization [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF][START_REF] Yurtsever | Scalable semidefinite programming[END_REF]. This permits to solve a given semidefinite relaxation with ad-hoc methods, like, e.g., limited-memory bundle methods, instead of the costly interior-point methods.

The present chapter is part of this effort.

Background on SDP with CTP

One way to exploit the CTP of matrices in SDPs is to consider the dual which reduces to minimize the maximum eigenvalue of a symmetric matrix pencil [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF]. For problems of moderate size one may solve the latter problem with interior-point methods [START_REF] Ben-Tal | Lectures on modern convex optimization: analysis, algorithms, and engineering applications[END_REF]. However for larger-scale instances, running a single iteration becomes computationally too demanding and therefore one has to use an alternative method, and in particular first-order methods.

To solve large-scale instances of this maximal eigenvalue minimization problem, two types of first-order methods can be used: subgradient descent or variants of the mirror-prox algorithm [START_REF] Nemirovsky | Problem complexity and method efficiency in optimization[END_REF], and spectral bundle methods [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF]. In other methods of interest based on non-convex formulations [START_REF] Burer | A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization[END_REF][START_REF] Journée | Low-rank optimization for semidefinite convex problems[END_REF], the problem is directly solved over the set of low rank matrices. These latter approaches are particularly efficient for problems where the solution is low rank, e.g., for matrix completion or combinatorial relaxations.

Despite their empirical efficiency, the computational complexity of spectral bundle and low rank methods is still not completely understood. This is in contrast with methods based on stochastic smoothing results for which explicit computational complexity estimates are available. For instance in [START_REF] Karoui | A stochastic smoothing algorithm for semidefinite programming[END_REF] smooth stochastic approximations of the maximum eigenvalue function are obtained via rank-one Gaussian perturbations. In [START_REF] Overton | Second derivatives for optimizing eigenvalues of symmetric matrices[END_REF] Newton's method is used, assuming that the multiplicity of the maximal eigenvalue is known in advance.

By combining quasi-Newton methods (e.g. Broyden-Fletcher-Goldfarb-Shanno (BFGS) method or its so-called "Limited-memory" version (L-BFGS) [START_REF]Updating quasi-newton matrices with limited storage[END_REF]) with adaptive gradient sampling [START_REF] Burke | A robust gradient sampling algorithm for nonsmooth, nonconvex optimization[END_REF][START_REF] Kiwiel | Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization[END_REF], convergence guarantees are obtained for certain non smooth problems while keeping good empirical performance [START_REF] Lewis | Nonsmooth optimization via quasi-Newton methods[END_REF][START_REF] Curtis | A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees[END_REF].

Another hybrid method is the Limited-Memory Bundle Method (LMBM) which combines L-BFGS with bundle methods [START_REF] Haarala | Globally convergent limited memory bundle method for large-scale nonsmooth optimization[END_REF][START_REF] Haarala | New limited memory bundle method for largescale nonsmooth optimization[END_REF]: Briefly, L-BFGS is used in the line search procedure to determine the step sizes in the bundle method. LMBM enjoys global convergence for locally Lipschitz continuous functions which are not necessarily differentiable.

Finally the more recent SketchyCGAL algorithm [START_REF] Yurtsever | Scalable semidefinite programming[END_REF] also uses limited memory and arithmetic. It combines a primal-dual optimization scheme together with a randomized sketch for low-rank matrix approximation. Assuming that zero duality holds, it provides a near-optimal low-rank approximation. A variant of SketchyCGAL can handle SDPs with bounded (instead of constant) trace property.

Concerning SDPs coming from relaxations in polynomial optimization, Malick and Henrion [78, Section 3.2.3] have used the CTP to provide an efficient algorithm for unconstrained polynomial optimization problems. At last but not least, the CTP trivially holds for Shor's relaxation [START_REF] Shor | Quadratic optimization problems[END_REF] of combinatorial optimization problems formulated as linear-quadratic POPs on the discrete hypercube {-1, 1} n . This fact has been exploited in Helmberg and Rendl [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF] to avoid solving the associated SDP via interior-point methods.

Contribution

A novelty with respect to previous efforts is to show that every POP on a compact basic semialgebraic set has an equivalent equality constrained POP formulation on an Euclidean sphere (possibly after adding some artificial variables) such that each of its semidefinite relaxations in the Moment-SOS hierarchy has the CTP. We call CTP-POP such a formulation of POPs. Therefore to solve each semidefinite relaxation of a CTP-POP one may avoid the computationally costly interiorpoint methods in some cases. Indeed as the dual reduces to minimize the largest eigenvalue of a matrix pencil, one may rather use efficient ad-hoc non smooth methods as those invoked above.

I. In Section 4.2.1, we prove that each semidefinite moment relaxation indexed by k ∈ N:

-τ k = sup X∈S (k) { C k , X : A k X = b k , X 0} , (4.0.1)
of the Moment-SOS hierarchy associated with an equality constrained POP on an Euclidean sphere of R n has CTP (see Lemma 4.4), i.e.,

∀ X ∈ S (k) , A k X = b k ⇒ trace(X) = a k ,
where

A k : S (k) → R m k is a linear operator with S (k) being the set of real symmetric matrices of size n+k n , C k ∈ S (k) and b k ∈ R m k with m k = O n+k n 2 .
Following the framework by Helmberg and Rendl [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF], SDP (4.0.1) boils down to minimizing the largest eigenvalue of a matrix pencil:

-τ k = inf z∈R m k a k λ 1 (C k -A k z) + b k z , (4.0.2)
where λ 1 (A) stands for the largest eigenvalue of A and A k denotes the adjoint operator of A k . Hence (4.0.2) form what we call a hierarchy of (non smooth, convex) spectral relaxations of the equality constrained POP on a sphere. Convergence of (τ k ) k∈N to the optimal value f of the initial POP is guaranteed with rate at least O(k -c ) (see Theorem 4.1), where c depends only on the polynomials describing the cost and constraints of the POP.

In addition, existence of an optimal solution of the spectral relaxation (4.0.2) is guaranteed for sufficiently large k under certain conditions on the POP (see Proposition 4.1). Finally, when the set of global minimizers of the equality constrained POP on the sphere is finite, we also describe how to obtain an optimal solution x via an optimal solution z of (4.0.2).

II. In Section 4.2 we prove that any POP on a compact basic semialgebraic set (including a ball constraint R -x 2 2 ≥ 0) has an equivalent equality constrained POP (called CTP-POP) on a sphere of R n+m+1 , where m is the number of inequality constraints of the initial POP. This CTP-POP can be solved by using spectral relaxations (4.0.2).

III. We describe Algorithm 3 to handle a given equality constrained POP on the sphere. It consists of handling each semidefinite relaxation (4.0.1) by solving the spectral formulation (4.0.2), with a nonsmooth optimization procedure chosen in advance by the user in our software library, called SpectralSOS. This library supports the three optimization subroutines LMBM [START_REF] Haarala | Globally convergent limited memory bundle method for large-scale nonsmooth optimization[END_REF][START_REF] Haarala | New limited memory bundle method for largescale nonsmooth optimization[END_REF], proximal bundle (PB) [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF], and SketchyCGAL [START_REF] Yurtsever | Scalable semidefinite programming[END_REF]. Our default method in Algorithm 3 is LMBM.

IV. Finally, efficiency and robustness of SpectralPOP are illustrated in Section 4.3 on extensive benchmarks. We solve several (randomly generated) dense equality constrained QCQPs on the unit sphere by running Algorithm 3 and compare results with those obtained with the standard Moment-SOS hierarchy. Suprisingly SpectralPOP can provide the optimal value as well as an optimal solution with high accuracy, and up to twenty five times faster than the semidefinite hierarchy. For instance, SpectralPOP can solve the first relaxation of minimization problem of dense quadratic polynomials on the unit sphere with up to n = 500 variables in about 35 seconds and up to 1500 variables in about 7000 seconds on a standard laptop computer. We emphasize that for some problems not randomly generated and scaled so as to fit our optimization framework on the unit sphere, we could observe a lack of high precision after transferring results (of the scaled formulation) back to the unscaled initial formulation.

We also provide extended applications of spectral relaxations to the following three decision problems: deciding nonnegativity of even degree forms, deciding convexity of even degree forms and deciding copositivity of real symmetric matrices, with very satisfactory results.

In [START_REF] Helmberg | A spectral bundle method for semidefinite programming[END_REF], Helmberg and Rendl propose a spectral bundle method (based on Kiwiel's proximal bundle method [START_REF] Kiwiel | Proximity control in bundle methods for convex nondifferentiable minimization[END_REF]) to solve an SDP relying on the maximal eigenvalue minimization problem of the form (4.0.2). This method works better than interior-point algorithms for very large-scale SDPs, when the number of trace equality constraints is not larger than the size of the positive semidefinite matrix (e.g., Shor's relaxation of MAXCUT problems). However this method is not always more efficient than interior-point solvers (e.g., SDPT3) for instance when the SDPs involve a number of trace equality constraints which is larger than the size of the positive semidefinite matrix, as reported in [START_REF] Helmberg | The spectral bundle method with second-order information[END_REF]. Unfortunately this latter type of SDP is the generic form of Moment-SOS relaxations for POPs and thus is not suitable to be solved by Helmberg-Rendl's spectral bundle method. By contrast with previous works, our numerical results show that the combination between Helmberg-Rendl's spectral formulation and LMBM is cheaper and faster than Mosek (the currently fastest SDP solver based on interior-point method) while maintaining the same accuracy when solving moment relaxations of equality constrained POPs on a sphere.

Background and Preliminary Results

General POPs on basic compact semialgebraic sets

We recall that a POP is of the form

f := inf{f (x) : x ∈ S(g, h)} , (4.1.1)
where S(g, h) is a basic semialgebraic set defined as follows:

S(g, h) := { x ∈ R n : g i (x) ≥ 0 , i ∈ [m] ; h j (x) = 0 , j ∈ [l] } (4.1.2) for some polynomials f, g i , h j ∈ R[x]. Here g := {g i } i∈[m] and h := {h j } j∈[l] . Set g i := deg(g i )/2 and h j := deg(h j )/2 .
If S(g, h) is nonempty and compact, then f < ∞ and POP (4.1.1) has at least one global minimizer. Next, as we are concerned with POPs on compact feasible sets, we assume that S(g, h) ⊂ B(0, R), where B(0, R) := {x ∈ R n : R -x 2 2 ≥ 0}. In addition, if m = 0 then we may and will assume that

g 1 := R -x 2 2 .

POPs on a variety contained in a sphere

We consider a special form of POP (4.1.1) which is of the form

f := inf { f (x) : x ∈ V (h)} , (4.1.3)
where V (h) is the real variety defined by:

V (h) := { x ∈ R n : h j (x) = 0 ; j ∈ [l] } , (4.1.4)
for some set of polynomials h

:= {h j } l j=1 ⊂ R[x]. We assume that h 1 := R -x 2 2 for some R > 0, so that V (h) ⊂ ∂B(0, R), where ∂B(0, R) := {x ∈ R n : R -x 2 2 = 0}. By assuming that V (h) = ∅, f < ∞ and POP (4.1.
3) has at least one global minimizer.

Given k ∈ N, define the truncated preordering of order k associated with the variety V (h) in (4.1.4) as follows:

P k (h) :=    σ 0 + l j=1 ψ j h j : σ 0 ∈ Σ[x] k , ψ j ∈ R[x] 2(k-hj ) , j ∈ [l]    . Remark 4.1. For every k ∈ N, P k (h) is also the truncated quadratic module Q k (h) associated with the semialgebraic set V (h) = S(∅, h).
As a consequence of Schweighofer's main result in [START_REF] Schweighofer | On the complexity of Schmüdgen's Positivstellensatz[END_REF]Theorem 4], one obtains the following result: Lemma 4.1. Let f be as in (4.1.3) with V (h) as in (4.1.4). There exists c > 0 depending on V such that for k ∈ N with k ≥ cd c n cd , one has

(f -f ) + cd 4 n 2d f k -1/c ∈ P k (h) .
Note that in the case of polynomial optimization on the sphere (i.e., h = {R -x 2 2 } for some R > 0), one can take c = 1 2 in Lemma 4.1, as a consequence of the convergence result from [START_REF] Fang | The sum-of-squares hierarchy on the sphere and applications in quantum information theory[END_REF]. Next, consider the hierarchy of semidefinite programs (SDP) indexed by k ∈ N:

ρ k := sup { ξ ∈ R : f -ξ ∈ P k (h)} . (4.1.5)
For every k ∈ N, the dual of (4.1.5) reads With V (h) in lieu of S(g, h), zero duality gap as well as analogues of Proposition 2.1 and 2.3, also hold.

τ k := inf y∈R b(n,2k) L y (f ) s.t. M k (y) 0 ; y 0 = 1 M k-hj (h j y) = 0 , j ∈ [l] .
1. For all k ∈ N, ρ k ≤ ρ k+1 ≤ f . 2. The sequence (ρ k ) k∈N converges to f with rate at least O(k -1/c ).

If the ideal I(h) is real radical and the second-order sufficiency conditions (Definition 2.1) hold at every global minimizer of POP

Spectral minimizations of SDP

Let s, l, s j ∈ N >0 , j ∈ [l], be fixed such that s = l j=1 s (j) . Let S be the set of real symmetric matrices of size s in a block diagonal form:

X = diag(X 1 , . . . , X l ) , (4.1.7)
such that X j is of size s (j) , j ∈ [l]. Let S + be the set of all X ∈ S such that X 0, i.e., X has only nonnegative eigenvalues. Then S is a Hilbert space with scalar product A, B = trace(B A) and S + is a self-dual cone.

Let us consider the following SDP:

-τ = sup X∈S { C, X : AX = b , X 0 } , (4.1.8)
where A : S → R m is a linear operator of the form

AX = [ A 1 , X , . . . , A m , X ] ,
with

A i ∈ S, i ∈ [m]
, C ∈ S is the cost matrix and b ∈ R m is the right-hand-side vector.

The dual of SDP (4.1.8) reads:

-ρ = inf z { b z : A z -C 0 } , (4.1.9)
where A : R m → S is the adjoint operator of A, i.e., A z = m i=1 z i A i . The following assumption will be used in the next two sections: 3. Dual attainability: SDP (4.1.9) has an optimal solution.

Constant trace property (CTP):

There exists a > 0 such that

∀ X ∈ S , AX = b ⇒ trace(X) = a .
(4.1.10)

Bounded trace property (BTP):

There exists a > 0 such that

∀ X ∈ S , AX = b ⇒ trace(X) ≤ a . (4.1.11)
In Assumption 5.3, conditions 1 and 5 (or conditions 1 and 4) imply condition 2. Indeed, if condition 5 holds, the feasible set of (5.3.25) is compact and if condition 1 holds, the feasible set of (5.3.25) is nonempty. Moreover, conditions 2 and 5 (or conditions 2 and 4) imply condition 1. Indeed, if conditions 2 and 5 hold, the set of optimal solutions of (5.3.25) is nonempty and bounded. Then Trnovska's result [197, Corollary 1] yields condition 1.

Remark 4.2. If condition 5 of Assumption 4.1 holds, by adding a slack variable y and noting

Y = diag(X, y), we obtain an equivalent SDP of (4.1.8) as follows:

-τ = sup Y∈ Ŝ{ Ĉ, Y : Âi , Y = b i , Y 0 , trace(Y) = a} , (4.1.12)
where Ŝ = {diag(X, y) : X ∈ S , y ∈ R}, Ĉ = diag(C, 0) and Âi = diag(A i , 0). Obviously, SDP (4.1.12) has CTP.

SDP with Constant Trace Property (CTP)

Recall that λ 1 (A) stands for the largest eigenvalue of a real symmetric matrix A. Lemma 4.2. Let conditions 1 and 4 of Assumption 4.1 hold and let ϕ : R m → R be the function:

z → ϕ(z) := aλ 1 (C -A z) + b z . (4.1.13)
Then:

-τ = inf z { ϕ(z) : z ∈ R m } . (4.1.14)
Moreover if condition 3 of Assumption 4.1 holds, i.e., SDP (4.1.9) has an optimal solution then problem (4.1.14) has an optimal solution.

The proof of Lemma 4.2 is available in [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF]Appendix]. Next, we describe Algorithm 1 to solve SDP (4.1.8), which is based on nonsmooth first-order optimization methods (e.g., LMBM [START_REF] Haarala | Globally convergent limited memory bundle method for large-scale nonsmooth optimization[END_REF]Algorithm 1]). As shown later on in Section 4.3, this algorithm works well in almost all cases and with significantly lower computational cost when compared to the (currently fastest) SDP solver Mosek 9.1.

For X ∈ S, the Frobenius norm of X is defined by X F := X, X . We denote by A the operator norm of A, i.e., A := max X∈S AX 2 / X F .

Remark 4.3. Before running Algorithm 1, we scale the problem's input as follows: C

F = A = a = 1 and A 1 F = • • • = A m F .
Algorithm 1 SDP-CTP Input: SDP (4.1.8) with unknown optimal value and optimal solution; method (T) for solving convex nonsmooth unconstrained optimization problems (NSOP). Output: optimal value -τ and optimal solution X of SDP (4.1.8).

1: Compute the optimal value -τ and an optimal solution z of the NSOP (4.1.14) by using method (T); 2: Compute a normalized eigenvector u corresponding to λ 1 (C -A z) and set X = auu .

The fact that Algorithm 1 is well-defined under certain conditions is a corollary of Lemma 4.

g., (T) is LMBM). Then output -τ of Algorithm 1 is well-defined. Moreover, if condition 3 of Assumption 4.1 holds, the vector z mentioned at Step 1 of Algorithm 1 exists. In this case, if λ

1 (C -A z) has multiplicity 1, the output X of Algorithm 1 is well-defined.
When λ 1 (C-A z) has multiplicity larger than 1, one can obtain a dual matrix G corresponding to an optimal solution X of SDP (4.1.8) as in the following corollary:

U := C -A z , G := λ 1 (U)I -U , (4.1.15)
where I is the identity matrix. Then G is positive semidefinite and satisfies

G = A z -C , ( 4.1.16)
for some optimal solution z of (4.1.9).

Proof. It is not hard to prove that G 0. Let us prove the other statement. By using Farkas' lemma, there exists y ∈ R m such that A y = I and y b = a (see [START_REF] Helmberg | The spectral bundle method with second-order information[END_REF]Section 2]). Then

G = λ 1 (U)A y -C + A z = A (λ 1 (U)y + z) -C = A z -C, where z := λ 1 (U)y + z. Since b z = λ 1 (U)b y + b z = ϕ(z) = -τ , z
is an optimal solution of (4.1.9).

Largest eigenvalue computation:

Step 1 of Algorithm 1 requires the largest eigenvalue and corresponding eigenvectors of C -A z to evaluate the function ϕ (resp. ψ) and a subgradient of the subdifferential ∂ϕ (resp. ∂ψ) given in [134, Proposition A.1] (resp. [134, Proposition A.2]) at z. Fortunately, solving the eigenvalue problem for C -A z ∈ S can be done on every block of C -A z. Indeed, with X ∈ S as in (4.1.7),

λ(X) = λ(X 1 ) ∪ • • • ∪ λ(X l ) , where λ(A) is the set of all eigenvalues λ 1 (A) ≥ • • • ≥ λ t (A) for every real symmetric matrix A of size t. In particular, λ 1 (X) = max{λ 1 (X 1 ), . . . , λ 1 (X l )} . If u ∈ R s (j)
is an eigenvector of X j corresponding to the eigenvalue λ i (X j ) for some i ∈ [s (j) ] and j ∈ [l], by adding zero entries in u,

ū = (0 R s (1) +•••+s (j-1) , u, 0 R s (j+1) +•••+s (l) ) is an eigenvector of X = diag(X 1 , . . . , X l ) corresponding to λ i (X j ).
The interested reader can refer to [START_REF] Saad | Numerical methods for large eigenvalue problems: revised edition[END_REF][START_REF] Lehoucq | ARPACK users' guide: solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods[END_REF] to solve largest eigenvalue problems of symmetric matrices of large sizes.

-τ = inf{ aλ 1 ( Ĉ -Â z) + b z : z ∈ R m } , (4.1.17)
where

 z = m i=1 z i Âi . Note that Ĉ - z = diag(C -A z, 0). It implies that λ 1 ( Ĉ - z) = max{λ 1 (C -A z), 0}
. Thus, (4.1.17) can be rewritten as

-τ = inf{ a max{λ 1 (C -A z), 0} + b z : z ∈ R m } . (4.1.18)
In the next section, we consider the spectral formulation (4.1.18) introduced by Ding et al. in [START_REF] Ding | An optimal-storage approach to semidefinite programming using approximate complementarity[END_REF]Section 6].

SDP with Bounded Trace Property (BTP)

In the last subsection, we have seen that SDPs with CTP can be solved efficiently with first-order methods. Similar results can be obtained for the larger class of SDPs with the weaker bounded trace property (BTP). In particular the semidefinite relaxations of the Moment-SOS hierarchy associated with a POP on a compact semialgebraic set have the BTP. So in principle there is no need to add auxiliary "slack" variables to obtain an equivalent CTP-POP, as shown in Remark 4.2. However, numerical experiments of Section 4.3 suggest that the CTP is a highly desirable property that justifies addition of auxiliary variables.

The analogue of Lemma 4.2 for BTP reads: The proof of Lemma 4.3 can be found in [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF]Appendix].

Then -τ = inf z { ψ(z) : z ∈ R m } . ( 4
We next describe Algorithm 2 to solve SDP (4.1.8). As Algorithm 1, it is also based on nonsmooth optimization methods such as LMBM.

Algorithm 2 SDP-BTP Input: SDP (4.1.8) with unknown optimal value and optimal solution; method (T) for solving convex NSOP. Output: optimal value -τ and optimal solution X of SDP (4.1.8).

1: Compute the optimal value -τ and an optimal solution z of NSOP (4.1.14) by using method (T); 2: Compute λ 1 (C -A z) and a corresponding normalized eigenvector u;

3: Let ξ > 0 such that ξ =      0 if λ 1 (C -A z) < 0 , ζa if λ 1 (C -A z) = 0 , a otherwise , (4.1.21) for some ζ ∈ [0, 1] such that X = ξuu satisfies AX = b.
The next result is a consequence of Lemma 4.

Application to polynomial optimization

We consider the following POP:

f := inf{f (x) : x ∈ S(g, h)} , (4.2.1)
where S(g, h) is defined as in (4.1.2) with m (resp. l) being the number of inequality (resp. equality) constraints. Assume that S(g, h) ⊂ B(0, R).

Remark 4.5. By setting X := diag(M k (y), M k-g1 (g 1 y), . . . , M k-gm (g m y)) and using the upper bound trace(X) ≤ āk with In order to overcome the accuracy issue mentioned in Remark 4.5, we convert every POP to a CTP-POP (i.e., a new POP formulation with CTP) by adding slack variables associated with inequality constraints. In the sequel, we consider three particular cases: equality constrained POPs on a sphere in Section 4.2.1, constrained POPs with single inequality (ball) constraint in Section 4.2.2, and constrained POPs on a ball in Section 4.2.3.

āk := R k b(n, k) + m i=1 g i 1 b(n, k -g i ) , ( 4

Equality constrained POPs on a sphere

Assume that m = 0 and

h 1 = R -x 2 2 . Note that x 2 2 = x 2 1 + • • • + x 2
n is a quadratic polynomial. In this case, we consider equality constrained POPs on a sphere, presented in Section 4.1.2. We propose to reduce SDP (4.1.6) to an NSOP. For each k ∈ N, let (θ k,α ) α∈N n k be a sequence of positive real numbers such that

(1 + x 2 2 ) k = α∈N n k θ k,α x 2α ,
and define the diagonal matrix

P n,k := diag((θ 1/2 k,α ) α∈N n k ) . (4.2.3)
For every k ∈ N, since P n,k 0, SDP (4.1.6) is equivalent to SDP:

τ k = inf y∈R b(n,2k) L y (f ) s.t. y 0 = 1 ; P n,k M k (y)P n,k 0 , M k-hj (h j y) = 0 , j ∈ [l] . (4.2.4)
For every k ∈ N, let a k := ( R + 1) k . We will use the following lemma:

Lemma 4.4. For all k ∈ N, M k-1 (( R -x 2 2 ) y) = 0 , y 0 = 1 ⇒ trace(P n,k M k (y)P n,k ) = a k . Proof. Let k ∈ N be fixed. From M k-1 (( R -x 2 2 ) y) = 0, L y (p( R -x 2 2 )) = 0, for every p ∈ R[x] 2(k-1) . For every r ∈ N ≤k-1 , by choosing p = x 2r 2 , L y ( x 2(r+1) 2 ) = -L y ( x 2r 2 ( R -x 2 2 )) + RL y ( x 2r 2 ) = RL y ( x 2r 2 ) . By induction, L y ( x 2r 2 ) = RL y ( x 2(r-1) 2 ) = • • • = Rk L y ( x 2×0 2 ) = Rk y 0 = Rr , for every r ∈ N ≤k . Thus, trace(P n,k M k (y)P n,k ) = α∈N n k θ 1/2 k,α y 2α θ 1/2 k,α = L y   α∈N n k θ k,α x 2α   = L y ((1 + x 2 2 ) k ) = L y k r=0 k r x 2r 2 = k r=0 k r L y ( x 2r 2 ) = k r=0 k r Rr = ( R + 1) k .
For each k ∈ N, we denote by S (k) the set of symmetric matrices of size b(k) and let A, B = trace(B A) be the usual scalar product on S (k) . For every k ∈ N, letting

X = P n,k M k (y)P n,k , (4.2.5) (4.2.4
) can be written in the form:

-τ k = sup X∈S (k) { C k , X : A k X = b k , X 0} , (4.2.6)
where

A k : S (k) → R m k is a linear operator of the form A k X = [ A k,1 , X , . . . , A k,m k , X ] , with A k,i ∈ S (k) , i ∈ [m k ], C k ∈ S (k) is the cost matrix and b k ∈ R m k is the right-hand-side vector. [134, Appendix A.2]
describes how to reduce SDP (4.2.4) to the form (4.2.6).

For every k ∈ N, the dual of SDP (4.2.6) reads:

-ρ k = inf z { b k z : A k z -C k 0 , } (4.2.7)
where

A k : R m k → S (k) is the adjoint operator of A k , i.e., A k z = m k i=1 z i A k,i . From Lemma 4.4 and since h 1 = R -x 2 2 , it implies that for every k ∈ N, ∀ X ∈ S (k) , A k X = b k ⇒ trace(X) = a k . (4.2.8)
We guarantee zero duality gap, primal attainability, and dual attainability for primal-dual (4.2.6)-(4.2.7) in the following proposition: Proposition 4.1. Let f be as in (4.1.3). Then:

1. Zero duality gap holds for primal-dual (4.2.6)-(4.2.7) for large enough k ∈ N.

SDP (4.2.6

) has an optimal solution for large enough k ∈ N.

Assume that one of the following two conditions holds: (a) I(h) is real radical and the second-order sufficiency conditions (Definition 2.1) hold at

every global minimizer of (4.1.3);

(b) V (h) is finite.
Then SDP (4.2.7) has an optimal solution for large enough k ∈ N. In this case, We illustrate the conversion from SDP (4.1.6) to SDP (4.2.6) in the following example.

τ k = ρ k = f . Proof. Since (4.

By replacing (A

k , A k,i , b k , C k , S (k) , b(k), m k , τ k , ρ k , a k ) by (A, A i , b, C, S, s, m, τ, ρ, a), primal- dual (4.
-1 = inf{x : 1 -x 2 = 0} .
Then the second order moment relaxation (k = 2) has the form:

τ 2 = inf y y 1 s.t.   y 0 y 1 y 2 y 1 y 2 y 3 y 2 y 3 y 4   0 , y 0 -y 2 y 1 -y 3 y 1 -y 3 y 2 -y 4 = 0 , y 0 = 1 .
It can be rewritten as

τ 2 = inf y y 1 s.t.   1 y 1 1 y 1 1 y 1 1 y 1 1   0 ,
by removing equality constraints. Obviously, the positive semidefinite matrix of this form has trace 3. In a different way, according to [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF]Appendix A.2], let us note

X =   1 0 0 0 √ 2 0 0 0 1     y 0 y 1 y 2 y 1 y 2 y 3 y 2 y 3 y 4     1 0 0 0 √ 2 0 0 0 1   , to obtain -τ 2 = sup X∈S2 { C, X : A i , X = b i , i ∈ [5] , X 0} , where b 1 = • • • = b 4 = 0, b 5 = 1 and C = - √ 2 4   0 1 0 1 0 0 0 0 0   , A 1 = √ 2 2   0 0 1 0 -1 0 1 0 0   , A 2 = 1 2   2 0 -1 0 0 0 -1 0 0   , A 3 = √ 2 4   0 1 0 1 0 -1 0 -1 0   , A 4 = 1 2   0 0 1 0 0 0 1 0 -2   , A 5 =   1 0 0 0 0 0 0 0 0   .
Remark that for any X ∈ S (2) ,

( A i , X = b i , i ∈ [5]) ⇒ trace(X) = 4 .
Next, we present an alternative iterative method, stated in Algorithm 3, to solve (4.1.3), based on nonsmooth optimization methods, e.g., LMBM. It performs well in practice for most cases and with significantly lower computational cost when compared to the (currently fastest) SDP solver Mosek 9.1.

Algorithm 3 SpectralPOP-CTP

Input: POP (4.1.3) with unknown optimal value f and optimal solutions; method (D) for solving SDP with CTP. Output: increasing real sequence (τ k ) k∈N and x ∈ R n .

1: for k ∈ N do 2:
Compute the optimal value -τ k and an optimal solution X of SDP (4.2.6) by using method (D);

3: Set M k (y ) := P -1
n,k X P -1 n,k (relying on (4.2.5)) and extract an atom x by using Henrion-Lasserre's algorithm in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] from M k (y );

4:

If x exists, set τ k+j = τ k , j ∈ N >0 , and terminate.

Note that one can choose method (D) in Algorithm 3 as Algorithm 1 with LMBM solver or SketchyCGAL.

Remark 4.6. In practice, to verify that an atom x extracted in Step 3 of Algorithm 3 is an approximate optimal solution of POP (4.1.3), with given ε ∈ (0, 1), we check the following inequalities:

|f (x ) -τ k | ≤ ε f max and |h j (x )| ≤ ε h j max , j ∈ [m] ,
where In Corollary 4.4, the flat extension condition implies that the SOS problem (4.1.5) has an optimal solution (due to [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF]Theorem 3.4 (b)] and τ k = ρ k ), so that SDP (4.2.7) has an optimal solution. In this case, X exists, which in turn implies the existence of x . In Step 4 of Algorithm 3, if the atom x exists, then we do not need to increase the relaxation order k.

It is due to the fact that f = τ k ≤ τ k+1 ≤ • • • ≤ f .

Remark 4.7. When Algorithm 1 with LMBM solver is used for method (D) in Algorithm 3, we have the following cases:

1. If the SDP relaxation (4.1.6) is exact then the value is f and one indeed may expect that generically the moment matrix is rank-one, which will yield X = uu for some u. Thus, X in the first step of Algorithm 3 is well-defined.

2. If the SDP relaxation (4.1.6) is not exact then we only use the relaxation value as a (supposedly accurate) lower bound on the global minimum f .

Remark 4.8. In practice, we use the following heuristic extraction algorithm when method (D) in Algorithm 3 is Algorithm 1 with LMBM solver:

1. Obtain a dual matrix G corresponding to an optimal solution X of SDP (4.2.6) based on Corollary 4.2;

2. Set Ḡ := P n,k GP n,k ;

3. Obtain an atom x by using the extraction algorithm of Henrion and Lasserre in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF], where the matrix V in [77, (6)] is taken such that the columns of V form a basis of the null space {u ∈ R b(k) : Ḡu = 0};

4. Verify that x is an approximate optimal solution of POP (4.1.3) as in Remark 4. 6.

This heuristic extraction algorithm performs practically well when the moment matrices are not rank-one. Note that Ḡ obtained in Step 2 is a Gram matrix corresponding to some moment matrix M k (y ).

Step 3 is well-defined when the complementary slackness, i.e., Ḡ M k (y ) = 0, and the strict complementarity, i.e., rank Ḡ + rankM k (y ) = b(k), hold (see [START_REF] Ding | An optimal-storage approach to semidefinite programming using approximate complementarity[END_REF]Section 1.3]). In this case, the range of M k (y ), which is the linear span of the columns of V in [77, (6)], is identical with the null space of Ḡ.

In the two following subsections, we consider POPs on general compact sets as stated in Section 4.1.1.

Constrained POPs with a single inequality (ball) constraint

Assume that m = 1 and g 1 = R -x 2 2 . In this case, g = {R -x 2 2 }. Let us show that POP (4.1.1) can be reduced to an equality constrained POP on a sphere. By adding one slack variable x n+1 , the inequality constraint R -x 2 2 ≥ 0 can be rewritten as an equality constraint R -x 2 2 -x 2 i+n = 0 and so

f := inf{ f (x) : (x, x n+1 ) ∈ V ( h)} , (4.2.9) where h := h ∪ {R -x 2 2 -x 2 n+1 } ⊂ R[x, x n+1 ]. Notice that: • If x = (x , x n+1
) is an optimal solution of POP (4.2.9), x is an optimal solution of POP (4.1.1).

• Conversely, if x is an optimal solution of POP (4.1.1), then x := x , R -x 2 2 is an optimal solution of POP (4.2.9).

Let us define n := n + 1 and x := (x, x n+1 ) to ease notation. For every k ∈ N, consider the order k moment relaxation of (4.2.9):

τk = inf y∈R b( n,2k) L y (f ) s.t. y 0 = 1 , M k (y) 0 , M k-1 ((R -x 2 2 ) y) = 0 , M k-hj (h j y) = 0 , j ∈ [l] . (4.2.10)
The corresponding dual SOS problem indexed by k ∈ N reads:

ρk := sup { ξ ∈ R : f -ξ ∈ P k ( h)} , (4.2.11)
where P k ( h) is the truncated preodering of all polynomials of the form

σ 0 + ψ 0 (R -x 2 2 ) + l j=1 ψ j h j , with σ 0 ∈ Σ[x] k , ψ 0 ∈ R[x] 2(k-1)
, and

ψ j ∈ R[x] 2(k-hj ) , j ∈ [l].
The following lemma will be used later on:

Lemma 4.5. If f -f ∈ Q k (g, h) for some k ∈ N then f -f ∈ P k ( h).
Proof. By assumption, there exist Compute the optimal value -τ k and an optimal solution y of SDP (4.2.10) with CTP (4.2.12) by using method (D);

σ 0 ∈ Σ[x] k , σ 1 ∈ Σ[x] k-1 , and ψ j ∈ R[x] 2(k-hj ) , j ∈ [l] such that f -f = σ 0 + σ 1 (R -x 2 2 ) + l j=1 ψ j h j = σ 0 + σ 1 x 2 n+1 + σ 1 (R -x 2 2 ) + l j=1 ψ j h j ,

3:

Extract an atom x = (x , x n+1 ) by using Henrion-Lasserre's algorithm in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] from M k (y );

Constrained POPs on a ball

Assume that m > 1 and

g 1 = R -x 2 2 .
Let us show that POP (4.1.1) can be reduced to an equality constrained POP on a sphere. After adding m slack variables x n+i , i ∈ [m], every inequality constraint g i (x) ≥ 0 can be rewritten as an equality constraint g i (x) = x 2 i+n and so

f := inf{ f (x) : (x, x n+1 , . . . , x n+m ) ∈ V ( ĥ)} ,
where ĥ :

= h ∪ {g i -x 2 i+n : i ∈ [m]} ⊂ R[x, x n+1 , . . . , x n+m ]. Let us take upper bounds b i ≥ sup{g i (x) : x ∈ S({g 1 }, h)}, i ∈ [m]. For every i ∈ [m]
, the bound b i can be computed by solving the order k moment relaxation:

-b i = inf y∈R b(n+1,2k) L y (-g i ) s.t. y 0 = 1 , M k (y) 0 , M k-1 ((R -(x, x n+1 ) 2 2 ) y) = 0 , M k-hj (h j y) = 0 , j ∈ [l] , (4.2.13)
based on the spectral minimization method presented in the previous section.

For every (x, x n+1 , . . . , x n+m ) ∈ V ( ĥ), x ∈ S(g, h) and

x 2 n+i = g i (x) ≤ b i , i ∈ [m], since S(g, h) ⊂ S({g 1 }, h). Therefore x 2 2 + m i=1 x 2 n+i ≤ R with R := R + m i=1 b i . (4.2.14)
Equivalently V ( ĥ) ⊂ B n+m R and after adding one more slack variable x n+m+1 :

f := inf{f (x) : x ∈ V ( h)} , (4.2.15)
where x := (x, x n+1 , . . . , x n+m+1 ) and

h := ĥ ∪ { R -x 2 2 } = h ∪ {g i -x 2 i+n : i ∈ [m]} ∪ { R -x 2 2 } ⊂ R[x] .
Notice that:

• If x = (x , x n+1 , . . . , x n+m+1
) is an optimal solution of POP (4.2.15), x is an optimal solution of POP (4.1.1).

• Conversely, if x is an optimal solution of POP (4.1.1), then

x :=   x , g 1 (x ), . . . , g m (x ), R - m i=1 g i (x ) -x 2 2  
is an optimal solution of POP (4.2.15).

Note n := n + m + 1 for simplicity. For every k ∈ N, consider the order k moment relaxation of (4.2.15): τk = inf

y∈R b( n,2k) L y (f ) s.t. y 0 = 1 , M k (y) 0 , M k-gi ((g i -x 2 n+i ) y) = 0 , i ∈ [m] , M k-1 (( R -x 2 2 ) y) = 0 , M k-hj (h j y) = 0 , j ∈ [l] . (4.2.16)
The corresponding dual SOS problem indexed by k ∈ N reads:

ρk := sup { ξ ∈ R : f -ξ ∈ P k ( h)} , ( 4.2.17)
where P k ( h) is the truncated preodering of all polynomials of the form

σ 0 + m i=1 ψ i (g i -x 2 n+i ) + ψ m+1 ( R -x 2 2 ) + l j=1 ψ m+1+j h j with σ 0 ∈ Σ[x] k , ψ i ∈ R[x] 2(k-gi ) , i ∈ [m], ψ m+1 ∈ R[x] 2(k-1)
, and

ψ m+1+j ∈ R[x] 2(k-hj ) , j ∈ [l].
We will use the following lemma later on:

Lemma 4.6. If f -f ∈ Q k (g, h) for some k ∈ N then f -f ∈ P k ( h).
Proof. By assumption, there exist

σ 0 ∈ Σ[x] k , σ i ∈ Σ[x] k-gi , i ∈ [m],
and

ψ j ∈ R[x] 2(k-hj ) , j ∈ [l] such that f -f = σ 0 + m i=1 σ i g i + l j=1 ψ j h j .
It implies that

f -f = σ 0 + m i=1 σ i x 2 i+n + m i=1 σ i (g i -x 2 i+n ) + 0 × ( R -x 2 2 ) + l j=1 ψ j h j ,

SDP (4.2.16

) has an optimal solution for large enough k ∈ N.

Assume one of the following two conditions holds:

(a) Q(g, h) is Archimedean, the ideal I(h) is real radical, and the second-order sufficiency conditions (Definition 2.1) hold at every global minimizer of POP (4.1.1);

(b) V (h) is finite.
Then SDP (4.2.17) has an optimal solution for large enough k ∈ N. In this case, τk = ρk = f .

Proof. The first and second statement follow from to Proposition 2.1 after replacing S(g, h) by V ( h). The third statement is due to Proposition 2.4 and Lemma 4.6.

For every k ∈ N, according to Lemma 4.4, if

M k-1 (( R -x 2 2 ) y) = 0 and y 0 = 1, trace(P n,k M k (y)P n,k ) = ( R + 1) k , ( 4
= R -x 2 2
, unknown optimal value f and optimal solutions; method (D) for solving SDP with CTP. Output: increasing real sequence (τ k ) k∈N and x ∈ R n .

1: for k ∈ N do 2:
Compute the optimal value b i of SDP (4.2.13) with CTP, i ∈ [m], by using method (D) and set R :

= R + m i=1 b i ; 3:
Compute the optimal value -τ k and an optimal solution y of SDP (4.2.16) with CTP (4.2.18) by using method (D);

4:

Extract an atom x = (x , x n+1 , . . . , x n+m+1 ) by using Henrion-Lasserre's algorithm in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] from M k (y );

Numerical experiments

Let us report numerical results obtained while relying on algorithms from Section 4.2 to solve equality constrained QCQPs on a sphere, quartic minimization problems on the unit sphere as well as further applications to three well-known NP-hard optimization problems on the unit sphere : deciding nonnegativity/convexity of even degree forms and copositivity of real symmetric matrices.

The experiments are performed in Julia 1.3.1 with the following packages:

• SumOfSquare.jl [START_REF] Weisser | Polynomial and Moment Optimization in Julia and JuMP[END_REF] is a modeling library to write and solve SDP relaxations of POPs, based on JuMP.jl and the SDP solver Mosek 9.1.

• LMBM.jl solves unconstrained NSOPs with the limited-memory bundle method of Haarala et al. [START_REF] Haarala | Globally convergent limited memory bundle method for large-scale nonsmooth optimization[END_REF][START_REF] Haarala | New limited memory bundle method for largescale nonsmooth optimization[END_REF]. LMBM.jl calls Karmitsa's Fortran implementation of LMBM algorithm [START_REF] Karmitsa | LMBM-FORTRAN subroutines for Large-Scale nonsmooth minimization: User's manual[END_REF]. We also use the package Arpack.jl, which is based on the implicitly restarted Lanczos's algorithm, to compute the largest eigenvalues and the corresponding eigenvectors of real symmetric matrices of (potentially) large size. When POPs have equality constraints, SumOfSquare.jl uses reduced forms with Groebner basis instead of creating SOS multipliers, in order to reduce solving time.

The implementation of algorithms described in Section 4.2 can be downloaded from the link: https://github.com/maihoanganh/SpectralSOS.

We use a desktop computer with an Intel(R) Core(TM) i7-8665U CPU @ 1.9GHz × 8 and 31.2 GB of RAM. The notation for our numerical results are given in Table 4.1.

Random dense equality constrained QCQPs on the unit sphere

Test problems: We construct several instances of POP (4.1.3) as follows:

1. Take h 1 = 1 -x 2 2 and choose f , h j , j ∈ [l]\{1} with degrees at most 2;

2. Each coefficient of the objective function f is taken randomly in (-1, 1) with respect to the uniform distribution;

3. Select a random point a ∈ R n in the unit sphere;

4. For every j ∈ [l]\{1}, all non-constant coefficients of h j are taken randomly in (-1, 1) with respect to the uniform distribution, and the constant coefficient of h j is chosen such that h j (a) = 0.

Note that by construction, a is a feasible solution. We use the method presented in Section 4.2.1 (actually the k-th iteration of Algorithm 3) to solve these problems. Numerical results are displayed

Efficiency comparison:

In Table 4.2, we minimize quadratic polynomials on the unit sphere. The SDP relaxation for a POP in n variables involves a matrix of size n + 1 and 2 trace equality constraints. In this table, LMBM is the fastest SDP solver while Mosek (the SDP solver used by SumOfSquares) is the slowest. It is due to the fact that Mosek relies on interior-point methods based on second order conditions to solve SDP while LMBM and SketchyCGAL only rely on algorithms based on first order conditions. Note that we use the same modeling technique to generate the SDP-CTP relaxation solved with either SketchyCGAL or LMBM, so both related modeling times are the same. The solving time of SketchyCGAL is a bit larger than the one of LMBM in this case.

In Table 4.3 and Table 4.4, we consider random equality constrained QCQPs and solve their first (k = 1) and second (k = 2) order moment relaxation, respectively. In Table 4.3, the size of the positive semidefinite matrix (resp. the number of trace equality constraints) involved in the SDP relaxation is equal to n + 1 (resp. l + 1). In Table 4.4, the matrices involved in the SDP relaxation have size b(n, 4) and the number of trace equality constraints is O(b(n, 4) 2 ), due to [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF]Appendix A.2]. Thus, the number of trace equality constraints for these SDP relaxations [START_REF] Dahl | Semidefinite optimization using mosek[END_REF] for more details). The linear system becomes harder to solve when the number of trace equality constraints is larger. This is in contrast with LMBM, which does not need to solve any such large size linear system of equations. By comparison with LMBM, SketchyCGAL may perform a larger number of operations [219, Algorithm 6.1], as emphasized later on.

Accuracy comparison:

When n ≤ 300 in Table 4.2, n ≤ 300 in Table 4.3 or n ≤ 30 in Table 4.4, LMBM converges to the exact optimal value of POPs with high accuracy, similarly to SumOfSquares. Both LMBM and SumOfSquares can extract at least one approximate optimal solution by Henrion-Lasserre's algorithm [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF], when n ≤ 300 in Table 4

Storage and evaluation comparisons:

In Table 4.5 and 4.6, we display some additional information related to Mosek, LMBM and SketchyCGAL, for the rows n = 5, 10, 15, 20, 25 of Table 4.4:

• storage;

• #A: the number of evaluations of the linear operator A in SDP (4.1.8);

• #A : the number of evaluations of the adjoint operator A ;

• s max : the largest size of symmetric matrices of which eigenvalues and eigenvectors are computed;

• N eig : the number of symmetric matrices of which eigenvalues and eigenvectors are computed.

Table 4.5 indicates that SumOfSquares requires a bit lower storage than LMBM only for the cases n = 5, 10. However, SketchyCGAL requires a bit smaller storage than LMBM. Note that SketchyCGAL performs a large number of evaluations of both A and A while relying on three specific primitive computations (see [START_REF] Yurtsever | Scalable semidefinite programming[END_REF]Section 2.3]). Compared to SketchyCGAL, LMBM performs a smaller number of evaluations of both A and A . For instance, the number of evaluations of LMBM is ten times smaller than the one of SketchyCGAL for the row n = 25 of Table 4.6. Because of the large number m of trace equality constraints, the evaluations of A and A in SDP relaxations of POPs is more expensive than the simple one related to the first order SDP relaxation of MAXCUT, which is solved very efficiently by SketchyCGAL (see [START_REF] Yurtsever | Scalable semidefinite programming[END_REF]Section 2.5]). These specific behaviors mainly come from the subroutines used by LMBM and SketchyCGAL to compute eigenvalues and eigenvectors. While LMBM computes directly the largest eigenvector (and corresponding eigenvalue) of the matrix C-A z involved in the nonsmooth function from [219, (2.4)] for more details), which yields a large number of evaluations of A . For instance, #A = 6134 for SketchyCGAL when n = 25 while the value #A is 337 for LMBM. Because of its slow convergence, SketchyCGAL also performs a larger number of iterations in Step 6 of [START_REF] Yurtsever | Scalable semidefinite programming[END_REF]Algorithm 6.1].

Based on the above comparison, we emphasize that LMBM is cheaper and faster than Mosek or SketchyCGAL while LMBM ensures the same accuracy as Mosek when solving SDP relaxations of equality constrained QCQPs on the unit sphere.

Random dense QCQPs on the unit ball

Test problems: We construct several samples of POP (4.1.1) as follows:

1. Take g 1 = 1 -x 2 2 and choose f , g i , i ∈ [m]\{1}, and h j , j ∈ [l] with degrees at most 2;

2. Each coefficient of the objective function f is taken randomly in (-1, 1) with respect to the uniform distribution;

3. Select a random point a ∈ R n in the unit ball, with respect to the uniform distribution;

4. For each i ∈ [m]\{1}, all non-constant coefficients of g i are taken randomly in (-1, 1) with respect to the uniform distribution, and the constant coefficient of g i is chosen such that g i (a) > 0;

5. For j ∈ [l], all non-constant coefficients of h j are taken randomly in (-1, 1) with respect to the uniform distribution, and the constant coefficient of h j is chosen such that h j (a) = 0.

Numerical results are displayed in Table 4.7 for the case (m, l) = (1, n/4 ) and Table 4.8 for the case (m, l) = ( n/8 , n/8 ). We recall the following notation:

• CTP (LMBM): the SDP relaxation is solved via the method described in Section 4.2.2 (the k-th iteration of Algorithm 4) or Section 4.2.3 (the k-th iteration of Algorithm 5) with the LMBM solver.

• BTP (LMBM): the SDP relaxation is solved via the method described in Remark 4.5 with the LMBM solver (Algorithm 2).

In Table 4.7 and Table 4.8, SumOfSquares and BTP solve relaxations involving matrices with the same size, corresponding exactly to the size of the moment relaxation (2.4.5). 4.8.

Efficiency and accuracy comparisons: In

the one obtained with SumOfSquares (Mosek). Note that when we use the method described in Section 4.2.2, the constant trace in (4.2.12) is always equal to 2 k , which is independent of n.

In Table 4.8, CTP (LMBM) provides inaccurate output as it only yields lower bounds, while SumOfSquares still preserves accuracy. Moreover, CTP (LMBM) is less efficient than SumOfSquares. We also emphasize that when one relies on the method stated in Section 4.2.3, we obtain a value of R, in (4.2.14), for the sphere constraint of CTP-POP, which becomes larger when n increases. It implies that the constant trace factor ( R + 1) k in (4.2.18) has a polynomial growth rate in R. Thus we minimize a nonsmooth function of the form (4.1.13) with a large constant trace factor a. The norm of the subgradient of this function at a point near its minimizers is rather large, which prevents LMBM to perform properly its minimization, by contrast with Table 4.7. This difference of magnitude is shown in Table 4.9, where we compute the subgradient norms during the last 10 iterations of CTP (LMBM) for the experiments from Table 4.7 and Table 4.8 with n = 10.

In both Table 4.7 and Table 4.8, BTP (LMBM) has the worst performance in terms on efficiency and accuracy. The trace bound (4.2.2) obtained in Remark 4.5 is usually much larger than the "exact" trace of the optimal solution of the SDP relaxation. The same issue occurs for the subgradient norm of the nonsmooth function at a point near its minimizers.

According to our experience, LMBM is suitable for spectral minimization of SDP problems with trace bounds which are small enough and close to the exact trace value of the optimal solution.

Random dense quartics on the unit sphere

Test problems: We construct several instances of POP (4.1.3) as follows:

1. Take l = 1 and h 1 = 1 -x 2 2 and choose f with degree at most 4;

2. Each coefficient of the objective function f is taken randomly in (-1, 1) with respect to the uniform distribution.

We use the method presented in Section 4.2.1 to solve these problems. The corresponding numerical results are displayed in Table 4.10.

Efficiency and accuracy comparisons: Table 4.10 indicates that LMBM is about twice faster than SumOfSquares when n ≥ 20 as well as SketchyCGAL.While SketchyCGAL can be rather inaccurate, LMBM has an accuracy which is similar to SumOfSquares (Mosek), yielding the ability to extract optimal solutions of POPs. For comparisons in Section 4.3.2 and Section 4.3.3, the coefficients of f have been randomly generated in (-1,1). However, for some non random problems that were scaled so as to fit the framework of optimization on the unit sphere, we could observe a lack of precision after transferring results (of the scaled formulation) back to results in the unscaled initial formulation.

In the next three subsections, we consider further applications of the minimization of forms on the unit sphere listed in [START_REF] Lasserre | Homogeneous polynomials and spurious local minima on the unit sphere[END_REF].

Deciding the nonnegativity of even degree forms

Given q ∈ R[x], we recall that q is a form of degree d if q = |α|=d q α x α for some d ∈ N and q α ∈ R. Given a form q ∈ R[x], q is nonnegative on R n iff q is nonnegative on the unit sphere. Moreover, given a polynomial

f ∈ R[x] 2d , f is nonnegative on R n iff its homogenization f := x 2d n+1 f ( x xn+1
) is nonnegative on R n+1 . Note that f is a form. Thus, in order to verify the nonnegativity of the polynomial f , we only verify the nonnegativity of its homogenization f on the unit sphere in R n+1 . Namely, given a form f ∈ R[x] of degree 2d, we consider the following POP:

f := min x∈R n {f (x) : x 2 2 = 1} . (4.3.1)
Note that if d = 1, problem (4.3.1) boils down to computing the minimal eigenvalue of the Gram matrix associated to f . Thus, we consider the case where d ≥ 2.

Test problems:

We construct several instances of the form f of degree 2d as follows:

1. Take f α randomly in (-1, 1) with respect to the uniform distribution, for each α ∈ N n with |α| = 2d.

Set

f := |α|=2d f α x α .
We use the method presented in Section 4.2.1 to solve problem (4.3.1). The corresponding numerical results are displayed in Table 4.11.

Efficiency and accuracy comparisons: Table 4.11 shows that LMBM is much faster than Mosek when n ≥ 20 and d = 2. In these cases, we were able to extract the solution of the resulting POPs. One can then conclude that f is not globally nonnegative since it has negative value at its atoms. For higher values of d = 3, 4, LMBM becomes less efficient and accurate than Mosek.

Deciding the convexity of even degree forms

Recall that a polynomial q

∈ R[x] is convex if q(tx+(1-t)y) ≤ tq(x)+(1-t)q(y) for all x, y ∈ R n and t ∈ [0, 1]. Moreover, q ∈ R[x] is convex iff the polynomial f (x, y) := q(y)-q(x)-∇q(x) (y-x)
is globally nonnegative, where ∇q stands for the gradient of q. If q is a form of degree d, f is also a form of degree d. In this case, the nonnegativity of f can be verified on the unit sphere in R 2n .

Given a form q ∈ R[x] of degree 2d, consider the following POP:

f := min x,y∈R n q(y) -q(x) -∇q(x) (y -x) s.t. (x, y) 2 2 = 1 . (4.3.2)
From the previous discussion, q is convex iff f ≥ 0. Test problems: We construct several instances of the form q of degree 2d as follows:

1. Take q α randomly in (-1, 1) with respect to the uniform distribution, for each α ∈ N n with |α| = 2d.

2. Set q := |α|=2d q α x α .

We use the method presented in Section 4.2.1 to solve problem (4.3.2). The corresponding numerical results are displayed in Table 4.12.

Efficiency and accuracy comparisons: Table 4.12 shows that LMBM is about 3 times faster than Mosek when n ≥ 10 and d = 2. In these cases, one concludes as above that q is nonconvex since f is negative. For d = 3 and n = 5, LMBM still returns a value with reasonably high accuracy (10 -5 ) even though it is slower than Mosek. In this case, one can also certify that q is nonconvex.

Deciding the copositivity of real symmetric matrices

Given a symmetric matrix A ∈ R n×n , we say that A is copositive if u Au ≥ 0 for all u ∈ R n + . Consider the following POP:

f := min x∈R n {(x 2 ) Ax 2 : x 2 2 = 1} , (4.3.3)
where

x 2 := (x 2 1 , . . . , x 2 n ). The matrix A is copositive iff f ≥ 0.
Test problems: We construct several instances of the matrix A as follows:

1. Take B ij randomly in (-1, 1) with respect to the uniform distribution, for all i, j ∈ [n].

2. Set B := (B ij ) 1≤i,j≤n and A := 1 2 (B + B ). We use the method presented in Section 4.2.1 to solve problem (4.3.3). The corresponding numerical results are displayed in Table 4. [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF].

Efficiency and accuracy comparisons:

Exploiting the constant trace property: Inequality constraints

Most of the content of this chapter is from [START_REF] Mai | Exploiting constant trace property in large-scale polynomial optimization[END_REF]. The goal of this chapter is to extend the CTP-exploiting framework introduced in the previous chapter to the case of POPs with inequality constraints. Based on this, we provide a method which returns the optimal value of the second order moment SDP-relaxation and which is suitable for a class of large-scale non-convex QCQPs with CTP. Ideally (i) it should avoid memory issues, and (ii) the resulting relative gap of the approximate value returned by this method w.r.t. the exact value should be less than 1%.

Contribution. We show that (i) a large class of POPs with inequality constraints have the constant trace property and (ii) this property can be exploited for solving their associated semidefinite relaxations via appropriate first-order methods. More precisely our contribution is threefold: I. In Section 5.1.2 we show that if a positive real number belongs to the interior of every truncated quadratic module associated with the inequality constraints, which is defined later in (2.3.4), then the corresponding POP has CTP. Moreover, we prove that this condition always holds when a ball constraint is present.

II. In Section 5.1.3 we provide a numerical procedure to check whether a POP has CTP. It consists in solving a certain linear program (LP) of the form: For practical implementation we provide a software library called ctpPOP. It models each moment SDP-relaxation of POPs as a standard SDP with CTP and then solves this SDP by CGAL or a spectral method (SM), based on nonsmooth optimization solvers (LMBM [START_REF] Karmitsa | LMBM-FORTRAN subroutines for Large-Scale nonsmooth minimization: User's manual[END_REF] or the Proximal Bundle Method [START_REF] Kiwiel | A tilted cutting plane proximal bundle method for convex nondifferentiable optimization[END_REF]).

In Section 5.2 and Appendix 5.3.3 we provide extensive numerical experiments to illustrate the efficiency and scalability of ctpPOP with the CGAL solver. In all our randomly generated POPs with different sparsity structures, the relative gap of the optimal value provided by CGAL w.r.t. the optimal value provided by Mosek is below 1%. Because of its cheap cost per iteration, CGAL is more suitable for particularly SDPs of form (1.4.1) with ζ = O((s max ) 2 ) (such as the second order moment SDP-relaxations of POPs) than other solvers (e.g., COSMO).

For instance for minimizing a dense quadratic polynomial on the unit ball with 100 variables, CGAL returns the optimal value of the second order moment SDP relaxation within 6 hours on a standard laptop while Mosek (considered as the state-of-the-art SDP solver using interior-point methods) runs out of memory. Similarly, for minimizing a sparse quadratic polynomial involving a thousand variables with a ball constraint on each clique of variables, CGAL spends around two thousand seconds to solve the second order moment SDP-relaxation while Mosek runs again out of memory. The largest clique of this POP involves 42 variables.

The classical OPF problem without constraints on current magnitudes (as in [START_REF] Josz | AC power flow data in MATPOWER and QCQP format: iTesla, RTE snapshots, and PEGASE[END_REF][START_REF] Godard | Novel approach towards global optimality of optimal power flow using quadratic convex optimization[END_REF]) can be formulated as a POP with ball and annulus constraints. For many instances Shor's relaxation provides the global optimum. However, for illustration purposes we have compared CGAL and Mosek for solving the second order CS-TSSOS relaxation from Chapter 3 of the "case89 pegase api" instance from the PGLib-OPF database 1 . The largest block size and the number of equality constraints of this SDP are around 1.7 thousands and 8 millions, respectively. While Mosek failed because of a memory issue, CGAL still returned the optimal value in two days, and with relative gap w.r.t. a local optimal value being less than 0.6%.

As in the previous chapter, a POP is defined as

f := inf{f (x) : x ∈ S(g) ∩ V (h)} , (5.0.2)
where S(g) and V (h) are a basic semialgebraic set and a real variety defined respectively by

S(g) := { x ∈ R n : g i (x) ≥ 0 , i ∈ [m] } V (h) := { x ∈ R n : h j (x) = 0 , i ∈ [l] } , (5.0.3) for some polynomials f, g i , h j ∈ R[x] with g := {g i } i∈[m] , h := {h j } j∈[l] . Also recall that g i = deg(g i )/2 , h j = deg(h j )/2 and k min = max i,j { f , g i , h j }.
We will assume in this chapter that POP (5.0.2) has at least one global minimizer.

Exploiting CTP for dense POPs

This section is devoted to developing a framework to exploit CTP for dense POPs. We provide a sufficient condition for a POP to have CTP, as well as a series of linear programs to check whether the sufficient condition holds. In addition we show that several special classes of POPs have CTP.

CTP for dense POPs

First let us recall CTP for a POP. To simplify notation, for every k ∈ N ≥kmin , denote by S (k) the set of real symmetric matrices -of size

s k := b(k) + i∈[m] b(k -g i ),
-in a block diagonal form X = diag(X 0 , . . . , X m ), and such that -X 0 (resp.

X i ) is of size b(k) (resp. b(k -g i ) for i ∈ [m]).
Letting D k (y) := diag(M k (y), M k-g1 (g 1 y), . . . , M k-gm (g m y)), SDP (2.4.5) can be rewritten in the form:

τ k := inf y∈R b(2k) L y (f ) D k (y) ∈ S (k) + , y 0 = 1 , M k-hj (h j y) = 0 , j ∈ [l] , (5.1.1)
where S

+ is the set of positive semidefinite matrices in S (k) . Definition 5.1. (CTP for a POP) We say that POP (5.0.2) has CTP if for every k ∈ N ≥kmin , there exists a k > 0 and a positive definite matrix P k ∈ S (k) such that for all y ∈ R b(2k) ,

M k-hj (h j y) = 0 , j ∈ [l] , y 0 = 1 ⇒ trace(P k D k (y)P k ) = a k . (5.1.2)
In other words, we say that POP (5.0.2) has CTP if each moment relaxation (5.1.1) has an equivalent form involving a psd matrix whose trace is constant. In this case, we call a k the constant trace and P k the basis transformation matrix. In the next subsection, we provide a sufficient condition for POP (5.0.2) to have CTP.

Example 5.1. We recall CTP for equality constrained POPs on a sphere in Chapter 4. If g = ∅ and h 1 = R -x 2 2 for some R > 0, then POP (5.0.2) has CTP with a k = (R + 1) k and P k := diag((θ

1/2 k,α ) α∈N n k ), where (θ k,α ) α∈N n k ⊆ R >0 satisfies (1 + x 2 2 ) k = α∈N n k θ k,α x 2α , for all k ∈ N ≥kmin .
We now provide a general method to solve a POP with CTP. We first convert the k-th order moment relaxation (5.1.1) of this POP to a standard primal SDP problem with CTP and then leverage appropriate first-order algorithms that exploit CTP to solve the resulting SDP problem.

Suppose POP (5.0.2) has CTP. For every k ∈ N ≥kmin , letting X = P k D k (y)P k , (5.1.1) can be rewritten as

τ k = inf X∈S (k) + { C k , X : A k X = b k } , ( 5.1.3)
where

A k : S (k) → R ζ k is a linear operator such that A k X = ( A k,1 , X , . . . , A k,ζ k , X ) with A k,i ∈ S (k) , i ∈ [ζ k ], C k ∈ S (k) and b k ∈ R ζ k . Appendix 5.
ρ k = sup z∈R ζ k { b k z : A k z -C k ∈ S (k) + } , (5.1.4)
where

A k : R ζ k → S (k) is the adjoint operator of A k , i.e., A k z = i∈[ζ k ] z i A k,i . After replacing (A k , A k,i , b k , C k , S (k) , ζ k , s k , τ k , ρ k , a k ) by (A, A i ,

A sufficient condition for a POP to have CTP

In this section, we provide a sufficient condition for POP (5.0.2) to have CTP. For k ∈ N ≥kmin , let Q • k (g) be the interior of the truncated quadratic module

Q k (g), i.e., Q • k (g) := {v k G 0 v k + i∈[m] g i v k-gi G i v k-gi : G i 0, i ∈ {0} ∪ [m]}.
Theorem 5.1. The following statements hold:

1. If one the following equivalent conditions hold for all k ∈ N ≥kmin :

R >0 ⊆ Q • k (g) + I k (h) ⇔ ∀δ > 0 , δ ∈ Q • k (g) + I k (h) ⇔ 1 ∈ Q • k (g) + I k (h) , ( 5
if R >0 ⊆ Q • k (g) , ∀k ∈ N ≥kmin . (5.1.6) Proof. 1. Let k ∈ N ≥kmin and assume that R >0 ⊆ Q • k (g) + I k (h). Then there exists a k > 0 such that a k = v k G 0 v k + i∈[m] g i v k-gi G i v k-gi + j∈[l] h j v 2(k-hj ) u j , (5.1.7)
for some G i 0, i ∈ {0} ∪ [m] and real vector u j , j ∈ [l]. We denote by G andy

1/2 i the square root of G i , i ∈ {0}∪[m]. Then G 1/2 i is well-defined and G 1/2 i 0. Set P k = diag(G 1/2 0 , . . . , G 1/2 m ). Let y ∈ R b(2k) such that M k (h j y) = 0, j ∈ [l],
0 = 1. Then L y   j∈[l] h j v 2(k-hj ) u j   = j∈[l] α∈N n 2(k-h j ) u j,α L y (h j x α ) = 0 . ( 5
a k = L y (v k G 0 v k + i∈[m] g i v k-gi G i v k-gi ) = trace(M k (y)G 0 ) + i∈[m] trace(M k-1 (g i y)G i ) = trace(G 1/2 0 M k (y)G 1/2 0 ) + i∈[m] trace(G 1/2 i M k-1 (g i y)G 1/2 i ) = trace(P k D k (y)P k ) ,
yielding the first statement.

2. The "if" part comes from the first statement. Let us prove the "only if" part. Assume that POP (5.0.2) has CTP (Definition 5.1). Let a ∈ S(g), y = (y α ) α∈N n be the moment sequence of the Dirac measure δ a . Let k ∈ N ≥kmin be fixed. Since P k ∈ S (k) ,

P k = diag(W 0 , . . . , W m ). Then W 2 i 0, i ∈ {0} ∪ [m] since P k 0. Let us define the polynomial w := v k W 2 0 v k + i∈[m] g i v k-gi W 2 i v k-gi .
By assumption,

a k = trace(P k D k (y)P k ) = trace(W 0 M k (y)W 0 ) + i∈[m] trace(W i M k-1 (g i y)W i ) = trace(M k (y)W 2 0 ) + i∈[m] trace(M k-1 (g i y)W 2 i ) = L y (v k W 2 0 v k + i∈[m] g i v k-gi W 2 i v k-gi ) = R n wdδ a (x) = w(a)
. It implies that w -a k vanishes on S(g). Since S(g) has nonempty interior, w = a k , yielding the second statement.

The following lemma will be used later on.

Lemma 5.1. Let R > 0. For all k ∈ N ≥1 , one has (R + 1) k = (1 + x 2 2 ) k + (R -x 2 2 ) k-1 j=0 (R + 1) j (1 + x 2 2 ) k-j-1 .
(5.1.9)

Proof. Let k ∈ N ≥1 . Letting a = R + 1 and b = 1 + x 2 2 , the desired equality follows from

a k -b k = (a -b) k-1 j=0 a j b k-1-j .
The next result states that the sufficient condition in Theorem 5.1 holds whenever a ball constraint is present in the POP's description. For a real symmetric matrix A, denote the largest eigenvalue of A by λ max (A). Theorem 5.2. If R -x 2 2 ∈ g for some R > 0 then the inclusions (5.1.6) hold and therefore POP (5.0.2) has CTP.

Proof. Without loss of generality, set g m := R -x 2 2 and let k ∈ N ≥kmin be fixed. By Lemma 5.1, (R + 1) k = Θ + g m Λ, where Θ := (1 + x 2 2 ) k and Λ :=

k-1 j=0 (R + 1) j (1 + x 2 2 ) k-j-1 . Note that • Θ = α∈N n k θ α x 2α = v k G 0 v k for some (θ α ) α∈N n k ⊆ R >0 ; • Λ = α∈N n k-1 λ α x 2α = v k-1 G m v k-1 for some (λ α ) α∈N n k-1 ⊆ R >0 . Here G 0 = diag((θ α ) α∈N n k ) and G m = diag((λ α ) α∈N n k-1 ) are both positive definite. Then we have (R + 1) k = v k G 0 v k + g m v k-1 G m v k-1 .
Denote by I t the identity matrix of size b(t) for t ∈ N.

Let W be a real symmetric matrix such that

i∈[m-1] g i v k-gi I k-gi v k-gi = v k Wv k . Since G 0 0, there exists δ > 0 such that G 0 -δW 0. Indeed, G 0 -δW 0 ⇔ I k δG -1/2 0 WG -1/2 0 ⇔ 1 > δλ max (G -1/2 0 WG -1/2 0
) ,

(5.1.10)

yielding the selection δ = 1/(|λ max (G -1/2 0 WG -1/2 0 )| + 1). Then (R + 1) k = v k (G 0 -δW)v k + δ i∈[m-1] g i v k-gi I k-gi v k-gi + g m v k-1 G m v k-1 , which implies (R + 1) k ∈ Q • k (g),
∈ Σ[x] k , σ ∈ Σ[x] k-1 and λ ∈ R such that f + λ = σ 0 + (R -x 2 2 )σ. Thus f + λ ∈ Q k (g). By Theorem 5.2, 1 ∈ Q • k (g) and therefore f + 1 + λ ∈ Q • k (g),
a k = (R + 1) k and P k = diag((G 0 -δW) 1/2 , √ δI k-g1 , . . . , √ δI k-gm-1 , G 1/2 m ) .
However, this choice leads to poor numerical properties. In the next section we provide a series of linear programs inspired from the inclusion in (5.1.5), to obtain a constant trace a k and a basis transformation matrix P k that achieve better numerical performance.

Verifying CTP for POPs by solving linear programs

For any k ∈ N ≥kmin , let Ŝ(k) be the set of real diagonal matrices of size b(k) and consider the following LP:

inf ξ,Gi,uj      ξ G 0 -I 0 ∈ Ŝ(k) + , G i -I i ∈ Ŝ(k-gi ) + , i ∈ [m] , ξ = v k G 0 v k + i∈[m] g i v k-gi G i v k-gi + j∈[l] h j v 2(k-hj ) u j      , ( 5
G i = G i,k , i ∈ {0} ∪ [m].
Since small constant traces are highly desirable for efficiency of first-order algorithms (e.g. CGAL), we search for an optimal solution of LP (5.1.11) instead of just a feasible solution.

Remark 5.2. One can extend the classes of diagonal matrices Ŝ(k) , Ŝ(kgi ) in (5.1.11) to obtain a smaller constant trace. For instance, one can define Ŝ(k) , Ŝ(kgi ) to be the class of symmetric block diagonal matrices with block size two. As shown in [START_REF] Wang | A second order cone characterization for sums of nonnegative circuits[END_REF]Lemma 4.3], (5.1.11) then becomes a second-order cone program which can be also efficiently solved.

Special classes of POPs with CTP

In this section we identify two classes of POPs whose CTP can be verified by LP (5.1.11). For I ⊆ [n], let x(I) := {x j : j ∈ I}. For matrices A and B of the same size, the Hadamard product of A and B, denoted by A • B, is the matrix with entries [A • B] i,j = A i,j B i,j .

POPs with ball or annulus constraints on subsets of variables

Consider the following assumption on the inequality constraints of POP (5.0.2).

Assumption 5.1.

There exists a nonnegative integer r ≤ m/2 and (1) m = 1, r = 0 and g Let u = (u j ) j∈[n] ⊆ N ≤m be defined by

• R i > R i > 0, T i ⊆ [n] for i ∈ [r]; • R j > 0, T j ⊆ [n] for j ∈ [m]\[2r] such that (1) (∪ i∈[r] T i ) ∪ (∪ j∈[m]\[2r] T j ) = [n]; (2) g i := x(T i ) 2 2 -R i , g i+r := R i -x(T i ) 2 2 for i ∈ [r]; (3) g i := R i -x(T i ) 2 2 for i ∈ [m]\[
1 := R 1 -x 2 2 , i.e., S(g) is a ball; (2) m = n, r = 0 and g i := R i -x 2 i for i ∈ [n], i.e., S(g) is a box; (3) m = 2, r = 1 and g 1 := x 2 2 -R 1 , g 2 := R 1 -x 2 2 (R 1 > R 1 > 0), i.e.,
u j := |{i ∈ [r] : j ∈ T i }| + |{i ∈ [m]\[2r] : j ∈ T i }| , ∀ j ∈ [n] . (5.1.12) Since (∪ i∈[r] T i ) ∪ (∪ i∈[m]\[2r] T i ) = [n], one has u j ∈ N ≥1 , j ∈ [n]. Moreover, u • x 2 2 = i∈[r] x(T i ) 2 2 + i∈[m]\[2r]
x(T i ) 2 2 .

(5.1.13)

With R := i∈[r] (R i + R i ) + i∈[m]\[2r] R i , by replacing x by u • x in Lemma 5.1, one obtains that for all k ∈ N ≥kmin , (R + 1) k = (1 + u • x 2 2 ) k + Λ k-1 i∈[m] δ i g i , (5.1.14) where Λ k-1 := k-1 j=0 (R + 1) j (1 + u • x 2
2 ) k-j-1 and

δ i := R i R i -R i , δ i+r := R i R i -R i , i ∈ [r] , and δ q = 1 , q ∈ [m]\[2r]. (5.1.15)
It is due to the fact that

R -u • x 2 2 = i∈[r] (R i + R i -x(T i ) 2 2 ) + i∈[m]\[2r] (R i -x(T i ) 2 2 ) , ( 5.1.16)
and

R i + R i -x(T i ) 2 2 = δ i g i + δ i+r g i+r , for all i ∈ [r]. For each k ∈ N ≥kmin , let (θ k,α ) α∈N n k ⊆ R >0 and (η k-1,α ) α∈N n k-1 ⊆ R >0 be such that (1 + u • x 2 2 ) k = α∈N n k θ k,α x 2α and Λ k-1 = α∈N n k-1 η k-1,α x 2α ,
and define the diagonal matrices

G (0) k := diag((θ k,α ) α∈N n k ) and G (i) k-1 := diag((δ i η k-1,α ) α∈N n k-1 ) , i ∈ [m] .
(5.1.17)

Then (5.1.14) yields that for every k ∈ N ≥kmin :

(R + 1) k = v k G (0) k v k + i∈[m] g i v k-1 G (i) k-1 v k-1 . Hence ((R + 1) k , G (i)
k , 0) is a feasible solution of (5.1.11), for every k ∈ N ≥kmin .

POPs with inequality constraints of equivalent degree

We say that polynomials q 1 , . . . , q t are of equivalent degree if

q 1 = • • • = q t .
:= g i , i ∈ [n + 1]. For every k ∈ N ≥kmin , letting Λ k-1 := k-1 j=0 (R + 1) j (1 + x 2 2 ) k-j-1 and Θ t := (1 + x 2 2 ) t , for t ∈ N, Lemma 5.1 yields (R + 1) k = Θ k + g m Λ k-1 . It implies that for every k ∈ N ≥kmin , (R + 1) k = (Θ k - L L + 1 Θ k-u ) + 1 L + 1 Θ k-u i∈[m-1] g i + g m Λ k-1 . (5.1.18)
It is due to the fact that i∈[m-1] g i = L. For each k ∈ N ≥kmin , let us consider the following sequences:

• (ν k,α ) α∈N n k ⊆ R >0 such that Θ k -L L+1 Θ k-u = α∈N n k ν k,α x 2α ; • (θ k-u,α ) α∈N n k-u ⊆ R >0 such that 1 L+1 Θ k-u = α∈N n k-u θ k-u,α x 2α ; • (η k-1,α ) α∈N n k-1 ⊆ R >0 such that Λ k-1 = α∈N n k-1 η k-1,α x 2α .
For each k ∈ N ≥kmin , define the diagonal matrices G (0)

k := diag((ν k,α ) α∈N n k ), G (1)
k-u := diag((θ k-u,α ) α∈N n k-u ) , and G (2)
k-1 := diag((η k-1,α ) α∈N n k-1 ) . Then (5.1.18) yields that for every k ∈ N ≥kmin , (R + 1) k = v k G (0) k v k + v k-u G (1) k-u v k-u i∈[m-1] g i + v k-1 G (2) k-1 v k-1 g m .
(5.1. [START_REF] Blekherman | There are significantly more nonegative polynomials than sums of squares[END_REF])

Hence ((R + 1) k , G (i)
k , 0) is a feasible solution of (5.1.11), for every k ∈ N ≥kmin . By using Lemma 5.2, the conclusion follows.

Example 5.3. Let R, L > 0 satisfy R ≥ L 2 and m = n + 2 , g i = x j for j ∈ [n] , g n+1 = L - j∈[n]
x j and g n+2 = R -x 2 2 .

(5.1.20)

Then Assumption 5.2 holds and S(g) is a simplex.

When S(g) is compact, we can always reformulate POP (5.0.2) such that Assumption 5.2 holds. Suppose S(g

) ⊆ B(0, R) for some R > 0. Let u = max i∈[m] g i . Set gi := g i (1 + x 2 2 ) u-gi for i ∈ [m]. Let L be a positive number such that i∈[m] gi ≤ L on S(g). Set gm+1 := L -i∈[m] gi and gm+2 := R -x 2
2 . Remark 5.3. For the latter case, one can choose any positive number L ≥ (R + 1) u i∈[m] g i 1 . Indeed, for any z ∈ S(g), and since z 2 2 ≤ R,

|z α | = i∈[n] |z i | αi ≤ i∈[n] (1 + z 2 2 ) αi/2 = (1 + z 2 2 ) |α|/2 ≤ (1 + R) t , ∀α ∈ N n 2t .

This implies that for every

i ∈ [m], gi (z) ≤ (1 + R) u-gi α∈N n 2 g i |g α ||z α | ≤ (1 + R) u-gi (R + 1) gi g i 1 = (1 + R) u g i 1 .
Thus we have i∈

[m] gi ≤ (1 + R) u i∈[m] g i 1 on S(g).
(j) k ∈ R b(2(k-hj )) , j ∈ [l],
and

(η (0) k,α ) α∈N n k ⊆ R >0 , (η (i) k-u,α ) α∈N n k-u ⊆ R >0 , i ∈ [m + 1] , (η (m+2) k-1,α ) α∈N n k-1 ⊆ R >0 such that 1 = v k diag((η (0) k,α ) α∈N n k )v k + i∈[m+1] gi v k-u diag((η (i) k-u,α ) α∈N n k-u )v k-u +g m+2 v k-1 diag((η (m+2) k-1,α ) α∈N n k-1 )v k-1 + j∈[l] h j v 2(k-hj ) u (j) k .
Let k ∈ N ≥kmin be fixed. We define the following polynomials:

• σ 0 := v k diag((η (0) k,α ) α∈N n k )v k = α∈N n k η (0) k,α x 2α , • σ i := v k-u diag((η (i) k-u,α ) α∈N n k-u )v k-u = α∈N n k-u η (i) k-u,α x 2α , i ∈ [m + 1],
•

σ m+2 := v k-1 diag((η (m+2) k-1,α ) α∈N n k-1 )v k-1 = α∈N n k-1 η (m+2) k-1,α x 2α , • ψ j := v 2(k-hj ) u (j) k , j ∈ [l].
From this and since gi :

= g i (1 + x 2 2 ) u-gi , for i ∈ [m], one has 1 = σ 0 + i∈[m] σ i gi + j∈[l] ψ j h j = σ 0 + i∈[m] σ i (1 + x 2 2 ) u-gi g i +g m+1 σ m+1 + gm+2 σ m+2 + j∈[l] ψ j h j .
(5.1.22)

Then there exist (θ

(i) k-gi ,α ) α∈N n k-g i ⊆ R >0 , i ∈ [m], such that σ i (1 + x 2 2 ) u-gi = α∈N n k-g i θ (i) k-gi ,α x 2α , i ∈ [m] .
(5.1.23)

Thus (5.1.22) becomes 1 = v k diag((η (0) k,α ) α∈N n k )v k + i∈[m] g i v k-gi diag((θ (i) k-gi ,α ) α∈N n k-g i )v k-gi +g m+1 v k-u diag((η (m+1) k-u,α ) α∈N n k-u )v k-u +g m+2 v k-1 diag((η (m+2) k-1,α ) α∈N n k-1 )v k-1 + j∈[l] h j v 2(k-hj ) u (j) k ∈ Q • k (g ∪ {g m+1 , gm+2 }) + I k (h) , since • diag((η (0) k,α ) α∈N n k ) 0, diag((θ (i) k-gi ,α ) α∈N n k-g i ) 0, i ∈ [m],
• diag((η

(m+1) k-u,α ) α∈N n k-u ) 0, and diag((η (m+2)
k-1,α ) α∈N n k-1 ) 0. It yields that (5.1.11) has a feasible solution with g ← g ∪ {g m+1 , gm+2 }, for every order k ∈ N ≥kmin .

In case that POP (5.0.2) does not have CTP and S(g) is compact, Corollary 5.2 provides a way to construct an equivalent POP by including two additional redundant constraints. Then CTP of this new POP can be verified by LP.

Numerical experiments for dense POPs

In this section we report results of numerical experiments obtained by solving the second order Moment-SOS relaxation of various randomly generated instances of QCQPs with CTP. The experiments were performed in Julia 1.3.1 with the following software packages:

• SumOfSquare [START_REF] Weisser | Polynomial and Moment Optimization in Julia and JuMP[END_REF] is a modeling library for solving the Moment-SOS relaxations of dense POPs, based on JuMP (with Mosek 9.1 as SDP solver).

• TSSOS [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF][START_REF] Wang | Chordal-tssos: a moment-sos hierarchy that exploits term sparsity with chordal extension[END_REF][START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF] is a modeling library for solving Moment-SOS relaxations of sparse POPs based on JuMP (with Mosek 9.1 as SDP solver).

• LMBM solves unconstrained non-smooth optimization with the limited-memory bundle method by Haarala et al. [START_REF] Haarala | Globally convergent limited memory bundle method for large-scale nonsmooth optimization[END_REF][START_REF] Haarala | New limited memory bundle method for largescale nonsmooth optimization[END_REF] and calls Karmitsa's Fortran implementation of the LMBM algorithm [START_REF] Karmitsa | LMBM-FORTRAN subroutines for Large-Scale nonsmooth minimization: User's manual[END_REF].

• Arpack [START_REF] Lehoucq | ARPACK users' guide: solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods[END_REF] is used to compute the smallest eigenvalues and the corresponding eigenvectors of real symmetric matrices of (potentially) large size, which is based on the implicitly restarted Arnoldi method.

The implementation of Algorithms 6 and 7 is available online via the link: https://github.com/maihoanganh/ctpPOP.

We use a desktop computer with an Intel(R) Core(TM) i7-8665U CPU @ 1.9GHz × 8 and 31.2 GB of RAM. The notation for the numerical results is given in Table 5.1.

For the examples tested in this chapter, the modeling time of SumOfSquares, TSSOS and ctpPOP is typically negligible compared to the solving time of the packages Mosek, CGAL, and LMBM. Hence the total running time mainly depends on the solvers and we compare their performances below. As mentioned in the introduction, the current framework differs from our previous work [START_REF] Mai | A hierarchy of spectral relaxations for polynomial optimization[END_REF], where we exploited CTP for equality constrained POPs on a sphere, which could be solved by LMBM efficiently. The reason is that the SDP relaxations of such equality constrained POPs involve a single psd matrix. For the benchmarks of this section, we consider POPs involving ball/annulus constraints, and so the resulting relaxations include several psd matrices. Our numerical experiments confirm that for such SDPs, LMBM returns inaccurate values since the gap w.r.t. the value of Mosek is typically larger than 1% while CGAL (without sketching) performs better for this type of SDPs in terms of accuracy. As showed in Section 5.2.1, the last columns of

Randomly generated dense QCQPs with a ball constraint

Test problems: We construct randomly generated dense QCQPs with a ball constraint as follows:

1. Generate a dense quadratic polynomial objective function f with random coefficients following the uniform probability distribution on (-1, 1);

2. Let m = 1 and g 1 := 1 -x 2 2 ;

3. Take a random point a in S(g) w.r.t. the uniform distribution;

4. For every j ∈ [l], generate a dense quadratic polynomial h j by (i) for each α ∈ N n 2 \{0}, taking a random coefficient h j,α for h j in (-1, 1) w.r.t. the uniform distribution;

(ii) setting h j,0 := -α∈N n 2 \{0} h j,α a α .
Then a is a feasible solution of POP (5.0.2).

The numerical results are displayed in Table 5.2 and 5.3.

Discussion:

As one can see from Table 5.2 and 5.3, CGAL is typically the fastest solver and returns an optimal value of gap within 1% w.r.t. the one returned by Mosek when n ≤ 30. Mosek runs out of memory when n ≥ 40 while CGAL works well up to n = 100. We should point out that LMBM is less accurate or even fails to converge to the optimal value when n ≥ 20. The reason might be that LMBM only solves the dual problem and hence looses information of the primal problem.

Randomly generated dense QCQPs with annulus constraints

Test problems: We construct randomly generated dense QCQPs as in Section 5.2.1, where the ball constraint is now replaced by annulus constraints. Namely, in Step 2 we take m = 2, g 1 := x 2 2 -1/2 and g 2 := 1 -x 2 2 . The numerical results are displayed in Table 5.4 and 5.5.

Discussion: Same remarks as in Section 5.2.1.

Randomly generated dense QCQPs with box constraints

Test problems: We construct randomly generated dense QCQPs as in Section 5.2.1, where the ball constraint is now replaced by box constraints. Namely, in Step 2 we take m = n, g j :=

-x 2 j + 1/n, j ∈ [n].
The numerical results are displayed in Table 5.6 and 5.7.

Discussion:

We observe similar behaviors of the solvers as in Section 5.2.1. The important point to note here is that solving a QCQP with box constraints is less efficient than solving the same one with ball constraints. This is because the efficiency of CGAL depends on the number of psd blocks involved in an SDP. For instance, when n = 50, CGAL takes around 1000 seconds to solve the second order moment relaxation of a QCQP with a ball constraint while it takes around 2100 seconds to solve this relaxation for a QCQP with box constraints.

Randomly generated dense QCQPs with simplex constraints

Test problems: We construct randomly generated dense QCQPs as in Section 5.2.1, where the ball constraint is now replaced by simplex constraints. Namely, in Step 2 we take g such that (5.1.20) holds with L = R = 1. The numerical results are displayed in Table 5.8 and 5.9.

Discussion: Again we observe a behavior of the solvers similar to that in Section 5.2.1. One can also see that solving a QCQP with simplex constraints by CGAL is significantly slower than solving the same one with box constraints. For instance, when n = 50, CGAL takes 2100 seconds to solve the second order moment relaxation for a QCQP with box constraints while it takes 9500 seconds with simplex constraints.

Numerical comparison between CGAL and ADMM

In Table 5.10, we make a numerical comparison between CGAL (with our CTP-exploiting method) and COSMO, an SDP solver based on ADMM (see Table 1.3), on some randomly generated dense QCQPs with a ball constraint (as in Section 5.2.1).

Discussion: Table 5.10 indicates that both CGAL and COSMO provide approximate values with gap within 1% w.r.t. the ones returned by Mosek when n ≤ 30. In addition, COSMO is slightly more accurate for n ∈ {20, 30} while CGAL offers an increasing speedup when n ≥ 30.

Dense POPs with a ball constraint

We construct randomly generated dense POPs as in Section 5.2.1, with input data of degree d ∈ {3, 4}.

The numerical results, displayed in Table 5.11, indicate that CGAL returns an optimal value with gap within 1% w.r.t. the one of Mosek, and is faster when the largest block size increases.

Appendix

Exploiting CTP for POPs with CS

In this section, we extend the CTP-exploiting framework to POPs with sparsity. For clarity of exposition we only consider correlative sparsity (CS). However, in Appendix 5.3.2 we also treat term sparsity (TS) [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF] as well as correlative-term sparsity (CS-TSSOS) [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]. Since the methodology is very similar to that in the dense case described earlier, we omit details and only present the main results.

To make this appendix self-contained, we recall some basic facts already stated in Chapter 3 on exploiting CS for POP (5.0.2) initially proposed in [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF]

[A • B] i,j = A i,j B i,j .

POPs with CS

Assume that {I c } c∈[p] (with n c := |I c |) are the maximal cliques of (a chordal extension of) the correlative sparsity pattern (csp) graph associated with POP (5.0.2), as defined in [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF].

Let

{J c } c∈[p] (resp. {W c } c∈[p] ) be a partition of [m] (resp. [l]) such that for all i ∈ J c , g i ∈ R[x(I c )] (resp. i ∈ W c , h i ∈ R[x(I c )]), c ∈ [p]. For each c ∈ [p], let m c := |J c |, R c := |W c | and g Jc := {g i : i ∈ J c }, h Wc := {h i : i ∈ W c }. Then Q(g Jc ) (resp. I(h Wc )) is a quadratic module (resp. an ideal) in R[x(I c )], for c ∈ [p].
For each k ∈ N ≥kmin , consider the following sparse SOS strengthening:

ρ cs k := sup    ξ : f -ξ ∈ c∈[p] (Q k (g Jc ) + I k (h Wc ))    . (5.3.1)
It is equivalent to the SDP:

ρ cs k = sup ξ,G (c) i , gi (c)              ξ G (c) i 0 , i ∈ {0} ∪ J c , c ∈ [p] , f -ξ = c∈[p] (v Ic k ) G (c) 0 v Ic k + i∈Jc g i (v Ic k-gi ) G (c) i v Ic k-gi + j∈Wc h j (v Ic 2(k-hj ) ) u (c) j              . (5.3.2)
The dual of (5.3.2) reads

τ cs k := inf y∈R b(2k)    L y (f ) M k (y, I c ) 0 , c ∈ [p] , y 0 = 1 . M k-gi (g i y, I c ) 0 , i ∈ J c , c ∈ [p] , M k-hj (h j y, I c ) = 0 , j ∈ W c , c ∈ [p]    . ( 5

Exploiting CTP for POPs with CS

For every c ∈ [p], we denote by S (c,k) the set of real symmetric matrices of size b(n c , k)

+ i∈Jc b(n c , k -g i ) in a block diagonal form: X = diag(X 0 , (X i ) i∈Jc ) such that X 0 is a block of size b(k, n c ) and X i is a block of size b(n c , k -g i ) for i ∈ J c .
Consider POP (5.0.2) with CS described in Section 5.3.1. For every c ∈ [p] and for every 2k) , SDP (5.3.3) can be rewritten as

k ∈ N ≥kmin , letting D k (y, I c ) := diag(M k (y, I c ), (M k-gi (g i y, I c )) i∈Jc ) for y ∈ R s(
τ cs k := inf y∈R b(2k) L y (f ) D k (y, I c ) 0 , j ∈ [p] , y 0 = 1 , M k-hj (h j y, I c ) = 0 , j ∈ W c , c ∈ [p] . ( 5.3.4)
We define CTP for POP with CS as follows.

M k-hj (h i y, I c ) = 0 , j ∈ W c , y 0 = 1 ⇒ trace(P (c) k D k (y, I c )P (c) k ) = a (c)
k .

(5.3.5)

The following result provides a sufficient condition for a POP with CS to have CTP.

Theorem 5.3. Assume that there is a ball constraint on each clique of variables, i.e., We are now in position to provide a general method to solve POPs with CS which have CTP.

∀ c ∈ [p], R c -x(I c ) 2 2 ∈ g for some R c > 0 . (5.3.6) Then one has R >0 ⊆ Q • k (g Jc ),
Consider POP (5.0.2) with CS described in Section 5.3.1. Assume that POP (5.0.2) has CTP and let k ∈ N ≥kmin be fixed.

Letting

X c = P (c) k D k (y, I c )P (c) k , c ∈ [p] ,
(5.3.7) SDP (5.3.4) can be rewritten as

τ cs k = inf Xc∈S (c,k) +    c∈[p] C c,k , X c : c∈[p] A c,k X c = b k , c ∈ [p]    , (5.3.8)
where for every c ∈

[p], A c,k : S (c,k) → R ζ k is a linear operator of the form A c,k X = ( A c,k,1 , X , . . . , A c,k,ζ k , X ) with A c,k,i ∈ S (c,k) , i ∈ [ζ k ], C c,k ∈ S (c,k) , c ∈ [p] and b k ∈ R ζ k .
ρ cs k = sup y∈R ζ b k y : A c,k y -C c,k ∈ S (c,k) + , c ∈ [p] ,
(5.3.9)

where

A c,k : R ζ → S (c,k) is the adjoint operator of A c,k , i.e., A c,k z = i∈[ζ] z i A c,k,i , c ∈ [p]
. By Definition 5.2, it holds that for every k ∈ N ≥kmin ,

∀ X c ∈ S (c,k) , c ∈ [p] c∈[p] A c,k X c = b k ⇒ trace(X c ) = a (c) k , c ∈ [p] . (5.3.10) After replacing (A c,k , A c,k,i , b k , C c,k , S (c,k) , ζ k , τ cs k , a (c) k ) by (A c , A i,c , b, C c , S (c) , ζ, τ, a c ), SDP (5.
s max = max c∈[p] b(n c , k).
If there is a ball constraint on each clique of variables then by Corollary 5.3, strong duality holds for the pair (5.3.8)-(5.3.9), for every k ∈ N ≥kmin .

The two algorithms (CGAL and SM) based on first-order methods are then leveraged to solve the primal-dual (5.3.8)-(5.3.9); see Appendix 5.3.4 and Appendix 5.3.5.

Verifying CTP for POPs with CS via LP

As in the dense case, we can verify CTP for a POP with CS via a series of LPs. For every k ∈ N ≥kmin and for every c ∈ [p], let Ŝ(c,k) be the set of real diagonal matrices of size b(n c , k) and consider the following LP:

inf ξ,Gi,ui      ξ G 0 -I 0 ∈ Ŝ(c,k) + , G i -I i ∈ Ŝ(c,k-gi ) + , i ∈ J c , ξ = (v Ic k ) G 0 v Ic k + i∈Jc g i (v Ic k-gi ) G i v Ic k-gi + j∈Wc h j (v Ic 2(k-hj ) ) u j      , ( 5.3.11)
where I i is the identity matrix, for every i ∈ {0} ∪ J c .

k = diag(G 1/2 0,k , (G 1/2 i,k ) i∈Ji ) and a (c) k = ξ (c) k , for k ∈ N ≥kmin and for c ∈ [p].
The proof of Lemma 5.3 is similar to that of Lemma 5.2. For instance, for POPs with ball or annulus constraints on subsets of each clique of variables, CTP can be verified by LP. Proposition 5.3. Let POP (5.0.2) with CS be described in Section 5. 3

.1. Let (T i ) i∈[r]∪([m]\[2r]) be as in Assumption 5.1 and further assume that for every

c ∈ [p], (∪ q∈Jc∩[r] T q ) ∪ (∪ q∈Jc\[2r] T q ) = I c .
Then LP (5.3.11) has a feasible solution for every k ∈ N ≥kmin , and therefore POP (5.0.2) has CTP. Proof. To prove that POP (5.0.2) has CTP on each clique of variables, it is sufficient to show that (5.3.11) has a feasible solution, for every k ∈ N ≥kmin and for every c ∈ [p] due to Lemma 5.3.

For every c ∈ [p], let u (c) = (u

(c) i ) i∈Ic ⊆ N ≤|Jc| be defined by u (c) i = |{q ∈ J c ∩ [r] : i ∈ T q }| + |{q ∈ J c \[2r] : i ∈ T q }| , i ∈ I c .
(5.3.12)

For every c ∈ [p], one has u

(c) i ∈ N ≥1 , i ∈ I c , according to (∪ q∈Jc∩[r] T q ) ∪ (∪ q∈Jc\[2r] T q ) = I c . Moreover, u (c) • x(I c ) 2 2 = i∈Jc∩[r] x(T i ) 2 2 + i∈Jc\[2r] x(T i ) 2 2 , ∀c ∈ [p] . (5.3.13) For every c ∈ [p], with R (c) := i∈Jc∩[r] (R i + R i ) + i∈Jc\[2r] R i , by replacing x (resp. R) by u (c) • x(I c ) (resp. R (c) ) in Lemma 5.1, we obtain (R (c) + 1) k = (1 + u (c) • x(I c ) 2 2 ) k + Λ (c) k-1 i∈Jc δ i g i , ∀c ∈ [p] , ∀k ∈ N ≥kmin , (5.3.14)
where Λ

(c) k-1 := k-1 r=0 (R (c) + 1) r (1 + u (c) • x(I c ) 2
2 ) k-r-1 and

δ i := R i R i -R i , δ i+r := R i R i -R i , i ∈ J c ∩ [r] and δ q = 1 , q ∈ J c \[2r]. (5.3.15)
It is due to the fact that

R (c) -u (c) • x(I c ) 2 = i∈Jc∩[r] (R i + R i -x(T i ) 2 2 ) + i∈Jc\[2r] (R i -x(T i ) 2 2 ) , (5.3.16) and R i + R i -x(T i ) 2 2 = δ i g i + δ i+r g i+r , i ∈ J c ∩ [r]. For every c ∈ [p], for each k ∈ N ≥kmin , let (θ (c) k,α ) α∈N Ic k ⊆ R >0 and (η (c) k-1,α ) α∈N Ic k-1 ⊆ R >0 be such that (1 + u (c) • x(I c ) 2 2 ) k = α∈N Ic k θ (c) k,α x 2α and Λ (c) k-1 = α∈N Ic k-1 η (c) k-1,α x 2α ,
and define the diagonal matrices

G (c,0) k := diag((θ (c) k,α ) α∈N Ic k ) and G (c,i) k-1 := diag((δ i η (c) k-1,α ) α∈N Ic k-1
) , i ∈ J c .

(5.3.17)

For every c ∈ [p], (5.3.14) yields that for every k ∈ N ≥kmin , (R

(c) + 1) k = (v Ic k ) G (c,0) k v Ic k + i∈Jc g i (v Ic k-1 ) G (c,i) k-1 v Ic k-1 . (5.3.18) Hence ((R (c) + 1) k , G (c,i) k
, 0) is a feasible solution of (5.3.11), for every k ∈ N ≥kmin and for every c ∈ [p].

Main algorithm

Algorithm 7 below solves POP (5.0.2) with CS and whose CTP can be verified by LP.

Algorithm 7

Approximating the optimal value of a POP with CS and CTP Input: POP (5.0.2) with CS and a relaxation order k ∈ N ≥kmin Output: The optimal value τ cs k of SDP (5.3.8)

1: for c ∈ [p] do 2:
Solve LP (5.3.11) to obtain an optimal solution (ξ

(c) k , G (c) i,k , u (c) c,k );

Exploiting CTP for POPs with TS and CS-TS

In this section, we restate the main results of TS in [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF] and CS-TSSOS in [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]. Similarly to dense POPs and POPs with CS, one can easily exploit CTP for POPs with TS and CS-TS. The central reason is that the diagonal of each moment/localizing matrix in a given moment relaxation of a dense POP (resp. POP with CS) does not change when TS (resp. CS-TS) is exploited.

Term sparsity (TS)

Fix a relaxation order k ∈ N ≥kmin and a sparse order t ∈ N\{0}. We compute as in [212, Section 5] the following block diagonal (up to permutation) (0, 1)-binary matrices:

G (0) k,t of size b(k); G (i) k,t of size b(k -g i ), i ∈ [m]; H (j) k,t of size b(k -h j ), j ∈ [l].
Then we consider the following sparse moment relaxation of POP (5.0.2):

τ ts k,t := inf y∈R b(2k)      L y (f ) G (0) k,t • M k (y) 0 , y 0 = 1 , G (i) k,t • M k-gi (g i y) 0 , i ∈ [m] , H (j) k,t • M k-hj (h j y) = 0 , j ∈ [l]      . ( 5
ρ ts k,t = sup ξ,Qi,Ui          ξ Qi = G (i) k,t • Q i 0 , i ∈ {0} ∪ [m] , Ūi = H (i) k,t • U i , i ∈ [l] , f -ξ = v k Q0 v k + i∈[m] g i v k-gi Qi v k-gi + j∈[l] h j v k-hj Ūj v k-hj          . ( 5.3.20)

Correlative-Term sparsity (CS-TSSOS)

The basic idea of correlative-term sparsity is to exploit term sparsity for each clique. The clique structure of the initial set of variables is derived from correlative sparsity (Section 5.3.1).

Fix a relaxation order k ∈ N ≥kmin . For every sparse order t ∈ N\{0} and for every c ∈ [p], we compute the following block diagonal (up to permutation) (0, 1)-binary matrices (see [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]):

G (0) k,t,c of size b(n c , k); G (i) k,t,c of size b(n c , k -g i ), i ∈ J c ; H (j) k,t,c of size b(n c , k -h j ), j ∈ W c .
Then let us consider the following CS-TSSOS moment relaxation:

τ cs-ts k,t := inf y∈R b(2k)      L y (f ) G (0) k,t,c • M k (y, I c ) 0 , c ∈ [p] , y 0 = 1 , G (i) k,t,c • M k-gi (g i y, I c ) 0 , i ∈ J c , c ∈ [p] , H (j) k,t,c • M k-hj (h j y, I c ) = 0 , j ∈ W c , c ∈ [p]      . (5.3.21) One has τ cs-ts k,t-1 ≤ τ cs-ts k,t ≤ τ cs k ≤ τ k ≤ f , for all (k, t)
. Moreover, we have the following theorem.

Theorem 5.5. (Wang et al. [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]) For each k ∈ N ≥kmin , the sequence (τ cs-ts k,t ) t∈N\{0} converges to τ cs k (the optimal value of SDP (5.3.3)) in finitely many steps. The dual of (5.3.21) reads

ρ cs-ts k,t = sup ξ,Q (c) i ,U (c) i                  ξ Q(j) i = G (i) k,t,c • Q (c) i 0 , i ∈ {0} ∪ J c , c ∈ [p] , Ū(c) i = H (i) k,t,c • U (c) i , i ∈ W c , c ∈ [p] , f -ξ = c∈[p] (v Ic k ) Q(c) 0 v Ic k + i∈Jc g i (v Ic k-gi ) Q(c) i v Ic k-gi + j∈Wc h j (v Ic k-hj ) Ū(c) j v Ic k-hj                  . ( 5

Numerical experiments for sparse POPs

In this section we report results of numerical experiments for sparse POPs with the same settings and notations as in Section 5.2.

Randomly generated QCQPs with TS and ball constraints

Test problems: We construct randomly generated QCQPs with TS and a ball constraint as follows:

1. Generate a quadratic polynomial objective function f such that for α ∈ N n 2 with |α| = 2, f α = 0 and for α ∈ N n 2 with |α| = 2, the coefficient f α is randomly generated in (-1, 1) w.r.t. the uniform distribution; 2. Take m = 1 and g 1 := 1 -x 2 2 ;

3. Take a random point a in S(g) w.r.t. the uniform distribution;

4. For every j ∈ [l], generate a quadratic polynomial h j by (i) setting h j,α = 0 for each α ∈ N n 2 \{0} with |α| = 2; (ii) for each α ∈ N n 2 \{0} with |α| = 2, taking a random coefficient h j,α for h j in (-1, 1) w.r.t. the uniform distribution;

(iii) setting h j,0 := -α∈N n 2 \{0} h j,α a α . Then a is a feasible solution of POP (5.0.2).

The numerical results are displayed in Table 5.12 and 5.13. Discussion: The behavior of solvers is similar to that in the dense case.

Randomly generated QCQPs with TS and box constraints

Test problems: We construct randomly generated QCQPs with TS as in Section 5.3.3, where the ball constraint is now replaced by box constraints. The numerical results are displayed in Table 5.14 and 5.15.

Discussion: Again the behavior of solvers is similar to that in the dense case.

Randomly generated QCQPs with CS and ball constraints on each clique of variables

Test problems: We construct randomly generated QCQPs with CS and ball constraints on each clique of variables as follows:

1. Take a positive integer u, p := n/u + 1 and let

I c =      [u], if c = 1 , {u(c -1), . . . , uc}, if c ∈ {2, . . . , p -1} , {u(p -1), . . . , n}, if c = p ; (5.3.23)

Generate a quadratic polynomial objective function

f = c∈[p] f c such that for each c ∈ [p], f c ∈ R[x(I c )]
:= -α∈N Ic 2 \{0} h c,α a α .
Then a is a feasible solution of POP (5.0.2).

The numerical results are displayed in Table 5.16 and 5.17.

Discussion:

The number of variables is fixed as n = 1000. We increase the clique size u so that the number of variable cliques p decreases accordingly. Again results in Table 5.16 and 5.17 show that CGAL is faster and returns an optimal value of gap within 1% w.r.t. the one returned by Mosek (for u ≤ 16). Moreover Mosek runs out of memory when u ≥ 21.

Randomly generated QCQPs with CS and box constraints on each clique of variables

Test problems: We construct randomly generated QCQPs with CS as in Section 5.3.3, where ball constraints are now replaced by box constraints. Namely, in Step 3 we take m = n, g j := -x Discussion: The number of variables is fixed as n = 1000. We increase the clique size u so that the number of variable cliques p decreases accordingly. From results in Table 5.16 and 5.17, one observes that when the largest size of variable cliques is relatively small (say u ≤ 11), Mosek is the fastest solver. However when the largest size of variable cliques is relatively large (say u ≥ 21), Mosek runs out of memory while CGAL still works well.

Randomly generated QCQPs with CS-TSSOS and ball constraints on each clique of variables

Test problems: We construct randomly generated QCQPs with CS-TSSOS and ball constraints on each clique of variables as follows:

1. Take a positive integer u, p := n/u + 1 and let (I c ) c∈[p] be defined as in (5.3.23); Discussion: The behavior of solvers is similar to that in Section 5.3.3. Here, we also emphasize that our framework is less efficient than interior-point methods for most benchmarks presented in [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]. The two underlying reasons are that (1) the block size of the resulting SDP relaxations is small, in which case Mosek performs more efficiently, e.g., for the benchmarks from [213, Section 5.2], and (2) it is harder to find the constant trace, e.g., for the benchmarks from [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF]Section 5.4]. Thus our proposed method complements that in [START_REF] Wang | CS-TSSOS: Correlative and term sparsity for large-scale polynomial optimization[END_REF] when the block size of the SDP relaxations is large and/or when CTP can be efficiently verified.

Generate a quadratic polynomial objective function

f = c∈[p] f c such that for each c ∈ [p], f c ∈ R[x(I c )]

Randomly generated QCQPs with CS-TSSOS and box constraints on each clique of variables

Test problems: We construct randomly generated QCQPs with CS-TSSOS as in Section 5.3.3, where ball constraints are now replaced by box constraints. Namely, in Step 3 we take m = n, g j := -x 2 j + 1/u, j ∈ [n]. The numerical results are displayed in Table 5.22 and 5.23.

Discussion:

The behavior of solvers is similar to that in Section 5.3.3.

Conditional gradient-based augmented Lagrangian

SDP with CTP

Let s, l, s (j) ∈ N ≥1 , j ∈ [ω], be fixed such that s = ω j=1 s (j) . Let S be the set of real symmetric matrices of size s in a block diagonal form: X = diag(X 1 , . . . , X ω ), such that X j is a block of size s (j) , j ∈ [ω]. Let s max := max j∈[ω] s (j) . Let S + be the set of all X ∈ S such that X 0, i.e., X has only nonnegative eigenvalues. Then S is a Hilbert space with scalar product A, B = trace(B A) and S + is a self-dual cone.

Let us consider the following SDP:

τ = inf X∈S+ { C, X : AX = b} , ( 5.3.25)
where

A : S → R ζ is a linear operator of the form AX = [ A 1 , X , . . . , A ζ , X ], with A i ∈ S, i ∈ [ζ], C ∈ S

Constant trace property (CTP): ∃a

> 0 : ∀ X ∈ S , AX = b ⇒ trace(X) = a.
For X ∈ S, the Frobenius norm of X is defined by X F := X, X . We denote by A the operator norm of A, i.e., A := max X∈S AX 2 / X F . The smallest eigenvalue of a real symmetric matrix D is denoted by λ min (D). Take an eigenvector u t corresponding to λ min (C + A (y t-1 + η t (AX t-1 -b)));

Algorithm. In

C F = A = a = 1 and A 1 F = • • • = A ζ F .

5:

Set X t := (1 -η t )X t-1 + η t au t u t ;

6:

Select γ t as the largest γ ∈ [0, 1] such that:

7: γ AX t -b 2 2 ≤ β t η 2 t a 2 A 2 and y t-1 + γ(AX t -b) 2 ≤ K; 8:
Set y t = y t-1 + γ t (AX t -b).

Remark 5.5. Given ε > 0, the for loop in Algorithm 8 terminates when: In Appendix 5.3.4, we provide an analogous method to solve an SDP with CTP on each subset of blocks.

| C, X t-1 -(aλ min (C + A (y t-1 + η t (AX t-1 -b))) -b y t-1 )| 1 + max{| C, X t-1 |, |aλ min (C + A (y t-1 + η t (AX t-1 -b))) -b y t-1 |} ≤ ε (5.3.

SDP with CTP on each subset of blocks

τ = inf Xc∈S (c) +    c∈[p] C c , X c : c∈[p] A c X c = b    , ( 5.3.28)
where where

A c : S (c) → R ζ is a linear operator of the form A c X = [ A 1,c , X , . . . , A ζ,c , X ], with A i,c ∈ S (c) , i ∈ [ζ], C c ∈ S (c) , c ∈ [p],
A c : R ζ → S (c) is the adjoint operator of A c , i.e., A c z = i∈[ζ] z i A i,c , c ∈ [p].
The following assumption will be used later on:

Assumption 5.4. Consider the following conditions:

1. Strong duality of primal-dual (5.3.28)-(5.3.29) holds, i.e., ρ = τ and ρ ∈ R.

Constant trace property (CTP)

: there exist a c > 0 and c ∈ [p], such that (1) , X , . . . , A (ζ) , X ], where

∀ X c ∈ S (c) , c ∈ [p] , c∈[p] A c X c = b ⇒ trace(X c ) = a c , c ∈ [p] . ( 5
= c∈[p] A c X c , for all X = diag(X c ) c∈[p] ∈ c∈[p] S (c) . Then for every X = diag(X c ) c∈[p] ∈ c∈[p] S (c) , we have C, X = c∈[p] C c , X c and AX = [ A
A (i) := diag((A i,c ) c∈[p] ), for i ∈ [ζ]. SDP (5.
∆ := c∈[p] ∆ c , K ← {b}, p ← ζ, Ax ← AX, f (x) ← C, X , λ 0 ← 1, λ k ← β k , σ k ← γ k . D Y k+1 ← K, L f ← 0, rk+1 ← b, D 2 X ← 2 c∈[p] a 2 c , v k ← C + A z k , arg min x∈X v k , x ← arg min X∈∆ C + A z k , X .
With fixed z k , we have:

min X∈∆ C + A z k , X = min diag((Xc) c∈[p] )∈ c∈[p] ∆c c∈[p] C c + A c z k , X c = c∈[p] min Xc∈∆c C c + A c z k , X c = c∈[p] a c λ min (C c + A c z k ) . Let u (c) k be a uniform eigenvector corresponding to λ min (C c + A c z k ), for c ∈ [p]. Then one has diag((a c u (c) k (u (c) k ) ) c∈[p] ) ∈ arg min X∈∆ C+A z k , X . Thus we can set s k ← diag((a c u (c) k (u (c) k ) ) c∈[p] ) in [218, Algorithm 1].
Set z t := y t-1 + η t ( c∈[p] A c X (t-1) c -b); 5: for c ∈ [p] do 6:
Take a uniform eigenvector u (c) t corresponding to λ min (C c + A c z t ); Set X (t)

c := (1 -η t )X (t-1) c + η t a c u (c) t (u (c) t ) ; 8:
Select γ t as the larges γ ∈ [0, 1] such that:

9: γ c∈[p] A c X (t) c -b 2 2 ≤ β t η 2 t ( c∈[p] a 2 c ) A 2 and y t-1 +γ( c∈[p] A c X (t) c -b) 2 ≤ K; 10: Set y t = y t-1 + γ t ( c∈[p] A c X (t) c -b).
Relying on [218, Theorem 3.1], we guarantee the convergence of the sequence ((X

(t) c ) c∈[p]
) t∈N in Algorithm 9 to the set of optimal solutions of SDP (5.3.28)

C F = A = a 1 = • • • = a p = 1 and A (1) F = • • • = A (ζ) F .
Remark 5.8. Given ε > 0, the for loop in Algorithm 9 terminates when:

| c∈[p] C c , X (t-1) c -c∈[p] (a c λ min (C c + A c z t ) -b y t-1 )| 1 + max{| c∈[p] C c , X (t-1) c |, | c∈[p] (a j λ min (C c + A c z t ) -b y t-1 )|} ≤ ε and c∈[p] A c X (t-1) c -b 2 / max{1, b 2 } ≤ ε.
In our experiments, we choose ε = 10 -2 .

Remark 5.9. To save memory at each iteration, we can run Algorithm 9 with implicit

X (t) c , c ∈ [p], by setting w t := c∈[p] A c X (t) c -b. In this case, Step 7 becomes w t := (1 -η t )w t-1 + η t [ c∈[p] A c (a c u (c) t (u (c) t ) ) -b].
Thus we only obtain an approximate dual solution y t of SDP (5.3.28) when Algorithm 9 terminates.

Spectral method

SDP with CTP

Consider SDP with CTP described in Appendix 5.3.4. The following assumption will be used later on. Algorithm 10 Spectral-SDP-CTP Input: SDP (5.3.25) with unknown optimal value and optimal solution; method (T) for solving convex nonsmooth unconstrained optimization problems (NSOP). Output: the optimal value τ of SDP (5.3.25).

1: Compute the optimal value τ and an optimal solution ȳ of the NSOP (5.3.31) by using method (T).

Corollary 5.4. Let Assumption 5.3 hold. Assume that the method (T) is globally convergent for NSOP (5.3.31) (e.g., (T) is LMBM). Then output τ of Algorithm 10 is well-defined. Moreover, if Assumption 5.5 holds, the vector ȳ mentioned at

Step 1 of Algorithm 10 exists.

SDP with CTP on each subset of blocks

Consider SDP with CTP on each subset of blocks described in Appendix 5.3.4.

The following assumption will be used later on. Algorithm 11 Spectral-SDP-CTP-Blocks Input: SDP (5.3.28) with unknown optimal value and optimal solution; method (T) for solving NSOP. Output: the optimal value ρ of SDP (5.3.28).

τ = inf Xc∈S (c) +    c∈[p] C c , X c c∈[p] A c X c = b , I c , X c = a c , c ∈ [p]    , ( 5
1: Compute the optimal value τ and an optimal solution ȳ of the NSOP (5.3.32) by using method (T).

The fact that Algorithm 11 is well-defined under certain conditions is a corollary of Lemma 5.5 and [134, Lemma A.2].

Corollary 5.5. Let Assumption 5.4 hold. Assume that the method (T) is globally convergent for NSOP (5.3.32) (e.g., (T) is LMBM). Then output τ of Algorithm 11 is well-defined. Moreover, if Assumption 5.6 holds, the vector ȳ involved at

Step 1 of Algorithm 11 exists.

Converting the moment relaxation to the standard SDP

The dense case

Let k ∈ N ≥kmin be fixed. We will present a way to transform SDP (5.1.1) to the form (5.1.3). By adding slack variables y (i) ∈ R s(2(k-gi )) , i ∈ [m], SDP (5.1.1) is equivalent to

τ k := inf y,y (i)    L y (f ) W k (y, y (1) , . . . , y (m) ) ∈ S (k) + , M k-gi (y (i) ) = M k-gi (g i y) , i ∈ [m] , M k-hj (h j y) = 0 , j ∈ [l]    , ( 5.3.35)
where W k (y, y (1) , . . . , y (m) ) := diag(M k (y), M k-g1 (y (1) ), . . . , M k-gm (y (m) )).

Let V = {M k (z) : z ∈ R s(2k) } and V i = {M k-gi (z) : z ∈ R s(2(k-gi )) }, i ∈ [m]. Then V and V i , i ∈ [m],
are the linear subspaces of the spaces of real symmetric matrices of size s(k) and s(k

-g i ), i ∈ [m], respectively. Denote by V ⊥ , V ⊥ i , i ∈ [m], the orthogonal complements of V, V i , i ∈ [m]
, respectively. In [134, Appendix A.2 ], we show how to take a basis { Âj } j∈[r] of V ⊥ . Similarly we can take a basis

{ Â(i) j } j∈[ri] of V ⊥ i , i ∈ [m]. Here r = dim(V ⊥ ) and r i = dim(V ⊥ i ), i ∈ [m].
Notice that if X 0 is a real symmetric matrix of size s(k), then X 0 = M k (y) for some y ∈ R s(2k) if and only if Âj , X 0 = 0, j ∈ [r]. It implies that if X = diag(X 0 , . . . , X m ) ∈ S (k) , then there exist y and y (i) , i ∈ [m], such that X = W k (y, y (1) , . . . , y (m) ) ⇔ Ā, X = 0 , Ā ∈ B 1 , where B 1 involves matrices Ā defined as • Ā = diag( Âj , 0, . . . , 0) for some j ∈ [r];

• Ā = diag(0, Â (1
) j , . . . , 0) for some j ∈ [r 1 ];

• . . .

• Ā = diag(0, 0, . . . , Â(m) j ) for some j ∈ [r m ].
Notice that

|B 1 | = r + i∈[m] r i = b(k)(b(k) + 1) 2 -b(2k) + i∈[m] b(k -g i )(b(k -g i ) + 1) 2 -b(2(k -g i )) . (5.3.36)
The constraints M k-gi (y

(i) ) = M k-gi (g i y), i ∈ [m], of SDP (5.3.35) are equivalent to y (i) α = γ∈N n 2 g i g i y α+γ , α ∈ N n 2(k-gi ) , i ∈ [m].
They can be written as Ā, W k (y, y (1) , . . . , y (m) ) = 0, for Ā ∈ B 2 , where B 2 involves matrices Ā defined by Ā = diag( Ã, 0, . . . , 0, Ã(i) , 0, . . . , 0), with à = ( õ,ν ) µ,ν∈N n k being defined as follows:

õ,ν =          g i,γ if µ = ν , µ + ν = α + γ , 1 2 g i,γ if µ = ν , (µ, ν) ∈ {(µ 1 , ν 1 ), (ν 1 , µ 1 )} with (µ 1 , ν 1 ) = minimal({( μ, ν) ∈ (N n k ) 2 : μ + ν = α + γ}) , 0 otherwise, (5.3.37)
and Ã(i) = ( Ã(i) µ,ν ) µ,ν∈N n k-g i being defined as follows:

Ã(i) µ,ν =          -1 if µ = ν , µ + ν = α , -1 2 if µ = ν , (µ, ν) ∈ {(µ 1 , ν 1 ), (ν 1 , µ 1 )} with (µ 1 , ν 1 ) = minimal({( μ, ν) ∈ (N n k ) 2 : μ + ν = α}) , 0 otherwise, (5.3.38) for some α ∈ N n 2(k-gi ) and i ∈ [m]. Notice that |B 2 | = i∈[m] b(2(k -g i ))
. Here minimal(T ) is the minimal element of T , for every T ⊆ N 2n with respect to the graded lexicographic order.

The constraints M k-hj (h j y) = 0, j ∈ [l], can be simplified as

γ∈N n 2 h j h j,γ y α+γ = 0, α ∈ N n 2(k-hj ) , j ∈ [l]
. They are equivalent to the following trace equality constraints: Ā, W k (y, y (1) , . . . , y (m) ) = 0 , Ā ∈ B 3 , where B 3 involves matrices Ā = diag( Ã, 0, . . . , 0), with à = ( õ,ν ) µ,ν∈N n k being defined as follows:

õ,ν =          h j,γ if µ = ν , µ + ν = α + γ , 1 2 h j,γ if µ = ν , (µ, ν) ∈ {(µ 1 , ν 1 ), (ν 1 , µ 1 )} with (µ 1 , ν 1 ) = minimal({( μ, ν) ∈ (N n k ) 2 : μ + ν = α + γ}) , 0 otherwise. 108CHAPTER 5. EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS Notice that |B 3 | = j∈[l] b(2(k -h j )). Let ∪ j∈[3] B j = ( Āi ) i∈[ζ k -1]
, where

ζ k = 1 + j∈[3] |B j | = 1 + b(k)(b(k) + 1) 2 -b(2k) + i∈[m] b(k -g i )(b(k -g i ) + 1) 2 + j∈[l] b(2(k -h j )) .
The final constraint y 0 = 1 can be rewritten as Āζ k , W k (y, y (1) , . . . , y (m) ) = 1 with Āζ k ∈ S (k) having zero entries except the top left one [ Āζ k ] 0,0 = 1. Thus we select real vector b k of length t k such that all entries of b k are zeros except the final one b k,ζ k = 1.

The function L y (f ) = γ f γ y γ is equal to C, W k (y, y (1) , . . . , y (m) ) with C := diag( C, 0, . . . , 0), where C = ( Cµ,ν ) µ,ν∈N n k is defined by

Cµ,ν =          f γ if µ = ν , µ + ν = γ , 1 2 f γ if µ = ν , (µ, ν) ∈ {(µ 1 , ν 1 ), (ν 1 , µ 1 )} with (µ 1 , ν 1 ) = minimal({( μ, ν) ∈ (N n k ) 2 : μ + ν = γ}) , 0 otherwise.
By writing X = W k (y, y (1) , . . . , y (m) ), SDP (5.3.35) has the standard form

τ k = inf X∈S (k) + { C, X : Ā X = b k } , (5.3.39)
where Ā : k) , by noting X = P k XP k , SDP (5.3.39) can be written as ( 5

S (k) → R ζ k is a linear operator of the form ĀX = Ā1 , X , . . . , Āζ k , X . Since U, V = P -1 k UP -1 k , P k VP k , for all U, V ∈ S (
.1.3) with A k,i = P -1 k Āi P -1 k , i ∈ [ζ k ],
and

C k = P -1 k CP -1 k .

The sparse case

Let k ∈ N ≥kmin be fixed. We will present a way to transform SDP (5. c) . From (5.3.7), the affine constraints of SDP (5.3.4) are now equivalent to AX = b.

α . For α ∈ ∪ c∈[p] N Ic k such that |T (α)| ≥ 2, for every c ∈ T (α), let Â(α) c ∈ S (c,k) be such that Â(α) c , X c = y α It implies the constraints Â(α) j0 , X j0 - Â(α) i , X i = 0, i ∈ T (α)\{j 0 }, for every α ∈ ∪ c∈[p] N Ic k such that |T (α)| ≥ 2, for some j 0 ∈ T (α). We denote by ÃX = 0 R ζ all these constraints with X = diag((X c ) c∈[p] ). Set ζ := c∈[p] ζc + ζ and b = [( bc ) c∈[p] , 0 R ζ ] ∈ R ζ . Define the linear operator A : c∈[p] S (c,k) → R ζ such that AX = [( Âc X c ) c∈[p] , ÃX], for all X = diag(X c ) c∈[p] ∈ c∈[p] S (
Let

A (i) := diag((A i,c ) c∈[p] ) ∈ c∈[p] S (c) , i ∈ [ζ], be such that AX = [ A (1) , X , . . . , A (ζ) , X ] , for all X = diag(X c ) c∈[p] ∈ c∈[p] S (c) . For every c ∈ [p], define A c : S (c) → R ζ as a linear operator of the form A c X := [ A 1,c , X , . . . , A ζ,c , X ]. Then AX = c∈[p] A c X c , for all X = diag(X c ) c∈[p] ∈ c∈[p] S (

Part II

Positivity certificates with denominators

Chapter 6

Polynomial optimization over noncompact semialgebraic sets

Most of the content of this chapter is from [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF].

In this chapter, we focus on polynomial optimization on noncompact semialgebraic sets by relying on positivity certificates involving denominators. The main motivation is to voluntarily avoid the big-ball trick which reduces the problem to the compact case. The big-ball "trick" is to simply assume that the global minimum is attained in some a priori known ball B(0, R) centered at zero of radius R > 0 potentially large. Therefore, by adding this additional constraint to the definition of the feasible set, one is back to the compact case. Why? This "trick" has definitely some merit since in some practical applications such an R can be sometimes determined with adhoc arguments. However, it is not satisfactory from a mathematical point of view. Indeed after one has found a minimizer x ∈ B(0, R), one is still left with the question: Is really x a global minimizer? Was R chosen sufficiently large? In other words, in doing so one does not obtain an certificate that x is a global minimizer. As we will see in this chapter, we deal with the challenge to adapt some certificates of positivity on noncompact sets already available in the literature, to turn them into a practical algorithm.

Let us recall the Positivstellensatz [START_REF] Putinar | Solving moment problems by dimensional extension[END_REF] of Putinar and Vasilescu in the following theorem:

1. Let f ∈ R[x] such that f > 0 on R n+1 \{0}. Then θ k f ∈ Σ[x] for some k ∈ N. 2. Let f, g 1 , . . . , g m ∈ R[x] satisfy the following two conditions: (a) f = f 0 + f 1 such that deg(f 0 ) < deg(f 1 ) and f1 > 0 on R n+1 \{0}; (b) f > 0 on S(g). Then θ 2k f ∈ Q(g) for some k ∈ N.
As a consequence, they also obtain: Corollary 6.1. Final remark 2]

) Let θ := 1 + x 2 2 . 1. Let f ∈ R[x] 2d be such that f ≥ 0 on R n . Then for all ε > 0, there exists k ∈ N such that θ k (f + εθ d ) ∈ Σ[x]. 2. Let f ∈ R[x] such that f ≥ 0 on S(g). Let d ∈ N such that 2d > deg(f ). Then for all ε > 0, there exists k ∈ N such that θ 2k (f + εθ d ) ∈ Q(g).
Marshall [START_REF] Marshall | Approximating positive polynomials using sums of squares[END_REF]Corollary 4.3] states a slightly more general result but with no explicit d, and Schweighofer [START_REF] Schweighofer | Iterated rings of bounded elements and generalizations of Schmüdgen's Positivstellensatz[END_REF]Corollary 6.3] provides a new algebraic proof of Marshall's result. To summarize, for every polynomial f nonnegative on a general basic semialgebraic set S(g), one obtains the following representation result: for a given ε > 0, there exist a nonnegative integer k and SOS polynomials σ 0 , σ 1 , . . . , σ m , such that

f + εθ d = σ 0 + σ 1 g 1 + • • • + σ m g m θ k . (6.0.1)
Although this representation is theoretically attractive, their previous proofs are not constructive and do not provide any explicit algorithm, especially in polynomial optimization.

Contribution. As already mentioned, our approach is to treat the noncompact case frontally and avoid the big-ball trick. Our contribution is threefold: I. In Section 6.1 we first provide an alternative proof of (6.0.1), with an explicit degree bound on the SOS weights, by relying on Jacobi's technique in the proof of [START_REF] Jacobi | A representation theorem for certain partially ordered commutative rings[END_REF]Theorem 7]; this is crucial as it has immediate implications on the algorithmic side. More precisely, the degrees of SOS weights σ i are bounded above by k + d -g i . First, one transforms the initial polynomials to homogeneous forms, then one relies on Putinar's Positivstellensatz for the compact case, and finally one transforms back the obtained forms to dehomogenized polynomials. As a consequence, with ε > 0 fixed, arbitrary, this degree bound allows us to provide hierarchies (ρ i k (ε)) k∈N , i = 1, 2, 3 for unconstrained polynomial optimization (m = 0 and i = 1, see Section 6.2.1) as well as for constrained polynomial optimization (m ≥ 1 and i = 2, 3, see Section 6.2.2). Computing each ρ i k (ε) boils down to solving a single SDP, with strong duality property. For k sufficiently large, ρ i k (ε) becomes an upper bound for the optimal value f of the corresponding polynomial optimization problem (POP) min x∈S(g) f (x). If this problem has an optimal solution x , the gap between ρ i k (ε) and f is at most εθ(x ) d . The related convergence rates are also analyzed in these sections.

II. In Section 6.2.3, we provide a new algorithm to find a feasible solution in the set S(g, h) defined in (1.1.1). The idea is to include appropriate additional spherical equality constraints ϕ t := ξ t -xa t 2 2 , t = 0, . . . , n, in S(g, h) so that the system S(g, h ∪ {ϕ 0 , . . . , ϕ n }) has a unique real solution. The nonnegative reals (ξ t ) t=0,...,n are computed with an adequate Moment-SOS hierarchy. Moreover, this solution might be extracted in certain cases by checking whether some (moment) matrix satisfies a flat extension condition.

III. Finally we use this method to approximate a global minimizer of f on S(g, h). Namely, we fix ε > 0 small and find a point in S(g ∪ {ρ i k (ε) -f }, h). This procedure works in certain cases, even if the set of minimizers is infinite. This is in deep contrast with the extraction procedure of [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] (via some flat extension condition) which works only for finite solution sets. Assuming that the set of solutions is finite, one may compare our algorithm with the procedure from [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] as follows. On the one hand, the latter extraction procedure provides global optimizers, provided that one has solved an SDP-relaxation with sufficiently large "k" (so as to get an appropriate rank condition). On the other hand, our algorithm that adds spherical equality constraints "divides" the problem into n + 1 SDP relaxations with additional constraints but with smaller order "k" (which is the crucial parameter for the SDP solvers). Numerical examples are provided in Section 6.3 to illustrate the difference between these two strategies.

For clarity of exposition, most proofs are omitted, they are available in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix].

Representation theorems

In this section we provide two exact representations of globally nonnegative polynomials and polynomials nonnegative on basic semialgebraic sets (not necessarily compact). The representations are obtained thanks to a perturbation argument as well as existing representations for positive definite forms. Let θ := 1 + x 2 2 .

Globally nonnegative polynomials

Let us note

q 1 := α |q α | for a given q ∈ R[x].
The following result provides a representation of globally nonnegative polynomials. Theorem 6.2. Let f ∈ R[x] 2d be nonnegative on R n . Then for every ε > 0, for k ε ∈ N and

k ε ≥ 2(n + 1)d(2d -1) 4 log 2 (ε -1 f 1 + 1) - n + 1 + 2d 2 ,
one has

θ kε f + εθ d ∈ Σ[x] kε+d . (6.1.1)
Proof. The proof consists of three steps:

1. Associate a positive definite form to the globally nonnegative polynomial f .

2. Use Reznick's representation from Theorem 1.2 to get a representation of this homogeneous form.

3. Transform back the homogeneous polynomial together with its representation to the original polynomial.

Let f = x 2d n+1 f (x/x n+1 ) be the degree 2d homogenization of f . Since f is globally nonnegative, f is nonnegative on R n+1 . Let ε > 0 be fixed. We claim that

f + ε (x, x n+1 ) 2d 2 ∈ R[x, x n+1 ]
is positive definite, i.e., is homogeneous and positive on R n+1 \{0}. Since

(x, x n+1 ) 2d 2 = (x 2 1 + • • • + x 2 n + x 2 n+1 ) d , the polynomial (x, x n+1 ) 2d
2 is homogeneous of degree 2d on R n+1 . From this and since f is homogeneous of degree 2d, f + ε (x, x n+1 ) 2d 2 is homogeneous of degree 2d. For every (x, x n+1 ) ∈ R n+1 \{0}, (x, x n+1 ) 2 > 0. From this and since f is nonnegative on R n+1 , f (x, x n+1 ) + ε (x, x n+1 ) 2d 2 > 0 , for all (x, x n+1 ) ∈ R n+1 \{0}. In addition, it is not hard to show that

inf{ f (x, x n+1 ) + ε (x, x n+1 ) 2d 2 : (x, x n+1 ) ∈ S n } ≥ ε and sup{ f (x, x n+1 ) + ε (x, x n+1 ) 2d 2 : (x, x n+1 ) ∈ S n } ≤ sup{ f (x, x n+1 ) : (x, x n+1 ) ∈ S n } + ε ≤ f 1 + ε . Thus, δ( f + . 2d 2 ) ≤ ( f 1 + ε)/ε = ( f 1 + ε)/ε.
From this and by applying Theorem 1.2 with

p = f + ε (x, x n+1 ) 2d 2 , for k ε ∈ N and k ε ≥ 2(n + 1)d(2d -1) (4 log 2) (ε -1 f 1 + 1) - n + 1 + 2d 2 , there exists σε ∈ Σ[x, x n+1 ] kε+d such that (x, x n+1 ) 2kε 2 ( f + ε (x, x n+1 ) 2d
2 ) = σε .

By replacing x n+1 by 1, one has

θ kε (f + εθ d ) = σε (x, 1) .
Let us note σ ε (x) := σε (x, 1), for every x ∈ R n . Since σε ∈ Σ[x, x n+1 ] kε+d , it follows that σ ε ∈ Σ[x] kε+d , yielding the desired result.

Polynomials nonnegative on a basic semialgebraic set

We recall the definition of the truncated quadratic module of order d associated with S(g):

Q d (g) :=    σ 0 + m j=1 σ i g i : σ 0 ∈ Σ[x] d , σ i ∈ Σ[x] d-gi    .
For every q ∈ R[x], let us define d 1 (q) := 1 + deg(q)/2 and d 2 (q) := deg(q)/2 = q .

The following result provides a degree bound for the SOS weights of [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF]Theorem 1].

Theorem 6.3. Let g = {g 1 , . . . , g m } ⊂ R[x] and f ∈ R[x]
such that f is nonnegative on S(g). Let ε > 0 and d ∈ N be such that at least one of the following two conditions is satisfied:

(i) d ≥ d 1 (f ); (ii) d ≥ d 2 (f ) and g m := f + λ for some real λ ≥ 0.
Then there exist k ε ∈ N such that

θ kε f + ε θ d ∈ Q kε+d (g) . (6.1.2)
The detailed proof of Theorem 6.3 relies on Jacobi's technique in his proof of [85, Theorem 7] and is available in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix]. This proof consists of three steps:

1. Associate a homogeneous polynomial f to the polynomial f . 2. Use Putinar's Positivstellensatz (Theorem 1.1 (i)) to obtain a representation of f . 3. Transform back the representation of f to obtain a representation of f . Remark 6.1. Theorem 6.3 is an extension of Putinar's Positivstellensatz to (possibly) noncompact sets S(g), and so does not require the Archimedean condition. The price to pay for such an extension is the presence of the multiplier θ kε in front of f and the perturbation term ε θ d . Note that (ii) involves a tighter bound for d, compared to (i), since d 1 (f ) ≥ d 2 (f ). The counterpart is that (ii) requires to include the additional constraint f + λ ≥ 0, for some λ ≥ 0. Assume that all assumptions of Theorem 6.3 hold and 0 R n ∈ S(g). Then there exist real numbers c j > 0 depending on g such that for all k ε ∈ N satisfying

Complexity of

k ε ≥ c 1 d c2 ε -1 f c3 -d , (6.1.3) one has θ kε f + ε θ d ∈ Q kε+d (g).
The proof of Proposition 6.1 is similar to [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix].

Discussion about the ε parameter. The (arbitrary small) positive parameter ε in Theorem 6.2 and Theorem 6.3 ensures the positivity of polynomials over the respective considered domain R n or S(g), excluding the origin in the homogenized representations. However these representations can still hold, even when ε = 0, as illustrated in the following two examples:

Example 6.1. (i) Motzkin's polynomial f = x 4 1 x 2 2 + x 2 1 x 4 2 + 1 -3x 2 1 x 2 2 is globally nonnegative but not SOS. However, θf is SOS since θf = 2( 1 2 x 3 1 x 2 + 1 2 x 1 x 3 2 -x 1 x 2 ) 2 + (x 2 1 x 2 -x 2 ) 2 + (x 1 x 2 2 -x 1 ) 2 + 1 2 (x 3 1 x 2 -x 1 x 2 ) 2 + 1 2 (x 1 x 3 2 -x 1 x 2 ) 2 + (x 2 1 x 2 2 -1) 2 . (ii) Let f = x 2 1 + x 2 2 x 2 1 x 2 2 -3x 2 1 x 2 2 and g = x 2 1 +x 2 2 -4. It is not hard to show that f is nonnegative on the noncompact set S(g). Moreover, f = 1 4 x 2 1 x 2 2 x 2 1 + x 2 2 + 3 4 x 2 1 x 2 2 g. Thus, θ 0 f ∈ Q 3 (g).
However, the certificate (6.1.1) for global nonnegativity with ε = 0 is not true in general, as shown in the following lemma. This is due to the fact that any SOS multiplier for Delzell's polynomial must have a zero at the "bad point", so one can not find any globally positive multiplier. Lemma 6.1. The nonnegative dehomogenized Delzell's polynomial [START_REF] Delzell | Continous, piecewise-polynomial functions which solve Hilbert's 17th problem[END_REF]:

f = x 4 1 x 2 2 + x 4 2 x 2 3 + x 2 1 x 4 3 -3x 2 1 x 2 2 x 2 3 + x 8 3 satisfies that θ k f / ∈ Σ[x] for all k ∈ N.
Proof. Assume by contradiction that θ K f ∈ Σ[x] for some K ∈ N. Note that n = 3 here. We denote by f the degree 8 homogenization of f , i.e.,

f = x 2 4 (x 4 1 x 2 2 + x 4 2 x 2 3 + x 2 1 x 4 3 -3x 2 1 x 2 2 x 2 3 ) + x 8 3 . Then (x, x n+1 ) 2K 2 f ∈ Σ[x, x n+1 ].
As shown in [START_REF] Reznick | Some concrete aspects of hilbert's 17th problem[END_REF]Section 6], it is impossible. This contradiction yields the conclusion.

The certificate (6.1.2) for global nonnegativity on basic semialgebraic sets with ε = 0 is also not true in general, as shown in the following lemma:

Lemma 6.2. With n = 1, let f = x and g = {x 3 , -x 3 }. Then f = 0 on S(g) = {0}. It follows that f is nonnegative on S(g), but: (i) θ k f / ∈ Q(g) for all k ∈ N and; (ii) for every ε > 0, θ k (f + εθ) ∈ Q k+1 (g) for all k ∈ N with k ≥ max{2 , ε -2 /4 -1}.
Proof. We will show statement (i). Assume by contradiction that there exists k ∈ N such that θ k f ∈ Q(g). Then there exists q j (x) ∈ R[x], j = 0, . . . , r such that

θ k f = m j=1 q j (x) 2 + q 0 (x)x 3 .
Assume that q j (x) = a j + b j x +

θ k (f + εθ) = (1 + kx 2 + x 4 e(x))(ε + x + εx 2 ) = ε + x + ε(k + 1)x 2 + x 3 q(x) , for some q ∈ R[x] 2k-2 . Assume that k ≥ ε -2 /4 -1. Then θ k (f + εθ) = ε - 1 4ε(k + 1) + x ε(k + 1) + 1 2 ε(k + 1) 2 + x 3 q(x) ∈ Q k+1 (g) .
From Lemma 6.1 and Lemma 6.2, we conclude that the strict positivity of the ε parameter is necessary in general although the certification with ε = 0 may happen in many cases.

When the certificate (6.1.1) with ε = 0 occurs, one has the following remark about the exponent of θ in (6.1.1). Remark 6.2. If n = 2, there does not exist a fixed K ∈ N such that for all nonnegative f ∈ R[x] 6 , θ K f is SOS. Indeed, assume by contradiction that there exists such a K. Then, the degree 6 homogenization f of f would be a positive ternary sextic such that (x, x n+1 ) 2K 2 f is SOS. By using [START_REF] Reznick | On the absence of uniform denominators in Hilbert's 17th problem[END_REF]Theorem 1] and the fact that the homogeneous Motzkin's polynomial is a positive ternary sextic which is not SOS, we obtain a contradiction.

In (6.1.1) with ε = 0, the multiplier θ kε can be replaced with other SOS multipliers; see, e.g., Leep-Starr's polynomial [START_REF] Leep | Polynomials in R[X,Y] that are sums of squares in R(X,Y)[END_REF]Example 2]. With the multiplier θ := x 2 1 + x 2 2 + x 2 3 and the homogenized Delzell's polynomial D, Schabert provides in [START_REF] Schabert | Uniform denominators in Hilbert's 17th Problem Theorems by Pólya and Reznick[END_REF]Example 4.4] the exact SOS decomposition of the product Θ(x 1 , x 2 , x 3 )D(x 1 , x 2 , x 3 , x 4 ).

Polynomial optimization

In this section, we exploit the two representations from Theorem 6.2 and Theorem 6.3 to construct new hierarchies of semidefinite programs for POPs of the form f = inf{f (x) : x ∈ Ω} where Ω = R n for the unconstrained case and Ω = S (g) for the constrained case (with no compactness assumption), respectively. Instead of solving the original problem, we are rather interested in the perturbed problem:

f ε := inf{f (x) + ε θ(x) d : x ∈ Ω} , (6.2.1)
where ε > 0 is fixed, θ(x) := 1 + x 2 2 , and 2d ≥ deg (f ). Now, assume that the optimal value f of the original problem is attained at some

x ∈ Ω. It is not difficult to show that if Ω is unbounded, the polynomial f + ε θ d is coercive on Ω, i.e. lim x∈Ω , x 2→∞ f (x) + εθ(x) d = ∞ ,
(see more in [START_REF] Bajbar | Coercive polynomials and their Newton polytopes[END_REF]). Indeed, it is due to the fact that f is bounded from below by f on Ω and θ(x) d → ∞ as x 2 → ∞. Thus, the optimal value f ε of the perturbed problem (6.2.1) is always attained at some global minimizer x ε even if Ω is noncompact. Then:

f + ε θ(x ) d = f (x ) + ε θ(x ) d ≥ f ε = f (x ε ) + ε θ(x ε ) d ≥ f (x ε ) ≥ f . Thus, f ∈ [f ε -ε θ(x ) d , f ε ], i.e.
, f ε is a perturbation of f and the gap between both of them is at most ε θ(x ) d . Next, observe that:

f ε = sup{λ ∈ R : f + ε θ d -λ ≥ 0 on Ω} = sup{λ ∈ R : θ k (f + ε θ d -λ) ≥ 0 on Ω}, k ∈ N .
The following hierarchies are based on the simple idea of replacing constraint "θ k (f + εθ d -λ) ≥ 0 on Ω" by relaxed constraint"θ k (f + εθ d -λ) is in the truncated quadratic module associated with Ω".

Unconstrained case

Given f ∈ R[x] 2d , let us consider the following problem:

f := inf x∈R n f (x) . (6.2.2)
In the sequel, we assume that f > -∞ and let ε > 0 be fixed. Consider the hierarchy of semidefinite programs indexed by k ∈ N:

τ 1 k (ε) := inf L y (θ k (f + εθ d )) s.t. y = (y α ) α∈N n 2(d+k) ⊂ R , M k+d (y) 0 , L y (θ k ) = 1 .
(6.2.3) Theorem 6.4. For every k ∈ N, the dual of (6.2.3) reads:

ρ 1 k (ε) := sup{λ ∈ R : θ k (f -λ + εθ d ) ∈ Σ[x] k+d } . (6.2.4)
The following statements hold:

1. The sequence (ρ 1 k (ε)) k∈N is monotone non-decreasing. 2. Assume that f in (6.2.2) is attained at x ∈ R n . Then there exists K ∈ N such that f ≤ ρ 1 k (ε) ≤ f + ε θ(x ) d for all k ≥ K.
In particular, K is upper bounded by O(ε -1 ) as ε ↓ 0.

Proof.

1. Let k ∈ N and fix ε > 0, arbitrary. By (6.2.4), there exists a real λ such that

ρ 1 k (ε) -ε ≤ λ and θ k (f -λ + εθ d ) ∈ Σ[x] k+d . Since θ ∈ Σ[x] 1 , θ k+1 (f -λ + εθ d ) ∈ Σ[x] k+d+1 . By (6.2.4), ρ 1 k+1 (ε) ≥ λ ≥ ρ 1 k (ε) -ε. This implies that ρ 1 k+1 (ε) ≥ ρ 1 k (ε). 2. By (6.2.2), f -f is nonnegative. By Theorem 6.2, there exists K ∈ N and K = O(ε -1 ) as ε ↓ 0 such that θ K (f -f + εθ d ) ∈ Σ[x] K+d . Let k ≥ K be fixed. Since θ ∈ Σ[x] 1 , one has θ k (f -f + εθ d ) = θ K+(k-K) (f -f + εθ d ) ∈ Σ[x] k+d . By (6.2.4), f ≤ ρ 1 k (ε). Thus, f ≤ ρ 1 k (ε) for all k ≥ K.
Let k ∈ N and fix ε > 0, arbitrary. By (6.2.4), there exists a real λ such that

ρ 1 k (ε) -ε ≤ λ and θ k (f -λ + εθ d ) ∈ Σ[x] k+d . It follows that f -λ + εθ d ≥ 0 on R n . From this, f + εθ(x ) d = f (x ) + εθ(x ) d ≥ λ ≥ ρ 1 k -ε . This implies f + εθ(x ) d ≥ ρ 1 k (ε), the desired result.
We guarantee strong duality for previous primal-dual problems: [START_REF] Boyd | Convex optimization[END_REF]Section 5.2.3], it suffices to show that (6.2.3) admits a strictly feasible solution. Let us denote by µ the measure with density χ [0,1] n θ -k with respect to the Lebesgue measure, where χ A is the characteristic function of a given set A ⊂ R n . Set y α := x α dµ for all α ∈ N n . We claim that y α ∈ R for all α ∈ N n , L y (θ k ) = 1 and M k+d (y) 0. Indeed, for all α ∈ N n

Proposition 6.2. Let k ∈ N. Then τ 1 k (ε) = ρ 1 k (ε). Moreover, if τ 1 k (ε) > -∞ then the optimal value ρ 1 k (ε) is attained. Proof. By Slater's constraint qualification
|y α | = | x α χ [0,1] n θ -k dx| = | [0,1] n x α θ -k dx| ≤ [0,1] n |x 1 | α1 . . . |x n | αn θ -k dx ≤ 1 , since θ -k ≤ 1. Thus, y α ∈ R for all α ∈ N n . In addition, L y (θ k ) = θ k χ [0,1] n θ -k dx = [0,1] n dx = 1 .
Let p ∈ R s(d+k) \{0} be fixed. We state that p M d+k (y)p > 0. Assume by contradiction that p M d+k (y)p ≤ 0. One has

0 ≥ p M d+k (y)p = p v d+k v d+k pdµ = p v d+k v d+k pχ [0,1] n θ -k dx = [0,1] n (p v d+k ) 2 θ -k dx ≥ 0 .
It follows that p v d+k = 0 on [0, 1] n , thus p = 0 yielding a contradiction. From this, (y α ) α∈N n d+k is a feasible solution of (6.2.3) with M k+d (y) 0. By strong duality, the conclusion follows.

118CHAPTER 6. POLYNOMIAL OPTIMIZATION OVER NONCOMPACT SEMIALGEBRAIC SETS

Constrained case

Consider the following problem:

f := inf x∈S(g) f (x) , (6.2.5)
where

f ∈ R[x], g = {g 1 , . . . , g m } ⊂ R[x].
Assume that S(g) = ∅ and f > -∞. Let ε > 0 be fixed.

Unknown lower bound

Let d := deg(f )/2 + 1 and consider the hierarchy of semidefinite programs indexed by k ∈ N:

τ 2 k (ε) := inf L y (θ k (f + εθ d )) s.t. y = (y α ) α∈N n 2(d+k) ⊂ R , M k+d (y) 0 , M k+d-gi (g i y) 0, i ∈ [m] , L y (θ k ) = 1 .
(6.2.6) Theorem 6.5. For every k ∈ N, the dual of (6.2.6) reads:

ρ 2 k (ε) := sup{λ ∈ R : θ k (f -λ + ε θ d ) ∈ Q k+d (g)} . (6.2.7)
The following statements hold:

1. The sequence (ρ 2 k (ε)) k∈N is monotone non-decreasing.

2. Assume that problem (6.2.5) has an optimal solution x . Then there exists

K ∈ N such that f ≤ ρ 2 k (ε) ≤ f + ε θ(x ) d for all k ≥ K.
In particular, K is upper bounded by O(ε -c ) -d as ε ↓ 0, for some c > 0 depending on g.

The proof of Theorem 6.5 relies on Theorem 6.3 (i) and is similar to Theorem 6.4. The upper bound on K is based on Proposition 6.1.

We guarantee strong duality for previous primal-dual problems:

Proposition 6.3. There exists K ∈ N such that τ 2 k (ε) = ρ 2 k (ε) for all k ≥ K. Moreover, if τ 2 k (ε) > -∞, the optimal value ρ 2 k (ε) is attained.
The proof of Proposition 6.3 can be found in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix].

Remark 6.3. If S(g) has nonempty interior then strong duality holds for all orders k of the primaldual problems (6.2.6)-(6.2.7). Indeed, by constructing a sequence of moments from the Lebesgue measure on an open ball contained in S(g), one can find a strictly feasible solution of (6.2.6) and then apply Slater's constraint qualification [START_REF] Boyd | Convex optimization[END_REF]Section 5.2.3].

Known lower bound

Assume that g m := f -f for some real f ≤ f and let d := deg(f )/2 . We then obtain the same conclusion as Theorem 6.5 with replacing here τ 2 k (ε) and ρ 2 k (ε) by τ 3 k (ε) and ρ 3 k (ε), respectively. The proof relies on Theorem 6.3 (ii) and is similar to Theorem 6.4. Note that here g m = (f -f ) + (f -f ) with f -f ≥ 0 on S(g) and f -f ≥ 0. The upper bound on K is also based on Proposition 6.1.

The next proposition states that strong duality is guaranteed for each relaxation order k.

Proposition 6.4. Let k ∈ N. Then τ 3 k (ε) = ρ 3 k (ε). Moreover, if τ 3 k (ε) > -∞ then the optimal value ρ 3 k (ε) is attained.
The proof of Proposition 6.4 can be found in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix].

Remark 6.4.

A lower bound f of problem (6.2.5) can be obtained by solving the following SDP:

sup{λ ∈ R : θ k (f -λ) ∈ Q k+d (g)} , k ∈ N .
Assume that we know a lower bound f of problem (6.2.5). By adding the inequality constraint f -f ≥ 0 in S(g), we obtain the same semialgebraic set, i.e. S(g ∪ {f -f }) = S(g). Thus, the two problems inf{f (x) : x ∈ S(g)} and inf{f (x) : x ∈ S(g ∪ {f -f })} are identical and the primal-dual SDP relaxations with values ρ 3 k (ε) and τ 3 k (ε) of the latter one satisfy strong duality for each relaxation order k. General case. Since g i ≥ 0 on S(g) and -g i ≥ 0 on S(g) is equivalent to g i = 0 on S(g), S(g) can be rewritten as S(g, h) with g = {g 1 , . . . , g m } is the set of polynomials involved in the inequality constraints and h = {h 1 , . . . , h l } is the set of polynomials involved in the equality constraints; in addition, R n = S({0}, ∅). Consider the general POP:

f := inf x∈S(g,h) f (x) , (6.2.8)
with f ∈ R and define

(d, i) =    ( deg(f )/2 , 1) if S(g, h) = R n , (1 + deg(f )/2 , 2) if S(g, h) = R n and lower bound f is unknown , ( deg(f )/2 , 3) otherwise and set g m := f -f .
For fixed ε > 0, one considers the following SDP relaxation of POP (6.2.8)

τ i k (ε) := inf L y (θ k (f + εθ d )) s.t. y = (y α ) α∈N n 2(d+k) ⊂ R , M k+d-gi (g i y) 0 , i = 0, . . . , m , M k+d-hj (h j y) = 0 , j ∈ [l] , L y (θ k ) = 1 , (6.2.9)
where g 0 := 1. Its dual is the semidefinite program:

ρ i k (ε) := sup{λ ∈ R : θ k (f -λ + εθ d ) ∈ Q k+d (g, h)} (6.2.10)
The zero-duality gap between SDP (6.2.9) and SDP (6.2.10) is guaranteed for large enough k.

Remark 6.5. The condition τ i k (ε) > -∞ is always satisfied whenever k is sufficently large. Indeed by weak duality, when ε is fixed and k is sufficiently large then

τ i k (ε) ≥ ρ i k (ε) ≥ f > -∞.
However, when k is small, τ i k (ε) = -∞ may happen. Let us now assume that the POP (6.2.8) has an optimal solution x . Then

ρ i k (ε) ∈ [f , f + εθ(x ) d ]
when ε > 0 is fixed and k is sufficiently large. Moreover, the gap between ρ i k (ε), and f is at most ε θ(x ) d . Therefore, ρ i k (ε) is indeed an approximation of f . In practice, (ρ i k (ε)) k∈N often converges to the optimal value f ε := min{f (x) + ε θ(x) d : x ∈ S(g, h)} after finitely many steps (see Section 6.3). Remark 6.6. (i) The term θ d in both (6.2.9) and (6.2.10) can be replaced by ϕ d (x, 1) where ϕ d : R n+1 → R is a positive form of degree 2d. For instant, one can select

ϕ d (x, 1) = x 2d 1 + • • • + x 2d n + 1.
(ii) Let r ∈ N be fixed. For every k divisible by 2r, the term θ k appearing in both (6.2.9) and (6.2.10) can be replaced by ψ r (x, 1) k/(2r) where ψ r : R n+1 → R is a coercive positive form of degree 2r. For instant, one can select ψ r (x, 1) =

x 2r 1 + • • • + x 2r n + 1.

Relation between classical optimality conditions and nonnegativity certificates.

When ρ i k (0) = f , the constraint qualification conditions hold at x . Proposition 6.5. Assume that ρ i k (0) = f for some k ∈ N, i.e., there exists

σ i ∈ Σ[x], i = 0, . . . , m and φ j ∈ R[x], j ∈ [l] such that θ k (f -f ) = σ 0 + m i=1 σ i g i + l j=1
φ j h j . Then the constraint qualification conditions hold at x :

120CHAPTER 6. POLYNOMIAL OPTIMIZATION OVER NONCOMPACT SEMIALGEBRAIC SETS 1. σ i (x ) ≥ 0 and g i (x ) ≥ 0, for all i ∈ [m]; 2. σ i (x )g i (x ) = 0, for all i ∈ [m]; 3. θ(x ) k ∇f (x ) = m i=1 σ i (x )∇g i (x ) + l j=1 φ j (x )∇h j (x ).
The proof of Proposition 6.5 is similar to [START_REF] Lasserre | An introduction to polynomial and semi-algebraic optimization[END_REF]Theorem 7.4]. If we take an arbitrary small ε > 0 then ρ i k (ε) is arbitrary close to f for large enough k. However, if one sets ε = 0, the statement "ρ i K (0) = f for some K ∈ N" is not true in general as stated in the following proposition: Proposition 6.6. If the first order optimality condition fails at a global minimizer of problem (6.2.8), then ρ i k (0) < f for all k ∈ N.

The proof of Proposition 6.6 is similar to [START_REF] Nie | Optimality conditions and finite convergence of Lasserre's hierarchy[END_REF]Proposition 3.4]. By applying Proposition 6.6 to POP min{x : x 3 = 0}, we obtain the statement (i) of Lemma 6.2. Indeed, the first order optimality condition fails at the global minimizer 0 of this problem. Therefore, the positivity of ε ensures convergence of (ρ i k (ε)) k∈N to the neighborhood [f , f + εθ(x ) d ] of the optimal value f . As proved by Huang, Nie and Yuan in [START_REF] Huang | Homogenization for polynomial optimization with unbounded sets[END_REF], ρ i K (0) = f for some K ∈ N when some optimality conditions hold at every global minimizer of (6.2.8). In many cases, ρ i K (0) = f with K = 0, 1 when the KKT conditions hold (see Example 6.1 and [146, Example 4.4] with x n = 1). However, the KKT conditions are not enough for this conjecture due to the fact that the minimizer x = (0, 0, 0) of dehomogenized Delzell's polynomial in Lemma 6.1 satisfies the KKT conditions and ρ i k (0) < f for all k ∈ N in this case.

Reducing the noncompact case to a compact case. Consider the POP: f := inf{f (x) :

x ∈ S(g, h)} where the feasible set S(g, h) is possibly noncompact, and the associated perturbed POP:

f ε := inf{f (x) + εθ(x) d : x ∈ S(g, h)} with fixed ε > 0.
Here one assumes that f is attained at x and 2d > deg(f ). As in Section 6.

2, f ∈ [f ε -ε θ(x ) d , f ε ]
. Suppose that a point x in S(g, h) is known. It is not hard to show that f + εθ d is coercive and therefore with C := f (x) + εθ(x) d , the set S({C -f -εθ d }) is compact. Moreover,

f ε = inf{f (x) + εθ(x) d : x ∈ S(g ∪ {C -f -εθ d }, h)} . (6.2.11)
Note that the quadratic module associated with the constraint set of POP (6.2.11) is Archimedean and so f ε can be approximated as close as desired by the Moment-SOS hierarchy. This approach is similar in spirit to that of [START_REF] Jeyakumar | On polynomial optimization over non-compact semi-algebraic sets[END_REF]. However, determining a point x in S(g, h) is not easy in general. The hierarchy (6.2.10) relying on Putinar-Vasilescu's Positivstellensatz goes beyond this restriction. In a different way, if one relies on the big ball trick, we consider the following POP, for all ε > 0:

f * ε := inf{f (x) : x ∈ S(g ∪ {1 -ε x 2 2 }, h)} . (6.2.12)
Obviously, one has f * ε ↓ f as ε ↓ 0. The quadratic module associated with the constraint set of POP (6.2.12) is Archimedean and f ε can be approximated by the Moment-SOS hierarchy:

ρ k (ε) := sup{λ ∈ R : f -λ ∈ Q(g ∪ {1 -ε x 2 2 }, h)} , k ∈ N , ε > 0 . (6.2.13)
Thus the SOS weight of 1-ε x 2 2 appears in the SOS decomposition of f -λ. However, the hierarchy (6.2.10) relying on Putinar-Vasilescu's Positivstellensatz does not require such SOS weight in the decomposition of f -λ, but requires a perturbation εθ d and a multiplier θ k .

Global optimizers

In this section we introduce a new method to find an approximation of a feasible point of a basic semialgebraic set S(g, h) as defined in (1.1.1). We then apply this method to obtain an approximation of a global minimizer x associated to f = min{f (x) : x ∈ S(g, h)} via finding a feasible solution of S(g

∪ {ρ i k (ε) -f }, h).
Remark 6.7. Let ε > 0 be fixed and k ∈ N be sufficiently large such that ρ i k (ε)is an upper bound of f . Let x be a global minimizer of f on S(g, h) and let x ∈ S(g

∪ {ρ i k (ε) -f }, h). Then x ∈ S(g, h) and f ≤ f (x) ≤ ρ i k (ε) ≤ f + εθ(x ) d .
Let us consider an arbitrary small ε > 0. The difference between ρ i k (ε) and f will be as close as desired to εθ(x ) d for large enough k. Assume that the solution set S(g ∪ {f -f }, h) is finite and denote by y ε an optimal solution of SDP (6.2.9). In practice, when k is sufficiently large, y ε satisfies numerically the flat extension condition defined in Section 2.5. One may then use the algorithm of Henrion and Lasserre [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] to extract numerically the support of a representing measure for y ε which may include global minimizers of f = min{f (x) : x ∈ S(g, h)} (see the same extraction in [START_REF] Jibetean | Semidefinite approximations for global unconstrained polynomial optimization[END_REF]Section 3.2]). However we cannot guarantee the success of this extraction procedure in theory because the set S(g

∪ {ρ i k (ε) -f }, h) may not be zero dimensional when ρ i k (ε) > f . For example, if f = x 2 2 and S(g, h) = R n , S(g ∪ {ρ i k (ε) -f }, h
) is a closed ball centered at the origin with radius ρ i k (ε) 1/2 . The following method aims at overcoming this issue from both theoretical and algorithmic sides. For further application cases of the flat moment criterion, we refer the interested reader to the framework from [START_REF] Quijorna | An experimental approach for global polynomial optimization based on moments and semidefinite programming[END_REF], which is based on altering the bottom right part of the moment matrix.

The Adding-Spherical-Constraints method (ASC): For a ∈ R n and r ≥ 0, let B(a, r) (resp. ∂B(a, r)) be the closed ball (resp. sphere) centered at a with radius r, i.e.,

B(a, r) = {x ∈ R n : x -a 2 ≤ r} (resp. ∂B(a, r) = {x ∈ R n : x -a 2 = r}) .
The following result provides an efficient way to find a sequence of additional spherical equality constraints for a given semialgebraic set such that (i) the resulting set is a singleton (i.e. it contains a single real point), and (ii) this point is solution of a non-singular system of linear equations.

Lemma 6.3. Assume that S(g, h) = ∅. Let (a t ) t=0,1,...,n ⊂ R n such that a t -a 0 , t ∈ [n] are linearly independent in R n . Let us define the sequence (ξ t ) t=0,1,...,n ⊂ R + as follows:    ξ 0 := min{ x -a 0 2 2 : x ∈ S(g, h)} , ξ t := min{ x -a t 2 2 : x ∈ S(g, h ∪ {ξ j -x -a j 2 2 : j = 0, . . . , t -1})} , t ∈ [n] . ( 6

.2.14)

Then there exists a unique real point x in S(g, h ∪ {ξ t -xa t 2 2 : t = 0, . . . , n}) which satisfies the non-singular linear system of equations

(a t -a 0 ) x = - 1 2 (ξ t -ξ 0 -a t 2 2 + a 0 2 2 ), t ∈ [n] . (6.2.15)
The proof of Lemma 6.3 is available in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix]. Geometrically speaking, we find a sequence of spheres ∂B(a t , ξ 1/2 t ), t = 0, . . . , n, such that the intersection between these spheres and S(g, h) is the singleton {x } (see Figure 6.1). Next, we use Lasserre's hierarchy to compute the optimal values ξ t , t = 0, . . . , n of problem (6.2.14). Theorem 6.6. Assume that S(g, h) ∩ B(0, R 1/2 ) = ∅ for some R > 0. Let (a t ) t=0,1,...,n ⊂ R n such that a t -a 0 , t ∈ [n] are linearly independent in R n . Assume that the Moment-SOS hierarchies associated with the following POPs:

a 0 a 1 a 2 x

S(g, h)

   ξ 0 := min{ x -a 0 2 2 : x ∈ S(g ∪ {R -x 2 2 }, h)} , ξ t := min{ x -a t 2 2 : x ∈ S(g, h ∪ {ξ j -x -a j 2 2 : j = 0, . . . , t -1})} , t ∈ [n] , (6.2.16)
have finite convergence, and let w := max{ g i , h j , 1}. For every k ∈ N, consider the following semidefinite programs:

                                         η 0 k := inf y∈R s(2(k+w)) L y ( x -a 0 2 2 ) s.t. M k+w-gi (g i y) 0 , M k+w-1 ((R -x 2 2 )y) 0 , M k+w-hj (h j y) = 0 , y 0 = 1 , η t k := inf y∈R s(2(k+w)) L y ( x -a t 2 2 ) s.t. M k+w-gi (g i y) 0 , M k+w-hj (h j y) = 0 , M k+w-1 ((η j k -x -a j 2 2 )y) = 0 , j = 0, . . . , t -1 , y 0 = 1 , t ∈ [n] .
(6.2.17)

Then there exists K ∈ N such that for all k ≥ K, η t k = ξ t , t = 0, . . . , n. Moreover, there exist t ∈ {0, . . . , n} and K ∈ N such that for all k ≥ K, the solution y of SDP (6.2.17) with value η t k satisfies the flat extension condition, i.e., rank (M k+w (y)) = rank (M k (y)). In addition, y has a representing rank (M k (y))-atomic measure µ and supp (µ) ⊂ S (g, h).

The proof of Theorem 6.6 is available in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Appendix]. Remark 6.8. The Moment-SOS hierarchy of each POP (6.2.16) has finite convergence when one of the following conditions is satisfied:

1. (Lasserre [START_REF] Lasserre | An introduction to polynomial and semi-algebraic optimization[END_REF]Theorem 7.5]) The ideal I(h) is real radical, and the second-order sufficient condition holds at every global minimizer of each POP in (6.2.16). [START_REF] Lasserre | An introduction to polynomial and semi-algebraic optimization[END_REF]Theorem 6.13]) The real variety V (h) (= S(∅, h)) is finite. Remark 6.9. In the final conclusion of Theorem 6.6, when y has a representing rank(M k (y))atomic measure µ, we may use the extraction algorithm from [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF] to obtain the atomic support of µ.

(Lasserre et al. [108, Proposition 1.1] and

Based on Theorem 6.6, Algorithm 6.2.3 below finds a feasible point in a nonempty (possibly noncompact) semialgebaric set S (g, h). Proposition 6.7. Let the assumptions of Theorem 6.6 hold. For k sufficiently large, Algorithm 6.

terminates and x ∈ S(g, h).

Proof. The proof follows from Theorem 6.6 and Remark 6.9.

In Algorithm 6.2.3, step 1 computes the radius L 1/2 of the ball B(0, L 1/2 ) which has non-empty intersection with S(g, h). Then step 2 checks the flat extension condition and extracts the solution x. Remark 6.10. At step 2 in Algorithm 6.2.3, for k sufficiently large, the rank of the moment matrix rank(M k+w (y)) decreases to one when t goes from 0 to n. Indeed, for each t between 0 and n, we replace the semialgebraic set S(g, h) by its intersection with the t spheres ∂B(a j , ξ 1/2 j ), j = 0, . . . , t -1. This intersection includes the support of the measure with moments y. Since S(g, h) ∩ n j=0 ∂B(a j , ξ 1/2 j ) = {x }, this support converges to {x } when t goes from 0 to n. Thus for large enough k, the solution y of SDP (6.2.17) with value η n k has a representing measure supported on x = (y e1 , . . . , y en ). Here e i , i ∈ [n] is canonical basis of R n .

EXAMPLES 123

Algorithm 12 PolySys

Input: S(g, h) = ∅, (a t ) t=0,1,...,n ⊂ R n such that a t -a 0 , t ∈ [n] are linearly independent, ε > 0 and k ∈ N. Output: x. Begin with t := 0 and do:

1: Solve SDP (6.2.10) with f = x 2 2 to obtain ρ i k (ε). Set L := ρ i k (ε) and go to step 2. 2: Solve SDP (6.2.17) to obtain η t k and an associated solution y. a: If t ≤ n and rank (M k+w (y)) = rank (M k (y)), i.e., y has a representing measure µ, extract supp(µ) from y by using the algorithm from [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF]. Take x ∈ supp(µ) and stop. b: If t ≤ n and rank (M k+w (y)) = rank (M k (y)), set t := t + 1 and do again step 2. c: If t = n + 1, stop. The decrease of the moment matrix rank in Algorithm 6.2.3 for the kissing number problem with

g 1 = x 2 1 + x 2 2 + x 2 3 + x 2 4 -2x 1 x 3 -2x 2 x 4 -1, h 1 = x 2 1 + x 2 2 -1 and h 2 = x 2 3 + x 2 4 -1 is illustrated in Table 6.1.
Here e i , i ∈ [4] is the canonical basis of R 4 . In this example, rank(M 1 (y)) decreases from 5 to 1 when t goes from 0 to 4 and M 1 (y) fulfills the flat extension condition at from t = 3.

Remark 6.11. ASC can be used to find an approximation of a real point in S(g, h) even if S(g, h)

is positive dimensional. This is illustrated later on by our numerical experiments from Section 6.3 (see the polynomial systems corresponding to Id 6,7,8 and 13).

Obtaining a minimizer by using the ASC method: We rely on the following algorithm to find the value ρ i k (ε) of SDP (6.2.10), which approximates f = min{f (x) : x ∈ S(g, h)}, together with an approximation x of a minimizer x for this problem.

Algorithm 13 PolyOpt

Input: f , S(g, h) = ∅, ε > 0 and k ∈ N. Output: ρ i k (ε) and x. 1: Solve SDP (6.2.10) to obtain ρ i k (ε). 2: Compute x in S(g ∪ {ρ i k (ε) -f }, h) by using Algorithm 6.2.3 and stop. Proposition 6.8. If POP f := inf{f (x) : x ∈ S(g, h)} admits an optimal solution at x , then for k large enough, Algorithm 6.2.3 terminates and f

≤ ρ i k (ε) ≤ f + εθ(x ) d . Moreover, for k large enough, x ∈ S(g, h) and f ≤ f (x) ≤ f + εθ(x ) d if the assumption of Theorem 6.6 holds for S(g ∪ {ρ i k (ε) -f }, h).
In practice, one performs Algorithm 6.2.3 several times by updating k := k +1 until one obtains x in S(g

∪ {ρ i k (ε) -f }, h). Obviously, one has f + εθ(x ) d ≥ ρ i k (ε) ≥ f (x) ≥ f .

Examples

In this section, we report results obtained after solving some instances of POP (6.2.8) with Algorithm 6.2.3. As before, let us note g = {g 1 , . . . , g m } and h = {h 1 , . . . , h l } the sets of polynomials involved in the inequality constraints and the equality constraints, respectively. In particular, the resulting set S(g, h) is unbounded for all examples. The experiments are performed with both MATLAB R2018a/Yalmip and Julia 1.1.1/JuMP to model the semidefinite optimization problems and Mosek 8.0 to solve these problems. The codes for Algorithm 6.2.3 (PolySys) and Algorithm 6.2.3 (PolyOpt) can be downloaded from the link: https://github.com/maihoanganh. In these codes, we always set a 0 := 0 R n and a 1 , . . . , a n as the canonical basis of R n . We use a desktop computer with an Intel(R) Pentium(R) CPU N4200 @ 1.10GHz and 4.00 GB of RAM. The input data given in Table 6.2 include examples of unconstrained and constrained POPs. The corresponding output data, the exact results and timings are given in Table 6.3. In these tables, the SOS hierarchy (6.2.10) is solved by optimization models in Yalmip (Y) and JuMP (J). The symbol "-" in a column entry indicates that the calculation did not finish in a couple of hours.

Id 1-5 are unconstrained POPs. Id 6-12 are POPs with inequality constrains, Id 13-18 are POPs with equality constraints and Id 19-25 are POPs with both inequality and equality constraints. Id 8, 11 and 12 correspond to examples from Jeyakumar et al. [START_REF] Jeyakumar | On polynomial optimization over non-compact semi-algebraic sets[END_REF][START_REF] Jeyakumar | Solving global optimization problems with sparse polynomials and unbounded semialgebraic feasible sets[END_REF]. Id 9 and 10 are selected from Demmel et al. [START_REF] Demmel | Representations of positive polynomials on noncompact semialgebraic sets via KKT ideals[END_REF]. Id 13-17 come from Greuet et al. [START_REF] Greuet | Probabilistic algorithm for polynomial optimization over a real algebraic set[END_REF]. Id 23, 24 and 25 are POPs constructed from some inequalities issued from Mathematics competitions mentioned in [START_REF] Slinko | USSR Mathematical Olympiads[END_REF][START_REF] Djukić | The IMO Compendium: A Collection of Problems Suggested for the International Mathematical Olympiads: 1959-2009 Second Edition[END_REF], yielding noncompact POPs with known optimal values and optimizers.

Even though the sets of minimizers associated to Id 6, 7, 8 and 13 are positive dimensional, we can still extract a single approximate optimal solution by using our ASC algorithm. Note that ASC computes a real point x in S(g

∪ {ρ i k (ε) -f }, h) which is an outer approximation of {x ∈ S(g, h) : f (x ) = f } for k sufficiently large.
In Table 6.3, Algorithm 6.2.3 terminates at some order k ≤ 5 for all POPs except Id 16. Note that for Id 16, the global minimum does not satisfy the KKT conditions. Thus the method of Demmel et al. [START_REF] Demmel | Representations of positive polynomials on noncompact semialgebraic sets via KKT ideals[END_REF][START_REF] Nie | Minimizing polynomials via sum of squares over the gradient ideal[END_REF] and Nie's [START_REF] Nie | Tight relaxations for polynomial optimization and Lagrange multiplier expressions[END_REF] cannot be used to solve this POP. Moreover, the convergence rate of (ρ i k (ε)) k∈N in Id 16 is very poor when ε ≤ 10 -5 . We overcome this issue by fixing k, multiplying ε by 10, and solving again the relaxations. The computational cost that we must pay here is due to the largest gap ε θ(x ) d between ρ i k (ε) and f . This behavior is illustrated in Table 6. 4.

In Id 18, even if the ideal I(h) is not radical and V (h) is not equidimensional (the assumptions required to apply the framework in [START_REF] Greuet | Probabilistic algorithm for polynomial optimization over a real algebraic set[END_REF] are not guaranteed) our ASC method can still extract one solution of the problem.

For Id 21, we can improve the quality of the approximation ρ i k (ε) of the optimal value f by fixing k = 1, dividing ε by 10, and solving again the relaxations. This is illustrated in Table 6. 5.

We emphasize that we can customize the ε parameter for different purposes. On the one hand, one increases ε to improve the convergence speed of the sequence (ρ i k (ε)) k∈N to the neighborhood [f , f +εθ(x ) d ] of f (see Table 6.4). On the other hand, one decreases ε to improve the accuracy of the approximate optimal value ρ i k (ε) and the approximate optimal solution x (see Table 6.5). Our numerical benchmarks also show that modeling in JuMP is faster and provides more accurate outputs than modeling in Yalmip. In particular, the JuMP implementation is the only one which provides solutions for Id 11,[START_REF] Bajbar | Coercive polynomials and their Newton polytopes[END_REF]17 and 23. Let us now denote by k ε the smallest nonnegative integer such that ρ i kε (ε) ≥ f , for each ε > 0. The graph of the function ε -1 → k ε on (0, 100] for Id 9 and Id 16 is illustrated in Figure 6. Chapter 7

On the complexity of Putinar-Vasilescu's Positivstellensatz

Most of the content of this chapter is from [START_REF] Mai | On the complexity of Putinar-Vasilescu's Positivstellensatz[END_REF].

In the previous chapter, we have applied Putinar-Vasilescu's Positivstellensatz to design a hierarchy of SDP relaxations that solves POPs over noncompact semialgebraic sets. In this chapter, we provide a constructive proof of the degree bound for Putinar-Vasilescu's Positivstellensatz. It allows us to analyze the convergence rate of the corresponding hierarchy of SDP relaxations.

For a positive m ∈ N, let us consider the polynomial optimization problem (POP):

f := inf x∈S(g) f (x) , (7.0.1)
where f ∈ R[x] and S(g

) := {x ∈ R n : g i (x) ≥ 0 , i ∈ [m]} , (7.0.2) for some g = {g i } i∈[m] ⊂ R[x].
Here R[x] denotes the ring of real polynomials in vector of variables x = (x 1 , . . . , x n ) and [m] stands for the set {1, . . . , m}. Assume that f has degree at most 2d f for some positive d f ∈ N. Recall that the set S(g) is a conjunction of finitely many polynomial inequalities, and therefore is called a basic semialgebraic set. Problem (7.0.1) can be written as

f = sup λ∈R {λ : f -λ > 0 on S(g)} . (7.0.3)
We can replace the inequality constraint of problem (7.0.3) by an equality constraint, if one can represent positive polynomials on S(g). Assume that S has nonempty interior and a ball constraint is present, i.e., g 1 = R -x 2 2 for some R > 0. Our first (minor) contribution is to rely on the representation of polynomials positive on S stated by Putinar and Vasilescu [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF], to obtain

f -λ = σ 0 + i∈[m] σ i g i (1 + x 2 2 ) k , (7.0.4) for some k ∈ N, σ i ∈ Σ[x], i ∈ [m], being such that deg(σ 0 ) ≤ 2(k + d f ) and deg(σ i g i ) ≤ 2(k + d f ).
Here Σ[x] denotes the set of sum-of-squares (SOS) polynomials and deg(•) stands for the degree of a polynomial. Such a representation of positive polynomials is called a Positivstellensatz.

After bounding the degrees of the SOS polynomials involved in (7.0.4), we obtain the following hierarchy of relaxations indexed by k ∈ N:

ρ (0) k := sup λ,σi λ s.t. λ ∈ R , σ i ∈ Σ[x] , (1 + x 2 2 ) k (f -λ) = σ 0 + i∈[m] σ i g i , deg(σ 0 ) ≤ 2(k + d f ) , deg(σ i g i ) ≤ 2(k + d f ) .
(7.0.5) Problem (7.0.5) can be solved numerically using semidefinite programming [START_REF] Ben-Tal | Lectures on modern convex optimization: analysis, algorithms, and engineering applications[END_REF]. As usual, one can show that for each k ∈ N, ρ (0) k is a lower bound of f , that the sequence (ρ (0) k ) k∈N is monotone nondecreasing, and converges to f .

In the present chapter, we answer the following two interesting questions:

1. How fast does (ρ

k ) k∈N converge to f ? We show the convergence rate O(k -1/c ) for some constant c > 0 depending on f and g i .

2. Is there any explicit example to illustrate this rate of convergence? If S(g) is the unit ball, i.e., m = 1 and g 1 = 1x 2 2 , the sequence (ρ

k ) k∈N converges to f with the rate O(k -1/65 ). Our contribution is concerned with the case of basic semialgebraic sets having nonempty interiors. Basically one obtains a convergence rate similar in spirit and magnitude of Schweighofer's bound cε -c , but still based on the quadratic module Q k (g) defined as in (1.3.8) (not the preordering P k (g) defined as in (1.3.6)) thanks to the prescribed denominator (1 + x 2 2 ) k involved in Putinar-Vasilescu's Positivstellensatz.

Before showing explicitly our contribution, we restate the original result of Putinar and Vasilescu (without degree bound) in (6.1).

Contribution. The construction of the hierarchy of semidefinite relaxations (7.0.5) is based on the Positivstellensatz stated in Corollary 7.2. More explicitly, if S(g) has nonempty interior such that g 1 = R -x 2 2 for some R > 0 and f is of degree at most 2d f such that f is nonnegative on S, then there exist positive constants c and c depending on f, g i such that for all ε > 0, for all

k ≥ cε -c , (1 + x 2 2 ) k (f + ε) = σ 0 + i∈[m] σ i g i , (7.0.6)
for some

σ i ∈ Σ[x] being such that deg(σ 0 ) ≤ 2(k + d f ) and deg(σ i g i ) ≤ 2(k + d f ).
In order to prove (7.0.6), we provide a degree bound on the weighted SOS polynomials for the homogenized Putinar-Vasilescu's Positivstellensatz [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF]. This is stated in Theorem 7.1 as follows: If f, g 1 , . . . , g m are homogeneous polynomials of even degrees such that S(g) has nonempty interior and f is nonnegative on S(g), then there exist positive constants c and c depending on f, g i such that for all ε > 0, for all k ≥ cε -c ,

x 2k 2 (f + ε x 2d f 2 ) = σ 0 + i∈[m] σ i g i , (7.0.7)
for some homogeneous SOS polynomials σ i being such that deg(σ

0 ) = deg(σ i g i ) = 2(k + d f ).
Here a polynomial q is homogeneous of degree 2t if q(λx) = λ 2t q(x) for all x ∈ R n and each λ ∈ R.

Remark that the original version of Putinar-Vasilescu's Positivstellensatz in [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF] does not include any degree bound on the weighted SOS polynomials σ i involved in the representation (7.0.7). Our proof of Theorem 7.1 consists of three main steps:

1. Construct iteratively some positive "weight" functions

ψ i such that f + ε -i∈[m] ψ i g i is positive on [-1, 1] n .
The idea of this step is similar in spirit to the proof of the inductive property in [186, Proposition 3.1] and relies on the Lojasiewicz inequality.

Approximate

√ ψ i with the multivariate Bernstein polynomial q i on [-1, 1] n such that the polynomial H = f + ε -i∈[m] q 2
i g i is positive on the unit sphere S n-1 .

3. Apply Reznick's Positivstellensatz [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF] to the homogenization of H.

The complexity analysis of every step is derived to get the final degree bound cε -c . Afterwards, we obtain in Corollary 7.1 the same degree bound for the dehomogenized Putinar-Vasilescu's Positivstellensatz. This improves the bound O(exp(ε -c )) obtained in the previous chapter, based on Nie-Schweighofer's complexity result [START_REF] Nie | On the complexity of Putinar's Positivstellensatz[END_REF] for Putinar's Positivstellensatz [START_REF] Putinar | Positive polynomials on compact semi-algebraic sets[END_REF]. Corollary 7.1 yields the convergence rate O(ε -c ) for the corresponding hierarchy of relaxations for polynomial optimization on general (not necessarily compact) basic semialgebraic sets. Remark 7.1. Concerning the assumption that the semialgebraic set S(g) has nonempty interior, a technique from [1] maybe helps us remove this assumption 1 .

Technical insights. We start to recall the two main steps in the proof of Nie and Schweighofer [START_REF] Nie | On the complexity of Putinar's Positivstellensatz[END_REF] for the degree bound of SOS polynomials involved in Putinar's Positivstellensatz:

1. Find a large enough k ∈ N such that the polynomial

F = f + ε -λ i∈[m] (g i -1) 2k g i (7.0.8) is positive on [-1, 1] n . An estimate k ≥ O(ε -c
) is obtained. Here ε > 0 measures how close the polynomial f (assumed to be nonnegative on S(g)) is to have a zero on S(g).

Apply Schmüdgen's Positivstellensatz to

F on [-1, 1] n .
Notice that Schweighofer's degree bound of Schmüdgen's Positivstellensatz is exponential in the degree of the given positive polynomial (n d f in (1.3.5)). Accordingly, Nie and Schweighofer obtain an exponential bound n O(ε -c ) in the second step since deg(F ) ∼ Ck as k → ∞ for some positive constant C. One notable difference in our proof is that the weight λ(g i -1) k in (7.0.8) is replaced by a non-differentiable positive function ψ i . Surprisingly, we can prove that the square root √ ψ i is a Lipschitz continuous function. Thus each √ ψ i can be approximated with a Bernstein polynomials q i on [-1, 1]. Here, the advantage of using Bernstein polynomials is that the approximation error between √ ψ i and q i decreases with a rate which only depends on a Lipschitz constant of √ ψ i , and

|q i | is upper bounded by the supremum of √ ψ i on [-1, 1] n .
Next, we apply Reznick's Positivstellensatz to the homogeneous polynomial H obtained from the homogenization of

H := f + ε -i∈[m] q 2 i g i , (7.0.9)
being such that the bounds of H and H on the unit sphere are the same. The important point to note here is that the degree bound of Reznick's Positivstellensatz is quadratic in the degree of H and linear in the ratio δ( H) (

). This is in deep contrast with Schmüdgen's Positivstellensatz, as there is no exponential dependency in these two quantities. This leads to the difference between our convergence rate O(ε -c ) and Nie-Schweighofer's rate O(exp(ε -c )).

One may ask whether with the same techniques from our proof, one could improve the existing degree bound for Putinar's Positivstellensatz. We have tried to apply the degree bound (1.3.3) of Pólya's Positivstellensatz to H after a change of coordinate, but unfortunately this leads to the same bound as Nie and Schweighofer. The underlying reason is that the norm q in (1.3.3) depends on the coefficients of q. In our situation, q coincides with H and the coefficients of H are bounded by a value involving the coefficients of the Bernstein polynomials. The bound on the largest coefficient, even for a univariate Bernstein polynomial, seems to be exponential in the approximation order t, namely, 2t t ∼ 4 t √ πt as t → ∞. The same issue occurs when we apply the degree bound of Schmüdgen's Positivstellensatz instead of the one of Pólya's Positivstellensatz.

Representation theorems and degree bounds

In this section, we derive representations of polynomials nonnegative on semialgebraic sets together with degree bounds. We extend these representations to the set of continuous functions being nonnegative on compact domains.

Polynomials nonnegative on general semialgebraic sets

We analyze the complexity of Putinar-Vasilescu's Positivstellensatz [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF] in the following theorem: Theorem 7.1. (Homogenized representation) Let g 1 , . . . , g m be homogeneous polynomials of even degrees such that the semialgebraic set

S := {x ∈ R n : g 1 (x) ≥ 0 . . . , g m (x) ≥ 0} (7.1.1)
132CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ has nonempty interior. Let f be a homogeneous polynomial of degree 2d f for some d f ∈ N such that f is nonnegative on S. Then there exist positive constants c and c depending on f, g 1 , . . . , g m such that for all ε > 0, for all k ∈ N satisfying k ≥ cε -c , (

there exist homogeneous SOS polynomials σ 0 , . . . , σ m such that

deg(σ 0 ) = deg(σ 1 g 1 ) = • • • = deg(σ m g m ) = 2(k + d f ) (7.1.3) and x 2k 2 (f + ε x 2d f 2 ) = σ 0 + σ 1 g 1 + • • • + σ m g m . (7.1.4)
In particular, if m = 1 and g 1 = x 2 n -x 2 2 with x := (x 1 , . . . , x n-1 ), then c = 65. Moreover, each SOS polynomial σ i involved in (7.1.4) can be chosen as the (single) square of a homogeneous polynomial, for i ∈ [m].

The proof of Theorem 7.1 is postponed to Subsection 7.1.4. In Theorem 7.1, all polynomials are assumed to have even degrees. Next, we provide two corollaries where each polynomial can have odd degree.

The following corollary is a direct consequence of Theorem 7.1.

Corollary 7.1. (Dehomogenized representation) Let g 1 , . . . , g m be polynomials such that the semialgebraic set

S := {x ∈ R n : g 1 (x) ≥ 0 . . . , g m (x) ≥ 0} (7.1.5)
has nonempty interior. Let f be a polynomial nonnegative on S. Denote

d f := deg(f )/2 + 1.
Then there exist positive constants c and c depending on f, g 1 , . . . , g m such that for all ε > 0, for all k ∈ N satisfying k ≥ cε -c , (

there exist SOS polynomials σ 0 , . . . , σ m such that

deg(σ 0 ) ≤ 2(k + d f ) and deg(σ i g i ) ≤ 2(k + d f ) , i ∈ [m] , (7.1.7
)

and θ k (f + εθ d f ) = σ 0 + σ 1 g 1 + • • • + σ m g m , ( 7.1.8
)

where θ := 1 + x 2 2 . Moreover, if m = 1 and g 1 = 1 -x 2 2 , then c = 65.
Proof. The proof of Corollary 7.1 is similar to the proof of [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Theorems 4 and 5]. We recall the basic ingredients. Let S be a homogenized version of S, defined by

S := {(x, x n+1 ) ∈ R n+1 : gi (x, x n+1 ) ≥ 0 , i ∈ [m]} , ( 7.1.9)
with gi (x,

x n+1 ) := x 2 gi n+1 g i (x/x n+1 ) being the degree-2 g i homogenization of g i , for i ∈ [m].
Then the proof consists of three steps:

1. Prove that the degree-2d f homogenization of f , denoted by f , is nonnegative on S.

2. Use Theorem 7.1 to obtain a representation of f together with the degree bound on SOS polynomials.

3. Obtain a representation of f by evaluating the representation of f at x n+1 = 1.

To apply Theorem 7.1, we need to show that if S has nonempty interior, then S has nonempty interior. This statement holds since when a belongs to the interior of S, one has gi (a, 1) = g i (a) > 0, implying that (a, 1) belongs to the interior of S.

Note that the ice cream constraint x 2 n+1 -x 2 2 is the degree-2 homogenization associated to the ball constraint 1 -x 2 2 .

Polynomials nonnegative on compact semialgebraic sets

The following corollary is deduced from Corollary 7.1.

Corollary 7.2. Let g 1 , . . . , g m be polynomials such that g 1 := R -x 2 2 for some R > 0 and the semialgebraic set

S := {x ∈ R n : g 1 (x) ≥ 0 . . . , g m (x) ≥ 0} (7.1.10)
has nonempty interior. Let f be a polynomial nonnegative on S. Denote

d f := deg(f )/2 + 1.
Then there exist positive constants c and c depending on f, g 1 , . . . , g m such that for all ε > 0, for all k ∈ N satisfying k ≥ cε -c , (

there exist SOS polynomials σ 0 , . . . , σ m such that

deg(σ 0 ) ≤ 2(k + d f ) and deg(σ i g i ) ≤ 2(k + d f ) , i ∈ [m] , (7.1.12)
and

(1 + x 2 2 ) k (f + ε) = σ 0 + σ 1 g 1 + • • • + σ m g m . (7.1.13)
Moreover, if m = 1 and L = 1, then c = 65.

Proof. By using Corollary 7.1, there exist positive constants c and c depending on f, g 1 , . . . , g m such that for all ε > 0, for all k ∈ N satisfying k ≥ cε -c , there exist SOS polynomials σ 0 , . . . , σ m such that deg(σ

0 ) ≤ 2(k + d f ) and deg(σ i g i ) ≤ 2(k + d f ) , i ∈ [m] . ( 7.1.14)
and

θ k (f + εθ d f ) = σ 0 + σ 1 g 1 + • • • + σ m g m , (7.1.15)
where θ := 1 + x 2 2 . In addition,

(R + 1) d f -θ d f = (R -x 2 2 ) d f -1 j=0 (R + 1) d f -1-j θ j = s 1 g 1 , (7.1.16)
where

s 1 = d f -1 j=0 (R + 1) d f -1-j θ j is an SOS polynomial of degree 2(d f -1). From this, θ k [f + ε(R + 1) d f ] = θ k (f + εθ d f ) + εθ k [(R + 1) d f -θ d f ] = σ 0 + (εs 1 θ k + σ 1 )g 1 + m i=2 σ i g i , ( 7
ε > 0, for all k ∈ N such that k ≥ O(ε -1 ), θ k (f +εθ d f ) is an SOS polynomial, so that θ k (f +ε) = σ 0 +σ 1 (R -x 2
2 ) for some SOS polynomials σ i , i = 0, 1. This is the so-called "big ball trick". This representation yields a linear convergence rate O(ε -1 ) for the minimization of polynomials on R n .

Remark 7.4. (Removing the denominator in Putinar

-Vasilescu's Positivstellensatz) Let f, g i ∈ R[x] with g 1 := 1. Set d = deg(f )/2 + 1. Assume that f ≥ 0 on S(g) ⊂ {x ∈ R n : 1 ≥ x 2 2 }. Then f ∈ R[x, x n+1 ] is nonnegative on S(g) × R. Let ψ(x, x n+1 ) := 1 + (x, x n+1 ) 2 2 . Putinar- Vasilescu's Positivstellensatz yields that there is k ∈ N such that ψ k (f + εψ d ) = m i=1 σ i g i 134CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ for some SOS σ i in R[x, x n+1 ]. Setting x n+1 = 1 -x 2 2 , we get ψ = 2 so f + ε2 d = 1 2 k m i=1 σ i (x, 1 -x 2 2 )g i .
By removing all terms which are not polynomial, we obtain

f + ε2 d = 1 2 k m i=1 [a i + b i (1 -x 2 2 )]g i . (7.1.18)
for some SOS

a i , b i in R[x]. It is due to the fact that each σ i (x, 1 -x 2 2
) is a sum of the following squares:

(u + v 1 -x 2 2 ) 2 = u 2 + v 2 (1 -x 2 2 ) + 2uv 1 -x 2 2 , for some u, v ∈ R[x].
Based on the degree bound for Putinar-Vasilescu's Positivstellensatz in Corollary 7.1, we obtain a similar one for representation (7.1.18).

Preliminary material

This subsection presents some important lemmas that we use to prove the main results.

Given Ω ⊂ R n , the distance of a ∈ R n to Ω is denoted by dist(a, Ω). Denote by B(a, r) (resp. B • (a, r)) the closed (resp. open) ball centered at a ∈ R n with radius r > 0.

We recall the Lojasiewicz inequality in the following lemma: Given an open set U ⊂ R n and a differentiable function ϕ :

U → R, denote by ∇ϕ(x) = [∂ x1 ϕ(x), . . . , ∂ xn ϕ(x)] the gradient of ϕ at x ∈ U . Given x = (x 1 , . . . , x n ) ∈ R n , let x := (x 1 , . . . , x n-1 ).
As mentioned in [START_REF] Pham | Genericity in polynomial optimization[END_REF]Theorem 3.8], one can prove that α ∈ {1, 2} in Lemma 7.1 if g is a quadratic polynomial. The following lemma states an instance of this result: The first condition of (7.1.21) implies that 2(xy) = µ -2y 2y n , so xy = -µy and x n -y n = µy n . Assume that µ / ∈ {1, -1}. Then y = x 1-µ and

Lemma 7.3. ( Lojasiewicz inequality with ice cream constraint) Let g := x 2 n -x 2 2 and Z := {x ∈ R n : g(x) = 0}. Then for all x ∈ R n , dist(x, Z) 2 ≤ |g(x)| 2 . ( 7
y n = xn 1+µ . Since g(y) = y 2 n -y 2 2 = 0, y n = ± y 2 .
Let us consider the first case

y n = y 2 . Then xn 1+µ = x 2 1-µ . It implies that µ = xn-x 2 xn+ x 2 . Note that x n = -x 2 since g(x) = 0. From this, y = (xn+ x 2)x 2 x 2 and y n = xn+ x 2 2 . Thus, x -y 2 2 = (xn-x 2 ) 2 2 .
Similarly, if we consider the case y n = -y 2 , then xy 2 2 = (xn+ x 2) 2

2

. Let us consider the case of µ ∈ {1, -1}. Assume that µ = 1. Then x = 0 and y n = xn 2 . From this and the fact that 0 = g(y) = y 2 n -y 2 2 , we obtain

y 2 2 = x 2 n 4 . It implies that x -y 2 2 = y 2 2 + (x n -y n ) 2 = x 2 n 4 + x 2 n 4 = x 2 n 2 = (xn-x 2) 2 2 . Thus, x -y 2 2 = (xn-x 2 ) 2 2 .
Similarly, if we consider the case µ = -1, then x -

y 2 2 = (xn+ x 2) 2 2 . Thus, d(x, Z) 2 ≤ 1 2 min{(x n -x 2 ) 2 , (x n + x 2 ) 2 } ≤ 1 2 (x n -x 2 ) 2 (x n + x 2 ) 2 = 1 2 |x 2 n -x 2 2 | = |g(x)| 2 , ( 7.1.22) yielding (7.1.20).
A real-valued function f :

U → R for some U ⊂ R n is called L-Lipschitz (or Lipschitz) continuous on K ⊂ U if there exits a real L > 0 such that |f (x) -f (y)| ≤ L x -y 2 , for all x, y ∈ K. In this case, L is called the Lipschitz constant of f on K. Given an open set U ⊂ R n , a function f : U → R is called locally Lipschitz continuous on K ⊂ U if for every x ∈ K there exists a neighborhood W ⊂ U of x such that f is Lipschitz continuous on W ∩ K.
The following lemma is similar in spirit to [163, Section 2.4, Lemma 2]:

Lemma 7.4. Given an open set U ⊂ R n , if the function f : U → R is locally Lipschitz on a compact set K ⊂ U , then f is Lipschitz on K.
Proof. Since f is locally Lipschitz on K, for each x ∈ K there is some r x > 0 and

L x > 0 such that B(x, r x ) ⊂ U and f is L x -Lipschitz on B(x, r x ) ∩ K.
Then the sets B(x, 1 2 r x ), x ∈ K form an open cover of K. Due to the compactness of K, there exists a finite subsequence of B(x, 1 2 r x ), x ∈ K covering K. For convenience, denote these by B(x k , 1 2 r k ) and

L k := L x k , k ∈ [l]. Let M := sup x∈K |f (x)|, r := 1 2 min k∈[l] r k , L 0 := 2M r and L := max{L 0 , L k : k ∈ [l]}.
Then L is a Lipschitz constant of f on K. To see this, pick x, y ∈ K. If xy 2 ≥ r then we see that

|f (x)-f (y)| x-y 2 ≤ 2M r = L 0 ≤ L. If x -y 2 < r, then for some x k we have x ∈ B(x k , 1 2 r k ). Then y ∈ B(x k , r k ) and so |f (x) -f (y)| ≤ L k x -y 2 ≤ L x -y 2 .
Lemma 7.5. (Kirszbraun's theorem [START_REF] Kirszbraun | Über die zusammenziehende und Lipschitzsche Transformationen[END_REF]) If U is a subset of R n and f : U → R is a Lipschitz continuous function, then there is a Lipschitz continuous function F : R n → R that extends f and has the same Lipschitz constant as f . Moreover the extension is provided by

F (x) := inf u∈U {f (u) + L f x -u 2 } , (7.1.23)
where L f is the Lipschitz constant of f on U .

Given Ω ⊂ R n , denote by C(Ω) the set of continuous functions from Ω to R. We recall basic properties of the multivariate Bernstein polynomials described, e.g., in [START_REF] Hildebrandt | On linear functional operations and the moment problem for a finite interval in one or several dimensions[END_REF][START_REF] Heitzinger | Simulation and inverse modeling of semiconductor manufacturing processes[END_REF].

f ∈ C([0, 1] n ) is L-Lipschitz, namely |f (x) -f (y)| ≤ L x -y 2 on [0, 1] n , then for all d ∈ N n , the inequality |B f,d (x) -f (x)| ≤ L 2 ( n j=1 1 dj ) 1 2 (7.1.26)
holds for all x ∈ [0, 1] n .

Proof. Let x ∈ [0, 1] n be fixed. To simplify the notation, we define b j := dj kj x kj j (1 -x j ) dj -kj , for j ∈ [n], and k = ( k1 d1 , . . . , kn dn ). Then, we have:

|B f,d (x) -f (x)| 2 ≤ ( |f (k) -f (x)|b 1 . . . b n ) 2 ≤ L 2 ( k -x 2 b 1 • • • b n ) 2 ≤ L 2 k -x 2 2 b 1 • • • b n ( b 1 • • • b n ) = L 2 ( r ( kr dr -x r ) 2 )b 1 • • • b n = L 2 r ( ( kr dr -x r ) 2 b 1 • • • b n ) = L 2 r xr(1-xr) dr ≤ L 2 r 1 4dr . (7.1.27)
For the first inequality, we have used the multinomial identity 1 = j [x j + (1 -x j )] dj = b 1 . . . b n and the triangle inequality. For the second one, we have used the fact that f is L-Lipschitz. For the third one, we use that for all a, b ∈ R n and all c ∈ R n + , one has (

a k b k c k ) 2 ≤ a 2 k c k b 2 k c k .
The last equality comes from the identities

dr kr=0 (k r -d r x r ) 2 b r = d r x r (1-x r ) and j =r ( dj kj =0 b j ) = 1.
The last inequality is obtained by noticing that we have x r (1

-x r ) ≤ 1/4.
Let e := (1, . . . , 1) ∈ R n . As a consequence of Lemma 7.6, we obtain the following result after a change of coordinates. Then 2L is a Lipschitz constant of g. Let k ∈ N ≥1 . Using Lemma 7.6, we get that for all x ∈ [0, 1] n , In addition, from (7.1.25),

Lemma 7.7. If f ∈ C([0, 1] n ) is L-Lipschitz, namely |f (x) -f (y)| ≤ L x -y 2 on [-1, 1] n ,
|B g,ke (x) -g(x)| ≤ 2L 2 ( n j=1 1 k ) 1 2 = L( n k ) 1 2 . (7.1.31) Let y ∈ [-1, 1] n . Then y+e 2 ∈ [0, 1] implies that |B g,ke ( y+e 2 ) -f (y)| = |B g,ke ( y+e 2 ) -g( y+e 2 )| ≤ L( n k ) 1 2 . ( 7
sup y∈[-1,1] |B g,ke ( y+e 2 )| = sup x∈[0,1] n |B g,ke (x)| ≤ sup x∈[0,1] n |g(x)| = sup y∈[-1,1] |g( y+e 2 )| = sup y∈[-1,1] |f (y)| , ( 7.1.33)
which yields (7.1.29).

The proof of Theorem 7.1

Recall that [l] := {1, . . . , l} for l ∈ N ≥1 . Given real value functions g, h, we use the notation {g * h} = {x ∈ R n : g(x) * h(x)}, where * ∈ {=, ≥, ≤, >, <}. Given a real value function q on Ω ⊂ R n , note q Ω := sup x∈Ω |q(x)|. With Ω ⊂ R n , denote by int(Ω) the interior of Ω.

Given U, V ⊆ R n and r ∈ R, note

U + V = {u + v : u ∈ U , v ∈ V } and rU = {ru : u ∈ U }. Given a function f : U → R and A ⊂ U ⊂ R n such that A = -A, f is called even on A if f (-x) = f (x) for all x ∈ A.
To begin the proof, let us fix ε > 0. By assumption, deg(f

) = 2d f , deg(g i ) = 2d gi for some d, d gi ∈ N, for i ∈ [m].

Construction of the positive weight functions

For j ∈ [m], define S j := {x ∈ R n : g i (x) ≥ 0 , i ∈ [j]} . (7.1.34)
Obviously, we have S m = S. Note S 0 := R n and f m := f . We will prove that there exist functions φm : R n → R such that the following conditions hold:

1. φm is positive, even and bounded from above by C φm = rm ε -rm on B(0, √ n + m) for some positive constants rm and r m independent of ε.

2. φm is Lipschitz with Lipschitz constant L φm = tj ε -tm for some positive constants tm and t m independent of ε.

3. f m-1 := f m + ε 2 -φ2 m g m satisfies: (a) f m-1 ≥ 0 on S m-1 ∩ B(0, √ n + m -1); (b) f m-1 ≤ C fm-1 on B(0, √ n+m), where C fm-1 = cm-1 ε -cm-1
for some positive constants cm-1 and c m-1 independent of ε;

(c) f m-1 is Lipschitz on B(0, √ n + m) with Lipschitz constant L fm-1 = lm-1 ε -lm-1
for some positive constants lm-1 and l m-1 independent of ε.

Let M m := inf x∈Sm∩B(0, √ n+m) f (x) + ε 2 g m (x) . (7.1.35)
The constant M m is a positive real number.

Let C gm = g m B(0, √ n+m) . We claim that ε 2Cg m < M m < ∞. Indeed, if z is a feasible solution of (7.1.35), z ∈ S yielding f (z) ≥ 0 so that f (z) + ε 2 g m (z) ≥ ε 2g m (z) ≥ ε 2C gm . ( 7.1.36)
From this, we have

M m > ε 2Cg m
. On the other hand, there exists a ∈ R n such that g i (a

) > 0 for i ∈ [m] since S has nonempty interior. For i ∈ [m], since g i is homogeneous, a = 0 yields g i (a) = 0. It implies that a = 0. With ā = a a 2 ∈ B n ⊂ B(0, √ n + m), we obtain g i (ā) > 0 for i ∈ [m] since g i (ā) = g i a a 2 = g i (a) a 2dg i 2 > 0 , ∀i ∈ [m] . (7.1.37)
Thus, ā is a feasible solution of (7.1.35) which yields

ε 2C gm ≤ M m ≤ f (ā) + ε 2 g m (ā) ≤ C f + ε 2 g m (ā) < ∞ , ( 7.1.38)
where

C f := f B(0, √ n+m)
. Let ψ m : R n → R be the function defined by

ψ m (x) := max{M m , f (x)+ ε 2 gm(x) } if g m (x) < 0 , M m otherwise. (7.1.39) 138CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ The function f + ε 2 -ψ m g m is nonnegative on S m-1 ∩ B(0, √ n + m). Namely, we claim that f + ε 2 -ψ m g m ≥ 0 on S m-1 ∩ B(0, √ n + m) . (7.1.40) Let y ∈ S m-1 ∩ B(0, √ n + m). If g m (y) < 0, then f (y) + ε 2 -ψ m (y)g m (y) = f (y) + ε 2 -g m (y) max{M m , f (y)+ ε 2 gm(y) } ≥ f (y) + ε 2 -g m (y) f (y)+ ε 2 gm(y) = 0 . (7.1.41)
Otherwise, g m (y) ≥ 0 gives

f (y) + ε 2 -ψ m (y)g m (y) = f (y) + ε 2 -g m (y)M m ≥ f (y) + ε 2 -g m (y) f (y)+ ε 2 gm(y) = 0 if g m (y) > 0 , = f (y) + ε 2 ≥ 0 if g m (y) = 0 , ( 7.1.42)
since y ∈ S is a feasible solution of (7.1.35).

The function ψ m is positive, even on B(0, √ n+m) and continuous on S m-1 ∩B(0, √ n+m). It is easy to see that ψ m is bounded from below by M m and continuous on B(0,

√ n + m)\{g m = 0} since the max function (t 1 , t 2 ) → max{t 1 , t 2 } is continuous. We claim that ψ m is continuous on S m-1 ∩ B(0, √ n + m) ∩ {g m = 0}. Indeed, let us consider a sequence (y l ) l ⊂ S m-1 ∩B(0, √ n+m)∩{g m < 0} such that y l → ȳ ∈ S m-1 ∩B(0, √ n+m)∩{g m = 0}. Then g m (y l ) → 0 -and f (y l ) → f (ȳ) ≥ 0 (since ȳ ∈ S) yielding that f (y l )+ ε 2 gm(y l ) → -∞. It implies that max{M m , f (y l )+ ε 2 gm(y l ) } → M m . Thus, ψ m = M m on a sufficiently small neighborhood of any point in S m-1 ∩B(0, √ n+m)∩{g m = 0}.
On the other hand, ψ m is even, i.e., ψ m (x) = ψ m (-x) due to the fact that f, g 1 , . . . , g m are even and B(0,

√ n + m) = -B(0, √ n + m).
The upper bound of ψ m depends on ε. It follows from (7.1.38) that ψ m = M m on B(0, √ n + m) ∩ {g m ≥ 0} and so is bounded from above by

f (ā)+ ε 2 gm(ā) . Let us compute an upper bound of ψ m on S m-1 ∩ B(0, √ n + m) ∩ {g m < 0}. Let y ∈ S m-1 ∩ B(0,
√ n + m) be such that g m (y) < 0 and

f (y)+ ε 2 gm(y) > M m . Then ψ m (y) = f (y)+ ε 2 gm(y)
. By using the Lojasiewicz inequality (see Lemma 7.2), there exist C m > 0 and α m > 0 depending on g 1 , . . . , g m such that for all x ∈ S m-1 ∩ B(0,

√ n + m) ∩ {g m < 0}, dist(x, S) αm ≤ -C m min{g 1 (x), . . . , g m (x), 0} = -C m g m (x) . (7.1.43) Let δ m = 1 Cm ( ε 2L f ) αm , where L f is a Lipschitz constant of f on B(0, √ n + m).
Consider the following two cases:

• Case 1: g m (y) ≤ -δ m < 0. Then ψ m (y) = f (y) + ε 2 g m (y) = -f (y) -ε 2 -g m (y) ≤ C f -g m (y) ≤ C f δ m ≤ C m C f 2L f ε αm . (7.1.44) • Case 2: -δ m ≤ g m (y) < 0. Let z ∈ S such that dist(y, S) = y -z 2 . Then (7.1.43) turns to -f (y) ≤ ε 2 according to -f (y) ≤ -f (z) + L f y -z 2 ≤ L f dist(y, S) ≤ L f (-C m g m (y)) 1 αm ≤ L f (C m δ m ) 1 αm = ε 2 . (7.1.45)
From this, we obtain

M m < f (y) + ε 2 g m (y) = -f (y) -ε 2 -g m (y) ≤ ε 2 -ε 2 -g m (y) = 0 < M m . (7.1.46)
The contradiction indicates that this case does not occur.

Thus, the bound is given as follows

sup x∈Sm-1∩B(0, √ n+m) ψ m (x) ≤ max f (ā) + ε 2 g m (ā) , C m C f 2L f ε αm =: C ψm . (7.1.47)
Moreover, we obtain the inclusion

S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ 0} ⊂ {g m ≤ -δ m } , ( 7.1.48)
where

ξ m (x) = f (x)+ ε 2 gm(x)
. Let ϕ m be the square root of ψ m , i.e., ϕ m (x) := ψ m (x). Then ϕ m is well-defined on B(0,

√ n + m) since ψ m is positive. Moreover, ϕ m is finitely bounded from above on S m-1 ∩ B(0, √ n + m) by C ϕm := C ψm and ϕ m is continuous on S m-1 ∩ B(0, √ n + m) since ξ m is continuous on S m-1 ∩ B(0, √ n + m).
The function ϕ m is Lipschitz continuous on S m-1 ∩ B(0, √ n + m -1). Keep in mind that ψ m is defined by the constant function M m and the function ξ m . Since ϕ m takes the constant value

√ M m on B(0, √ n + m)\({ξ m ≥ M m } ∩ {g m ≤ 0}), ϕ m is Lipschitz continuous on B(0, √ n + m)\({ϕ m ≥ M m } ∩ {g m ≤ 0}) with zero Lipschitz constant.
On the other hand,

ϕ m = √ ξ m on B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ 0}.
As a consequence of (7.1.48), we have

S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ 0} = S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ -δ m } . ( 7.1.49)
It implies that

ϕ m (x) = ξ m (x) if x ∈ S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ -δ m } , √ M m if x ∈ (S m-1 ∩ B(0, √ n + m))\({ξ m ≥ M m } ∩ {g m ≤ -δ m }) . ( 7.1.50)
The second equality is due to the fact that

ϕ m = √ M m on (S m-1 ∩ B(0, √ n + m))\({ξ m ≥ M m } ∩ {g m ≤ 0}) and (S m-1 ∩ B(0, √ n + m))\({ξ m ≥ M m } ∩ {g m ≤ 0}) = (S m-1 ∩ B(0, √ n + m))\[S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ 0}] = (S m-1 ∩ B(0, √ n + m))\[S m-1 ∩ B(0, √ n + m) ∩ {ξ m ≥ M m } ∩ {g m ≤ -δ m }] = (S m-1 ∩ B(0, √ n + m))\({ξ m ≥ M m } ∩ {g m ≤ -δ m }) . ( 7.1.51)
Let L gm be a Lipschitz constant of g m on B(0, √ n + m). Set

w m := min 1, δ m 2L gm , εδ 2 m 8C gm [L f C gm + (C f + ε 2 )L gm ] . ( 7.1.52)
and

W m := B(0, √ n + m -1) ∩ {ξ m ≥ M m } ∩ {g m ≤ -δ m } + w m B n . (7.1.53) Then B(0, √ n + m -1) ∩ {ξ m ≥ M m } ∩ {g m ≤ -δ m } ⊂ W m . Next, we prove that W m ⊂ B(0, √ n + m) ∩ {ξ m ≥ M m 2 } ∩ {g m ≤ - δ m 2 } . (7.1.54) Let y ∈ W m . Then y = z+w m u for some z ∈ S m-1 ∩B(0, √ n+m-1)∩{ξ m ≥ M m }∩{g m ≤ -δ m } and for some u ∈ B n . Combining z 2 ≤ √ n + m -1, 0 < w m < 1 and u 2 ≤ 1, one has y 2 ≤ z 2 + w m u 2 ≤ √ n + m, yielding y ∈ B(0, √ n + m). Since g m (z) ≤ -δ m , we have g m (y) ≤ g m (z) + L gm y -z 2 ≤ -δ m + L gm w m u 2 ≤ -δ m + L gm δ m 2L gm ≤ - δ m 2 . ( 7
|ξ m (y) -ξ m (z)| = f (y)+ ε 2 gm(y) - f (z)+ ε 2 gm(z) = |(f (y)+ ε 2 )gm(z)-(f (z)+ ε 2 )gm(y)| |gm(y)||gm(z)| ≤ 2 δ 2 m |(f (y) + ε 2 -f (z) -ε 2 )g m (z) + (f (z) + ε 2 )(g m (z) -g m (y))| ≤ 2 δ 2 m [|f (y) -f (z)||g m (z)| + (|f (z)| + ε 2 )|g m (z) -g m (y)|] ≤ 2 δ 2 m [L f y -z 2 C gm + (C f + ε 2 )L gm z -y 2 ] ≤ 2 δ 2 m [L f C gm + (C f + ε 2 )L gm ]w m u 2 ≤ ε 4Cg m ≤ Mm 2 . (7.1.56) Since ξ m (z) ≥ M m , we obtain ξ m (y) ≥ ξ m (z) -|ξ m (y) -ξ m (z)| ≥ M m -Mm 2 = Mm 2 , yielding y ∈ {ξ m ≥ Mm 2 }
, which concludes the proof of (7.1.54) and ensures that

√ ξ m is well-defined on W m . Let us prove that √ ξ m is Lipschitz on W m . Let y, z ∈ W m such that y = z. Then | √ ξm(y)- √ ξm(z)| y-z 2 = |ξm(y)-ξm(z)| y-z 2 ( √ ξm(y)+ √ ξm(z)) ≤ f (y)+ ε 2 gm (y) - f (z)+ ε 2 gm(z) 2 Mm 2 y-z 2 ≤ |(f (y)+ ε 2 )gm(z)-(fm(z)+ ε 2 )gm(y)| 2gm(y)gm(z) ε 4Cg m y-z 2 ≤ 2|(f (y)+ ε 2 )gm(z)-(fm(z)+ ε 2 )gm(y)| δ 2 m ε 4Cg m y-z 2 = 2|(f (y)+ ε 2 -f (z)-ε 2 )gm(z)+(f (z)+ ε 2 )(gm(z)-gm(y))| δ 2 m ε 4Cg m y-z 2 ≤ 2[|f (y)-f (z)||gm(z)|+(|f (z)|+ ε 2 )|gm(z)-gm(y)|] δ 2 m ε 4Cg m y-z 2 ≤ 2[L f y-z 2 Cg m +(C f + ε 2 )Lg m z-y 2] δ 2 m ε 4Cg m y-z 2 ≤ 2[L f Cg m +(C f + ε 2 )Lg m ] δ 2 m ε 4Cg m =: L √ ξm . (7.1.57) Thus, L √ ξm is a Lipschitz constant of √ ξ m on W m . Set K := S m-1 ∩B(0, √ n+m-1), K 1 := K ∩{ξ m ≥ M m }∩{g m ≤ -δ m } and K 2 := K\({ξ m ≥ M m } ∩ {g m ≤ -δ m }). Note that K = K 1 ∪ K 2 and K 1 ∩ K 2 = ∅. From (7.1.50), ϕ m = √ ξ m on K 1 and ϕ m = √ M m on K 2 .
To conclude that ϕ m is Lipschitz on K according to Lemma 7.4 (see Figure 7.1), it is sufficient to prove that ϕ m is locally Lipschitz on K.

Explicitly, we will show that for all z ∈ K, ϕ m is Lipschitz on

B(z, wm 2 ) ∩ K with Lipschitz constant L √ ξm . Let z ∈ K. Let u, v ∈ B(z, wm
2 ) ∩ K and consider the following cases:

• Case 1: u, v ∈ K 1 . Then u, v ∈ W m by definition of W m . Moreover, ϕ m (u) = ξ m (u) and ϕ m (v) = ξ m (v). In this case, by the Lipschitz continuity of √ ξ m on W m , |ϕ m (u) -ϕ m (v)| = | ξ m (u) -ξ m (v)| ≤ L √ ξm u -v 2 . (7.1.58) • Case 2: u, v ∈ K 2 . In this case, ϕ m (u) = ϕ m (v) = √ M m , so that |ϕ m (u) -ϕ m (v)| = 0 ≤ L √ ξm u -v 2 . (7.1.59) • Case 3: u ∈ K 1 and v ∈ K 2 . We claim that B(z, wm 2 ) ⊂ W m . Let q ∈ B(z, wm 2 ). Then q -u 2 ≤ q -z 2 + z -u 2 ≤ w m yielding q ∈ u + w m B n ⊂ K 1 + w m B n ⊂ W m . Then u, v ∈ B(z,
y ∈ B(z, wm 2 ) ∩ {ξ m = M m } ∩ {tu + (1 -t)v : t ∈ [0, 1]}. Then with y = λu + (1 -λ)v for some λ ∈ [0, 1], we have |ϕ m (u) -ϕ m (v)| ≤ |ϕ m (u) -ϕ m (y)| + |ϕ m (y) -ϕ m (v)| ≤ | ξ m (u) -ξ m (y)| + | √ M m - √ M m | ≤ L √ ξm u -y 2 ≤ L √ ξm u -λu -(1 -λ)v 2 ≤ L √ ξm (1 -λ) u -v 2 ≤ L √ ξm u -v 2 .
(7.1.60)

From the proof of Lemma 7.4, the Lipschitz constant of ϕ m on K is given by where the latter inequality is due to the fact that ϕ m is even (since ξ m , g m are even). From this, φm is even.

L φm := max 4C ϕm w m , L √ ξm , ( 7
= ϕ m = √ ψ m on S m-1 ∩ B(0, √ n + m -1), f m-1 ≥ 0 on S m-1 ∩ B(0, √ n + m -1). Since φm is Lipschitz continuous, f m-1 is Lipschitz continuous on B(0, √ n + m).
A bound and a Lipschitz constant of f m-1 on B(0, Then, L fm-1 is a Lipschitz constant of f m-1 on B(0, √ n + m). Notice that C φm , L φm , C fm-1 , L fm-1 are obtained by composing finitely many times the following operators: "+", "-", "×", "÷", "| • |","(x 1 , x 2 ) → max{x 1 , x 2 }", "(x 1 , x 2 ) → min{x 1 , x 2 }", "(•) αm " and " √ •", where all arguments possibly depend on ε. Without loss of generality we can assume

√ n + m) both depend on ε. Let us compute an upper bound of |f m-1 | on B(0, √ n + m). Let y ∈ B(0, √ n + m). Then |f m-1 (y)| ≤ |f (y)| + ε 2 + φm (y) 2 |g m (y)| ≤ C f + ε 2 + C gm C 2 φm =: C fm-1 . (7.1.67) Thus, f m-1 B(0, √ n+m) ≤ C fm-1 . ( 7
C φm = rm ε -rm , L φm = tm ε -tm , C fm-1 = cm-1 ε -cm-1 , L fm-1 = lm-1 ε -lm-1
for some rm , r m , tm , t m , cm-1 , c m-1 , lm-1 , l m-1 large enough and independent of ε. Backward induction. Repeating the above process (after replacing f j by f j-1 ) several times, we obtain functions φj : R n → R, j = m, m -1, . . . , 1, such that, 1. φj is positive, even and bounded from above by C φj = rj ε -rj on B(0, √ n + j) for some positive constants rj and r j independent of ε.

2. φj is Lipschitz with Lipschitz constant L φj = tj ε -tj for some positive constants tj and t j independent of ε.

3. f j-1 := f j + ε 2 m-j+1 -φ2 j g j satisfies: (a) f j-1 ≥ 0 on S j-1 ∩ B(0, √ n + j -1); (b) f j-1 ≤ C fj-1 on B(0, √ n + j)
, where C fj-1 = cj-1 ε -cj-1 for some positive constants cj-1 and c j-1 independent of ε;

(c) f j-1 is Lipschitz on B(0, √ n + j) with Lipschitz constant L fj-1 = lj-1 ε -lj-1 for some positive constants lj-1 and l j-1 independent of ε.

Then f 0 = f 1 + ε 2 m -φ2 1 g 1 = f 2 + ε 2 m-1 -φ2 2 g 2 + ε 2 m -φ2 1 g 1 = f 2 + ε 2 m-1 + ε 2 m -φ2 2 g 2 -φ2 1 g 1 = • • • = f m + ε m i=1 1 2 i - m i=1 φ2 i g i = f + ε 2 1-1 2 m 1-1 2 - m i=1 φ2 i g i = f + ε(1 -1 2 m ) - m i=1 φ2 i g i . (7.1.70)
From this and since

f 0 ≥ 0 on S 0 ∩ B(0, √ n) = B(0, √ n) ⊃ [-1, 1] n , we obtain f + ε - m i=1 φ2 i g i ≥ ε 2 m on [-1, 1] n . ( 7
d i := 2u i with u i = 2C 2 gi C 2 φi nL 2 φi (m + 1) 2 2 2m ε 2 , ( 7.1.75)
where

C gi := g i B(0, √ n+i) , for i ∈ [m]. Then for all x ∈ [-1, 1] n , |B (di) i (x) -φi (x)| ≤ L φi n di 1 2 ≤ L φi n 4C 2 g i C 2 φi nL 2 φi (m+1) 2 2 2m ε 2 1 2 = ε 2Cg i C φi (m+1)2 m . (7.1.76)
Converting to homogeneous approximations. For i ∈ [m], we write B

(di) i = ndi j=0 h (j) i such that h (j) i
is a homogeneous polynomial with deg(h

(j) i ) = j. Set p i := 1 2 [B (di) i (x) + B (di) i (-x)], for i ∈ [m]. Then p i = nui t=0 h (2t) i , for i ∈ [m], since h (j) i (x) = h (j) i (-x) if j is even and h (j) i (x) = -h (j) i (-x) otherwise. Since φi is even, φi (x) = 1 2 [ φi (x) + φi (-x)]. It implies that for x ∈ [-1, 1] n , for i ∈ [m], |p i (x) -φi (x)| = | 1 2 [B (di) i (x) + B (di) i (-x)] -1 2 [ φi (x) + φi (-x)]| ≤ 1 2 |B (di) i (x) -φi (x)| + 1 2 |B (di) i (-x) -φi (-x)| ≤ ε 4Cg i C φi (m+1)2 m + ε 4Cg i C φi (m+1)2 m = ε 2Cg i C φi (m+1)2 m .
|p i (x)| ≤ 1 2 (|B (di) i (x)| + |B (di) i (x)|) ≤ 1 2 (C φi + C φi ) = C φi . (7.1.78) Set q i := nui t=0 h (2t) i x 2(nui-t) 2
. Then q i is a homogeneous polynomial of degree 2nu i and

q i = p i on S n-1 , for i ∈ [m]. Thus for i ∈ [m], |q i (x) -φi (x)| ≤ ε 2Cg i C φi (m+1)2 m and |q i (x)| ≤ C φi ,
for all x ∈ S n-1 . From these and (7.1.71), for all x ∈ S n-1 ,

f (x) + ε - m i=1 q i (x) 2 g i (x) = f (x) + ε - m i=1 φi (x) 2 g i (x) + m i=1 g i (x)[ φi (x) 2 -q i (x) 2 ] ≥ ε 2 m - m i=1 |g i (x)|| φi (x) + q i (x)|| φi (x) -q i (x)| ≥ ε 2 m - m i=1 C gi (| φi (x)| + |q i (x)|) ε 2Cg i C φi (m+1)2 m ≥ ε 2 m - m i=1 2C gi C φi ε 2Cg i C φi (m+1)2 m = ε 2 m - mε (m+1)2 m = ε (m+1)2 m . (7.1.79)
Moreover, for all x ∈ S n-1 ,

f (x) + ε - m i=1 q i (x) 2 g i (x) ≤ C f + ε + m i=1 C 2 φi C gi =: C F . (7.1.80)
Applying the global positivity certificate

Set D := max i∈[m] {2nu i + d gi , d f } and F = x 2(D-d f ) 2 (f + ε x 2d f 2 ) - m i=1 g i q 2 i x 2(D-2nui-dg i ) 2 . (7.1.81)
Then F is a homogeneous polynomial of degree 2D and for all x ∈ S n-1 , Clearly there exist positive constants c and c independent of ε such that cε -c ≥ K. Let K ∈ N and K ≥ cε -c ≥ K. According to Lemma 1.3.1, there exists a homogeneous SOS polynomial s 0 of degree 2(D + K) such that x 2K 2 F = s 0 . It implies that

C F ≥ F (x) = f (x) + ε - m i=1 q i (x) 2 g i (x) ≥ ε (m+1)2 m . ( 7
x 2(D-d f +K) 2 (f + ε x 2d f 2 ) = s 0 + m i=1 g i q 2 i x 2(D-2nui-dg i +K) 2 = s 0 + m i=1 g i s i , (7.1.86)
where

s i := q 2 i x 2(D-2nui-dg i +K) 2
is a homogeneous SOS polynomial such that deg(g

i s i ) = 2(K + D), for i ∈ [m]. Set k = D -d f + K. Then x 2k 2 (f + ε x 2d f 2 ) = s 0 + m i=1 g i s i with deg(s 0 ) = deg(g i s i ) = 2(k + d f ), for i ∈ [m].
The case of the ice cream constraint. Assume that m = 1 and g 1 = x 2 n -x 2 2 with x := (x 1 , . . . , x n-1 ). We shall show that c = 65. Using Lemma 7.3, we can take α m = 2 in (7.1.43). We then obtain the following asymptotic equivalences as ε → 0 + :

δ m ∼ R 1 ε 2 ⇒ C ψm ∼ R 2 ε -2 ⇒ C ϕm ∼ R 3 ε -1 ⇒ w m ∼ R 4 ε 5 ⇒ L √ ξm ∼ R 5 ε -9 2 ⇒ L φm ∼ R 6 ε -6 ⇒ C φm ∼ R 7 ε -6 ⇒ u m ∼ R 8 ε -26 ⇒ d m ∼ R 9 ε -26 ⇒ C F ∼ R 10 ε -12 ⇒ D ∼ R 11 ε -26 ⇒ b = 12 ⇒ d = 26 ⇒ K ∼ R 12 ε -65 ⇒ c = 65. (7.1.87)
for some R j > 0 independent of ε, j ∈ [START_REF] Bajbar | Coercive polynomials and their Newton polytopes[END_REF]. This completes the proof of Theorem 7.1.

Polynomial optimization

This section is concerned with some applications to polynomial optimization. Consider the following POP:

f := inf x∈S(g) f (x) , (7.2.1)
where

f ∈ R[x] and S(g) = {x ∈ R n : g i (x) ≥ 0 , i ∈ [m]} , (7.2.2) for some g = {g i } i∈[m] ∈ R[x]. Recall that θ = 1 + x 2 2 .

General case

In this subsection, we improve the convergence rate of the Moment-SOS hierarchy described in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Theorem 4.3], based on Putinar-Vasilescu's Positivstellensatz [START_REF] Putinar | Positive polynomials on semi-algebraic sets[END_REF]. Theorem 7.2. Let f, g 1 , . . . , g m be polynomials such that f defined as in (7.2.1) and S(g) defined as in (7.2.2) satisfy that S(g) has nonempty interior and f > -∞. Let ε > 0 and denote g 0 := 1. Let d := deg(f )/2 + 1. Consider the hierarchy of semidefinite programs indexed by k ∈ N:

τ (ε) k := inf L y (θ k (f + εθ d )) s.t. y = (y α ) α∈N n 2(d+k) ⊂ R , M k+d (y) 0 , M k+d-gi (g i y) 0, i ∈ [m] , L y (θ k ) = 1 . (7.2.3)
For every k ∈ N, the dual of (7.2.3) reads as:

ρ (ε) k := sup λ∈R {λ : θ k (f -λ + ε θ d ) ∈ Q k+d (g)} , (7.2.4)
where Q r (g) is defined as in (1.3.8). The following statements hold:

1. For all k ∈ N, ρ (ε) k ≤ ρ (ε) k+1 ≤ f . (7.2.5)
2. Assume that problem (7.2.1) has an optimal solution x . Then there exist positive constants c and c depending on f, g 1 , . . . , g m such that for all k ≥ cε -c ,

0 ≤ ρ (ε) k -f ≤ εθ(x ) d . ( 7

Compact case

In this subsection, we consider the case when S(g) is compact by assuming that a ball constraint is present. We can then remove the perturbation term εθ d in the hierarchy based on Putinar-Vasilescu's Positivstellensatz, described in the previous subsection. Theorem 7.3. Let f, g 1 , . . . , g m be polynomials such that f defined as in (7.2.1) and S(g) defined as in (7.2.2) satisfy that S(g) has nonempty interior and f > -∞. Denote g 0 := 1. Let d := deg(f )/2 +1. Assume that g 1 = R-x 2 2 for some R > 0. Consider the hierarchy of semidefinite programs indexed by k ∈ N:

τ (0) k := inf L y (θ k f ) s.t. y = (y α ) α∈N n 2(d+k) ⊂ R , M k+d (y) 0 , M k+d-gi (g i y) 0, i ∈ [m] , L y (θ k ) = 1 . (7.2.7)
For every k ∈ N, the dual of (7.2.7) reads as:

ρ (0) k := sup λ∈R {λ : θ k (f -λ) ∈ Q k+d (g)} , (7.2.8)
where Q r (g) is defined as in (1.3.8). The following statements hold:

1. For all k ∈ N, ρ (0) k ≤ ρ (0) k+1 ≤ f . (7.2.

9)

2. There exist positive constants c and c depending on f, g 1 , . . . , g m such that

0 ≤ f -ρ (0) k ≤ c k 1 c
(7.2.10)

3.

Strong duality holds for all orders k of the primal-dual problems (7.2.7)-(7.2.8).

Proof. The first and third statements of Theorem 7.3 can be proved similarly to the ones of Theorem 7.2. Let us prove the second statement. By using Corollary 7.2, there exist positive constants c and c depending on f, g 1 , . . . , g m such that for any ε > 0, for all k ≥ cε -c ,

θ k (f -f + ε) ∈ Q k+d (g) . (7.2.11) Let K ∈ N. Set = ( c K ) 1 c
. Then > 0 and K = c -c , so that

θ K (f -f + ) ∈ Q K+d (g) . (7.2.12)
It implies that f -is a feasible solution of (7.2.8) with relaxation order K, yielding

0 ≤ f -ρ (0) K ≤ f -(f -) = = c K 1 c . (7.2.13)
Hence the desired result follows.

Remark 7.5. The authors' bounds in Theorems 7.2 and 7.3 are only for worst cases. In fact, for generic cases of polynomials, the Moment-SOS hierarchy based on Putinar's Positivstellensatz (in [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF]) has finite convergence [START_REF] Nie | Optimality conditions and finite convergence of Lasserre's hierarchy[END_REF].

Chapter 8

A sparse version of Reznick's Positivstellensatz

Most of the content of this chapter is from [START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF].

Inspired by correlative sparsity due to Waki et al. [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF][START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF] and Putinar-Vasilescu's Positivstellensatz that is applied for polynomial optimization in the previous two chapters, we prove a sparse version of Reznick's Positivstellensatz. Accordingly we obtain some representations that involve uniform denominators and quadratic modules in the sparse setting.

Exploiting sparsity pattern.

Let n, m ∈ N >0 . For T ⊂ [n], denote by R[x(T )] (resp. Σ[x(T )]) the ring of polynomials (resp. the subset of SOS polynomials) in the variables x(T ) := {x j : j ∈ T }. Also denote by R[x(T )] t (resp. Σ[x(T )] t ) the restriction of R[x(T )] (resp. Σ[x(T )]) to polynomials of degree at most t (resp. 2t). For W ⊂ [m], we note g W := {g i : i ∈ W }.

Designing alternative hierarchies for solving f := inf{f (x) : x ∈ S(g)}, significantly (computationally) cheaper than their dense version (1.2.4), while maintaining convergence to the optimal value f is a real challenge with important implications.

One first such successful contribution is due to Waki et al. [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] when the input polynomial data f, g i are sparse, where by sparse we mean the following:

i ∈ J c , g i ∈ R[x(I c )], c ∈ [p].
(iv) Additional redundant quadratic constraints: There exists R > 0 such that x 2 2 ≤ R for all x ∈ S(g) and R -

x(I c ) 2 2 ∈ g Jc , c ∈ [p].
With τ (≤ n) being the maximum number of variables appearing in each index subset I c , i.e., τ := max{n c : c ∈ [p]}, Table 8.1 displays the respective computational complexity of the sparse hierarchy of Waki et al. [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF] and the dense hierarchy of Lasserre [START_REF] Lasserre | Global optimization with polynomials and the problem of moments[END_REF] for SDPs with same relaxation order k ∈ N. Obviously the sparse hierarchy provides a potentially high computational saving when compared to the dense one. In addition, convergence of the hierarchy of Waki et al. to the optimal value of the original POP was proved in [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF], resulting in the following sparse version of Putinar's Positivstellensatz:

0,c ∈ Σ[x(I c )] k , σ i,c ∈ Σ[x(I c )] k-gi , i ∈ J c , c ∈ [p] such that f = p c=1 σ 0,c + i∈Jc σ i,c g i . (8.0.1)
Compactness of the feasible set S(g) is a crucial ingredient of the proof in [START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF]; shortly after, Grimm et al. [START_REF] Grimm | A note on the representation of positive polynomials with structured sparsity[END_REF] provided another (simpler) proof where int(S(g)) = ∅ is not needed, but where compactness of S(g) is still a crucial assumption.

Motivation for sparse representations on noncompact sets. We remark that Theorem 8.1 requires the additional redundant quadratic constraints (Assumption 8.1 (iv)), which is slightly stronger than just assuming the compactness of S(g). When S(g) is compact, we can always add these constraints but we need to know the radius R > 0 of a ball centered at the origin and containing S(g). In this case, adding such constraints increases the number of positive semidefinite matrices from m to m + p in each SDP. In addition, it may be hard to verify compactness of S(g) and obtain such a radius R.

To the best of our knowledge, in the noncompact case there is still no Positivstellensatz allowing one to build hierarchies for POPs satisfying:

-the RIP and the structured sparsity pattern from Assumption 8.1 (i)-(iii), -and a guarantee of convergence to the global optimum.

In fact we provide Examples 6.1, 8.3, and 8.4, which show that in both unconstrained and constrained cases, there exist sparse nonnegative polynomials which do not have a sparse SOSbased decomposition (8.0.1) à la Putinar. Such examples have been our motivation to investigate existence of sparse representations in the noncompact case, as well as to construct converging SDP-hierarchies for sparse polynomial optimization in general.

Contribution. Our contribution is threefold:

I. We first provide a rational SOS representation for a positive definite rational form which is a sum of sparse rational functions with uniform denominators, satisfying the structured sparsity pattern and the RIP stated in Assumption 8.1 (i). This representation is provided in Theorem 8.2. As a direct consequence, we obtain a sparse version of Reznick's Positivstellensatz in Corollary 8.1.

II. Then, we provide two positivity certificates for arbitrary small perturbations of -globally nonnegative polynomials in Corollary 8.2 -and polynomials nonnegative on a (possibly noncompact) basic semialgebraic set in Corollary 8.3, when the input data satisfy a similar sparsity pattern. These two certificates are obtained via a sparse version of Putinar-Vasilescu's Positivstellensatz and do not require the additional constraints from Assumption 8.1 (iv).

III. In Section 8.2, we build up a hierarchy of semidefinite relaxations for polynomial optimization based on the sparse version of Putinar-Vasilescu's Positivstellensatz. Convergence of the hierarchy is guaranteed and illustrated on minimization of random quadratic forms on the nonnegative orthant. However a naive implementation of this hierarchy leads to a heavy computational burden when the number of variables is larger than 10.

Illustrations of such positivity certificates for polynomials nonnegative on noncompact basic semialgebraic sets are provided in Example 8.1, 6.1, 8.3 and 8.4, for which positivity certificates (1.2.4) do not exist. The existence of such sparse SOS-representations is proved by combining different tools:

(a) First, we use an idea similar to that developed in Grimm et al. [START_REF] Grimm | A note on the representation of positive polynomials with structured sparsity[END_REF] (in the compact case) to prove that a sparse positive definite form can be decomposed as SOS of sparse positive definite rational forms; as expected the noncompact case is technically more involved. This yields a sparse version of Hilbert-Artin's representation theorem in the case of positive definite forms.

(b) Next, we use generalizations of Schmüdgen's Positivstellensatz presented by Schweighofer [START_REF] Schweighofer | Iterated rings of bounded elements and generalizations of Schmüdgen's Positivstellensatz[END_REF], Berr-Wörmann [START_REF] Berr | Positive polynomials on compact sets[END_REF], Jacobi [START_REF] Jacobi | A representation theorem for certain partially ordered commutative rings[END_REF], and Marshall [START_REF] Marshall | Extending the Archimedean Positivstellensatz to the non-compact case[END_REF][START_REF] Marshall | A general representation theorem for partially ordered commutative rings[END_REF], for a finitely generated R-algebra in each term of the sum, to obtain again a sparse version, this time of Reznick's Positivstellensatz for positive definite forms.

(c) Finally we combine the homogenization/dehomogenization method that we already used in [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF] together with limit tools, to provide the two sparse versions of Putinar-Vasilescu's Positivstellensatz.

We acknowledge that the computational benefits are so far rather limited and that the present contribution is essentially theoretical, as it provides a sparse analogue of Reznick's Positivstellenstaz. In our opinion, the sparse analogue of Putinar's Positivstellenstaz is still the "champion" algorithm to beat. Indeed the versatility of its power, which applies to both dense (but of modest size) problems and large size sparse problems, is yet to be surpassed. So the practical benefits of sparse Reznick Positivstellensatz are not immediately available with its obvious (but naive) implementation.

In order to address this computational issue, we propose to use a sampling technique in the spirit of that advocated by Parrilo and Löfberg [START_REF] Lofberg | From coefficients to samples: a new approach to sos optimization[END_REF] for polynomial optimization, and briefly described in Section 8.2.3. In our context it allows us to avoid clearing denominators and its efficiency is illustrated on some numerical experiments presented in Section 8.2.4. Its complete validation for rational functions is beyond the scope of the present chapter and we believe that in view of the appealing form of our sparse version of Reznick's Positivstellensatz, additional investigation of powerful algorithmic implementations are worth pursuing.

Representation theorems

Notation and definitions

A function h is homogeneous of degree t if h(λx) = λ t h(x) for all x ∈ R n and each λ ∈ R. Therefore a homogeneous polynomial can be written as h = |α|=t h α x α . A function f : R n → R is even if f (x) = f (-x) for all x. A rational function h is the ratio of two polynomials and denote by R(x) the space of all rational functions. A homogeneous rational function (also called be a rational form, or form in short) can be written as the ratio of two homogeneous polynomials.

The degree-d homogenization h of h ∈ R(x 1 , . . . , x n ) is a homogeneous rational function in R(x 1 , . . . , x n+1 ) of degree d defined by h(x, x n+1 ) = x d n+1 h(x/x n+1 ). A rational positive definite form of degree t is a homogeneous rational function of degree t which is positive everywhere except at the origin. Equivalently, a homogeneous rational function h of degree t is a rational positive definite form of degree t if and only if there exists ε > 0 such that h ≥ ε x 2t 2 . For (i, j) ∈ N 2 , we denote the Kronecker delta function by

δ i,j := 1 if i = j , 0 if i = j .
When Assumption 8.1 (i) holds, define

Φ c := x( Îc ) 2(1-δc,1) 2 p j=c+1 x( Îj ) 2δc,s j 2 if c ∈ [p -1] ,
x( Îc )

2(1-δc,1) 2 if c = p .
Obviously, one has Φ c ∈ R[x(I c )], for each c ∈ [p].

A key result

Let us state the first main result of this chapter which is of independent interest and very useful in proving our representation results. In particular it yields a sparse version of Reznick's Positivstellensatz as a particular case.

f 1 := x 2 4 (x 4 1 x 2 2 + x 4 2 x 2 3 + x 2 1 x 4 3 -3x 2 1 x 2 2 x 2 3 ) + x 8 3
is the so-called Delzell's polynomial and f 2 := x 2 1 x 2 2 x 2 3 x 2 5 . The polynomial f 1 is nonnegative, but not SOS as shown in [176, Section 6]. Let I 1 := {1, 2, 3, 4} and I 2 := {1, 2, 3, 5}. Then f 1 ∈ R[x(I 1 )] and f 2 ∈ R[x(I 2 )] are nonnegative and homogeneous of degree 8. Since f 1 is nonnegative then f is nonnegative. The following statements hold:

1. f is a nonnegative form, but is not positive definite; 2. f / ∈ Σ[x(I 1 )] + Σ[x(I 2 )], but f ∈ Σ[x(I1)]6 x(I1) 2 2 Φ1 + Σ[x(I2)]6
x(I2) 2 2 Φ2 . The first statement follows from the fact that f (0, 0, 0, 1, 1) = 0, ensuring that f is not a positive definite form. Proof of the second statement: Assume by contradiction that

f = σ 1 + σ 2 for some σ c ∈ Σ[x(I c )], c = 1, 2. Evaluation at x 5 = 0 yields f 1 = σ 1 + σ 2 (x 1 , x 2 , x 3 , 0), so that f 1 is an SOS, which is impossible. Thus, f / ∈ Σ[x(I 1 )] + Σ[x(I 2 )]. However, (x 2 1 + x 2 2 + x 3 3 )f 1 is SOS according to [182, Example 4.4], so (x 2 1 + x 2 2 + x 2 3 )f ∈ Σ[x(I 1 )] 5 + Σ[x(I 2 )] 5 . Note that Φ 1 = Φ 2 = x 2 1 + x 2 2 + x 2 3 . Therefore f ∈ Σ[x(I 1 )] 5 Φ 1 + Σ[x(I 2 )] 5 Φ 2 ⊂ Σ[x(I 1 )] 6 H 1 + Σ[x(I 2 )] 6 H 2 .

Remark 8.1. A possibly simpler proof of Corollary 8.1 based on Carathéodory's theorem.

In [START_REF] Barvinok | A course in convexity[END_REF]Chapter I,Section 3], Barvinok provides a simple proof of Reznick's theorem, based on Carathéodory's theorem. The basic idea is that a large enough power of the linear form v, x 2k is a Dirac function on the unit sphere, centered at v. Therefore x 2k 2 f (which is the same as f on the unit sphere, and strictly positive) can naturally be written as a positive combination of Dirac functions, which can be in turn decomposed as a positive weighted sum of powers of linear forms. Compared to Reznick's proof, this

f 1 := 8 + 1 2 x 2 1 x 4 2 + (x 2 1 -2x 3 1 )x 3 2 + (2x 1 + 10x 2 1 + 4x 3 1 + 3x 4 1 )x 2 2 + 4(x 1 -2x 2 1 )x 2
is the so-called Leep-Starr's polynomial and f 2 := x 2 1 x 2 3 . Let I 1 := {1, 2} and I 2 := {1, 3}, so that [START_REF] Leep | Polynomials in R[X,Y] that are sums of squares in R(X,Y)[END_REF]Example 2], f 1 is nonnegative but not an SOS. In addition, f 2 is an SOS, so that f is nonnegative.

f 1 ∈ R[x(I 1 )] and f 2 ∈ R[x(I 2 )]. As shown in
We claim that f / ∈ Σ[x(I 1 )] + Σ[x(I 2 )]. Indeed, assume by contradiction that

f = σ 1 + σ 2 for some σ c ∈ Σ[x(I c )], c = 1, 2. Evaluation at x 3 = 0, yields f 1 = σ 1 + σ 2 (x 2 , 0), so that f 1 is an SOS, which is impossible.
However, (x 2 1 + 1) 2 f 1 is a sum of three squares of polynomials according to [START_REF] Leep | Polynomials in R[X,Y] that are sums of squares in R(X,Y)[END_REF]Example 2], so

(x 2 1 + 1) 2 f ∈ Σ[x(I 1 )] 5 + Σ[x(I 2 )] 5 . Note that D 1 = D 2 = x 2 1 + 1. Thus, f ∈ Σ[x(I 1 )] 5 D 2 1 + Σ[x(I 2 )] 5 D 2 2 ⊂ Σ[x(I 1 )] 7 Θ 2 1 + Σ[x(I 2 )] 7 Θ 2 2 .
Example 8.3. As shown in [START_REF] Klep | Sparse noncommutative polynomial optimization[END_REF]Example 5.2], the nonnegative polynomial

f = x 2 1 -2x 1 x 2 + 3x 2 2 -2x 2 1 x 2 + 2x 2 1 x 2 2 -2x 2 x 3 + 6x 2 3 + 18x 2 2 x 3 -54x 2 x 2 3 + 142x 2 2 x 2 3 satisfies f ∈ R[x(I 1 )] + R[x(I 2 )] and f / ∈ Σ[x(I 1 )] + Σ[x(I 2 )], with I 1 = {1, 2} and I 2 = {2, 3}. However, f ∈ Σ[x(I1)]4 Θ1 + Σ[x(I2)]4 Θ2
, where Θ 1 = (x 2 2 + 1)(x 2 1 + x 2 2 + 1) and Θ 2 = (x 2 2 + 1)(x 2 2 + x 2 3 + 1). It is due to the fact that f = σ1 D1 + σ2 D2 , where D 1 = D 2 = x 2 2 + 1 and σ 1 , σ 2 are SOS polynomials given in [START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF]Appendix].

Positivity on a semialgebraic set

We next state our second main result, namely a sparse version of Putinar-Vasilescu's Positivstellensatz for polynomials nonnegative on (possibly noncompact) basic semialgebraic sets.

Let f = f 1 + f 2 , where f 1 = x 1 x 2 and f 2 = x 2 2 x 3 . Let g = {g 1 , g 2 , g 3 }, where g 1 = x 3
2 , g 2 = -g 1 and g 3 = x 3 . It is not hard to show that f = 0 on S(g), so that f ≥ 0 on S(g). By noting I 1 := {1, 2} and

I 2 := {2, 3}, one has {f 1 , g 1 , g 2 } ⊂ R[x(I 1 )] and {f 2 , g 3 } ⊂ R[x(I 2 )].
We claim the following statements:

1. f / ∈ Σ[x(I 1 )] + g 1 R[x(I 1 )] + Σ[x(I 2 )] + g 3 Σ[x(I 2 )];
2. for every ε > 0,

f + ε(θ 2 1 + θ 2 2 ) ∈ Σ[x(I 1 )] 2k+2 + g 1 R[x(I 1 )] 4k+1 Θ k 1 + Σ[x(I 2 )] 2k+2 + g 3 Σ[x(I 2 )] 4k+3 Θ k 2 ,
for some k ∈ N depending on ε.

Proof of the first statement: Assume by contradiction that there exist σ 1 ∈ Σ[x(I 1 )], ψ 1 ∈ R[x(I 1 )] and σ 2 , σ 3 ∈ Σ[x(I 2 )] such that f = σ 1 + ψ 1 g 1 + σ 2 + σ 3 g 3 . Evaluation at x 1 = 1 and x 3 = 0 yields

x 2 = σ 1 (1, x 2 ) + ψ 1 (1, x 2 )x 3 2 + σ 2 (x 2 , 0) ∈ Σ[x 2 ] + x 3 2 R[x 2 ] ,
which is impossible due to [START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF]Lemma 3.3 (i)].

Proof of the second statement: With ε > 0 fixed,

f 1 + εθ 2 1 = x 1 x 2 + ε(1 + x 2 1 + x 2 2 ) 2 = x 1 x 2 + ε + εx 2 1 + σ 4 , for some σ 4 ∈ Σ[x(I 1 )] 2 . Let k ∈ N ≥2 be fixed. Then D k 1 = (1 + x 2 2 ) k = 1 + kx 2 2 + x 4 2 σ 5 for some σ 5 ∈ Σ[x 2 ] k-2 , which implies D k 1 (f 1 + εθ 2 1 ) = x 1 x 2 + εx 2 1 + εkx 2 2 + σ 6 + ψ 2 x 3 2 ,
for some

σ 6 ∈ Σ[x(I 1 )] k+2 and ψ 2 ∈ R[x(I 1 )] 2k+1 . Assume that k ≥ ε -2 /4. Then D k 1 (f 1 + εθ 2 1 ) = x 2 1 ε -1 4εk + x 2 √ εk + x1 2 √ εk 2 + σ 6 + ψ 2 x 3 2 ∈ Σ[x(I 1 )] k+2 + g 1 R[x(I 1 )] 2k+1 , which implies f 1 + εθ 2 1 ∈ Σ[x(I1)] 2k+2 +g1R[x(I1)] 4k+1 Θ k 1 .
We also have

f 2 + εθ 2 2 ∈ Σ[x(I 2 )] 2k+2 + g 3 Σ[x(I 2 )] 4k+3 Θ k 2 since f 2 ∈ g 3 Σ[x(I 2 )] 1
, proving the second statement.

General case

For f ∈ R[x], define

d := deg(f )/2 if S(g) = R n , 1 + deg(f )/2 if S(g) = R n . (8.1.5)
For l ∈ [p], let us note

Q(g Jc )[x(I c )] k = σ 0,c + i∈Jc σ i,c g i σ 0,c ∈ Σ[x(I c )] k , σ i,c ∈ Σ[x(I c )] k-gi , i ∈ J c (8.1.6)
the truncated quadratic module of order k associated with the polynomials g Jc = {g i : i ∈ J c }.

Gathering the two situations from Corollary 8.2 and Corollary 8.3, we obtain the following general statement. Theorem 8.3. Let f ∈ R[x] be nonnegative on S(g). Let us suppose that the conditions (i) and (ii) of Assumption 8.1 hold if S(g) = R n , otherwise let us suppose that the conditions (i), (ii) and (iii) of Assumption 8.1 hold. Let us fix ε > 0 and let d be defined as in (8.1.5). Then there exists k ∈ N such that

f + ε p c=1 θ d c ∈ p c=1 Q(g Jc )[x(I c )] d+kωc Θ k c . (8.1.7)

Application to polynomial optimization

Based on Theorem 8.3, we build up two hierarchies of SDP relaxations for unconstrained sparse POPs and sparse POPs on possibly noncompact basic semialgebraic sets.

Consider the general POP:

f := inf x∈S(g) f (x) . (8.2.1)
When the condition (i) of Assumption 8.1 holds, with d defined as in (8.1.5) one notes:

• ψ d := p c=1 θ d c ; • φ := p c=1 Θ c ; • φ c := φ/Θ c = p r=1,r =c Θ r , c ∈ [p].
Let ε > 0 be fixed. Let us suppose that the conditions (i) and (ii) of Assumption 8.1 hold if S(g) = R n , otherwise let us suppose that the conditions (i), (ii) and (iii) of Assumption 8.1 hold.

Semidefinite relaxations

Consider the hierarchy of semidefinite programs indexed by k ∈ N:

ρ sparse k (ε) := sup λ ∈ R : φ k (f -λ + εψ d ) ∈ p c=1 φ k c Q(g Jc )[x(I c )] d+kωc . ( 8

Duality

For every k ∈ N, the dual of (8.2.2) reads:

τ sparse k (ε) := inf y L y (φ k (f + εψ d )) s.t. y = (y α ) α∈N n 2(d+pk maxc ωc ) ⊂ R , M d+kωc (φ k c y, I c ) 0 , c ∈ [p] , M d+kωc-ui (φ k c g i y, I c ) 0 , i ∈ J c , c ∈ [p] , L y (φ k ) = 1 .
(ε) ≥ ρ sparse k (ε) ≥ f > -∞. However, when k is small, τ sparse k (ε) = -∞ may happen.

Sampling technique

Even though our sparse variant of Reznick's Positivstellensatz has an appealing shape, the computational benefits can be limited in the context of polynomial optimization. It is due to the very large number of constraints involved in the SDP relaxations when we clear the denominators in these certificates. To avoid this explosion of SDP constraints, one possible route is to use the sampling technique suggested in Parrilo and Löfberg [START_REF] Lofberg | From coefficients to samples: a new approach to sos optimization[END_REF] for semidefinite relaxations in polynomial optimization. To state that two polynomials are identical, instead of equating their coefficients one rather states that their respective values at sufficiently many points are equal.

More explicitly, in our context we take a sample (a i ) N i=1 ⊂ R n and consider the following SDP relaxations indexed by k ∈ N:

ρ sample k,N (ε) := sup λ s.t f (a i ) -λ + ε p c=1 θ c (a i ) d = p c=1 σ0,c(ai)+ j∈Jc σj,c(ai)gj (ai) Θc(ai) k , i ∈ [N ] , σ 0,c ∈ Σ[x(I c )] d+kωc , σ j,c ∈ Σ[x(I c )] d+kωc-gj , j ∈ J c , c ∈ [p] . (8.2.4) It is not hard to see that ρ sparse k (ε) ≤ ρ sample k,N (ε) ≤ ρ sample k,N -1 (ε) . (8.2.5)
Since the denominators Θ k c have fixed forms, their evaluations Θ c (a i ) k become constants in SDP (8.2.4). Thus, we do not need to clear the denominators as in the SDP (8.2.2) with value ρ sparse k (ε). The constraints

f (a i ) -λ + ε p c=1 θ c (a i ) d = p c=1 σ 0,c (a i ) + j∈Jc σ j,c (a i )g j (a i ) Θ c (a i ) k , i ∈ [N ] ,
directly provide linear constraints on the coefficients of the polynomial weights σ j,c . The underlying rationale behind (8.2.4) comes from the following observation in the case of sparse polynomial optimization, i.e., without denominators. Let us consider the simplified equality constraint:

p c=1 f c = p c=1 (σ 0,c + i∈Jc σ i,c g i ) , (8.2.6) with σ 0,c ∈ Σ[x(I c )] k and σ i,c ∈ Σ[x(I c )] k-gi .
The polynomials involved in both left-hand side and right-hand side of (8.2.6) have degree less than 2k.

If we do not exploit sparsity, we need to sample N = 2k+n n points (a i ) N i=1 ⊂ R n , in order to potentially obtain a set of equality constraints equivalent to (8.2.6). Let us consider the multidimensional Vandermonde matrix associated to (a i ) N i=1 , with rows indexed by all monomials of degree ≤ 2k. If this Vandermonde matrix is invertible, then the set of equations obtained after sampling is equivalent to (8.2.6) as a consequence of Theorem 4.1 from [START_REF] Olver | On multivariate interpolation[END_REF], namely the N points are in this case so-called "polynomially poised" if and only if they do not belong to a common algebraic hypersurface of degree ≤ 2k.

If we exploit sparsity, we can do a similar reasoning with the multi-dimensional sparse Vandermonde matrix, with rows indexed by all sparse monomials of degree ≤ 2k. In this case, the number of rows is upper bounded by

p c=1 2k+c c
, and so is the number of required sample points. By using Theorem 4.1 from [START_REF] Olver | On multivariate interpolation[END_REF], the points are poised if and only if the Vandermonde matrix is non-singular, which guarantees a unique sparse interpolating polynomial of degree ≤ 2k passing through the data points.

Obtaining similar equivalent conditions in the case of rational function evaluation is left for future work. Our numerical experiments presented in the following section suggest that we obtain the same value as ρ sparse k (ε) with (8.2.4), after selecting a large enough number N of samples.

Numerical experiments

The main goal of this section is to illustrate the correctness of our representation results, on a sample of nontrivial polynomials involving up to 10 variables. Let us report the numerical results obtained with SDP (8.2.2) to approximate the minimum of quadratic polynomials on the nonnegative orthant. The quadratic polynomials are generated randomly as follows:

1. Take u ∈ N, p := n/u and

I c =      {1, . . . , u} if c = 1 , {u(c -1), . . . , uc} if c ∈ {2, . . . , p -1} , {u(p -1), . . . , n} if c = p . (8.2.7) 2. Let f = p c=1 f c such that f c ∈ R[x(I c )] 2 , c ∈ [p].
For each c ∈ [p], the coefficient f c,α of f c is generated randomly in (0, 1) with respect to the uniform distribution, for every α ∈ N Ic 2 \{0} and f c,0 = 0.

3. Take m = n and g j := x j , j ∈ [n]. Then S(g) is the nonnegative orthant which is a noncompact set. Set J 1 = I 1 and J c = I c \ ∪ c-1 i=1 I i , for c = 2, . . . , p.

4. Take a sequence (a i ) N i=1 ⊂ [-1, 1] n of uniformly random sample points as mentioned in Section 8.2.3.

With the above setting, the conditions (i), (ii) and (iii) of Assumption 8.1 hold and the optimal value of the corresponding POP (8.2.1) should be f = 0. The experiments are performed with the JuMP Julia package [START_REF] Dunning | Jump: A modeling language for mathematical optimization[END_REF], relying on the Mosek solver [START_REF] Dahl | Semidefinite optimization using mosek[END_REF] to solve the SDP relaxation (8.2.2). We use a desktop computer with an Intel(R) Core(TM) i7-8665UCPU @ 1.9GHz×8 and 31.2 GB of RAM. The numerical results are displayed in Table 5.16 with the following information:

• nummat: number of semidefinite matrices involved in SDP (8.2.2);

• numcons: number of constraints involved in SDP (8.2.2);

• val: the optimal value of SDP (8.2.2);

• time: the total time in seconds;

• the symbol "-" means that the SDP solver runs out of memory. , where d is the smallest positive integer such that 2d > deg(f ). Note that k needs to be sufficiently large to ensure that the cone

p c=1 φ k c Q(g Jc )[x(I c )] d+kωc in (8.2.2) is well-defined, namely 2(d + k min c∈[p] ω c ) ≥ max i∈[m]
deg(g i ). Nevertheless, we still obtain the optimal value of SDP relaxation (8.2.4) for n ≥ 12 by using the sampling technique from Section 8.2.3. The value is also close to the exact optimal value of POP (8.2.1). The number of constraints involved in SDP (8.2.4) is equal to the sample size N which is much smaller than the one involved in SDP (8.2.2).

Chapter 9

Exploiting nonnegativity of variables

In the previous three chapters, we have relied on uniform denominators to solve POPs over possibly noncompact semialgebraic sets. In this chapter, we present another advantage related to uniform denominators. More explicitly, such denominators enable us to decompose the Gram matrices, arising in the SOS strengthenings of POPs over the nonnegative orthant, into finitely many smaller ones with prescribed sizes.

For each A ⊂ N n , denote v A (x) = (x α ) α∈A . We say that a polynomial q is even in each variable if for every j ∈

[n], q(x 1 , . . . , x j-1 , -x j , x j+1 , . . . , x n ) = q(x 1 , . . . , x j-1 , x j , x j+1 , . . . , x n ). A polynomial q is called a SOS of monomials if q = α∈N n λ α x 2α for some λ α ≥ 0. Accordingly, if q is an SOS of monomials, then q = v d diag(u)v d for some d ∈ N and u ∈ R b(n,d) + .
Factor width: Originally defined in [START_REF] Boman | On factor width and symmetric H-matrices[END_REF], the factor width of a real positive semidefinite matrix G is the smallest integer s for which there exists a real matrix P such that G can be decomposed as G = PP and each column of P contains at most s nonzeros. In this case, if u is a vector of several monomials in x, the SOS polynomial u Gu can be written as u(x) Gu(x) = i (q i u(x)) 2 , where q i is the i-th column of P. It is not hard to prove that the Gram matrix of each square (q i u(x)) 2 has size at most s since q i has at most s nonzeros. Thus, if an SOS polynomial has Gram matrix of factor width at most s, it can be written as a sum of SOS polynomials with Gram matrix sizes at most s. The inverse also holds true thanks to eigen decomposition. The applications of factor width for polynomial optimization can be found in, e.g., [3,[START_REF] Miller | Decomposed structured subsets for semidefinite and sum-of-squares optimization[END_REF].

POP with nonnegative variables:

In the present chapter, we focus on the following POP on the nonnegative orthant:

f := inf x∈S f (x) , (9.0.1)
where f is a polynomial and S is a semialgebraic set defined by

S := {x ∈ R n : x j ≥ 0 , j ∈ [n] , g i (x) ≥ 0 , i ∈ [m]} , (9.0.2) for some g i ∈ R[x], i ∈ [m]
with g m := 1. Letting q(x) := q(x 2 ) (with x 2 := (x 2 1 , . . . , x 2 n )) whenever q ∈ R[x], it follows immediately that problem (9.0.1) is equivalent to solving

f = inf x∈ Š f (x) , (9.0.3)
where Š is a subset of R n defined by

Š := {x ∈ R n : ǧi (x) ≥ 0 , i ∈ [m]} . ( 9
.0.4)

DSOS and SDSOS:

Recent work of Ahmadi and Majumdar [3] presents two alternative cones for SOS cones, namely, DSOS and SDSOS, which involve factor widths at most 2 and are more tractable than SOS cones. In the unconstrained case of POP (9.0.3), our semidefinite hierarchy based on the extension of Pólya's Positivstellensatz can be seen as a generalization of DSOS and SDSOS while using the notion of factor width, see Remark 9.17. In fact, to obtain our semidefinite relaxations for the constrained case (9.0.3), we replace each SOS of monomials involved in the certificate (9.0.5) by an SOS polynomial whose Gram matrix has factor width at most s; see Remark 9.10.

Representation theorems

In this section, we derive representations of polynomials nonnegative on semialgebraic sets together with degree bounds.

Polynomials nonnegative on general semialgebraic sets

Extension of Pólya's Positivstellensatz:

We analyze the complexity of the extension of Pólya's Positivstellensatz in the following theorem: Theorem 9.1. (Homogenized representation) Let g 1 , . . . , g m be homogeneous polynomials such that g 1 , . . . , g m are even in each variable. Let S be the semialgebraic set defined by

S := {x ∈ R n : g 1 (x) ≥ 0 . . . , g m (x) ≥ 0} . (9.1.1)
Let f be a homogeneous polynomial of degree 2d f for some d f ∈ N such that f is even in each variable and nonnegative on S. Then the following statements hold:

1. For all ε > 0, there exists K ε ∈ N such that for all k ≥ K ε , there exist homogeneous SOS of monomials σ i satisfying

deg(σ 0 ) = deg(σ 1 g 1 ) = • • • = deg(σ m g m ) = 2(k + d f ) (9.1.2) and x 2k 2 (f + ε x 2d f 2 ) = σ 0 + σ 1 g 1 + • • • + σ m g m . (9.1.3)
2. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that for all ε > 0, one can take K ε = cε -c .

The proof of Theorem 9.1 is postponed to Section 9.4.2. Note that some other homogeneous representations for globally nonnegative polynomials even in each variable have been studied in [START_REF] Goel | The analogue of Hilbert's 1888 theorem for even symmetric forms[END_REF][START_REF] Harris | Real even symmetric ternary forms[END_REF][START_REF] Choi | Even symmetric sextics[END_REF].

Remark 9.1. The Gram matrix associated with each SOS of monomials is diagonal. In other word, it is a block-diagonal matrix with maximal block size one. It would be interesting to know for which types of input polynomials we could obtain other representations involving SOS with blockdiagonal Gram matrices of very small maximal block size, similarly to Theorem 9.1. Some of them have been discussed in [START_REF] Gouveia | On sums of squares of k-nomials[END_REF][START_REF] Magron | SONC Optimization and Exact Nonnegativity Certificates via Second-Order Cone Programming[END_REF] that includes SOS of binomials, trinomials, tetranomials and SOS of any s-nomials. We emphasize that such representations allow one to build up SDP relaxations of small maximal matrix size that can be solved efficiently by using interior-point methods as shown later in Section 9.3.

The following corollary is a direct consequence of Theorem 9.1. Corollary 9.1. (Dehomogenized representation) Let g 1 , . . . , g m be polynomials even in each variable. Let S be the semialgebraic set defined by (9.1.1). Let f be a polynomial even in each variable and nonnegative on S. Denote d f := deg(f )/2 + 1. Then the following statements hold: Proposition 9.1. Let f, g 1 , . . . , g m be polynomials in R[x] such that f, g i are even in each variable. Assume that there exists a decomposition:

f = m i=1 g i v di G (i) v di , (9.1.9)
for some d i ∈ N and real symmetric matrices

G (i) = (G (i) α,β ) α,β∈N n d i . For every i ∈ [m], define Ḡ(i) := ( Ḡ(i) α,β ) α,β∈N n d i
, where:

Ḡ(i) α,β := G (i) α,β if α + β ∈ 2N n , 0 otherwise . (9.1.10)
Then Ḡ(i) are block-diagonal up to permutation and

f = m i=1 g i v di Ḡ(i) v di . (9.1.11) Moreover, if G (i) 0, then Ḡ(i) 0.
Proof. The proof is inspired by [START_REF] Gatermann | Symmetry groups, semidefinite programs, and sums of squares[END_REF]Section 8.1]. Removing all terms in (9.1.9) except the terms of monomials x 2α , α ∈ N n , we obtain (9.1.11). It is due to the fact that f, g i only have terms of the form x 2α , α ∈ N n and

v di G (i) v di = α,β∈N n d i G (i) α,β x α+β . (9.1.12)
Next, we show the block-diagonal structure of Ḡ(i) . For every γ ∈ {0, 1} n , define

Λ (i) γ := {α ∈ N n di : , α -γ ∈ 2N n } . (9.1.13) Then Λ (i) γ ∩Λ (i) η = ∅ if γ = η and N n di := ∪ γ∈{0,1} n Λ (i)
γ . In addition, for all α, β ∈ Λ This yields the desired results.

Remark 9.3. The block-diagonal structure in Proposition 9.1 can be obtained by using TSSOS [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF]. For general input polynomials f, g i , we cannot ensure that the maximal block size in this form is upper bounded or possibly goes to infinity as each d i increases. However, as shown in Remark 9.2, we cannot obtain blocks of size one for this form. In order to improve this, we provide another representation with diagonal Gram matrices in the next corollary.

Extension of Handelman's Positivstellensatz

The following corollary is a consequence of Theorem 9.1. 1. For all ε > 0, there exists K ε ∈ N such that for all k ≥ K ε , there exist SOS of monomials σ i,j satisfying deg(σ i,j g j 1 g i ) ≤ 2k (9.1.15)

and f + ε = m i=1 k-dg i j=0 σ i,j g j 1 g i . ( 9
:= (x, x n+1 ) 2d f 2 f ( x √ R (x,xn+1) 2 ) and ḡi := (x, x n+1 ) 2dg i 2 g i ( x √ R (x,xn+1) 2 ) . (9.1.17)
It is not hard to prove that f is nonnegative on the semialgebraic set {(x, x n+1 ) ∈ R n+1 : ḡi (x, x n+1 ) ≥ 0 , i ∈ [m]}, so that Theorem 9.1 yields the representation where η 0 = 1 4 , η 1 = η 2 = 1 are SOS of monomials. Note that the degrees of these SOS of monomials are zero while the degrees of the ones from (9.1.8) for the extension of Pólya's Positivstellensatz are 8 and 4.

(x, x n+1 ) 2k 2 ( f + ε (x, x n+1 ) 2d f 2 ) = σ 1 ḡ1 + • • • + σ m ḡm , ( 9
f = η 0 + η 1 (1 -x 2 ) + η 2 (1 -x 2 ) 2 , ( 9
Remark 9.6. In Section 9.4.3, we provide some variations of Pólya's and Handelman's Positivstellensatz where the input polynomials are not required to be even in each variable. Moreover, the weighted SOS polynomials of these representations are still associated with Gram matrices of factor width one thanks to a change of monomial basis.

Polynomial optimization on the nonnegative orthant: Compact case

This section is concerned with some applications of the extensions of Pólya's and Handelman's Positivstellensatz for polynomial optimization on compact semialgebraic subsets of the nonnegative orthant. The noncompact case is postponed to Section 9.4.4. Moreover, Section 9.4.6 is devoted to some applications of the sparse representation provided in Section 9.4.5 for polynomial optimization with correlative sparsity.

Consider the following POP:

f := inf x∈S f (x) , (9.2.1)
where f ∈ R[x] and

S = {x ∈ R n : x j ≥ 0 , j ∈ [n] , g i (x) ≥ 0 , i ∈ [m]} , (9.2.2) for some g i ∈ R[x], i ∈ [m], with g m = 1.
Throughout this section, we assume that f > -∞ and problem (9.2.1) has an optimal solution x .

Remark 9.7. Every general POP in variable x = (x 1 , . . . , x n ) can be converted to the form (9.2.1) by replacing each variable x j by the difference of two new nonnegative variables x + j -x - j . In particular, if there are several constraints x j ≥ a j , we can obtain an equivalent POP on the nonnegative orthant by defining new nonnegative variables y j := x j -a j . However, we restrict ourselves to POPs on the nonnegative orthant in this chapter. Remark 9.11. At fixed s ∈ N >0 , the sequence (ρ Pól k,s ) k∈N may not be monotonic w.r.t. k, and similarly at fixed k ∈ N.

Example 9.2. (AM-GM inequality) Consider the case where

n = 3, f = x 1 + x 2 + x 3 and S = {x ∈ R 3 : x j ≥ 0 , j ∈ [3] , x 1 x 2 x 3 -1 ≥ 0 , 3 -x 1 -x 2 -x 3 ≥ 0}. Using AM-GM inequality, we have f (x) ≥ 3(x 1 x 2 x 3 ) 1/3 ≥ 3 , ∀x ∈ S , ( 9

Based on the extension of Handelman's Positivstellensatz

Consider the hierarchy of semidefinite programs indexed by s ∈ N >0 and k ∈ N:

τ Han k,s := inf y L y ( f ) s. t. y = (y α ) α∈N n 2k ⊂ R , y 0 = 1 , M A (s,k ij ) r ((ǧ i ǧj 1 )y) 0 , r ∈ [b(n, k ij )] , i ∈ [m] , j ∈ {0} ∪ [k -d gi ] , (9.2.21)
where

k ij := k -d gi -j, for i ∈ [m], for j ∈ {0} ∪ [k -d gi ]. Note that ǧm = 1. Theorem 9.5. Let f, g i ∈ R[x], i ∈ [m], with g m = 1 and g 1 := R -j∈[n]
x j for some R > 0.

Consider POP (9.2.1) with S being defined as in (9.2.2). For every s ∈ N >0 and for every k ∈ N, the dual of (9.2.21) reads as:

ρ Han k,s := sup λ,Gijr λ s. t. λ ∈ R , G ijr 0 , j ∈ [b(n, k ij )] , i ∈ [m] , j ∈ {0} ∪ [k -d gi ] , f -λ = i∈[m] k-dg i j=0 ǧi ǧj 1 r∈[b(n,kij )] v A (s,k ij ) r G ijr v A (s,k ij ) r . ( 9

.2.22)

The following statements hold:

1. For all k ∈ N and for every s

∈ N >0 , ρ Han k = ρ Han k,1 ≤ ρ Han k,s ≤ f .
2. For every s ∈ N >0 , the sequence (ρ Han k,s ) k∈N converges to f . 3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that for every s ∈ N >0 and for every k

∈ N, 0 ≤ f -ρ Han k,s ≤ k c -1 c .
4. If S has nonempty interior, for every k ∈ N and for every s ∈ N >0 , strong duality holds for the primal-dual problems (9.2.21)-(9.2.22).

The proof of Theorem 9.5 is based on Theorem 9.3 and similar to the proof of Theorem 9.3.

Remark 9.12. To make the use of the extended Handelman's Positivstellensatz, we need at least one ball constraint ǧ1 := R -x 2 2 for some R > 0. Thus, Theorem 9.5 is applicable only when the domain S of POP (9.2.1) is compact. To deal with the noncompact case, we might combine it with the so-called "big ball trick".

Obtaining an optimal solution

A real sequence (y α ) α∈N n t has a representing measure if there exists a finite Borel measure µ such that y α = R n x α dµ(x) is satisfied for every α ∈ N n t . Next, we discuss about the extraction of an optimal solution x of POP (9.2.1) from the optimal solution y = (y α ) α∈N n 2(d f +k) of the semidefinite relaxations (9.2.18). ) 2 . Then Ḡj 0 and

v N n km Ḡj v N n km = v A (s,km) j G j v A (s,km ) j ; (9.2.23) 2: Let G := j∈[b(n,km)] Ḡj . Then G is the Gram matrix corresponding to σ m in the SOS decomposition θ k ( f -λ) = i∈[m] ǧi σ i , (9.2.24)
where σ i are SOS polynomials and ǧm = 1; 3: Obtain an atom z ∈ R n by using the extraction algorithm of Henrion and Lasserre in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF],

where the matrix V in [77, (6)] is taken such that the columns of V form a basis of the null space {u ∈ R ω k : Gu = 0}; 4: Verify that z is an approximate optimal solution of POP (9.2.3) by checking the following inequalities:

| f (z ) -λ| ≤ ε f max and ǧi (z ) ≥ -ε ǧi max , i ∈ [m] , (9.2.25)
where q max := max α |q α | for any q ∈ R[x]. 5: If the inequalities (9.2.25) hold, set x := z 2 .

Numerical experiments

In this section we report results of numerical experiments obtained by solving the Moment-SOS relaxations of some random and nonrandom instances of POP (9.0.1). Other results for some random instances of POP (9.0.1) can be found in Section 9.4.7. Notice that our relaxations from Section 9.2 are to deal with dense POPs while the ones from Section 9.4.6 are for POPs with correlative sparsity.

For numerical comparison purposes, recall the semidefinte relaxation based on Putinar's Positivstellensatz for solving POP (9.0.1) indexed by k ∈ N:

τ Put k := inf y L y (f ) s. t. y = (y α ) α∈N n 2k ⊂ R , y 0 = 1 , M k-gi (g i y) 0 , i ∈ [ m] , (9.3.1)
and its sparse version: least O(ε -c ) when POP (9.0.1) has a ball constraint, e.g., g 1 := R -x 2 2 for some R > 0. If g 1 = R -j∈ [n] x j for some R > 0, then (τ Put k ) k∈N still converges to f due to Jacobi-Prestel [86, Theorem 4.2] (see also [8, Theorem 1 (JP)]). Remark 9.14. If we assume that g 1 := R -j∈ [n] x j for some R > 0, SDP (9.3.1) may be unbounded when k is too small since its variables y are possibly unbounded. This issue occurs later on, see, e.g., Section 9.3.1. However, if we assume that g 1 := R -x 2 2 for some R > 0, then SDP (9.3.1) is feasible for any order k ≥ 1 (see Section 9.4.7).

τ spPut k := inf y L y (f ) s. t. y = (y α ) α∈N n 2k ⊂ R , y 0 = 1 , M N Ic k-g i (g i y) 0 , i ∈ J c , c ∈ [p] .
The experiments are performed in Julia 1.3.1. We rely on TSSOS [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF] to solve the Moment-SOS relaxations of sparse POPs.

The implementation of our method is available online via the link: https://github.com/maihoanganh/InterRelax.

We use a desktop computer with an Intel(R) Core(TM) i7-8665U CPU @ 1.9GHz × 8 and 31.2 GB of RAM. The notation for the numerical results is given in Table 9.2.

Dense QCQPs

Test problems: We construct randomly generated dense quadratically constrained quadratic programs (QCQPs) in the form (9.0.1)-(9.0.2) as follows: 1. Take a in the simplex

∆ n := {x ∈ R n : x j ≥ 0 , j ∈ [n] , j∈[n] x j ≤ 1} (9.3.3)
w.r.t. the uniform distribution.

Let g

1 := 1 -j∈[n]
x j and g 2 := 1.

3. Take every coefficient of f and g i , i = 2, . . . , m, in (-1, 1) w.r.t. the uniform distribution.

4. Update g i (x) := g i (x) -g i (a) + 0.125, for i = 2, . . . , m ineq .

5. Update g i+mineq (x) := g i+mineq (x) -g i+mineq (a) and set g i+meq+mineq = -g i+mineq , for i ∈ [m eq ].

Here m = m ineq + 2m eq with m ineq (resp. m eq ) being the number of inequality (resp. equality) constraints except the orthogonal constraints x j ≥ 0. If m ineq = 2 and m eq = 0, we obtain the case of the minimization of a polynomial on the simplex ∆ n . The point a is a feasible solution of POP (9.0.1). The numerical results are displayed in Table 9.3.

Discussion: Table 9.3 shows that Pól and Han are typically faster and more accurate than Put. For instance, when n = 20, m ineq = 5 and m eq = 0, Put takes 342 seconds to return the lower bound -0.350601 for f , while Pól only takes 9 seconds to return the better bound -0.265883 and an approximate optimal solution. It is due to the fact that Pól has 44 matrix variables with the maximal matrix size 20, while Put has 25 matrix variables with the maximal matrix size 231 in this case. In addition, Han provides slightly better bounds than Pól in Pb 3 and the same bounds with Pól in the others. Moreover, Pól runs about five times faster than Han in Pb 5 and 6.

Sparse QCQPs

Test problems: We construct randomly generated QCQPs in the form (9.0.1)-(9.0.2) with correlative sparsity as follows:

1. Take a positive integer u, p := n/u + 1 and let

I c =      [u], if c = 1 , {u(c -1), . . . , uc}, if c ∈ {2, . . . , p -1} , {u(p -1), . . . , n}, if c = p ; (9.3.4) 2. Generate a quadratic polynomial objective function f = c∈[p] f c such that for each c ∈ [p], f c ∈ R[x(I c )] 2 , and the coefficient f c,α , α ∈ N Ic
2 of f c is randomly generated in (-1, 1) w.r.t. the uniform distribution; 3. Take a random point a such that for every c ∈ [p], a(I c ) belongs to the simplex

∆ (c) := {x(I c ) ∈ R nc : x j ≥ 0 , j ∈ I c , j∈Ic x j ≤ 1} (9.3.5)
4. Let q := m ineq /p and 5. Take g ic := 1 -i∈Ic x i , for some i c ∈ J t , for c ∈ [p];

J c := {(c -1)q + 1, . . . , cq}, if c ∈ [p -1] , {(p -1)q + 1, . . . , l}, if c = p . ( 9
6. Let r := m eq /p and Here m = m ineq + 2m eq with m ineq (resp. m eq ) being the number of inequality (resp. equality) constraints except the orthogonal constraints x j ≥ 0. The point a is a feasible solution of POP (9.0.1). The numerical results are displayed in Table 9.4. Discussion: Similarly to the previous discussion, spPól and spHan in Table 9.4 are also much faster and more accurate than spPut. For instance, when u = 20, m ineq = 201 and m eq = 0, spPól takes 20 seconds to return the lower bound -65.9794 of f , while spPut takes 56360 seconds to return a worse bound of -66.1306. In this case, spPól has 399 matrix variables with maximal matrix size 15, while spPut has 1250 matrix variables with maximal matrix size 253. In particular, spHan provides slightly better bounds than spPól for Pb 1, 2, 4 while it is seven (resp. five) times slower than spPól in Pb 4 (resp. Pb 2).

W c := {(c -1)r + 1, . . . , cr}, if c ∈ [p -1] , {(p -1)r + 1, . . . , l}, if c = p . ( 9

Stability number of a graph

In order to compute the stability number α(G) of a given graph G, we solve the following POP on the unit simplex:

1 α(G) = min x∈R n + {x (A + I)x : j∈[n] x j = 1} , (9.3.8)
where A is the adjacency matrix of G and I is the identity matrix.

Test problems:

We take some adjacency matrices of known graphs from [START_REF] Rossi | The network data repository with interactive graph analytics and visualization[END_REF]. The numerical results are displayed in Tables 9.5 and 9.6. Note that in Table 9.6 we solve POP (9.3.8) with an additional unit ball constraint 1 -x 2 2 ≥ 0. The columns under "val" show the approximations of α(G).

Discussion: The graphs from Table 9.5 are relatively dense so that we cannot exploit term sparsity or correlative sparsity for POP (9.3.8) in these cases. For the graph GD02 a in Table 9.5, Pól and Han provide better bounds for α(G) compared to the ones returned by the second order relaxations of Put. In Table 9.6, Put provides negative values for the first order relaxations. The additional unit ball constraint does not help to improve the bound for the second order relaxation for Id 2. Besides, Table 9.5 shows that Han provides slightly better bounds than Pól for johnson16-2-4, but its value is less accurate than the corresponding one from Table 9.6.

The MAXCUT problems

The MAXCUT problem is given by: max x∈{0,1} n x W(ex) , (9.3.9) where e = (1, . . . , 1) and W is the matrix of edge weights associated with a graph (see [START_REF] Commander | Maximum Cut Problem, MAX-CUT[END_REF]Theorem 1]).

Test problems:

The data of graphs is taken from TSPLIB [START_REF] Reinelt | TSPLIB-A traveling salesman problem library[END_REF].

The numerical results are displayed in Table 9.7. Note that all instance of matrix W are dense.

Discussion: The behavior of our method is similar to that in Section 9.3.1.

Robustness certification of deep neural networks

In [START_REF] Raghunathan | Semidefinite relaxations for certifying robustness to adversarial examples[END_REF], the robustness certification problem of a multi-layer neural network with ReLU activation function is formulated as the following QCQP:

l y (x, ȳ) := max x 0 ,...,x L (c y -c ȳ ) x L s.t. x i t (x i t -W i-1 t x i-1 ) = 0 , x i t ≥ 0 , x i t ≥ W i-1 t x i-1 , t ∈ [m i ] , i ∈ [L] -ε ≤ x 0 t -xt ≤ ε , t ∈ [m 0 ] , (9.3.10)
where we use the same notation as in [172, Section 2] and write

W i-1 =   W i-1 1 . . . W i-1 mi   .
We say that the network is certifiably ε-robust on (x, ȳ) if l y (x, ȳ) < 0 for all y = ȳ. Test problems: To obtain an instance of weights W i , we train a classification model by using Keras1 . Explicitly, we minimize a loss function as follows:

min W 0 ,...,W L-1 1 2 (x 0 ,y 0 )∈D f (x 0 ) -e y 0 2 2 , (9.3.11)
where the network f is defined as in [172, Section 2] and e y 0 has 1 at the y 0 -th element and zeros at the others. Here the input set D is a part of Boston House Price Dataset (BHPD). The class label y 0 is assigned to the input x 0 . We classify the inputs from BHPD into 3 classes according to the MEDian Value of owner-occupied homes (MEDV) in $1000 as follows:

y 0 =      1 if MEDV < 10 , 2 if 10 ≤ MEDV < 20 , 3 otherwise . (9.3.12)
We also take a clean input label pair (x, ȳ) / ∈ D with ȳ = 3 from BHPD. As shown in [35, Step 2: Construction of the positive weight functions. We process similarly to the proof of [132, Theorem 1] (see [START_REF] Mai | On the complexity of Putinar-Vasilescu's Positivstellensatz[END_REF]Appendix A.2.1]) to obtain functions φj : R n → R, j ∈ [ m], such that, 1. φj is positive and bounded from above by C φj = rj ε -rj on B(0, √ n + j) for some positive constants rj and r j independent of ε.

S := {x ∈ R n : gi (x) ≥ 0 , i ∈ [ m]} . ( 9
2. φj is Lipschitz with Lipschitz constant L φj = tj ε -tj for some positive constants tj and t j independent of ε.

The inequality

f + ε - m i=1 φ2 i gi ≥ ε 2 m on [-1, 1] n , ( 9.4.7)
holds.

Note that we do not need to prove the even property for each weight φi above. (9.4.12)

Step 4: Estimating the lower and upper bounds of f (x) + ε -m i=1 q i (x) 2 gi (x) on ∆ n . From these and (9.4.7), for all x ∈ ∆ n , f (x) + ε -m i=1 q i (x) 2 gi (x) = f (x) + ε -m i=1 φi (x) 2 gi (x) + m i=1 gi (x)[ φi (x) 2 -q i (x) 2 ] ≥ ε 2 m -m i=1 |g i (x)|| φi (x) + q i (x)|| φi (x) - Step 5: Estimating the upper bound of q i . For i ∈ [ m], we write (9.4.17)

q i (x)| ≥ ε 2 m -m i=1 C gi (| φi (x)| + |q i (x)|) ε 2Cg i C φi ( m+1)2 m ≥ ε 2 m -m i=1 2C gi C φi ε 2Cg i C φi ( m+1)2 m = ε 2 m - mε (
q i = B (2ui) i = 2ui k1=0 • • •
The second inequality is due to x j + 1 = 1 -x j = 1 and 2ui ui ≥ 2ui kj , for k j = 0, . . . , 2u i . The third inequality is implied from Lemma 9.1.

Step 6: Converting to homogeneous polynomials. Thanks to (9.4.14), we get

f + 2ε -i∈[ m] (q 2 i + ε mCg i )g i ≥ ε ( m+1)2 m
on ∆ n , (9.4.18) since |g i | ≤ C gi on ∆ n . Note that f , gi are homogeneous polynomials of degree d f , d gi , respectively. For each q ∈ R[x] d , q is a d-homogenization of q if q = d t=0 h (t) ( j∈ [n] x j ) d-t , (9.4.19) for some h (t) is the homogeneous polynomial of degree t satisfying q = d t=0 h (t) . In this case, q = q on ∆ n .

Let p i := q2 i + ε mCg i

( j∈ [n] x j ) 4nui with qi being a 2nu i -homogenization of q i , for i ∈ [ m]. Then p i is a homogeneous polynomial of degree 4nu i , since j∈ [n] x j = 1.

p i = q 2 i + ε mCg i ≥ ε mCg i on ∆ n , ( 9
Step 7: Applying the degree bound of Pólya's Positivstellensatz. Using Lemma 9.2, we obtain:

• For all k ∈ N satisfying Notice that K i , i = 0, . . . , m, are obtained by composing finitely many times the following operators: "+", "-", "×", "÷", "| • |", " • ", "(x 1 , x 2 ) → max{x 1 , x 2 }", "(x 1 , x 2 ) → min{x 1 , x 2 }", "(•) αm " and " √ •", where all arguments possibly depend on ε. Without loss of generality, let c, c be positive constants independent of ε such that cε -c ≥ max{K 0 , . . . , K m}.

Let k ≥ cε -c be fixed. Multiplying two sides of (9. [179, (3), page 4]. The difference here is that the weight of each constrained polynomial g i , i / ∈ {1, m}, in (9.4.38) is the polynomial k-dg i r=0 g r 1 v k-dg i -r diag(η (i,r) )v k-dg i -r , (9.4.39) which does not involve g i . This is in contrast with the weight associated to each g i in [179, (3), page 4], which is of the form σ i (U i -g i ), where U i is an upper bound of g i on the ball {x ∈ R n : g 1 (x) ≥ 0} and σ i is a univariate polynomial, e.g., σ i (t) = t 2ξ for some ξ ∈ N. Here ǧm = 1. The following statements hold:

Variations of Pólya's and Handelman's Positivstellensatz

1. For all k ∈ N, ρ Pól k (ε) ≤ ρ Pól k+1 (ε) ≤ f .

2. There exists K ∈ N such that for all k ≥ K, 0 ≤ f -ρ Pól k (ε) ≤ εθ(x 2 ) d f . 3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that for all k ≥ cε -c , 0 ≤ f -ρ Pól k (ε) ≤ εθ(x 2 ) d f . The proof of Theorem 9.6 relies on Corollary 9.1 and is exactly the same as the proof of Theorem 6.5.

Semidefinite relaxations

Given ε > 0, consider the hierarchy of semidefinite programs indexed by s ∈ N >0 and k ∈ N: It implies that the degree-2d f homogenization of f -λ + εθ d f belongs to the cone k-DSOS n+1,2d f (resp. k-SDSOS n+1,2d f ) when s = 1 (resp. s = 2) according to [3,Definition 3.10]. More generally, the polynomial θ k ( f -λ + εθ d f ) belongs to the cone of SOS polynomials whose Gram matrix has factor width at most s (see [3,Section 5.3]).

Sparse representation theorem

For every I = {i 1 , . . . , i r } ⊂ [n] with i 1 < • • • < i r , denote x(I) = (x i1 , . . . , x ir ). We will make the following assumptions:

Extension of Pólya's Positivstellensatz

We state the sparse representation in the following theorem:

Theorem 9.8. Let g 1 , . . . , g m be polynomials such that g 1 , . . . , g m are even in each variable and Assumption 9.1 holds. Let S be the semialgebraic set defined by S := {x ∈ R n : g 1 (x) ≥ 0 . . . , g m (x) ≥ 0} . This decomposition is simpler than the ones provided in [START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF] and thus is more applicable to polynomial optimization.

Another sparse representation without denominators can be found in the next theorem. However, the number of SOS of monomials is not fixed in this case.

Extension of Handelman's Positivstellensatz

We process similarly to the proof of Theorem 9.8 and apply Corollary 9.3 to obtain the following theorem: with optimal solution x 2 . We will make the following assumptions: The following statements hold:

1. For all k ∈ N, ρ spHan k ≤ ρ spHan k+1 ≤ f .

The sequence (ρ spHan

) k∈N converges to f .

The proof of Theorem 9.11 relies on Theorem 9.9 and is similar to Theorem 9.3.

Semidefinite relaxations

For every I ⊂ [n], we write N I = {α is written explicitly as in (9.2.15). We obtain two blocks:

Obtaining an optimal solution

In other to extract an optimal solution of POP (9.4.60) with correlative sparsity, we first extract atoms on each clique similarly to Algorithm 14 and then connect them together to obtain atoms in R n . Explicitly, we use the following heuristic extraction algorithm: where each σ i,c is an SOS polynomial and ǧm = 1; c: Obtain an atom z (c) ∈ R nc by using the extraction algorithm of Henrion and Lasserre in [START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF], where the matrix V in [77, (6)] is taken such that the columns of V form a basis of the null space {u ∈ R ω k : G (c) u = 0}; 2: Let z ∈ R n such that z (I c ) = z (c) , for c ∈ [p]. where f is a polynomial of degree at most 2d. It is equivalent to the following POP on the nonnegative orthant: min

x∈R n + {f (x) : x j (1 -x j ) = 0 , j ∈ [n]} , ( 9.4.94)
Test problems: We construct several instances by taking the coefficients of f randomly in (-1, 1) w.r.t. to the uniform distribution.

The numerical results are displayed in Table 9.14.

Discussion:

The behavior of our method is similar to that in Section 9.3.1. Note that Pól with order k = 0 provides worse bounds than Put with order k = 2. However, as shown in Table 9.14, Pól with order k = 1 provides the same bounds as Put with order k = 2. 1) The standard procedure of extracting optimal solutions for the dense Moment-SOS hierarchy does not apply to the CS-TSSOS hierarchy. It would be interesting to develop a procedure for extracting (approximate) solutions from partial information of moment matrices.

2) Recall that chordal extension plays an important role for both correlative and term sparsity patterns. It turns out that the size of the resulting maximal cliques is crucial for the overall computational efficiency of the CS-TSSOS hierarchy. So far, we have only considered maximal chordal extensions (for convergence guarantee) and approximately smallest chordal extensions. It would be worth investigating more general choices of chordal extensions.

3) The CS-TSSOS strategy could be adapted to other applications involving sparse polynomial problems, including deep learning [START_REF] Chen | Semialgebraic optimization for lipschitz constants of relu networks[END_REF].

4) At last but not least, a challenging research issue is to establish serious computationally cheaper alternatives to interior-point methods for solving SDP relaxations of POPs. The recent work [START_REF] Yurtsever | Scalable semidefinite programming[END_REF] which reports spectacular results for standard SDPs (and Max-Cut problems in particular) is a positive sign in this direction. Such a computationally cheaper alternative is presented in Chapter 5, Appendix 5.3.

• In Chapter 4 we have provided a nonsmooth hierarchy of SDP relaxations for optimization of polynomials on varieties contained in a Euclidean sphere. The advantage of this hierarchy is to circumvent the hard constraints involved in the standard SDP hierarchy (4.1.6) by minimizing the maximal eigenvalue of a matrix pencil. This in turn boils down to solving an unconstrained convex nonsmooth optimization problem by LMBM and to computing largest eigenvalues by means of the modified Lanczos's algorithm. Our numerical experiments indicate that solving this nonsmooth hierarchy is more efficient and more robust than solving the classical semidefinite hierarchy by interior-point methods, at least for a class of interesting POPs, including equality constrained QCQPs on the sphere, QCQPs with a single inequality (ball) constraint, and minimization of quartics on the sphere. Our CTP framework can be further applied for an interesting class of noncommutative polynomial optimization problems [START_REF] Mai | The constant trace property in noncommutative optimization[END_REF], in particular for eigenvalue maximization problems arising from quantum information theory, where the variables are unitary operators [START_REF] Navascués | A convergent hierarchy of semidefinite programs characterizing the set of quantum correlations[END_REF].

Eventually, we have tried to use spectral methods to solve SDP relaxations of QCQPs involving inequalities only, systems of polynomial equations, MAXCUT problems, 0/1 linear constrained quadratic problems and computation of stability numbers of graphs. However, our preliminary experiments for these problems have not been convincing in terms of efficiency and accuracy. In order to improve upon these results, one possible remedy would be to index the moment matrices by alternative Legendre/Chebychev bases, rather than with the standard monomial basis.

• In Chapter 5, we have proposed a general framework for exploiting the constant trace property in solving large-scale SDPs, typically SDP-relaxations arising from the Moment-SOS hierarchy for POPs with CTP. Extensive numerical experiments strongly suggest that with this CTP formulation, the CGAL solver based on first-order methods is more efficient and more scalable than Mosek without exploiting CTP, especially when the block size is large. In addition, the optimal value returned by CGAL is typically within 1% w.r.t. the one returned by Mosek.

We have also integrated sparsity-exploiting techniques into the CTP-exploiting framework in order to handle large-scale POPs. For SDP-relaxations of large-scale POPs with a term and/or correlative sparsity pattern, and in applications for which only a medium accuracy of optimal solutions is enough, we believe that our framework should be very useful.

As a topic of further investigation, we would like to improve the LP-based formulation for verifying CTP, for instance by relying on more general second-order cone programming. We also would like to generalize the CTP-exploiting framework to noncommutative POPs [START_REF] Burgdorf | Optimization of polynomials in non-commuting variables[END_REF][START_REF] Klep | Sparse noncommutative polynomial optimization[END_REF][START_REF] Wang | Exploiting term sparsity in noncommutative polynomial optimization[END_REF] which have attracted a lot of attention in the quantum information community.

Another line of research would be to investigate whether CTP could be efficiently exploited by interior-point solvers.

• In Chapter 6, we have established two representations of (i) globally nonnegative polynomials and (ii) polynomials nonnegative on semialgebraic sets, based on the homogeneous representations of [START_REF] Reznick | Uniform denominators in Hilbert's seventeenth problem[END_REF] and [START_REF] Putinar | Solving moment problems by dimensional extension[END_REF]. These representations have a distinguishing feature. They can be converted into a practical numerical scheme for approximating the global minimum, yielding converging appropriate hierarchies of semidefinite relaxations for unconstrained and constrained polynomial optimization problems.

We have also introduced a new method (based on adding spherical constraints (ASC)) to solve systems of polynomial equalities and inequalities, and to obtain global solutions of polynomial optimization problems as well. In view of its practical efficiency, a topic of further investigation is to provide a more detailed comparison with other methods for solving polynomial systems.

• In Chapter 7 we have provided a new degree bound on the sum-of-squares (SOS) polynomials involved in Putinar-Vasilescu's Positivstellensatz. The resulting associated Moment-SOS hierarchy provides a sequence of lower bounds that converges to the global minimum (with prescribed accuracy ε > 0) at an O(ε -c ) rate.

A topic of further investigation is the analysis of the convergence rate of the Moment-SOS hierarchy for lower bounds in some special cases of basic (compact) semialgebraic sets. A first natural idea is to find the explicit constant α in the Lojasiewicz inequality stated in Lemma 7.2. We could then proceed analogously to the proof of the rate O(ε -65 ) for the minimization of a polynomial on the unit ball. It would be interesting to investigate whether our convergence analysis would be improved after replacing Bernstein approximations involved in our proof by Jackson kernels from [START_REF] Bagby | Multivariate simultaneous approximation[END_REF][START_REF] Newman | Jackson's theorem in higher dimensions[END_REF].

• In Chapter 8, we have provided:

1) a sparse version for Reznick's Positivstellensatz (resp. Putinar-Vasilescu's Positivstellensatz) for positive definite forms (resp. nonnegative polynomials).

2) a sparse version of Putinar-Vasilescu's Positivstellensatz for polynomials that are nonnegative on a possibly noncompact basic semialgebraic set. All these certificates involve sums of squares of rational functions with uniform denominators. For additional efficiency, our positivity certificates have been combined with appropriate sampling (evaluation) techniques (to impose that two polynomials are identical). The full computational benefit of such sampling techniques remains to be investigated.

• In Chapter 9 we have considered the case of dense POPs on the nonnegative orthant. By applying a positivity certificate involving SOS of monomials for a POP with input polynomials being even in each variable, one obtains a specific hierarchy of linear relaxations. Afterwards we replace each SOS of monomials by an SOS associated with a block-diagonal Gram matrix, where each block has a prescribed size. This ensures a practical efficiency of the corresponding hierarchy of SDP relaxations. Its convergence is still maintained with a O(ε -c ) rate, similar to the one of Baldi and Mourrain [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF].

As a topic of further applications, we would like to use this methodology for solving large-scale POPs for phase retrieval and feedforward neural networks.

Additional future research directions 10.2.1 Deep Neural networks

Robustness certification. Evaluation and certification of robustness of Deep Neural Networks (DNNs) has become an important issue (DNNs), especially in view of certain of their applications. We refer the interested readers to [START_REF] Parhi | What kinds of functions do deep neural networks learn? Insights from variational spline theory[END_REF][START_REF] Caragea | Quantitative approximation results for complex-valued neural networks[END_REF][START_REF] Li | Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks[END_REF][START_REF] Baykal | Sensitivity-informed provable pruning of neural networks[END_REF] for some recent research advances on DNNs. Figure 10.1 illustrates a neural network with two hidden layers. The widely used ReLU activation function (x → max(0, x)) can be modeled by polynomial (in)equalities. As a result, robustness certification of such DNNs problem can be formulated as maximizing a linear function with linear and quadratic polynomial constraints, as described, e.g., in [START_REF] Raghunathan | Semidefinite relaxations for certifying robustness to adversarial examples[END_REF]. A promising research track is to take advantage of special structures of the objective and constraining polynomials to find a Positivstellensatz adapted to this situation. In Chapter 9, we have exploited nonnegativity of variables involved in the robustness certification problems. In the context of DNNs, the idea is to also exploit sparsity of the constraints which arise from their structure.

Matrix singular values and phase retrieval. We also intend to improve the approach of Chapter 9, to obtain tighter upper bounds for positive maximal singular values of a given matrix. Such bounds are very useful in the stability analysis of recurrent neural networks [START_REF] Ebihara | Stability Analysis of Recurrent Neural Networks by IQC with Copositive Mutipliers[END_REF][START_REF] Ebihara | l2 induced norm analysis of discrete-time LTI systems for nonnegative input signals and its application to stability analysis of recurrent neural networks[END_REF]. Computing such singular values boils down to maximizing a quadratic polynomial over the intersection of the unit sphere and the nonnegative orthant. Another important application is phase retrieval, which can be similarly formulated as a (nonconvex) optimization problem involving quadratic polynomials, with several hidden structures. Another possibility is to develop a hierarchy of semidefinite relaxations similar to the one based on Pólya's Positivstellensatz in Chapter 9.

Finally, we also envision to adapt the method of chapters 4 and 5 that exploits the constant trace property to certify robustness of multi-layer neural networks, to compute positive maximal singular values, and to solve phase retrieval problems at global optimality.

Rates of convergence

An important challenge in polynomial optimization is to provide rates of convergence for various convex relaxations, and notably for the Moment-SOS hierarchy of semidefinite relaxations (and some variants described in this thesis). That is, the goal is to provide explicit bounds on the degrees of the polynomials involved in the various Positivstellensätze. We will particularly investigate (i) how different types of positive weights in the Positivstellensätze affect the convergence rate and (ii) if the corresponding hierarchies of semidefinite relaxations have finite convergence.

In addition to a theoretical interest in its own, analyzing the convergence rate also provides useful insights about the practical efficiency of the underlying semidefinite relaxations. A higher convergence rate is likely to yield a smaller computational effort to approximate closely the optimal value. Moreover, if the hierarchy has finite convergence, then the optimal value can be computed exactly.

One possibility is to follow the methodology that we have used to obtain the convergence rates of Putinar-Vasilescu's and Dickinson-Povh's Positivstellensatz in chapters 7 and 9, respectively, and which proceeds in two steps: (i) provide a constructive existence proof for the Positivstellensatz, and (ii) provide degree bounds of its explicit SOS weights.

Another possibility is to use Fritz-John optimality conditions as in our recent works [START_REF] Mai | Exact polynomial optimization strengthened with Fritz John conditions[END_REF][START_REF] Mai | On the exactness for polynomial optimization strengthened with Fritz John conditions[END_REF][START_REF] Mai | Complexity for exact polynomial optimization strengthened with Fritz John conditions[END_REF]. Developed directly from the works of Nie-Demmel-Sturmfels [START_REF] Nie | Minimizing polynomials via sum of squares over the gradient ideal[END_REF] and Demmel-Nie-Powers [START_REF] Demmel | Representations of positive polynomials on noncompact semialgebraic sets via KKT ideals[END_REF], this method allows us to guarantee finite convergence and to compute global minimizers of POPs even when the Karush-Kuhn-Tucker conditions no longer hold.

  pour des polynômes SOS σ i avec deg(σ 0 ) et deg(σ i g i ) au plus 2(d f + k). Ici • 2 désigne la norme vectorielle 2 . En conséquence, nous obtenons une hiérarchie convergente de relaxations semidéfinies pour les bornes inférieures en optimisation polynomiale sur des ensembles semialgébriques compacts basiques. La complexité de cette hiérarchie est O(ε -c ) pour une précision prescrite ε > 0. En particulier, si m = L = 1 alors c = 65, ce qui donne une complexité de calcul O(ε -65 ) pour minimiser un polynôme sur la boule unité. Dans une autre contribution, nous dérivons une variante creuse du Positivstellensatz de Reznick. Si f est une forme définie positive, le Positivstellensatz de Reznick indique qu'il existe k ∈ N tel que x 2k 2 f est un SOS. Si nous supposons maintenant que f = p c=1 f c , où chaque forme f c dépend d'un sous-ensemble des variables initiales, et supposons que ces sous-ensembles satisfont la propriété d'intersection courante (RIP). Alors il existe k ∈ N tel que f = p c=1 σ c /H k c , où σ c est une somme de carrés de polynômes, H c est un dénominateur polynomial uniforme, et les deux polynômes σ c , H c dépendent uniquement des mêmes variables que f c , pour chaque c ∈ [p]. En d'autres termes, le modèle de parcimonie de f se reflète également dans cette version parcimonieuse du certificat de positivité de Reznick. Nous utilisons ensuite ce résultat pour obtenir également des certificats de positivité pour (i) les polynômes non négatifs sur tout l'espace et (ii) les polynômes positifs sur un ensemble semialgébrique basique (éventuellement non compact), en supposant que les données d'entrée satisfont la RIP. Les deux sont des versions parcimonieuses du Positivstellensatz de Putinar-Vasilescu.

3. 1

 1 An example of chordal extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The support extension of G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 The tsp graphs of Example 3.2. The dashed edge is added after the maximal chordal extension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 The csp graph of Example 3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 The tsp graph for the first clique of Example 3.3 . . . . . . . . . . . . . . . . . . . 3.6 The tsp graph for the second clique of Example 3.3 . . . . . . . . . . . . . . . . . . 3.7 The tsp graph without decomposing variables of Example 3.3 . . . . . . . . . . . . 3.8 The block-band sparsity pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 . 1 1 . 2 1 . 3 4 . 1 4 . 2 5 . 1

 111213414251 10. . . . . . . . . . . . . . . . . . . 6.1 Illustration of Lemma 6.3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Plot of the complexity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Illustration for the proof of the Lipschitz continuity of ϕ m on K (rectangle). Here K = S m-1 ∩B(0, √ n+m-1) and U = K + wm 2 B • (0, 1) with the notation of Lemma 7.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 A two hidden layer neural network. . . . . . . . . . . . . . . . . . . . . . . . . . . .List of Tables1 Table of Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some known Positivstellensätze. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Recent fastest convergence rates in polynomial optimization. . . . . . . . . . . . . . . . Complexity comparison (in terms of floating-point operations) of several methods for solving SDP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 The result for Broyden banded functions (k = 3) . . . . . . . . . . . . . . . . . . . 3.3 The result for the generalized Rosenbrock function (k = 2) . . . . . . . . . . . . . . 3.4 The result for the Broyden tridiagonal function (k = 2) . . . . . . . . . . . . . . . 3.5 The result for the chained Wood function (k = 2) . . . . . . . . . . . . . . . . . . . 3.6 The result for Max-Cut instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 The data for AC-OPF instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 The result for AC-OPF instances . . . . . . . . . . . . . . . . . . . . . . . . . . . . Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, 1) and k = 1. . . . . . . . . . . . . . . . . . . 4.3 Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, n/4 ) and k = 1. . . . . . . . . . . . . . . . . 4.4 Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, n/4 ) and k = 2. . . . . . . . . . . . . . . . . 4.5 Storage comparisons for the rows n = 5, 10, 15, 20, 25 of Table 4.4. . . . . . . . . . . . . 4.6 Evaluation comparisons for the rows n = 5, 10, 15, 20, 25 of Table 4.4. . . . . . . . . . . . 4.7 Numerical results of random dense QCQPs on the unit ball, described in Section 4.3.2, with (m, l) = (1, n/4 ), and k = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Numerical results of random dense QCQPs on the unit ball, described in Section 4.3.2, with (m, l) = ( n/8 , n/8 ), and k = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Subgradient norms computed during the last 10 iterations of CTP (LMBM) for the experiments from Table 4.7 and Table 4.8 with n = 10. . . . . . . . . . . . . . . . . . . . . . 4.10 Numerical results for random dense quartics on the unit sphere, described in Section 4.3.3, with (m, l) = (0, 1) and k = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 Numerical results for deciding the nonnegativity of even degree forms, described in Section 4.3.4, with k = d. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.12 Numerical results for deciding the convexity of even degree forms, described in Section 4.3.5, with k = d. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.13 Numerical results for deciding the copositivity of real symmetric matrices, described in Section 4.3.6, with k = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Numerical results for minimizing a dense quadratic polynomial on a unit ball . . . . . . 5.3 Numerical results for randomly generated dense QCQPs with a ball constraint . . . . . . 5.4 Numerical results for minimizing a dense quadratic polynomial on an annulus . . . . . . 5.5 Numerical results for randomly generated dense QCQPs with annulus constraints . . . . 5.6 Numerical results for minimizing a dense quadratic polynomial on a box . . . . . . . . .

127 8. 1 156 9. 1

 12711561 Numerical results for randomly generated QCQPs with CS and ball constraints on each clique of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.18 Numerical results for minimizing a random quadratic polynomial with CS and box constraints on each clique of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 5.19 Numerical results for QCQPs with CS and box constraints on each clique of variables . . 100 5.20 Numerical results for minimizing a random quadratic polynomial with CS-TSSOS and ball constraints on each clique of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.21 Numerical results for QCQPs with CS-TSSOS and ball constraints on each clique of variables101 5.22 Numerical results for minimizing a random quadratic polynomial with CS-TSSOS and box constraints on each clique of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.23 Numerical results for QCQPs with CS-TSSOS and box constraints on each clique of variables102 6.1 Decrease of the moment matrix rank in Algorithm 6.2.3. . . . . . . . . . . . . . . . . . 123 6.2 Examples of POPs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 6.3 Numerical experiments with ε = 10 -5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.4 Numerical experiments for Id 16 with various values of ε. . . . . . . . . . . . . . . . . . 127 6.5 Numerical experiments for Id 21 with various values of ε. . . . . . . . . . . . . . . . . . Comparing the computational complexity of the sparse and dense hierarchies. . . . . . . 148 8.2 Numerical results obtained when solving the SDP relaxations (8.2.2), (8.2.4) associated to the minimization of quadratic forms on the nonnegative orthant with ε = 10 -5 and k = 1. Numerical values (in the first subtable) and computing time (in the second subtable) for τ Pól k,s in Example 9.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 9.2 The notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 9.3 Numerical results for randomly generated dense QCQPs. . . . . . . . . . . . . . . . . . 171 9.4 Numerical results for randomly generated QCQPs with correlative sparsity of n = 1000 and d = deg(f ) = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 9.

  with an additional unit ball constraint. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 9.7 Numerical results for some instances of MAXCUT problems. . . . . . . . . . . . . . . . 176 9.8 Information for the training model (9.3.11). . . . . . . . . . . . . . . . . . . . . . . 177 9.9 Numerical results for robustness certification on BHPD, n = 43, mineq = 43, meq = 30 and d = deg(f ) = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 9.10 Numerical results for positive maximal singular values. . . . . . . . . . . . . . . . . . . 191 9.11 Numerical results for stability number of randomly generated graphs. . . . . . . . . . . 192 9.12 Numerical results for deciding the copositivity of a real symmetric matrix. . . . . . . . . 193 9.13 Numerical results for deciding the nonnegativity of an even degree form on the nonegative orthant, with d = 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 9.14 Numerical results for minimizing polynomials over the boolean hypercube, with d = 1. . . 195

N

  set of natural integers [m] set {1, . . . , m} of m first consecutive positive integers with [0] := ∅ R set of real numbers R + set of nonnegative real numbers N * d set {u ∈ N : u * d}, for * ∈ {≥, ≤, >, <} Linear algebra R n×m space of real matrices with n rows and m columns I n identity matrix in R n×n trace(M) trace of a square matrix M M transposition of a matrix M S n space of real symmetric matrices with n rows S n + cone of symmetric positive semidefinite matrices M 0 M ∈ S n + S n ++ open cone of symmetric positive definite matrices M 0 M ∈ S n ++ M 0 -M 0 M ≺ 0 -M 0 M N N -M 0 Euclidean geometry

  With x = (x 1 , . . . , x n ), let R[x] stand for the ring of real polynomials and let Σ[x] ⊂ R[x] be the subset of sum of squares (SOS) polynomials. Their restrictions to polynomials of degree at most d and 2d are denoted by R[x] d and Σ[x] d respectively. For α

  let [r] := {1, . . . , r}. Let (x α ) α∈N n be the canonical monomial basis of R[x] (sorted w.r.t. the graded lexicographic order) and v d (x) be the vector of monomials of degree up to d, with length b(n, d) := n+d n . When it is clear from the context, we also write b(d) instead of b(n, d). A polynomial q ∈ R[x] d can be written as q

Figure 3 . 1 :

 31 Figure 3.1: An example of chordal extension

Theorem 3 . 1 (+ 3 )

 313 [2], Theorem 2.3). Let G(V, E) be a chordal graph and assume that C 1 , . . . , C t are the list of maximal cliques of G(V, E). Then a matrix G ∈ S |V | + ∩ S G if and only if G can be written as G = t i=1 G i , where G i ∈ S |V | + has nonzero entries only with row and column indices coming from C i for i ∈ [t].Given a graph G(V, E), let Π G be the projection from S |V | to the subspace S G , i.e., for G ∈ S |V | , Π G (G) βγ = G βγ , if β = γ or {β, γ} ∈ E, ) denotes matrices that are projections of PSD matrices onto S G . More precisely, Π G (S |V | + ) = {Π G (G) | G ∈ S One can easily check that the cone Π G (S |V | + ) and the cone S |V | + ∩ S G form a pair of dual cones in S G (see [199, Chapter 10]). Moreover, for a chordal graph G, the decomposition result for matrices in S |V | + ∩ S G given in Theorem 3.1 leads to the following characterization of matrices in the cone Π G (S |V | + ).

Theorem 3 . 2 (

 32 [START_REF] Grone | Positive definite completions of partial hermitian matrices[END_REF],Theorem 7). Let G(V, E) be a chordal graph and assume that C 1 , . .. , C t are the list of maximal cliques of G(V, E). Then a matrixG ∈ Π G (S |V | + ) if and only if G[C i ] 0 for i ∈ [t],where G[C i ] denotes the principal submatrix of G indexed by the clique C i . By Theorem 3.2, to check G ∈ Π G (S |V |

  Next, with c ∈ {1, . . . , p} fixed, for d ∈ N and g ∈ R[x(I c )], let M d (y, I c ) (resp. M d (gy, I c )) be the moment (resp. localizing) submatrix obtained from M d (y) (resp. M d (gy)) by retaining only those rows and columns indexed by β = (β i ) ∈ N n d of M d (y) (resp. M d (gy)) with supp(β) ⊂ I c , where supp(β) := {i | β i = 0}.

.1. 5 )

 5 where (2N) n := {2α | α ∈ N n }. For a graph G(V, E) with V ⊆ N n , let supp(G) := {β + γ | β = γ or {β, γ} ∈ E}. We define the graphs G (0) k,0 := G tsp k and, for i ∈ [m], G (0) k,i is the empty graph with node set V k,i := N n k-gi and empty edge set. Note that supp(G (0)

Example 3 . 1 (

 31 support extension). Assume m = 0, k = 2, and consider the graph G with solid edges shown in Figure3.2. Then by support extension, the two dashed edges are added to G for x 1 x 2 x 3 ∈ supp(G).

Figure 3 . 2 :

 32 Figure 3.2: The support extension of G

Figure 3 . 3 :

 33 Figure 3.3: The tsp graphs of Example 3.2. The dashed edge is added after the maximal chordal extension.1

Figure 3 . 4 :Figure 3 . 5 :

 3435 Figure 3.4: The csp graph of Example 3.3

Figure 3 . 6 :

 36 Figure 3.6: The tsp graph for the second clique of Example 3.3 1

Figure 3 . 7 :

 37 Figure 3.7: The tsp graph without decomposing variables of Example 3.3 1

Theorem 3 . 4 (

 34 sparse representation). Let f ∈ R[x] and S(g) be as in (3.3.2) with the additional quadratic constraints (3.3.1). Let I c , J c be defined as in Section 3.2.1 and A = supp(f ) ∪ m i=1 supp(g i ). Assume that the sign symmetries of A are represented by the column vectors of the binary matrix R. If f is positive on S(g), then

Figure 3 . 8 :

 38 Figure 3.8: The block-band sparsity pattern

( 4 . 1 . 6 )Theorem 4 . 1 .

 41641 By invoking Lemma 4.1 and Proposition 2.4, one obtains the convergence behavior of the sequence (ρ k ) k∈N in the following result. Let f be as in (4.1.3) with V (h) = ∅ as in (4.1.4). Then:

( 4 . 1 . 3 )

 413 then τ k = ρ k = f for some k and (4.1.5) has an optimal solution, i.e., f -f ∈ P k (h).

4 .

 4 If V (h) defined as in (4.1.4) is finite, τ k = ρ k = f for some k ∈ N and both primal-dual (2.4.3)-(2.4.5) have optimal solutions. In this case, the flatness condition holds at order k.

Assumption 4 . 1 .

 41 Consider the following conditions: 1. Zero duality gap of primal-dual (4.1.8)-(4.1.9) holds, i.e., τ = ρ and τ ∈ R. 2. Primal attainability: SDP (4.1.8) has an optimal solution.

Corollary 4 . 1 .

 41 2 and [134, Lemma A.22]. Let conditions 1 and 4 of Assumption 4.1 hold. Assume that the method (T) is globally convergent for NSOP (4.1.14) (e.

Corollary 4 . 2 .

 42 Let condition 4 of Assumption 4.1 hold. Let z be an optimal solution of the NSOP (4.1.14). Define

Remark 4 . 4 .

 44 Let conditions 1, 2 and 5 of Assumption 4.1 hold. We keep all notation from Remark 4.2. By applying Lemma 4.2 for SDP (4.1.12) with CTP, one has

Lemma 4 . 3 .

 43 Let conditions 1, 2 and 5 of Assumption 4.1 hold, and let ψ : R m → R be the function: z → ψ(z) := a max{λ 1 (C -A z), 0} + b z . (4.1.19)

.1. 20 )

 20 Moreover if condition 3 of Assumption4.1 holds, then problem (4.1.20) has an optimal solution.

1 . 6 )

 16 (resp. (4.1.5)) and (4.2.6) (resp. (4.2.7)) are equivalent, the first and second statements follow from Proposition 2.1. The third statement is due to Theorem 4.1.

  2.6)-(4.2.7) becomes primal-dual (4.1.8)-(4.1.9), we then go back to Section 4.1.3 with l = 1.

Example 4 . 1 .

 41 Consider a simple example of POP (4.1.3) with n = 1:

Following Proposition 2 . 3 ,Corollary 4 . 4 .

 2344 p max := max α |p α | for any p ∈ R[x]. We take ε = 0.01 for the experiments in Section 4.3. Corollary 4.1 and Proposition 4.1, we obtain the following corollary: (i) Sequence (τ k ) k∈N of Algorithm 3 is well defined and τ k ↑ f as k → ∞. (ii) Assume that condition (a) or (b) of Proposition 4.1.3 holds. If there exists an optimal solution y of SDP (4.1.6) for some order k ∈ N such that the flat extension condition holds, x exists at the k-th iteration of Algorithm 3. In this case, if X in the first step of Algorithm 3 is well-defined, Algorithm 3 terminates at the k-th iteration, x is an optimal solution of POP (4.1.3) and f = τ k .

5 :

 5 If x exists, set τk+j = τk , j ∈ N >0 , and terminate. As in the single (ball) constraint case, one can choose method (D) in Algorithm 5 as Algorithm 1 with LMBM solver or SketchyCGAL. Following Proposition 2.3, Corollary 4.1 and Proposition 4.3, we obtain the following corollary: Corollary 4.6. (i) The sequence (τ k ) k∈N of Algorithm 5 is well defined and τk ↑ f as k → ∞. (ii) Assume that either condition (a) or condition (b) of Proposition 4.3.3 holds. If there exists an optimal solution y of SDP (4.2.16) at order k ∈ N such that the flat extension condition holds, then x exists at the k-th iteration of Algorithm 5 . In this case, Algorithm 5 terminates at the k-th iteration, x is an optimal solution of POP (4.1.1) and f = τk .

68CHAPTER 4 .

 4 EXPLOITING THE CONSTANT TRACE PROPERTY: EQUALITY CONSTRAINTS

  (4.1.13), SketchyCGAL computes indirectly the smallest eigenvalue of the matrix C + A (y + β(z -b)) in Step 8 of [219, Algorithm 6.1] while relying on the so-called "ApproxMinEvec" subroutine. When the ApproxMinEvec subroutine is implemented via [219, Algorithm 4.2], SketchyCGAL provides approximations of the smallest eigenvalue and eigenvector of each matrix C + A (y + β(z -b)) by using the randomized Lanczos method. It only requires to compute the smallest eigenvalue and eigenvector of a tridiagonal matrix of small size (e.g. s max = 42 when n = 25 in Table 4.6 while the value s max of LMBM is 351). Besides, SketchyCGAL computes v i (C + A (y + β(z -b))v i 1 within the loop from Step 5 of [219, Algorithm 4.2] while relying on three primitive computations (see

Figure 4 . 4 :

 44 Figure 4.4: Efficiency and accuracy comparison for Table 4.7.

Figure 4 . 5 :

 45 Figure 4.5: Efficiency and accuracy comparison for Table 4.8.

Figure 4 . 6 :

 46 Figure 4.6: Efficiency and accuracy comparison for Table 4.10.

  inf y∈R a + {c y : Ay = b} , (5.0.1) where c ∈ R a , A ∈ R b×a and b ∈ R b . With this approach we prove in Section 5.1.4 that several special classes of POPs (including POPs on a ball, annulus, simplex) have CTP. III. Our final contribution, postponed in Appendices 5.3.1 and 5.3.2, is to handle sparse largescale POPs by integrating sparsity-exploiting techniques from Chapter 3 into the CTP-exploiting framework.

3 . 6

 36 describes how to convert SDP (5.1.1) to the form (5.1.3).The dual of SDP (5.1.3) reads

  b, C, S, ζ, s, τ, ρ, a), the primaldual (5.1.3)-(5.1.4) has an equivalent formulation as the primal-dual (5.3.25)-(5.3.26); see also Appendix 5.3.4 with ω = m + 1 and s max = s(k). Then two first-order algorithms (CGAL and SM) are leveraged for solving the primal-dual (5.3.25)-(5.3.26); see Appendix 5.3.4 and Appendix 5.3.5.

.1. 5 ) 2 .

 52 then POP (5.0.2) has CTP, as in Definition 5.1. Assume that h = ∅ and S(g) has nonempty interior. Then POP (5.0.2) has CTP if and only

Remark 5 . 1 .

 51 which yields the desired conclusion. From the proofs of Theorem 5.2 and Theorem 5.1, the constant trace a k and the basis transformation matrix P k (Definition 5.1) can be taken as

Lemma 5 . 2 .

 52 .1.11) where I i is the identity matrix for i ∈ {0} ∪ [m]. If LP (5.1.11) has a feasible solution (ξ k , G i,k , u j,k ) for every k ∈ N ≥kmin , then POP (5.0.2) has CTP with a k = ξ k and P k = diag(G ). The proof of Lemma 5.2 is similar to that of Theorem 5.1 with a k = ξ k and

1 . 5 . 2 .

 152 2r]. Notice that if Assumption 5.1 holds then POP (5.0.2) has r annulus constraints and (m -2r) ball constraints on subsets of variables. Moreover, Q(g) + I(h) is Archimedean due to (1-3) in Assumption 5.Example Assumption 5.1 holds in the following cases:

5 . 1 .

 51 S(g) is an annulus. Proposition If Assumption 5.1 holds then LP (5.1.11) has a feasible solution for every k ∈ N ≥kmin , and therefore POP (5.0.2) has CTP. Proof. Let Assumption 5.1 hold. It is sufficient to show that (5.1.11) has a feasible solution for every k ∈ N ≥kmin .

Algorithm 6 1 : 2 : 3 : 5 . 1 . 5

 6123515 Approximating the optimal value of a dense POP with CTP Input: POP (5.0.2) and a relaxation order k ∈ N ≥kmin Output: The optimal value τ k of SDP (5.1.3) Solve LP(5.1.11) with an optimal solution (ξ k , G i,k , u j,k ); Let a k = ξ k and P k = diag(G Compute the optimal value τ k of SDP (5.1.3) by running an algorithm based on first-order methods and which exploits CTP. Main algorithm Algorithm 6 below solves POP (5.0.2) whose CTP can be verified by LP. Examples of algorithms based on first-order methods and which exploit CTP are CGAL (Algorithm 8 in Appendix 5.3.4) or SM (Algorithm 10 in Appendix 5.3.5).

  by Waki et al. For α ∈ N n , let supp(α) := {j ∈ [n] : α j > 0}. For I ⊆ [n], let x(I) := {x j : j ∈ I} and N I d := {α ∈ N n d : supp(α) ⊆ I}. Assume I ⊆ [n]. Given y = (y α ) α∈N n 2d , the moment (resp. localizing) submatrix associated with I of order d is defined by M d (y, I) := (y α+β ) α,β∈N I d (resp. M d (qy, I) := ( γ q γ y α+β+γ ) α,β∈N I d for q ∈ R[x(I)]). Let v I d := (x α ) α∈N I d with length b(|I|, d) := |I|+d n . For matrices A and B of same sizes, the Hadamard product of A and B, denoted by A • B, is the matrix with entries

Definition 5 . 2 .

 52 (CTP for a POP with CS) We say that POP (5.0.2) with CS has CTP if for every k ∈ N ≥kmin and for every c ∈ [p], there exists a positive number a (c) k and a positive definite matrix P (c) k ∈ S (c,k) such that for all y ∈ R b(2k) ,

Corollary 5 . 3 .

 53 for all k ∈ N ≥kmin and for all c ∈ [p]. As a consequence, POP (5.0.2) has CTP. The proof of Theorem 5.3 being very similar to that of Theorem 5.2 by considering each clique of variables, is omitted. Again by considering each clique of variables, the following result can be obtained from Theorem 5.3 in the same way Corollary 5.1 was obtained. If (5.3.6) holds then Slater's condition for SDP (5.3.2) holds for all k ∈ N ≥kmin .

3 . 8 )

 38 then becomes SDP (5.3.28); see Appendix 5.3.4 with ω c = m c +1 and

Lemma 5 . 3 .

 53 Let POP (5.0.2) with CS be described in Section 5.3.1. If LP (5.3.11) has a feasible solution (ξ ), for every k ∈ N ≥kmin and for every c ∈ [p], then POP (5.0.2) has CTP with P (c)

4 :

 4 Compute the optimal value τ cs k of SDP (5.3.8) by running an algorithm based on first-order methods and which exploits CTP. In Step 4 of Algorithm 7 the two algorithms CGAL (Algorithm 9 in Appendix 5.3.4 or SM (Algorithm 11 in Appendix 5.3.5) are good candidates.

98CHAPTER 5 .

 5 EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS

  2 , and the coefficient f c,α , α ∈ N Ic 2 of f c is randomly generated in (-1, 1) w.r.t. the uniform distribution;3. Take m = p and gi := -x(I i ) 2 2 + 1, i ∈ [m];4. Take a random point a in S(g) w.r.t. the uniform distribution;

2 4 . 5 .

 245 and the nonzero coefficient f c,α with α ∈ N Ic 2 and |α| = 2 is randomly generated in (-1, 1) w.r.t. the uniform distribution; 3. Take m = p and g i := -x(I i ) 2 2 + 1, i ∈ [m]; Take a random point a in S(g) w.r.t. the uniform distribution; Let r := l/p and (W c ) c∈[p] be as in (5.3.24). For every c ∈ [p] and every i ∈ W c , generate a quadratic polynomial h i ∈ R[x(I c )] 2 by (a) for each α ∈ N Ic 2 \{0} with |α| = 2, taking h i,α = 0; (b) for each α ∈ N Ic 2 with |α| = 2, taking a random coefficient h i,α of h i in (-1, 1) w.r.t. the uniform distribution; (c) setting h i,0 := -α∈N Ic 2 \{0} h c,α a α . Then a is a feasible solution of POP (5.0.2).The numerical results are displayed in Table5.20 and 5.21.

Assumption 5 . 3 .

 53 is the cost matrix and b ∈ R ζ is a vector. The dual of SDP (5.3.25) reads ρ = sup y∈R ζ { b y : A y -C ∈ S + } , (5.3.26) where A : R ζ → S is the adjoint operator of A, i.e., A y = i∈[ζ] y i A i . The following assumption will be used later on. Consider the following conditions: 1. Strong duality of primal-dual (5.3.25)-(5.3.26) holds, i.e., ρ = τ and ρ ∈ R.

  Let p ∈ N ≥1 , s c , ω c ∈ N, c ∈ [p], and s (i,c) ∈ N ≥1 , i ∈ [ω p ], c ∈ [p], be fixed such that s c = i∈[ωc] s (i,c) , c ∈ [p]. For every c ∈ [p], let S (c) be the set of real symmetric matrices of size s c in a block diagonal form: X c = diag(X 1,c , . . . , X ωc,c ), such that X i,c is a block of size s (i,c) , i ∈ [ω c ]. Let s max := max i∈[ωp],c∈[p] s (i,c) . For every c ∈ [p], let S (c) + be the set of all X c ∈ S (c) such that X c 0. Then for every c ∈ [p], S (c) is a Hilbert space with scalar product A, B = trace(B A) and S (c) + is a self-dual cone. Let us consider the following SDP:

  and b ∈ R ζ . The dual of SDP (5.3.28) reads ρ = sup y∈R ζ b y : A c y -C c ∈ S

  .3.30) 104CHAPTER 5. EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS Recall that λ min (D) stands for the smallest eigenvalue of a real symmetric matrix D. We denote by c∈[p] S (c) the set of all X = diag(X c ) c∈[p] such that X c ∈ S (c) , for c ∈ [p]. Let C := diag(C c ) c∈[p] and let A : c∈[p] S (c) → R ζ be a linear operator of the form: AX

3 . 28 )

 328 can be rewritten as τ = inf X∈ c∈[p] S (c) + { C, X : AX = b}.The dual operatorA : R ζ → c∈[p] S (c) of A reads A z = diag((A c z) c∈[p] ). Note ∆ c := {X c ∈ S (c) + : trace(X c ) = a c } , for c ∈ [p].Algorithm. In order to solve SDP (5.3.28) with CTP on each subset of blocks, we use[218, Algorithm 1] due to Yurtsever et al. to describe Algorithm 9 with the following setting: X ←

Algorithm 9 1 : 3 :

 913 CGAL-SDP-CTP-Blocks Input: SDP (5.3.28) such that Assumption 5.4 holds; Parameter K > 0. Output: ((X (t) c ) c∈[p] ) t∈N . Set (X (0) c ) c∈[p] := (0 S ) c∈[p] and y 0 := 0 R ζ . 2: for t ∈ N do Set β t := √ t + 1 and η t := 2/(t + 1); 4:

Assumption 5 . 5 .

 55 Dual attainability: SDP (5.3.26) has an optimal solution. Lemma 5.4. Let Assumption 5.3 hold and let ϕ : R ζ → R be a function defined by y → ϕ(y) := aλ min (C -A y) + b y. Then, Moreover, if Assumption 5.5 holds, then problem (5.3.31) has an optimal solution. Notice that ϕ in Lemma 5.4 is concave and continuous but not differentiable in general. The subdifferential of ϕ at y reads: ∂ϕ(y) = {b -aAU : U ∈ conv(Γ(C -A y))}, where for each A ∈ S, Γ(A) := {uu : Au = λ min (A)u , u 2 = 1}. Next, we describe Algorithm 10 to solve SDP (5.3.25), which is based on nonsmooth first-order optimization methods (e.g., LMBM [71, Algorithm 1]).

Assumption 5 . 6 .Lemma 5 . 5 .

 5655 Dual attainability: SDP (5.3.29) has an optimal solution. Let Assumption 5.4 hold and let ψ : R ζ → R be a function defined by y → ψ(y) := b y + c∈[p] a j λ min (C c -A c y). Then, τ = sup y∈R ζ ψ(y) .(5.3.32) Moreover, if Assumption 5.6 holds, then problem (5.3.32) has an optimal solution. 106CHAPTER 5. EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS Proof. From (5.3.28) and Condition 4 of Assumption 5.4,

aProposition 5 . 4 .

 54 .3.33) where I c ∈ S(c) is the identity matrix, for c ∈ [p]. Note that I c , X c = trace(X c ), forX c ∈ S (c) , c ∈ [p]. The dual of this SDP reads c ξ c + b y : C c -A c y -ξ c I c ∈ S ρ = sup ξ,y { c∈[p] a c ξ c + b y : ξ c ≤ λ min (C c -A c y) , c ∈ [p] }.From this, the result follows since ρ = τ . The function ψ in Lemma 5.5 has the following properties:1. ψ is concave and continuous but not differentiable in general. 2. The subdifferential of ψ at y satisfies: ∂ψ(y) = b + c∈[p] a j ∂ψ j (y), where for every c ∈ [p], ψ c : R ζ → R is a function defined by ψ c (y) = λ min (C c -A c y) and ∂ψ c (y) = {-A c U : U ∈ conv(Γ(C c -A c y))}. Proof. It is not hard to prove the first statement. Indeed, ψ is a positive combination of z → b z, ψ c , c ∈ [p], which are convex, continuous functions. The second statement follows by applying the subdifferential sum rule and notice that the domains of z → b z, ψ c , c ∈ [p], are both R n . Next, we describe Algorithm 11 to solve SDP (5.3.28), which is based on nonsmooth first-order optimization methods (e.g., LMBM [71, Algorithm 1]).

3 . 4 )

 34 to the form (5.3.8). Doing a similar process as in Appendix 5.3.6 on every clique, by noting (5.3.7), for every c ∈ [p], the constraints D k (y, I c ) 0 , y 0 = 1 , M k-hj (h j y, I c ) = 0 , j ∈ W c , (5.3.40) become Âc X c = bc for some linear operator Âc : S (c,k) → R ζc and vector bc ∈ R ζc . Moreover, L y (f c ) = C c , X c for some matrix C c ∈ S (c,k) since f c ∈ R[x(I c )], for every c ∈ [p]. Then from (5.3.7), the objective function of SDP (5.3.4) is L y (f ) = c∈[p] C c , X c . Next we describe the constraints depending on common moments on cliques. For every α ∈ ∪ c∈[p] N Ic k , note T (α) := {c ∈ [p] : α ∈ N Ic k }. In other words, T (α) indices the cliques sharing the same moment y

  c) . Hence we obtain the data (C c,k , A c,k , b k , ζ k ) = (C c , A c , b, ζ) of the standard form (5.3.8) by plugging k.

Theorem 6 . 1 .

 61  Corollary 4.3 and 4.4]) Let θ ∈ R[x] be the quadratic polynomial x → θ(x) := 1 + x2 2 , and denote by p ∈ R[x, x n+1 ] the homogeneous polynomial associated with p ∈ R[x], defined by x → p(x) := x deg(p) n+1 p(x/x n+1 ).

  Putinar-Vasilescu's Positivstellensatz. For each q ∈ R[x] 2r , we denote q max,r := max α {|q α |/c n+1 (α, 2r -|α|)}. Let us recall c n (α) := |α|! α1!...αn! for each α ∈ N n . By relying on Baldi-Mourrain's result (Theorem 1.3) after the homogenization trick, it is straightforward to analyze the complexity of Putinar-Vasilescu's Positivstellensatz: Proposition 6.1.

aa j b j = 1 .

 1 x 2 d j (x), where a j , b j ∈ R and d j ∈ R[x], j ∈ [r]. From this and since θ k = 1 + x 2 e(x) for some e ∈ R[x], one has (1 + x 2 e(x))x = j b j x + x 2 p(x) , for some q ∈ R[x]. By comparing coefficients of monomials 1 and x in the two sides of the above equality, It implies that a j = 0, j ∈ [r], and 2 r j=1 a j b j = 1. It follows that 0 = 1. It is impossible. Let us prove the statement (ii). Let ε > 0 and k ∈ N, k ≥ 2. Since θ k = 1 + kx 2 + x 4 e(x) for some e ∈ R[x] 2k-4 , one has

Figure 6 . 1 :

 61 Figure 6.1: Illustration of Lemma 6.3.

Figure 6 . 2 :

 62 Figure 6.2: Plot of the complexity.

Lemma 7 . 1 .Lemma 7 . 2 .

 7172 Let g : U → R be an analytic function on an open set U ⊂ R n and Z := {x ∈ K : g(x) = 0} for some compact set K ⊂ U . Then there exists α > 0 and C > 0 such thatd(x, Z) α ≤ C|g(x)| , ∀x ∈ K .As a consequence of Lemma 7.1, we obtain the following result: ( [22, Corollary 2.6.7]) Let r > 0 and the semialgebraic set S := {x ∈ R n : g i (x) ≥ 0 , i ∈ [m]}, where g 1 , . . . , g m are polynomials. Then there exist positive constants α and C such that, for all x in B(0, r), dist(x, S) α ≤ -C min{g 1 (x), . . . , g m (x), 0} .(7.1.19)

.1. 20 )∇ y x -y 2 2 =

 202 Proof. If x ∈ Z, both sides of (7.1.20) are zeros. Let x ∈ R n \Z be fixed. Then d(x, Z) 2 = min y { xy 2 2 : g(y) = 0}. Assume that (y, µ) ∈ R n × R satisfies the Karush-Kuhn-Tucker conditions: µ∇ y g(y) , g(y) = 0 . (7.1.21)

Definition 7 . 1 .( 1 -Lemma 7 . 6 .

 71176 (Multivariate Bernstein polynomials) Let d ∈ N n and f ∈ C([0, 1] n ). The polynomials B f,d (x) x j ) dj -kj(7.1.24) are called the multivariate Bernstein polynomials of f . Note that deg(B f,d ) = j∈[n] d j and the binomial identity implies sup x∈[0,1] n |B f,d (x)| ≤ sup x∈[0,1] n |f (x)| . (7.1.25) 136CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ (Error bound) If

k 1 2 ( 7 . 1 . 28 ) 2 |

 271282 then for all k ∈ N ≥1 , the inequality B y →f (2y-e),ke x + e 2 -f (x) ≤ L n holds for all x ∈ [-1, 1] n . Moreover, we have sup x∈[-1,1] n |B y →f (2y-e),ke x+e ≤ sup x∈[-1,1] n |f (x)| . (7.1.29) Proof. Define g : [0, 1] n → R by g(x) := f (2x -e). Let us compute a Lipschitz constant of g. With x, y ∈ [0, 1] n , by the Lipschitz continuity of f , we have |g(x) -g(y)| = |f (2x -e) -f (2y -e)| ≤ L 2x -e -2y + e 2 = 2L xy 2 . (7.1.30)

wm 2 ) 2 ξ m = M m g m = -δm 2 gFigure 7 . 1 :

 22271 Figure 7.1: Illustration for the proof of the Lipschitz continuity of ϕ m on K (rectangle). Here K = S m-1 ∩ B(0, √ n + m -1) and U = K + wm 2 B • (0, 1) with the notation of Lemma 7.4.

.1. 61 )

 61 Here we have covered K by a finite sequence of balls with radii wm 2 and centers lying on K.The function ϕ m has a Lipschitz continuous extension φm . Let φm : R n → R be the function defined by φm (x) := infy {ϕ m (y) + L φm xy 2 : y ∈ S m-1 ∩ B(0, √ n + m -1)} . (7.1.62) By Kirszbraun's theorem (stated in Lemma 7.5), φm is Lipschitz continuous with Lipschitz constant L φm and φm = ϕ m on S m-1 ∩ B(0, √ n + m -1). The function φm is even, positive and has a finite upper bound on B(0, √ n + m) depending on ε. Let us prove that φm is even. Consider φm (-x) = inf y {ϕ m (y) + L φm -xy 2 : y ∈ S m-1 ∩ B(0, √ n + m -1)} . (7.1.63)Let y be any feasible solution of (7.1.63). Since g 1 , . . . , g m-1 are even,S m-1 ∩ B(0, √ n + m) is symmetric, i.e., S m-1 ∩ B(0, √ n + m) = -S m-1 ∩ B(0, √ n + m), it turns out that -y is a feasible solution of(7.1.63). Thus, φm (-x) = inf -y {ϕ m (-y) + L φm -x + y 2 : -y ∈ S m-1 ∩ B(0, √ n + m -1)} = inf y {ϕ m (y) + L φm yx 2 : y ∈ S m-1 ∩ B(0, √ n + m -1)} = φm (x) ,(7.1.64)

.1. 68 )-z 2 ≤y-z 2 ≤y-z 2 +y-z 2 =y-z 2 ≤ 2 φm-z 2 =

 682222222 We now estimate the Lipschitz constant of f m-1 on B(0,√ n + m). Let y, z ∈ B(0, √ n + m) such that y = z. Then |fm-1(y)-fm-1(z)| y|f (y)-f (z)|+| φm(y) 2 gm(y)-φm(z) 2 gm(z)| L f + | φm(y) 2 gm(y)-φm(z) 2 gm(y)| | φm(z) 2 gm(y)-φm(z) 2 gm(z)| L f + |gm(y)|| φm(y)+ φm(z)|| φm(y)-φm(z)|+ φm(z) 2 |gm(y)-gm(z)| L f + 2Cg m C φm L φm y-z 2+C Lg m y-z 2 yL f + 2C gm L φm C φm + L gm C 2 φm =: L fm-1 .(7.1.69)

.1. 71 )d 1 2 ,

 712 Polynomial approximations for the weight functionsApproximating with Bernstein polynomials. For each i ∈ [m], we now approximate φi on [-1, 1] n with the following Bernstein polynomials:B (d) i (x) = B y → φi(2y-e),de x + e 2 , d ∈ N , (7.1.72)with e = (1, . . . , 1) ∈ R n . By using Lemma 7.7, for allx ∈ [-1, 1] n , for i ∈ [m], |B (d) i (x) -φi (x)| ≤ L φi n d ∈ N ,(7.1.73)and the following inequality holds for all x ∈ [-1, 1] n , for i ∈ [m]: |B (d) i (x)| ≤ sup x∈[-1,1] n | φi (x)| ≤ C φi . (7.1.74) For i ∈ [m], let

( 7 .

 7 1.77) 144CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ and

.1. 82 )

 82 It implies that F is a positive definite form of degree 2D with inf x∈S n-1 F (x) ≥ ε (m+1)2 m and sup x∈S n-1 F (x) ≤ C F . There is no loss of generality in assuming C F = bε -b for some large enough b > 0 independent of ε. Similarly assume that D ≥ dε -d for some large enough d > 0 independent of ε. From this,δ(F ) ≤ bε -b ε (m+1)2 m = b(m + 1)2 m ε -b-1 . (7.1.83) Set K := 2ndε -d (2dε -d -1) 4 log 2 b(m + 1)2 m ε -b-1 .

Assumption 8 . 1 .

 81 The following conditions hold: (i) Running intersection property (RIP): [n] = p c=1 I c with p ∈ N ≥2 , I c = ∅, c ∈ [p], and for every c ∈ {2, . . . , p}, there exists s c ∈ [c -1], such that Îc ⊂ I sc , where Îc := I c ∩ c-1 j=1 I j . Note that s 2 = 1 and w.l.o.g, set Î1 := ∅. Denote n c := |I c | and nc := | Îc |, c ∈ [p]. (ii) Structured sparsity pattern for the objective function 1 : f = p c=1 f c where f c ∈ R[x(I c )] deg(f ) , c ∈ [p]. (iii) Structured sparsity pattern for the constraints: [m] = p c=1 J c and for every

Theorem 8 . 2 . 2 , 1 )Corollary 8 . 1 . 2 )Example 8 . 1 .

 822181281 Let Assumption 8.1 (i) hold. Let f ∈ R(x) be a positive definite rational form of degree 2d with d ∈ N >0 such that whereq c ∈ R[x(I c )] is homogeneous of degree 2(d + k c ) for some k c ∈ N, c ∈ [p]. Then there exist k ∈ N and σ c ∈ Σ[x(I c )] d+k(1+deg(Φc)/2) , c ∈ [p], such thatThe proof of Theorem 8.2 can be found in [133,Section 4]. As a consequence, we obtain the following sparse version of Reznick's Positivstellensatz. Let Assumption 8.1 (i) hold. Assume that f is a positive definite form of degree2d with d ∈ N >0 and f = p c=1 f c , where f c ∈ R[x(I c )] is homogeneous of degree 2d, c ∈ [p]. Then there exist k ∈ N and σ c ∈ Σ[x(I c )] d+k(1+deg(Φc)/2) , c ∈ [p], such that f = where H c := x(I c ) 2 2 Φ c , c ∈ [p].To prove Corollary 8.1, we apply Theorem 8.2 with k c = 0, c ∈ [p]. The representation (8.1.2) can still hold even when f is not a positive definite form, as illustrated in the following example. Let f = f 1 + f 2 , where

8. 1 . 3 cCorollary 8 . 2 . 3 )Example 8 . 2 .

 1382382 Global nonnegativityWhen Assumption 8.1 (i) holds, define the following polynomials, for each c ∈ [p]:• θ c := x(I c ) 2 2 + 1 and θc := x( Îc ) if c = p ; • Θ c := θ c D c and ω c := deg(Θ c )/2. Note that Θ c ∈ Σ[x(I c )] ωc ,for each c ∈ [p]. We next state the following sparse version of Putinar-Vasilescu's Positivstellensatz for polynomials nonnegative on R n . Let f be a nonnegative polynomial such that the conditions (i) and (ii) of Assumption 8.1 hold. Let ε > 0 and d ≥ deg(f )/2. Then there exist k ∈ N and σ c ∈ Σ[x(I c )] d+kωc , c ∈ [p], such that The proof of Corollary 8.2 can be found in [133, Section 4]. The representation (8.1.3) can still hold even if ε = 0, as illustrated in the following examples. Let f = f 1 + f 2 , where

Corollary 8 . 3 .. ( 8 . 1 . 4 )

 83814 Let f ∈ R[x] be nonnegative on S(g) such that the conditions (i), (ii) and (iii)of Assumption 8.1 hold. Let ε > 0 and d ≥ 1 + deg(f )/2 . Then there exist k ∈ N, σ 0,c ∈ Σ[x(I c )] d+kωc and σ i,c ∈ Σ[x(I c )] d+kωc-gi , i ∈ J c , c ∈ [p], such that c + i∈Jc σ i,c g i Θ k cThe proof of Corollary 8.3 can be found in[START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF] Section 4].Example 8.4.

( 8 . 2 . 3 )Proposition 8 . 1 .Remark 8 . 2 .

 8238182 We guarantee strong duality for the pair of primal-dual problems (8.2.3)-(8.2.2). Assume that S(g)has nonempty interior. Let k ∈ N. Then τ sparse k (ε) = ρ sparse k (ε). Moreover, if τ sparse k (ε) > -∞, the optimal value ρ sparse k (ε) is attained.The proof of Proposition 8.1 can be found in[START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF] Section 4]. The condition τ sparse k (ε) > -∞ is always satisfied whenever k is sufficently large. Indeed by weak duality, when ε is fixed and k is sufficiently large then τ sparse k

γ

  , α+β ∈ 2N n . Moreover, if α, β ∈ N n di and α + β ∈ 2N n , then there exists γ ∈ {0, 1} n such that α, β ∈ Λ (i) γ . It implies that all blocks on the diagonal of Ḡ(i) must be ( Ḡ(i) α,β ) α,β∈Λ (i) γ , γ ∈ {0, 1} n . (9.1.14)

Corollary 9 . 3 .

 93 (Dense representation without multiplier) Let f, g i , S be as in Corollary 9.1 such that g 1 := R -x2 2 for some R > 0 and g m := 1. Denote d gi := deg(g i )/2 . Then the following statements hold:

  .1.19)

( 9 . 3 . 2 )

 932 Here m := m + n and g m+j := x j , j ∈ [n]. Here g m := 1 and m ∈ J c ⊂ [ m], for c ∈ [p]. As shown by Baldi and Mourrain [13], the sequence (τ Put k ) k∈N converges to f with the rate of at

.3. 6 )

 6 For every c ∈ [p] and every i ∈ J c , generate a quadratic polynomialg i ∈ R[x(I c )]2 by (a) for each α ∈ N Ic 2 \{0}, taking a random coefficient G i,α of h i in (-1, 1) w.r.t. the uniform distribution; (b) setting g i,0 := 0.125 -α∈N Ic 2 \{0} g j,α a α .

.4. 6 )

 6 Note that deg(gi ) = d gi , i ∈ [ m]. Since S has nonempty interior and ∪ n j=1 {x ∈ R n : x j = 0} has zero Lebesgue measure in R n , S\(∪ n j=1 {x ∈ R n : x j = 0}) also has nonempty interior. Then there exists a ∈ S\(∪ n j=1 {x ∈ R n :x j = 0}) such that g i (a) > 0, i ∈ [m]. Let b = ( |a 1 |, . . . , |a n |). Then b ∈ (0, ∞) n and b 2 = (|a 1 |, . . . , |a n |). Since each g i is even in each variable, gi (b) = g i (b 2 ) = g i (a) > 0, i ∈ [m], yielding S has nonempty interior.

Step 3 :d 1 2 ,ε

 32 Approximating with Bernstein polynomials. For each i ∈ [ m], we now approximate φi on [-1, 1] n with the following Bernstein polynomials defined as in [132, Definition 1]: B (d) i (x) = B y → φi(2y-e),de x + e 2 , d ∈ N , (9.4.8) with e = (1, . . . , 1) ∈ R n . By using [132, Lemma 6], for allx ∈ [-1, 1] n , for i ∈ [ m], |B (d) i (x) -φi (x)| ≤ L φi n d ∈ N , (9.4.9)and for all x ∈ [-1, 1] n , for i ∈ [ m]:|B (d) i (x)| ≤ sup x∈[-1,1] n | φi (x)| ≤ C φi . (9.4.10)For i ∈ [ m], letd i := 2u i with u i C gi is an upper bound of |g i | on B(0, √ n + i). Set q i := B (di) i , i ∈ [ m]. Then for all x ∈ [-1, 1] n , |q i (x) -φi (x)| = |B 2Cg i C φi ( m+1)2 m .

1 2

 1 i ) ≤ 2nu i ,(9.4.16) for i ∈ [ m]. From (9.4.1), we have 2u i x j + 1 kj 1 -x j

(

  j∈[n] x j ) k F has positive coefficients.• For each i ∈ [ m] and for all k ∈ N satisfying k ≥ 4nu i (4nu i -1)T pi ε mCg i =: K i , (9.4.26)( j∈[n] x j ) k p i has positive coefficients.

2 (f + 2ε x 2d f 2 ) 2 = 2 ,σ i = s i (x 2 ) x 2 ( 2 (f + 2ε x 2d f 2 )

 2222222 4.22) with ( j∈[n] x j ) k , we gets 0 = ( j∈[n] x j ) D-d f +k ( f + 2ε( j∈[n] x j ) d f ) -i∈[ m] gi s i ( j∈[n] x j ) D-4nui-dg i , (9.4.27)where s 0 := ( j∈[n] x j ) k F and s i := ( j∈[n] x j ) k p i are homogeneous polynomials having nonnegative coefficients. Replacing x by x 2 , we obtain:x 2(D-d f +k) = σ 0 + i∈[m] g i σ i , (9.4.28)whereσ 0 = s 0 (x 2 ) + j∈[n] gj+m (x 2 )s j+m (x 2 ) x 2(D-4nuj+m-dg j+m ) s 0 (x 2 ) + j∈[n] x 2 j s j+m (x 2 ) x 2(D-4nuj+m-dg j+m ) D-4nui-dg i ) 2 , i ∈ [m] , (9.4.30) are SOS of monomials. Set K = D -d f + K. Then x 2K = σ 0 + m i=1 g i σ i with deg(σ 0 ) = deg(g i σ i ) = 2(K + d f ), for i ∈ [m]. This completes the proof of Theorem 9.1.

For

  every t ∈ N, denote vt (x) := v t ( 1 2 (x + e), 1 2 (xe)) = ( 1 2 |α+β| (x + e) α (xe) β ) (α,β)∈N 2n t , (9.4.31)where e := (1, . . . , 1) ∈ R n . As a consequence of Corollary 9.2, the next proposition shows that the weighted SOS polynomials in Putinar-Vasilescu's Positivstellensatz can be associated with diagonal Gram matrices via a change of monomial basis.

Remark 9 . 15 .

 915 Representation (9.4.38) in Proposition 9.3 is similar in spirit to the one ofRoebers et al. in

9. 4 . 4 Theorem 9 . 6 .

 4496 Polynomial optimization on the nonnegative orthant: Noncompact caseLinear relaxationsGiven ε > 0, consider the hierarchy of linear programs indexed by k ∈ N:τ Pól k (ε) := inf y L y (θ k ( f + εθ d f )) s. t. y = (y α ) α∈N n 2(d f +k) ⊂ R , L y (θ k ) = 1 , diag(M ki (ǧ i y)) ∈ R b(n,ki) + , i ∈ [m] , (9.4.40)wherek i := k + d f -d gi , i ∈ [m]. Here ǧm = 1. Note that diag(M ki (ǧ i y)) = ( γ∈N n 2dg i y 2α+γ ǧi,γ ) Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.2.1) with S being defined as in (9.2.2). Let ε > 0 be fixed. For every k ∈ N, the dual of (9.4.40) reads as:ρ Pól k (ε) := sup λ,ui λ s. t. λ ∈ R , u i ∈ R b(n,ki) + , i ∈ [m] , θ k ( f -λ + εθ d f ) = m i=1 ǧi v ki diag(u i )v ki .(9.4.42)

τTheorem 9 . 7 ..( 9 . 4 . 44 )Remark 9 . 17 .

 979444917 Pól k,s (ε) := inf y L y (θ k ( f + εθ d f )) s. t. y = (y α ) α∈N n 2(d f +k) ⊂ R , L y (θ k ) = 1 , M A (s,k i ) j (ǧ i y) 0 , j ∈ [b(n, k i )] , i ∈ [m] ,(9.4.43) wherek i := k + d f -d gi , i ∈ [m]. Here ǧm = 1. Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.2.1) with S being defined as in (9.2.2). Let ε > 0 be fixed. For every k ∈ N and for every s ∈ N >0 , the dual of (9.4.43) reads as:ρ Pól k,s (ε) := sup λ,Gij λ s. t. λ ∈ R , G ij 0 , j ∈ [b(n, k i )] , i ∈ [m] , θ k ( f -λ + εθ d f ) = i∈[m] ǧi j∈[b(n,ki)] v A (s,k i ) j G ij v A (s,k i ) jThe following statements hold:1. For all k ∈ N >0 and for every s∈ N >0 , ρ Pól k = ρ Pól k,1 (ε) ≤ ρ Pól k,s (ε). 2. For every s ∈ N >0 , there exists K ∈ N such that for every k ∈ N satisfying k ≥ K, 0 ≤ f -ρ Pól k,s (ε) ≤ εθ(x 2 ) d f .3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such thatfor every s ∈ N >0 and for every k∈ N satisfying k ≥ cε -c , 0 ≤ f -ρ Pól k,s (ε) ≤ εθ(x 2 ) d f . 4. IfS has nonempty interior, for every s ∈ N >0 and for every k ∈ N strong duality holds for the primal-dual problems (9.4.43)-(9.4.44). The proof of Theorem 9.7 is based on Theorem 9.6, Theorem 7.2 and the inequalities ρ Pól k(ε) ≤ ρ Pól k,s (ε) ≤ ρ (ε) k , where ρ (ε) k is defined as in (7.2.4). For each q ∈ R[x] d , denote the degree-d homogenization of q by x d n+1 q( x xn+1 ) ∈ R[x, x n+1 ]. Let (λ, G ij )be a feasible solution of (9.4.44) and consider the case of m = 1. Then the equality constraint of (9.4.44) becomesθ k ( f -λ + εθ d f ) = j∈[b(n,km)] v A (s,km) j G mj v A (s,km )

Assumption 9 . 1 . 2 . 3 .

 9123 With p ∈ N >0 , the following conditions hold:1. There exists(I c ) c∈[p] being a sequence of subsets of [n] such that ∪ c∈[p] I c = [n] and ∀c ∈ {2, . . . , p} , ∃r c ∈ [c -1] : I c ∩ (∪ c-1 t=1 I t ) ⊂ I rc . (9.4.46) Denote n c := |I c |, for c ∈ [p]. With m ∈ N >0 and (g i ) i∈[m] ⊂ R[x], there exists (J c ) c∈[p] being a finite sequence of subsets of [m] such that ∪ c∈[p] J c = [m] and ∀c ∈ [p] , (g i ) i∈Jc ⊂ R[x(I c )] . (9.4.47) For every c ∈ [p], there exists i c ∈ J c and R c > 0 such that g ic := R c -x(I c ) 2 2 . (9.4.48) The condition (9.4.46) is called the running intersection property (RIP). Let θ c := 1 + x(I c ) 2 2 , c ∈ [p].

( 9 . 4 . 49 ) 2 .Remark 9 . 18 .

 94492918 Let f = f 1 +• • •+f p be a polynomial such that f is positive on S and for every c ∈ [p], f c ∈ R[x(I c )] is even in each variable. Then there exist d, k ∈ N,h c ∈ R[x(I c )], σ 0,c , σ j,c ∈ R[x(I c )], for j ∈ J c and c ∈ [p], such that the following conditions hold:1. The equality f = h 1 + • • • + h p holds and h c is a polynomial of degree at most 2d which is even in each variable. For all i ∈ J c and c ∈ [p], σ 0,c , σ i,c are SOS of monomials satisfyingdeg(σ 0,c ) ≤ 2(k + d) and deg(σ i,c g i ) ≤ 2(k + d) (9.4.50) and θ k c h c = σ 0,c + i∈Jc σ i,c g i . (9.4.51) Proof. Let ε > 0. Similarly as in Step 1 of the proof of Theorem 9.1, f = f1 + • • • + fm is positive on the semialgebraic set S defined as in (9.4.5). For every c ∈ [p], let Jc := J c ∪ (m + I c ). Recall that gm+j := x j , j ∈ [n]. By applying [68, Lemma 4], there exist polynomials sc , q i,c ∈ R[x(I c )], for j ∈ Jc and c ∈ [p], such that f = p c=1 (s c + i∈ Jc q 2 i,c gi ) ,(9.4.52)and for all c ∈ [p], s c is positive on the set{x(I c ) ∈ R nc : x j ≥ 0 , j ∈ I c , gic (x) = R c -j∈Ic x j ≥ 0} . (9.4.53) Set h c := s c (x 2 ) + i∈ Jc q i,c (x 2 ) 2 gi (x 2 ), c ∈ [p]. Let d ∈ N such that 2d -1 ≥ max{deg(h c ) : c ∈ [p]}. Then f = p c=1 h c with h c ∈ R[x(I c )]2d being even in each variable and positive on the semialgebraic setS c := {x(I c ) ∈ R nc : g i (x) ≥ 0 , i ∈ J c } . (9.4.54)Note that g ic := R c -x(I c ) 2 2 with i c ∈ J c . By applying Corollary 9.2, there exists k c ∈ N such that for all K ≥ k c , there exist σ 0,c , σ i,c ∈ R[x(I c )], i ∈ J c , such that σ 0,c , σ i,c are SOS of monomials satisfyingdeg(σ 0,c ) ≤ 2(K + d) and deg(σ i,c g i ) ≤ 2(K + d) (9.4.55)for all i ∈ J c , andθ K c h c = σ 0,c + i∈Jc σ i,c g i . (9.4.56) Set k = max{k (c) : c ∈ [p]}. Finally, we obtain the desired results. In Theorem 9.8, it is not hard to see that f has a rational SOS decomposition f = c∈[p]

Theorem 9 . 9 . 9 . 4 . 6

 99946 (Sparse representation without multiplier) Let f, g 1 , . . . , g m be as in Theorem 9.8. Assume that g m := 1 and m ∈ J c , for all c ∈ [p]. Denote d gi := deg(g i )/2 . Then there existk ∈ N, SOS of monomials σ i,j,c ∈ R[x(I c )], for j = 0, . . . , k -d gi , i ∈ J c and c ∈ [p], satisfying deg(σ i,j,c g j ic g i ) ≤ 2k (9.4.58) and f = c∈[p] i∈Jc k-dg i j=0 σ i,j,c g j ic g i . (9.4.59) Sparse polynomial optimization on the nonnegative orthant Consider the following POP: f := inf x∈S f (x) , (9.4.60)where f ∈ R[x] andS = {x ∈ R n : x j ≥ 0 , j ∈ [n] , g i (x) ≥ 0 , i ∈ [m]} , (9.4.61)for someg i ∈ R[x], i ∈ [m], with g m = 1. Assume that f > -∞ and problem (9.4.60) has an optimal solution x . Then POP (9.4.60) is equivalent tof := inf x∈ Š f , (9.4.62)where Š = {x ∈ R n : ǧi (x) ≥ 0 , i ∈ [m]} ,(9.4.63)

Assumption 9 . 2 . 2 .Theorem 9 . 10 . 1 . 2 .Theorem 9 . 11 .

 92291012911 With p ∈ N >0 , the first two conditions of Assumption 9.1 and the following conditions hold:1. For every c ∈ [p], there exist i c ∈ J c and R c > 0 such thatg ic = R c -j∈Ic x j .(9.4.64) There existf c ∈ R[x(I c )], for c ∈ [p], such that f = f 1 + • • • + f p .Linear relaxationsBased on the extension of Pólya's Positivstellensatz: Consider the hierarchy of linear programs indexed by k, d ∈ N: . y = (y α ) α∈N n 2d ⊂ R , y (c) = (y(c) α ) α∈N n 2(d+k) ⊂ R , c ∈ [p] , diag(M d (y, I c )) = diag(M d (θ k c y (c) , I c )) , c ∈ [p] , diag(M k (d) i (ǧ i y (c) , I c )) ∈ R b(nc,k (d) i ) + , i ∈ [m] , c ∈ [p] , y 0 = 1 , (9.4.65)where k(d) i := k + d -d gi . Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.4.60) with S being defined as in(9.4.61). Let Assumption 9.2 hold. The dual of SDP(9.4.65) reads as: . λ ∈ R , u c ∈ R b(nc,d) , w J c , c ∈ [p] , f -λ = c∈[p] h c , h c = v N Ic d diag(u c )v N Ic d , c ∈ [p] , θ k c h c = i∈Jc ǧi v For all k ∈ N and for every s ∈ N >0 , ρ spPól k-1,d ≤ ρ spPól k,d ≤ ρ spPól k,d+1 ≤ f . 2. One has sup{ρ spPól k,d : (k, d) ∈ N 2 } = f . (9.4.67) Proof. It is fairly easy to see that the first statement holds. Let us prove the second one. Let ǧic := R c -x(I c ) 2 2 and ε > 0. Then f -(f -ε) > 0 on S. By applying Theorem 9.8, there exist d, k ∈ N, h c ∈ R[x(I c )], σ 0,c , σ j,c ∈ R[x(I c )], for j ∈ J c and c ∈ [p], such that the following conditions hold: 1. The equality f -(f -ε) = h 1 + • • • + h p holds and h c is a polynomial of degree at most 2d which is even in each variable. For all i ∈ J c and c ∈ [p], σ 0,c , σ i,c are SOS of monomials satisfying deg(σ 0,c ) ≤ 2(k + d) and deg(σ i,c ǧi ) ≤ 2(k + d) (9.4.68) and θ k c h c = σ 0,c + i∈Jc σ i,c ǧi . (9.4.69)It implies that there existsu c ∈ R b(nc,d) , w i ∈ J c and c ∈ [p]. It implies that (f -ε, u c , w(c)i ) is an optimal solution of LP(9.4.66). Thus ρ Pól k,d ≥ f -ε, yielding(9.4.67).Based on the extension of Handelman's Positivstellensatz: Consider the hierarchy of linear programs indexed by k ∈ N: . y = (y α )α∈N n 2k ⊂ R , y 0 = 1 , diag(M kij ((ǧ i ǧj ic )y, I c )) ∈ R b(nc,kij ) + , c ∈ [p] , i ∈ [m] , j ∈ {0} ∪ [k -d gi ] , (9.4.71) where k ij := k -d gi -j, for i ∈ [m], for j ∈ {0} ∪ [k -d gi ]. Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.4.60) with S being defined as in(9.4.61). Let Assumption 9.2 hold. The dual of SDP (9.4.71) reads as: . λ ∈ R , w(c) ij ∈ R b(nc,kij ) + , c ∈ [p] , i ∈ J c , j ∈ {0} ∪ [k -d gi ] , f -λ = c∈[p] i∈Jc k-dg i j=0

= 9 . 3 .

 93 j ∈ 2N I } , j ∈ N >0 , I ⊂ [n] . (9.4.74) Then for all j ∈ N >0 and for all I ⊂ [n], W (I) j = ∅ since j ∈ W (I) j . For every j ∈ N and for every I ⊂ [n], we write W I d , I ⊂ [n] , j, s ∈ N >0 , d ∈ N . (9.4.75) For every s ∈ N >0 , for every d ∈ N and for every I ⊂ [n], define A j = 2, . . . , b(|I|, d), define A ∅ , ∀l ∈ [j -1] , [b(|I|, d)] (9.4.77) are overlapping blocks of size at most s in (α + β) (α,β∈N I d ) . Example Consider the case of n = d = s = 2, I 1 = {1} and I 2 = {2}. Matrix (α+β) (α,β∈N 2 2 )

Theorem 9 . 12 . 1 . 2 .Theorem 9 . 13 . 2 .

 912129132 = ∅. For every I ⊂ [n], with B = {β 1 , . . . , β r } ⊂ N I such that β 1 < • • • < β r , for every h = γ∈N I h γ x γ ∈ R[x(I)] and y = (y α ) α∈N n ⊂ R, denote M B (hy, I) := ( γ∈N I h γ y γ+β i +β j ) i,j∈[r] . Based on the extension of Pólya's Positivstellensatz: Consider the hierarchy of linear programs indexed by k, d ∈ N and s ∈ N >0 :τ spPól k,d,s := inf y,y (c) L y ( f ) s. t. y = (y α ) α∈N n 2d ⊂ R , y (c) = (y (c) α ) α∈N n 2(d+k) ⊂ R , c ∈ [p] , diag(M d (y, I c )) = diag(M d (θ k c y (c) , I c )) , c ∈ [p] , y 0 = 1 , M A (s,k (d) i ) j,Ic (ǧ i y (c) , I c ) 0 , j ∈ [b(n c , k (d) i )] , i ∈ J c , c ∈ [p] , (9.4.80)where k(d) i := k + d -d gi . Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.4.60) with S being defined as in(9.4.61). Let Assumption 9.2 hold. The dual of SDP (9.4.80) reads as:ρ spPól k,d,s := sup λ,uc,G (c) i,j λ s. t. λ ∈ R , u c ∈ R b(nc,d) , G (c) i,j 0 , j ∈ [b(n c , k (d) i )] , i ∈ J c , c ∈ [p] , f -λ = c∈[p] h c , h c = v N Ic d diag(u c )v N Ic d , c ∈ [p] , θ k c h c = i∈Jc ǧi j∈[b(nc,k For all k, d ∈ N and for every s ∈ N >0 , ρ spPól k,d = ρ spPól k,d,1 ≤ ρ spPól k,d,s ≤ f . For every s ∈ N >0 , sup{ρ spPól k,d,s : (k, d) ∈ N 2 } = f .Proof. It is not hard to prove the first statement. The second one is due to the second statement of Theorem 9.10 and the inequalities ρ spPólk,d ≤ ρ spPól k,d,s ≤ f .Based on the extension of Handelman's Positivstellensatz:Consider the hierarchy of linear programs indexed by k ∈ N and s ∈ N >0 : . y = (y α )α∈N n 2k ⊂ R , y 0 = 1 , M A (s,k ij ) r,Ic ((ǧ i ǧj ic )y, I c ) 0 , c ∈ [p] , i ∈ J c , j ∈ {0} ∪ [k -d gi ] , r ∈ [b(n c , k ij )] ,(9.4.82)wherek ij := k -d gi -j, for i ∈ [m], for j ∈ {0} ∪ [k -d gi ]. Let f, g i ∈ R[x], i ∈ [m], with g m = 1. Consider POP (9.4.60) with S being defined as in(9.4.61). Let Assumption 9.2 hold. The dual of SDP (9.4.82) reads as:. λ ∈ R , G (c) ijr 0 , c ∈ [p] , i ∈ J c , j ∈ {0} ∪ [k -d gi ] , r ∈ [b(n c , k ij )] , ˇf -λ = c∈[p] i∈Jc k-dg i j=0 ǧi ǧj ic r∈[b(nc,kij )] v A (s,k ij )The following statements hold:1. For all k ∈ N and for every s ∈ N >0 , For every s ∈ N >0 , the sequence (ρ spHan k,s) k∈N converges to f . Proof. It is not hard to prove the first statement. The second one is due to the second statement of Theorem 9.11 and the inequalities ρ spHan k

Algorithm 15

 15 Extraction algorithm for sparse POPs on the nonnegative orthant Input: precision parameter ε > 0 and an optimal solution (λ, u c , G (c) i,j ) of SDP(9.4.81). Output: an optimal solution x of POP(9.4.60).

1 :

 1 For c ∈ [p], do: a: For j ∈ [b(n c , k j) pq = 0 if (p, q) / ∈ (A (s,k (d) m ) j,Ic

)

  Let G (c) := j∈[b(nc,k (d) m )] Ḡ(c) j .Then G(c) is the Gram matrix corresponding to σ m,c in the rational SOS decomposition f -λ = c∈[p]

3 : 4 :

 34 If z exists, verify that z is an approximate optimal solution of POP (9.4.62) by checking the following inequalities:| f (z ) -λ| ≤ ε f max and ǧi (z ) ≥ -ε ǧi max , i ∈ [m] , (9.4.87)where q max := max α |q α | for any q ∈ R[x]. If the inequalities (9.4.87) hold, set x := z 2 .

Figure 10 . 1 :

 101 Figure 10.1: A two hidden layer neural network.

Table 4 .

 4 2. . . . . . . . . . . . . . . . . . . . 4.2 Efficiency and accuracy comparison for Table 4.3. . . . . . . . . . . . . . . . . . . . 4.3 Efficiency and accuracy comparison for Table 4.4. . . . . . . . . . . . . . . . . . . . 4.4 Efficiency and accuracy comparison for Table 4.7. . . . . . . . . . . . . . . . . . . . 4.5 Efficiency and accuracy comparison for Table 4.8. . . . . . . . . . . . . . . . . . . . 4.6 Efficiency and accuracy comparison for Table 4.

Table 1 :

 1 Table of Notation

Table 1 . 1 :

 11 Some known Positivstellensätze.

	18

Table 1 . 2 :

 12 Recent fastest convergence rates in polynomial optimization.

	Year	Author(s) for com-plexity	Author(s) Positivstellen-for satz	Semialgebraic set	Cone	Programming Rate
	1995 Reznick [175]		Reznick [175]	whole space	SOS	SDP	O(ε -1 )
	2001 Powers-Reznick [167]	Pólya [166]	simplex	nonnegative orthant	LP	O(ε -1 )
	2004 Schweighofer [188]	Schmüdgen [185]	compact	preordering SDP	O(ε -c )
	2016	Fawzi-Saunderson-Parrilo [58]	Schmüdgen [185]	boolean hypercube	preodering SDP	O(ε -1 )
	2020 Fang-Fawzi [57]		Schmüdgen [185]	sphere	preodering SDP	O(ε -1/2 )
	2021 Laurent-Slot [112]	Schmüdgen [185]	hypercube	preodering SDP	O(ε -1/2 )
	2021 Baldi-Mourrain [13]	Putinar [168]	archimedean	quadratic module	SDP	O(ε -c )
	2021 Slot [191]		Schmüdgen [185]	ball, simplex	preordering SDP	O(ε -1/2 )
	2022	Mai-Magron (Proposition 6.1)	[132]	Putinar-Vasilescu [170]	general	quadratic module	SDP	O(ε -c )
	2022	Mai-Magron-Lasserre-Toh [135]	Dickinson-Povh [45]	contained in the nonnega-tive orthant	nonnegative module orthant, quadratic	LP, SDP	O(ε -c )

Table 1 . 3 :

 13 Complexity comparison (in terms of floating-point operations) of several methods for solving SDP.

	Method	Software	SDP type Convergence rate	The most expensive part per iteration
	IP [76] (second-order)	SDPT3 [196], Mosek [6]	arbitrary	O(log(1/ε)) [201]	system of linear equations solving with complexity O((s

max ) 6 ) [200, Table

  2.1) for c ∈ [p]. As before, we set k min := max{ f , g 1 , . . . , g m } and g 0 := 1. Fix a relaxation order k ≥ k min and let N nc k-gi be the standard monomial basis for i ∈ {0} ∪ J c , c ∈ [p]. Let G tsp k,c be the tsp graph with nodes N nc k associated with A c defined as in Section 3.1.3, i.e., its node set is N nc k and {β, γ} is an edge if β + γ ∈ A c ∪ (2N) nc . Note that we embed N nc into N n via the map α

Proposition 3.1. Fixing

  .2.5) a relaxation order k ≥ k min , the sequence (ρ

				(t) k ) t≥1 is monotonically
	non-decreasing and ρ	(t) k ≤ ρ k for all t.
	Proof. By construction, we have G (t) k,c,j ⊆ G (t+1) k,c,j for all k, c, j and all t. It follows that each maximal
	clique of G	(t) k,c,j is a subset of some maximal clique of G (t+1) k,c,j . Hence by Theorem 3.2, (3.2.5) with
	value ρ (t) k is a relaxation of (3.2.5) with value ρ	(t+1) k	and is clearly also a relaxation of (3.1.4) with
	value ρ k . Therefore, (ρ	(t) k ) t≥1 is monotonically non-decreasing and ρ	(t) k ≤ ρ k for all t.
	Proposition 3.2. Fixing a sparse order t ≥ 1, the sequence (ρ (t) k ) k≥kmin is monotonically non-
	decreasing.
	Proof. The conclusion follows if we can show that G (t) k,c,j ⊆ G (t) k+1,c,j for all k, c, j, t since by Theorem
	3.2 this implies that (3.2.5) is a relaxation of (3.2.5) with value ρ	(t) k+1 . Let us prove G (t) k,c,j ⊆ G (t) k+1,c,j
	by induction on t. For t = 1, from (3.1.5), we have

  + x 1 x 2 x 3 + x 3 x 4 x 5 + x 3 x 4 x 6 + x 3 x 5 x 6 + x 4 x 5 x 6

	Example 3.3. Let f = 1 +	6 i=1 x 4 i
					(3.2.6)
		. .	. . .	. . .	. . .
	The array of lower bounds (3.2.6) (and its associated SDP relaxations (3.2.5)) is what we call the
	CS-TSSOS hierarchy associated with (3.0.1) (3.0.1).

Lemma 3.1. Let

  + s 2 ) 2 ∈ S. The minimal set containing S with elements which are closed under addition modulo 2 is denoted by S Z2 . It is easy to proveS Z2 = {( i s i ) 2 | s i ∈ S} which is the subspace spanned by S in Z n 2 . S ⊆ Z n 2 .Then (S ⊥ ) ⊥ = S Z2 . Proof. It is immediate from the definitions.

	the
	orthogonal complement space of S in Z n 2 , denoted by S ⊥ , is the set {α ∈ Z n 2 | α s ≡ 0 (mod 2) , ∀s ∈
	S}.
	Remark 3.4. By definition, the set of sign symmetries of A is exactly the orthogonal complement
	space (α) ⊥ 2 in Z n 2 , which therefore can be essentially represented by a basis of the subspace (A) ⊥ 2 in Z n 2 .
	For a subset S ⊆ Z n 2 , we say that S is closed under addition modulo 2 if s 1 , s 2 ∈ S implies
	(s 1

Lemma 3.2. Suppose

  G is a graph with V (G) ⊆ N n . Then it holds (supp(G )) 2 ⊆ (supp(G)) 2 Z2 .Proof. By definition, we need to show (β + γ) 2 ∈ (supp(G)) 2 Z2 for any {β, γ} ∈ E(G ). Since in the process of chordal extensions, edges are added only if two nodes belong to the same connected component, for any {β, γ} ∈ E(G ) there is a path connecting β and γ in G: {β, υ 1 , . . . , υ r , γ} with {β, υ 1 }, {υ r , γ} ∈ E(G) and {υ

Table 3 .

 3

		1: Notation
	var	number of variables
	cons	number of constraints
	mc	maximal size of variable cliques
	mb	maximal size of SDP blocks
	opt	optimal value
	time	running time in seconds
	gap	optimality gap
	CE	type of chordal extensions used in (3.2.4)
	min	approximately smallest chordal extension
	max	maximal chordal extension
	0	a number whose absolute value less than 1e-5
	-	an out of memory error

Table 3 .2: The result for Broyden banded functions

 3

	(k = 3)

Table 3 .

 3 3: The result for the generalized Rosenbrock function (k = 2)

	var	mb	CSSOS opt	time mb	TSSOS opt	time mb	CS-TSSOS opt	time	SDSOS opt time
	40	231 38.051	126	41	38.049 0.61	21	38.049 0.23 37.625	115
	60	231 57.849	232	61	57.845 3.31	21	57.845 0.32	-	-
	80	231 77.647	306	81	77.641 11.7	21	77.641 0.41	-	-
	100	231 97.445	377	101 97.436 31.3	21	97.436 0.54	-	-
	120	231 117.24	408	121 117.23 75.4	21	117.23 0.60	-	-
	140	231 137.04	495	141 137.03	190	21	137.03 0.75	-	-
	160	231 156.84	570	161 156.82	367	21	156.82 0.90	-	-
	180	231 176.64	730	181 176.62	628	21	176.62 1.09	-	-
	200	231	-	-	201 196.41 1327	21	196.41 1.27	-	-
	300	231	-	-	-	-	-	21	295.39 2.26	-	-
	400	231	-	-	-	-	-	21	394.37 3.36	-	-
	500	231	-	-	-	-	-	21	493.35 4.65	-	-
	1000 231	-	-	-	-	-	21	988.24 15.8	-	-
		Table 3.4: The result for the Broyden tridiagonal function (k = 2)
	var	mb	CSSOS opt	time mb	TSSOS opt	time mb	CS-TSSOS opt	time	SDSOS opt time
	40	231 31.234	168	43	31.234 1.95	23	31.234 0.64 -5.8110	138
	60	231 47.434	273	63	47.434 8.33	23	47.434 1.14	-	-
	80	231 63.634	413	83	63.634 33.9	23	63.634 1.50	-	-
	100	231 79.834	519	103 79.834	104	23	79.834 1.96	-	-
	120	231 96.034	671	123 96.034	199	23	96.034 2.30	-	-
	140	231 112.23	872	143 112.23	490	23	112.23 2.94	-	-
	160	231 128.43 1002 163 128.43	783	23	128.43 3.67	-	-
	180	231 144.63 1066 183 144.63 1329	23	144.63 4.46	-	-
	200	231	-	-	-	-	-	23	160.83 4.88	-	-
	300	231	-	-	-	-	-	23	241.83 8.67	-	-
	400	231	-	-	-	-	-	23	322.83 13.3	-	-
	500	231	-	-	-	-	-	23	403.83 19.9	-	-
	1000 231	-	-	-	-	-	23	808.83 57.5	-	-
			Table 3.5: The result for the chained Wood function (k = 2)
	var	mb	CSSOS opt	time mb	TSSOS opt	time mb	CS-TSSOS opt	time	SDSOS opt time
	40	231 574.51	164	41	574.51 0.81	21	574.51 0.26 518.11	110
	60	231 878.26	254	61	878.26 3.61	21	878.26 0.40	-	-
	80	231 1182.0	393	81	1182.0 15.3	21	1182.0 0.57	-	-
	100	231 1485.8	505	101 1485.8 43.2	21	1485.8 0.73	-	-
	120	231 1789.5	516	121 1789.5 88.4	21	1789.5 0.93	-	-
	140	231 2093.3	606	141 2093.3	195	21	2093.3 1.16	-	-
	160	231 2397.0	700	161 2397.0	403	21	2397.0 1.39	-	-
	180	231 2700.8	797	181 2700.8	867	21	2700.8 1.54	-	-
	200	231	-	-	201 3004.5 1238	21	3004.5 1.91	-	-
	300	231	-	-	-	-	-	21	4523.6 3.39	-	-
	400	231	-	-	-	-	-	21	6042.0 5.72	-	-
	500	231	-	-	-	-	-	21	7560.7 7.88	-	-
	1000 231	-	-	-	-	-	21	15155		23.0	-	-

Table 3 . 6

 36

				: The result for Max-Cut instances
	instance nodes edges mc	Shor opt	mb	CSSOS opt	time mb	CS-TSSOS opt time
	g20	505	2045	14	570 120 488	51.2	92	488	19.6
	g40	1005	3441	14 1032 120 885	134	92	893	41.1
	g60	1505	4874	14 1439 120 1227 183	92 1247 71.3
	g80	2005	6035	15 1899 136 1638 167 106 1669 84.8
	g100	2505	7320	14 2398 120 2073 262	92 2128 112
	g120	3005	8431	14 2731 120 2358 221	79 2443 127
	g140	3505	9658	13 3115 105 2701 250	79 2812 153
	g160	4005 10677 14 3670 120 3202 294	79 3404 166
	g180	4505 12081 13 4054 105 3525 354	79 3666 246
	g200	5005 13240 13 4584 105 4003 374	79 4218 262

Table 3 .

 3

		7: The data for AC-OPF instances
	case name	var	cons	mc	AC	opt	Shor	gap
	3 lmbd api	12	28	6	1.1242e4 1.0417e4 7.34%
	5 pjm	20	55	6	1.7552e4 1.6634e4 5.22%
	24 ieee rts api	114	315	10 1.3495e5 1.3216e5 2.06%
	24 ieee rts sad	114	315	14 7.6943e4 7.3592e4 4.36%
	30 as api	72	297	8	4.9962e3 4.9256e3 1.41%
	73 ieee rts api	344	971	16 4.2263e5 4.1041e5 2.89%
	73 ieee rts sad	344	971	16 2.2775e5 2.2148e5 2.75%
	162 ieee dtc	348	1809	21 1.0808e5 1.0616e5 1.78%
	162 ieee dtc api 348	1809	21 1.2100e5 1.1928e5 1.42%
	240 pserc	766	3322	16 3.3297e6 3.2818e6 1.44%
	500 tamu api	1112 4613	20 4.2776e4 4.2286e4 1.14%
	500 tamu	1112 4613	30 7.2578e4 7.1034e4 2.12%
	793 goc	1780 7019	18 2.6020e5 2.5636e5 1.47%
	1888 rte	4356 18257 26 1.4025e6 1.3748e6 1.97%
	3022 goc	6698 29283 50 6.0143e5 5.9278e5 1.44%

Table 3 .

 3

	case name	mb	CSSOS opt time	gap	mb	opt	CS-TSSOS time	gap	CE
	3 lmbd api	28	1.1242e4 0.21 0.00%	22	1.1242e4 0.09 0.00% max
	5 pjm	28	1.7543e4 0.56 0.05%	22	1.7543e4 0.30 0.05% max
	24 ieee rts api	66	1.3442e5 5.59 0.39%	31	1.3396e5 2.01 0.73% max
	24 ieee rts sad	120	7.6943e4 94.9 0.00%	39	7.6942e4 14.8 0.00% max
	30 as api	45	4.9927e3 4.43 0.07%	22	4.9920e3 2.69 0.08% max
	73 ieee rts api	153	4.2246e5	758	0.04%	44	4.2072e5 96.0 0.45% max
	73 ieee rts sad	153	2.2775e5	504	0.00%	44	2.2766e5 71.5 0.04% max
	162 ieee dtc	253	-	-	-	34	1.0802e5	278	0.05% min
	162 ieee dtc api	253	-	-	-	34	1.2096e5	201	0.03% min
	240 pserc	153	3.3072e6	585	0.68%	44	3.3042e6 33.9 0.77% max
	500 tamu api	231	4.2413e4 3114 0.85%	39	4.2408e4 46.6 0.86% max
	500 tamu	496	-	-	-	31	7.2396e4	410	0.25% min
	793 goc	190	2.5938e5	563	0.31%	33	2.5932e5 66.1 0.34% max
	1888 rte	378	-	-	-	27	1.3953e6	934	0.51% min
	3022 goc	1326	-	-	-	76	5.9858e5 1886 0.47% max

8: The result for AC-OPF instances

  For every k ∈ N, according toLemma 4.4, if M k-1 ((Rx 22 ) y) = 0 and y 0 = 1, then one has trace(P n,k M k (y)P n,k ) = (R + 1) k , (4.2.12)where P n,k is defined as in (4.2.3) after replacing n by n. Thus SDP (4.2.10) has the CTP. We now do a similar process as in Section 4.2.1.Next, we present an iterative method, stated in Algorithm 4, to solve (4.1.1) with g = {Rx2 2 }, based on a nonsmooth optimization method such as LMBM. Output: increasing real sequence (τ k ) k∈N and x ∈ R n .

	Algorithm 4 SpectralPOP-CTP-WithSingleBallConstraint
	Input: POP (4.1.1) with g = {R -x 2 2 }, unknown optimal value f and optimal solutions;
	method (D) for solving SDP with CTP.
	3. Assume that one of the following two conditions holds:
	(a) Q(g, h) is Archimedean, the ideal I(h) is real radical, and the second-order sufficiency
	conditions (Definition 2.1) hold at every global minimizer of POP (4.1.1);
	(b) V (h) is finite.

yielding the result. Zero duality gap, primal attainability, and dual attainability for primal-dual (4.2.10)-(4.2.11) are guaranteed in the following proposition: Proposition 4.2. Let f be as in (4.1.1) with g = {R -x 2 2 }. Then: 1. Zero duality gap holds for primal-dual (4.2.10)-(4.2.11) for large enough k ∈ N. 2. SDP (4.2.10) has an optimal solution for large enough k ∈ N. Then SDP (4.2.11) has an optimal solution for large enough k ∈ N. In this case, τk = ρk = f . Proof. The first and second statement follow from Proposition 2.1, after replacing S(g, h) by V ( h). The third statement is due to Proposition 2.4 and Lemma 4.5. 1: for k ∈ N do 2:

4 :

 4 If x exists, set τk+j = τk , j ∈ N >0 , and terminate.Note that one can choose method (D) in Algorithm 4 as Algorithm 1 with LMBM solver or SketchyCGAL.

Following Proposition 2.3, Corollary 4.1 and Proposition 4.2, we obtain the following corollary: Corollary 4.5. (i) Sequence (τ k ) k∈N of Algorithm 4 is well defined and τk ↑ f as k → ∞. (ii) Assume that condition (a) or (b) of Proposition 4.2.3 holds. If there exists an optimal solution y of SDP (4.2.10) for some order k ∈ N such that the flat extension condition holds, x exists at the k-th iteration of Algorithm 4. In this case, if Algorithm 4 terminates at the k-th iteration, x is an optimal solution of POP (4.1.1) and f = τk .

  .2.18) where P n,k is defined as in (4.2.3) with n replaced by n. Thus SDP (4.2.16) has the CTP. It remains to follow a process which is similar to the one from Section 4.2.1.

Next, we present an iterative method, stated in Algorithm 5, to solve POP (4.1.1) with g = {R -x

2

2 }, based on nonsmooth optimization methods such as LMBM.

Algorithm 5 SpectralPOP-CTP-WithBallConstraint Input: POP (4.1.1) with g 1

Table 4 .

 4

	1: Notation

*

there exists at least one optimal solution of the POP, which can extracted by Henrion-Lasserre's algorithm in

[START_REF] Henrion | Detecting global optimality and extracting solutions in Glop-tiPoly[END_REF]

time the total computation time of the SDP relaxation in seconds the calculation did not finish in 3000 seconds or ran out of memory • SketchyCGAL is a MATLAB package to handle SDP problems with CTP/BTP, implemented by Yurtsever et al. [219]. We have implemented a Julia version (SketchyCGAL.jl) of SketchyCGAL to ensure fair comparison with LMBM.jl and SumOfSquare.jl. In this section, SketchyCGAL is used as a solver for SDP (4.2.6) in Algorithm 3 instead of Algorithm 1 or 2.

Table 4 . 2 :

 42 Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, 1) and k = 1.

				POP size		SumOfSquares (Mosek)	SpectralPOP (CTP) LMBM SketchyCGAL
				n		val	time	val	time	val	time
				50	-6.24844 *	0.3	-6.24844 *	0.7	-6.11351	0.7
				75	-7.25323 *	2	-7.25326 *	0.7	-6.95325	0.8
				100	-7.00957 *	8	-7.00957 *	0.9	-6.75991	1
				125	-9.76963 *	23	-9.76963 *	1	-9.39907	1
				150	-8.49449 *	64	-8.49449 *	1	-8.15382	2
				175	-10.7286 *	140	-10.72866 *	1	-10.1323	2
				200	-11.3521 *	300	-11.3521 *	2	-10.4724	3
				250	-13.8881 * 1152	-13.8881 *	4	-13.5571	5
				300	-13.9957 * 3708	-13.9958 *	6	-13.8327	12
				400		-	-	-15.7584 *	15	-15.5036	28
				500		-	-	-17.5838 *	35	-17.2513	65
				700		-	-	-22.3584 *	218	-22.0710	355
				900		-	-	-25.6117 *	621	-25.2435	947
				1200		-	-	-28.3170 *	1401 -27.8270 2074
				1500		-	-	-30.8475 *	7120 -30.2347 9020
		3,500
			SumOfSquares
			LMBM
			SketchyCGAL
	time	2,000
		1,000						2.5 • 10 -2
		500						1 • 10 -2
		0							0
		50	100	150	200	250	300		50	100	150	200	250	300
				n

  Efficiency and accuracy comparison forTable 4.2. in Table 4.2 for the case l = 1 and Table 4.3, 4.4 for the case l = n/4 . For these results, we use the Julia version of SketchyCGAL, which runs much faster than the MATLAB version without compromising accuracy.

	0.1
		SumOfSquares
	8 • 10 -2	LMBM SketchyCGAL
	gap
		n
	Figure 4.1:

Table 4 . 3 :

 43 Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, n/4 ) and k = 1. Efficiency and accuracy comparison for Table4.3.

			POP size		SumOfSquares (Mosek)	SpectralPOP (CTP) LMBM SketchyCGAL
			n	l		val	time	val	time	val	time
			50	14		-4.26516	0.4	-4.26516	1		-4.24511	1
			60	16	-6.42900 *	1	-6.42929 *	1		-6.36177	2
			70	19		-5.08320	3	-5.08322	2		-5.01911	3
			80	21		-5.35178	5	-5.35178	2		-5.29900	4
			100	26		-7.50097	15	-7.50097	10	-7.42432	11
			120	31		-5.89903	33	-5.89903	12	-5.81244	18
			150	39		-7.44920	127	-7.44921	26	-7.32154	36
			200	51		-8.93976	363	-8.93976	51	-8.79487	71
			300	76		-12.4295	3753	-12.4295	530	-12.2180	480
			400 101		-	-	-14.7190	2553 -14.4830 2318
		3,500							3	•10 -2
			SumOfSquares							SumOfSquares
			LMBM							LMBM
		2,500	SketchyCGAL					2		SketchyCGAL
	time	1,500						gap
									1
		500
		0							0
		50	100	150	200	250	300			50	100	150	200	250	300
				n								n
			Figure 4.2:

Table 4 . 4 :

 44 Numerical results for random dense equality constrained QCQPs on the unit sphere, described in Section 4.3.1, with (m, l) = (0, n/4 ) and k = 2. Efficiency and accuracy comparison for Table4.4. is more than 200 times larger than the matrix size, for almost all instance of Table 4.4. LMBM and SketchyCGAL still happen to be faster than SumOfSquares in both Table 4.3 and Table 4.4, but LMBM is not much more efficient than SketchyCGAL. The most expensive step performed by Mosek (used by SumOfSquares) is to solve a system of linear equations coming from certain complementarity conditions (see page 13 in

	POP size	SDP size	SumOfSquares (Mosek)	SpectralPOP (CTP) LMBM SketchyCGAL
	n	l	s	m	val	time	val	time	val	time
	5	3	21	169	-2.43822 *	0.04 -2.43823 *	1	-2.43982	1
	10	4	66	1475	-1.42006 *	0.4	-1.42013 *	1	-1.41268	1
	15	5	136	6121	-2.87129 *	7	-2.87142 *	2	-2.86744	6
	20	6	231	17557	-3.28734 *	73	-3.28736 *	5	-3.27733	26
	25	8	351	40834	-3.32902 *	592	-3.32918 *	13	-3.31634	65
	30	8	496	81345	-4.34398 * 4678 -4.34407 *	60	-4.32974	294
	35	10	666	146521	-	-	-4.77580 *	275	-4.75946	450
	40	11	861	244812	-	-	-2.95099	390	-2.91856 1225
	45	13	1081 386999	-	-	-3.95743	1588 -3.88533 2905
	50	14	1326 582115	-	-	-4.01846	6126	-	-

  .2 or n ≤ 35 in Table 4.4. Moreover, LMBM can provide an approximate optimal solution even for large-scale problems with n = 1500 in Table 4.2 (resp. n = 40 in Table 4.4) and in a case in Table 4.3. Unfortunately SketchyCGAL cannot do the extraction procedure successfully, because of its inaccurate output.

Table 4 . 5 :

 45 Storage comparisons for the rows n = 5, 10, 15, 20, 25 of Table 4.4.

		SumOfSquares	SpectralPOP (CTP)
		(Mosek)	LMBM	SketchyCGAL
	n	storage	storage	storage
	5	80.21 k	1.19 M	679.48 k
	10	803.00 k	1.02 M	881.75 k
	15	3.74 M	2.44 M	2.29 M
	20	12.06 M	6.51 M	6.46 M
	25	32.67 M	19.04 M	19.02 M

Table 4 . 6 :

 46 Evaluation comparisons for the rows n = 5, 10, 15, 20, 25 of Table 4.4.

				SpectralPOP (CTP)
			LMBM			SketchyCGAL
	n	#A #A	smax Neig #A #A	smax Neig
	5	18	19	21	19	191	2036	12	192
	10	26	27	66	27	108	1350	14	109
	15 404	405	136	405	205	3383	19	206
	20 469	470	231	470	328	6605	24	329
	25 336	337	351	337	292	6134	25	293

Table 4 . 7 :

 47 Numerical results of random dense QCQPs on the unit ball, described in Section 4.3.2, with (m, l) = (1, n/4 ), and k = 2.

	POP size	SDP size (CTP)	SumOfSquares (Mosek)	SpectralPOP CTP (LMBM) BTP (LMBM)
	n	l	s	m	val	time	val	time	val	time
	5	2	28	281	-0.33125 *	0.07 -0.33126 *	1	-0.98254	0.7
	10	3	78	2029	-2.30410 *	0.5	-2.30411 *	1	-3.64371	5
	15	4	153	7702	-2.26182 *	10	-2.26195 *	2	-4.21202	134
	20	5	253	21000	-2.24031 *	112	-2.24033 *	4	-5.77860	1722
	25	7	378	47251	-2.88952 * 1484 -2.88770 *	15	-6.81243 16185
	30	7	528	92049	-4.15791 * 4694 -4.15798 *	49	-	-
	35	9	703	163097	-	-	-4.10015	150	-	-
	40	10	903	269095	-	-	-4.47927	694	-	-
	45	12	1128 421121	-	-	-5.50988	849	-	-
	50	13	1378 628369	-	-	-5.52884	2086	-	-

Table 4

 4

.7, we consider POPs which involve a single inequality (ball) constraint. In this case, CTP (LMBM) is the most efficient and accurate solver. Numerical results emphasize that SumOfSquares and CTP (LMBM) behave in a similar way as in Table

4

.4. This indicates that converting a POP with a single inequality (ball) constraint to a CTP-POP by adding one slack variable, and solving the resulting SDP-CTP relaxation by means of spectral methods allows one to reduce the computing time while ensuring the same accuracy as

Table 4 . 8 :

 48 Numerical results of random dense QCQPs on the unit ball, described in Section 4.3.2, with (m, l) = ( n/8 , n/8 ), and k = 2.

		POP size		SDP size (CTP)	SumOfSquares (Mosek)	SpectralPOP CTP (LMBM) BTP (LMBM)
		n	m l	s	m	val	time	val		time	val	time
		10	2	2 105	3711	-3.17792	0.2	-3.18434	31	-4.71914	6
		15	2	2 190 11781 -2.14424	6	-2.16250	69	-4.45435	649
		20	3	3 190 11781 -2.92513	190	-3.09124	469	-81.3719 2573
		2,500								2
			SumOfSquares							SumOfSquares
		CTP (LMBM)							CTP (LMBM)
		BTP (LMBM)							BTP (LMBM)
	time	1,000							gap	1
		0								0
		10		15		20			10	15	20
				n							n

Table 4 . 9 :

 49 Subgradient norms computed during the last 10 iterations of CTP (LMBM) for the experiments from Table 4.7 and Table 4.8 with n = 10.

	Table 4.7 0.035 0.033 0.028 0.022 0.024 0.019 0.015 0.009 0.007 0.004
	Table 4.8 0.871 0.959 0.947 0.792 1.684 0.794 0.579 0.559 0.230 0.916

Table 4 . 10 :

 410 Numerical results for random dense quartics on the unit sphere, described in Section 4.3.3, with (m, l) = (0, 1) and k = 2.

	POP size	SDP size	SumOfSquares (Mosek)	SpectralPOP (CTP) LMBM SketchyCGAL
	n	s	m	val	time	val	time	val	time
	5	21	127	-2.92483 *	0.02 -2.92485 *	1	-2.84710	1
	10	66	1277	-3.59964 *	0.4	-3.59964 *	1	-3.48501	2
	15	136	5577	-4.18773 *	7	-4.18778 *	12	-4.03882	18
	20	231 16402 -3.92438 *	88	-3.92440 *	35	-3.67857	87
	25	351 38377 -6.36891 *	711	-6.36894 *	74	-5.93774	251

Table 4 .

 4 11: Numerical results for deciding the nonnegativity of even degree forms, described in Section 4.3.4, with k = d.

	POP size	SDP size	SumOfSquares (Mosek)	SpectralPOP (CTP) (LMBM)
	n	d	s	m	val	time	val	time
	5	2	21	127	-2.62353 *	0.01 -2.62353 *	1
	10	2	66	1277	-6.31389 *	0.4	-6.31390 *	1
	15	2	126	5577	-12.1405 *	6	-12.1405 *	2
	20	2	231	16402	-19.9981 *	76	-19.9981 *	2
	25	2	351	38377	-29.4812 *	633	-29.4812 *	4
	30	2	496	77377	-40.6934 * 3471 -40.6934 *	9
	35	2	666	140527	-	-	-54.2561 *	24
	40	2	861	236202	-	-	-69.4700 *	57
	45	2	1081 374027	-	-	-86.4113 *	127
	50	2	1326 564877	-	-	-105.532 *	250
	5	3	56	1261	-1.56744	0.1	-1.60032	23
	5	4	126	7177	-1.35267	1	-1.35315	235

Table 4 .

 4 12: Numerical results for deciding the convexity of even degree forms, described in Section 4.3.5, with k = d.

	POP size	SDP size	SumOfSquares (Mosek)	SpectralPOP (CTP) (LMBM)
	n	d	s	m	val	time	val	time
	5	2	66	1277	-3.87350 *	0.2	-3.87350 *	1
	7	2	120	4321	-4.44260 *	3	-4.44280 *	11
	10	2	231 16402 -4.98855 *	63	-4.98883 *	17
	12	2	325 32826 -4.47495 *	336	-4.49239 *	91
	5	3	286 34035 -3.89581 *	53	-3.89586 *	70

Table 4 .

 4 [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF] indicates that LMBM is twice faster than Mosek when n ≥ 20. In all cases, we can extract the solutions of the resulting POP and certify that A is not copositive since f is negative. 78CHAPTER 4. EXPLOITING THE CONSTANT TRACE PROPERTY: EQUALITY CONSTRAINTS

Table 4 . 13 :

 413 Numerical results for deciding the copositivity of real symmetric matrices, described in Section 4.3.6, with k = 2.

	POP size	SDP size	SumOfSquares (Mosek)	SpectralPOP (CTP) (LMBM)
	n	s	m	val	time	val	time
	10	66	1277	-0.89102 *	0.2	-0.89102 *	1
	15	136	5577	-0.91701 *	6	-0.91701 *	9
	20	231 16402 -0.98474 *	57	-0.98474 *	19
	25	351 38377 -0.97873 *	558	-0.97873

*

Chapter 5

  which in turn yields the desired conclusion.The next result is a consequence of Theorem 5.2. It states that if a POP has a ball constraint then the corresponding SOS strengthenings satisfy Slater's condition. Assume that R -x2 2 ∈ g for some R > 0. Then Slater's condition holds for SDP (2.4.4) for all k ≥ k min . As a consequence, strong duality holds for the primal-dual (2.4.4)-(2.4.5) for all k ≥ k min .Proof. It suffices to prove that SDP (2.4.4) has a strictly feasible solution for all k ≥ k min . Let k ≥ k min be fixed. By[START_REF] Marshall | Representations of non-negative polynomials, degree bounds and applications to optimization[END_REF] Proposition 5.8], there exist σ 0

	Corollary 5.1.

  Assumption 5.2. Let m ≥ 3 and {g i } i∈[m-2] be of equivalent degree. L > 0 and R > 0 are such that g m-1 = L -

	g i and g m = R -x 2 2 .
	i∈[m-2]
	Proposition 5.2. If Assumption 5.2 holds then LP (5.1.11) has a feasible solution for every
	k ∈ N ≥kmin , and therefore POP (5.0.2) has CTP.
	Proof. Let Assumption 5.2 hold with u

  Corollary 5.2. With the above notation, S(g ∪ {g m+1 , gm+2 }) = S(g) and LP(5.1.11) has a feasible solution when replacing g by g ∪ {g m+1 , gm+2 } for each k ∈ N ≥kmin . As a result, POP (5.0.2) is equivalent to the following POP Let g := {g i } i∈[m+2] . Then {g i } i∈[m] are of equivalent degree, i.e., there exists u ∈ N such that gi = u, for all i ∈ [m]. Thus Assumption 5.2 holds with g ← g, m ← m + 2. By Proposition 5.2, (5.1.11) has a feasible solution with g ← g for every order k ∈ N ≥kmin . It implies that for every k ∈ N ≥kmin , there exist u

	h)}	(5.1.21)
	which has CTP.
	Proof.

f := inf{f (x) : x ∈ S(g ∪ {g m+1 , gm+2 }) ∩ V (

Table 5 .

 5 2 and Table 5.3 illustrate how inaccurate LMBM can be for large problems (n ≥ 20), thus we do not report LMBM results in the other experiments. 88CHAPTER 5. EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS

Table 5 . 1 :

 51 Notation

	n	the number of variables of a POP
	m	the number of inequality constraints of a POP
	l	the number of equality constraints of a POP
	u max	the largest size of variable cliques of a sparse POP
	p	the number of variable cliques of a sparse POP
	k	the relaxation order of the Moment-SOS hierarchy
	t	the sparse order of the sparsity adapted Moment-SOS hierarchy (for TS and CS-TS)
	ω	the number of psd blocks of an SDP
	s max	the largest size of psd blocks of an SDP
	ζ	the number of affine equality constraints of an SDP
	a max	the largest constant trace
	Mosek	the SDP relaxation modeled by SumOfSquares (for dense POPs) or TSSOS (for sparse POPs) and solved by Mosek 9.1
	CGAL	the SDP relaxation modeled by our CTP-exploiting method and solved by the CGAL algorithm
	LMBM	the SDP relaxation modeled by our CTP-exploiting method and solved by the SM algorithm with the LMBM solver
	val	the optimal value of the SDP relaxation
	gap	the relative optimality gap w.

r.t. the value returned by Mosek, i.e., gap = |val -val(Mosek)|/|val(Mosek)| time the running time in seconds (including modeling and solving time) -the calculation runs out of space

Table 5 . 2 :

 52 Numerical results for minimizing a dense quadratic polynomial on a unit ball • POP size: m = 1, l = 0; Relaxation order: k = 2; SDP size: ω = 2, a max = 3.

	POP size	SDP size	Mosek	CGAL	LMBM
	n	s max	ζ	val	time	val	time	val	time
	10	66	1277	-2.2181	0.3	-2.2170	0.2	-2.2187	0.3
	20	231	16402	-3.7973	4	-3.7947	0.6	-3.7096	7
	30	496	77377	-3.6876 3474 -3.6858	104	-3.8530	59
	40	861	236202	-	-	-4.1718	33	-4.7730	179
	50	1326	564877	-	-	-6.3107	1007	-7.3874	139
	60	1891 1155402	-	-	-6.5326	1085	-7.4733	674
	70	2556 2119777	-	-	-7.3379	1262	-9.5223	1486
	80	3321 3590002	-	-	-7.9559	4988	-10.0260 1241
	90	4186 5718077	-	-	-7.3425	5187	-9.4477	5313
	100	5151 8676002	-	-	-7.7374 22451	-10.684	5355

Table 5 . 3 :

 53 Numerical results for randomly generated dense QCQPs with a ball constraint

	POP size	SDP size	Mosek	CGAL	LMBM
	n	l	s max	ζ	val	time	val	time	val	time
	10	3	66	1475	-2.0686	1.7	-2.0674	0.8	-2.0874	0.3
	20	5	231	17557	-3.0103	61	-3.0075	7	-3.0750	18
	30	8	496	81345	-3.3293 4573 -3.3249	80	-3.6863	123
	40	10	861	244812	-	-	-4.6977	194	-5.3488	488
	50	13	1326	582115	-	-	-4.2394	951	-6.1325	837
	60	15	1891 1183767	-	-	-5.7793 1387 -7.5718	3781
	70	18	2556 2165785	-	-	-6.1278 4335 -8.1181 15854

• POP size: m = 1, l = n/4 ; Relaxation order: k = 2; SDP size: ω = 2, a max = 3.

Table 5 . 4 :

 54 Numerical results for minimizing a dense quadratic polynomial on an annulus• POP size: m = 2, l = 0; Relaxation order: k = 2; SDP size: ω = 3, a max = 4.

	POP size	SDP size	Mosek	CGAL
	n	s max	ζ	val	time	val	time
	10	66	1343	-3.0295	0.5	-3.0278	1
	20	231	16633	-3.6468	69	-3.6458	5
	30	496	77873	-3.9108 2546 -3.9079	9
	40	861	237063	-	-	-4.7469	28
	50	1326	566203	-	-	-6.4170	112
	60	1891 1157293	-	-	-5.5841	226
	70	2556 2122333	-	-	-7.9325	730
	80	3321 3593323	-	-	-7.6164 1355
	90	4186 5722263	-	-	-8.1900 3563

Table 5 . 5 :

 55 Numerical results for randomly generated dense QCQPs with annulus constraints

	POP size	SDP size	Mosek	CGAL
	n	l	s max	ζ	val	time	val	time
	10	3	66	1541	-2.7950	0.5	-2.7934	2
	20	5	231	17788	-3.5048	95	-3.5027	10
	30	8	496	81841	-3.3964 4237 -3.3937	45
	40	10	861	245673	-	-	-4.6573	140
	50	13	1326	583441	-	-	-3.8236	437
	60	15	1891 1185658	-	-	-4.5246 1076
	70	18	2556 2168341	-	-	-6.2924 4783

• POP size: m = 2, l = n/4 ; Relaxation order: k = 2; SDP size: ω = 3, a max = 4.

Table 5 . 6

 56

	POP size	SDP size	Mosek	CGAL
	n	s max	ζ	val	time	val	time
	10	66	1871	-2.7197	0.5	-2.7189	1
	20	231	20791	-3.3560	98	-3.3501	57
	30	496	91761	-4.6372 5150 -4.6242	285
	40	861	269781	-	-	-4.5788	409
	50	1326	629851	-	-	-4.2313	2083
	60	1891 1266971	-	-	-4.0135	5525
	70	2556 2296141	-	-	-5.4019 15172

: Numerical results for minimizing a dense quadratic polynomial on a box

• POP size: m = n, l = 0; Relaxation order: k = 2; SDP size: ω = n + 1, a max = 3.

Table 5 . 7 :

 57 Numerical results for randomly generated dense QCQPs with box constraints• POP size: m = n, l = n/7 ; Relaxation order: k = 2; SDP size: ω = n + 1, a max = 3.

	POP size	SDP size	Mosek	CGAL
	n	l	s max	ζ	val	time	val	time
	10	2	66	2003	-1.8320	0.6	-1.8321	3
	20	3	231	21484	-3.1797	175	-3.1781	106
	30	5	496	94241	-2.2949 6850 -2.2982	528
	40	6	861	274947	-	-	-3.8651	933
	50	8	1326 640459	-	-	-3.6267 6159

Table 5 . 8 :

 58 Numerical results for minimizing a dense quadratic polynomials on a simplex• POP size: m = n + 2, l = 0; Relaxation order: k = 2; SDP size: ω = n + 3, a max = 5.

	POP size	SDP size	Mosek	CGAL
	n	s max	ζ	val	time	val	time
	10	66	2003	-1.9954	0.3	-1.9950	7
	20	231	21253	-1.5078	58	-1.5055	116
	30	496	92753	-2.0537 2804 -2.0480	377
	40	861	271503	-	-	-2.3034	950
	50	1326 632503	-	-	-1.8366 9539

Table 5 . 9 :

 59 Numerical results for randomly generated dense QCQPs with simplex constraints• POP size: m = n + 2, l = n/7 ; Relaxation order: k = 2; SDP size: ω = n + 3, a max = 5.

	POP size	SDP size	Mosek	CGAL
	n	l	s max	ζ	val	time	val	time
	10	2	66	2135	-1.0605	0.4	-1.0606	176
	20	3	231	21946	-1.6629	72	-1.6628	512
	30	5	496	95233	-1.0091 6206 -1.0249	1089
	40	6	861	276669	-	-	-0.3256	2314
	50	8	1326 643111	-	-	-1.4200 10035

Table 5 . 10 :

 510 Numerical comparison with ADMM (COSMO) on randomly generated dense QCQPs with a ball constraint• POP size: m = 1, l = n/4 ; Relaxation order: k = 2; SDP size: ω = 2, a max = 3.

	POP size	SDP size	Mosek	CGAL	COSMO
	n	l	s max	ζ	val	time	val	time	val	time
	10	3	66	1475	-2.3153	0.7	-2.3134	0.1	-2.3125	0.2
	20	5	231	17557	-3.6585	57	-3.6562	7	-3.6582	5
	30	8	496	81345	-4.6221 4670 -4.6177	69	-4.6230	91
	40	10	861	244812	-	-	-4.9932	173	-4.9989	532
	50	13	1326	582115	-	-	-5.0394	524	-5.0418 2468
	60	15	1891 1183767	-	-	-5.3537	735	-5.3548 6176

Table 5 .

 5 11: Numerical results for randomly generated dense POPs with a ball constraint

	POP size			SDP size		Mosek	CGAL
	n	l d k a max s max	ζ	val	time	val	time
	15 4 3	2 3	3 4	136 816	5581 288933	-3.0127 -	5 -	-3.0089 -3.0021	1 290
	10 3 4	2 3	3 4	66 286	1280 35443	-2.0327 -1.9337	0.3 41	-2.0194 -1.9310	0.6 16

• POP size: m = 1, l = n/4 ; SDP size: ω = 2.

  .3.3)It is shown in[START_REF] Lasserre | Convergent SDP-relaxations in polynomial optimization with sparsity[END_REF] Theorem 3.6] that convergence of the primal-dual (5.3.2)-(5.3.3) to f is guaranteed if there are additional ball constraints on each clique of variables.

  t∈N\{0} converges to τ k (the optimal value of SDP (2.4.5)) in finitely many steps.

	The dual of (5.3.19) reads

.3.19) One has τ ts k,t-1 ≤ τ ts k,t ≤ τ k ≤ f , for all (k, t). Moreover, we have the following theorem. Theorem 5.4. (Wang et al. [212, Theorem 5.1]) For each k ∈ N ≥kmin , the sequence (τ ts k,t )

Table 5 .

 5 12: Numerical results for minimizing a random quadratic polynomial with TS on the unit ball • POP size: m = 1, l = 0; Relaxation order: k = 2; Sparse order: t = 1; SDP size: ω = 4, a max = 3.

	POP size	SDP size	Mosek	CGAL
	n	s max	ζ	val	time	val	time
	10	56	937	-1.5681	4	-1.5527	0.7
	20	211	13722	-2.4275	36	-2.3996	1
	30	466	68357	-3.0748 1930 -3.0577	8
	40	821	214842	-	-	-3.6999	20
	50	1276	523177	-	-	-4.1603	128
	60	1831 1083362	-	-	-4.1914	655
	70	2486 2005397	-	-	-4.9578 1461
	80	3241 3419282	-	-	-5.6452 7253

.3.22)

Table 5 .

 5 [START_REF] Baldi | On moment approximation and the effective putinar's positivstellensatz[END_REF]: Numerical results for randomly generated QCQPs with TS and a ball constraint

	POP size	SDP size	Mosek		CGAL
	n	l	s max	ζ	val	time	val	time
	10	3	56	1105	-0.60612	0.7	-0.60550	2
	20	5	211	14777	-2.3115	47	-2.3097	17
	30	8	466	72085	-2.8344	3102	-2.8321	112
	40	10	821	223052	-	-	-3.4081	476
	50	13	1276	539765	-	-	-3.3552	1845
	60	15	1831 1110827	-	-	-3.5620	2992

• POP size: m = 1, l = n/4 ; Relaxation order: k = 2; Sparse order: t = 1; SDP size: ω = 4, a max = 3.

Table 5 . 14 :

 514 Numerical results for minimizing a random quadratic polynomial with TS on a box• POP size: m = n, l = 0; Relaxation order: k = 2; Sparse order: t = 1; SDP size: a max = 3.

	POP size		SDP size	Mosek	CGAL
	n	ω	s max	ζ	val	time	val	time
	10	22	56	1441	-1.0539	3	-1.0519	14
	20	42	211	17731	-1.3925	93	-1.3802	161
	30	62	466	81871	-2.2301 4392 -2.2128	567
	40	82	821	246861	-	-	-2.5209	1602
	50	102 1276	585701	-	-	-3.0282	2583
	60	122 1831 1191391	-	-	-3.0470 10858

Table 5 .

 5 15: Numerical results for randomly generated QCQPs with TS and box constraints

	• POP size: m = n, l = n/7 ; Relaxation order: k = 2; Sparse order: t = 1.; SDP size: ω = n + 1,
	a max = 3.
	POP size		SDP size	Mosek		CGAL
	n	l	ω	s max	ζ	val	time	val	time
	10	2	22	56	1553	-0.77189	0.2	-0.77214	9
	20	3	42	211	18364	-1.7962	71	-1.8009	150
	30	5	62	466	84201	-1.8529	5814	-1.8625	650
	40	6	82	821	251787	-	-	-2.1930	2994
	50	8	102 1276 595909	-	-	-2.4655	8397

Table 5 .

 5 16: Numerical results for minimizing a random quadratic polynomial with CS and ball constraints on each clique of variables • POP size: n = 1000, m = p, l = 0, u max = u + 1; Relaxation order: k = 2; SDP size: ω = 2p, a max = 3.

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	182	91	222712	-240.54	124	-240.37	98
	16	63	126	171	550692	-205.45 1389 -205.19	280
	21	48	96	276	1107682	-	-	-175.60	321
	26	39	78	406	1955879	-	-	-165.65	559
	31	33	66	561	3167072	-	-	-149.10	973
	36	28	56	741	4758727	-	-	-140.21 1315
	41	25	50	946	6839993	-	-	-126.55 1926

Table 5 .

 5

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	182	91	235023	-224.15	163	-224.09	204
	16	63	126	171	572905	-192.45 1830 -192.30	335
	21	48	96	276	1139460	-	-	-162.79	537
	26	39	78	406	2005124	-	-	-148.77 1014
	31	33	66	561	3239573	-	-	-142.38 2115
	36	28	56	741	4862292	-	-	-124.97 5304
									the uniform
	distribution;
	(b) setting h i,0

17: Numerical results for randomly generated QCQPs with CS and ball constraints on each clique of variables • POP size: n = 1000, m = p, l = 143, u max = u + 1; Relaxation order: k = 2; SDP size: ω = 2p, a max = 3. 5. Let r := l/p and W c := {(c -1)r + 1, . . . , cr}, if c ∈ [p -1] , {(p -1)r + 1, . . . , l}, if c = p . (5.3.24) For every c ∈ [p] and every i ∈ W c , generate a quadratic polynomial h i ∈ R[x(I c )] 2 by (a) for each α ∈ N Ic 2 \{0}, taking a random coefficient h i,α of h i in (-1, 1) w.r.t.

Table 5 .

 5 18: Numerical results for minimizing a random quadratic polynomial with CS and box constraints on each clique of variables • POP size: n = m = 1000, l = 0, u max = u+1; Relaxation order: k = 2; Constant trace: a max ∈[3, 4].

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	1181	91	313361	-204.89	443	-204.69	753
	16	63	1125	171	720323	-163.11 3082 -162.88 3059
	21	48	1095	276	1380918	-	-	-147.92 5655
	26	39	1077	406	2357161	-	-	-131.00 8889

2 j + 1/u, j ∈ [n]. The numerical results are displayed in Table 5.18 and 5.19. 100CHAPTER 5. EXPLOITING THE CONSTANT TRACE PROPERTY: INEQUALITY CONSTRAINTS

Table 5 .

 5 19: Numerical results for QCQPs with CS and box constraints on each clique of variables • POP size: n = m = 1000, l = 143, u max = u + 1; Relaxation order: k = 2; Constant trace: a max ∈ [3, 4].

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	1181	91	325672	-187.01	402	-186.98 1915
	16	63	1125	171	742536	-142.16 4323 -142.27 4126
	21	48	1095	276	1412696	-	-	-131.14 5334
	26	39	1077	406	2406406	-	-	-113.44 8037

Table 5 . 20 :

 520 Numerical results for minimizing a random quadratic polynomial with CS-TSSOS and ball constraints on each clique of variables• POP size: n = 1000, m = p, l = 0, u max = u + 1; Relaxation order: k = 2; Sparse order: t = 1; SDP size: a max = 3.

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	364	79	169654	-160.05	163	-160.01	498
	16	63	252	154	448354	-135.78 1422 -135.74	768
	21	48	192	254	939619	-	-	-117.17 1605
	26	39	156	379	1705763	-	-	-106.26 3150

Table 5 .

 5 21: Numerical results for QCQPs with CS-TSSOS and ball constraints on each clique of variables • POP size: n = 1000, m = p, l = 143, u max = u + 1; Relaxation order: k = 2; Sparse order: t = 1;SDP size: a max = 3.

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	364	79	180303	-155.91	158	-155.87	604
	16	63	252	154	468290	127.42	1707 -127.36 1053
	21	48	192	254	939619	-	-	-114.85 2877
	26	39	156	379	1751556	-	-	-102.30 6878

Table 5 .

 5 22: Numerical results for minimizing a random quadratic polynomial with CS-TSSOS and box

	constraints on each clique of variables
	• POP size: n = m = 1000, l = 0, u max = u + 1; Relaxation order: k = 2; Sparse order: t = 1;
	Constant trace: a max ∈ [3, 4].
	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	2362	79	248335	-126.15	151	-126.04	1982
	16	63	2250	154	601081	-100.75 2225 -100.64	7323
	21	48	2190	254	1191001	-	-	-87.804 10734
	26	39	2154	379	2080265	-	-	-81.908 20294

Table 5 .

 5 23: Numerical results for QCQPs with CS-TSSOS and box constraints on each clique of variables• POP size: n = m = 1000, l = 143, u max = u + 1; Relaxation order: k = 2; Sparse order: t = 1;Constant trace: a max ∈[3, 4].

	POP size		SDP size	Mosek	CGAL
	u	p	ω	s max	ζ	val	time	val	time
	11	91	2362	79	258984	-114.53	325	-114.27	482
	16	63	2250	154	621017	-96.199 4450 -96.079	1245
	21	48	2190	254	1220027	-	-	-83.013	8204
	26	39	2154	379	2126058	-	-	-74.532 27600

  [START_REF] Yurtsever | Scalable semidefinite programming[END_REF], Yurtsever et al. stated Algorithm 8 (see below) to solve SDP(5.3.25) with CTP. This procedure is based on the augmented Lagrangian paradigm combined together with the conditional gradient method.The convergence of the sequence (X t ) t∈N in Algorithm 8 to the set of optimal solutions of SDP (5.3.25) is guaranteed as follows:

	Theorem 5.6. [219, Fact 3.1 ] Consider SDP (5.3.25) such that Assumption 5.3 holds. Let
	(X

t ) t∈N be in the output of Algorithm 8. Then X t 0, for all t ∈ N and

AX t -b 2 → 0, | C, X t -τ | → 0 as t → ∞,

with the rate of order O(1/ √ t). Remark 5.4. In order to achieve the best convergence rate for Algorithm 8, we scale the problem's input as follows:

  in the following theorem:

	Theorem 5.7. Consider SDP (5.3.28) such that Assumption 5.4 holds. Let ((X	(t) c ) c∈[p] ) t∈N be the
	output of Algorithm 9. Then X	(t) c	0, for all c ∈ [p] and for all t ∈ N and	c∈[p] A c X (t) c -b

2 → 0 and c∈[p] C c , X (t) c -τ → 0 as t → ∞ with the rate O(1/ √ t).

Remark 5.7. Before running Algorithm 9, we scale the problem's input as follows:

Table 6 . 1 :

 61 Decrease of the moment matrix rank in Algorithm 6.2.3.

	t at	η t 0	rank(M1(y)) rank(M0(y))	x
	0 0 R 4 2.0000 5	1	-
	1 e1	1.0000 3	1	-
	2 e2	2.9997 3	1	-
	3 e3	1.9998 1	1	(0.9999, 0.0001, 0.5028, -0.8611)
	4 e4	1.3089 1	1	(1.0000, 0.0002, 0.4968, 0.8329)

  We can apply the technique used in the proof of Corollary 7.2, which consists of replacing the perturbation εθ d f by ε, to represent polynomials nonnegative on R n . Let us consider an arbitrary large positive constant L and a polynomial f of degree 2d f which is nonnegative on R n . Then, thanks to[START_REF] Mai | Positivity certificates and polynomial optimization on non-compact semialgebraic sets[END_REF] Theorem 3.2], for any

.

1.17)

which yields the desired result. Remark 7.2. In Corollary 7.2, if m = 1 and L > 0, we still obtain c = 65. Indeed, up to a scaling, one can always assume L = 1. Remark 7.3.

  .1.55) 140CHAPTER 7. ON THE COMPLEXITY OF PUTINAR-VASILESCU'S POSITIVSTELLENSATZ Thus y ∈ {g m ≤ -δm 2 }. This in turn implies

  .2.6) 3. Strong duality holds for all orders k of the primal-dual problems (7.2.3)-(7.2.4). The proof of Theorem 7.2 is exactly the same as the proof of [130, Theorem 7]. The second statement relies on Corollary 7.1. The third statement is due to the Slater condition [201, Theorem 3.1] since S(g) has nonempty interior (see in detail [130, Proposition 2 and Remark 3]).

Table 8 . 1 :

 81 Comparing the computational complexity of the sparse and dense hierarchies.

	SDP of order k	sparse hierarchy	dense hierarchy
	number of variables	O(τ 2k )	O(n 2k )
	largest size of SDP matrix	O(τ k )	O(n k )
	Theorem 8.1. (Lasserre, Waki et al.) Let Assumption 8.1 hold. If a polynomial f is positive on
	S(g), then there exist σ

  simple proof has a less constructive flavor. It is due to the fact that we need to know the power of the linear form v, x 2k to construct the convex hull of such Dirac functions in order to apply Carathéodory's theorem. If the shape of the uniform denominators involved in Reznick's result is not available, finding such convex hull is hard and unnatural. To prove our sparse version of Reznick's result via the application of Carathéodory's theorem, we would need to construct the Dirac functions of the form σ c /H k c . However these Dirac unctions are complicated because each denominator H k c depends on the RIP. In our proof, we obtain the general form of H c via inductions. In case that such denominators H k c are known in advance, it might be possible to prove our result in a simpler way.

  The proof of Theorem 8.4 can be found in[START_REF] Mai | A sparse version of Reznick's Positivstellensatz[END_REF] Section 4].

		.2.2)
	Theorem 8.4. The following statements hold:
	1. The sequence (ρ sparse k	(ε)) k∈N is monotone non-decreasing.

2. Assume that problem (8.2.1) has an optimal solution x . Then there exists

K ∈ N such that f ≤ ρ sparse k (ε) ≤ f + ε ψ d (x ) for all k ≥ K.

Table 8 . 2 :

 82 Numerical results obtained when solving the SDP relaxations (8.2.2), (8.2.4) associated to the minimization of quadratic forms on the nonnegative orthant with ε = 10 -5 and k = 1.For n ≤ 10, the optimal value obtained at the first order SDP relaxations (8.2.2) and (8.2.4) is very close to the exact optimal value of POP (8.2.1). Besides the solver runs out of memory for n ≥ 12 when we compute ρ sparse k (ε) as the number of equality constraints in SDP (8.2.2) is already very large even for k = 1. This large number of constraints arises while computing the common denominator within the sparse representation.Explicitly, it is the number of terms of the polynomial φ k (f -λ + εψ d ), according to (8.2.2). This number is upper bounded by

	POP size		SDP size	ρ sparse k (JuMP+Mosek) (ε)		ρ sample k,N (JuMP+Mosek) (ε)
	n	u nummat numcons	val	time	N	val	time
	5	2	10	432	3.0 × 10 -5	3	100 3.0 × 10 -5	3
	7	2	14	2368	6.0 × 10 -5	3	140 6.0 × 10 -5	4
	10 3	20	22528	9.0 × 10 -5	109	200 9.0 × 10 -5	17
	12 4	22	229520	-	-	240 1.0 × 10 -4	65
	15 5	29	1157120	-	-	300 1.3 × 10 -4 2601
	n+2d+k deg(φ)
	n

  Denote d f := deg(f )/2 +1. With an additional variable x n+1 , we first define the following homogeneous polynomials:

	9.2. POLYNOMIAL OPTIMIZATION ON THE NONNEGATIVE ORTHANT: COMPACT CASE163
	Proof. f

.

1.16)

2. If S has nonempty interior, then there exist positive constants c and c depending on f, g i such that for all ε > 0, one can take K ε = cε -c .

  .1.18) for some SOS of monomials σ i . By replacing x n+1 by R -x2 2 , we obtain the results. -d gi +1) which becomes larger when k increases, while the number of SOS of monomials in the representation (9.1.6) is m + 1, thus does not depend on k. However, a large number of Gram matrices is not a computational issue, since the complexity of interior-point methods mainly depend on the maximal block sizes of the Gram matrices and are still efficient when their number is large. With f being defined as in Remark 9.2, the following decomposition is an instance of the extended Handelman's Positivstellensatz:

	Remark 9.4. The number of SOS of monomials in the representation (9.1.16) is i=1 (k Remark 9.5. m

  .2.20) yielding f = 3. We solve SDP(9.2.18) with Mosek and report the corresponding numerical results in Table9.1. The table displays τ Pól 2,4 = 2.9999 which is very close to f . However, τ Pól 17 = τ Pól 17,1 = 1.5030 is smaller than τ Pól 16 = τ Pól 16,1 = 2.4000, which violates the theoretical inequality (9.2.7). The underlying reason is that the matrix A used to define the convex polytope P = {x ∈ R n : x ≥ 0 , Ax ≤ b} in the equivalent form min x∈P c x of LP (9.2.5) is ill-conditioned, and the solver is not able to solve accurately the LP corresponding to τ Pól 17 .

Table 9 .

 9 Remark 9.13. A naive idea is to define the new sequence of moments u = (u α ) α∈N n 2(d f +k) given by u α := y 2 α , for α ∈ N n 2(d f +k) . Obviously, if y has a representing Dirac measure δ z , then u has a representing Dirac measure δ z 2 . In this case, we take x := z 2 . However, there is no guarantee that u has a representing measure in general even if y has one. Algorithm 14 Extraction algorithm for POPs on the nonnegative orthant Input: precision parameter ε > 0 and an optimal solution (λ, G ij ) of SDP(9.2.19). For j ∈ [b(n, k m )], let Ḡj = (w

	1: Numerical values (in the first subtable) and computing time (in the second subtable)
	for τ Pól k,s in Example 9.2
	k	s	1	2		3		4	5		6	7	8
	0	0.0000 0.0000 0.0000		0.0000	0.0000		0.0000	0.0000	0.0000
	1	0.0000 0.0000 0.0000		0.0000	0.0000		0.0000	0.0000	0.0000
	2	0.0000 0.0000 0.4999 2.9999 2.9999 2.9999 2.9999 2.9999
	3	1.0000 0.9999 0.9999		2.7454	2.8368		2.8383	2.9999 2.9999
	4	1.4399 1.4999 1.4999		1.4999	1.4999		1.4999	2.9999 2.9999
	5	1.8615 1.9961 1.9999		1.9999	1.9999		1.9999	2.9999 2.9999
	6	2.1999 2.4526 2.4998		2.4999	2.4999		2.4999	2.4999	2.4999
	7	2.3971 2.8090 2.9633		2.9950	2.9996		2.9999 2.9999 2.9999
	8	2.4109 2.9022 2.9989 2.9999 2.9999 2.9999 2.9999 2.9999
	9	2.5161 2.9137 2.9997 2.9999 2.9999 2.9999 2.9999 2.9999
	10 2.5896 2.9520 2.9993 2.9999 2.9999 2.9999 2.9999 2.9999
	11 2.6210 2.9607 2.9983 2.9999 2.9999 2.9999 2.9999 2.9999
	12 2.6937 2.9615 2.9973		2.9998	2.9999 2.9999 2.9999 2.9999
	13 2.7330 2.9662 2.9977 2.9999 2.9999 2.9999 2.9999 2.9999
	14 2.7390 2.9687 2.9974 2.9999 2.9999 2.9999 2.9999 2.9999
	15 2.3704 2.9697 2.9972		2.9998	2.9999 2.9999 2.9999 2.9999
	16 2.4000 2.9710 2.9971		2.9997	2.9999 2.9999 2.9999 2.9999
	17 1.5030 2.9723 2.9968 2.9999 2.9999 2.9999 2.9999 2.9999
	18 0.5833 2.9732 2.9966		2.9996	2.9999 2.9999 2.9999 2.9999
	19 0.8121 0.0000 0.0000		2.9995	2.9999 2.9999 2.9999 2.9999
	20 0.7457 0.0000 0.0000		2.9994	2.9999 2.9999 2.9999 2.9999
			k	s	1	2	3	4	5	6	7	8
			0	1.1 1.3 1.0	1.0	1.0	1.1	1.0	1.0
			1	1.1 1.1 1.1	1.2	1.1	1.1	1.1	1.1
			2	1.1 1.1 1.1	1.1	1.1	1.1	1.1	1.1
			3	1.1 1.1 1.1	1.1	1.1	1.1	1.5	1.1
			4	1.1 1.2 1.1	1.1	1.1	1.1	1.2	1.2
			5	1.1 1.1 1.1	1.2	1.2	1.3	1.2	1.3
			6	1.2 1.2 1.2	1.3	1.3	1.5	1.3	1.2
			7	1.4 1.2 1.2	1.4	1.4	1.6	1.6	1.4
			8	1.2 1.2 1.3	1.3	1.5	1.4	1.8	1.9
			9	1.3 1.2 1.3	1.4	1.5	1.7	1.7	1.6
					1.3 1.3 1.5	1.8	1.7	1.9	2.2	1.9
					1.3 1.5 1.4	1.9	1.9	2.0	2.2	2.3
					1.3 1.7 1.8	2.1	2.2	2.3	2.7	2.6
					1.4 1.6 1.9	2.2	2.3	2.4	2.9	3.4
					1.2 1.5 2.0	2.5	2.6	2.9	3.5	3.8
					1.2 1.6 2.3	2.8	3.1	3.5	4.2	5.0
					1.3 2.5 2.8	3.5	3.9	4.4	5.9	7.1
					1.4 2.3 3.8	5.3	6.2	7.2	7.9	9.7
					1.6 2.9 5.2	7.2	7.9	9.7	10.6 12.3
					1.5 2.7 4.1	9.8	13.3 14.0 14.1 16.6
					1.4 3.4 4.8 12.6 16.5 20.8 24.5 27.2

1:

Table 9 .

 9

		2: The notation
	Pb	the ordinal number of a POP instance
	Id	the ordinal number of an SDP instance
	n	the number of nonnegative variables in POP (9.0.1)
	mineq	the number of inequality constraints of the form gi ≥ 0 in POP (9.0.1)
	meq	the number of equality constraints of the form gi = 0 in POP (9.0.1)
	Put	the SDP relaxation based on Putinar's Positivstellensatz (9.3.1) modeled by TSSOS and solved by Mosek 9.1
	Pól	the SDP relaxation based on the extension of Pólya's Positivstellensatz (9.2.19) modeled by our software InterRelax and solved by Mosek 9.1
		the SDP relaxation based on the extension of Handelman's Positivstellen-
	Han	satz (9.2.22) modeled by our software InterRelax and solved by Mosek
		9.1
	spPut	the SDP relaxation for a sparse POP based on Putinar's Positivstellensatz (9.3.2) modeled by TSSOS and solved by Mosek 9.1
		the SDP relaxation for a sparse POP based on the extension of Pólya's
	spPól	Positivstellensatz (9.4.81) modeled by our software InterRelax and
		solved by Mosek 9.1
		the SDP relaxation for a sparse POP based on the extension of Han-
	spHan	delman's Positivstellensatz (9.4.83) modeled by our software InterRelax
		and solved by Mosek 9.1
	k	the relaxation order
	s	the factor width upper bound used in SDP (9.0.7) and SDP (9.4.81)
	d	the sparsity order of the SDP relaxation (9.4.81)
	nmat	the number of matrix variables of an SDP
	msize	the largest size of matrix variables of an SDP
	nscal	the number of scalar variables of an SDP
	naff	the number of affine constraints of an SDP
	val	the value returned by the SDP relaxation
	*	there exists at least one optimal solution of the POP, which can be ex-
		tracted by Algorithm 14 or 9.4.6
	time	the running time in seconds (including modeling and solving time)
	∞	the SDP relaxation is unbounded or infeasible
	-	the calculation runs out of space

Table 9 . 3 :

 93 Numerical results for randomly generated dense QCQPs.

	Id Pb	n	POP size mineq meq k	Put val	time	k	s		Pól val	time k	s	Han val	time
	1 2	1	20	2	0	1 2 -1.99792 ∞	0.0 92	0 17	-1.99792 *	1	2	5	-1.99792	1
	3 4	2	20	5	0	1 2 -0.350601 ∞	0.03 342	1 20 -0.265883 *	9	3	1	-0.265883	1
	5 6	3	20	5	4	1 2 -0.431543 ∞	0.02 356	1	7	-0.429442	5	3	7	-0.429430	9
	7 8	4	30	2	0	1 2 -2.31695 ∞	0.0 3545	0 20	-2.31695 *	2	2 10	-2.31695	1
	9 10	5	30	7	0	1 2	∞ -2.13423	0.2 15135	0 31	-1.79295	45	3 20	-1.79295	238
	11 12	6	30	7	6	1 2 -1.56374 12480 ∞ 0.1	1 31	-1.56374	54	3 15	-1.56374	236
	Id	Put nmat msize nscal	naff	Pól nmat msize nscal		naff	Han nmat msize nscal		naff
	1 2	1 22		21 231	22 1	231 10626	5	17	232		231	17	5	255		231
	3 4	1 25		21 231	25 1	231 10626	44	20	1604 1771	0	1	2344 1771
	5 6	1 25		21 231	29 925	231 10626	330	7	1688 1771	345	7	1945 1771
	7 8	1 32		31 496	32 1	496 46376	11	21	497		496	22	10	530		496
	9 10	1 37		31 496	37 1	496 46376	32	31	5116 5456	396	20	5650 5456
	11 12	1 37		31 496	43 2977 46376 496	32	31	5302 5456	561	15	5836 5456

Table 9 . 4 :

 94 Numerical results for randomly generated QCQPs with correlative sparsity of n = 1000 and

	d = deg(f ) = 2.
	Id Pb	u	POP size mineq meq k	spPut val	time	k	s	spPól val	time k	s	spHan val	time
	1 2	1	10	201	0	1 2 -129.061 ∞	1.5 385	0 10	-128.906	15	2	7	-128.660	20
	3 4	2	10	201	200	1 2 -66.0696 ∞	2.0 475	1 12	-65.3195	51	3 10 -65.3050	283
	5 6	3	20	201	0	1 2 -66.1306 56360 ∞ 3.6	0 15	-65.9794	19	2 15 -65.8646	24
	7 8	4	20	201	200	1 2	∞ -	9 -	1 22	-38.2061	319	3 20 -38.2035 2146
	Id	spPut nmat msize nscal	naff	nmat msize	spPól nscal	naff	spHan nmat msize nscal	naff
	1 2	100 1300		12 78	1201 1	7491 135641	299	10		7889	7491	599	7		9288	7491
	3 4	100 1300		12 78	1401 15577 135641 7491	1299	12		39920	43813	4184	10		41419	35926
	5 6	50 1250		22 253	1201 1	12481 630231	399	15		25407	25109	399	15		13978	12481
	7 8	50 1250		22 253	1401 50513 630231 12481	1149	22		108641 113428	3574	20		109990 100751

Table 9 . 7 :

 97 Numerical results for some instances of MAXCUT problems.

	Id	Pb	POP size n	k	Put val	time k	s	Pól val		time k	s	Han val	time
	1 2	burma14		14	1 2 30301.999 30310.915	0.2 4	1 16	30302.000	1	3 16 30301.999	1
	3 4	gr17		17	1 2 24985.999 25089.044	0.2 24	1 19	24986.000	1	3 19 24985.999	2
	5 6	fri26		26	1 2 22217.999 1970 22220.657 0.4	1 28	22218.000	12	3 28 22217.999	28
	7 8	att48		48	1 799281.420 2 -	1 -	1 50 798857.049 1129 3 50 798890.722 3600
	Id	nmat msize	Put nscal	naff	nmat msize	Pól nscal	naff	Han nmat msize nscal	naff
	1 2	1 15	15	29 1681	130 3060	15	15		666	680	16	15		787	680
	3 4	1 18	18	35 2908	171 5985	18	18		1123	1140	19	18		1295	1140
	5 6	1 27	27	53 9829	378 27405	27	27		3628	3654	28	27		4007	3654
	7 8	1 30	49	97 13486 40920 1225	49	49	20777 20825	50	49		22003 20825

Table 9 .

 9 8: Information for the training model(9.3.11).

	Dataset	BHPD
	Number of hidden layers	L = 2
	Length of an input	13
	Number of inputs	506
	Test size	20%
	Number of classes	k = 3
	Numbers of units in layers	m = (13, 20, 10)
	Number of weights	490
	Opimization method	Adadelta algorithm 2
	Accuracy	70%
	Batch size	128
	Epochs	200

Table 9 . 9 :

 99 Numerical results for robustness certification on BHPD, n = 43, mineq = 43, meq = 30 and

	d = deg(f ) = 2.
		Id	Pb	k	spPut val	time k	s	spPól val	time k	s	spHan val time
		1 2	y = 1	1 88.1571 2 -	0.4 -	1 35 -11.8706	625	3 35	-	1364
		3 4	y = 2	1 208.934 2 -	0.4 -	1 35 -13.3240	518	3 35	-	1270
	Id	spPut nmat msize nscal	naff	spPól nmat msize nscal	naff	spHan nmat msize nscal	naff
	1,3 2,4	23 97	22 595	117 14431 86285 737	297	35	46233 28195	308	35	47629 9670

  Section 4.2], POP(9.3.10) has correlative sparsity. To use our method, we convert (9.3.10) to a POP on the nonnegative orthant by defining new nonnegative variables zt := x 0 t -xt + ε. Here we choose ε = 0.1. More detailed information for our training model are available in Table9.8.The numerical results are displayed in Table9.9. Compared to spPut, spPól and spHan provide better upper bounds in less total time. Moreover, in Table9.9, the values returned by spPut with k = 1 are positive and are much when apply Pólya's Positivstellensatz. To obtain (9.1.3), we replace x by x 2 in the representation of f + ε( n j=1 x j ) d f . We shall prove the second statement. Assume that S has nonempty interior. Set m := m + n and g m+j := x 2 j with d gm+j := 1, j ∈ [n]. Then gm+j := x j , j ∈ [n], and

Discussion:

  Then F is a homogeneous polynomial of degree D andF = f + 2ε -i∈[ m] (q 2 i + ε

					mCg i	)g i ≥	ε ( m+1)2 m	on ∆ n ,	(9.4.23)
	Moreover,
	F ≤	j∈[n] x j	D-d f ( f + 2ε	j∈[n] x j	d f )
	≤	+ i∈[ m] gi p i f + 2ε + i∈[ m] T pi gi =: T F , j∈[n] x j D-4nui-dg i	(9.4.24)
						.4.20)
	and
	p i ≤ q i	2 + ε mCg i	≤ T 2 qi + ε mCg i	=: T pi .	(9.4.21)

Set D := max{d f , 4nu i + d gi : i ∈ [ m]} and F := ( j∈[n] x j ) D-d f ( f + 2ε( j∈[n] x j ) d f ) -i∈[ m] gi p i ( j∈[n] x j ) D-4nui-dg i . (

9

.4.22)

  Remark 9.16. In view of Propositions 9.2 and 9.3, replacing the standard monomial basis v t by the new basis vt can provide a Positivstellensatz involving Gram matrix of factor width 1. Thus, ones can build up hierarchies of semidefinite relaxations with any maximal matrix size, based on both representations (9.4.38) and (9.4.33). However, expressing the entries of the basis vt is a time-consuming task within the modeling process. A potential workaround is to impose(9.4.38) and (9.4.33) on a set of generic points similarly to[START_REF] Lasserre | A bounded degree SOS hierarchy for polynomial optimization[END_REF] Section 2.3]. This needs further study.

Table 9 .

 9 13: Numerical results for deciding the nonnegativity of an even degree form on the nonegative orthant, with d = 2.

	Id Pb	POP size n	k	Put val	time	k	s	Pól val	time
	1 2	1	5		2 3 -0.68020 -1.87958	0.001 0 0.06	8	-0.68020 *	1
	3 4	2	10		2 3 -0.87524 -1.87491	0.1 10	0 11 -0.87524 *	5
	5 6	3	15		2 3 -0.86938 -2.01566	6 7675	0 44 -0.86938 *	79
	Id	Put nmat msize nscal	naff	Pól nmat msize nscal	naff
	1 2		6 6	21 56	22 232	126 462	31		6	72	126
	3 4		11 11	66 268	67 2212	1001 8008	111		11	617	1001
	5 6		16 16	136 816	137 9317 54264 3876	213		44	2637 3876

Table 9 .

 9 14: Numerical results for minimizing polynomials over the boolean hypercube, with d = 1.

	Id Pb	POP size n	k	Put val	time	k	s	Pól val	time
	1 2	1	10		1 2 -4.34345 -4.61386	0.008 1 11 -4.34345 0.2	1
	3 4	1	20		1 2 -14.9455 -15.4584	0.02 108	1 21 -14.9455	4
	5 6	3	30		1 2 -27.6311 -29.3433	0.1 8068	1 31 -27.6311	41
	Id		nmat msize	Put nscal	naff	Pól nmat msize nscal	naff
	1 2		1 6	11 56	21 232	66 462	11		11	276	286
	3 4		1 21	21 231	41 4621	231 10626	21		21	1751 1771
	5 6		1 31	31 496	61 14881 46376 496	31		31	5426 5456

Though we only include inequality constraints in the definition of S(g) (3.0.2) for the sake of simplicity, equality constraints can be treated in a similar way.

A smallest chordal extension is a chordal extension with the smallest clique number. Computing a smallest chordal extension is generally NP-complete. So in practice we compute approximately smallest chordal extensions instead with efficient heuristic algorithms.

The instances are available at https://wangjie212.github.io/jiewang/code.html.

The expressions involving angles of complex variables can be converted to polynomials by using tan(∠z) = y/x for z = x + iy ∈ C.

the vector v i is updated in Step 6 of[START_REF] Yurtsever | Scalable semidefinite programming[END_REF] Algorithm 4.2]

https://github.com/power-grid-lib/pglib-opf

Tien-Son Pham, Personal communication,

If there are fc in the sum f such that deg(fc) > deg(f ), we can always remove the high degree redundant term in fc which cancel with each other to make degree of fc at most deg(f ).

https://keras.io/api/models/model_training_apis/

CONTENTS

 IMO 1990] 3 0 {(±1, ±1), (0, 0)} 1 0.0000 (-0.0000, -0.0000) Y: 3.26 J: 0.22 4 0 {(1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (0, 1, 1), (1, 0, 1)} 1 0.0000 (0.00, 0.00, 0.

Contribution. Our contribution is twofold: I. In our first contribution, we provide in Corollary 9.2 a degree bound for the extension of Pólya's Positivstellensatz originally stated in [START_REF] Dickinson | On an extension of Pólya's Positivstellensatz[END_REF]. Explicitly, if f , ǧ1 , . . . , ǧm are polynomials even in each variable, -Š defined as in (9.0.4) has nonempty interior, ǧ1 = R -x 2 2 for some R > 0, -f is of degree at most 2d f , each ǧi is of degree at most 2d gi , and f -f is nonnegative on Š, then there exist positive constants c and c depending on f , ǧi such that for all ε > 0, for all

for some σ i being SOS of monomials such that deg(σ i ǧi ) ≤ 2(k + d f ). (Here ǧm := 1.)

Consequently, the resulting LP-hierarchy of lower bounds (ρ Pól k ) k∈N for POP (9.0.3):

where

. converges to f with a rate at least O(ε -c ). This linear hierarchy is originally stated by Dickinson and Povh in [START_REF] Dickinson | A new approximation hierarchy for polynomial conic optimization[END_REF] without convergence rate.

Unfortunately, for large relaxation order k this LP is potentially ill-conditioned (see for instance Example 9.2). In order to address this issue, we replace each diagonal Gram matrix diag(u j ) in LP (9.0.6) by a Gram matrix of factor width at most s ∈ N >0 to obtain a semidefinite relaxation, which is tighter than LP (9.0.6). Namely, consider the following SDP indexed by k ∈ N and s ∈ N >0 : is at most s. Here ǧm := 1. We call s the factor width upper bound associated with the semidefinite relaxation (9.0.7). It is easy to see that the size of each Gram matrix G ij in (9.0.7) is at most s. In addition, due to (9.0.8), we obtain the following estimate for every s ∈ [b(n, k)]:

.0.9) so that for every fixed s ∈ N >0 , τ Pól k,s → f as k increases, with a rate at least O(ε -c ). Notice that when s = 2, (9.0.7) becomes an SOCP thanks to [START_REF] Magron | SONC Optimization and Exact Nonnegativity Certificates via Second-Order Cone Programming[END_REF]Lemma 15].

We emphasize that in our semidefinite relaxation (9.0.7), for fixed k the size of Gram matrices G ij can be bounded from above by any s ∈ N >0 while the maximal matrix size of the standard semidefinite relaxation for POP (9.0.1) (defined as in Section 2.4) is fixed for each relaxation order k. Nevertheless, since we convert (9.0.1) to the form (9.0.3) (so as to use Corollary 9.2), the degrees of the resulting objective and constraint polynomials are doubled, i.e., deg( f ) = 2 deg(f ) and deg(ǧ i ) = 2 deg(g i ).

However, numerical experiments in Sections 9.3 and 9.4.7 suggest that our method works better than existing methods on examples of POPs with nonnegative variables. For instance, for 20variable dense POPs on the nonnegative orthant, the standard SOS-relaxations based on Putinar's Positivstellensatz provide a lower bound for f in 356 seconds while we can provide a better lower bound in 5 seconds.

Next, in Sections 9.4.6 and 9.4.6 we provide two convergent hierarchies of linear and semidefinite relaxations for POPs on the nonegative orthant, that exploit correlative sparsity, and with properties similar to those of (9.0.6) and (9.0.7). Accordingly, for POPs on the nonnegative orthant with up to 1000 variables, we can provide lower bounds in 19 seconds which are better than those obtained in 56360 seconds with the sparsity-adapted version of the standard SOS-relaxations of Waki et al. [START_REF] Waki | Sums of squares and semidefinite program relaxations for polynomial optimization problems with structured sparsity[END_REF].

II. In our second contribution, we provide a degree bound for an extended version of Handelman's Positivstellensatz to arbitrary compact basic semialgebraic sets. More explicitly, Corollary 9.3 states the following result. If f , ǧ1 , . . . , ǧm are polynomials even in each variable, -Š defined as in (9.0.4) has nonempty interior, ǧ1 = R -x 2 2 for some R > 0, -ǧi is of degree at most 2d gi and f -f is nonnegative on Š, then there exist positive constants c and c depending on f , ǧi such that for all ε > 0, for all

for some σ ij being SOS of monomials such that deg(σ ij ǧi ǧj 1 ) ≤ 2k. (Here ǧm := 1.) When compared with the extension of Pólya's Positivstellensatz in (9.0.5), the one of Handelman's Positivstellensatz in (9.0.10) does not have multiplier (1 + x 2 2 ) k but its number of SOS of monomials is m i=1 (k -d gi + 1) becomes larger when k increases. In contrast, the extension of Pólya's Positivstellensatz involves the same multiplier and its number of SOS of monomials is m + 1, so does not depend on k.

As a consequence, we obtain in Section 9.2.2 the rate of convergence for the hierarchy of linear relaxations (9.2.8) based on the extension of Handelman's Positivstellensatz. In addition, we also propose a new hierarchy of semidefinite relaxations in (9.2.22) based on even symmetry and the concept of factor width similarly to the one relying on Pólya's Positivstellensatz. A sparse version of this semidefinite hierarchy is also obtained in Section 9.4.6.

As shown in Sections 9.3 and 9.4.7, these hierarchies of semidefinite relaxations have the same numerical behavior as the ones based on Pólya's Positivstellensatz. In almost all cases, the ones based on the extension of Handelman's Positivstellensatz are several times slower but provide slightly better accurate bounds, compared to the ones based on the extension of Pólya's Positivstellensatz.

Related works

Exploiting sparsity: Structure exploitation in (9.0.7), is comparable to term sparsity and correlative sparsity (see Chapter 3) but here we can deal with dense POPs of the form (9.0.1). Moreover, the maximal block sizes involved in the sparsity-exploiting SDP relaxations mainly depend on the POP itself as well as on the relaxation order. By comparison, the maximal block size of our SDP relaxations is controllable. Under mild conditions, the rate of convergence ρ Han k,s → f as k increases, is at least O(ε -c ). [START_REF] Dickinson | On an extension of Pólya's Positivstellensatz[END_REF] a specific constrained version of Pólya's Positivstellensatz. Explicitly, if f, g 1 , . . . , g m are homogeneous polynomials, S is defined as in (9.0.2), and f is positive on S\{0}, then

Dickinson-Povh's hierarchy of linear relaxations: Dickinson and Povh state in

for some homogeneous polynomials σ i with positive coefficients. (Here g m := 1.) They also construct a hierarchy of linear relaxations associated with (9.0.11).

The extension of Pólya's Positivstellensatz restated in Corollary 9.2 is indeed analogous to (9.0.11). However the approach is different and importantly, the result is more convenient as we provide degree bounds for the SOS of monomials involved in the representation. Similarly, our corresponding linear relaxations (9.2.6) are the analogues to those of Dickinson and Povh [START_REF] Dickinson | A new approximation hierarchy for polynomial conic optimization[END_REF]. As shown in Example 9.2 and other examples in Sections 9.3 and 9.4.7, this hierarchy of linear relaxations usually have a poor numerical behavior in practice when k is large. Our new hierarchy of semidefinite relaxations (9.2. [START_REF] Blekherman | There are significantly more nonegative polynomials than sums of squares[END_REF]) is used to improve this issue.

1. For all ε > 0, there exists

where θ := 1 + x 2 2 .

2. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that for all ε > 0, one can take

The proof of Corollary 9.1 is similar to the proof of [132, Corollary 1].

Polynomials nonnegative on compact semialgebraic sets

In this section, we provide a representation of polynomials nonnegative on semialgebraic sets when the input polynomials are even in each variable. We also derive in Section 9.4.5 some sparse representations when the input polynomials have correlative sparsity.

Extension of Pólya's Positivstellensatz

The following corollary is deduced from Corollary 9.1. Corollary 9.2. Let f, g i , S, d f be as in Corollary 9.1 such that g 1 := R -x 2 2 for some R > 0. Then the following statements hold:

1. For all ε > 0, there exists K ε ∈ N such that for all k ≥ K ε , there exist SOS of monomials σ i satisfying (9.1.4) and

2 ) 2 and assume that f = σ 0 + σ 1 (1 -x 2 ) for some SOS of monomials σ i , i = 0, 1. Note that f is even and positive on [-1, 1]. We write

Then we obtain the system of linear equations:

This contradiction yields the conclusion. Thus, Putinar's Positivstellensatz with SOS of monomials does not exist when the input polynomials are even in each variable. However, we are still able to exploit term sparsity/even symmetry for Putinar's Positivstellensatz in this case as shown later in Proposition 9.1.

It is not hard to see that with the multiplier (1 + x 2 ) 2 , we obtain the Pólya's Positivstellensatz as follows:

where σ0 := x 8 and σ1 := x 4 + 15 4 x 2 + 9 4 are SOS of monomials.

We prove in the following proposition the existence of block-diagonal Gram matrices in Putinar's Positivstellensatz when the input polynomials are even in each variable: Recall that q(x) := q(x 2 ), for a given polynomial q. In this case, q is even in each variable. Then POP (9.2.1) is equivalent to

with x 2 being an optimal solution.

Let

and let diag(•) stand for the vector of diagonal entries of a square matrix.

Linear relaxations

Based on the extension of Pólya's Positivstellensatz

Consider the hierarchy of linear programs indexed by k ∈ N:

where

Remark 9.8. The optimal value τ Pól k only depends on the subset of variables {y 2α : α ∈ N n d f +k }, i.e., the optimal value of LP (9.2.5) does not change when we assign each of the other variables with any real number. It is due to the fact that θ, f , and ǧi only have nonzero coefficients associated to the monomials x 2α for some α ∈ N n .

, with g m = 1 and g 1 := R -j∈ [n] x j for some R > 0. Consider POP (9.2.1) with S being defined as in (9.2.2). For every k ∈ N, the dual of (9.2.5) reads as:

The following statements hold:

2. The sequence (ρ Pól k ) k∈N converges to f .

3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that

The proof of Theorem 9.2 relies on Corollary 9.2 and can be proved in almost the same way as the proof of [START_REF] Mai | On the complexity of Putinar-Vasilescu's Positivstellensatz[END_REF]Theorem 4].

Based on the extension of Handelman's Positivstellensatz

Consider the hierarchy of linear programs indexed by k ∈ N:

where

, with g m = 1 and g 1 := R -t∈ [n] x t for some R > 0.

Consider POP (9.2.1) with S being defined as in (9.2.2). For every k ∈ N, the dual of (9.2.8) reads as:

(9.2.9)

The following statements hold:

The sequence (ρ Han

k

) k∈N converges to f .

3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that

The proof of Theorem 9.3 relies on Corollary 9.3 and can be proved in almost the same way as the proof of Theorem 9.2.

Semidefinite relaxations

In this subsection, we construct the sparsity pattern A We write

Then for all j ∈ N >0 , W j = ∅ since j ∈ W j . For every j ∈ N, we write W j := {i 2 ) can be written explicitly as

.2.15)

In this matrix, the entries in bold belong to 2N 2 . Then A

) , j ∈ [5], are as follows:

and for every y = (y α ) α∈N n ⊂ R, let us define

(9.2.17)

Based on the extension of Pólya's Positivstellensatz

Consider the hierarchy of semidefinite programs indexed by s ∈ N >0 and k ∈ N:

where

Here ǧm = 1. Remark 9.9. If we assume that θ = 1 then (9.2.18) becomes a moment relaxation based on Putinar's Positivstellensatz for POP (9.0.3). Here each constraint M ki (ǧ i y) 0 is replaced by the constraint M A (s,k i ) j (ǧ i y) 0. If s is large enough, (9.2.18) corresponds to an SDP relaxation obtained after exploiting term sparsity (see [START_REF] Wang | TSSOS: A Moment-SOS hierarchy that exploits term sparsity[END_REF]).

, with g m = 1 and g 1 := R -j∈ [n] x j for some R > 0. Consider POP (9.2.1) with S being defined as in (9.2.2). For every s ∈ N >0 and for every k ∈ N, the dual of (9.2.18) reads as:

.2.19)

The following statements hold:

1. For all k ∈ N and for every s

2. For every s ∈ N >0 , the sequence (ρ Pól k,s ) k∈N converges to f . 3. If S has nonempty interior, there exist positive constants c and c depending on f, g i such that for every s ∈ N >0 and for every k larger than the negative ones returned by spPut with k = 2. Since in Table 9.9, the upper bounds on l y (x, ȳ) are negative, for all y = ȳ, l y (x, ȳ) must be negative. Thus, we conclude that this network is certifiably ε-robust on (x, ȳ).

Appendix

Preliminary material

For each q = α q α x α ∈ R[x], we note q := max α |qα| cα with c α := |α|! α1!...αn! for each α ∈ N n . This defines a norm on the real vector space R[x]. Moreover, for p 1 , q 2 ∈ R[x], we have q 1 q 2 ≤ q 1 q 2 , (9.4.1)

according to [START_REF] Schweighofer | On the complexity of Schmüdgen's Positivstellensatz[END_REF]Lemma 8].

We recall the following bound for central binomial coefficient stated in [94, page 590]:

We recall the degree bound for Pólya's Positivstellensatz [START_REF] Pólya | Über Positive Darstellung von Polynomen[END_REF]:

Lemma 9.2. (Powers and Reznick [START_REF] Powers | A new bound for Pólya's Theorem with applications to polynomials positive on polyhedra[END_REF]) If q is a homogeneous polynomial of degree d positive on ∆ n , then for all k ∈ N satisfying

( j∈ [n] x j ) k q has positive coefficients.

Let us recall the concept and the properties of polynomials even in each variable in [START_REF] Schabert | Uniform denominators in Hilbert's 17th Problem Theorems by Pólya and Reznick[END_REF]Definition 3.3]. A polynomial q is even in each variable if for every j ∈ [n], q(x 1 , . . . , x j-1 , -x j , x j+1 , . . . , x n ) = q(x 1 , . . . , x j-1 , x j , x j+1 , . . . , x n ) .

(9.4.4)

If q is even in each variable, then there exists a polynomial q such that q = q(x 2 1 , . . . , x 2 n ). Indeed, let q = α∈N n q α x α be a polynomial even in each variable. Let j ∈ [n] be fixed. Then q(x) = 1 2 (q(x) + q(x 1 , . . . , x j-1 , -x j , x j+1 , . . . , x n )). It implies that q α = 0 if α j is odd. Thus, q = α∈N n q 2α x 2α since j is arbitrary in [n]. This yields q = α∈N n q 2α x α .

For convenience, we denote x 2 := (x 2 1 , . . . , x 2 n ). Moreover, if q is even in each variable and homogeneous of degree 2d q , then q is homogeneous of degree d q . Conversely, if q is a polynomial of degree at most 2d such that q is even in each variable, the degree-2d homogenization of q is even in each variable.

The proof of Theorem 9.1

Step 1: Converting to polynomials on the nonnegative orthant. We claim that f is nonnegative on the semialgebraic set

is homogeneous and positive on S\{0}. To prove the first statement, we proceed exactly as in the proof of [START_REF] Dickinson | On an extension of Pólya's Positivstellensatz[END_REF]Theorem 2.4] for f + ε( n j=1 x j ) d f and derive the bound on the degree of polynomials having positive coefficients Proposition 9.2. (Putinar-Vasilescu's Positivstellensatz with diagonal Gram matrices) Let g 1 , . . . , g m be polynomials such that g 1 := R -x 2 2 for some R > 0 and g m := 1. Let S be the semialgebraic set defined by

Let f be a polynomial of degree at most 2d f such that f is nonnegative on S. Denote d gi := deg(g i )/2 . Then the following statements hold:

1. For all ε > 0, there exists K ε ∈ N such that for all k ≥ K ε , there exist vectors

2. If S has nonempty interior, then there exist positive constants c and c depending on f, g i such that for all ε > 0, one can take

Proof. Take two vectors of n variables y = (y 1 , . . . , y n ) and z = (z 1 , . . . , z n ). Given q ∈ R[x], denote the polynomial q(y,

Note that ĝ1 := R -y 2 -z 2 2 2 and ĝm := 1. Since f ≥ 0 on S, replacing x by y 2 -z 2 gives f ≥ 0 on Ŝ. From this and Corollary 9.2, there exist

With y = 1 2 (x + e) and z = 1 2 (xe), it becomes

Moreover, if a belongs to the interior of S, then ( 1 2 (a + e), 1 2 (ae)) belongs to the interior of Ŝ. Thus, the desired result follows.

As a consequence of Corollary 9.3, the next proposition states a new representation associated with diagonal Gram matrices for a polynomial positive on a compact semialgebraic set without assumption on even property. Proposition 9.3. (Representation without even symmetry) Let f, g i , S, d gi be as in Proposition 9.2. Then the following statements hold:

1. For all ε > 0, there exists K ε ∈ N such that for all k ≥ K ε , there exist vectors η (i,r)

2. If S has nonempty interior, then there exist positive constants c and c depending on f, g i such that for all ε > 0, one can take

The proof of Proposition 9.3 relies on Corollary 9.3 and can be proved in almost the same way as the proof of Proposition 9.2.

Numerical experiments

In this section we report results of numerical experiments for random instances with the same settings and notation as in Section 9.3.

Positive maximal singular values

Test problems: We generate a matrix M as in [54, (12) The numerical results are displayed in Table 9.10. The columns of val show the approximations of σ + (M) 2 .

Discussion: The behavior of our method is similar to that in Section 9.3.1.

Stability number of a graph

Let us consider POP (9.3.8) which returns the stability number of a graph G. Test problems: We generate the adjacency matrix A = (a ij ) j,j∈ [n] of the graph G by the following steps:

1. Set a ii = 0, for i ∈ [n].

2. For i ∈ [n], for j ∈ {1, . . . , i -1}, let us select a ij = a ji uniformly {0, 1}.

The numerical results are displayed in Table 9.11.

Note that the columns of val show the approximations of α(G).

Discussion: The behavior of our method is similar to that in Section 9.3.1. Note that the graphs from Tables 9.11 are dense so that we cannot exploit term sparsity or correlative sparsity for POP (9.3.8) in these cases. Moreover, for all graphs in Table 9.11, spPól provides the better bounds for α(G) compared to the ones returned by the second order relaxations of Put.

Remark 9.19. In Pb 3, 4 of Table 9.11, Pól with k = 1 provides a better bound than Pól with k = 0. As shown in Remark 9.9, each SDP relaxation of Pól with k = 0 and sufficiently large s corresponds to an SDP relaxation obtained after exploiting term sparsity.

Deciding the copositivity of a real symmetric matrix

Given a symmetric matrix A ∈ R n×n , we say that A is copositive if u Au ≥ 0 for all u ∈ R n + . Consider the following POP:

.4.90)

The matrix A is copositive iff f ≥ 0.

Test problems:

We construct several instances of the matrix A as follows:

1. Take B ij randomly in (-1, 1) w.r.t. the uniform distribution, for all i, j ∈ {1, . . . , n}.

2. Set B := (B ij ) 1≤i,j≤n and A := 1 2 (B + B ). The numerical results are displayed in Table 9.12. Discussion: The behavior of our method is similar to that in Section 9.3.1. In all cases, we can extract the solutions of the resulting POP and certify that A is not copositive since f is negative.

Deciding the nonnegativity of an even degree form on the nonegative orthant

Given a form q ∈ R[x], q is nonnegative on R n + iff q is nonnegative on the unit simplex Note that if d = 1, problem (9.4.92) boils down to deciding the copositivity of the Gram matrix associated to f . Thus, we consider the case where d ≥ 2.

Test problems:

We construct several instances of the form f of degree 2d as follows:

1. Take f α randomly in (-1, 1) w.r.t. the uniform distribution, for each α ∈ N n with |α| = 2d.

2. Set f := |α|=2d f α x α .

The numerical results are displayed in Table 9.13.

Discussion:

The behavior of our method is similar to that in Section 9.3.1. In these cases, we were able to extract the solution of the resulting POPs. One can then conclude that f is not nonnegative on the nonnegative orthant since it has negative value at its atoms.

Minimizing a polynomial over the boolean hypercube

Consider the optimization problem:

General discussion

We have provided several methods that use positivity certificates with and without denominators, for solving POPs on compact and non-compact basic semi-algebraic sets.

On the one hand, we have exploited correlative-term sparsity and constant trace property for the Moment-SOS hierarchy based on Putinar's Positivstellensatz without denominator. On the other hand, we have relied on Putinar-Vasilescu's Positivstellensatz with uniform denominator to provide an appropriate Moment-SOS hierarchy for solving POPs on noncompact semialgebraic sets.

It is worth pointing out that correlative-term sparsity enables us to reduce the matrix sizes involved in the SDP relaxations, and hence to solve them efficiently with interior-point methods. Besides, one advantage of exploiting constant trace property is to benefit from a class of first-order methods to speed up the resolution of the SDP relaxations. The sparse setting for methods based on Reznick's Positivstellensatz with uniform denominators has been also considered. We emphasize that the same uniform denominator allows us to control the sizes of the SDP relaxations for POPs on the nonnegative orthant.

On the theoretical side, we have proved that methods with uniform denominators can have polynomial time complexity, similarly to methods without denominators. On a practical side, the efficiency of our methods has been illustrated on real-life problems arising from engineering, machine learning, and power networks.

Discussion and perspectives specific to each chapter

We list the detailed conclusions for each chapter:

• In Chapter 3 we have introduced the CS-TSSOS hierarchy, a sparse variant of the Moment-SOS hierarchy, to solve large-scale real-world nonlinear optimization problems whose input data are sparse polynomials. In addition to its theoretical convergence guarantees, CS-TSSOS allows one to make a trade-off between the quality of optimal values and the computational efficiency by controlling the types of chordal extensions and the sparse order k.

By fully exploiting sparsity, CS-TSSOS allows one to go beyond Shor's relaxation and solve the second-order Moment-SOS relaxation associated with large-scale POPs to obtain more accurate bounds. Indeed CS-TSSOS can handle second-order relaxations of POP instances with thousands of variables and constraints on a standard laptop in few minutes. Such instances include the optimal power flow (OPF) problem, an important challenge in the management of electricity networks. In particular, our plan is to perform advanced numerical experiments on HPC cluster, for OPF instances with larger numbers of buses [START_REF] Eltved | On the robustness and scalability of semidefinite relaxation for optimal power flow problems[END_REF].

Some additional further potential research directions are listed below: