Je Voudrais

Vinh Lgipm

Tran Thanh

Aurelié Sarah Bach

Viet Blind

Phuong Anh

Sara Anh

Pham Samir

Tuyet Tuan

Khac Trinh

Truc Linh

Prof Xuan Vy

Vinh

Mme Gaëlle

Mme Wiemert Greppi

J'apprécie

Xuan Phong

Thai Duy

Van Khoi

Ngoc Quynh

Ngoc Phuc

Kim Trinh

Quoc Viet

Tri

Quoc

Thuan Nhanh

An

Huy, Thuong Le

H A Le Thi

T Pham Dinh

H P H Luu

Deterministic H M Le

Communications in national / international conferences

Cette thèse n'aurait pas été possible sans les encouragements, les conseils et le soutien de mes conseillers, de mon comité de thèse, de mes collègues, de mes amis et de ma famille. Trois années ont passé avec beaucoup de souvenirs que je n'oublierais jamais. La thèse a été réalisée au LGIPM de l'Université de Lorraine pendant une période de turbulence et d'incertitude causée par la pandémie de COVID-19.

La thèse touche maintenant à sa fin et il est temps de dire merci.

Tout d'abord, je tiens à remercier ma directrice de thèse, le professeur Le Thi Hoai An, Université de Lorraine, pour ses conseils qui sont essentiels à la réalisation de ma thèse. Je lui suis reconnaissant pour sa patience, sa confiance et son attention pendant ma thèse de doctorat et aussi pendant la période où j'ai travaillé au LGIPM en tant qu'étudiant stagiaire. Tout au long de ces années éprouvantes, elle m'a donné une liberté totale pour poursuivre une variété de sujets de recherche intéressants. Ses commentaires constructifs sur mon travail m'ont appris à mener des recherches significatives et ont grandement amélioré mes capacités de réflexion critique.

Je suis reconnaissant à mon co-directeur de thèse, le professeur Le Hoai Minh, Université de Lorraine, pour ses encouragements, son soutien et ses discussions utiles qui m'ont aidé à améliorer mes recherches sous de nombreux aspects. Ses conseils sont essentiels pour me permettre de mener à bien cette thèse.

Il m'a également donné d'excellentes conditions de travail et des outils pour mes recherches.

Je suis reconnaissant au professeur Pham Dinh Tao, l'INSA de Rouen, qui m'a offert d'innombrables discussions qui ont réellement inspiré ma passion pour les mathématiques. Ces discussions instructives

Je dédie cette thèse à mon père, Luu Van Hieu, et à ma mère, Hoang Thi Do, qui me soutiennent toujours dans ma vie.

iii

Abbreviations and Notations

The key abbreviations and notations used in this thesis are listed as follows.

DC

F (x) = E(f (x, Z)), (1)
où Z est une variable aléatoire dont la distribution, P Z , est inconnue en général. En général, on ne peut accéder à Z que par l'intermédiaire de ses réalisations i.i.d. (indépendantes et identiquement distribuées) Z 1 , Z 2 , . . . , Z N (appelées échantillons). Dans un scénario plus difficile, ces échantillons ne sont plus i.i.d., c'est-à-dire qu'ils sont corrélés les uns aux autres, et leurs distributions sont différentes de la distribution de Z, mais convergent vers celle-ci. En particulier, une chaîne de Markov Z 1 → Z 2 → . . . → Z N → . . . avec une distribution d'équilibre de P Z est un exemple typique de ce dernier cas.

Dans le cadre (1), la distribution de Z ne dépend pas de la variable d'optimisation (la décision) x, on parle alors d'incertitude exogène. Dans le cas où la décision x affecte la distribution de Z, nous sommes face à une incertitude endogène, un objet bien plus compliqué que le premier.

Comme les modèles d'apprentissage automatique ont tendance à être de plus en plus compliqués pour capturer la structure complexe des données, les problèmes d'optimisation qui en résultent sont plus susceptibles d'être non convexes. Bien qu'il existe une littérature importante sur l'optimisation convexe dans des contextes déterministes et stochastiques, la recherche sur l'optimisation stochastique non convexe reste limitée. Pour contribuer à cette direction de recherche, dans cette thèse, l'optimisation stochastique non convexe est le principal sujet d'étude. La structure non convexe étudiée est la DC (différence de fonctions convexes). Comme son nom l'indique, une fonction est dite DC si elle peut être xvii exprimée comme une différence de deux fonctions convexes, ce qui donne le programme DC suivant min x∈R n {f (x) := g(x) -h(x)}, où g, h : R n → R ∪ {+∞} sont convexes, propres, semicontinus inférieurs. La classe des programmes DC est fermée sous l'opérateur habituel considéré en optimisation et elle est assez large pour englober la plupart des fonctions objectives du monde réel [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF]. La programmation DC est une approche fondamentale de l'optimisation non convexe, non seulement en raison de sa polyvalence mais aussi de sa maniabilité, grâce à un algorithme bien établi connu sous le nom de DCA (algorithme DC) [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF]. Le DCA [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF][START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF], introduit par Pham De nos jours, la maintenance est au centre de toute activité industrielle pour assurer la stabilité, l'efficacité et la productivité des entreprises. À l'ère de l'industrie intelligente, une politique de maintenance correcte peut faire une grande différence dans la compétitivité d'une entreprise. Grâce à sa capacité à prévoir les défauts de fonctionnement et à estimer l'état de santé des actifs, la maintenance prédictive peut minimiser les risques, les coûts de maintenance et les temps d'arrêt tout en améliorant la productivité. Pour mettre en oeuvre la maintenance prédictive, il faut soit une connaissance considérable du domaine pour modéliser précisément le mécanisme physique sous-jacent de la dégradation, soit utiliser l'intelligence artificielle pour prédire les défauts directement à partir des données des capteurs. Les connaissances d'experts n'étant pas disponibles dans la plupart des cas, cette dernière approche est préférable grâce à sa facilité de déploiement, à la technologie de pointe des capteurs qui permet une collecte de données de haute qualité, et aux améliorations rapides de l'apprentissage automatique et de l'apprentissage profond. Le développées sur la base de la diffusion de Langevin sous-amortie avec un nouveau schéma de discrétisation introduit dans [START_REF] Cheng | Underdamped langevin mcmc: A non-asymptotic analysis[END_REF].

Sur le plan pratique, les méthodes proposées ont été appliquées à un certain nombre de problèmes d'apprentissage automatique/apprentissage profond, où leurs vertus ont été justifiées. Le

Convex and variational analysis

To begin, we review some notions from Convex Analysis and Nonsmooth Analysis that will be useful later (see, e.g., [START_REF] Rockafellar | Convex analysis[END_REF][START_REF] Rockafellar | The theory of subgradients and its applications to problems of optimization: Convex and nonconvex functions[END_REF][START_REF] Clarke | Optimization and nonsmooth analysis[END_REF][START_REF] Rockafellar | Variational analysis[END_REF][START_REF] Boyd | Convex optimization[END_REF][START_REF] Mordukhovich | An easy path to convex analysis and applications[END_REF]). If we relax the condition λ ∈ [0, 1] in the definition of convex set, we obtain the notion of affine set: a set C ⊂ R n is affine if the line passing through any two distinct points in C remains in C, i.e., for x 1 , x 2 ∈ C and λ ∈ R, λx 1 +(1-λ)x 2 ∈ C. As a matter of fact, a nonempty affine set is always parallel to a unique linear subspace of R n . One then defines the dimension of an affine set to be the dimension of the corresponding parallel linear subspace.

Likewise, in the above definition of convex hull, if the condition λ 1 , λ 2 , . . . , λ k ≥ 0 is relaxed, we obtain the notion of affine hull,

aff C := {λ 1 x 1 + λ 2 x 2 + . . . + λ k x k : x i ∈ C, ∀i = 1, k, λ 1 + λ 2 + . . . + λ k = 1}.
The affine hull of C is the smallest affine set containing C.

Let Ω be a subset of R n , we call Ω a cone if λx ∈ Ω whenever λ ≥ 0 and x ∈ Ω. Let C ⊂ R n be a convex set and x ∈ C. The normal cone of C at x is defined as follows

N C (x) := {v ∈ R n : ⟨v, x -x⟩ ≤ 0, ∀x ∈ C}.
For a set C, we define the relative interior of C, denoted by ri C, as follows:

ri C := {x ∈ C : B(x, r) ∩ aff C ⊂ C for some r > 0}.

When the affine hull of C is full dimension, i.e., aff C = R n , the notion of relative interior coincides with the notion of interior (denoted by int).

For a nonempty convex set C ⊂ R n , the above notion of relative interior is reduced to ri C := {x ∈ C : ∀y ∈ C, ∃λ > 1 such that λx + (1 -λ)y ∈ C}. An important geometrical characteristic of a convex set is that a convex set "bends outwards". Due to this characteristic, it is possible to support a convex set -at each boundary point of it -by a hyperplane, i.e., a set of the form {x ∈ R n : ⟨v, x⟩ = α} with v ̸ = 0 (see Figure 1.3) and it is possible to separate two disjoint convex sets by a hyperplane (see Figure 1.4). Formal statements for these properties can be found, for example, in [START_REF] Boyd | Convex optimization[END_REF].

Let f : R n → R ∪ {+∞} be an extended-real-valued function. The effective domain of f is defined as

dom f := {x ∈ R n : f (x) < +∞}.
The function f is said to be proper if dom f ̸ = ∅.

For f : R n → R ∪ {+∞}, the epigraph of f is defined as

epi f = {(x, t) : x ∈ dom f, f (x) ≤ t}.
Meanwhile, the graph of f , gph f , is given by gph f = {(x, t) : x ∈ dom f, f (x) = t}. The function f is said to be convex if its epigraph is a convex set in R n+1 . Or equivalently,

f (λx + (1 -λ)y) ≤ λf (x) + (1 -λ)f (y), for all x, y ∈ R n , λ ∈ [0, 1]. (1.1)
Furthermore, f is called ρ-convex for some ρ ≥ 0, if f -(ρ/2)∥•∥ 2 is a convex function. Or equivalently, for all x, y ∈ R n , λ ∈ [0, 1],

f (λx + (1 -λ)y) ≤ λf (x) + (1 -λ)f (y) - ρ 2 λ(1 -λ)∥x -y∥ 2 .
(1.

2)

The supremum of all ρ ≥ 0 such that the inequality (1.2) holds is called the convex modulus of f , denoted by ρ(f ) or ρ f . If ρ(f ) > 0, f is said to be strongly convex. Note that, the notion of convexity (and strong convexity) can be defined locally over a nonempty convex set contained in dom f as follows: f is convex on C if the inequality (1.1) holds true for all x, y ∈ C, λ ∈ [0, 1].

A function f : R n → R ∪ {+∞} is Lipschitz continuous on Ω ⊂ dom f if there is a constant L ≥ 0 such that |f (x) -f (y)| ≤ L∥x -y∥, ∀x, y ∈ Ω.
Meanwhile, f is said to be locally Lipschitz continuous around x ∈ dom f if there exists L x ≥ 0 and δ x > 0 such that

|f (x) -f (y)| ≤ L x∥x -y∥, ∀x, y ∈ B(x, δ x).
If f is a convex function, it is locally Lipschitz continuous in the interior of its domain, int(dom f ).

Let Ω ⊂ dom f be an open set and suppose that f is differentiable on Ω, f is said to have L-Lipschitz continuous gradient for some L > 0 (or L-smooth) on Ω if ∥∇f (x) -∇f (y)∥ ≤ L∥x -y∥, ∀x, y ∈ Ω.

Let x ∈ dom(f ), a vector z ∈ R n is called a subgradient of f at x if f (y) -f (x) ≥ ⟨z, y -x⟩, ∀y ∈ R n .

The set of all subgradients of f at x is called the subdifferential of f at x, denoted by ∂f (x), ∂f (x) := {z ∈ R n : f (y) -f (x) ≥ ⟨z, y -x⟩, ∀y ∈ R n }.

(1.3)

By definition, the subdifferential of f at x, if nonempty, offers affine minorants to f at x (affine functions whose graphs are below the graph of f and touch the graph of f at x), y → f (x) + ⟨z, y -x⟩, for z ∈ ∂f (x). Figure 1.6 illustrates affine minorants of f . And, f is said to be subdifferentiable at x if ∂f (x) ̸ = ∅. Also, by definition,

dom ∂f = {x ∈ R n : ∂f (x) ̸ = ∅}.
For a proper convex function f , the following relations hold ri(dom f ) ⊂ dom ∂f ⊂ dom f.

For ϵ ≥ 0, the ϵ-subdifferential (which is a relaxation of the notion of subdifferential) of f at x is defined as follows

∂ ϵ f (x) := {z ∈ R n : f (y) -f (x) ≥ ⟨z, y -x⟩ -ϵ, ∀y ∈ R n }.
While the the concepts of subgradient and subdifferential do not assume that f is convex, they do not make much sense for a nonconvex function f as the subdifferential of a nonconvex function can easily be an empty set. Therefore, when it comes to subgradient and subdifferential, one usually refers to a convex function (in such a case, dom ∂f ⊃ ri(dom f )). Furthermore, for x ∈ dom f where f is convex, the subdifferential ∂f (x) can be characterized in terms of normal cone of epi f as follows,

∂f (x) = {v ∈ R n : (v, -1) ∈ N epi f ((x, f (x)))}.
In particular, if f is the indicator function of a convex set C,

f (x) = χ C (x) :=    0 if x ∈ C, +∞ otherwise, the above relation implies ∂f (x) = N C (x).
The function f is said to be lower semicontinuous at

x ∈ R n if f (x) ≤ lim inf y→x f (y).
Moreover, f is called lower semicontinuous on R n if f is lower semicontinuous at every point of R n . It should be noted that a function is lower semicontinuous if and only if its epigraph is closed.

For a proper function f and The conjugate of a function f is defined as

x * ∈ dom f , x * is called a global minimizer of f if f (x * ) ≤ f (x), ∀x ∈ R n ; Meanwhile, x * is called a local minimizer (resp. strict local minimizer) of f if there is r > 0 such that f (x * ) ≤ f (x), ∀x ∈ B(x * , r) (resp. f (x * ) < f (x), ∀x ∈ B(x * , r) \ {x * }).
f * (y) := sup{⟨x, y⟩ -f (x) : x ∈ R n }, for y ∈ R n .
It follows directly from the definition of f * that

f * (y) + f (x) ≥ ⟨x, y⟩, ∀x, y ∈ R n ,
which is usually referred to as Young's inequality. An interesting property of conjugation is that the conjugate of the conjugate (known as biconjugate) of a proper, convex, lower semicontinuous function is itself, i.e., f * * = f , which is known as the Fenchel-Moreau theorem. Hereafter, we denote the class of proper, convex, lower semicontinuous functions by Γ 0 (R n ).

A direct implication of the Fenchel-Moreau theorem is the following equivalence. Let

f ∈ Γ 0 (R n ), then y ∈ ∂f (x) ⇔ x ∈ ∂f * (y). (1.4)
In summary, the following theorem encompasses essential properties of functions in Γ 0 (R n ).

Theorem 1. Let f ∈ Γ 0 (R n ).
The following properties hold true.

1. f * ∈ Γ 0 (R n ), f * * = f. 2. y ∈ ∂f (x) ⇔ x ∈ ∂f * (y) ⇔ f (x) + f * (y) = ⟨x, y⟩.
Meanwhile, the inequality f (x) + f * (y) ≥ ⟨x, y⟩ always holds true for all x, y ∈ R n .

1.1. Convex and variational analysis 3. f is continuous on int(dom f ).

4.

The subdifferential mapping ∂f is closed: for two sequences {x k } and {y k } such that y k ∈ ∂f (x k ) and x k → x * , y k → y * , then y * ∈ ∂f (x * ).

5.

Let S be a compact set of int(dom f ), then ∂f (S) := ∪{∂f (x) : x ∈ S} is a compact set, and let

γ := sup{∥y∥ : y ∈ ∂f (S)} < +∞.
In this case f is Lipschitz continuous on S with

|f (u) -f (x)| ≤ γ ∥u -x∥ , ∀(u, x) ∈ S × S 6. 0 ∈ ∂f (x * ) if and only if x * is a solution of the convex program inf{f (x) : x ∈ R n }.
As mentioned earlier, although the notion of subdifferential (1.3) can be defined without the requirement of convexity, it is less meaningful in the nonconvex setting since this subdifferential is easily an empty set for nonconvex functions. Therefore, one needs some alternatives. We next recall some different notions of subdifferentials and stationarity for nonsmooth, nonconvex functions. For a lower semicontinuous function, the Fréchet subdifferential of f at x ∈ dom f is defined as

∂ F f (x) = z ∈ R n : lim inf h→0 f (x + h) -f (x) -⟨z, h⟩ ∥h∥ ≥ 0 . For x / ∈ dom f, we set ∂ F f (x) = ∅. Note that, even if x ∈ dom f , ∂ F f (x)
can be empty, e.g., take f (u) = -|u| where u ∈ R. The set ∂ F f (x) is a closed and convex set. However, the Fréchet subdifferential is not a closed mapping, hence it is computationally unstable [START_REF] Bolte | Clarke subgradients of stratifiable functions[END_REF]. For this reason, one considers the following "limiting version" of Fréchet subdifferential, called limiting subdifferential. By definition, the limiting subdifferential of

f at x ∈ dom f is ∂f (x) = {z ∈ R n : ∃(x k , f (x k )) → (x, f (x)), z k ∈ ∂ F f (x k ), z k → z},
and we put ∂f (x) = ∅ if x / ∈ dom f. Now, the limiting subdifferential is a closed mapping. However, the set ∂f (x) is closed but not necessarily convex.

If f is locally Lipschitz at x ∈ R n , then the Clarke directional derivative f C (x, •) at x and the Clarke subdifferential ∂ C f (x) are given respectively by

f C (x; d) = lim sup (t,u)→(0 + ,x) f (u + td) -f (u) t and ∂ C f (x) = {y ∈ R n : ⟨y, d⟩ ≤ f C (x; d), ∀d ∈ R n }. A point x ∈ R n is called a Fréchet (resp. limiting/ Clarke) stationary point for the function f , if 0 ∈ ∂ F f (x) (resp. 0 ∈ ∂f (x)/ 0 ∈ ∂ C f (x)).

Chapter 1. Preliminaries

The following inclusions describe the relations between Fréchet, limiting, and Clarke subdifferentials:

∂ F f (x) ⊂ ∂f (x) ⊂ ∂ C f (x).
When f is a convex function, the Fréchet, limiting, and Clarke subdifferential coincide with the Convex Analysis subdifferential (1.3).

Let f be a proper function and x ∈ dom f , the directional derivative of f at x along a direction d is defined by

f ′ (x; d) = lim t↓0 f (x + td) -f (x) t ,
given that the limit exists.

A point x ∈ dom f is d-stationary point of f iff f ′ (x; d) ≥ 0 for all d ∈ R n . If f is convex, if x ∈ int(dom f ), f ′ (x, d
) exists as a real number, for all direction d ∈ R n . Moreover, for x ∈ dom f , the set ∂f (x) can be characterized using directional derivative as follows:

y ∈ ∂f (x) iff f ′ (x; d) ≥ ⟨d, y⟩ for all d ∈ R n .

DC programming and DCA

Recall that Γ 0 (R n ) denotes the convex cone of all proper, lower semicontinuous, convex functions on R n . The standard DC program takes the form [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF] (P dc )

α := inf{f (x) := g(x) -h(x) : x ∈ R n }
where g, h ∈ Γ 0 (R n ). Such a function f is called DC, g -h is DC decomposition while g and h are DC components of f . It is worth noting that a DC function has infinitely many DC decompositions. We may assume that g and h are strongly convex without losing generality since f can be recast as the difference of two strongly convex functions as follows

f = g + ρ 2 ∥ • ∥ 2 -h + ρ 2 ∥ • ∥ 2 , with ρ > 0.
The class of DC functions is quite rich so as to encompass most real-world optimization problems [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF].

Moreover, it is possible to uniformly approximate any continuous function over a compact set by a sequence of DC functions [START_REF] Bačák | On difference convexity of locally lipschitz functions[END_REF]. The class of DC functions is closed under usually operators considered in optimization, e.g., linear combination, maximum, or minimum of a finite number of DC functions are again DC functions. However, the class of DC functions is unstable under pointwise convergence and the supremum operator (of an infinite number of DC functions, of course) [START_REF] Correa | Necessary and sufficient optimality conditions in dc semi-infinite programming[END_REF].

A DC program with a convex constraint x ∈ C can be written in form of the standard DC program (P dc ) by integrating the indicator function of C, χ C , into the first DC component, ĝ = g + χ C . When a DC function is minimized over nonconvex DC constraints, the optimization problem is no longer a standard DC program. Rather, it is referred to as a general DC program [START_REF] Le Thi | DC programming and DCA for general DC programs[END_REF][START_REF] Pham Dinh | Recent advances in DC programming and DCA[END_REF]. For general DC programs with DC constraints, there are two approaches to convert them to standard DC programs: applying penalty technique or convexifying DC constraints [START_REF] Le Thi | DC programming and DCA for general DC programs[END_REF][START_REF] Pham Dinh | Recent advances in DC programming and DCA[END_REF]. In this thesis, we are mostly concerned with standard DC programs.

DC programming and DCA

The dual DC program of (P dc ) is defined by

(D dc ) β := inf{h * (y) -g * (y) : y ∈ R n } .
It can be verified that α = β and there is the perfect symmetry of the DC duality: the dual of (D dc ) is exactly (P dc ).

For a DC function f = g -h, a global minimizer of f is characterized as follows [103, Theorem 1]:

x * is the global minimizer of f if and only if ∂ ϵ h(x * ) ⊂ ∂ ϵ g(x * ), ∀ϵ ≥ 0. It is fascinating that -for DC programming -globality can be expressed in terms of ϵ-subdifferentials. In practice, however, the

condition ∂ ϵ h(x * ) ⊂ ∂ ϵ g(x *
) for all ϵ ≥ 0 is almost impossible to achieve as it requires the inclusion to hold for all ϵ ≥ 0. The foregoing inclusion is naturally relaxed to hold solely for ϵ = 0, giving rise to the notion of strong (DC) criticality:

A point x * is called strongly (DC) critical for f = g -h if ∅ ̸ = ∂h(x * ) ⊂ ∂g(x *
) (note that ∂ 0 coincides with the standard notion of subdifferential in convex analysis). This requirement -however -is also very hard to develop corresponding iterative algorithms.

Therefore, the notion of (DC) criticality was invented as a further relaxation of strong DC criticality:

a point x * is called a (DC) critical point of f = g -h if 0 ∈ ∂g(x * ) -∂h(x * ), or equivalently, ∅ ̸ = ∂g(x * ) ∩ ∂h(x * ).
When h is differentiable at x * , DC criticality becomes strong DC criticality. As a convex function is differentiable almost everywhere, a DC critical point is very likely to be strongly DC critical. However, the notion of DC criticality is now dependent on DC decompositions of f as a result of this relaxing process. Because a DC function can be decomposed in infinitely many ways, to be better understood, the above definitions of DC criticality should be related to other generalized subdifferentials of f in nonsmooth nonconvex analysis, such as Fréchet, or Clarke subdifferentials. For instance, the following inclusions give a link between DC criticality and Clarke/Fréchet stationarity

∂ C f (x) ⊆ ∂g(x) -∂h(x), ∂ F f (x) ⊆ ∂g(x) -∂h(x). (1.5)
The first (resp. second) equality is verified if either g or h (resp. h) is differentiable at x. On the other hand, strong DC criticality coincides with d-stationarity on ri(dom g) ∩ ri(dom h) [START_REF] Le Thi | Convergence analysis of difference-of-convex algorithm with subanalytic data[END_REF][START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF].

DC criticality and strong DC criticality are the necessary conditions for local optimality. The sufficient conditions for local optimality are presented as follows [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF]. Let x * be a point admitting a neighbor-

hood U such that ∂h(x) ∩ ∂g(x * ) ̸ = ∅, ∀x ∈ U ∩ dom g, then x * is a local minimizer of f . On the other hand, if x * ∈ int(dom h) satisfies ∂h(x * ) ⊂ int(∂g(x * )), then x * is a strict local minimizer of f .
In practice, to measure the goodness of the found solution, one also uses the notions of ϵ-criticality and nearly ϵ-criticality [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF].

A point x * is called ϵ-critical of (P dc ) if dist(∂g(x * ), ∂h(x * )) ≤ ϵ, while it is called nearly ϵ-critical if there exists a point x such that ∥x -x * ∥ = O(ϵ) and dist(∂g(x), ∂h(x)) ≤ ϵ ,
where dist(A, B) denotes the distance between two sets A and B.

The DCA [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF][START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF][START_REF] Le Thi | The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems[END_REF] is an effective method for solving DC programs, especially when the DC decomposition is tailored. Iteratively linearizing the second DC component,

h k (x) = h(x k ) + ⟨x -x k , y k ⟩,
where y k ∈ ∂h(x k ), and solving the resultant convex program to yield x k+1 , is essentially how DCA works. The DCA subsequentially converges to DC critical points while guaranteeing a decrease in the objective value across iterations.

Standard DCA.

Initialization: Let x 0 ∈ dom ∂h and k = 0.

repeat

Step 1: Compute the subgradient y k ∈ ∂h(x k ).

Step 2: Solve the convex program

x k+1 ∈ arg min{g(x) -h k (x) : x ∈ R n }.
Step 3:

k = k + 1.
until Stopping criterion.

The reader is referred to [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF][START_REF] Le Thi | The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems[END_REF][START_REF] Le Thi | The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems[END_REF] for the DCA's convergence properties and the theoretical underpinning of DC programming. Some essential properties of the sequence generated by DCA are presented in Theorem 2.

Theorem 2. The sequence {x k } generated by DCA has the following properties:

1. The sequence {(g -h)(x k )} is decreasing, (g -h)(x k+1 ) ≤ (g -h)(x k ) - ρ(g) + ρ(h) 2 ∥x k+1 -x k ∥ 2 , ∀k ∈ N. 2. If (g -h)(x k+1 ) = (g -h)(x k ), then
x k and x k+1 are critical points of (P dc ):

y k ∈ ∂g(x k ) ∩ ∂h(x k ) and y k ∈ ∂g(x k+1 ) ∩ ∂h(x k+1 ).
3. If ρ(g) + ρ(h) > 0 then the series ∞ k=1 ∥x k+1 -x k ∥ 2 is convergent, and

min{∥x l+1 -x l ∥ : l = 0, 1, . . . , k} ≤ √ 2(f (x 0 ) -α) 1/2 (ρ(g) + ρ(h)) 1/2 (k + 1) 1/2 , so, min{∥x l+1 -x l ∥ : l = 0, 1, . . . , k} = O(k -1/2 ).
4. If the optimal value α of the problem (P dc ) is finite and the sequences {x k } and {y k } are bounded, then every limit point of {x k } is a critical point of (P dc ), and every limit point of {y k } is a critical point of (D dc ).

We observe that item 3 of Theorem 2 guaranteeing ∞ k=1 ∥x k+1 -x k ∥ 2 < +∞ is one step away from the convergence of the whole sequence {x k }. Indeed, if a stronger condition ∞ k=1 ∥x k+1 -x k ∥ < +∞ holds, it follows that {x k } is a Cauchy sequence, so it is convergent.

We visualize how DCA works through the following simple example. Let us consider a DC function f = g -h where g = x 2 + 2x and h = 5|x| + 0.005(x + 1) 4 .

A duality point of view

Step 1 of the DCA,

y k ∈ ∂h(x k ), is equivalent to x k ∈ ∂h * (y k ) thanks to (1.4), or h * (z) ≥ h * (y k ) + ⟨x k , z -y k ⟩, ∀z ∈ R n , (1.6)
which is further equivalent to

y k ∈ arg min{h * (z) -⟨x k , z⟩ : z ∈ R n }.
Likewise, Step 2 of the DCA, x k+1 ∈ arg min{g(x) -⟨y k , x⟩}, is equivalent to y k ∈ ∂g(x k+1 ). Thanks to (1.4), it is further equivalent to

x k+1 ∈ ∂g * (y k ). (1.7)
From (1.6) and (1.7), we have the following observation: if {(x k , y k )} is the sequence generated by DCA applied to the primal DC program (P dc ), then {(y k , x k+1 )} is the sequence generated by DCA applied to the dual DC program (D dc ). Consequently, for each convergence property of the primal sequence {x k }, there is a corresponding convergence property for the dual sequence {y k }.

The convergence of DCA for subanalytic functions

It is noteworthy that Theorem 2 only guarantees the subsequence convergence of the sequence generated by DCA. Recent advances in DC programming and DCA have shed some light on the convergence of the whole sequence for subanalytic functions, as well as the convergence rates of the sequence to DC critical points. Such convergence results will be presented in the remainder of this introductory chapter.

First, let us review some subanalytic set and function concepts [START_REF] Łojasiewicz | Sur la géométrie semi-et sous-analytique[END_REF][START_REF] Bolte | The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF].

1. A function f : R n → R is called real-analytic if for each point p ∈ R n
, there is a convergent series that converges to f in some neighborhood of p.

2.

A subset C of R n is said to be semianalytic if for each point of R n there exists a neighborhood V such that C ∩ V is of the following form:

C ∩ V = p i=1 q j=1 {x ∈ V : f ij (x) = 0, g ij (x) > 0},
where

f ij , g ij : V → R (1 ≤ i ≤ p, 1 ≤ j ≤ q) are real-analytic functions. 3. A subset C of R n is called subanalytic if for each point of R n there exists a neighborhood V such that C ∩ V = {x ∈ R n : ∃ y ∈ R m , (x, y) ∈ D},
where D is a bounded semianalytic subset of R n × R m for some m ≥ 1.

A function

f : R n → R ∪ {+∞} is said to be subanalytic if its graph gph f is a subanalytic subset of R n+1 .
Subanalytic functions enjoy the following Łojasiewicz subgradient inequality.

Theorem 3. (Theorem 3.1 [START_REF] Bolte | The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems[END_REF]) Let f : R n → R∪{+∞} be a subanalytic function such that its domain dom f is closed and f | dom f is continuous and let x 0 be a Fréchet critical point of f. Then, there exist θ ∈ [0, 1), L > 0 and a neighborhood V of x 0 such that the following inequality holds,

|f (x) -f (x 0 )| θ ≤ L∥x * ∥ ∀ x ∈ V, x * ∈ ∂ F f (x),
where a convention 0 0 = 1 is used.

The number θ is called a Łojasiewicz exponent of the function f at the critical point x 0 .

Under subanalytic data, Theorem 4 establishes the convergence of the whole sequence generated by DCA, and Theorem 5 provides the convergence rate of such a sequence to its limit point (the limit point is also a DC critical point), where the convergence rate depends on the Łojasiewicz exponent of f at that limit point (see [START_REF] Le Thi | Convergence analysis of difference-of-convex algorithm with subanalytic data[END_REF] for more details).

DC programming and DCA

Theorem 4. Let us consider DC problem (P dc ) with α ∈ R. Suppose that the sequences {x k } and {y k } are defined by the DCA. Suppose that the DC function f := g -h is subanalytic such that dom f is closed; f | dom f is continuous and that around every critical point of (P dc ), either g or h is differentiable with locally Lipschitz derivative. Assume that ρ := ρ(g) + ρ(h) > 0, where ρ(g) and ρ(h) are modulus of the strong convexity of g and h, respectively. If either the sequence {x k } or {y k } is bounded, then {x k } and {y k } are convergent to critical points of (P dc ) and (D dc ), respectively.

Theorem 5. Suppose that the assumptions of Theorem 4 are satisfied. Let x ∞ be the limit point of {x k }, where the Łojasiewicz exponent of the function f at x ∞ is denoted by θ ∈ [0, 1). Then, there exist constants τ 1 , τ 2 > 0 such that, for all k ∈ N,

∥x k -x ∞ ∥ ≤ ∞ j=k ∥x j -x j+1 ∥ ≤ τ 1 ∥x k -x k-1 ∥ + τ 2 ∥x k -x k-1 ∥ 1-θ θ .
As a consequence, one has

• If θ ∈ (1/2, 1), then ∥x k -x ∞ ∥ ≤ ck 1-θ
1-2θ for some c > 0.

• If θ ∈ (0, 1/2], then ∥x k -x ∞ ∥ ≤ cq k for some c > 0 and q ∈ (0, 1).

• If θ = 0, then {x k } is convergent in a finite number of steps.

Chapter 2

Stochastic DCA with Variance Reduction Abstract. In this chapter, we develop stochastic DCA schemes for tackling a class of structured DC programs. Because the standard DCA requires full information of (sub)gradients, which can be costly in large-scale contexts, stochastic methods depend instead on stochastic information. Stochastic estimations, on the other hand, generate additional variance terms, making stochastic algorithms unstable. Therefore, we incorporate several variance reduction strategies into our design, such as SVRG and SAGA. The proposed algorithms' almost sure convergence to critical points is established, and the algorithms' complexity are examined. We apply our algorithms to three important machine learning tasks to investigate their efficiency: nonnegative principal component analysis, group variable selection in multiclass logistic regression, and sparse linear regression. Numerical studies have demonstrated the effectiveness of our proposed algorithms over existing cutting-edge stochastic approaches for addressing nonconvex large-sum problems.

Introduction

The following nonconvex optimization problem is of our interest

(P ) min {F (x) = G(x) -H(x) + r 1 (x) -r 2 (x) : x ∈ R n } ,
where

H := 1 N N i=1 h i , G, h i : R n → R and r 1 , r 2 :
R n → R ∪ {+∞} are convex, lower semicontinuous functions while each h i has Lipschitz continuous gradient (with a common constant L) on the effective domain of r 1 . The optimal value α of the problem (P ) is assumed to be finite, i.e., α > -∞, which implies dom r 1 ⊂ dom r 2 . Moreover, we assume that dom r 1 ⊂ dom ∂r 2 . This type of problem is common in many areas, particularly machine learning, where G -H represents the data-fitting term (the loss function) and r 1 -r 2 represents the regularization term to encourage some desired properties on the found solutions (for example, low-rank or sparsity) or to model constraints on x. Furthermore, in the age of big data, optimization models are expected to take the large-sum structure into account so as to represent the high volume of data. As a result, the loss function frequently has a (nonconvex) large-sum structure.

Chapter 2. Stochastic DCA with Variance Reduction

In terms of the regularizer, we investigate two cases of r 2 : r 2 is convex (the regularizer is a DC function), or r 2 is a composite function defined by r 2 (x) = m i=1 l i (p i (x)) with l i : R → R being convex, decreasing and p i : R n → R being convex. In the second case, r 2 , in general, is no longer convex. Most nonconvex sparsity-promoting regularizers (usually used to approximate the ℓ 0 norm) are known to have the above composite form of r 2 [START_REF] Le Thi | Feature selection in machine learning: an exact penalty approach using a difference of convex function algorithm[END_REF][START_REF] Ong | Learning sparse classifiers with difference of convex functions algorithms[END_REF], for instance, log-sum penalty [START_REF] Candes | Enhancing sparsity by reweighted l1 minimization[END_REF], smoothly clipped absolute deviate (SCAD) [START_REF] Fan | Variable selection via nonconcave penalized likelihood and its oracle properties[END_REF], capped ℓ 1 [START_REF] Zhang | Analysis of multi-stage convex relaxation for sparse regularization[END_REF], minimax concave penalty [START_REF] Zhang | Nearly unbiased variable selection under minimax concave penalty[END_REF], (nonconcave) piecewise linear function [START_REF] Le | A new approximation for the ℓ 0 -norm[END_REF].

The problem (P ) catches our attention because of two prominent features that play an important role in machine learning and big data analytics: the (nonconvex) DC structure of the objective (on both loss and regularizer) and the large-sum structure of H. Indeed, as machine learning models tend to be more complicated, nonconvex optimization is then indispensable. The nonconvexity is represented here by the DC structure, which is applicable to a wide range of nonconvex optimization problems [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF].

Furthermore, as previously stated, the large-sum structure is one of the most commonly seen forms in practice for modeling massive data-driven problems. The large-sum structure, for example, emerges naturally in empirical risk minimization (ERM) in stochastic programming. That is, for instance, the stochastic problem

min x∈D E(f (x, ξ)),
where ξ is a random variable, can be approximated by

(ERM) min x∈D 1 N N i=1 f (x, ξ i ) + λΩ(x),
where {ξ 1 , ξ 2 , . . . ξ N } is a set of i.i.d. realizations of ξ, and Ω is the regularization term. In particular, when Ω is a convex function, D is a convex set, and f (•, ξ i ) has L-Lipschitz gradients for all i, the problem (ERM) falls into the spectrum of the problem (P ) with G(x) = (L/2)∥x∥ 2 , h i (x) = (L/2)∥x∥ 2 -f (x, ξ i ), i = 1, 2, . . . , N , and r 1 (x) = λΩ(x) + χ D (x), r 2 (x) = 0. Many important applications such as LASSO, principal component analysis, logistic regression, etc., can be expressed in this form.

Related works and our motivation

The two previously mentioned aspects of the problem (P ) also pose a significant challenge. On the one hand, there are very few efficient and scalable algorithms for dealing with nonconvex problems, while on the other hand, deterministic algorithms struggle to manage the large-sum structure. As a result, combining stochastic techniques for dealing with the large-sum structure with an powerful algorithm for nonconvex programming could be an efficient strategy. A stochastic algorithm often incorporates some stochastic approximation techniques based on the framework of a deterministic algorithm. This deterministic frame is critical to the overall quality of a stochastic algorithm and should thus be well suited to the problem's structure. And the stochastic approximation used should be inexpensive and has small noise.

The literature on stochastic optimization in the convex context is vast. In particular, the problem (ERM) with f (•, ξ) and Ω being convex, D = R n , has been studied extensively. Since the influential idea of stochastic gradient descent, which can be traced back to the seminal work [START_REF] Robbins | A stochastic approximation method[END_REF], many variants have been developed, for example, stochastic dual coordinate descent [START_REF] Shalev-Shwartz | Stochastic dual coordinate ascent methods for regularized loss minimization[END_REF], stochastic average gradi-2.1. Introduction ent (SAG) [START_REF] Schmidt | Minimizing finite sums with the stochastic average gradient[END_REF], stochastic variance reduced gradient (SVRG) [START_REF] Johnson | Accelerating stochastic gradient descent using predictive variance reduction[END_REF], SAGA [START_REF] Defazio | SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives[END_REF], StochAstic Recursive grAdient algoritHm (SARAH) [START_REF] Nguyen | SARAH: A novel method for machine learning problems using stochastic recursive gradient[END_REF], etc. Other works studied the (strongly) convex sum function while relaxing the convexity of each component function f (•, ξ) [START_REF] Allen-Zhu | Improved svrg for non-strongly-convex or sum-of-non-convex objectives[END_REF][START_REF] Shalev-Shwartz | Sdca without duality, regularization, and individual convexity[END_REF].

The number of research on nonsmooth, nonconvex large-sum problems remains small. The (ERM) problem with L-smooth f (•, ξ) and (possibly nonsmooth) convex regularizer Ω (which is a special case of (P )) is probably the most prevalent model that has been studied via the proximal-based approach [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF][START_REF] Pham | Proxsarah: An efficient algorithmic framework for stochastic composite nonconvex optimization[END_REF]. Among stochastic methods for solving the (ERM), the prox-SVRG and prox-SAGA [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF] are the most related to our approach since they use the SVRG and SAGA estimators, respectively. Another work based on the Majorization-Minimization (MM) approach was proposed in [START_REF]Incremental majorization-minimization optimization with application to large-scale machine learning[END_REF] which studied a large-sum of nonconvex functions where each function admit surrogates with L-smooth error (i.e. the difference between a function and its surrogates is L-smooth).

The existing body of research for nonsmooth DC large-sum problems is even more limited. Some recent publications [START_REF] Le Thi | Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification[END_REF][START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF][START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF] developed stochastic methods based on DCA (DC Algorithm). The first stochastic DCA was presented in [START_REF] Le Thi | Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification[END_REF] for a large-sum of (nonconvex) L-smooth functions with a DC regularizer, and it was later expanded to a more general problem where the L-smooth condition was relaxed in [START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF]. The stochastic DCA has been constructed in these studies based on the SAG estimator [START_REF] Schmidt | Minimizing finite sums with the stochastic average gradient[END_REF], which is a variance reduction estimator.

Another approach was [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF] in which the data-fitting term is large-sum DC and the regularizer is nonconvex, nonsmooth whose proximal operator can be efficiently computed. The author approximated the regularizer by its Moreau envelope, resulting in a DC program, in which stochastic DCA schemes were developed. In essence, the proposed algorithms can be viewed as the standard DCA where the (large-sum) convex subproblems are solved by stochastic algorithms (e.g., Adagrad [START_REF] Duchi | Adaptive subgradient methods for online learning and stochastic optimization[END_REF], SVRG [START_REF] Johnson | Accelerating stochastic gradient descent using predictive variance reduction[END_REF])

to a certain degree of precision.

Based on the above discussion, we are inspired to develop novel stochastic algorithms based on DCA for solving (P ). In addition, several research studies have demonstrated the theoretical and practical value of stochastic variance-reduced gradients provided by SVRG and SAGA. Consequently, combining them with the DCA could be an effective strategy.

Our contribution We present two stochastic DCA schemes, DCA-SVRG and DCA-SAGA, that incorporate SVRG and SAGA. We investigate two typically used sampling strategies in our design: sampling with replacement and sampling without replacement. Sampling is called with replacement if one sample selected at random from the population is returned to the population, then the next sample is selected in the same manner; hence, it allows repetition. Meanwhile, the sampling without replacement strategy sequentially samples from the population, with each picked unit not being placed back. Algorithms for tackling large-sum problems in the literature primarily use the former sampling method due to its useful property: each sample is independent of the others. However, in practice, the latter is a much more natural method to apply and can occasionally produce significantly superior outcomes (as demonstrated by our numerical tests in Section 2.5). As a result, we give analyses for both sampling procedures.

The convergence of the DCA-SVRG and DCA-SAGA has been thoroughly investigated. The proposed algorithms' almost sure convergence to critical points is established: each accumulating point of the sequence generated by DCA-SVRG and DCA-SAGA is shown to be a DC critical point of

F = (G + r 1 ) -(H + r 2 ). Furthermore, if r 2 is L-smooth, we obtain the O(k -1/2
) convergence rate with respect to the measure of proximity to criticality, where k is the number of iterations. Consequently, to find ϵ-criticality, DCA-SVRG and DCA-SAGA have the complexity of O(N 2/3 /ϵ 2 ) and O(N + N 3/4 /ϵ 2 ) (respectively) in terms of gradient evaluations. Another contribution is the novel arguments used in the analysis of DCA-SAGA, where we introduce an elegant idea of how to escape the non-independence (which is the main difficulty in our analysis) and provide good quantitative links between E∥x t -α t i ∥ 2 and E∥x t -x t-1 ∥ 2 , E∥x t-1 -x t-2 ∥ 2 , etc., where {α t i } N i=1 are SAGA-introduced local auxiliary variables that are used to build variance reduction terms and are updated progressively.

These new arguments should be helpful in analyzing future SAGA-type algorithms.

Moreover, we provide additional convergence analysis to deal with the composite structure of r 2 .

By introducing new optimization variables, z i = p i (x) with i = 1, m, this problem can be reformulated as a DC program in the form of (P ) with respect to a pair of old and new variables. Our goal here is to provide a more practical convergence result by not simply treating this pair of variables as a whole.

Instead, we still concentrate on the primary role of the variable x. This special treatment provides us with more flexible results that can be used in practice while avoiding an undesirable situation when using the original convergence analysis of the proposed algorithms (discussed in Section 2.4). Finally, we perform numerical experiments to thoroughly investigate the behaviors of the proposed methods. Nonnegative principal component analysis, group variable selection in multiclass logistic regression, and sparse linear regression are the real-world problems under consideration.

Comparison with related works In contrast to [START_REF] Le Thi | Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification[END_REF][START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF], we apply the SVRG and SAGA estimators in the construction of convex subproblems, whereas [START_REF] Le Thi | Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification[END_REF][START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF] used the SAG estimator. Because of its averaging feature, the SAG appears to be quite "conservative": at each iteration, it computes the average of b new gradients and N -b kept gradients (b is the minibatch size and N is the number of functions in the large sum) to form a stochastic direction used to construct a subproblem. As a result, if b is small in proportion to N , the old information dominates the new information, causing this stochastic direction to shift slowly from iteration to iteration. In the convex optimization scenario, numerical tests in the publication [START_REF] Defazio | SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives[END_REF] have shown that SVRG and SAGA have some advantages over SAG. As a result, SVRG and SAGA are likely to maintain their advantages in the context of DC programming and DCA.

On the other hand, our proposed algorithms are considerably different from [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF]: their algorithms can be regarded as the deterministic DCA in which stochastic convex solvers are utilized to solve convex subproblems, whereas our techniques stochastically build convex subproblems. Furthermore, in their study, the authors used the SVRG algorithm to solve their subproblem, resulting in the SSDC-SVRG algorithm. Our DCA-SVRG distinguishes itself from the SSDC-SVRG by the fact that the SVRG estimator is utilized as an outer technique to compute stochastic gradients in the DCA-SVRG, whereas the SVRG is used as an inner convex solver in the SSDC-SVRG.

Concerning the proximal-SVRG and the prox-SAGA in [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF] for solving the (ERM) problem where each f (•, ξ i ) is L-smooth and Ω is convex, which is a special case of (P ): the DCA-SVRG via the option with replacement applied to this particular problem with G, h i , r 1 , r 2 defined above recovers the prox-SVRG. Meanwhile, the DCA-SAGA via the option with replacement does not correspond to the prox-SAGA, but rather a related algorithm, where the "stochastic variance-reduced gradient" of prox-SAGA, ∇prox-SAGA , is drifted by another unbiased variance-reduction term: ∇DCA-SAGA = ∇prox-SAGAdrift,

Control variates

where drift = L N N i=1 α t i - L b i∈I α t i ,
with I being a set of random indexes. Furthermore, in the prox-SAGA step of updating the table of gradients, the authors do not update directly on I. Instead, they draw another independent set of indexes J, which mostly serves to facilitate their theoretical analysis. This procedure additionally consumes -in the worst scenario -b gradients computation, where b is the minibatch size. In contrast, the DCA-SAGA does not need to employ another set J, rather, we just update the table of gradients using I directly, omitting the extra computations.

Control variates

We begin with an introduction to control variates, a fundamental concept underlying recent advances in variance reduction techniques in machine learning. The following discussion is a variant of the discussion in [START_REF] Defazio | SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives[END_REF]. The method of control variates [START_REF] Lavenberg | Statistical results on control variables with application to queueing network simulation[END_REF][START_REF] Lavenberg | A perspective on the use of control variables to increase the efficiency of monte carlo simulations[END_REF] is a variance reduction technique that is widely used in Monte Carlo simulation. Let µ be an unknown parameter of interest and X be an unbiased estimator of µ, E(X) = µ. In order for X to be a good estimator of µ, the variance of X, Var(X), should be small. Suppose there is a random variable Y with known mean E(Y ) = τ . Consider a random variable defined as follows

Z = X + c(τ -Y ),
where c > 0 is some constant. It is observed that Z is an unbiased estimator of µ since E(Z) = E(X) = µ. The variance of Z is given by

Var(Z) = Var(X) + c 2 Var(Y ) -2c Cov(X, Y ). (2.1)
Hence, if X and Y are highly correlated in the sense that either 2 Cov(X, Y ) > Var(Y ) (then Var(Z) < Var(X) for c ≤ 1) or one can choose small c such that c 2 Var(Y ) < 2c Cov(X, Y ) (note that c 2 is one degree of smallness away from c, so this inequality is possible when Cov(X, Y ) > 0), all of which lead to Var(Z) < Var(X).

When X and Y have positive correlation, as a well-known fact, the optimal value of c minimizing the variance of Z is (the minimizer of the parabola (2.1))

c * = Cov(X, Y ) Var(Y ) .
With such a value of c,

Var(Z) = Var(X) - Cov(X, Y ) 2 Var(Y ) = 1 -Corr(X, Y ) 2 Var(X) < Var(X),
where Corr(X, Y ) is the correlation between X and Y ,

Corr(X, Y ) = Cov(X, Y )
Var(X) Var(Y ) .

However, such a perfect c * is hard to compute in practice. Therefore, the best scenario is that X, Y are highly correlated, and then Z serves as a variance-reduced unbiased estimator for µ even when c is not very carefully chosen.

In machine learning, the quantity of interest is often the gradient of a large-sum objective function, say, at some point x k ,

µ := 1 N N i=1 ∇f i (x k ). A natural estimator for µ is X = 1 b i∈I ∇f i (x k )
, where I is a set of b indexes randomly drawn from {1, 2, . . . , N }. Now, let x be point near to x k , the random variable Y := 1 b i∈I ∇f i (x) is expected to be highly correlated with X since x k and x are not too far from each other so they can share some common geometry of the objective. Therefore, the following unbiased estimator

Z := 1 b i∈I ∇f i (x k ) - 1 b i∈I ∇f i (x) + 1 N N i=1 ∇f i (x)
is expected to have smaller variance than that of X. The question is that what is the point of using Z when we have to compute the expensive full gradient N -1 N i=1 ∇f i (x)? The point is that once N -1 N i=1 ∇f i (x) is computed, it can be used in the update of several x k , in which only 2b gradient computations are needed for each update. Then, x is updated, from time to time, accordingly to the progress of the sequence {x k } so as to make sure it is relevant in terms of variance reduction. This leads to the creation of a technique known as stochastic variance reduction gradient (SVRG) [START_REF] Johnson | Accelerating stochastic gradient descent using predictive variance reduction[END_REF].

Another technique named SAGA [START_REF] Defazio | SAGA: A fast incremental gradient method with support for non-strongly convex composite objectives[END_REF] for using control variates in computing µ is to utilize a table of references {α k 1 , α k 2 , . . . , α k N } where each α k i is among the set {x 0 , x 1 , . . . , x k-1 }. The random variable Y := b -1 i∈I ∇f i (α k i ) is expected to be highly correlated with X, so the following unbiased estimator

Z := 1 b i∈I ∇f i (x k ) - 1 b i∈I ∇f i (α k i ) + 1 N N i=1 ∇f i (α k i )
is expected to have small variance. The key is that one always keeps the table of gradients

{∇f i (α k i )} N i=1
in the memory so that N -1 N i=1 ∇f i (α k i ) is ready to use, and the table is updated progressively to ensure the high correlation between X and Y .

Stochastic DCA with Variance Reduction

Throughout this section, we study the problem (P ) with r 2 being a convex function.

The first stochastic DCA: DCA-SVRG

In this part, we design the first stochastic DCA, DCA-SVRG, to solve the problem (P ). The approach can be thought of as a combination of deterministic DCA and SVRG-style gradient updating. That is, in the deterministic DCA, we substitute the gradient of H with the "stochastic variance-reduced gradient" of H. The technique is epoch-based, with the full gradient of H computed at the start of each epoch and used as a variance-reduction term inside that epoch.

To build the stochastic variance reduction gradient of H, we can sample in one of two ways: with or without replacement. If sampling is with replacement, a set of indexes I b = {i 1 , i 2 , . . . , i b } is, at each iteration, randomly chosen from {1, 2, . . . , N } where each i j is independent of the others. Otherwise, the repetition in I b is not permitted. On the other hand, since a critical point

x * of F = (G + r 1 ) -(H + r 2 ) satisfies dist(∂(G + r 1 )(x * ), ∂(H + r 2 )(x *
)) = 0, we define the measure of proximity to criticality as follows

d K = min k=0,1,...,K-1,j=0,1,...,M -1 E dist(∂(H + r 2 )(x k+1 j+1 ), ∂(G + r 1 )(x k+1 j+1 )).
For brevity, we denote xk = {x k 0 , x k 1 , . . . , x k M -1 } and ȳk = {y k 0 , y k 1 , . . . , y k M -1 } for all k ∈ N * . Let P k+1 j be the σ-algebra defined as

P k+1 j = σ(x
ing b = ⌊N 2/3 ⌋, M = ⌊µ √ b⌋, where µ = 1 4 √ e -1 ρ G+r 1 + ρ H + ρ r 2 L ,
the complexity in terms of the number of gradient evaluations to obtain ϵ-criticality in expectation is O(N 2/3 /ϵ 2 ); Meanwhile, the complexity in terms of the number of convex subproblems being

solved is O(1/ϵ 2 ).
Because the proofs for the two sampling choices largely overlap, we shall prove Theorem 6 concurrently for both options with and without replacement. We will emphasize parts of the proof that must be studied differently for each option. Before proving Theorem 6, we first introduce the following lemma for the option sampling without replacement.

Lemma 1. For the option of sampling without replacement, the following inequality holds

E(∥t k+1 j -∇H(x k+1 j )∥ 2 |P k+1 j ) ≤ L 2 b 1 - b -1 N -1 ∥x k+1 j -xk ∥ 2 ≤ L 2 b ∥x k+1 j -xk ∥ 2 .
Proof of Lemma 1. We have,

E   1 b i∈I b ∇h i (x k+1 j ) - 1 b i∈I b ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2 |P k+1 j   = E I b 1 b i∈I b ∇h i (x k+1 j ) - 1 b i∈I b ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2 = I⊂[N ],|I|=b P(I b = I) 1 b i∈I ∇h i (x k+1 j ) - 1 b i∈I ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2 ,
where the probability P(I b = I) = 1/C b N .

Stochastic DCA with Variance Reduction

For a fixed I, we compute

R = 1 b i∈I ∇h i (x k+1 j ) - 1 b i∈I ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2 = 1 b 2 i∈I ∥∇h i (x k+1 j ) -∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j )∥ 2 + 1 b 2 i̸ =l,i,l∈I ⟨∇h i (x k+1 j ) -∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ), ∇h l (x k+1 j ) -∇h l (x k ) + ∇H(x k ) -∇H(x k+1 j )⟩ := 1 b 2 (R 1 + R 2 ). R 1 = i∈I ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 + b∥∇H(x k ) -∇H(x k+1 j )∥ 2 + 2⟨∇H(x k ) -∇H(x k+1 j ), i∈I (∇h i (x k+1 j ) -∇h i (x k ))⟩, R 2 = A 2 b ∥∇H(x k ) -∇H(x k+1 j )∥ 2 + ⟨∇H(x k ) -∇H(x k+1 j ),
i̸ =l,i,l∈I

(∇h i (x k+1 j ) -∇h i (x k ) + ∇h l (x k+1 j ) -∇h l (x k ))⟩ + i̸ =l,i,l∈I ⟨∇h i (x k+1 j ) -∇h i (x k ), ∇h l (x k+1 j ) -∇h l (x k )⟩.
Therefore,

R = 1 b 2 i∈I ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 + 2 b ⟨∇H(x k ) -∇H(x k+1 j ), i∈I (∇h i (x k+1 j ) -∇h i (x k ))⟩ + ∥∇H(x k ) -∇H(x k+1 j )∥ 2 + 1 b 2 i̸ =l,i,l∈I ⟨∇h i (x k+1 j ) -∇h i (x k ), ∇h l (x k+1 j ) -∇h l (x k )⟩.
On the other hand, we have the following computations

I⊂[N ],|I|=b i∈I ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 = C b-1 N -1 N i=1 ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 , I⊂[N ],|I|=b ⟨∇H(x k ) -∇H(x k+1 j ), i∈I (∇h i (x k+1 j ) -∇h i (x k ))⟩ = -C b-1 N -1 • N • ∥∇H(x k ) -∇H(x k+1 j )∥ 2 , I⊂[N ],|I|=b i̸ =l,i,l∈I ⟨∇h i (x k+1 j ) -∇h i (x k ), ∇h l (x k+1 j ) -∇h l (x k )⟩ = C b-2 N -2 i̸ =j ⟨∇h i (x k+1 j ) -∇h i (x k ), ∇h l (x k+1 j ) -∇h l (x k )⟩.
Therefore,

E I b 1 b i∈I b ∇h i (x k+1 j ) - 1 b i∈I b ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2 = 1 bN N i=1 ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 -∥∇H(x k ) -∇H(x k+1 j )∥ 2 + b -1 bN (N -1) i̸ =l ⟨∇h i (x k+1 j ) -∇h i (x k ), ∇h l (x k+1 j ) -∇h l (x k )⟩ = 1 bN 1 - b -1 N -1 N i=1 ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 + N (b -1) b(N -1) -1 ∥∇H(x k ) -∇H(x k+1 j )∥ 2 ≤ L 2 b 1 - b -1 N -1 ∥x k+1 j -xk ∥ 2 .
Now we prove Theorem 6 as follows.

Proof of Theorem 6. 1. Since H is ρ H -convex and by the definition of x k+1 j+1 ,

H(x k+1 j+1 ) ≥ H(x k+1 j ) + ⟨∇H(x k+1 j ), x k+1 j+1 -x k+1 j ⟩ + ρ H 2 ∥x k+1 j+1 -x k+1 j ∥ 2 , r 2 (x k+1 j+1 ) ≥ r 2 (x k+1 j ) + ⟨y k+1 j , x k+1 j+1 -x k+1 j ⟩ + ρ r 2 2 ∥x k+1 j+1 -x k+1 j ∥ 2 .
It follows from the definition of x k+1 j+1 that

(G + r 1 )(x k+1 j ) ≥ (G + r 1 )(x k+1 j+1 ) + ⟨z k+1 j , x k+1 j -x k+1 j+1 ⟩ + ρ G+r 1 2 ∥x k+1 j -x k+1 j+1 ∥ 2 .
These inequalities imply

F (x k+1 j+1 ) ≤ F (x k+1 j ) + ⟨x k+1 j+1 -x k+1 j , t k+1 j -∇H(x k+1 j )⟩ - ρ 2 ∥x k+1 j+1 -x k+1 j ∥ 2 ,
where ρ = ρ H + ρ r 2 + ρ G+r 1 . Let γ > 0 that will be determined later. By applying Schwartz inequality and AM-GM inequality,

F (x k+1 j+1 ) ≤ F (x k+1 j ) + 1 2γ ∥t k+1 j -∇H(x k+1 j )∥ 2 - ρ -γ 2 ∥x k+1 j+1 -x k+1 j ∥ 2 .
By taking conditional expectation with respect to P k+1 j ,

E(F (x k+1 j+1 )|P k+1 j ) ≤ F (x k+1 j ) - ρ -γ 2 E(∥x k+1 j+1 -x k+1 j ∥ 2 |P k+1 j ) + 1 2γ E(∥t k+1 j -∇H(x k+1 j )∥ 2 |P k+1 j
).

If option is sampling with replacement, the set I b consists of independent indexes, we therefore evaluate ) as follows,

E(∥t k+1 j -∇H(x k+1 j )∥ 2 |P k+1 j ) = E I b   1 b i∈I b ∇h i (x k+1 j ) - 1 b i∈I b ∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2   = 1 b E i ∇h i (x k+1 j ) -∇h i (x k ) + ∇H(x k ) -∇H(x k+1 j ) 2
, where i uni

∼ [N ] ≤ 1 b E i ∥∇h i (x k+1 j ) -∇h i (x k )∥ 2 ≤ L 2 b ∥x k+1 j -xk ∥ 2 . (2.2)
Together with Lemma 1, the following inequality holds for two options

E(∥t k+1 j -∇H(x k+1 j )∥ 2 |P k+1 j ) ≤ L 2 b ∥x k+1 j -xk ∥ 2 . (2.3)
Therefore,

E(F (x k+1 j+1 )|P k+1 j ) ≤ F (x k+1 j ) + L 2 2bγ ∥x k+1 j -xk ∥ 2 - ρ -γ 2 E(∥x k+1 j+1 -x k+1 j ∥ 2 |P k+1 j ).
Consider the sequence of Lyapunov functions

V k+1 j = F (x k+1 j ) + c j ∥x k+1 j -xk ∥ 2
, where {c j } are non-negative numbers determined later (the idea of such sequence of Lyapunov functions is adopted from [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF]). We have

E(V k+1 j+1 |P k+1 j ) = E(F (x k+1 j+1 ) + c j+1 ∥x k+1 j+1 -xk ∥ 2 |P k+1 j ) ≤ E(F (x k+1 j+1 ) + c j+1 (1 + β)∥x k+1 j+1 -x k+1 j ∥ 2 + c j+1 (1 + 1/β)∥x k+1 j -xk ∥ 2 |P k+1 j ), where β > 0 ≤ F (x k+1 j ) + L 2 2bγ + c j+1 1 + 1 β ∥x k+1 j -xk ∥ 2 + c j+1 (1 + β) - ρ -γ 2 E(∥x k+1 j+1 -x k+1 j ∥ 2 |P k+1 j ). (2.4)
To obtain V k+1 j in the right-hand side of (2.4), we choose the sequence {c j } in such a way c M = 0

and c j = L 2 2bγ + c j+1 1 + 1 β if j < M. These relations yield c j = βL 2 2bγ 1 + 1 β M -j -1 . Next,
to achieve the descent property on the Lyapunov sequence, i.e.

E(V k+1 j+1 |P k+1 j ) < V k+1 j , we want c j+1 (1 + β) + γ 2 ≤ ρ 4 , ∀j = 0, M -1, or equivalently, (1 + β) βL 2 2bγ 1 + 1 β M -j-1 -1 + γ 2 ≤ ρ 4 , ∀j = 0, M -1,
which is further equivalent to

(1 + β) βL 2 2bγ 1 + 1 β M -1 -1 + γ 2 ≤ ρ 4 . (2.5)
By choosing β = M -1, and noticing that 1

+ 1 M -1 M -1 ≤ e and (1 + β)β < (1 + β) 2 = M 2 , (2.5)
holds if the following stronger inequality holds

M 2 L 2 2bγ (e -1) + γ 2 ≤ ρ 4 . (2.6) Now by choosing γ = M L √ e-1 √ b
to optimize the LHS of (2.6), the inequality (2.6) becomes

M √ b ≤ ρ 4L √ e-1 .
Consequently, if the minibatch size b and the inner-loop length M satisfy this inequality, together with (2.4) we get

E(V k+1 j+1 |P k+1 j ) ≤ V k+1 j - ρ 4 E(∥x k+1 j+1 -x k+1 j ∥ 2 |P k+1 j
).

(2.7)

From the definitions of V k+1 j and P k+1 j , it is observed that

V k+1 j is P k+1 j -measurable, for k ∈ N, j ∈ {0, . . . , M -1}.
Moreover, it can be verified that P k+1 M -1 ⊂ P k+2 0 , ∀k ∈ N. Therefore, we have the following filtration

P 1 0 Q 0 ⊂ P 1 1 Q 1 ⊂ . . . ⊂ P 1 M -1 Q M -1 ⊂ P 2 0 Q M ⊂ P 2 1 Q M +1 ⊂ . . . ,
in which the following sequence is an adapted process:

V 1 0 W 0 , V 1 1 W 1 , . . . , V 1 M -1 W M -1 , V 2 0 W M , V 2 1 W M +1
, . . .

On the other hand, it is observed that V k+1 M = V k+2 0 , ∀k ∈ N. So by plugging j = M -1 to (3.15),
we obtain

E(V k+2 0 |P k+1 M -1 ) ≤ V k+1 M -1 - ρ 4 E(∥x k+1 M -x k+1 M -1 ∥ 2 |P k+1 M -1 ). (2.8)
From (3.15), (2.8), and the fact that

V k+1 j ≥ F (x k+1 j ) ≥ α, ∀k ∈ N, j ∈ {0, 1, . . . , M -1}, we can
apply the supermartingale convergence theorem [START_REF] Bertsekas | Convex Analysis and Optimization[END_REF]: there exists a random variable

V ∞ such that the sequence {W k } converges to V ∞ almost surely. As a consequence, the sequence {F (x k )} converges to V ∞ almost surely since V k+1 0 = F (x k+1 0 ) = F (x k ). 2. From (3.15), we obtain ρ 4 E∥x k+1 j+1 -x k+1 j ∥ 2 + E(V k+1 j+1 ) ≤ E(V k+1 j
). From this inequality and the fact that

E(V 1 0 ) = E(F (x 0 )) = F (x 0 ) < +∞, by induction, one obtains E(V k+1 j
) < +∞, for all k, j. By telescoping with j and k, ρ 4

∞ k=0 M -1 j=0 E∥x k+1 j+1 -x k+1 j ∥ 2 ≤ E(V 1 0 ) -α < +∞.
(2.9)

3. From the evaluation (2.3), we obtain

E t k+1 j -∇H(x k+1 j ) 2 ≤ L 2 (M -1) b E∥x k+1 1 -x k+1 0 ∥ 2 + . . . + E∥x k+1 M -1 -x k+1 M -2 ∥ 2 , which implies ∞ k=0 M -1 j=0 E t k+1 j -∇H(x k+1 j ) 2 < +∞. (2.10) Let S 1 = t k+1 j -∇H(x k+1 j ) → 0, ∀j = 0, M -1 .
It follows from (2.10) that P(S 1 ) = 1. On the other hand, let S 2 = x k+1 j+1 -x k+1 j → 0, ∀j = 0, M -1 , it follows from (2.9) that P(S 2 ) = 1. Furthermore, P(S 3 ) = 1, where S 3 = (y k j is bounded, ∀j = 1, M -1).

Now, consider an event in S 1 ∩ S 2 ∩ S 3 which gives rise to a realization {x k j } (here, {x k j } is a sequence of real numbers rather than a sequence of random variables). Let x * be a limit point of {x k j }. Note that the sequence {x k j } generated by the algorithm is expressed explicitly in the following order

x 1 0 , x 1 1 , . . . , x 1 M -1 , x 1 M ≡ x 2 0 , x 2 1 , . . . , x 2 M -1 , x 2 M ≡ x 3 0 , x 3 1 , . . .
We observe that a subsequence of {x k j } has the following form {x

u(l) v(l) } l∈N , where u : N → N * and v : N → {0, 1, . . . , M -1} satisfy the following relation: for every i < t in N, either u(i) < u(t) or u(i) = u(t) and v(i) < v(t). Since x * is a limit point of {x k j }, there exists a subsequence {x u(l) v(l) } converging to x * as l → ∞, which implies x u(l) v(l)+1 → x * . As a consequence, t u(l) v(l) → ∇H(x * ).
By passing to a subsequence if necessary, we assume that y

u(l) v(l) converges to y * . Since y u(l) v(l) ∈ ∂r 2 x u(l) v(l) , we obtain y * ∈ ∂r 2 (x * )
∈ ∂(G + r 1 )(x * ), hence x * is a critical point of F = (G + r 1 ) -(H + r 2 ). 4. We first evaluate E dist(∇H(x k+1 j+1 ) + ∇r 2 (x k+1 j+1 ), ∂(G + r 1 )(x k+1 j+1 )). Since z k+1 j ∈ ∂(G + r 1 )(x k+1 j+1 ), we obtain E dist(∇H(x k+1 j+1 ) + ∇r 2 (x k+1 j+1 ), ∂(G + r 1 )(x k+1 j+1 )) ≤ E∥∇H(x k+1 j+1 ) -t k+1 j ∥ + E∥∇r 2 (x k+1 j+1 ) -∇r 2 (x k+1 j )∥ ≤ E∥∇H(x k+1 j+1 ) -∇H(x k+1 j )∥ + E∥∇H(x k+1 j ) -t k+1 j ∥ + E∥∇r 2 (x k+1 j+1 ) -∇r 2 (x k+1 j )∥ ≤ (L + L r 2 )E∥x k+1 j+1 -x k+1 j ∥ + E∥t k+1 j -∇H(x k+1 j )∥ 2 1 2 ≤ (L + L r 2 )E∥x k+1 j+1 -x k+1 j ∥ + L √ b E∥x k+1 j -xk ∥ 2 1 2 .
(2.11) Furthermore, we have

∥x k+1 j -xk ∥ 2 = ∥x k+1 j -x k+1 0 ∥ 2 ≤ (M -j + M -j -1 + . . . + M -1) × 1 M -j ∥x k+1 j -x k+1 j-1 ∥ 2 + . . . + 1 M -1 ∥x k+1 1 -x k+1 0 ∥ 2 .
Chapter 2. Stochastic DCA with Variance Reduction Therefore,

M -1 j=1 E∥x k+1 j -xk ∥ 2 1 2 ≤ M -1 j=1 j r=1 (M -r) 1 2 j r=1 1 M -r E∥x k+1 r -x k+1 r-1 ∥ 2 1 2 ≤   M -1 j=1 j r=1 (M -r)   1 2   M -1 j=1 j r=1 1 M -r E∥x k+1 r -x k+1 r-1 ∥ 2   1 2 = (M -1)M (2M -1) 6
1 2 M -1 r=1 E∥x k+1 r -x k+1 r-1 ∥ 2 1 2
.

(2.12)

It follows from (2.11) and (2.12) that

K-1 k=0 M -1 j=0 E dist(∇H(x k+1 j+1 ) + ∇r 2 (x k+1 j+1 ), ∂(G + r 1 )(x k+1 j+1 )) ≤ (L + L r 2 ) K-1 k=0 M -1 j=0 E∥x k+1 j+1 -x k+1 j ∥ + L √ b (M -1)M (2M -1) 6
1 2 K-1 k=0 M -1 r=1 E∥x k+1 r -x k+1 r-1 ∥ 2 1 2 ≤ √ K (L + L r 2 ) √ M + L √ b (M -1)M (2M -1) 6 1 2   K-1 k=0 M -1 j=0 E∥x k+1 j+1 -x k+1 j ∥ 2   1 2
.

As a consequence, the iteration convergence rate is given by d

K = O(1/ √ K).
Now we derive the complexity to find ϵ-criticality as follows. After K iterations, the output x a is chosen uniformly from {x k+1 j+1 } k=0,...,K-1 j=0,...,M -1 , it holds, for some C > 0,

KM • E dist(∇H(x a ) + ∇r 2 (x a ), ∂(G + r 1 )(x a )) ≤ √ K (L + L r 2 ) √ M + L √ b (M -1)M (2M -1) 6 1 2 C. Let's choose M = ⌊µ √ b⌋, where µ = 1 4 √ e -1 ρ G+r 1 + ρ H + ρ r 2 L one gets KM • E dist(∇H(x a ) + ∇r 2 (x a ), ∂(G + r 1 )(x a )) ≤ C √ K (L + L r 2 ) √ M + L 2 √ b M 3/2 ≤ C √ K (L + L r 2 ) √ M + Lµ 2 √ M ≤ C √ M K L + L r 2 + Lµ 2 .
Therefore,

E dist(∇H(x a ) + ∇r 2 (x a ), ∂(G + r 1 )(x a )) = O 1 √ KM .
By choosing b = ⌊N 2/3 ⌋, the algorithm needs

K = O( 1 ϵ 2 N 1/3
) outer iterations to attain an ϵ-DC critical point. The total number of gradient evaluations is then:

K (N + 2bM ) ≤ K(N + 2N 2/3 µN 1/3 ) = KN (1 + 2µ) = O N 2/3 ϵ 2 .
On the other hand, there are KM convex subproblems to be solved, leading to the complexity of O(1/ϵ 2 ) in terms of the number of convex subproblems.

γ 2 ∥ • ∥ 2 (γ > 0) to both DC components, F = G + r 1 + γ 2 ∥ • ∥ 2 - H + r 2 + γ 2 ∥ • ∥ 2 . With this new DC decomposition ( Ḡ = G, H = H, r1 = r 1 + γ 2 ∥ • ∥ 2 , r2 = r 2 + γ 2 ∥ • ∥ 2 )
, the larger value is obtained since

ρ L = ρ + 2γ L → +∞ as γ → +∞.
However, adopting the above technique should be done with caution because adding a strongly convex term to both DC components increases the "gap" between the second DC component and its linear minorant, which can lead to bad approximations. We denote

d T = min t=0,1,...,T -1 E dist(∂(H + r 2 )(x t+1 ), ∂(G + r 1 )(x t+1 )).
For brevity, we de-

note ᾱt = {α t 1 , α t 2 , . . . , α t N }, ∀t ∈ N.
We denote the sequence of increasing σ-algebra {P t } as P t = σ(x 0 , x 1 , . . . , x t , y 0 , y 1 , . . . , y t , ᾱ0 , ᾱ1 , . . . , ᾱt ). The convergence results of DCA-SAGA is described as follows.

Theorem 7. If the minibatch size b satisfies N √ N +b b 2 ≤ ρ H +ρr 2 +ρ G+r 1 4L
for the option of sampling with

replacement or N √ N +1 b 2 ≤ ρ H +ρr 2 +ρ G+r 1 4L
for the option of sampling without replacement, then 1.

∞ t=0 E∥x t+1 -x t ∥ 2 < ∞. 2. ∞ t=0 N i=1 E∥x t -α t i ∥ 2 < ∞.
3. The sequence {F (x t )} converges almost surely.

4. Suppose the sequence {y t } is bounded almost surely. Then, every limit point of {x t } is a critical

point of F = (G + r 1 ) -(H + r 2 ) almost surely.
Algorithm 2 DCA-SAGA Initialization: x 0 ∈ dom r 1 , set α 0 i = x 0 , ∀i = 1, N , t = 0, option (either with replacement or without replacement).

Compute the full gradient ν

0 = 1 N N i=1 ∇h i (α 0 i ). repeat if option is with replacement then Randomly choose with replacement the set I t of b elements of [N ]. else
Randomly choose without replacement the set

I t of b elements of [N ]. end if Compute the "stochastic variance reduced gradient" v t , v t = 1 b i∈It ∇h i (x t ) + ν t - 1 b i∈It ∇h i (α t i ).
Compute y t ∈ ∂r 2 (x t ), and let

z t = v t + y t . Solve the convex program x t+1 ∈ arg min{G(x) + r 1 (x) -⟨z t , x⟩ : x ∈ R n }. Update α t+1 j = x t , if j ∈ I t α t j , otherwise Update the "full gradient" ν t+1 = ν t -1 N i∈It ∇h i (α t i ) + 1 N i∈It ∇h i (α t+1 i ). Update t = t + 1. until Stopping criterion. 5. If r 2 has Lipschitz continuous gradient over dom r 1 , then d T = O(1/ √ T ). Furthermore, by choosing b = 4 √ 2 √ µ N 3/4 , where µ = ρ H + ρ r 2 + ρ G+r 1 4L ,
the complexity in terms of number of gradient evaluations to obtain ϵ-criticality in expectation is O(N + N 3/4 /ϵ 2 ); Meanwhile, the complexity in terms of number of convex subproblems solved

is O(1/ϵ 2 ).
Proof of Theorem 7. First, we prove the above properties for the sampling with replacement option. The proof for the sampling without replacement option is then outlined, where the key different arguments are highlighted.

Consider the sampling with replacement option.

1. Since H is ρ H -convex, r 2 is ρ r 2 -convex,
and by the definition of x t+1 ,

H(x t+1 ) ≥ H(x t ) + ⟨∇H(x t ), x t+1 -x t ⟩ + ρ H 2 ∥x t+1 -x t ∥ 2 . r 2 (x t+1 ) ≥ r 2 (x t ) + ⟨y t , x t+1 -x t ⟩ + ρ r 2 2 ∥x t+1 -x t ∥ 2 , (G + r 1 )(x t+1 ) ≤ (G + r 1 )(x t ) + ⟨z t , x t+1 -x t ⟩ - ρ G+r 1 2 ∥x t+1 -x t ∥ 2 .

Stochastic DCA with Variance Reduction

Combining these inequalities,

F (x t+1 ) ≤ F (x t ) + ⟨x t+1 -x t , v t -∇H(x t )⟩ - ρ 2 ∥x t+1 -x t ∥ 2 , (2.13)
where ρ = ρ H + ρ r 2 + ρ G+r 1 . By taking conditional expectation with respect to P t ,

E(F (x t+1 )|P t ) ≤ F (x t ) + E(⟨x t+1 -x t , v t -∇H(x t )⟩|P t ) - ρ 2 E(∥x t+1 -x t ∥ 2 |P t ),
or can be rewritten as follows to address the expectation is taken over the randomness of I t ,

E It (F (x t+1 )) ≤ F (x t ) + E It ⟨x t+1 -x t , v t -∇H(x t )⟩ - ρ 2 E It ∥x t+1 -x t ∥ 2 ,
which implies

E It (F (x t+1 )) ≤ F (x t ) + 1 2γ E It ∥v t -∇H(x t )∥ 2 - ρ -γ 2 E It ∥x t+1 -x t ∥ 2 (2.14)
where γ > 0 that will be determined later to gain some advantages.

We now evaluate E It ∥v t -∇H(x t )∥ 2 as follows,

E It ∥v t -∇H(x t )∥ 2 = E It 1 b i∈It ∇h i (x t ) + ν t - 1 b i∈It ∇h i (α t i ) -∇H(x t ) 2 = 1 b E i ∇h i (x t ) -∇h i (α t i ) + ν t -∇H(x t ) 2 where i uni ∼ {1, 2, . . . , N } ≤ 1 b E i ∥∇h i (x t ) -∇h i (α t i )∥ 2 ≤ L 2 b E i ∥x t -α t i ∥ 2 = L 2 bN N i=1 ∥x t -α t i ∥ 2 . (2.15)
Therefore,

E It (F (x t+1 )) ≤ F (x t ) - ρ -γ 2 E It ∥x t+1 -x t ∥ 2 + L 2 2γbN N i=1 ∥x t -α t i ∥ 2 .
We define the sequence of Lyapunov functions

V t = F (x t ) + c t N i=1 ∥x t -α t i ∥ 2
, where the sequence {c t } will be determined later. For all β > 0,

E It (V t+1 ) = E It F (x t+1 ) + c t+1 N i=1 ∥x t+1 -α t+1 i ∥ 2 ≤ E It F (x t+1 ) + c t+1 (β + 1) N i=1 ∥x t+1 -x t ∥ 2 + c t+1 (1 + 1/β) N i=1 ∥x t -α t+1 i ∥ 2 ≤ E It F (x t ) + N c t+1 (β + 1) - ρ -γ 2 ∥x t+1 -x t ∥ 2 + L 2 2γbN N i=1 ∥x t -α t i ∥ 2 + c t+1 (1 + 1/β) i / ∈It ∥x t -α t i ∥ 2   = F (x t ) + N c t+1 (β + 1) - ρ -γ 2 E It ∥x t+1 -x t ∥ 2 + L 2 2γbN + c t+1 (1 + 1/β) N i=1 ∥x t -α t i ∥ 2 -c t+1 1 + 1 β bE i ∥x t -α t i ∥ 2 = F (x t ) + N c t+1 (β + 1) - ρ -γ 2 E It ∥x t+1 -x t ∥ 2 + L 2 2γbN + c t+1 1 + 1 β 1 - b N N i=1 ∥x t -α t i ∥ 2 .
Similar to the proof of DCA-SVRG, we set

c t = L 2 2γbN + c t+1 1 + 1 β 1 -b N , or equivalently, c t+1 = c t -L 2 2γbN βN (β+1)(N -b)
. We obtain by recursion that

c t = c 0 - L 2 β 2γb(βb + b -N ) βN (β + 1)(N -b) t + L 2 β 2γb(βb + b -N ) .
From the above recursion, if we choose the initial value c 0 =

L 2 β 2γb(βb+b-N ) , we will get c 0 = c 1 = c 2 = . . . = c t = . . . Next, we choose β such that βb+b-N > 0 ⇔ β > N -b b (⋆) to make the sequence {c t } nonnegative. With this choice, we obtain E It (V t+1 ) ≤ V t + N c t+1 (β + 1) -ρ-γ 2 E It ∥x t+1 -x t ∥ 2 .
In order to obtain the "decreasing property" of the sequence {V t }, we want the term N c t+1 (β +1)-ρ-γ 2 to be negative. More specifically, we want N c t+1 (β + 1) + γ 2 ≤ ρ 4 , or equivalently,

N (β + 1) L 2 β 2γb(βb + b -N ) + γ 2 ≤ ρ 4 . (2.16)
Now by choosing γ > 0 to make the LHS of (2.16) as small as possible (AM-GM inequality),

γ = L β(β+1)N b(βb+b-N ) , we need N (β+1)L 2 β b(βb+b-N ) ≤ ρ 4 , or equivalently N (β + 1)β b(βb + b -N ) ≤ ρ 4L . (2.17)
From (⋆), we choose β = N/b. The inequality (2.17) becomes

N √ N + b b 2 ≤ ρ 4L . (2.18)
Consequently, if b satisfies (2.18), we obtain

E It (V t+1 ) ≤ V t -ρ 4 E It ∥x t+1 -x t ∥ 2 . Since c t ≥ 0, we obtain V t ≥ F (x t ) ≥ α > -∞.
By applying supermartingale convergence theorem, we obtain: there

exits V ∞ such that V t → V ∞ a.s. Next, we have ρ 4 E∥x t+1 -x t ∥ 2 ≤ E(V t ) -E(V t+1
). By induction based on this inequality, we obtain E(V t ) is finite for all t. By telescoping the above inequality, we

derive ρ 4 ∞ t=0 E∥x t+1 -x t ∥ 2 ≤ E(V 0 ) -α < +∞.
2. We now evaluate the term E∥x t -α t i ∥ 2 . This is a key for us to establish the convergence to critical points. By the definition, we can write

α t i = x t-1 • 1 {i∈I t-1 } + α t-1 i • 1 {i / ∈I t-1 }
. Similarly, we can track back one further step

α t i = x t-1 • 1 {i∈I t-1 } + x t-2 • 1 {i∈ Īt-1 ∩I t-2 } + α t-2 i • 1 {i∈ Īt-1 ∩ Īt-2 } , where Ī is the complement of I, Ī = {1, 2, . . . , N } \ I.
By doing so repeatedly,

α t i =x t-1 • 1 {i∈I t-1 } + x t-2 • 1 {i∈ Īt-1 ∩I t-2 } + x t-3 • 1 {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } + . . . + x 0 • 1 {i∈ Īt-1 ∩ Īt-2 •••∩ Ī1 } . It is worth noting that {i ∈ I t-1 }, {i ∈ Īt-1 ∩ I t-2 }, {i ∈ Īt-1 ∩ Īt-2 ∩ I t-3 }, . . . , {i ∈ Īt-1 ∩ Īt-2 • • • ∩ Ī1
} form a partition of the sample space Ω, and we obtain

α t i =                      x t-1 on {i ∈ I t-1 } x t-2 on {i ∈ Īt-1 ∩ I t-2 } x t-3 on {i ∈ Īt-1 ∩ Īt-2 ∩ I t-3 } • • • x 0 on {i ∈ Īt-1 ∩ Īt-2 • • • ∩ Ī1 }.
Therefore,

E∥x t -α t i ∥ 2 = Ω ∥x t -α t i ∥ 2 dP = {i∈I t-1 } ∥x t -x t-1 ∥ 2 dP + {i∈ Īt-1 ∩I t-2 } ∥x t -x t-2 ∥ 2 dP + {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } ∥x t -x t-3 ∥ 2 dP + . . . + {i∈ Īt-1 ∩ Īt-2 •••∩ Ī1 } ∥x t -x 0 ∥ 2 dP. (2.19)
Our main aim is to prove ∞ t=1 E∥x t -α t i ∥ 2 < +∞. Here we have several remarks playing as guiding light in our proof:

• We already proved ∞ t=0 E∥x t+1 -x t ∥ 2 < +∞, therefore, we will find links between E∥x t -

α t i ∥ 2 and E∥x t -x t-1 ∥ 2 , E∥x t-1 -x t-2 ∥ 2 , . . .
• Our main challenge when dealing with the RHS of (2. [START_REF] Cao | On explicit l2-convergence rate estimate for underdamped langevin dynamics[END_REF]) is that it is very hard to compute explicitly those integrals since, for instance, ∥x t -x t-1 ∥ 2 and {i ∈ Īt-1 } are not independent.

• We know that if a random variable X is independent to the set of events B, then we have the property B XdP = E(X)P(B). We will therefore evaluate in such a way that after our evaluations, by grouping integrals with the common function integrated, the combined region is independent to that function. For example, after some evaluations, we obtain something of the form

A 1 XdP + A 2 XdP + . . . + Am XdP = A 1 ∪A 2 ∪...∪Am

XdP,

where A 1 , A 2 , . . . , A m are pair-wise disjoint. Here our aim is to have

A 1 ∪ A 2 ∪ . . . ∪ A m being
independent to X, so we can use the mentioned property.

With these meta-ideas kept in mind, we are back to the proof. We denote {ϵ i } the sequence as ϵ i = 1 i 2 . The sequence {ϵ i } plays a crucial role in our proof. Moreover, we denote

A = ∞ i=1 ϵ i = ∞ i=1 1 i 2 <
+∞. We define another sequence {θ i } as

θ i = ϵ i A , which implies ∞ i=1 θ i = 1. We will evaluate E∥x t -α t i ∥ 2 , (t ≥ 2) based on (2.19) as follows. Cauchy-Schwartz inequality implies {i∈ Īt-1 ∩I t-2 } ∥x t -x t-2 ∥ 2 dP ≤ 1 θ 1 {i∈ Īt-1 ∩I t-2 } ∥x t -x t-1 ∥ 2 dP + 1 1 -θ 1 {i∈ Īt-1 ∩I t-2 } ∥x t-1 -x t-2 ∥ 2 dP, {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } ∥x t -x t-3 ∥ 2 dP ≤ 1 θ 1 {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } ∥x t -x t-1 ∥ 2 dP + 1 θ 2 {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } ∥x t-1 -x t-2 ∥ 2 dP + 1 1 -θ 1 -θ 2 {i∈ Īt-1 ∩ Īt-2 ∩I t-3 } ∥x t-2 -x t-3 ∥ 2 dP, . . . {i∈ Īt-1 ∩...∩ Ī1 } ∥x t -x 0 ∥ 2 dP ≤ 1 θ 1 {i∈ Īt-1 ∩...∩ Ī1 } ∥x t -x t-1 ∥ 2 dP + 1 θ 2 {i∈ Īt-1 ∩...∩ Ī1 } ∥x t-1 -x t-2 ∥ 2 dP + . . . + 1 θ t-1 {i∈ Īt-1 ∩...∩ Ī1 } ∥x 2 -x 1 ∥ 2 dP + 1 1 -θ 1 -θ 2 -. . . -θ t-1 {i∈ Īt-1 ∩...∩ Ī1 } ∥x 1 -x 0 ∥ 2 dP.
By adding (2.19) and these inequalities, then grouping integrals sharing common the function integrated,

namely ∥x t -x t-1 ∥ 2 , ∥x t-2 -x t-1 ∥ 2 , . . . , ∥x 1 -x 0 ∥ 2 , and noticing {i ∈ Īt-1 ∩ I t-2 } ∪ {i ∈ Īt-1 ∩ Īt-2 ∩ I t-3 } . . . ∪ {i ∈ Īt-1 ∩ . . . Ī1 } = {i ∈ Īt-1 }, {i ∈ Īt-1 ∩ Īt-2 ∩ I t-3 } ∪ . . . ∪ {i ∈ Īt-1 ∩ . . . Ī1 } = {i ∈ Īt-1 ∩ Īt-2 }, . . . we get E∥x t -α t i ∥ 2 ≤ {i∈I t-1 } ∥x t -x t-1 ∥ 2 dP + 1 θ 1 {i∈ Īt-1 } ∥x t -x t-1 ∥ 2 dP + 1 1 -θ 1 {i∈ Īt-1 ∩I t-2 } ∥x t-1 -x t-2 ∥ 2 dP + 1 θ 2 {i∈ Īt-1 ∩ Īt-2 } ∥x t-1 -x t-2 ∥ 2 dP + . . . + 1 1 -θ 1 -. . . -θ t-2 {i∈ Īt-1 ∩...∩I 1 } ∥x 2 -x 1 ∥ 2 dP + 1 θ t-1 {i∈ Īt-1 ∩...∩ Ī1 } ∥x 2 -x 1 ∥ 2 dP + 1 1 -θ 1 -θ 2 -. . . -θ t-1 {i∈ Īt-1 ∩...∩ Ī1 } ∥x 1 -x 0 ∥ 2 dP. By observing that 1 ≤ 1 θ 1 , 1 1-θ 1 ≤ 1 θ 2 , . . . , 1 1-θ 1 -...-θ t-2 ≤ 1 θ t-1 , 1 1-θ 1 -...-θ t-1 ≤ 1 θt , we have E∥x t -α t i ∥ 2 ≤ 1 θ 1 Ω ∥x t -x t-1 ∥ 2 dP + 1 θ 2 {i∈ Īt-1 } ∥x t-1 -x t-2 ∥ 2 dP + . . . + 1 θ t-1 {i∈ Īt-1 ∩...∩ Ī2 } ∥x 2 -x 1 ∥ 2 dP + 1 θ t {i∈ Īt-1 ∩...∩ Ī1 } ∥x 1 -x 0 ∥ 2 dP.
By the independence of each pair

∥x t-1 -x t-2 ∥ 2 and {i ∈ Īt-1 },..., ∥x 2 -x 1 ∥ 2 and {i ∈ Īt-1 ∩. . .∩ Ī2 }, ∥x 1 -x 0 ∥ 2 and {i ∈ Īt-1 ∩ . . . ∩ Ī1 }, E∥x t -α t i ∥ 2 ≤ 1 θ 1 E∥x t -x t-1 ∥ 2 + 1 θ 2 E∥x t-1 -x t-2 ∥ 2 P({i ∈ Īt-1 }) + . . . + 1 θ t-1 E∥x 2 -x 1 ∥ 2 P({i ∈ Īt-1 ∩ . . . ∩ Ī2 }) + 1 θ t E∥x 1 -x 0 ∥ 2 P({i ∈ Īt-1 ∩ . . . ∩ Ī1 }) = 1 θ 1 E∥x t -x t-1 ∥ 2 + 1 θ 2 E∥x t-1 -x t-2 ∥ 2 N -1 N b + . . . + 1 θ t-1 E∥x 2 -x 1 ∥ 2 N -1 N b(t-2) + 1 θ t E∥x 1 -x 0 ∥ 2 N -1 N b(t-1)
.

For short, let us denote

λ = N -1 N b < 1.
The inequality can be written as follows

E∥x t -α t i ∥ 2 ≤ t k=1 λ t-k θ t-k+1 E∥x k -x k-1 ∥ 2 .
Therefore,

∞ t=1 E∥x t -α t i ∥ 2 ≤ ∞ t=1 t k=1 1 θ t-k+1 λ t-k E∥x k -x k-1 ∥ 2 = ∞ k=1 ∞ t=k 1 θ t-k+1 λ t-k E∥x k -x k-1 ∥ 2 = ∞ k=1 E∥x k -x k-1 ∥ 2 ∞ t=k λ t-k θ t-k+1 = ∞ k=1 E∥x k -x k-1 ∥ 2 ∞ t=0 λ t θ t+1 = A ∞ t=0 λ t ϵ t+1 ∞ k=1 E∥x k -x k-1 ∥ 2 = A ∞ t=0 (t + 1) 2 λ t ∞ k=1 E∥x k -x k-1 ∥ 2 < +∞,
since the power series u(x) = ∞ t=0 (t + 1) 2 λ t has the convergence radius 1. 3. As a consequence of properties 1,2 above, we derive that {F (x t )} converges to V ∞ almost surely.

If follows from (2.15) and property

2 that E ∞ k=1 ∥v k -∇H(x k )∥ 2 < +∞. Let S = v k -∇H(x k ) → 0, x k -x k+1 → 0, {y k } is bounded .
It is evident that P(S) = 1. We consider a fixed event in S to obtain a realization {x k }. Let x * be a limit point of {x k }, there exists a subsequence {x

l k } such that x l k → x * , implying x l k +1 → x * . Since H is L-smooth, we obtain ∇H(x l k ) → ∇H(x * ), which implies, v l k → ∇H(x * ). Since {y l k } is bounded,
by passing to a subsequence if necessary, we assume that y l k → y * , therefore, y * ∈ ∂r 2 (x * ). On the other hand, we have z l k ∈ ∂(G + r 1 )(x l k +1 ). By the closedness of the graph subdifferential operator,

we obtain ∇H(x * ) + y * ∈ ∂(G + r 1 )(x * ). Therefore, x * is a critical point of F = (G + r 1 ) -(H + r 2 ). 5. Since z t ∈ ∂(G + r 1 )(x t+1 ), dist ∇H(x t+1 ) + ∇r 2 (x t+1 ), ∂(G + r 1 )(x t+1 ) ≤ ∥∇H(x t+1 ) + ∇r 2 (x t+1 ) -v t -y t ∥ ≤ ∥∇H(x t+1 ) -∇H(x t )∥ + ∥∇H(x t ) -v t ∥ + ∥∇r 2 (x t+1 ) -∇r 2 (x t )∥ ≤ (L + L r 2 )∥x t+1 -x t ∥ + ∥∇H(x t ) -v t ∥.
Therefore,

E dist ∇H(x t+1 ) + ∇r 2 (x t+1 ), ∂(G + r 1 )(x t+1 ) ≤ (L + L r 2 )E∥x t+1 -x t ∥ + E∥∇H(x t ) -v t ∥ 2 1 2 ≤ (L + L r 2 ) E∥x t+1 -x t ∥ 2 1 2 + L √ bN N i=1 E∥x t -α t i ∥ 2 1 2
.

Summing these inequalities for

t = 0 to T -1, T -1 t=0 E dist(∇H(x t+1 ) + ∇r 2 (x t+1 ), ∂(G + r 1 )(x t+1 )) ≤ (L + L r 2 ) T -1 t=0 E∥x t+1 -x t ∥ 2 1 2 + L √ bN T -1 t=0 N i=1 E∥x t -α t i ∥ 2 1 2 ≤ (L + L r 2 ) √ T T -1 t=0 E∥x t+1 -x t ∥ 2 1 2 + L √ bN √ T T -1 t=0 N i=1 E∥x t -α t i ∥ 2 1 2 ≤ (L + L r 2 ) √ T C + L √ T C √ b ≤ (2L + L r 2 ) C √ T ,
for some C > 0.

It follows that the iteration convergence rate is

d T = O(1/ √ T ).
If the output x a is chosen randomly from {x t } T t=1 ,

E dist(∇H(x a ) + ∇r 2 (x a ), ∂(G + r 1 )(x a )) = O 1 √ T .
Therefore, to get ϵ-criticality, the algorithm needs

T = O(1/ϵ 2 ) iterations. From the condition for b, if we choose b = 4 √ 2 √ µ N 3/4 , where µ = ρ H + ρ r 2 + ρ G+r 1 4L ,
one obtains the complexity of O(N + N 3/4 /ϵ 2 ) in terms of the number of gradient evaluations and the complexity of O(1/ϵ 2 ) in terms of the number of convex subproblems.

Sketched proof for the sampling without replacement option.

1. We arrive at the inequality (2.14) by following the arguments offered in the previous proof. We then introduce the following lemma whose proof is similar to that of Lemma 1.

Lemma 2. The following inequality holds,

E It ∥v t -∇H(x t )∥ 2 ≤ L 2 bN 1 - b -1 N -1 N i=1 ∥x t -α t i ∥ 2 .
Similar to the arguments above, by introducing the Lyapunov sequence

V t = F (x t )+c t N i=1 ∥x t -α t i ∥ 2 with c t = βL 2 (N -b) 2γb(N -1)(b-N +βb)
, where β = N/b, and γ is determined by the AM-GM inequality as above, we want

N √ N 2 -b 2 b 2 √ N -1 ≤ ρ 4L . (2.20) Consequently, if N √ N +1 b 2 ≤ ρ 4L
, (2.20) holds and we obtain that {V t } converges almost surely and

∞ t=0 E∥x t+1 -x t ∥ 2 < ∞.
2. The arguments above can be employed with the following notice:

P({i ∈ Īt-1 }) = 1 - b/N, P({i ∈ Īt-1 ∩ Īt-2 }) = (1 -b/N ) 2 , etc.
3,4,5. These properties are established similar to the option of sampling with replacement.

Remark 2. i. We can apply the same technique discussed in Remark 1 to improve the number ρ L of the problem if necessary.

ii. The analysis providing links between E∥x t -α t i ∥ 2 and ∞ t=1 E∥x t+1 -x t ∥ 2 is quite novel in the literature, which is expected to be useful in analyzing SAGA-type algorithms. The analysis offers an elegant solution to escape the "non-independence" of the function and the region of an integral.

Remark 3. Several comparisons to related algorithms: i. From the convergence of the standard DCA [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF], one can verify that to find an ϵ-criticality, under the L-smooth assumption of H and r 2 , it requires O(1/ϵ 2 ) iterations. As a result, the standard DCA has the complexity of O(N/ϵ 2 ) in terms of gradient evaluations and O(1/ϵ 2 ) in terms of the number of convex subproblems. While the proposed DCA-SVRG and DCA-SAGA enjoy the same complexity as the standard DCA in terms of the number of convex subproblems, they significantly outperform the standard DCA in terms of gradient evaluations, DCA-SVRG:

O(N 2/3 /ϵ 2 ), DCA-SAGA: O(N + N 3/4 /ϵ 2 ).
ii. [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF] considered the following large-sum problem

min x∈R n F (x) = G(x) -H(x) + r(x) := 1 N N i=1 g i (x) - 1 N N j=1 h j (x) + r(x),
where g i , h j are convex and the regularizer r is convex. The authors also examine the case that r is nonconvex (not necessarily DC) where the proximal mapping can be computed efficiently. For the sake of simplicity, we only discuss the case where r is convex (note that their complexity for the case of nonconvex r is obviously worse than in the case where r is convex). When each h j is L-smooth, the best gradient complexity to find (nearly) ϵ-criticality is given in [134, Corollary 9], which is O(( N +N )/ϵ 2 ).

Obviously, this complexity takes the cost of solving each large-sum convex subproblem by the SVRG into account. In order to compare this complexity to our algorithms, we must express it as the number of gradient evaluations of H and the number of convex subproblems being solved. Our algorithms enjoy superior gradient complexity, while the complexity in terms of convex subproblems is somewhat incomparable since their algorithmic design also takes into account the inexactness of solving each such problem, while we do not quantify this issue here.

Additional convergence analysis for the composite structure of r 2

In this section, we consider

r 2 (x) = m i=1 l i (p i (x))
where l i : R → R is convex and decreasing, p i : R n → R is convex and hence continuous, for i = 1, m. The optimization problem (P ) becomes

(Q) α = min F (x) = G(x) - 1 N N i=1 h i (x) + r 1 (x) - m i=1 l i (p i (x))
.

By denoting Ω = {(x, z) ∈ R n × R m : p i (x) ≤ z i , ∀i = 1, m}, we can solve the following problem instead (Q ′ ) α = min φ(x, z) = G(x) - 1 N N i=1 h i (x) + r 1 (x) + χ Ω (x, z) - m i=1 l i (z i ) , since if (x * , z * )
is the optimal solution of (Q ′ ), x * is the optimal solution of (Q). We notice that the problem (Q ′ ) with respect to the optimization variables (x, z) takes the form of (P ). As a result, we can use the proposed DCA-SVRG and DCA-SAGA to solve (Q ′ ). However, because x and z do not play the same role, the convergence analysis of DCA-SVRG and DCA-SAGA should be modified for this case. We can see that the modulus of convexity of G and H with respect to (x, z) is 0 due to the newly introduced optimization variable z, which is undesirable. By Remark 1, we can add the convex term γ 2 ∥(x, z)∥ 2 to both DC components. However, this technique has the risk of resulting in bad approximations. Therefore, we still want to use the modulus of strong convexity of G and H with respect to x only in our analysis, which necessitates adapting the convergence analysis of DCA-SVRG and DCA-SAGA to this case. For convenience of presentation, we provide in detail the DCA-SVRG scheme applied to (Q ′ ) and state the convergence results with its sketched proof. For DCA-SAGA applied to (Q ′ ), only present convergence results are presented.

The DCA-SVRG scheme applied to (Q ′ ) is given in the Algorithm 3, where we denote

r 3 (z) = m i=1 l i (z i ).
Henceforth, we still denote ρ H , ρ G+r 1 the modulus of strong convexity of H and G + r 1 and L the Lipschitz constant of ∇h i with respect to x only. We derive the following convergence results.

Theorem 8. If the minibatch size b and the inner-loop length M satisfy

M √ b ≤ 1 4 √ e -1 ρ G+r 1 + ρ H L ,
∇h i (x k+1 j ) + νk - 1 b i∈I b ∇h i (x k ).
Compute y k+1 j ∈ ∂r 3 (z k+1 j

). Solve the convex problem

(x k+1 j+1 , z k+1 j+1 ) = arg min{G(x) + r 1 (x) + χ Ω (x, z) -⟨t k+1 j , x⟩ -⟨y k+1 j , z⟩} end for Set xk+1 = x k+1 M , and k = k + 1. until Stopping criterion. then 1. The sequence {F (x k )} converges almost surely. 2. ∞ k=0 M -1 j=0 E∥x k+1 j+1 -x k+1 j ∥ 2 < +∞.
3. Suppose the sequence {y k j } is bounded almost surely, let x * be a limit point of {x k j }, then (x * , z * ) is a critical point of F = (G+r 1 +χ Ω )-(H+r 3 ) almost surely, where z * = (p 1 (x * ), p 2 (x * ), . . . , p m (x * )) ⊤ .

Proof of Theorem 8. 1, 2. The convex subproblem can be solved as follows

x k+1 j+1 = arg min G(x) + r 1 (x) -⟨t k+1 j , x⟩ - m r=1 y k+1 j,r p r (x) (2.21) z k+1 j+1,r = p r (x k+1 j+1 ), ∀r = 1, m. (2.22) It follows from (2.21) that t k+1 j ∈ ∂ G + r 1 -m r=1 y k+1 j,r p r (x k+1 j+1 ), which implies G(x k+1 j+1 ) + r 1 (x k+1 j+1 ) ≤G(x k+1 j ) + r 1 (x k+1 j ) + ⟨t k+1 j , x k+1 j+1 -x k+1 j ⟩ - m r=1 y k+1 j,r (p r (x k+1 j ) -p r (x k+1 j+1 )) - ρ G+r 1 2 ∥x k+1 j+1 -x k+1 j ∥ 2 . (2.23)
From the convexity of H and r 3 , we obtain

H(x k+1 j+1 ) ≥ H(x k+1 j ) + ⟨∇H(x k+1 j ), x k+1 j+1 -x k+1 j ⟩ + ρ H 2 ∥x k+1 j+1 -x k+1 j ∥ 2 , (2.24) r 3 (z k+1 j+1 ) ≥ r 3 (z k+1 j ) + ⟨y k+1 j , z k+1 j+1 -z k+1 j ⟩. (2.25)
From (2.23), (2.24) and (2.25),

F (x k+1 j+1 ) ≤ F (x k+1 j ) + ⟨t k+1 j -∇H(x k+1 j ), x k+1 j+1 -x k j ⟩ - ρ G+r 1 + ρ H 2 ∥x k+1 j+1 -x k+1 j ∥ 2 , by noting that φ(x k+1 j+1 , z k+1 j+1 ) = F (x k+1 j+1 ), φ(x k+1 j , z k+1 j ) = F (x k+1 j ).
Here, by considering the Lyapunov sequence

V k+1 j = F (x k+1 j ) + c j ∥x k+1 j
-xk ∥ 2 similar to the proof of Theorem 6, we arrive at the conclusion 1 and 2.

Let

S = t k+1 j -∇H(x k+1 j ) → 0, x k+1 j+1 -x k+1 j
→ 0, {y k j } is bounded . We see that P(S) = 1 and let {x k j } be a realization with respect to a fixed event in S. Let x * be a limit point of {x k j }, there exists a subsequent x

u(l) v(l) → x * , which further implies x u(l) v(l)+1 → x * . From (2.22
) and the continuity of p r , we obtain z u(l) v(l) → (p 1 (x * ), . . . , p m (x * )) ⊤ . Without loss of generality, we assume that y u(l) v(l) → y * , which yields y * ∈ ∂r 3 ((p 1 (x * ), . . . , p m (x * )) ⊤ ). On the other hand, t

u(l) v(l) → ∇H(x * ). It follows from t u(l) v(l) ∈ ∂ G + r 1 - m r=1 y u(l) v(l),r p r (x u(l) v(l)+1 ) that ∇H(x * ) ∈ ∂(G + r 1 -m r=1 y * r p r )(x * ). Therefore, ∇H(x * ) y * ∈ ∂(G(x) + r 1 (x) + χ Ω (x, z))(x * , z * ), which implies ∂(G(x) + r 1 (x) + χ Ω (x, z))(x * , z * ) ∩ ∂(H(x) + r 3 (z))(x * , z * ) ̸ = ∅.
Similarly, we have the following convergence results when applying DCA-SAGA to minimize the function φ(x, z).

Theorem 9. If the minibatch size b satisfies N √ N +b b 2 ≤ ρ H +ρ G+r 1 4L
for the option of sampling with

replacement or N √ N +1 b 2 ≤ ρ H +ρ G+r 1 4L
for the option of sampling without replacement, then 1.

∞ t=0 E∥x t+1 -x t ∥ 2 < ∞. 2. ∞ t=0 N i=1 E∥x t -α t i ∥ 2 < ∞.
3. The sequence {F (x t )} converges almost surely.

4. Suppose the sequence {y t } is bounded almost surely, let x * be a limit point of {x t }, then (x * , z * ) is a critical point of F = (G+r 1 +χ Ω )-(H+r 3 ) almost surely, where z * = (p 1 (x * ), p 2 (x * ), . . . , p m (x * )) ⊤ .

Applications

In this section, we apply DCA-SVRG and DCA-SAGA to three important machine learning problems to investigate their practical behaviors: nonnegative principal component analysis, group variable selection in multi-class logistic regression, and sparse linear regression.

All numerical experiments in this section are performed on a Processor Intel(R) core(TM) i7-8700, CPU @ 3.20GHz, RAM 16 GB.

Nonnegative principal component analysis

Principal component analysis is arguably one of the most effective tools for reducing dimensionality.

PCA has a long history of success in a wide range of applied sciences, including neuroscience, medicine, psychology, material science, scientometry, astronomy, geography, and social sciences.

We conduct numerical experiments on a type of PCA structure known as Non-negative component analysis (NN-PCA). The term "non-negative" implies that we limit the search domain to be the nonnegative portion of the search space. This restriction implicitly assumes that we know some additional information about the PCA solution: the solution belongs to the non-negative orthant. As discussed in [START_REF] Montanari | Non-negative principal component analysis: Message passing algorithms and sharp asymptotics[END_REF], this additional information offers some theoretical advantages. The NN-PCA has the following form

min ∥x∥≤1,x≥0 - 1 2N N i=1 ⟨x, z i ⟩ 2 , (2.26)
where {z i } N i=1 is the given dataset.

DCA-SVRG and DCA-SAGA applied to (2.26)

We apply DCA-SVRG and DCA-SAGA to (2.26) with the following DC decomposition

G(x) = ρ 2 ∥x∥ 2 , h i (x) = ρ 2 ∥x∥ 2 + 1 2 ⟨x, z i ⟩ 2 , r 1 (x) = χ S (x), r 2 (x) = 0
, where S = {x ∈ R n : x ≥ 0, ∥x∥ ≤ 1}, and ρ > 0. In our experiments, we fix ρ = 1.

With this setting, we implement two versions of DCA-SAGA and two versions of DCA-SVRG, namely, DCA-SAGA-v1 (sampling with replacement) and DCA-SAGA-v2 (sampling without replacement), DCA-SVRG-v1 (sampling with replacement) and DCA-SVRG-v2 (sampling without replacement), respectively.

Numerical experiments

Datasets We use standard machine learning datasets in LIBSVM2 , namely, a9a (32561 × 123), aloi (108000 × 128), cifar10 (50000 × 3072), SensIT Vehicle (78823 × 100), connect-4 (67557 × 126), letter (15000 × 16), mnist (60000 × 780), protein (17766 × 357), shuttle (43500 × 9), YearPredictionMSD (463715 × 90). Each sample of these datasets is normalized as

∥z i ∥ = 1.
Comparative algorithms We implement stochastic DCA (abbreviated SDCA) [START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF] as a main competitor to justify the merits of the variance-reduction estimators used. For a more in-depth comparison, we further implement prox-SGD and prox-SAGA [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF], prox-SARAH [START_REF] Pham | Proxsarah: An efficient algorithmic framework for stochastic composite nonconvex optimization[END_REF], prox-SpiderBoost [START_REF] Wang | Spiderboost and momentum: Faster variance reduction algorithms[END_REF].

We do not compare with the prox-SVRG here because, as previously discussed, the DCA-SVRG-v1 applied to the above setting recovers a version of the prox-SVRG.

For proximal-type algorithms (prox-SGD, prox-SAGA, prox-SARAH, prox-SpiderBoost), the convex regularizer is set to be χ S (x), while each L-smooth component of the large sum is set as

f i (x) = -1 2 ⟨x, z i ⟩ 2 (f i is 1-smooth).

Experiment setups and results

We study the evolution process of the objective in relation to the computational resource used. This resource is measured by the number of stochastic first-order oracle calls (SFO). That is, one SFO call is counted each time we access one stochastic gradient (the gradient of one summand) of the large sum. We set the fixed budget of SFO calls to be 15N . We do, however, allow all algorithms to use some additional SFO calls to complete their final iteration.

We observe that each function x → 1 2 ⟨x, z i ⟩ 2 is 1-smooth. Based on theoretical convergence results, we choose hyperparameters for the algorithms as follows. The minibatch size b is chosen as ⌊N √ b⌋. On the other hand, we use a neutral minibatch size of 10%N which usually yields good results [START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF] for the SDCA. For the prox-SGD, η = 1/(2L) [START_REF] Ghadimi | Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization[END_REF] and we choose a neutral minibatch size of 500. For the prox-SAGA, we set η = 1/(5L) and the minibatch size b = ⌊N Lastly, for the prox-SpiderBoost, we choose η = 1/(2L) and the inner loop length and the mini-batch

size M = b = ⌊ √ N ⌋ [129, Theorem 1].
We report the suboptimality of the objective, f (x) -f (x), where x is the "optimal" solution found by running the deterministic DCA 500 iterations (which means that this deterministic DCA uses 500N SFO calls) 10 times with different initial points. The final optimal value f (x) is the smallest value found by the deterministic DCA and all the competitive algorithms in this experiment. In order to increase visibility, we plot this suboptimality on the logarithmic scale of base 10. Figure 2.1 shows the results over 10 random runs of all comparative algorithms. The bold lines depict the mean curves, while the shaded regions represent the mean absolute deviation (MAD). Note that, in the normal scale, one might plot x ± MAD(x); Meanwhile, in the log scale, a reasonable way is to plot log 10 (x) ± (1/ ln(10)) MAD(x)/x ≈ log 10 (x) ± 0.434 MAD(x)/x (see, e.g., [START_REF] Stuve | Estimating and plotting logarithmic error bars[END_REF]).

Comments In this experiment, the prox-SAGA also obtains good results with suboptimality less than 10 -5 across most datasets. The SDCA performs marginally worse than prox-SAGA, whereas prox-SGD fluctuates around 10 -5 and cannot reduce suboptimality any further.

Group Variables Selection in Multi-class Logistic Regression

Logistic regression is without a doubt one of the most successful classification tools. Logistic regression has a wide range of applications including cancer detection [START_REF] Kim | A gradient-based optimization algorithm for lasso[END_REF], social sciences [START_REF] King | Logistic regression in rare events data[END_REF], medical [START_REF] Bagley | Logistic regression in the medical literature:: Standards for use and reporting, with particular attention to one medical domain[END_REF],

etc. For high-dimensional data, on the other hand, feature selection is typically used to select relevant features and encourage weights corresponding to irrelevant features to go to 0.

Let {(x i , y i ) : i = 1, 2, . . . , N } be a training set with feature vectors x i and the labels y i ∈ {1, 2, . . . , Q} where Q is the number of classes. Let W be the d×Q matrix with columns W :,1 , W :,2 , . . . , W :,Q

and b = (b 1 , b 2 , . . . , b Q ).
The conditional probability of class y given the observation X = x can be modeled by

p(Y = y|X = x) = exp (b y + W ⊤ :,y x) Q k=1 exp (b k + W ⊤ :,k x)
.

To find (W, b), one minimizes the following negative log-likelihood function over the whole training set

L(W, b) := 1 N N i=1 ℓ(x i , y i , W, b) where ℓ(x i , y i , W, b) = -log p(Y = y i |X = x i ).
On the other hand, to select relevant features, one employs the following ℓ q,0 -norm of W ∥W ∥ q,0 = |{j ∈ {1, 2, . . . , d} : ∥W j,: ∥ q ̸ = 0}|, which leads to the following optimization problem

min W,b 1 N N i=1 ℓ(x i , y i , W, b) + λ∥W ∥ q,0 . (2.27)
In practice, the discontinuous ℓ q,0 -norm is usually approximated by some continuous function. In this work, we approximate ℓ q,0 with the following nonconvex functions, which have been shown to be efficient in a variety of problems such as individual variable selection in SVM [START_REF] Bradley | Feature selection via concave minimization and support vector machines[END_REF], sparse optimal scoring problem [START_REF] Le Thi | Dc programming and dca for sparse optimal scoring problem[END_REF], sparse covariance matrix estimation problem [START_REF] Phan | Sparse covariance matrix estimation by dca-based algorithms[END_REF],

Exponential: η exp α (s) = 1 -exp(-αs), Capped -ℓ 1 : η cap-ℓ 1 α (s) = min{1, αs}.
The corresponding approximation problem of (2.27) takes the form

min W,b    1 N N i=1 ℓ(x i , y i , W, b) + λ d j=1 η α (∥W j,: ∥ q )    .
(2.28)

DCA-SVRG and DCA-SAGA applied to (2.28)

It is worth noting that an L-smooth function φ admits the following DC decomposition

φ(x) = γ 2 ∥x∥ 2 - γ 2 ∥x∥ 2 -φ(x) (2.29) whenever γ ≥ L. We can show that ℓ(x i , y i , W, b) is 2 √ 2 3 √ 3 √ Q(∥x i ∥ 2 + 1)
-smooth, so the DC decomposition (2.29) can be used. On the other hand, η α is concave and increasing. Therefore, the problem (2.28) takes the form of (Q) with

G(W, b) = γ 2 ∥(W, b)∥ 2 , h i (W, b) = γ 2 ∥(W, b)∥ 2 -ℓ(x i , y i , W, b), r 1 (W, b) = 0, l i (s) = -λη α (s), p i (W, b) = ∥W i,: ∥ q , ∀i = 1, d, where γ ≥ 2 √ 2 3 √ 3 √ Q(max i=1,N ∥x i ∥ 2 + 1)
. We can apply DCA-SVRG and DCA-SAGA to solve (2.28).

In general, the convex subproblem of DCA-SVRG and DCA-SAGA requires the computing of the proximal operator of r∥ • ∥ q with q ∈ {1, 2}, where the proximal operator of a function f is defined by

prox f (x) = arg min y 1 2 ∥x -y∥ 2 + f (y) .
It is worth noting that, these proximal operators can be computed explicitly as follows

prox r∥•∥ 2 (x) =      1 - r ∥x∥ 2 x if ∥x∥ 2 ≥ r, 0 otherwise prox r∥•∥ 1 (x) = max(|x| -r, 0) • sign(x).
Therefore, the corresponding DCA-SVRG and DCA-SAGA schemes applied to (2.28) is explicit, i.e., no convex solver is needed for solving convex subproblems.

In this experiment, we found that DCA-SAGA with replacement yields poor performance and is not numerically stable. Hence, we only study the behaviors of three algorithms: DCA-SAGA without replacement, DCA-SVRG-v1 (with replacement), and DCA-SVRG-v2 (without replacement).

Numerical experiments

Datasets We use standard machine learning datasets obtained from LIBSVM for multiclass classifica-

Comparative algorithms

The DCA and the SDCA are chosen as comparative algorithms in this experiment.

Experiment setups and results

We set the budget of SFO calls at 20N . Throughout this experiment, regularization parameter λ and the number α that controls the tightness of the ℓ q,0 approximation are set to 1 and 0.5, respectively. As mentioned, it can be demonstrated that each function ℓ(

x i , y i , W, b) is L-smooth, where L = 2 √ 2 3 √ 3 √ Q(max i=1,N ∥x i ∥ 2 + 1).
To ensure the convexity of two DC components, we need to set γ ≥ L. In our experiment, we fix γ = 2L.

By the convergence analysis above and the numerical experiments in [START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF], we choose hyperparameters for algorithms as follows. The minibatch size b is chosen as over the SDCA ranges from 0.56% to 54.73%.

2 N √ N + 1 , ⌊N 2 3 ⌋, ⌊N 2

Sparse Linear Regression

Sparse linear regression is a linear regression model that aims to eliminate features that are redundant, noisy, or irrelevant. Given a dataset {(x i , y i )} N i=1 , sparse linear regression attempts to fit ⟨β, x i ⟩ ≈ y i with a sparse solution β in order to select right features, particularly in the high-dimensional regime.

Formally, the optimization problem associated with sparse linear regression is given by

min β 1 2N N i=1 (y i -⟨β, x i ⟩) 2 + λ∥β∥ 0 , (2.30)
where λ > 0 is a parameter that makes a trade-off between the data-fitting term and the sparsityencouraging term.

As previously discussed, the discontinuous ℓ 0 -norm is typically approximated by a continuous one to make the problem tractable. In this section, we focus on the Capped-ℓ 1 that is used to approximate the ℓ 0 -norm to give rise to the following approximation problem

min β F (β) = 1 2N N i=1 (y i -⟨β, x i ⟩) 2 + λ m j=1 min(1, α|β j |).
(2.31)

DCA-SVRG and DCA-SAGA applied to (2.31)

Firstly, each function β → 1 2 (y i -⟨β, x i ⟩) 2 is ∥x i ∥ 2
-smooth, so it admits the DC decomposition (2.29). On the other hand, the Capped-ℓ 1 function is DC with the DC decomposition min(1, α|β|) = g(β) -h(β), where g(β) = 1 + α|β|, and h(β) = max (1, α|β|). Therefore, the problem (2.31) falls into our framework with the following setting

G(β) = γ 2 ∥β∥ 2 , h i (β) = γ 2 ∥β∥ 2 - 1 2 ⟨β, x i ⟩ 2 + y i ⟨β, x i ⟩, r 1 (β) = λα∥β∥ 1 , r 2 (β) = λ m j=1 max(1, α|β j |),
where γ ≥ max i=1,N ∥x i ∥ 2 . When applying DCA-SAGA and DCA-SVRG to this setting, the subproblem is nothing but the proximal operator of ℓ 1 , which has a closed-form solution.

On the other hand, we discovered that DCA-SAGA with replacement performs poorly and is numerically unstable in this experiment, which is a phenomenon that also occurred in the previous experiments on group variables selection in multi-class logistic regression. As a result, we only take into account DCA-SAGA without replacement, DCA-SVRG-v1, and DCA-SVRG-v2.

Numerical experiments

Datasets We use regression datasets, namely, cadata (20640×8), YearPredictionMSD (463715× 90) from LIBSVM, and SGEMM GPU kernel performance dataset, sgemm_product (241600 × 14) [START_REF] Nugteren | Cltune: A generic auto-tuner for opencl kernels[END_REF]. Each dataset is standardized to have mean (µ) of 0 and standard deviation (σ) of 1.

Comparative algorithms We compare our proposed algorithms with the standard DCA, the SDCA, the stochastic stagewise DC (SSDC) proposed in [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF], and the mini-batch stochastic proximal gradient (MB-SPG) [START_REF] Xu | Non-asymptotic analysis of stochastic methods for non-smooth nonconvex regularized problems[END_REF].

Experiment setups and results

The budget of SFO calls is set to be 30N . The regularization parameter λ and the parameter α of the Capped-ℓ 1 are both set to be 1 throughout this experiment. To guarantee the convexity of DC components, we choose γ = max i=1,N ∥x i ∥ 2 . On the other hand, the minibatch size b is chosen as ⌊N About the SSDC, the nonconvex regularizer Capped-ℓ 1 is replaced by the Monreau envelope r µ [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF]Sect. 4]. Moreover, the proximal operator of the Capped-ℓ 1 can be efficiently computed (see, e.g., [START_REF] Gong | A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems[END_REF]), which allows the SSDC to work. The parameter µ controlling the Monreau envelope is required to match the (square) order of the final error ϵ [134, Theorem 10c], so it is set to be a relatively small number, µ = 10 -6 . Note that, since the convex subproblem has a closed-form solution and the first DC component is not given as a large sum, no stochastic convex solver is required for the SSDC in this case. On the other hand, as remarked in [START_REF] Xu | Non-asymptotic analysis of stochastic methods for non-smooth nonconvex regularized problems[END_REF], the use of the Moreau envelope could be a bad idea as it introduces the approximation error while slowing down the convergence. Therefore, we further implement the mini-batch stochastic proximal gradient (MB-SPG) [START_REF] Xu | Non-asymptotic analysis of stochastic methods for non-smooth nonconvex regularized problems[END_REF] that uses directly the proximal operator of the Capped-ℓ 1 . The minibatch size of MB-SPG is of order O(1/ϵ 2 ) where ϵ is the expected final error, so it is proper to set the minibatch size at ⌊10%N ⌋. The constant c (see [START_REF] Xu | Non-asymptotic analysis of stochastic methods for non-smooth nonconvex regularized problems[END_REF]) is in (0, 0.5), so we set it at 0.25 which is a neutral number between the two extremes; Meanwhile, as discussed earlier, the Lipschitz smoothness parameter can be chosen as max i=1,N ∥x i ∥ 2 .

We report the mean curve and the mean absolute deviation (over 10 runs) of the objective value of five comparative algorithms in Figure 2.6, where the mean absolute deviation is scaled up by a factor of 20 for visibility.

Comments It is observed that, once again, DCA-SAGA is the best algorithm over all tested cases where it decreases the objective value in a fast rate to obtain good solutions. Meanwhile, the performances of DCA-SVRG-v1 and DCA-SVRG-v2 are almost identical and are the second best on sgemm_product and YearPredictionMSD datasets. On the other hand, the MB-SPG is the second best on the cadata dataset and pushes the DCA-SVRGs to the third place in this case. The gain of DCA-SAGA over DCA-SVRG-v1 are 5.22%, 0.24%, and 0.56% on cadata, sgemm_product, and YearPredictionMSD, respectively. On the other hand, the graphs of DCA and SDCA are also almost the same on all three datasets. The gain of DCA-SVRG-v1 over SDCA on cadata, sgemm_product, and YearPredictionMSD are 2.11%, 11.45%, and 0.98%, respectively. Finally, in this experiment, while sampling with replacement makes DCA-SAGA numerically unstable, sampling without replacement consistently makes DCA-SAGA the best algorithm over all considered tested cases. Finally, the SSDC performs quite poorly in this experiment. Although it still manages to decrease the objective function, this decrease is insufficient to be visible in the decreasing magnitude of other comparative algorithms. This can be attributed to the slow convergence caused by the use of the Moreau envelope.

Conclusion

In this chapter, we introduced two stochastic DCA methods with integrated variance reduction techniques, DCA-SVRG and DCA-SAGA, for tackling a wide class of nonconvex DC problems, including the large-sum case, where both the data-fitting and regularization terms are given as (nonsmooth) DC functions. The DCA framework addresses the DC structure, while the unbiased estimators SVRG and SAGA address the large-sum structure. In this study, the benefits of DC programming and DCA are multiple.

DC programming and DCA give a broad overview of the problems under consideration and advise on how to use the structural information in these problems. They create a flexible deterministic frame around which stochastic approaches are developed to deal with the stochastically problematic aspects of the problems. Our proposed methods inherit both the advantages and the inherent limitations of the stochastic estimators used. The DCA-SVRG is an epoch-based algorithm with the benefit of requiring just one gradient vector in memory to produce the stochastic variance reduction term. However, the pivot point xk is not changed within its corresponding epoch, which appears to be a flaw of this approach.

Indeed, when the points x k+1 j in an epoch proceed far away from its pivot xk , ∇h i (x k+1 j

) and ∇h i (x k ) (i is a random index) are no longer highly correlated, undermining the effect of the variance reduction term. In contrast, the progressive updating of the variance reduction term across iterations allows DCA-SAGA to partially address this issue. However, DCA-SAGA has a limitation of its own, which is the memory need for storing a table of gradients. Although one can retain a vector of scalars only in some specific problems, this issue persists generally and is the algorithm's bottleneck. We have observed that a biased estimator named SARAH [START_REF] Nguyen | SARAH: A novel method for machine learning problems using stochastic recursive gradient[END_REF] can potentially address both of the drawbacks of SVRG and SAGA: it progressively updates the variance reduction term within each epoch and does not need the storage of the gradient table. This observation stimulates our future work on the DCA-SARAH combination. Additionally, we have researched both sampling with and without replacement in the step of selecting a subset of the large sum. The independence of the selected elements is enjoyed by sampling with replacement, making convergence analysis easier. The latter strategy, however, is more widely used and prevents some low-probability bad events that occur in sampling with replacement (high repetition leading to bad approximations). In our numerical experiments on three problems, the results of DCA-SAGA with the two sampling options are considerably different from one another: sampling without replacement makes DCA-SAGA more stable and makes it the best method in two of the studied problems. On the other hand, DCA-SVRG's performances using the two sampling options are almost identical.

From a theoretical standpoint, we establish the almost sure convergence of the proposed algorithms to DC critical points. Furthermore, for the class of problem (P ) where r 2 in the regularizer r(x) = r 1 (x) -r 2 (x) has a composite form, we provide additional convergence analysis to facilitate parameters selection in practice. By exploiting the special composite structure of r 2 , we reformulate this problem as a DC program. We then provide new relationships between the parameters of the proposed algorithms (minibatch size, inner-loop length) and the parameters of the problems (strong convexity, Lipschitz smoothness, dataset size), which ensures the almost sure convergence property to DC critical points.

The key benefit of the extra analysis is that it enables more relaxing convergence criteria than those that would be obtained by just applying the full previous convergence analysis to the reformulated problem.

Overall, the proposed algorithms appear to outperform several state-of-the-art stochastic methods for large-sum nonconvex problems in three experiments. These experiments also demonstrate the advantages of the SAGA and SVRG estimators within the proposed algorithms, as well as the limitations of the SAG estimator used in SDCA.

Chapter 3

Online Stochastic DCA Abstract. In this chapter, we propose stochastic DCA schemes for solving stochastic DC programs whose involved random variable can have an arbitrary distribution, with the focus on handling streaming data. The context of streaming data is pervasive in real-world applications, necessitating a fast and scalable approach. The proposed algorithms' convergence is thoroughly investigated using tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important machine learning problem: the expected problem in Principal Component Analysis.

Introduction

Stochastic optimization is a means for resolving many challenges in the age of big data, such as data processing, storage bottlenecks, noisy measurements, high dimensionality, and so on. A typical stochastic optimization problem takes the form

min{F (w) := E(f (w, Z)) : w ∈ S ⊂ R m }, (3.1)
where Z is a random vector with unknown distribution in general, and f is a loss function defined on R m . There is an extensive literature studying this problem in the convex setting, i.e. when f (•, Z) is convex for all Z, and S is a convex set. When either In this chapter, we are concerned with stochastic DC programs, the major class of nonconvex stochastic optimization, which have the form of (3.1) with f being a DC function,

f (•, Z) or S is nonconvex (resp. f (•, Z) is nonsmooth), ( 3
min{F (w) = E(g(w, Z) -h(w, Z)) : w ∈ S}, (3.2)
where Z is a random vector determined in some complete probability space (Ω, M, P) Secondly, in (3.2), g and h are possibly nonsmooth, resulting in a very large class of stochastic nonsmooth, nonconvex DC programs that encompasses the vast majority of real-world problems [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF].

(Z : Ω → R n ), while S ⊂ R m is
F (w) = 1 N N i=1 g(w, z i ) - 1 N N i=1 h(w, z i ) . ( 3
Our research is primarily inspired by the role of stochastic DC programs in machine learning, as well as new issues emerging from optimization problems in this field. To begin, let us note that machine learning is a mine of DC programs, and that many machine learning problems can be expressed in the form of the stochastic DC program (3.2). Indeed, formally, a machine learning task results in an optimization problem that involves minimizing a loss function f (w, Z) on a training set of samples, measuring the goodness of the prediction model parameterized by w using the data point Z. A regularizer is frequently added to the loss to promote certain types of solution structure, such as sparsity or low rank. When the loss function and the regularizer are both DC, this learning problem is a DC program. A typical instance of DC regularizers arises in learning with sparsity where the zero-norm (the zero-norm of a vector is the number of its nonzero components) is used to model sparsity. The main approach to deal with the zero-norm is nonconvex approximation, in which the zero-norm is replaced by its nonconvex approximation functions, and it is demonstrated in [START_REF] Le Thi | DC approximation approaches for sparse optimization[END_REF] that all existing nonconvex approximations (e.g., capped ℓ 1 , log-sum penalty, minimax concave penalty, etc.) are DC. There are various examples of DC loss functions. In supervised learning, for example, classification or regression problems such as robust logistic regression [START_REF] Park | Robust penalized logistic regression with truncated loss functions[END_REF], truncated linear regression [START_REF] Chen | Personalized dose finding using outcome weighted learning[END_REF], robust support vector machines [START_REF] Zhou | Unified svm algorithm based on ls-dc loss[END_REF][START_REF] Collobert | Trading convexity for scalability[END_REF],

robust support vector regression [START_REF] Wang | Robust non-convex least squares loss function for regression with outliers[END_REF], phase retrieval [START_REF] Davis | Stochastic model-based minimization of weakly convex functions[END_REF], the nonconvex subproblem of Uzawa Algorithm for Neyman-Pearson classification [START_REF] Gasso | Batch and online learning algorithms for nonconvex neyman-pearson classification[END_REF], or again neural network with ReLU activations [START_REF] Cui | Composite difference-max programs for modern statistical estimation problems[END_REF] (combined with popularly used losses including cross-entropy, Huber loss, square error, LogCosh loss), etc. Several other unsupervised/semi-supervised learning problems possessing DC structures are as follows: Principal Component Analysis [START_REF] Montanari | Non-negative principal component analysis: Message passing algorithms and sharp asymptotics[END_REF], t-Distributed Stochastic Neighbor Embedding [START_REF] Le Thi | Novel DCA based algorithms for a special class of nonconvex problems with application in machine learning[END_REF], Positive

Unlabeled learning [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF], Semi-supervised support vector machines [START_REF] Le | Sparse semi-supervised support vector machines by DC programming and DCA[END_REF], etc.

If we consider the preceding learning problems in the context of stochastic data, i.e., Z is a random vector, then we have to minimize the expectation loss F (w) = E(f (w, Z)) (and possibly with a regu-3.2. Related works larizer), and this optimization problem takes the form of (3.2).

Our contributions. To address the primary difficulty of the problem (3.2), which is its nonconvexity, we study the DCA approach. Moreover, DCA is also introduced in an online manner to handle streaming data. Our contributions are as follows.

First, we develop a general online stochastic DCA scheme for solving the problem (3.2) (where the problem setting will be described in more detail in section 3.3). We combine DCA with Stochastic Approximations (SA) in the online manner: at each iteration, new data is used to construct a stochastic approximation of G and the one of a subgradient of H, and the solution is updated via the DCA's principle. Since the update steps of the proposed algorithms require new fresh samples from the distribution of Z, we refer to our algorithm as online stochastic DCA (abbreviated osDCA). The proposed osDCA enjoyes a double benefit from being an online algorithm: it is suitable to perform streaming data coming from an unknown distribution.

Second, we propose two additional osDCA schemes for situations in which we can compute directly (i.e. the SA procedure is not used) the function G(w) := E(g(w, Z)) (the second osDCA) or H(w) := E(h(w, Z)) (the third osDCA). These two algorithms have some advantages over the first osDCA as they seek to use the additional information of data to make better updates (which are less noisy than the first one) at the expense of computing the exact expectations. Nevertheless, while the first osDCA can be applied to any data, the latter two usually necessitate extra knowledge about Z. A typical case is that the distribution of some partial measurement of Z, µ(Z), is known and either g(w, •) or h(w, •) is a function with respect to µ (see Remark 6).

Third, we rigorously study the convergence analysis of the three proposed methods. For the first algorithm, we establish the subsequential convergence to critical points with probability one. In addition, the proposed algorithm and its convergence analysis can be extended to a more general setting which is F (w) = E(g(w, Z) -h(w, Z)), where Z and Z are two different random vectors. The extension aims to address optimization problems involving with two parallel streams of data. For the second and the third algorithms, the subsequent convergence to DC critical points is also established with milder conditions than those of the first osDCA scheme.

Fourth, we study the practical performance of the proposed algorithms via the expected problem of principal component analysis. Numerical experiments have been conducted carefully on a variety of real-world as well as synthetic datasets to study the proposed algorithms' behaviors in different respects.

Related works

Stochastic optimization has been studied thoroughly for convex problems since the seminal work [START_REF] Robbins | A stochastic approximation method[END_REF],

where the well-known Stochastic Gradient Descent (SGD) was introduced, which really opened a door in numerical optimization for large-scale problems [START_REF] Bottou | Optimization methods for large-scale machine learning[END_REF][START_REF] Bottou | Large-scale machine learning with stochastic gradient descent[END_REF]. In nonconvex setting, stochastic algorithms remain rare, and most of them require the objective to be smooth or partially smooth (some components of the objective are smooth). Overall, there are four approaches to tackle nonconvex stochastic problems. The first, inspired by the aforementioned SGD, includes stochastic (proximal) (sub)gradient-based methods designed primarily for smooth or weakly convex objective functions [START_REF] Ghadimi | Stochastic first-and zeroth-order methods for nonconvex stochastic programming[END_REF][START_REF] Bertsekas | Gradient convergence in gradient methods with errors[END_REF][START_REF] Davis | Stochastic model-based minimization of weakly convex functions[END_REF]. The second is stochastic MM (Majorization-Minimization) for partially smooth objective [START_REF]Stochastic majorization-minimization algorithms for large-scale optimization[END_REF][START_REF] Razaviyayn | A stochastic successive minimization method for nonsmooth nonconvex optimization with applications to transceiver design in wireless communication networks[END_REF], which minimizes a stochastic convex surrogate iteratively to obtain an updated optimization variable. Here the authors con-sidered either smooth error function or smooth surrogate, which implies -in case of DC programs -one DC component should be smooth. The third is stochastic Successive Convex Approximation [START_REF] Ye | Stochastic successive convex approximation for general stochastic optimization problems[END_REF][START_REF] Yang | A parallel decomposition method for nonconvex stochastic multi-agent optimization problems[END_REF] (mainly for smooth objective functions) that is similar to stochastic MM where the sequence of approximation functions are convex but not necessary upper bounds of sample objective functions. The fourth is stochastic DCA for stochastic DC programs -a substantially large class to cover almost all real-world nonconvex optimization problems [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF]. Initial works in this approach include [START_REF] Le Thi | Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification[END_REF][START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF][START_REF] Liu | Two-stage stochastic programming with linearly biparameterized quadratic recourse[END_REF][START_REF] Nitanda | Stochastic Difference of Convex Algorithm and its Application to Training Deep Boltzmann Machines[END_REF][START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF] that consider some special classes of DC problems such as large-sum and/or (partially) smooth, as well as [START_REF] Le Thi | Stochastic difference-of-convexfunctions algorithms for nonconvex programming[END_REF] working on a very general class of stochastic nonsmooth DC programs. To extend beyond the DC programming framework, [START_REF] Metel | Simple stochastic gradient methods for non-smooth non-convex regularized optimization[END_REF] used Moreau envelope which is a DC function to approximate a nonsmooth, nonconvex regularizer, and then developed a stochastic DCA for solving the resulting problem.

It should be noted that, as indicated in [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF], while the (stochastic) MM proposes a general idea to majorize the objective function, (stochastic) DCA gives the simplest and the most closed convex surrogate thanks to DC structures of the objective. Furthermore, usual choices of surrogates of MM result in DCA versions [START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF].

So far, there are very few algorithms for the general setting (3.2). To our knowledge, the paper [START_REF] Le Thi | Stochastic difference-of-convexfunctions algorithms for nonconvex programming[END_REF] is the first work dealing with (3.2) where both DC components are nonsmooth. In that article, several stochastic DCA (SDCA) schemes are proposed in the aggregated update style. That is, all past information (sample realizations) is used to construct subproblems. These algorithms therefore need to store all samples in the computer memory during the computational process. In the present work, we investigate online stochastic DCA for the general problem (3.2) to deal with fast streaming data where we do not need to store samples all the time. Furthermore, thanks to the online mechanism, our proposed algorithms have the adaptive ability which is a great advantage over the SDCA schemes proposed in [START_REF] Le Thi | Stochastic difference-of-convexfunctions algorithms for nonconvex programming[END_REF].

Numerical experiments justify this claim.

Proposed methods

This section develops osDCA schemes for solving the problem (3.2) which can be described as follows.

Problem setting

Let P Z be the probability distribution of Z on R n and Ξ = supp(P Z ) be the support of P Z . By definition, a point x ∈ R n is in supp(P Z ) if P Z (N x ) > 0, for all neighborhood N x of x. Since a measure "lives" in its support, we only need to work in Ξ instead of R n . For instance, a Dirac measure δ a concentrating at a single point a admits a support containing only one point a; a discrete measure µ = ∞ i=1 β i δ a i with β i > 0 admits a support {a 1 , a 2 , . . .}. A basic property of Ξ is that it is closed in R n . Moreover, P Z (Ξ ∁ ) = 0 since R n is the topological Hausdoff space and P Z is a Radon measure in R n . Therefore, only the values of g and h on S × Ξ matter. For simplicity of presentation, we assume that dom g = dom h = S × Ξ. That is, the value of g and h outside S × Ξ is set to +∞. This implies that dom G = dom H = S (recall that G(w) := E(g(w, Z)) and H(w) := E(h(w, Z))). Here we use the convention +∞ -(+∞) = +∞. Moreover, g and h are assumed to be bounded below and Borel measurable, and g(w, Z), h(w, Z) are integrable for all w ∈ S. It is noted that the Borel σ-algebra on R ∪ {+∞} is generated by the order topology of R ∪ {+∞}. As g(•, z) and h(•, z) are convex, lower semi-continuous for all z ∈ Ξ, so are G and H. In the sequel we will consider the problem (3.2) in its DC form min{F (w) = G(w) -H(w) : w ∈ S}.

(3.4)

Moreover, we need some mild assumptions as follows.

Assumption 1. i. For all z ∈ Ξ, dom ∂h(•, z) = S.

ii. ρ := ρ H + inf z∈Ξ ρ(g(•, z)) > 0.

iii. There exists a Borel measurable selector τ such that

∀w ∈ S, z ∈ Ξ, τ (w, z) ∈ ∂ w h(w, z),
where τ is L 2 uniformly bounded in the sense that there exists a Borel measurable function τ such that τ (Z) 2 is integrable and ∀w ∈ S, z ∈ Ξ, ∥τ (w, z)∥ ≤ τ (z).

iv. sup w∈S |F (w)| < +∞.

Remark 4. It is observed that the assumptions i), iii) and iv) are mild. On another hand, thanks to the regularization technique introduced in [START_REF] Pham Dinh | Convex analysis approach to DC programming: theory, algorithms and applications[END_REF], Assumption 1(ii) is easily fulfilled by adding an L 2 regularizer to both DC components.

Next, to measure the complexity of a class of functions, we adopt the following notion of Rademacher average (or Rademacher complexity), see, e.g. [START_REF] Ermoliev | Sample average approximation method for compound stochastic optimization problems[END_REF].

Rademacher average/complexity

For a set of points {z 1 , z 2 , . . . , z l } := z l in Ξ, the Rademacher average R l (g, z l ) is defined as

R l (g, z l ) = E σ sup w∈S 1 l l i=1 σ i g(w, z i ) ,
where σ ′ i s are i.i.d. random numbers such that σ i ∈ {±1} with P(σ i = 1) = P(σ i = -1) = 1/2. The Rademacher average of a family of functions {g(•, z) : z ∈ Ξ}, denoted by R l (g, Ξ), is defined as

R l (g, Ξ) = sup z 1 ∈Ξ,z 2 ∈Ξ,...,z l ∈Ξ R l (g, z l ).
Assumption 2. i. There exists a Borel measurable function g :

R n → R such that g(Z) is integrable and |g(w, z)| ≤ g(z), ∀w ∈ S, z ∈ Ξ.
ii. R k (g, Ξ) ≤ N g /k α with N g > 0 and α > 0.

It is noteworthy that Assumption 2(ii) holds for various cases described as follows [START_REF] Ermoliev | Sample average approximation method for compound stochastic optimization problems[END_REF].

Case 1. Holder functions g(•, z), z ∈ Ξ: Let D be the length of a cube in R m containing the compact set S. Suppose that ∃M, L > 0 and γ ∈ (0, 1] such that

1. |g(w, z)| ≤ M, ∀w ∈ S, z ∈ Ξ, 2. |g(x, z) -g(y, z)| ≤ L∥x -y∥ γ , ∀x, y ∈ S, z ∈ Ξ.
Then, for any α ∈ (0, 1/2), R k (g, Ξ) ≤ N g /k α , where

N g = LD γ m γ 2 + M √ m γ(1 -2α)e .
Case 2. Holder functions g(w, •), w ∈ S: Suppose that Ξ is compact, let D be the length of a cube in R n that contains Ξ. Suppose that there exists M, L, γ > 0 such that 1. |g(w, z)| ≤ M, ∀w ∈ S, z ∈ Ξ.

2. |g(w, u) -g(w, v)| ≤ L∥u -v∥ γ , ∀w ∈ S, u, v ∈ Ξ.
Then R k (g, Ξ) ≤ N g /k α where N g = M + LD γ n γ 2 and α = γ/(2γ + n). Case 3. Discrete set Ξ: Suppose that the number of elements of Ξ is finite, say |Ξ| = N Ξ . Furthermore, assume that there exists

M > 0 such that |g(w, z)| ≤ M, ∀w ∈ S, z ∈ Ξ. Then, R k (g, Ξ) ≤ M N Ξ /k, hence, α = 1/2.
It turns out that Assumption 2 is not strong; hence a class of functions meeting the criteria is wide to cover many problems arising in practice. In three cases of Rademacher complexity presented above, though α in case 2 can be very small in the high-dimension regime, which makes our next algorithm impractical, the other two cases have α = 1/2 or arbitrarily near to 1/2, which are appropriate sample rates in practice. It should be stressed that the Rademacher complexity measures the richness of a class of functions. Therefore, roughly speaking, the function g must be quite "simple" in this Rademacher sense. This criterion naturally fulfills our demand to control the variability of stochastic approximations made on g.

Online Stochastic DCA schemes

We introduce an osDCA scheme described in Algorithm 4.

Algorithm 4 Online Stochastic DCA

Initialization: Choose w 0 ∈ S and a sequence of sample sizes {n k }, set k = 0. repeat 1. Draw independently n k samples Z k,1 , . . . , Z k,n k from the distribution of Z in such a way that they are also independent of the past.

Compute t

k = 1 n k n k i=1 τ (w k , Z k,i ). 3.
Compute w k+1 , an optimal solution to the problem

min 1 n k n k i=1 g(w, Z k,i ) -⟨t k , w⟩ : w ∈ R m . (3.5) 4. Set k = k + 1. until Stopping criterion.
Algorithm 4 is well defined with probability 1. To be more specific, the set of events that makes

Algorithm 4 work is V = ∩ ∞ k=1 ∩ n k i=1 (Z k,i ∈ Ξ)
and hence P(V) = 1. We denote Z k = Z k,1:n k and P k = σ(Z 0 , Z 1 , . . . , Z k-1 , w 0 , w 1 , . . . , w k ) which is the σ-algebra generated by associated random variables. We observe that {w k } ∞ k=0 is a predictable process and {t k } ∞ k=0 is an adapted process with respect to the filtration {P k+1 } ∞ k=0 . The convergence results of Algorithm 4 are presented in Theorem 10.

Theorem 10. Under Assumptions 1 and 2, let β = min{α, 1}, if the sequence of sample sizes {n k } satisfies ∞ k=1 n -β k < +∞, the iterations of Algorithm 4 satisfy: 1. There exists F ∞ integrable such that F (w k ) → F ∞ a.s.

∞

k=1 ∥w k+1 -w k ∥ 2 < +∞ a.s. 3. There exists a measurable set L ⊂ Ω with P(L) = 1 such that for each ω ∈ L, every limit point

of {w k (ω)} is a critical point of F = G -H.
Proof. 1. Let ν(w) := E(τ (w, Z)). It follows from the definition of the selector τ that: for every w ∈ S,

h(w ′ , Z) ≥ h(w, Z) + ⟨τ (w, Z), w ′ -w⟩, ∀w ′ ∈ R m . (3.6)
By taking expectation on both sides of (3.6), we obtain

H(w ′ ) ≥ H(w) + ⟨ν(w), w ′ -w⟩, ∀w ′ ∈ R m , or ν(w) ∈ ∂H(w). Since ν(w k ) ∈ ∂H(w k ), H(w k+1 ) ≥ H(w k ) + ⟨ν(w k ), w k+1 -w k ⟩ + ρ H 2 ∥w k+1 -w k ∥ 2 . (3.7)
On the other hand, it follows from definition of w k+1 that

1 n k n k i=1 g(w k , Z k,i ) ≥ 1 n k n k i=1 g(w k+1 , Z k,i ) + ⟨t k , w k -w k+1 ⟩ + inf z∈Ξ ρ(g(•, z)) 2 ∥w k+1 -w k ∥ 2 . (3.8)
From (3.7) and (3.8), we obtain

1 n k n k i=1 g(w k+1 , Z k,i ) ≤ H(w k+1 ) + 1 n k n k i=1 g(w k , Z k,i ) -H(w k ) - ρ 2 ∥w k+1 -w k ∥ 2 + ⟨t k -ν(w k ), w k+1 -w k ⟩, (3.9)
with ρ = ρ H + inf z∈Ξ ρ(g(•, z)). By taking conditional expectation with respect to P k both sides of (3.9), we obtain

E(F (w k+1 ) -F (w k )|P k ) ≤ E(⟨t k -ν(w k ), w k+1 -w k ⟩|P k ) + E G(w k+1 ) - 1 n k n k i=1 g(w k+1 , Z k,i )|P k - ρ 2 E(∥w k+1 -w k ∥ 2 |P k ). (3.10)
By applying Schwartz inequality and Holder inequality,

E(⟨t k -ν(w k ), w k+1 -w k ⟩|P k ) ≤ E(∥t k -ν(w k )∥ 2 |P k ) 1 2 E(∥w k+1 -w k ∥ 2 |P k ) 1 2 . (3.11)
By using AM-GM inequality, we obtain

E(∥t k -ν(w k )∥ 2 |P k ) 1 2 E(∥w k+1 -w k ∥ 2 |P k ) 1 2 ≤ 1 2ρ E(∥t k -ν(w k )∥ 2 |P k ) + ρ 2 E(∥w k+1 -w k ∥ 2 |P k ). (3.12)
It follows from the independence of Z k,i and Z k,j (i ̸ = j) that

E ∥t k -ν(w k )∥ 2 |P k = 1 n 2 k n k i=1 E ∥τ (w k , Z k,i ) -ν(w k )∥ 2 |P k .
We observe that

E ∥τ (w k , Z k,i ) -ν(w k )∥ 2 |P k = E Z ∥τ (w k , Z)∥ 2 -∥ν(w k )∥ 2 = V Z (τ (w k , Z)).
Thus,

E ∥t k -ν(w k )∥ 2 |P k = 1 n k V Z (τ (w k , Z))
. Combining this and (3.10), (3.11), (3.12), and we obtain

E(F (w k+1 ) -F (w k )|P k ) ≤ V Z (τ (w k , Z)) 2ρ × n k + E G(w k+1 ) - 1 n k n k i=1 g(w k+1 , Z k,i )|P k . (3.13)
Next, we make an upper bound on the right-hand side of (3.13). Firstly, the (nonnegative) term V Z (τ (w k , Z))

is bounded above by E(τ (Z) 2 ). Secondly, we show that

E sup w∈S G(w) - 1 n k n k i=1 g(w, Z k,i ) ≤ 2R n k (g, Ξ). (3.14)
To prove (3.14), let us first introduce "ghost samples" Z ′ k,1 , Z ′ k,2 , . . . , Z ′ k,n k (similar to the arguments in [START_REF] Boucheron | Theory of classification: a survey of some recent advances[END_REF]) that are independent of all Z k,i and identically distributed with Z. By Jensen's inequality, we get

1 n k n k i=1 g(w, Z k,i ) -E(g(w, Z)) ≤ E 1 n k n k i=1 g(w, Z k,i ) -g(w, Z ′ k,i ) |Z k,i , i = 1, n k .
Thus, E sup

w∈S 1 n k n k i=1 g(w, Z k,i ) -E(g(w, Z)) ≤ E sup w∈S 1 n k n k i=1 g(w, Z k,i ) - 1 n k n k i=1 g(w, Z ′ k,i ) .
Now let σ 1 , σ 2 , . . . , σ n k be independent random variables with P(σ i = 1) = P(σ i = -1) = 1 2 in such a way that they are also independent of Z k,i and Z ′ k,i . Then,

E sup w∈S 1 n k n k i=1 g(w, Z k,i ) - 1 n k n k i=1 g(w, Z ′ k,i ) = E sup w∈S 1 n k n k i=1 σ i (g(w, Z k,i ) -g(w, Z ′ k,i )) ≤ 2E E σ sup w∈S 1 n k n k i=1 σ i g(w, Z k,i ) = 2E(R n k (g, Z n k )) ≤ 2E(R n k (g, Ξ)) = 2R n k (g, Ξ).
Now, we establish the almost sure convergence of the sequence {F (w k )} as follows. Assumption 1(iv)

implies that there exists R such that

F (w) ≥ R, ∀w ∈ S. Let D(w) = F (w) -R ≥ 0 and S k = [E(D(w k+1 ) -D(w k )|P k ) > 0]
. Since S k is P k -measurable and by using (3.13), (3.14) , we obtain

∞ k=1 E(1 S k (D(w k+1 ) -D(w k ))) = ∞ k=1 E E(1 S k (D(w k+1 ) -D(w k ))|P k ) ≤ 1 2ρ ∞ k=1 E(V Z (τ (w k , Z))) n k + 2 ∞ k=1 R n k (g, Ξ) ≤ E(τ (Z) 2 ) 2ρ ∞ k=1 1 n k + 2N g ∞ k=1 1 n α k < +∞.
It follows from semimartingale convergence theorem [START_REF] Metivier | Semimartingales[END_REF] that there exists D ∞ integrable such that D(w k ) → D ∞ a.s., which implies F (w k ) → F ∞ = D ∞ + R a.s.

By AM-GM inequality, ⟨t

k -ν(w k ), w k+1 -w k ⟩ ≤ 1 ρ ∥t k -ν(w k )∥ 2 + ρ 4 ∥w k+1 -w k ∥ 2 .
Combining this inequality with (3.9), we get

ρ 4 ∥w k+1 -w k ∥ 2 ≤ F (w k ) -F (w k+1 ) + G(w k+1 ) -G(w k ) - 1 n k n k i=1 g(w k+1 , Z k,i ) -g(w k , Z k,i ) + 1 ρ ∥t k -ν(w k )∥ 2 .
By applying Lebesgue dominated convergence theorem [17, Theorem 4.2], we get

ρ 4 E ∞ k=1 ∥w k -w k+1 ∥ 2 ≤ E(F (w 1 )) -E(F ∞ ) + M ρ ∞ k=1 1 n k + 2N g ∞ k=1 1 n α k < ∞.
Therefore, ∞ k=1 ∥w k -w k+1 ∥ 2 < +∞ a.s.

Denote by

G k (w) = 1 n k n k i=1 g(w, Z k,i ). It follows from t k ∈ ∂G k (w k+1 ) and ν(w k ) ∈ ∂H(w k ) that ⟨w k+1 , t k ⟩ = G k (w k+1 ) + G * k (t k ), ⟨ν(w k ), w k ⟩ = H(w k ) + H * (ν(w k )).
Then we obtain

G k (w k ) -H(w k ) ≥ H * (ν(w k )) -G * k (t k ) + ⟨t k -ν(w k ), w k ⟩ ≥ G k (w k+1 ) -H(w k+1 ) + ⟨t k -ν(w k ), w k -w k+1 ⟩, which implies G k (w k ) -H(w k ) -H * (ν(w k )) + G * k (t k ) → 0, (3.15) since G k (w k ) -H(w k ) → F ∞ , t k -ν(w k ) → 0, and G k (w k+1 ) -H(w k+1 ) → F ∞ .
Hence, (3.15) implies G(w k ) + G * k (t k ) -⟨w k , ν(w k )⟩ → 0. On another hand, from the definition of the conjugate functions G * and G * k we get

|G * k (t k ) -G * (t k )| ≤ sup x∈S |G k (x) -G(x)| → 0.
Thus G(w k ) + G * (t k ) -⟨w k , ν(w k )⟩ → 0 a.s. Now let L be an intersection of sets with probability 1 gained from all almost surely true statements from the beginning of the proof, we have P(L) = 1

since there are at most countably finite statements. Let ω ∈ L, we have {w k (ω)} and {ν(w k (ω))} are bounded. Let w * ∈ S be a limit point of {w k (ω)}, there exists a subsequence {w k j (ω)} such that w k j (ω) → w * . By extracting a subsequence of {ν(w k j (ω))} if necessary, we can assume that ν(w k j (ω)) → ν * , which implies t k j (ω) → ν * . Therefore, G(w k j (ω)) + G * (t k j (ω)) → ⟨w * , ν * ⟩. By letting j → +∞ and noting that θ(w, z) = G(w) + G * (z) is lower semicontinuous, we obtain G(w * ) + G * (ν * ) ≤ ⟨w * , ν * ⟩. On the other hand, according to Young's inequality, G(w * ) + G * (ν * ) ≥ ⟨w * , ν * ⟩.

Therefore, G(w * ) + G * (ν * ) = ⟨w * , ν * ⟩. In other words, ν * ∈ ∂G(w * ). Furthermore, for each w ∈ S, it follows from ν(w k j (ω)) ∈ ∂H(w k j (ω)) that H(w) ≥ H(w k j (ω)) + ⟨ν(w k j (ω)), w -w k j (ω)⟩, which implies H(w) ≥ H(w * ) + ⟨ν * , w -w * ⟩. Therefore, ν * ∈ ∂H(w * ), and we conclude that w * is a critical

point of F = G -H since ∂G(w * ) ∩ ∂H(w * ) ̸ = ∅.
Remark 5. (i) The algorithm only uses samples at the current time to update the solution (past samples are no longer used). Therefore, even if the distribution of Z changes at a certain time (suppose that, due to some real-world events, Z becomes Z ′ at the iteration k), the algorithm will automatically solve the problem (3.2) with Z being replaced by Z ′ . Indeed, the current solution w k can be considered as the initial point for restart, the algorithm continues operating based on new samples from the distribution of Z ′ . Theorem 10 is still valid, and the subsequential convergence with probability one to DC critical points of the DC problem associated with the new distribution is guaranteed. This is indeed an advantage of the osDCA. In contrast, intuitively, stochastic algorithms using aggregated update (still using old samples to compute the current solution) barely have this kind of adaptivity. We will conduct numerical experiments to study this aspect.

(ii) Our algorithm and the convergence analysis can be extended to deal with the more general problem whose the random variables inside the first and the second DC components are not necessarily the same, i.e., F (w) = E(g(w, Z)) -E(h(w, Z)). At the iteration k, we approximate values of G and the subgradients of H by using n k independent random samples from the distribution of Z and ñk independent random samples from the distribution of Z, respectively. The sample size sequences {n k } and {ñ k } need to increase in such a way that ∞

k=1 n -α k < ∞ and ∞ k=1 ñ-1 k < ∞.
Next, we will discuss two scenarios where one can directly compute (without stochastically approximation) values of G or H. Since the information of G (resp. H) can be achieved, we will modify Algorithm 4 to exploit this advantage. Note that these two schemes are not special cases of Algorithm 4, but they will coincide with Algorithm 4 in some cases (Remark 6).

G can be directly computed without approximation In this case, G does not need to be stochastically approximated, we replace the approximation of G in step 3 of Algorithm 4 by its true value, which results in Algorithm 5. With this algorithm, we obtain stronger convergence results since G is computed exactly. Note that, in the convergence results of Algorithm 4, we impose Assumption 2 in order to control the variance of the stochastic estimator of G. To study the convergence of Algorithm 5, we do not need such an assumption.

Furthermore, in Assumption 1, we replace the convexity condition ρ H + inf z∈Ξ ρ(g(•, z)) > 0 by the weaker one ρ H + ρ G > 0, which gives rise to a milder assumption called Assumption 1'. We obtain Theorem 11 whose proof is similar to that of Theorem 10.

Theorem 11. Under Assumption 1', if the sequence of sample sizes {n k } satisfies ∞ k=1 n -1 k < +∞, then the iterations of Algorithm 5 satisfy:

1. There exists F ∞ integrable such that F (w k ) → F ∞ a.s.

∞

k=1 ∥w k+1 -w k ∥ 2 < +∞ a.s. 3. There exists a measurable set L ⊂ Ω with P(L) = 1 such that for each ω ∈ L, every limit point

of {w k (ω)} is a critical point of F = G -H.
H can be directly computed without approximation In this case, we replace the stochastic estimator of the subgradient of H in Algorithm 4 by the true subgradient of H to obtain the following algorithm.

Algorithm 6 Online Stochastic DCA with exact H Algorithm 4, in which the step 2 is replaced by 2. Compute t k ∈ ∂H(w k ).

Since we work directly on H, we replace Assumption 1(i) by dom ∂H = S. Likewise, Assumption 1(iii) is replaced by: there exists M > 0 such that ∀w ∈ S, ∀t ∈ ∂H(w) : ∥t∥ ≤ M. These modifications bring about a new set of assumptions called Assumption 1". We obtain the following convergence results whose proof is similar to that of Algorithm 10.

Theorem 12. Under Assumptions 1" and 2, if the sequence of sample sizes {n k } satisfies ∞ k=1 n -α k < +∞, the iterations of Algorithm 6 satisfy:

1. There exists F ∞ integrable such that F (w k ) → F ∞ a.s.

∞

k=1 ∥w k+1 -w k ∥ 2 < +∞ a.s. 3. There exists a measurable set L ⊂ Ω with P(L) = 1 such that for each ω ∈ L, every limit point of {w k (ω)} is a critical point of F = G -H. Remark 6. (i) In practice, thanks to the flexibility of DC decompositions, one can usually formulate the problem (3.1) as a stochastic DC program, f (w, z) = g(w, z) -h(w, z), where either g or h does not depend on z. When g(w, z) does not depend on z, Algorithm 5 coincides with Algorithm 4. For example, if the functions f (•, z) are L-smooth with the same constant L for all z ∈ Ξ, f has the following DC decomposition: f (w, z) = (L/2)∥w∥ 2 -(L/2)∥w∥ 2 -f (w, z) . Likewise, when h(w, z) does not depend on z, Algorithm 6 and Algorithm 4 coincide. For instance, suppose that there exists a convex function φ(w) such that f (w, z) + φ(w) are convex for all z ∈ Ξ (in particular, when φ(w) = (κ/2)∥w∥ 2 , f (•, z) are weakly convex), f has the following DC decomposition: f (w, z) = (f (w, z) + φ(w)) -φ(w).

(ii) Apart from the use of the above appropriate DC decompositions so that G or H is a deterministic function and thereafter Algorithm 5 or 6 is applicable, the exact computation of G and H requires additional knowledge about Z. For instance, suppose that the distribution of Z is unknown but the distribution of its measurement, µ(Z), is known (P µ ). For example, µ can be the magnitude ∥Z∥, a partial observation Z * (Z * is a part of the vector Z), the difference Z 1 -Z 2 (where Z = (Z 1 , Z 2 )), etc. In such cases, if g(w, •) is a function with respect to µ, i.e., g(w, z) = u(w, µ(z)), the expectation E(g(w, z)) = u(w, µ)dP µ which is possibly computed. Similar arguments apply to h(w, z).

(iii) In big data analytics, large-sum problems play a key role. We consider the following large-sum objective function

F (w) = N i=1 α i f i (w) = N i=1 α i g i (w) - N i=1 α i h i (w),
where g i , h i are convex, α i ≥ 0 for all i = 1, N and N i=1 α i = 1. The function F can be rewritten as F (w) = E(g I (w)) -E(h I (w)), where I is a random index with P(I = i) = α i . In this case, the distribution of I is known completely. However, as N can be very large, we may still need to apply osDCA schemes. Furthermore, since I is known, we have full freedom to choose Algorithm 4, Algorithm 5, or Algorithm 6 to apply, which leads to -in general -three distinctive algorithms. The practical trade-off between these algorithms would be which DC component (or none of them) is cheaper to be computed directly.

Applications: solving the Expected PCA

In this section, we will apply osDCA schemes to the expected problem of PCA to study the generalization capacity of the proposed methods.

osDCA schemes for solving Expected PCA

We consider the following expected problem of PCA (denoted by E-PCA) as follows [START_REF] Montanari | Non-negative principal component analysis: Message passing algorithms and sharp asymptotics[END_REF],

min{- 1 2 E(⟨w, Z⟩ 2 ) : ∥w∥ ≤ 1, (E-PCA)
where Z is a normalized random vector, i.e. ∥Z∥ = 1, with unknown distribution. The situation in which we are interested is the data obtained online. The problem (E-PCA) can be considered as the theoretical problem of the classic PCA (and -vice versa -the classic PCA is the empirical problem of (E-PCA)). The problem (E-PCA) aims to generalize the compressing capacity of the classical PCA on unseen data.

Firstly, we see that the problem (E-PCA) is nonconvex and it can be formulated as a DC problem of the form (3.4), where

G(w) = λ 2 ∥w∥ 2 , H(w) = E λ 2 ∥w∥ 2 + 1 2 ⟨w, Z⟩ 2 , (3.16)
S = {w ∈ R m : ∥w∥ ≤ 1} and λ > 0. Although we have a very natural DC decomposition with G(w) = 0, H(w) = E 1 2 ⟨w, Z⟩ 2 , here we add λ 2 ∥w∥ 2 to both DC components to fulfill to Assumption 1(ii). Since the values G are directly obtained without approximation, Algorithm 4 coincides with Algorithm 5. We call this scheme osDCA-1, where the k-th iteration is described as follows.

1. Receive n k samples Z k,1 , . . . , Z k,n k . 2. Compute t k = λw k + 1 n k n k i=1 ⟨w k , Z k,i ⟩Z k,i . 3. Update w k+1 =    λ -1 t k if ∥t k ∥ ≤ λ t k /∥t k ∥ otherwise.
Secondly, it is well-known that if a function θ has L-Lipschitz continuous gradient, then (L/2)∥ • ∥ 2 -θ and (L/2)∥ • ∥ 2 + θ are convex. Therefore, we have another DC decomposition for the problem (E-PCA) as follows,

G(w) = E L 2 ∥w∥ 2 -1 2 ⟨w, Z⟩ 2 , H(w) = E L 2 ∥w∥ 2 + 1 2 ⟨w, Z⟩ 2 .
(3.17)

Since G and H remain unknown, we apply Algorithm 4 to this DC problem. Obviously the family {g(•, z) : ∥z∥ = 1} is uniformly Lipschitz and uniformly bounded by a constant, therefore, the rate α in Assumption 2 can be chosen arbitrarily in (0, 1/2). With this setup, we obtain a second scheme called osDCA-2 whose the k-th iteration is described as follows.

1. Receive n k samples Z k,1 , Z k,2 , . . . , Z k,n k .

Compute the stochastic gradient

t k = Lw k + 1 n k n k i=1 ⟨w k , Z k,i ⟩Z k,i .
3. Solve the following convex program to get w k+1 ,

min w∈S L 2 ∥w∥ 2 - 1 2n k n k i=1 ⟨w, Z k,i ⟩ 2 -⟨t k , w⟩ . (3.18)
The problem (3.18) is convex and can be solved by an existing convex optimization solver. Thanks to the efficiency of DCA we use again it to solve (3.18), via the following "false" DC decomposition We get a simple DCA scheme where the solutions of convex subproblems have closed-form. The (deterministic) DCA takes the current solution u 0 = w k as the initial point, then operates until the stopping criterion which is ∥u l+1 -u l ∥ < ϵ is met, where ϵ > 0 is the error tolerance.

g(w) = L 2 ∥w∥ 2 , h(w) = 1 2n k n k i=1 ⟨w, Z k,i ⟩ 2 + ⟨t k , w⟩.

Numerical experiments

Datasets

The numerical experiments are conducted on standard machine learning datasets on LIBSVM. The information of the used datasets is described in Table 3

Comparative algorithms

We compare our algorithms with two versions of Projected Stochastic Subgradient method (PSS) [START_REF] Davis | Stochastic model-based minimization of weakly convex functions[END_REF] an online algorithm for weakly convex objective functions, four Stochastic DCA schemes (SDCA) [START_REF] Le Thi | Stochastic difference-of-convexfunctions algorithms for nonconvex programming[END_REF] proposed for nonconvex, nonsmooth DC programs, and two benchmark algorithms for solving online PCA including the AdaOja [START_REF] Henriksen | Adaoja: Adaptive learning rates for streaming PCA[END_REF] and MaxAP [START_REF] Dhanaraj | Stochastic principal component analysis via mean absolute projection maximization[END_REF].

Experiment setup and results

The numerical experiments comprise of five parts. The first experiment is a comparison between algorithms, the second experiment studies our algorithms' behaviors when the DC decomposition of the problem varies, the third experiment compares between convex solvers for solving subproblems, the fourth experiment studies the adaptive capacity of osDCA schemes, and the fifth experiment compares directly the differences in the practical performance of the three proposed osDCA schemes.

Experiment 1. We compare osDCA schemes with two versions of PSS (constant stepsize policy and diminishing stepsize policy), four SDCA schemes, AdaOja, and MaxAP. Firstly, we run cross-validation 10 times on each dataset to find the best hyperparamters for the comparative algorithms. To be specific, for the PSS with constant stepsize, we cross-validate to find the best stepsizes from {0.001, 0.005, 0.01, 0.015, 0.02}; Meanwhile, for the PSS with adaptive stepsize α k = c/k, c is searched from {4, 5, 6, . . . , 15}. On the other hand, the AdaOja automatically tunes its stepsize based on the gradient information. The MaxAP uses diminishing stepsize α k = c/k where c is chosen from {1, 2, . . . , 10}. For the four SDCA schemes, it should be stressed that SDCA1 and SDCA3 require the first DC component of the objective to be explicitly defined, meanwhile, SDCA2 and SDCA4 can handle the unknown first DC component. Therefore, we apply SDCA1 and SDCA3 to (3.16) with λ = 10 -6

(small λ yields good results [START_REF] Le Thi | Stochastic difference-of-convexfunctions algorithms for nonconvex programming[END_REF]); meanwhile, SDCA2 and SDCA4 are applied to (3.17) where L = 1.

We use the sequence of equal weights for all four SDCA schemes. On the other hand, based on the theoretical analysis, the parameters of osDCA schemes are chosen as follows. For the osDCA-1, we choose the sequence of sample sizes as n k = k 2 , and λ = 1 which is a neutral number. For the osDCA-2, the sequence of sample sizes is chosen as All experiments are performed on a PC Intel(R) Core(TM) i7-8700 CPU @3.20GHz of 16 GB RAM.

n k = k 3 ,
Figures 3.1 and 3.2 illustrate the performance of all algorithms (we only display six out of twelve datasets). Moreover, to show the final effect of the learned system, we report in Table 3.2 the average reconstruction errors (on test sets) when reconstructing compressed data from the learned eigenspace.

Comparisons between osDCA schemes and PSS, AdaOja, MaxAP. Our algorithms take a very short amount of time to pass through the training sets while obtaining really small suboptimality values, say 10 -4 ∼ 10 -5 in most of the cases. In contrast, the PSS with constant stepsize and the AdaOja converge slower and usually can not obtain the comparable objective values. On the other hand, PSS with diminishing stepsize performs very well and obtains similar suboptimality to osDCA schemes, where the differences are negligible. Lastly, the MaxAP performs quite poorly in this experiment.

Comparisons between osDCA and SDCA. About the solutions' quality, osDCA schemes and SDCA1, SDCA3, SDCA4 are similar in all cases; Meanwhile, the SDCA2 is slighly worse in some cases (e.g. letter, SensIT Vehicle). On the other hand, the osDCA-2 usually obtains such solutions at a faster rate than SDCA2 and SDCA4.

Overall, among the top six best algorithms (osDCA-1,2, SDCA1,3,4, PSS diminishing stepsize), time to pass through the entire training datasets (only considered in datasets where the computational time is significant) of osDCA-1 is the shortest, osDCA-2 is the second-best. In comparison, SDCA1, Furthermore, it is well-known that there are two main factors needed to be carefully considered when designing any DCA (or its variants), namely the DC decomposition of the problem and the convex solver for subproblems. Therefore, we consider the following experiments to study our proposed algorithms' behaviors within these two mentioned perspectives.

Experiment 2. Our aim is to study the behavior of osDCA-1 when λ varies (change the DC decomposition of the problem). It is observed that, to surely fulfill the strong convexity condition ρ G +ρ H > 0,

we add the regularization term λ∥ • ∥ 2 to both G and H components. A natural question raised is that:

suppose H is already strongly convex, will we obtain some "optimal" performance if we do not use this regularization term? This curiosity motivates us to perform the osDCA-1 scheme with DC decomposition g(w, z) = 0, h(w, z) = 1 2 ⟨w, z⟩ 2 . Before presenting the experimental results, let us discuss a little bit about the condition ρ H > 0 in this case. We know that this condition does not always hold and it is equivalent to E(ZZ ⊤ ) being positive definite. By definition, the positive definiteness of E(ZZ ⊤ ) is equivalent to E (w ⊤ Z) 2 > 0, ∀w ̸ = 0. Therefore, this condition is violated if there exists w 0 ̸ = 0 such that E((w ⊤ 0 Z) 2 ) = 0, or equivalently w T 0 Z = 0 almost surely. In other words, the condition ρ H > 0 does not hold if there is a perfectly linear dependence between features of the random vector Z. on the YearPredictionMSD dataset. We observe that, the optimal performance of osDCA-1 is achieved at some moderate values of λ, say, from 1 to 5. Besides, the quality of the performance is not monotone with respect to λ. With large value of λ, osDCA-1 somehow gets stuck at the beginning.

The performance of osDCA-1 is gradually improved as λ decreases up to a certain value, and then the performance slightly deteriorates as λ continues to approach 0. Experiment 3. We study the performance of osDCA-2 with different convex solvers for subproblems. To be specific, beside the (deterministic) DCA used in the osDCA-2 scheme, we want to use the industrial CPLEX for solving the convex subproblems. Figure 3.4 shows the difference between osDCA-2 using deterministic DCA and CPLEX for solving convex subproblems. It is observed that while the suboptimality values of these two algorithms are similar, osDCA-2 using DCA for the convex subproblem is faster than osDCA-2 with CPLEX.

Experiment 4. We study the adaptive capacity of osDCA schemes compared with SDCA schemes when there is an abrupt change in the distribution of Z. We describe the context of the problem as follows. We are receiving streaming data from an unknown distribution (the data is -in fact -realizations of Z). At a certain time, suppose that there is a real-world event that makes the distribution of Z change (Z becomes some Z ′ ). We do not know this event (and hence, the change of Z is also unknown to us)

and continue to receive streaming data from the changed distribution. From that time, we want to solve (3.2) with Z being replaced by Z ′ since the new random variable Z ′ is more relevant than Z.

To this end, we generate a synthetic dataset as follows. The dataset consists of two subdatasets Experiment 5. Finally, we compare the practical performance of the three proposed algorithms in a case on which they are all applicable, i.e. G and H can be exactly computed. Let us suppose that the random variable Z is known to have a discrete distribution over a set Ξ of N elements, P(Z = z i ) = α i for all z i ∈ Ξ, where α i > 0 and N i=1 α i = 1. We use the DC decomposition (3.17) to test the performance of the three algorithms. Since α i are known, we can compute G(w) and ∇H(w) exactly.

The probability vector α = (α 1 , α 2 , . . . , α N ) is randomly generated. Figure 3.6 shows the difference on the practical performance of our three proposed algorithms. We observe that Algorithms 5 and 6 always obtain better solutions in comparison to Algorithm 4. This result can be explained as Algorithms 5 and 6 use more information about the DC decomposition than Algorithm 4 which approximates both DC components at each iteration.

Conclusion

We have designed three online stochastic algorithms based on the DCA to handle stochastic nonsmooth, nonconvex DC programs. The first scheme stochastically approximates both DC components while the other two are designed for the context that one of two DC components can be directly computed.

The convergence properties of the proposed algorithms are rigorously studied, and the almost sure convergence to critical points is established. As online stochastic algorithms, the osDCA schemes gain a competitive edge when dealing with streaming data. The benefits of osDCA schemes include remedying storage burden and the ability to adapt to new changes of data distribution, as well as the efficiency and flexibility of DCA. On the other hand, it is well-known that the variance of stochastic estimators of online stochastic algorithms is high, which creates difficulties in the convergence analysis, especially in nonconvex and nonsmooth settings. Our algorithms' convergence results hold thanks to the increase of sample sizes. Moreover, the rate of this increase is determined based on the Rademacher complexity of the family of functions {g(•, z) : z ∈ Ξ}. Nevertheless, such complexity is not always easy to compute.

In future works, we would like to improve this condition and provide a better rate. To study the practical behaviors of the proposed algorithms, we conduct numerical experiments on the expected problem of PCA with streaming data from an unknown distribution. The numerical experiments justify the efficiency of proposed algorithms. Indeed, the proposed osDCA schemes obtain good solutions within a short time. In addition, the adaptive capacity of osDCA schemes have been confirmed: after a change of the data distribution, our algorithms quickly adapt to the new distribution. As a comparison, SDCA schemes do not have this ability. Further experimental insights confirm the important role of DC decompositions and convex solvers for subproblems in DCA. It should be noted that DCA plays a double roles in these osDCA schemes, as it has also been used for solving the convex subproblems. And numerical experiments showed that the (deterministic) DCA is a very efficient and robust convex solver. Finally, via a direct comparison between the three proposed algorithms, it seems consistent that Algorithms 5

and 6 using the exact information of G or H (when possible) improve the performance of Algorithm 4.

Chapter 4

Markov chain Stochastic DCA and applications in Deep learning

Abstract. This chapter studies a broad class of nonsmooth nonconvex stochastic DC algorithms with endogenous uncertainty and the absence of i.i.d. (independent and identically distributed) samples. Instead, we assume that only Markov chains with sequences of distributions that converge to the target distributions can be accessed. Markovian noise occurs in a variety of contexts, including Bayesian inference, reinforcement learning, and stochastic optimization in high-dimensional or combinatorial spaces, therefore this setting is legitimate. We then develop a stochastic algorithm termed Markov chain stochastic DCA (MCSDCA). The convergence analysis is established in both asymptotic and nonasymptotic senses. The MCSDCA is then applied to deep learning via PDEs regularization, where two realizations of MCSDCA are constructed, namely MCSDCA-odLD and MCSDCA-udLD, based on overdamped and underdamped Langevin dynamics, respectively.

Introduction

We study the following nonsmooth nonconvex stochastic DC program

min x∈R n {F (x) := G(x) -H(x)}, (4.1)
where G and H are convex, and a subgradient of H is given in the expectation form

E P (ξ|x) (v(x, ξ)) ∈ ∂H(x)
involving with the endogenous uncertainty of ξ (its distribution is determined by the optimization variable x). Furthermore, we suppose that i.i. domains of machine learning such as variational inference, local entropy minimization, Boltzmann machine, as well as operational research where the uncertainty is influenced by the decision. The considered problem can be categorized as a nonconvex stochastic program with (endogenous) correlated noise. To our knowledge, there is no existing work in the literature that formally studies this problem.

Besides the important role of DC programming, our motivation to study this problem is twofold.

Firstly, endogenous uncertainty occurs naturally in a variety of applications and is more difficult to deal with than exogenous uncertainty. In general, transitioning from exogeneity to endogeneity results in the loss of convexity [START_REF] Dupačová | Optimization under exogenous and endogenous uncertainty[END_REF]. While the majority of stochastic optimization research focuses on exogenous uncertainty, endogenous uncertainty has received little attention. Our work attempts to discuss this topic to a certain extent. Third, while the availability of i.i.d. samples, on which classic approaches like Sample Average Approximation (SAA) and Stochastic Approximation (SA) rely, is a common assumption in stochastic optimization, this is not always the case. Such samples are frequently unfeasible or prohibitively expensive in simulation, for example, in high-dimensional or combinatorial space [START_REF] Duchi | Ergodic mirror descent[END_REF][START_REF] Sun | On markov chain gradient descent[END_REF], and also in the context of Bayesian posterior sampling. Working with a sequence of dependent samples, namely a Markov chain, becomes more viable. Dimension is less of an issue with this approach because some strategies, such as Gibbs sampling, can break down high-dimensional target densities into low-dimensional conditionals. However, because of the autocorrelation between the chain elements, Markov sampling has a higher variance than i.i.d. sampling. In terms of modeling, data obtained in the real world can be highly correlated, therefore modeling them as i.i.d. data is unreasonable. Dependent samples (Markovian or general ergodic data) are utilized for various special classes of stochastic programming under exogenous uncertainty, such as least-square regression [START_REF] Nagaraj | Least squares regression with markovian data: Fundamental limits and algorithms[END_REF] and convex/L-smooth objective [START_REF] Duchi | Ergodic mirror descent[END_REF][START_REF] Sun | On markov chain gradient descent[END_REF][START_REF] Doan | Convergence rates of accelerated markov gradient descent with applications in reinforcement learning[END_REF].

Examples of problem (4.1) The following exogenous stochastic DC program has got a lot of attention recently and has been studied in Chapter 3,

min F (x) = G(x) -H(x) := E P (ξ) (g(x, ξ)) -E P (ξ) (h(x, ξ)) s.t. x ∈ X, (4.2)
where g(•, ξ), h(•, ξ) are convex functions and X is a convex set. Obviously, (4.2) is a special case of (4.1) with E P (ξ) (v(x, ξ)) ∈ ∂H(x), where v(x, ξ) is a subgradient of h(x, ξ).

We next consider the endogenous stochastic program min θ E P (ξ|θ) (f (θ, ξ)), where θ represents the hyperparameters of a family of distributions or a general decision variable. Suppose that P (ξ|θ) is a continuous distribution with a density p(ξ|θ). If -for each ξ -the function f (θ, ξ)p(ξ|θ) admits the DC decomposition f (θ, ξ)p(ξ|θ) = g(θ, ξ)p(ξ|θ) -h(θ, ξ)p(ξ|θ), then the above problem is DC:

E P (ξ|θ) (f (θ, ξ)) = E P (ξ|θ) (g(θ, ξ)) -E P (ξ|θ) (h(θ, ξ)).
Suppose h(θ, ξ), p(ξ|θ) are differentiable. The gradient of the second DC component is given by

∇E P (ξ|θ) (h(θ, ξ)) = ∇ h(θ, ξ)p(ξ|θ)dξ = E P (ξ|θ) (∇ θ h(θ, ξ) + h(θ, ξ)∇ θ log p(ξ|θ)) .
Our contributions To begin, we develop a new stochastic algorithm called Markov chain stochastic DCA (MCSDCA) to handle a class of stochastic (nonsmooth) DC programs with endogenous uncertainty in the absence of i.i.d. samples. Asymptotic and non-asymptotic convergence analyses are established using the theory of the general state-space Markov chain and modern convex analysis. On the non-asymptotic side, the algorithm -in expectation -promises to locate (nearly) ϵ-critical points with reasonable convergence rates if either G or H is L-smooth. On the asymptotic side, we get almost sure convergence to critical points where the L-smooth assumption of both G and H are relaxed, but this requires a greater level of accuracy when solving convex subproblems as well as more Markov chain iterations than in the non-asymptotic case. Second, we broaden our study to include Markov chains that are not always time-homogeneous. This is significant because some diffusion-based Markov

Markov chain Stochastic DCA

In this section, all Markov chains are understood as time-homogeneous. Additional analysis for timeinhomogeneous Markov chains will be presented in Section 4.3.

Markov chain in general state space

Let X be some state space (possibly uncountable) and π be a distribution on X . Let {Z k : k ≥ 0} be a time-homogeneous X -valued Markov chain associated with a Markov transition kernel P (x, dy). We call π an invariant (or stationary) distribution of the Markov chain

{Z k } if for all x, y ∈ X , x∈X π(dx)P (x, dy) = π(dy). (4.3)
The distribution of Z 0 is called the initial distribution of the chain. The relation (4.3) literally reads: if the Markov chain starts from its invariant distribution, the distribution of all later elements will remain invariant. We denote P n (x, A) := P(Z n ∈ A|Z 0 = x) the n-step transition law of the chain. For a fixed x, P n (x, dy) describes the distribution of the n-th state of the Markov chain, Z n , given the initial state Z 0 = x. We denote by E x (resp. E π ) the expectation given the chain starts at Z 0 = x (resp. Z 0 ∼ π). The invariant distribution π of a chain is called an equilibrium distribution if for π-almost all

x, lim n→∞ P n (x, A) = π(A), for all measurable sets A. If π is an invariant distribution of {Z k }, under mild assumptions (irreducible, aperiodic), π is the unique invariant distribution and also an equilibrium distribution of the chain. In fact, a stronger convergence holds under the total variation (TV) distance [START_REF] Tierney | Markov chains for exploring posterior distributions[END_REF]. In other words, for n being large enough, Z n looks similar to a sample drawn from π (but still different, nevertheless). In this research, we want to measure how fast this convergence occurs. We employ the notion of geometric ergodicity introduced as follows (see, e.g., [START_REF] Belomestny | Variance reduction for markov chains with application to mcmc[END_REF]). For a π-integrable function f , we denote π(f ) := f (x)π(dx). For convenience, sometimes we write ⟨π, f ⟩ with the same meaning as π(f ). Let W : X → [1, +∞), the W -norm of a function h : X → R is defined as ∥h∥ W = sup x∈X {|h(x)|/W (x)}. For two measures µ and ν on X with µ(W ), ν(W ) < +∞, we define the W -norm of µ -ν as ∥ν -

µ∥ W = sup ∥f ∥ W ≤1 |µ(f ) -ν(f )|.
Geometric ergodicity Let W : X → [1, +∞) be a measurable function. The Markov chain associated with the Markov transition kernel P admits a unique invariant measure π with π(W ) < ∞ and there exist ς > 0, 0 < ρ < 1 such that for all x ∈ X , n ∈ N,

∥P n (x, dy) -π(dy)∥ W ≤ ςW (x)ρ n .

Problem's setting and assumptions

Firstly, we suppose that the optimal value (called F * ) of (4.1) is finite. Let Ξ be a (general) state space equipped with the underlying σ-algebra M. For each x ∈ R n , P (ξ|x) specifies a probability distribution over Ξ. Consider the problem (4.1), for each x, one subgradient of ∂H(x) is of the form

E P (ξ|x) (v(x, ξ)) for some R n -valued function v = (v 1 , v 2 , . . . , v n ) ⊤
. We assume that one can construct a geometrically ergodic Markov chain (corresponding to a Markov kernel P x (ξ, dξ ′ )) whose equilibrium distribution is P (ξ|x). To be specific, the Markov chain is geometrically ergodic w.r.t. to some W x : Ξ → [1, +∞), ς x > 0, ρ x ∈ (0, 1). We further assume some uniform boundedness as:

sup x ς x < +∞, sup x ρ x < 1, sup x ∥v j (x, •) -⟨P (•|x), v j (x, •)⟩∥ (Wx) 1/2 < +∞ for j = 1, n, sup x ⟨P (•|x), W x ⟩ < +∞.
Finally, we assume that P (ξ|x) and P x (ξ, dξ ′ ) have densities with respect to some common underlying measure on (Ξ, M). It should be stressed that this assumption is very weak: it holds in most cases, where the underlying measure is usually either the counting measure (for countable state space) or the Lebesgue measure (for continuous state space).

Algorithmic design

To solve the program (4.1), we devise a Markov chain Stochastic DCA, whose major characteristics are as follows. Generally, at iteration k, a geometrically ergodic Markov chain ξ 0 → ξ 1 → ξ 2 → . . . whose equilibrium distribution is P (ξ|x k ) is simulated. Then, to address the burn-in period, some early samples are possibly discarded. The remaining samples are used to construct a biased estimator of a subgradient of H. Finally, the algorithm stochastically linearizes the second DC component and solves the resulting convex program up to a certain level of accuracy.

For technical convenience, the starting point of the Markov chain at iteration k, ξ k 0 , is chosen such that there is no randomness of ξ k 0 given x k , i.e., ξ k 0 = φ k (x k ) where φ k is a deterministic function.

Algorithm 7 Markov chain Stochastic DCA

Initialization. A starting point x 0 , a sequence of positive numbers {γ k }, a sequence of Markov chains' length {n k }, the number of burn-in samples b, a symmetric positive definite matrix

A ⪰ I, set k = 0. repeat Run a Markov chain ξ k 0 → ξ k 1 → . . . → ξ k n k -1 whose equilibrium distribution is P (ξ|x k ). Compute y k = 1 n k -b n k -1 i=b v(x k , ξ k i ).
Solve the following convex problem up to a (stochastic) level of accuracy ϵ k to obtain x k+1 ,

min x G(x) -⟨x, y k ⟩ + γ k 2 ∥x -x k ∥ 2 A . (4.4)
k = k + 1. until Stopping criterion Remark 7. 1)
The number b (burn-in) is the number of samples we wait before the Markov chain enters the sampling period where it produces samples looking similar to those drawn from the target distribution.

2) The convex problem (4.4) is only needed to be solved up to the (stochastic) level of accuracy ϵ k , i.e.,

G(x k+1 ) -⟨x k+1 , y k ⟩ + γ k 2 ∥x k+1 -x k ∥ 2 A ≤ inf x G(x) -⟨x, y k ⟩ + γ k 2 ∥x -x k ∥ 2 A + ϵ k .
This feature is desirable in case G is complicated or even stochastic by nature.

Convergence results

Let F k denote the information up to time k, F k = σ({ξ i j } i=0:k-1 j=0:n i , x 0 , x 1 , . . . , x k , ϵ 1 , ϵ 2 , . . . , ϵ k-1 ) which is the σ-algebra generated by the associated random variables. Firstly, the non-asymptotic convergence is presented in Theorem 13.

Theorem 13 (Non-asymptotic convergence). If the sequences of {n k }, {γ k } are chosen in such a way

that n k = b + ⌊k λ ⌋, γ k = γk β for some b ≥ b, λ > 0, β ∈ [0, 1), γ > 0. Let ν > β, suppose that -at iteration k -we solve the convex subproblem (4.4) up to the level of accuracy k -ν , i.e., E(ϵ k |F k ) ≤ k -ν .
Moreover, we assume that the sequences

{EF (x k )} k , E[W x k (φ k (x k ))] n k -b k are bounded.
After T iterations, let κ be sampled from {1, 2, . . . , T } with probability P(κ = k) ∝ k α for some α ≥ max{2β + λ, β + ν}, then 1) If G is L-smooth, x κ is an ϵ-critical point in expectation with the following rate

E dist(∇G(x κ ), ∂H(x κ )) = O 1 T min{ 1-β 2 , λ 2 , ν-β 2 }
.

2) If H is L-smooth, x κ is a nearly ϵ-critical point in expectation with the following rate:

there exists z κ : E∥z κ -x κ ∥ A = O 1 T min{ β+1 2 , 2β+λ 2 , β+ν 2 }
,

and E dist(∂G(z κ ), ∇H(z κ )) = O 1 T min{ 1-β 2 , λ 2 , ν-β 2 }
, where z κ can be defined explicitly as follows

z κ = arg min G(x) -⟨E P (ξ|x κ ) (v(x κ , ξ)), x⟩ + γ κ 2 ∥x -x κ ∥ 2 A .
To prove Theorem 13, we need the following two lemmas.

Lemma 3. Let X be a general state space and {Z k } be a time-homogeneous X -valued Markov chain (corresponding to a Markov transition kernel P (x, dy)) with an equilibrium distribution π. Assume that π, P (x, dy) have densities with respect to some common underlying distribution. Let W :

X → [1, ∞)
such that π(W ) < +∞. Suppose the Markov chain is geometrically egordic w.r.t. W, ρ ∈ (0, 1), ς > 0.

Let h be a function such that π(|h|) < +∞ and denote h = h -π(h). Then

1) |E x [ h(Z 0 ) h(Z s )]| ≤ √ 2ς 1/2 ρ s/2 W (x)∥ h∥ 2 W 1/2 , ∀x ∈ X . 2) |E π [ h(Z 0 ) h(Z s )]| ≤ √ 2ς 1/2 ρ s/2 π(W )∥ h∥ 2 W 1/2 . 3) |E x [ h(Z k ) h(Z k+s )]| ≤ √ 2ς 1/2 ρ s/2 π(W )∥ h∥ 2 W 1/2 + √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ k+s/2 , ∀x ∈ X .
Proof of Lemma 3. The proof is standard (for example, see [START_REF] Belomestny | Variance reduction for markov chains with application to mcmc[END_REF]), but let us include it here (and make it clearer) for the sake of completeness.

1) Since π( h) = 0, we have

|E x [ h(Z 0 ) h(Z s )]| = X h(x) h(y)P s (x, dy) -h(x) X h(y)π(dy) ≤ ∥ h∥ W 1/2 W 1/2 (x) X h(y)P s (x, dy) - X h(y)π(dy) .
Since P s (x, dy) and π have densities that -with a slight abuse of notation -we still call P s (x, •) and π, respectively. Continuing the evaluation,

|E x [ h(Z 0 ) h(Z s )]| ≤ ∥ h∥ W 1/2 W 1/2 (x) X h(y)(P s (x, y) -π(y))dy ≤ ∥ h∥ 2 W 1/2 W 1/2 (x) X W 1/2 (y)(P s (x, y) -π(y))dy ≤ ∥ h∥ 2 W 1/2 W 1/2 (x) X W (y)|P s (x, y) -π(y)|dy 1/2 X |P s (x, y) -π(y)|dy 1/2 ≤ √ 2∥ h∥ 2 W 1/2 W 1/2 (x) X W (y)|P s (x, y) -π(y)|dy 1/2 ≤ √ 2∥ h∥ 2 W 1/2 W 1/2 (x) X u(y)(P s (x, y) -π(y))dy 1/2 ≤ √ 2∥ h∥ 2 W 1/2 W 1/2 (x)∥P s (x, dy) -π(dy)∥ 1/2 ≤ √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 W (x),
where

u(y) =    W (y) if P s (x, y) -π(y) ≥ 0 -W (y) otherwise.
2) In property 1, the initial state Z 0 is fixed to be x, while Z 0 ∼ π in this case. We notice that

|E π [ h(Z 0 ) h(Z s )]| = |E π E( h(Z 0 ) h(Z s )|Z 0 ) | ≤ E π |E Z 0 ( h(Z 0 ) h(Z s ))| ≤ √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 E π (W (Z 0 )) = √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 π(W ).
3) This property evaluates the (auto)correlation between two arbitrary elements of the chain given the initial state. We have

E x [ h(Z k ) h(Z k+s )] -E π [ h(Z k ) h(Z k+s )] = E x E( h(Z k ) h(Z k+s )|Z k ) -E π E( h(Z k ) h(Z k+s )|Z k ) ▷ law of total expectation = E x h(Z k )E( h(Z k+s )|Z k ) -E π h(Z k )E( h(Z k+s )|Z k ) ▷ pulling out what is known = E x h(Z k )E Z k ( h(Z s )) -E π h(Z k )E Z k ( h(Z s ))
▷ due to the homogeneity of the chain

= y h(y)E y ( h(Z s ))P k (x, y)dy - y h(y)E y ( h(Z s ))π(y)dy ▷ Z 0 ∼ π then Z k ∼ π = y E y ( h(y) h(Z s ))(P k (x, y) -π(y))dy ≤ √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 ∥P k (x, dy) -π(dy)∥ W ▷ thanks to property 1 ≤ √ 2ς 3/2 ρ k+s/2 ∥ h∥ 2 W 1/2 W (x) ▷ due to geometric ergodicity.
On another hand, by noticing that

E π ( h(Z k ) h(Z k+s )) = E π ( h(Z 0 ) h(Z s ))
since the chain starts from stationarity and it is homogeneous, the proof is complete thanks to property 2.

Lemma 4. Under the same settings as in Lemma 3, the following inequality holds

E x 1 N -b N -1 i=b h(Z i ) -π(h) 2 ≤ 1 + √ ρ 1 - √ ρ √ 2ς∥ h∥ 2 W 1/2 N -b ςW (x) (1 -ρ)(N -b) + π(W ) .
Proof of Lemma 4. Expanding

E x 1 N -b N -1 i=b h(Z i ) -π(h) 2 = 1 (N -b) 2 N -1 i=b E x (h(Z i ) -π(h)) 2 + 2 (N -b) 2 N -2 i=b N -1 j=i+1 E x (h(Z i ) -π(h))(h(Z j ) -π(h))) = 1 (N -b) 2 N -1 i=b E x h(Z i ) 2 P∼variance + 2 (N -b) 2 N -2 i=b N -1 j=i+1 E x h(Z i ) h(Z j ) Q∼autocovariance .
By using Lemma 3, item 3,

P ≤ N -1 i=b √ 2ς 1/2 π(W )∥ h∥ 2 W 1/2 + √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ i ≤ √ 2ς 1/2 π(W )∥ h∥ 2 W 1/2 (N -b) + √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x) ρ b 1 -ρ , and
Q ≤ N -2 i=b N -1 j=i+1 √ 2ς 1/2 ρ (j-i)/2 π(W )∥ h∥ 2 W 1/2 + √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ i+(j-i)/2 = √ 2ς 1/2 π(W )∥ h∥ 2 W 1/2 N -2 i=b N -1 j=i+1 ρ (j-i)/2 + √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x) N -2 i=b N -1 j=i+1 ρ i+(j-i)/2 . Let A = √ 2ς 1/2 π(W )∥ h∥ 2 W 1/2 , B = √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)
, by standard calculations, it holds

Q ≤ B (1 - √ ρ) 2 (1 + √ ρ) ρ (2b+1)/2 + A √ ρ(N -b) 1 - √ ρ .
Putting together these evaluations, we obtain

E x 1 N -b N -1 i=b h(Z i ) -π(h) 2 ≤ 1 (N -b) 2 A(N -b) + B ρ b 1 -ρ + 2B (1 - √ ρ) 2 (1 + √ ρ) ρ (2b+1)/2 + 2A √ ρ(N -b) 1 - √ ρ = Bρ b (1 - √ ρ) 2 (N -b) 2 + A(1 + √ ρ) (1 - √ ρ)(N -b) ≤ B (1 - √ ρ) 2 (N -b) 2 + A(1 + √ ρ) (1 - √ ρ)(N -b) ≤ 1 + √ ρ 1 - √ ρ √ 2ς∥ h∥ 2 W 1/2 N -b ςW (x) (1 -ρ)(N -b) + π(W ) .
We now prove Theorem 13 as follows.

Proof of Theorem 13. Firstly, we denote

z k = arg min G(x) -⟨E P (ξ|x k ) (v(x k , ξ)), x⟩ + γ k 2 ∥x -x k ∥ 2 A .
By the first-order optimality condition of z k ,

0 ∈ ∂G(z k ) -E P (ξ|x k ) (v(x k , ξ)) + γ k A(z k -x k ),
or equivalently,

E P (ξ|x k ) (v(x k , ξ)) -γ k A(z k -x k ) ∈ ∂G(z k ).
Consequently,

G(x k ) ≥ G(z k ) + ⟨E P (ξ|x k ) (v(x k , ξ)) -γ k A(z k -x k ), x k -z k ⟩ or G(x k ) -⟨E P (ξ|x k ) (v(x k , ξ)), x k ⟩ ≥ G(z k ) -⟨E P (ξ|x k ) (v(x k , ξ)), z k ⟩ + γ k ∥z k -x k ∥ 2 A . (4.5)
By the definition of x k+1 ,

G(z k ) -⟨y k , z k ⟩ + γ k 2 ∥z k -x k ∥ 2 A + ϵ k ≥ G(x k+1 ) -⟨y k , x k+1 ⟩ + γ k 2 ∥x k+1 -x k ∥ 2 A . (4.6)
From (4.5) and (4.6),

G(x k ) -⟨y k , z k ⟩ -⟨E P (ξ|x k ) (v(x k , ξ)), x k ⟩ + γ k 2 ∥z k -x k ∥ 2 A + ϵ k ≥ G(x k+1 ) -⟨E P (ξ|x k ) (v(x k , ξ)), z k ⟩ -⟨y k , x k+1 ⟩ + γ k ∥z k -x k ∥ 2 A + γ k 2 ∥x k+1 -x k ∥ 2 A ,
or equivalently,

G(x k ) -⟨y k , z k -x k+1 ⟩ + ⟨E P (ξ|x k ) (v(x k , ξ)), z k -x k ⟩ + ϵ k ≥ G(x k+1 ) + γ k 2 ∥z k -x k ∥ 2 A + γ k 2 ∥x k+1 -x k ∥ 2 A .
By the convexity of H,

⟨E P (ξ|x k ) (v(x k , ξ)), z k -x k ⟩ + ⟨y k , x k+1 -z k ⟩ = ⟨E P (ξ|x k ) (v(x k , ξ)), x k+1 -x k ⟩ + ⟨E P (ξ|x k ) (v(x k , ξ)), z k -x k+1 ⟩ + ⟨y k , x k+1 -z k ⟩ ≤ H(x k+1 ) -H(x k ) + ⟨x k+1 -z k , y k -E P (ξ|x k ) (v(x k , ξ))⟩ ≤ H(x k+1 ) -H(x k ) + γ k 8 ∥x k+1 -z k ∥ 2 + 2 γ k ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 ≤ H(x k+1 ) -H(x k ) + γ k 8 ∥x k+1 -z k ∥ 2 A + 2 γ k ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 ≤ H(x k+1 ) -H(x k ) + γ k 4 ∥x k+1 -x k ∥ 2 A + γ k 4 ∥z k -x k ∥ 2 A + 2 γ k ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 .
Therefore, we arrive at

γ k 4 ∥z k -x k ∥ 2 A + γ k 4 ∥x k+1 -x k ∥ 2 A ≤ F (x k ) -F (x k+1 ) + 2 γ k ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 + ϵ k . (4.7)
Expanding

∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 = 1 n k -b n k -1 i=b v(x k , ξ k i ) -E P (ξ|x k ) v(x k , ξ) 2 = n j=1 1 n k -b n k -1 i=b v j (x k , ξ k i ) -E P (ξ|x k ) v j (x k , ξ) 2 .
For j ∈ {1, 2, . . . , n}, by applying Lemma 4,

E ξ k 0 1 n k -b n k -1 i=b v j (x k , ξ k i ) -E P (ξ|x k ) v j (x k , ξ) 2 ≤ 1 + √ ρ x k 1 - √ ρ x k √ 2ς x k n k -b ∥ṽ j (x k , •)∥ 2 W 1/2 x k ς x k W x k (φ k (x k )) (1 -ρ x k )(n k -b) + ⟨P (•|x k ), W x k ⟩ .
By the boundedness assumptions, it holds, for some universal B > 0 independent of k,

E 1 n k -b n k -1 i=b v j (x k , ξ k i ) -E P (ξ|x k ) v j (x k , ξ) 2 ≤ B n k -b . (4.8)
Denoting nk = n k -b, from (4.7) and (4.8), for some C > 0,

γ k 4 E∥z k -x k ∥ 2 A ≤ EF (x k ) -EF (x k+1 ) + C γ k nk + E(ϵ k ). (4.9)
Let w k = k α (where α > 0), (4.9) is equivalent to

w k 4 E∥z k -x k ∥ 2 A ≤ w k γ k (EF (x k ) -EF (x k+1 )) + Cw k γ 2 k nk + w k E(ϵ k ) γ k .
Taking sum from k = 1 to T ,

T k=1 w k 4 E∥z k -x k ∥ 2 A ≤ T k=1 w k γ k (EF (x k ) -EF (x k+1 )) + T k=1 Cw k γ 2 k nk + T k=1 w k E(ϵ k ) γ k .
The rest of the proof follows the arguments in [START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF], which -in turn -are based on [START_REF] Chen | Universal stagewise learning for nonconvex problems with convergence on averaged solutions[END_REF]. We have the following evaluations

T k=1 w k γ k (EF (x k ) -EF (x k+1 )) = T k=1 w k γ k EF (x k ) - T k=1 w k γ k EF (x k+1 ) = T k=1 w k-1 γ k-1 EF (x k ) - w k γ k EF (x k+1 ) + T k=1 w k γ k - w k-1 γ k-1 EF (x k ) = w 0 γ 0 EF (x 1 ) - w T γ T EF (x T +1 ) + T k=1 w k γ k - w k-1 γ k-1 EF (x k ) = T k=1 w k γ k - w k-1 γ k-1 (EF (x k ) -EF (x T +1 )) ≤ T k=1 w k γ k - w k-1 γ k-1 ≥0 (EF (x k ) -F * ) ≤ T k=1 w k γ k - w k-1 γ k-1 D = D w T γ T ,
for some D > 0. Therefore,

T k=1 w k 4 E∥z k -x k ∥ 2 A ≤ D w T γ T + C T k=1 w k γ 2 k nk + T k=1 w k E(ϵ k ) γ k . (4.10)
By denoting wk = w k w 1 + w 2 + . . . + w T , (4.10) becomes

T k=1 wk 4 E∥z k -x k ∥ 2 A ≤ D wT γ T + C T k=1 wk γ 2 k nk + T k=1 wk E(ϵ k ) γ k . Case 1: G is L-smooth.
From the first-order optimality,

0 = ∇G(z k ) -E P (ξ|x k ) (v(x k , ξ)) + γ k A(z k -x k ).
Therefore, by noting there exists c > 0 such that A ⪯ cI,

d(∇G(x k ), ∂H(x k )) ≤ ∥∇G(x k ) -E P (ξ|x k ) (v(x k , ξ))∥ ≤ ∥∇G(x k ) -∇G(z k )∥ + ∥∇G(z k ) -E P (ξ|x k ) (v(x k , ξ))∥ ≤ L∥x k -z k ∥ + γ k (z k -x k ) ⊤ A 2 (z k -x k ) ≤ L∥x k -z k ∥ A + γ k √ c (z k -x k ) ⊤ A(z k -x k ) = L∥x k -z k ∥ A + γ k √ c∥x k -z k ∥ A = (L + γ k √ c)∥x k -z k ∥ A , which implies Ed(∇G(x k ), ∂H(x k )) ≤ (L + γ k √ c)E∥x k -z k ∥ A . Since P(κ = k) = wk , Ed(∇G(x κ ), ∂H(x κ )) = T k=1 wk Ed(∇G(x k ), ∂H(x k )) ≤ (L + γ T √ c) T k=1 wk E∥x k -z k ∥ A ≤ (L + γ T √ c) T k=1 wk E∥x k -z k ∥ 2 A 1/2 ≤ (L + γ T √ c) T k=1 wk 1/2 T k=1 wk E∥x k -z k ∥ 2 A 1/2 ≤ (L + γ T √ c) 4D wT γ T + 4C T k=1 wk γ 2 k nk + 4 T k=1 wk E(ϵ k ) γ k 1/2 . ( 4.11)
The next evaluations will employ the standard calculus:

S s=1 s α ≥ S 0 x α dx = 1 α + 1 S α+1 .

Now by plugging γ

k = γ • k β , w k = k α , nk = ( b -b) + ⌊k λ ⌋ ≥ ⌊k λ ⌋ to (4.11), where γ > 0, 0 ≤ β < 1, α ≥ max{2β + λ, β + ν}, λ > 0, and noticing that E(ϵ k |F k ) ≤ k -ν and ν > β, it holds Ed(∇G(x κ ), ∂H(x κ )) ≤ (L + γ T √ c) 4D wT γ T + 4C T k=1 wk γ 2 k nk + 4 T k=1 wk E(ϵ k ) γ k 1/2 ≤ 2(L + γ • T β √ c) D T α γ • T β T k=1 k α + C T k=1 k α T k=1 k α γ 2 k 2β ⌊k λ ⌋ + 1 T k=1 k α T k=1 k α E(ϵ k ) γk β 1/2 ≤ 2(L + γ • T β √ c) D(α + 1) γ • T β+1 + C(α + 1) γ 2 T α+1 T k=1 k α-2β c λ k λ + α + 1 γT α+1 T k=1 k α-β-ν 1/2 , c λ > 0 ≤ 2(L + γ • T β √ c) D(α + 1) γ • T β+1 + C(α + 1) γ 2 c λ T α+1 T α-2β-λ+1 + α + 1 γT α+1 T α-β-ν+1 1/2 = 2(L + γ • T β √ c) D(α + 1) γ • T β+1 + C(α + 1) γ 2 c λ T 2β+λ + α + 1 γT β+ν 1/2 .
Therefore, x κ is an ϵ-critical point in expectation with the following rate

Ed(∇G(x κ ), ∂H(x κ )) = O 1 T min{ 1-β 2 , λ 2 , ν-β 2 } . Case 2: H is L-smooth. d(∂G(z k ), ∇H(z k )) ≤ d(∂G(z k ), ∇H(x k )) + ∥∇H(x k ) -∇H(z k )∥ ≤ γ k (z k -x k ) ⊤ A 2 (z k -x k ) + L∥x k -z k ∥ ≤ γ k √ c∥z k -x k ∥ A + L∥x k -z k ∥ A ≤ (L + γ k √ c)∥z k -x k ∥ A .
Similarly, x κ is a nearly ϵ-critical point in expectation with the following rate

Ed(∂G(z κ ), ∇H(z κ )) = O 1 T min{ 1-β 2 , λ 2 , ν-β 2 }
where

E∥z κ -x κ ∥ A = O 1 T min{ β+1 2 , 2β+λ 2 , β+ν 2 }
.

We next obtain the almost sure convergence results stated in Theorem 14.

Theorem 14 (Asymptotic convergence). Let γ > 0 and set

γ k = γ, for all k ∈ N. If the sequence {n k } is chosen such that ∞ k=1 1 n k -b
< +∞, and it is further supposed that ∃R > 0 : ∀k,

W x k (φ k (x k )) n k -b ≤ R a.s.,
and the sequence of random errors in solving convex subproblems satisfies ∞ k=1 E(ϵ k ) < +∞, then 1) The sequence of the objective values {F (x k )} is convergent almost surely.

2) ∞ k=1 ∥x k+1 -x k ∥ 2 A < +∞ almost surely. 3) Assume that {x k } and {y k } are bounded almost surely, then, almost surely, every limit point of {x k } is a critical point of F . Proof of Theorem 14. 1) It follows from (4.7) that

γ 4 ∥z k -x k ∥ 2 A + γ 4 E(∥x k+1 -x k ∥ 2 A |F k ) ≤ F (x k ) -E(F (x k+1 )|F k ) + 2 γ E ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 |F k + E(ϵ k |F k ) = F (x k ) -E(F (x k+1 )|F k ) + 2 γ E ξ k 0 ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 + E(ϵ k |F k ).
As a result, for some C > 0,

E(F (x k+1 )|F k ) + γ 4 E(∥x k+1 -x k ∥ 2 A |F k ) ≤ F (x k ) + C γ(n k -b) + E(ϵ k |F k ). (4.12) Noting that the condition ∞ k=1 E(ϵ k ) < +∞ implies ∞ k=1 E(ϵ k |F k ) < +∞ a.s. By supermartin- gale theorem [110, Theorem 1], {F (x k )} is convergent almost surely.
2) From (4.12),

γ 4 E(∥x k+1 -x k ∥ 2 A ) ≤ E(F (x k )) -E(F (x k+1 )) + C γ(n k -b) + E(ϵ k ).
It follows that

E ∞ k=1 ∥x k+1 -x k ∥ 2 A < +∞, implying ∞ k=1 ∥x k+1 -x k ∥ 2 A < +∞ a.s.
3) It has been established from the proof of Theorem 1 that

E∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 ≤ E n k -b
, for some E > 0.

As a consequence,

∞ k=1 E∥y k -E P (ξ|x k ) (v(x k , ξ))∥ 2 < +∞,
which yields

∥y k -E P (ξ|x k ) (v(x k , ξ))∥ → 0 a.s.
ergodic to π with respect to W : X → [1, +∞) (where π(W ) < +∞) and ς > 0, 0 < ρ < 1 if ∥P k+s k (x, dy) -π(dy)∥ W ≤ ςW (x)ρ s , ∀x ∈ X , and ∀k, s ∈ N.

Apart from the Markov chains being now time-inhomogeneous and geometrically ergodic in the above sense, other settings in subsection 4.2.2 remain unchanged. Furthermore, the convergence results stated in Theorems 13 and 14 are still valid in this new setting.

Theorem 15. With the above setting, Theorems 13 and 14 hold true.

Proof. In essence, it is sufficient to verify property 3) of Lemma 3. Everything else remains the same.

We prove:

E x h(Z k ) h(Z k+s ) ≤ √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ k+s/2 + √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 π(W ).
It is observed that We evaluate A as follows

E x h(Z k ) h(Z k+s ) = E h(Z k ) h(Z k+s )|Z 0 = x = E E h(Z k ) h(Z k+s )|Z k , Z 0 = x |Z 0 = x = E E h(Z k ) h(Z k+s )|Z k |Z 0 = x = E h(Z k )E h(Z k+s )|Z k |Z 0 = x = E h(Z k ) h(z)P k+s k (Z k , dz)|Z 0 = x = h(
A ≤ h(y) h(z) P k+s k (y, z) -π(z) P k (x, y) -π(y) dzdy ≤ ∥ h∥ 2 W 1/2 W 1/2 (y)W 1/2 (z) P k+s k (y, z) -π(z) • P k (x, y) -π(y) dzdy ≤ ∥ h∥ 2 W 1/2 W 1/2 (y) P k (x, y) -π(y) W 1/2 (z) P k+s k (y, z) -π(z) dz ∆ dy.
It follows from Hölder's inequality that

∆ ≤ W (z) P k+s k (y, z) -π(z) dz 1/2 P k+s k (y, z) -π(z) dz 1/2 ≤ √ 2∥P k+s k (y, dz) -π(dz)∥ 1/2 W ≤ √ 2ς 1/2 ρ s/2 W (y) 1/2 .
Therefore,

A ≤ √ 2ς 1/2 ∥ h∥ 2 W 1/2 ρ s/2 W (y) P k (x, y) -π(y) dy ≤ √ 2ς 1/2 ∥ h∥ 2 W 1/2 ρ s/2 P k (x, dy) -π(dy) W ≤ √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ k+s/2 . ( 4.15)
From (4.13), (4.14), (4.15), It holds

E x h(Z k ) h(Z k+s ) ≤ √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ k+s/2 + h (
⊠ ≤ | h(z)| P k+s k (y, z) -π(z) dz ≤ ∥ h∥ W 1/2 W 1/2 (z) P k+s k (y, z) -π(z) dz ≤ √ 2∥ h∥ W 1/2 W (z) P k+s k (y, z) -π(z) dz 1/2 ≤ √ 2∥ h∥ W 1/2 P k+s k (y, dz) -π(dz) 1/2 W ≤ √ 2ς 1/2 ∥ h∥ W 1/2 W (y) 1/2 ρ s/2 .
Therefore,

3 ≤ | h(y)|π(y) √ 2ς 1/2 ∥ h∥ W 1/2 W (y) 1/2 ρ s/2 dy ≤ √ 2ς 1/2 ρ s/2 ∥ h∥ W 1/2 | h(y)|π(y)W (y) 1/2 dy ≤ √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 W (y)π(y)dy = √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 π(W ). (4.17)
From (4.16) and (4.17), we conclude that where f is a large sum of compositions between a neural network and a loss function, and x represents the network's trainable parameters. Rather than immediately minimizing f , which can be quite rugged and only represents the training data, it is preferable to minimize its smoothed form f , which seeks to capture the primary trend of f and is expected to generalize well on unseen data [START_REF] Chaudhari | Deep relaxation: partial differential equations for optimizing deep neural networks[END_REF][START_REF] Chaudhari | Entropy-sgd: Biasing gradient descent into wide valleys[END_REF]. The following Hamilton-Jacobi equation, presented by [START_REF] Chaudhari | Deep relaxation: partial differential equations for optimizing deep neural networks[END_REF], can be used to obtain such a smooth function

E x h(Z k ) h(Z k+s ) ≤ √ 2ς 3/2 ∥ h∥ 2 W 1/2 W (x)ρ k+s/2 + √ 2ς 1/2 ρ s/2 ∥ h∥ 2 W 1/2 π(W ).
u t + 1 2 |∇u| 2 = 0 in R n × (0, +∞),
and its "viscous" version (for ϵ > 0),

u t + 1 2 |∇u| 2 = ϵ 2 ∆u in R n × (0, +∞),
where the initial condition of both PDEs is set as u(x, 0) = f (x). The variables x and t represent space and time, respectively. The fundamental idea is to let the systems evolve up to time t, then minimize u(x, t) instead of minimizing f , which is the initial condition (at time 0) of the above PDEs.

The former PDE gives rise to the viscous solution, which is the inf-convolution of f , bringing down local maxima while enlarging local minima, whereas the latter implies local entropy which favors local minima in wide valleys while suppressing those in sharp thin valleys. The local entropy, which is the viscous approximation of the inf-convolution, is the focus of this research. The local entropy plays a particularly important role in deep learning. In the context of training deep neural networks, Chaudhari et al. [START_REF] Chaudhari | Entropy-sgd: Biasing gradient descent into wide valleys[END_REF] argued that local minima lying in large flat region generalize better than those located in sharp valleys (although this argument is still controversial [START_REF] Dinh | Sharp minima can generalize for deep nets[END_REF]). They also empirically verified that standard optimizers such as Adam or SGD applied to the original loss surface usually end up at solutions lying in flat regions (by checking the spectrum of the Hessian), explaining why these optimizers usually generalize well. Therefore, they suggested the construction of the local entropy that proactively modifies the original loss surface to favor flat-region local minima. The local entropy is a DC function that comes within the scope of this study. In fact, the DC structure can be found in a broader class of regularization structures based on a more general Hamilton-Jacobi equation

u t + 1 2 ⟨∇u, H -1 ∇u⟩ = 0 in R n × (0, +∞),
with the initial condition u(x, 0) = f (x), where H is a symmetric positive definite matrix. According to Hopf-Lax formula [START_REF] Evans | Partial differential equations[END_REF], the above PDE has the following viscous solution:

u(x, t) = inf x ′ f (x ′ ) + 1 2t ⟨x ′ -x, H(x ′ -x)⟩ . (4.18)
We next present a fact that explains why the local entropy is a viscous approximation to the infconvolution (ϵ → 0 + , the local entropy tends to the inf-convolution) and allows us to construct an approximation for the more general structure (4.18).

Fact [START_REF] Gomes | Entropy penalization methods for hamilton-jacobi equations[END_REF]: if

F ∈ C(R n ) is superlinear, then lim ϵ→0 + R n e -F (v) ϵ dv ϵ = e -inf F . So, let F(x ′ ) = f (x ′ ) + 1 2t ⟨x ′ -x, H(x ′ -x)⟩ which is superlinear if f is bounded below, the above fact implies ũ(x, t) := -ϵ log R n exp 1 ϵ -f (x ′ ) -1 2t ⟨x ′ -x, H(x ′ -x)⟩ dx ′ ϵ→0 + ---→ u(x, t).
When we replace H = I for the above limit, we get the fact that the local entropy tends to the inf-convolution as ϵ → 0 + , which is usually referred to as viscosity vanishing in the PDE literature.

We point out a DC structure of ũ(x, t) as follows,

ũ(x, t) = 1 2t x ⊤ Hx G(x) -ϵ log R n exp - 1 ϵ f (x ′ ) + 1 2t x ′⊤ Hx ′ - 1 t x ⊤ Hx ′ dx ′ H(x) . (4.19)
It is obvious that G is convex, while the convexity of H is verified as follows.

Verification of the convexity of H. Denote v(x, x ′ ) = exp -1 ϵ f (x ′ ) + 1 2t x ′⊤ Hx ′ -1 t x ⊤ Hx ′ . We compute the partial derivative of H,

∂H ∂x i = ϵ ∂ ∂x i v(x, x ′ )dx ′ v(x, x ′ )dx ′ = ϵ v(x, x ′ )( 1 ϵt [Hx ′ ] i )dx ′ v(x, x ′ )dx ′ = 1 t v(x, x ′ )[Hx ′ ] i dx ′ v(x, x ′ )dx ′ = 1 t E p(x ′ |x) [Hx ′ ] i ,
where p(x ′ |x) ∝ v(x, x ′ ). Therefore, ∇H(x) = 1 t E p(x ′ |x) (Hx ′ ). For i, j ∈ {1, 2, . . . , n}, the second order derivative is given by

∂ 2 H ∂x j ∂x i = 1 t ∂ ∂x j v(x, x ′ )[Hx ′ ] i dx ′ v(x, x ′ )dx ′ -v(x, x ′ )[Hx ′ ] i dx ′ ∂ ∂x j v(x, x ′ )dx ′ v(x, x ′ )dx ′ 2 = 1 t v(x, x ′ ) 1 ϵt [Hx ′ ] j [Hx ′ ] i dx ′ v(x, x ′ )dx ′ -v(x, x ′ )[Hx ′ ] i dx ′ v(x, x ′ ) 1 ϵt [Hx ′ ] j dx ′ v(x, x ′ )dx ′ 2 = 1 ϵt 2 v(x, x ′ )[Hx ′ ] j [Hx ′ ] i dx ′ v(x, x ′ )dx ′ -v(x, x ′ )[Hx ′ ] i dx ′ v(x, x ′ )[Hx ′ ] j dx ′ v(x, x ′ )dx ′ 2 = 1 ϵt 2 E p(x ′ |x) [Hx ′ ] i [Hx ′ ] j -E p(x ′ |x) [Hx ′ ] i E p(x ′ |x) [Hx ′ ] j .
We prove the Hessian matrix H = ∂ 2 H ∂x j ∂x i i,j is positive semidefinite. It amounts to showing z ⊤ Hz ≥ 0, for all z ∈ R n , or equivalently, i,j

z i z j E p(x ′ |x) [Hx ′ ] i [Hx ′ ] j ≥ i,j z i z j E p(x ′ |x) [Hx ′ ] i E p(x ′ |x) [Hx ′ ] j .
The above inequality is subsequently rewritten as follows

E p(x ′ |x)   i,j z i z j [Hx ′ ] i [Hx ′ ] j   ≥ n i=1 z i E p(x ′ |x) [Hx ′ ] i 2 ,
and then

E p(x ′ |x) n i=1 z i [Hx ′ ] i 2 ≥ E p(x ′ |x) n i=1 z i [Hx ′ ] i 2 ,
which is true thanks to Hölder's inequality.

Langevin dynamics

The gradient of the H is given by ∇H(x) = 1 t E p(x ′ |x) (Hx ′ ), as shown previously. As a result, the MCSDCA for minimizing ũ(x, t) requires -at each iteration -a simulation of a Markov chain with the

target distribution p(x ′ |x) ∝ exp -1 ϵ f (x ′ ) + 1 2t x ′⊤ Hx ′ -1 t x ⊤ Hx ′ .
To this goal, we use Langevin dynamics, which was originally developed to model the dynamics of molecular systems. Let us consider the general target distribution π(x) ∝ exp(-βU (x)) where β is referred to as the inverse temperature.

The overdamped Langevin diffusion is of the form dX t = -∇U (X t )dt + 2β -1 dB t where B t is standard Brownian motion [START_REF] Mörters | Brownian motion[END_REF]. The overdamped Langevin diffusion, under suitable conditions, admits π(x) as its unique invariant measure and the convergence to π(x) happens at the geometric rate [START_REF] Roberts | Exponential convergence of langevin distributions and their discrete approximations[END_REF][START_REF] Durmus | Nonasymptotic convergence analysis for the unadjusted langevin algorithm[END_REF]. To be implementable, this diffusion must be discretized. If the Euler-Maruyama discretization is employed, we obtain the following Markov chain named Unadjusted Langevin Algorithm

X k+1 = X k -η∇U (X k ) + 2ηβ -1 W k , (4.20)
where W k is the standard Gaussian noise. Some discretization error occurs but it is not quantified here.

Furthermore, for computational feasibility, the full gradient of ∇U required in (4.20) is replaced by the mini-batch gradient, resulting in the stochastic gradient Langevin dynamics [START_REF] Welling | Bayesian learning via stochastic gradient langevin dynamics[END_REF].

On the other hand, the underdamped Langevin diffusion takes the following form

dV t = -µV t dt -∇U (X t )dt + 2µβ -1 dB t (4.21) dX t = V t dt
where µ is the friction coefficient. Basically, this diffusion lifts the original space to a higher dimensional space (2x higher). Under mild conditions, the diffusion process (X t , V t ) admits

π(x, v) ∝ exp -β 1 2 ∥v∥ 2 + U (x)
as the invariant measure and the convergence to π(x, v) is exponentially fast [START_REF] Cao | On explicit l2-convergence rate estimate for underdamped langevin dynamics[END_REF]. The marginal distribution of the first component X is then π(x) ∝ exp(-βU (x)), which is exactly the target distribution we want to sample from. Furthermore, it is noteworthy that

E π(x) (F (x)) = x F (x)π(x)dx = x F (x) v π(x, v)dvdx = x v F (x)π(x, v)dvdx = E π(x,v) (F (x)).
Therefore, instead of constructing a Markov chain with target distribution π(x), one can consider a Markov chain with the target π(x, v), namely,

(X 0 , V 0 ) → (X 1 , V 1 ) → . . . → (X k-1 , V k-1 ), then E π(x) (F (x)) = E π(x,v) (F (x)) ≈ 1 k k-1 i=0 F (X i ).
Now the question is what is a good discretization scheme for (4.21). Although the simple Euler-Maruyama discretization scheme can again be applied to this diffusion (resulting in the so-named inertial Langevin dynamics), it also generates large error [START_REF] Hu | Optimal underdamped langevin mcmc method[END_REF]. In this work, we make use of the state-of-the-art discretization scheme introduced in [START_REF] Cheng | Underdamped langevin mcmc: A non-asymptotic analysis[END_REF]. Basically, let's say with the specific friction µ = 2, given the current state (X i , V i ), the next state is sampled from a multivariate normal distribution with mean and covariance as follows (here the expectation is understood as the conditional expectation given (X i , V i ))

E(V i+1 ) = V i e -2δ - 1 2 (1 -e -2δ )∇U (X i ), E(X i+1 ) = X i + 1 2 (1 -e -2δ )V i - 1 2 δ - 1 2 (1 -e -2δ ) ∇U (X i ), E (X i+1 -E(X i+1 ))(X i+1 -E(X i+1 )) ⊤ = 1 β δ - 1 4 e -4δ - 3 4 + e -2δ • I d×d , E (V i+1 -E(V i+1 ))(V i+1 -E(V i+1 )) ⊤ = 1 β (1 -e -4δ ) • I d×d , E (X i+1 -E(X i+1 ))(V i+1 -E(V i+1 )) ⊤ = 1 2β [1 + e -4δ -2e -2δ ] • I d×d , (4.22)
where δ ∈ (0, 1) is the step size. Again, in the context of big data, the full gradient ∇U (X i ) is replaced by the stochastic gradient ∇U (X i ) computed over a mini-batch, resulting in a stochastic version of the above scheme (see [START_REF] Cheng | Underdamped langevin mcmc: A non-asymptotic analysis[END_REF][Subsection 2.2.1.]).

MCSDCA-odLD and MCSDCA-udLD

In this subsection, by putting necessary components together, we derive two specific MCSDCA schemes for training deep neural networks. In this practical setting, we simply choose the matrix A = I in (4.4), and the matrix H = I in (4.18).

The first scheme, MCSDCA-odLD, is obtained straightforwardly by integrating the overdamped Langevin dynamics (4.20) into the inner loop of Algorithm 7. Note that with (we drop the outer iteration index for simplicity)

p(x ′ |x) ∝ exp - 1 ϵ f (x ′ ) + 1 2t x ′⊤ x ′ - 1 t x ⊤ x ′ ,
the stochastic version of the Langevin dynamics (4.20) reads

x ′ i+1 ∼ x ′ i -η ∇f (x ′ i ) + 1 t (x ′ i -x) + 2ηϵN (0, I),
where N (0, I) is the standard Gaussian distribution. Furthermore, the starting state of the Markov chain {x ′ i } is simply set to be x (the current point of the outer loop). The detailed MCSDCA-odLD is presented in Algorithm 8. It is worth noting that, with the mentioned specific settings, MCSDCA-odLD can be viewed as a sibling of the Entropy-SGD developed in [START_REF] Chaudhari | Entropy-sgd: Biasing gradient descent into wide valleys[END_REF][START_REF] Chaudhari | Deep relaxation: partial differential equations for optimizing deep neural networks[END_REF]. Only two distinctions exist:

(1) the MCSDCA-odLD takes a traditional approach to the Markov chain burn-in period (discard early samples) while the Entropy-SGD employs exponential averaging, (2) the step size of the MCSDCA-odLD (in general, MCSDCA-odLD does not have a step size, but when applied to the local entropy with the given DC decomposition, it appears to have a step size) is specified by the DC decomposition (and partially by users via {γ k }) and is always smaller than t, while the step size of the Entropy-SGD is specified arbitrarily by users.

Algorithm 8 MCSDCA overdamped Langevin dynamics

Initialization. A starting point x 0 , a sequence of positive numbers {γ k }, a sequence of Markov chains' length {n k }, the number of burn-in samples b, ϵ > 0, η > 0, t > 0, set k = 0, repeat Set the starting state of a Markov chain x k 0 := x k . for i = 0 to n k -2 do 1. Receive a minibatch of data D minibatch .

2. Compute a stochastic gradient ∇f (x k i ) of f using D minibatch . 3. Sample the next state of the Markov chain as

x k i+1 ∼ x k i -η ∇f (x k i ) + 1 t (x k i -x k ) + 2ηϵN (0, I)
where N (0, I) is the standard Gaussian distribution.

end for Compute y k = 1 n k -b n k -1 i=b x k i .
Solve the following convex problem,

x k+1 = arg min x 1 2t ∥x∥ 2 - 1 t ⟨x, y k ⟩ + γ k 2 ∥x -x k ∥ 2 , (4.23)
which has the closed-form solution

x k+1 = tγ k 1 + tγ k x k + 1 1 + tγ k y k . Set k = k + 1. until Stopping criterion
The second scheme, named MCSDCA-udLD, is obtained by using the underdamped Langevin dynamics (4.22) to approximately sample from p(x ′ |x). It then reads,

E(v ′ i+1 ) = v ′ i e -2δ - 1 2 (1 -e -2δ ) ∇f (x ′ i ) + 1 t (x ′ i -x) E(x ′ i+1 ) = x ′ i + 1 2 (1 -e -2δ )v ′ i - 1 2 δ - 1 2 (1 -e -2δ ) ∇f (x ′ i ) + 1 t (x ′ i -x) E (x ′ i+1 -E(x ′ i+1 ))(x ′ i+1 -E(x ′ i+1 )) ⊤ = ϵ δ - 1 4 e -4δ - 3 4 + e -2δ • I d×d := c 1 • I d×d E (v ′ i+1 -E(v ′ i+1 ))(v ′ i+1 -E(v ′ i+1 )) ⊤ = ϵ(1 -e -4δ ) • I d×d := c 2 • I d×d E (x ′ i+1 -E(x ′ i+1 ))(v ′ i+1 -E(v ′ i+1 )) ⊤ = ϵ 2 [1 + e -4δ -2e -2δ ] • I d×d := c 3 • I d×d .
The question is how to efficiently sample (x ′ i+1 , v ′ i+1 ) with the above mean and covariance matrix. In the context of deep learning, x ′ i+1 represents all trainable parameters of a neural network, and so does v ′ i+1 . Therefore, the size of x ′ i+1 and v ′ i+1 is enormous. Furthermore, in deep learning frameworks, the structure of trainable parameters is not given as a vector. Rather, it is a list containing of many matrices and vectors, where each matrix (resp. vector) represents a weight (resp. bias) of a particular layer. Clearly, it is not a wise choice to flatten these matrices and concatenate them together to form a super long vector, then construct a super large covariance matrix to do sampling. It is easier to sample layer by layer. We have several observations and remarks:

• The elements (the features) of x ′ i+1 are independent from each other, and so are the elements of v ′ i+1 . There is only correlation between each element of x ′ i+1 and v ′ i+1 at the same location, e.g., the element at row 2, column 3 of the weight matrix of the 4-th layer of x ′ i+1 is correlated with the element at row 2, column 3 of the weight matrix of the 4-th layer of v ′ i+1 .

•

μ = µ 1 + Σ 12 Σ -1 22 (a -µ 2 ), Σ = Σ 11 -Σ 12 Σ -1 22 Σ 21 .
Therefore, instead of sampling (x ′ i+1 , v ′ i+1 ) simultaneously, we will sample it sequentially and this can be done in a layer-by-layer manner. Firstly, we sample v ′ i+1 from the normal multivariate with mean

E(v ′ i+1 ) = v ′ i e -2δ - 1 2 (1 -e -2δ ) ∇f (x ′ i ) + 1 t (x ′ i -x) and covariance matrix c 2 • I, or equivalently, v ′ i+1 ∼ v ′ i e -2δ - 1 2 (1 -e -2δ ) ∇f (x ′ i ) + 1 t (x ′ i -x) + √ c 2 N (0, I).
Next, we sample x ′ i+1 |v ′ i+1 . Thanks to the second remark above,

x ′ i+1 |v ′ i+1 ∼ N (μ, σ) with μ = E(x ′ i+1 ) + c 3 c -1 2 (v ′ i+1 -E(v ′ i+1 )), Σ = (c 1 -c 2 3 c -1 2 )I. Consequently, x ′ i+1 |v ′ i+1 ∼ μ + (c 1 -c 2 3 c - 1
2 )N (0, I). Finally, as discussed earlier, for computational feasibility, the gradient ∇f (x ′ i ) in the previous computations is replaced by the stochastic gradient ∇f (x ′ i ) computed over a mini-batch. The detailed MCSDCA-udLD is presented in Algorithm 9.

Algorithm 9 MCSDCA underdamped Langevin dynamics

Initialization. A starting point x 0 , a sequence of positive numbers {γ k }, a sequence of Markov chains' length {n k }, the number of burn-in samples b, ϵ > 0, δ > 0, t > 0, set k = 0. repeat Set the starting state of the Markov chain x k 0 := x k , and v k 0 := 0.

for i = 0 to n k -2 do 1.
Receive a minibatch of data D minibatch .

2. Compute a stochastic gradient ∇f (x k i ) of f using D minibatch . 3. Compute the conditional expectation of v k i+1 and x k i+1 given x k i and v k i as follows

E(v k i+1 ) = e -2δ v k i - 1 2 (1 -e -2δ ) ∇f (x k i ) + 1 t (x k i -x k ) , E(x k i+1 ) = x k i + 1 2 (1 -e -2δ )v k i - 1 2 δ - 1 2 (1 -e -2δ ) ∇f (x k i ) + 1 t (x k i -x k ) . 4. Sample v k i+1 ∼ E(v k i+1 ) + √ c 2 N (0, I). 5. Sample x k i+1 |v k i+1 ∼ E(x k i+1 ) + c 3 c -1 2 (v k i+1 -E(v k i+1 )) + (c 1 -c 2 3 c -1 2 )N (0, I). end for Compute y k = 1 n k -b n k -1 i=b x k i .
Solve the following convex problem,

x k+1 = arg min x 1 2t ∥x∥ 2 - 1 t ⟨x, y k ⟩ + γ k 2 ∥x -x k ∥ 2 , (4.24)
which has the closed-form solution

x k+1 = tγ k 1 + tγ k x k + 1 1 + tγ k y k . k = k + 1. until Stopping criterion Technical note. In implementation, the step of computing y k = 1 n k -b n k -1 i=b x k
i in Algorithms 8 and 9 is integrated into the inner loop, say, y k is computed progressively.

Conclusion

In this chapter, we present a Markov chain stochastic DCA for solving (nonsmooth) endogenous stochastic DC programs using only Markovian data. There are strong guarantees in locating DC critical points of the proposed algorithm when both asymptotic and non-asymptotic convergence results are proved.

As an important extension, we give additional analysis to time-inhomogeneous Markov chains, proving that the previously proved convergence results still hold in this scenario. This study should open the way for a more complete approach with firm convergence analysis to a larger class of nonconvex stochastic algorithms with diverse intrinsic noise properties. We next use PDEs regularization to apply the proposed technique to deep learning optimization problems, yielding two MCSDCA realizations, MCSDCA-olLD and MCSDCA-ulLD, which are based on the overdamped and underdamped Langevin dynamics, respectively. In the next chapter, these two MCSDCA realizations will be used to tackle time series forecasting problems utilizing some different deep neural network structures. safety hazards, thereby diminishing quickly the advantages of reactive maintenance. The next stepping stone in the timeline of maintenance is preventive maintenance, where the main philosophy is to maintain machines before malfunctions occur. It follows a set schedule based on expected statistical information of equipment (provided by manufacturers) to carry out maintenance even if the equipment is operating normally. It helps us to gain proactivity in prioritizing the maintenance schedules. The last milestone is predictive maintenance, which refers to a class of techniques that actively predict the status of operating equipment based on actual (and time-based) data in order to determine when maintenance should be performed. This type of maintenance is probably the most advancing and sophisticated, which is backed by the cutting-edge technologies such as machine learning, deep learning, high-sensitivity sensor, and so on.

Predictive maintenance, once properly implemented, can have decisive advantages over reactive and preventive approaches. On the one hand, reactive maintenance, while simple to deploy, is costinefficient and unsuitable for industries where safety is of utmost priority such as transportation, health, and aerospace. Predictive maintenance addresses these limitations by actively maintaining equipment based on early alert detection, preventing catastrophic failures, reducing unplanned downtime, improving safety and liability, and prolonging equipment lifespan. On the other hand, while preventive maintenance is also capable of preventing disastrous crashes and enhancing safety, it is not cost-effective as equipment possibly still operates regularly for quite some time after the scheduled repair. Predictive maintenance, on the contrary, can potentially save a lot of unnecessary costs by making "just-in-time" decisions based on current equipment data, which is more reliable and informative than average population statistics used in preventive maintenance.

In predictive maintenance, two top-priority problems are remaining useful life (RUL) prediction and state-of-health (SoH) estimation. The RUL of a machine is the length of time in which the machine is expected to continue functioning as intended. The RUL, once accurately predicted, can aid engineers with crucial information to schedule maintenance. Meanwhile, the SoH indicates the current health condition of a battery (or a cell) in comparison with the initial health status. The SoH estimation provides important information in battery-powered systems, enabling smart replacement decisions possible.

Predictive maintenance methodologies, for RUL prediction and SoH estimation, can be categorized into model-based and data-driven methods. The model-based approach [START_REF] Virkar | A model for degradation of electrochemical devices based on linear nonequilibrium thermodynamics and its application to lithium ion batteries[END_REF][START_REF] Jouin | Particle filter-based prognostics: Review, discussion and perspectives[END_REF][START_REF] Jouin | Degradations analysis and aging modeling for health assessment and prognostics of pemfc[END_REF] requires extensive domain knowledge and the understanding of the underlying physical system to accurately model the degradation. For example, partial differential equations have been used to describe the electrochemical reactions that happen inside the lithium-ion batteries [START_REF] Zheng | Co-estimation of state-of-charge, capacity and resistance for lithium-ion batteries based on a high-fidelity electrochemical model[END_REF]. Once the system degradation is modeled precisely, this approach can be very accurate and reliable [START_REF] Qian | A multi-time scale approach to remaining useful life prediction in rolling bearing[END_REF]. However, this modeling procedure is quite demanding, and in many circumstances, such knowledge is unavailable. Furthermore, because the model is not always linear, precise analysis of the solutions can be challenging at times [START_REF] Gao | Prediction of remaining useful life of lithium-ion battery based on multikernel support vector machine with particle swarm optimization[END_REF]. Meanwhile, the data-driven approach [START_REF] Benkedjouh | Remaining useful life estimation based on nonlinear feature reduction and support vector regression[END_REF][START_REF] Dong | A segmental hidden semi-markov model (hsmm)-based diagnostics and prognostics framework and methodology[END_REF][START_REF] Dong | A segmental hidden semi-markov model (hsmm)-based diagnostics and prognostics framework and methodology[END_REF][START_REF] Wu | Remaining useful life estimation of engineered systems using vanilla lstm neural networks[END_REF][START_REF] Sateesh Babu | Deep convolutional neural network based regression approach for estimation of remaining useful life[END_REF][START_REF] Zhang | Long short-term memory for machine remaining life prediction[END_REF][START_REF] Park | Lstm-based battery remaining useful life prediction with multi-channel charging profiles[END_REF] treats the underlying mechanism as a blackbox and learns to predict the relevant values directly from time-based data. In essence, with this approach, the two mentioned problems (RUL, SoH) are characterized as time series forecasting, Time series data → Machine learning/Deep learning model → RUL/SoH where the intermediate machine learning/deep learning model attempts to capture the relationship be-tween up-to-date time series data and the corresponding RUL/SoH. This approach has lately acquired popularity due to its ease of implementation and the allowance of the lack of prior domain expertise. Because of the rapid advancement of machine learning and artificial intelligence, the data-driven approach is now prevalent in practically all domains of applied sciences, particularly predictive maintenance. A considerable amount of high-quality data is usually required for the training process in order to produce a successful model. This demand may be addressed thanks to high-precision sensors and cutting-edge internet-of-things technologies.

We further classify the data-driven approach into two categories: classical machine learning such as support vector machine [START_REF] Benkedjouh | Remaining useful life estimation based on nonlinear feature reduction and support vector regression[END_REF], hidden Markov model [START_REF] Dong | A segmental hidden semi-markov model (hsmm)-based diagnostics and prognostics framework and methodology[END_REF], relevance vector machine [START_REF] Liu | Lithium-ion battery remaining useful life estimation with an optimized relevance vector machine algorithm with incremental learning[END_REF][START_REF] Zhou | A novel health indicator for on-line lithium-ion batteries remaining useful life prediction[END_REF], or deep learning using deep neural networks as prediction functions [START_REF] Wu | Remaining useful life estimation of engineered systems using vanilla lstm neural networks[END_REF][START_REF] Sateesh Babu | Deep convolutional neural network based regression approach for estimation of remaining useful life[END_REF][START_REF] Zhang | Long short-term memory for machine remaining life prediction[END_REF][START_REF] Park | Lstm-based battery remaining useful life prediction with multi-channel charging profiles[END_REF]. In recent years, the latter category, deep learning, is becoming increasingly popular, owing to the capacity of deep neural networks in learning highly nonlinear relationships without requiring much effort to extract information manually. Deep neural networks, in other words, can automatically extract features from raw data and use them to produce accurate predictions. On the other hand, some network topologies such as recurrent neural network (RNN) and long short-term memory (LSTM) are specialized for time series forecasting, making them much more prevalent in the context of predictive maintenance. Furthermore, one of the key strengths of deep neural networks is their flexibility: one can design different prediction models by simply modifying the model settings such as the number of hidden layers, the number of neurons in each layer, the activation functions, and so on. This increases the likelihood of obtaining a model that fits the data at hand.

Contribution

We study two important problems in predictive maintenance, which are remaining useful life prediction and state-of-heath/capacity estimation. Two typical datasets used in our study are the turbofan engine degradation simulation dataset (CMAPSS) and the Li-ion battery aging dataset, both of which are provided by NASA (The National Aeronautics and Space Administration). In light of the preceding discussion, we are motivated to adopt the data-driven approach and rely on the wellestablished strength of deep neural networks. Three typical network structures are investigated in our study, namely, feed-forward neural networks, recurrent neural networks, and long short-term memory.

It is worth noting that training a neural network is essentially about optimizing the loss function. We adopt the PDEs-based approach introduced in Chapter 4 to transform the original optimization problems into their smooth versions, in which some latent DC structures are observed. The neural networks are then trained using our proposed MCSDCA-odLD and MCSDCA-udLD algorithms. Numerical findings demonstrate the merits of our methods as they typically achieve lower generalization errors than several baseline optimizers in deep learning, including Adam, Adagrad, RMSProp, and SGD. Overall, the learned systems are capable of capturing the true RUL and the capacity of the turbofan engines and the lithium-ion batteries, respectively.

Our case studies are optimization-oriented, meaning that we focus on the optimization aspect of training deep neural networks to confirm the learning capability of the two proposed algorithms, MCSDCA-odLD and MCSDCA-udLD, over other state-of-the-art optimizers in deep learning.

ŷk+s =

y k + y k-1 + . . . + y 1 k .

Hence, observations are considered to be of equal relevance for the prediction. Another naïve method is to forecast all future values to be the last observed value, ŷk+s = y k .

Between these two extremes, one can also assign higher weights to the latter observations than the former ones,

ŷk+s = αy k + α(1 -α)y k-1 + α(1 -α) 2 y k-2 + . . . + (1 -α) k-1 y 1 .
In order to capture the trend of the time series, Holt's linear trend method [START_REF] Holt | Forecasting seasonals and trends by exponentially weighted moving averages[END_REF] takes into account, beside the forecast equation, two auxiliary smoothing equations

ŷt+s = l t + sb t l t = αy t + (1 -α)(l t-1 + b t-1 ) b t = β(l t -l t-1 ) + (1 -β)b t-1
where the latter two equations are to capture the level and the trend, respectively, and α, β ∈ [0, 1] are smoothing parameters. To capture the seasonality, the above Holt's method was extended to the Holt-Winters' seasonal method [START_REF] Winters | Forecasting sales by exponentially weighted moving averages[END_REF].

The application of this approach is limited, for example, it is inapplicable when the history of the time series is missing, as in formula (5.3).

Statistical forecasting methods This class of methods, which has a strong mathematical foundation, mainly uses linear regression to forecast. Some prominent methods include: AutoRegression (AR),

Moving Average (MA), and AutoRegressive Integrated Moving Average (ARIMA) (see, e.g., [START_REF] Hyndman | Forecasting: principles and practice[END_REF]).

Autoregression uses a linear combination of past values to forecast. An autoregression model of order p can be written as

y t = c + ϕ 1 y t-1 + ϕ 2 y t-2 + . . . + ϕ p y t-p + ϵ t ,
where ϵ t is Gaussian noise and ϕ 1 , ϕ 2 , . . . , ϕ p are regression parameters.

Moving average model, on the other hand, uses a linear combination of past forecast errors

y t = c + ϵ t + θ 1 ϵ t-1 + θ 2 ϵ t-2 + . . . + θ q ϵ t-q ,
where ϵ t , ϵ t-1 , . . . , ϵ t-q are Gaussian noise and θ 1 , θ 2 , . . . , θ q are regression parameters.

Autoregression and Moving average are closely related. Indeed, when we enroll the autoregression formula, it can be written in form of a linear combination of errors. For example, a simple autoregression Machine leaning approach Machine learning techniques have less constraints in terms of linearity and stationarity than statistical forecasting techniques [START_REF] Cheng | Time series forecasting for nonlinear and non-stationary processes: a review and comparative study[END_REF]. The machine learning regression approach for time series prediction includes support vector regression, decision tree, random forest, and deep neural networks such as the vanilla feed-forward neural network, convolutional neural network, recurrent neural network, and long short-term memory. The ability to learn highly nonlinear regression is the approach's key strength, allowing it to cope with difficult time series prediction applications. With this approach, the prediction function f can be a decision tree, a random forest, a neural network, and so on.

The next section introduces three common neural networks that will be employed in this study.

Deep neural networks and optimization problems

In this section, we give a brief introduction to three types of neural networks used in our research:

feed-forward neural network, recurrent neural network, and long short-term memory. For each neural network, we clearly state the associated optimization problem which we aim to solve.

Deep feed-forward neural networks

Deep feed-forward neural networks, or multilayer perceptrons, are the classic deep learning models that are formed by stacking layers together. The phrase "feed-forward" alludes to the fact that information goes through the function from input to hidden layers to output, and there is no feedback connection where the output of a model is fed back to itself [START_REF] Goodfellow | Deep learning[END_REF]. The term deep learning comes from the fact that there are many hidden layers (refers to as the depth of a network) between the input and the output that can extract meaningful features from raw data.

Figure 5.2 presents a typical vanilla feed-forward neural network 6 , knowing that there exist other advanced structures, e.g., convolutional neural networks using special layers such as convolution and pooling. Since each neuron is connected to all neurons in the previous layer, the feed-forward neural network in 5.2 is also called fully-connected (FC) neural network.

The connection between two consecutive layers is in fact an affine transformation followed by a nonlinear transformation (each neuron is activated by a function σ), resulting in the following mathematical expression,

input layer k = W k-1 output layer k-1 + b k-1 , output layer k = σ(input layer k ).
Hence, a feed-forward neural network is a function of the following composition form

ŷ(x) = W l σ (W l-1 σ (W l-2 σ (• • • (W 0 x + b 0 ) • • • ) + b l-2 ) + b l-1 ) + b l .
Given a training dataset {(x 1 , y 1 ), (x 2 , y 2 ), . . . , (x N , y N )}, one tries to adjust the internal parameters of the network to match the network output ŷ to the true corresponding label, yielding the following optimization problem

N i=1 L W l σ W l-1 σ W l-2 σ • • • (W 0 x i + b 0 ) • • • + b l-2 + b l-1 + b l , y i , where Θ = (W 0 , b 0 , W 1 , b 1 , . . . , W l , b l )
is the optimization variable.

Recurrent neural networks

The "feed-forward" feature of the classic neural networks appears to be a limitation in problems where the sequential correlation of data is of major importance. In such a case, it is desirable to have a feedback connection. To this end, RNN maintains a hidden variable that is transferred across time steps. Hence, now the input to be fed into an RNN cell at time t is not just the current data x t , but also the previous hidden state h t-1 , leading to the following update of the hidden state

h t = σ(W xh x t + W hh h t-1 + b h )
and the output

o t = W hq h t + b q ,

Long short-term memory

An LSTM cell [START_REF] Hochreiter | Long short-term memory[END_REF][START_REF] Gers | Learning to forget: Continual prediction with lstm[END_REF] is made up of three gates: forget, input, and output, and it maintains a hidden state that is recurrently updated. At each time step, the LSTM gates are fed with the current input data x t and the previous hidden state h t-1 via three fully-connected layers activated by σ,

i t = σ(W xi x t + W hi h t-1 + b i ), f t = σ(W xf x t + W hf h t-1 + b f ), o t = σ(W xo x t + W ho h t-1 + b o ), where W xi , W xf , W xo , W hi , W hf , W ho , b i , b f ,
c t = f t ⊙ c t-1 + i t ⊙ ct .
The new memory c t going through the tanh activation together with the output gate o t are used to update the hidden state

h t = o t ⊙ tanh(c t ).
Finally, c t and h t are broadcast to the next time step to continue the recurrent update. Figure 5.4 depicts an LSTM cell with all of the components listed 8 .

In summary, the complete equations for an LSTM cell are described as follows

i t = σ(W xi x t + W hi h t-1 + b i ), f t = σ(W xf x t + W hf h t-1 + b f ), o t = σ(W xo x t + W ho h t-1 + b o ), ct = tanh(W xc x t + W hc h t-1 + b c ), c t = f t ⊙ c t-1 + i t ⊙ ct , h t = o t ⊙ tanh(c t ).
Given a sequence of training sample {x 1 , x 2 , . . . , x K } and the label y, similar to the RNN, we stack

ŷ =b f c + W f c [σ(W xo x K + W ho σ(W xo x K-1 + W ho o K-2 ⊙ tanh(c K-2 ) + b o ) ⊙ tanh(σ(W xf x K-1 + W hf h K-2 + b f ) ⊙ c K-2 +σ(W xi x K-1 + W hi h K-2 + b i ) ⊙ tanh(W xc x K-1 + W hc h K-2 + b c )) + b o )]
knowing that h K-2 , c K-2 , o K-2 will be traced back all the way down to h 0 , c 0 (note that o 1 is defined via h 0 ). The starting initial hidden state h 0 and the initial memory c 0 are usually set to be zero vectors.

Given a training dataset consisting of a number of training sequences with their associated labels,

S = {({x 1 1 , x 1 2 , . . . , x 1 K }, y 1 ), ({x 2 1 , x 2 2 , . . . , x 2 K }, y 2 ), . . . , ({x N 1 , x N 2 , . . . , x N K }, y N )}, the optimization problem reads min Θ 1 N N i=1 L(b f c + W f c σ(W xo x i K + W ho σ(W xo x i K-1 + W ho o K-2 ⊙ tanh(c K-2 ) + b o ) ⊙ tanh(σ(W xf x i K-1 + W hf h K-2 + b f ) ⊙ c K-2 +σ(W xi x i K-1 + W hi h K-2 + b i ) ⊙ tanh(W xc x i K-1 + W hc h K-2 + b c )) + b o ) , y i ), here Θ = (W xi , W hi , b i , W xf , W hf , b f , W xo , W ho , b o , W xc , W hc , b c , W f c , b f c
) is the optimization variable to be learned.

Remaining useful life prediction

Data exploration and preprocessing

One of the utmost important tasks in predictive maintenance is to predict the remaining useful life (RUL)

as the RUL provides critical information so that one can arrange an appropriate maintenance schedule.

We study the turbofan engine degradation simulation dataset (CMAPSS) [START_REF] Saxena | Turbofan engine degradation simulation data set[END_REF], where the engine model is of the 90000 lb thrust class [START_REF] Saxena | Damage propagation modeling for aircraft engine run-to-failure simulation[END_REF]. A turbofan engine is an engine that generates thrust and thereby propels an airplane through the air. The CMAPSS dataset contains four subdatasets (FD001, FD002, FD003, FD004), and each -in turn -contains multiple sensor data of hundreds of engines. The information of the CMAPSS dataset is presented in as the engines progressively operate towards the end of their useful life, the values of sensor 7 tend to decrease, while the values of sensor 4 exhibit an increasing trend.

Exponential smoothing Exponential moving average (EMA) is a well-known method to smooth time series. Given a time series {x n }, EMA constructs a smoothed (filtered) time series {s n } as follows

s 0 = x 0 s n = αx n + (1 -α)s n-1 ,
where α ∈ (0, 1) is a parameter controlling the smoothness. If α is close to 0, the filtered time series is nearly constant and the smoothing effect is strong. On the other hand, if α is close to 1, {s n } is similar (see, e.g., [START_REF] Pasa | Operating condition-invariant neural network-based prognostics methods applied on turbofan aircraft engines[END_REF]).

Truncated RUL As the initial wear and tear of each engine are unknown, instead of using the normal linear RUL, we use the truncated RUL for the training data (see, e.g., [START_REF] Sateesh Babu | Deep convolutional neural network based regression approach for estimation of remaining useful life[END_REF]). The truncated RUL is defined as truncated_RUL = min(RUL, T ), where T is the truncation level.

Data exploration and preprocessing

State-of-health (SoH) estimation plays a prominent role in many predictive maintenance paradigms as it indicates the status of the batteries that power the whole system and channels if the batteries' health has crossed the safety threshold so that they will be replaced. In this work, we conduct a case study in forecasting the SoH of Lithium-ion batteries. Lithium-ion battery technology is used as cuttingedge technology in various commercial applications, including electric vehicles, consumer electronics [START_REF] Berecibar | Critical review of state of health estimation methods of li-ion batteries for real applications[END_REF][START_REF] Wang | Probability based remaining capacity estimation using data-driven and neural network model[END_REF], and aerospace/aeronautics industries [START_REF] Marsh | Li ion batteries for aerospace applications[END_REF]. Javelin anti-tank missile, a symbol of Ukraine's resistance in the 21st century, is powered by lithium-ion batteries [START_REF]Javelin-close combat missile system, medium[END_REF]. We use the satellite lithium-ion battery degradation dataset [START_REF] Saha | Battery data set[END_REF]. The SoH at time t of a lithium-ion battery is defined as SoH(t) = Capacity at time t Initial capacity . As the initial capacity is given by the manufacturers, the prediction task is amount to estimating the capacity of the battery. Usually, the battery is marked as reaching its end of life if the SoH is under 70%.

The dataset contains four batteries, namely, B0005, B0006, B0007, and B0018. Each battery underwent many repeated charge-discharge cycles at room temperature (24 Celsius degrees), resulting in the accelerated aging. For each cycle, batteries are charged in a constant current (CC) mode (1.5 A) until the battery voltage hit 4.2 V, then it is switched to the constant voltage (CV) mode until the current falls to 20 mA. Batteries are discharged at the constant current (CC) level of -2 A until the voltage drops to 2.7 V, 2.5 V, 2.2 V, and 2.5 V for batteries 5, 6, 7, and 18, respectively. Figure 5.30 illustrates a charge and discharge process.

Along the process, the battery terminal voltage, the battery output current, the battery temperature, the charger measurement current, the charger measurement voltage and the cycle time are recorded.

During the discharge process, the actual capacity of the batteries is also provided, which serves as the groundtruth label in our study. Figure 5.31 shows the degradation of batteries, where the capacity value at each timestamp is what we want to accurately estimate.

Furthermore, additional impedance measurements obtained by electrochemical impedance spectroscopy (EIS) frequency swept from 0.1 Hz to 5 kHz are provided but are not taken into account here.

As the charge and discharge operations are repeated, distinct patterns can be seen in critical measurements such as battery current and voltage profiles, as well as battery surface temperature, as illustrated in Figures 5.32 , where the MCSDCA-udLD appears to be overfitting. Finally, the results using both charge and discharge data reveal that the MCSDCA-udLD outperform all competitive optimizers on all test batteries and two neural networks.

In comparison between the prediction results obtained using charge/discharge or both forms of data, the MCSDCA-udLD achieves much superior results when discharge data is used (only discharge, or both types of data), and underfits when just charge data is used. In general, the MCSDCA-udLD works best on the RNN when both data types are used; however, on the LSTM, the MCSDCA-udLD performs best when only discharge data is used. Chapter 6

Conclusion and perspectives

In this thesis, we have studied three classes of DC programs: large-sum, stochastic with exogenous uncertainty with i.i.d. samples, stochastic that involves endogenous uncertainty with non-i.i.d. samples.

For each class of programs, we proposed the corresponding stochastic DCA schemes to handle the associated challenges. Several applications in machine learning have been used to verify the effectiveness of the proposed methods. Especially, with the predictive maintenance applications, the MCSDCA-odLD and MCSDCA-udLD have been successful in training a number of neural networks for the associated prediction tasks.

For solving the class of large-sum DC programs, we have proposed two stochastic DCA schemes, DCA-SVRG and DCA-SAGA, integrating variance reduction techniques. The theoretical convergence analysis is studied intensively under both sampling strategies, namely, with and without replacement.

The numerical experiments have shown the virtues of the proposed methods via three machine learning problems: nonnegative principal component analysis, group variable selection in multiclass logistic regression, and sparse linear regression. Our proposed algorithms inherit the virtues as well as the unavoidable limitations of the stochastic estimators used. The proposed methods outperform the standard DCA in terms of gradient complexity, theoretically and experimentally, and they also outperform experimentally the stochastic DCA proposed in [START_REF] Le Thi | Stochastic DCA for minimizing a large sum of dc functions with application to multi-class logistic regression[END_REF] that uses the SAG estimator. However, the SVRG estimator has a limitation that is the variance reduction term inside each epoch is not updated, which undermines the high correlation requirement of the control variate principle. On the other hand, the SAGA, in general, suffers from the storage burden. Our perspective is to use another estimator called SARAH [START_REF] Nguyen | SARAH: A novel method for machine learning problems using stochastic recursive gradient[END_REF] that can possibly address the above drawbacks. Similar to SVRG, SARAH is an epochbased estimator where the full gradient is computed at the beginning of each epoch. However, unlike SVRG where the variance-reduction term is fixed within each epoch, SARAH progressively updates that term as

ν k = ∇f i k (x k ) -∇f i k (x k-1 ) + ν k-1 .
This update comes at the cost of the gradient estimator no longer being unbiased, which is likely to be more difficult to theoretically analyze. However, the performance of such a biased estimator is pretty well, as evidenced by [START_REF] Nguyen | SARAH: A novel method for machine learning problems using stochastic recursive gradient[END_REF][START_REF] Pham | ProxSARAH: An efficient algorithmic framework for stochastic composite nonconvex optimization[END_REF] and our experiments in Chapter 2. This may inspire our future work on the combination between DCA and SARAH.

For solving the class of stochastic DC programs with exogenous uncertainty and i.i.d. samples com-Chapter 6. Conclusion and perspectives ing as a stream, we have designed the online stochastic DCA and its two variants. The convergence properties of the proposed algorithms are rigorously studied, where we established the almost sure convergence to critical points. In addition to the efficiency and flexibility of DCA, the advantages of osDCA schemes include the reduction of storage burden and the capacity to cope with the changes in the data distribution (if any). As a limitation, our main algorithm, in its most general form, requires the knowledge on the Rademacher complexity of the family of functions {g(•, z) : z ∈ Ξ}, which is not always easy to compute. The proposed algorithms' practical behaviors have been thoroughly studied on the expected problem of PCA. The numerical experiments validate the effectiveness of the proposed algorithms, in which they yield good results in a short period of time. Furthermore, the adaptability of osDCA schemes has been demonstrated: when the data distribution changes, our algorithms swiftly adapt to the new distribution. Additional experimental findings confirm the importance of the DC decomposition and the convex solver (for solving subproblems) in DCA. In future work, we will extend our study to include (exogenous) stochastic DC programs with stochastic DC constraints. To the best of our knowledge, the most recent advances in the field of stochastic programming with stochastic constraints are studies for L-smooth functions [START_REF] Ye | Stochastic successive convex approximation for general stochastic optimization problems[END_REF][START_REF] Liu | Stochastic successive convex approximation for non-convex constrained stochastic optimization[END_REF]. Meanwhile, in the context of deterministic optimization, general DC programs (DC objective and DC constraints) have been extensively researched, and the general DCA has been established [START_REF] Le Thi | DC programming and DCA for general DC programs[END_REF][START_REF] Pham Dinh | Recent advances in DC programming and DCA[END_REF]. We aim to investigate some stochastic versions of the general DCA for solving the mentioned class of stochastic DC programs with stochastic DC constraints.

For solving a class of stochastic DC programs where endogenous uncertainty is involved and i.i.d.

samples are unavailable, we have designed a Markov chain stochastic DCA that uses only Markovian data. The proposed algorithm is guaranteed to find DC critical points in both asymptotic and nonasymptotic senses. As an important extension, we provide additional analysis to time-inhomogeneous Markov chains, demonstrating that previously established convergence results still hold true in this case.

We next apply the proposed algorithm to deep learning via PDEs regularization, yielding two MCSDCA realizations, MCSDCA-odLD and MCSDCA-udLD, which are based on overdamped and underdamped Langevin dynamics, respectively. The efficiency of these two specific algorithms have been shown in the predictive maintenance applications via the deep learning approach. Two case studies are carried out: RUL prediction and SoH estimate. Given several suitable neural network topologies, the study confirms the learnability of the MCSDCA-odLD and MCSDCA-udLD. More importantly, the MCSDCA-udLD has exhibited acceleration in cases where the MCSDCA-odLD appears to be slow to converge.

This finding matches our expectation as the underdamped Langevin dynamics, once tailored, should show acceleration over the overdamped Langevin dynamics. However, as a disadvantage, the proposed methods have not yet been able to address the general class of stochastic programs with endogenous uncertainty in which the analytical form of the family of distributions is unknown since it is nearly impossible to highlight a DC decomposition in such a case. This class of problems is much beyond the scope of DC programming and requires a brand new methodology. Furthermore, there is a gap between theory and practice that will be a subject in our future work: the discretization errors (when discretizing the overdamped and underdamped Langevin diffusions) and the stochastic noise induced by the use of mini-batch not yet to be quantified.

  DCA-SVRG et le DCA-SAGA sont appliqués à trois problèmes d'apprentissage automatique : l'analyse en composantes principales non négatives, la sélection de variables de groupe dans la régression logistique multiclasse et la régression linéaire éparse. Deux versions du DCA-SVRG (échantillonnage avec/sans remplacement) et une version du DCA-SAGA (échantillonnage sans remplacement) figurent parmi les meilleurs gagnants dans les trois problèmes. Plus important encore, les avantages de DCA-SVRG et DCA-SAGA en tant qu'algorithmes stochastiques ont été démontrés par rapport au DCA déterministe, et les mérites des techniques de réduction de variance utilisées (estimateurs SAGA et SVRG) ont été démontrés par rapport au DCA stochastique proposé dans [73] employant l'estimateur SAG. Deuxièmement, les comportements pratiques de la DCA stochastique en ligne sont étudiés en détail par le biais du problème théorique de l'analyse en composantes principales. En particulier, étant une méthode en ligne qui utilise de nouveaux échantillons frais à chaque itération, la DCA stochastique en ligne possède une capacité d'adaptation lorsqu'il y a un changement abrupt dans la distribution de Z. Cette remarque est justifiée expérimentalement. Troisièmement, et surtout, les modèles MCSDCA-odLD et MCSDCA-udLD sont utilisés pour entraîner avec succès des réseaux neuronaux profonds, tels que les réseaux neuronaux à action directe, les réseaux neuronaux récurrents et la mémoire à long terme, pour des tâches de maintenance prédictive, telles que la prédiction de la durée de vie utile restante (RUL) et l'estimation de l'état de santé (SoH)/capacité. D'une part, le MCSDCA-odLD a montré sa force dans le problème de prédiction de la durée de vie utile résiduelle, où il surpasse ou rivalise avec un certain nombre d'optimiseurs de base dans l'apprentissage profond, notamment Adam, Adagrad, RMSprop et SGD. D'autre part, le MCSDCA-udLD a démontré qu'il avait une capacité d'accélération supérieure à celle du MCSDCA-odLD dans la tâche d'estimation de la capacité. Étant donné que la dynamique de Langevin sous-amortie présente un certain nombre d'avantages par rapport à la dynamique de Langevin suramortie, notamment l'accélération, ce comportement correspond à nos attentes initiales. Le MCSDCA-udLD bat également les optimiseurs de base mentionnés pour la tâche d'estimation de la capacité. Dans l'ensemble, les deux méthodes proposées ont entraîné efficacement les réseaux neuronaux profonds, car les valeurs prédites de RUL/capacité correspondent étroitement aux valeurs réelles. xx Organisation de la thèse La thèse se compose de six chapitres. Le chapitre 1 contient des informations préliminaires sur la programmation DC et la DCA. Dans le chapitre 2, nous étudions une classe de programmes DC dont les fonctions objectives ont une structure à somme élevée, et nous proposons le DCA-SVRG et le DCA-SAGA. Dans le chapitre 3, nous considérons les programmes DC stochastiques, où nous concevons la DCA stochastique en ligne et ses variantes. La DCA stochastique par chaîne de Markov est développée dans le chapitre 4 pour une classe de programmes DC stochastiques endogènes avec des échantillons non-i.i.d.. Le chapitre 5 traite des applications de la maintenance prédictive. Enfin, le chapitre 6 conclut la thèse.

Firstly, forFigure 1 . 1 :Figure 1 . 2 :

 1112 Figure 1.1: Examples on convex and nonconvex sets

Figure 1 . 3 :Figure 1 . 4 :

 1314 Figure 1.3: Supporting hyperplane of a convex set

Figure 1 . 5 :

 15 Figure 1.5: Graph and epigraph of a function f

Figure 1 . 6 :

 16 Figure 1.6: Affine minorants

  For a convex function, a local minimizer is also a global minimizer. This fundamental fact differentiates the level of difficulty between convex optimization and nonconvex optimization, where the latter does not guarantee the equivalence between locality and globality. It follows straightforwardly from the definition of subgradient that: x * is a global minimizer of f if and only if 0 ∈ ∂f (x * ).

Figure 1 .Figure 1 . 7 : 8 Figure 1 . 8 :

 117818 Figure 1.7: DC decomposition of f = g -h, where g = x 2 + 2x and h = 5|x| + 0.005(x + 1) 4 .

Algorithm 1 .

 1 DCA-SVRG Initialization: x0 ∈ dom r 1 , inner-loop length M , minibatch size b, k = 0, option (either with replacement or without replacement). repeat Compute the full gradient νk = 1 N N i=1 ∇h i (x k ) and set x k+1 0 = xk . for j = 0 : M -1 do if option is with replacement then Randomly choose with replacement the set I b of b elements of [N ]. else Randomly choose without replacement the set I b of b elements of [N ]. end if Compute the "stochastic variance reduced gradient" Solve the convex problem x k+1 j+1 ∈ arg min{G(x) + r 1 (x) -⟨z k+1 j , x⟩ : x ∈ R n }. end for Set xk+1 = x k+1 M , and k = k + 1. until Stopping criterion.

  thanks to the closedness of the graph of the subdifferential operator. By the definition of x u(l) v(l)+1 , we obtain z u(l) v(l) ∈ ∂(G+r 1 ) x u(l) v(l)+1 . Since the graph of the subdifferential operator is closed, by letting l → ∞ we derive ∇H(x * ) + y *

Remark 1 .

 1 About the constraint between b and M , the DC decomposition admits a large number ρ L , we have great flexibility to choose b and M . ii. If the DC problem has a very small value of ρ L , we can adjust the DC decomposition to increase ρ L . Indeed, to this end, we add

2. 3 . 2

 32 The second stochastic DCA: DCA-SAGA In this part, we propose a new stochastic DCA named DCA-SAGA presented in Algorithm 2. The new algorithm, like DCA-SVRG, is a combination of deterministic DCA with stochastic gradient update in the SAGA style. That is, the deterministic DCA's gradient of H is replaced by the stochastic gradient provided by SAGA.

2. 4 . 2 Algorithm 3

 423 Additional convergence analysis for the composite structure of r DCA-SVRG applied to (Q ′ ) Initialization: x0 ∈ dom r 1 , inner-loop length M , minibatch size b, k = 0, option (either with replacement or without replacement). repeat Compute the full gradient νk = 1 N N i=1 ∇h i (x k ) and set x k+1 0 = xk , z k+1 0 = (z k+1 0,1 , z k+1 0,2 , . . . , z k+1 0,m ) ⊤ with z k+1 0,r = p r (x k+1 0 ). for j = 0 : M -1 do if option is with replacement then Randomly choose with replacement the set I t of b elements of [N ]. else Randomly choose without replacement the set I t of b elements of [N ]. end if Compute the "stochastic variance reduced gradient" t k+1 j

3 4

 3 ⌉, ⌈2 N √ N + 1⌉ for DCA-SVRG-v1, DCA-SVRG-v2, DCA-SAGA-v1, DCA-SAGA-v2, respectively. We set the inner loop length M for DCA-SVRG-v1 and DCA-SVRG-v2 to be ⌊

2 3 ⌋

 3 [START_REF] Reddi | Proximal stochastic methods for nonsmooth nonconvex finite-sum optimization[END_REF]. The hyperparamters of prox-SARAH are chosen as follows[101, Theorem 8]: the minibatch size b = ⌊ √ n⌋, the inner loop length M = ⌊N/b⌋, the step size η and the averaging parameter γ: ω := 3(N -b) 2b(N -1) .

  Figure 2.1 shows that two versions of DCA-SVRG, prox-SARAH, and prox-SpiderBoost perform very well and typically take the lead, with suboptimality around 10 -15 the majority of cases. It is then followed by the sampling without replacement version of DCA-SAGA (DCA-SAGA-v2) which obtains a good suboptimality (usually less than 10 -10 ). While the performance of DCA-SVRG-v1 and

  (a) a9a (b) aloi (c) cifar10 (d) SensIT Vehicle (e) connect-4 (f) letter (g) mnist (h) protein (i) shuttle (j) YearPredictionMSD

Figure 2 . 1 :

 21 Figure 2.1: The suboptimality of all algorithms over 10 datasets

3

  ⌋, ⌊10%N ⌋ for DCA-SAGA, DCA-SVRG-v1, DCA-SVRG-v2, SDCA, respectively. The inner loop length M of DCA-SVRG-v1 and DCA-SVRG-v2 is set to be ⌊

Figures 2 .

 2 Figures 2.2, 2.3, 2.4, and 2.5 report the mean curves of the objective (averaged over 10 runs) and the mean absolute deviation of five algorithms. To improve visibility, the MAD is scaled up by a factor of 20.

4 Figure 2 . 2 :

 422 Figure 2.2: The (averaged) objective value of five algorithms for the case η exp and q = 1.

Figure 2 . 3 :

 23 Figure 2.3: The (averaged) objective value of five algorithms for the case η exp and q = 2.

4 Figure 2 . 4 :

 424 Figure 2.4: The (averaged) objective value of five algorithms for the case η cap-ℓ 1 and q = 1.

Figure 2 . 5 :

 25 Figure 2.5: The (averaged) objective value of five algorithms for the case η cap-ℓ 1 and q = 2.

  , ⌊10%N ⌋ for DCA-SVRG-v1, DCA-SVRG-v2, DCA-SAGA, and SDCA, respectively. For two versions of DCA-SVRG, we set M = ⌊

Figure 2 . 6 :

 26 Figure 2.6: The performance of seven algorithms on the sparse linear regression problem

Algorithm 5

 5 Online Stochastic DCA with exact G Algorithm 4, in which the problem (3.5) in step 3 is replaced by min G(w) -⟨t k , w⟩ : w ∈ R m .

  the Lipschitz smoothness constant L = 1 and the tolerance error in solving subproblems ϵ = 10 -5 .Next, the training dataset and the validation dataset are merged together and are fed to all algorithms (one pass -to meet the online setting context and guarantee the independence of samples). As a preprocessing step, each training dataset is randomly shuffled before each run. The starting points are also randomly initialized in S. The performance of our algorithms are measured on the test set to guarantee their generalization capability. To enhance visualization, we report the suboptimality graph F (w n ) -F (w * ) averaging over 20 runs, where F (w * ) is the optimal value found on each test set by computing the largest eigenvalue of the covariance matrix. Furthermore, we classify osDCA-1, SDCA1, SDCA3 in one group and osDCA-2, SDCA2, SDCA4 in another group (since the former three use the DC decomposition (3.16) and the latter three use (3.17)) to plot them in two different figures.

Figure 3 .

 3 Figure 3.3 shows the behaviors of osDCA-1 with different λ > 0 and an extreme case where λ = 0

Figure 3 . 1 :

 31 Figure 3.1: The performance of osDCA-1 compared with SDCA1, SDCA3, two versions of PSS, AdaOja, and MaxAP

Figure 3 . 2 :Figure 3 . 3 :Figure 3 . 4 :Figure 3 . 5 :

 32333435 Figure 3.2: The performance of osDCA-2 compared with SDCA2, SDCA4, two versions of PSS, AdaOja, and MaxAP

  representing data collected before and after the abrupt change. The first subdataset includes a training set (200000×500) and a test set (500000×500) that are generated from multivariate normal distribution with a mean vector 0 and a positive definite covariance matrix Σ = Λ ⊤ Λ + I where Λ is randomly generated. Then, we change Λ and generate the second subdataset consisting of a training set (200000 × 500) and a test set (200000 × 500). All data is then normalized as ∥z i ∥ = 1. We concatenate two training sets to create one unified training set in order to feed to the algorithms. Before the change, we measure the performance of each algorithm on the first test set, and after the change, we use the second test set.

Figure 3 .

 3 Figure 3.5 shows the average results of 20 runs, here we separate the results into two subfigures because the running times of SDCA2, SDCA3, SDCA4 are remarkably longer than osDCA-1, osDCA-2, and SDCA1. The numerical results confirm the adaptive capacity of osDCA schemes over SDCA schemes. Indeed, after the abrupt change, osDCA schemes quickly regain suboptimality values that are as good as the ones obtained before the change. Meanwhile, SDCA schemes barely adapt to the change and decrease the suboptimality slowly.

Figure 3 . 6 :

 36 Figure 3.6: Comparison between the three proposed osDCA schemes

  chains that play a key role in sampling are time-inhomogeneous. Then, we show how to decompose a broad class of PDE-based regularization structures into DC decompositions that may be utilized to train deep neural networks. In the future, this could help to stimulate the use of the DCA-based approach to solve other nonconvex regularized deep learning challenges. Finally, thanks to the discovered DC structure, the MCSDCA is applied specifically to deep learning. There are two realizations of MCS-DCA in this application. The first one is named MCSDCA-odLD (MCSDCA overdamped Langevin dynamics) where Markov chains are constructed based on the overdamped Langevin diffusion with Euler-Maruyama discretization. The second one is named MCSDCA-udLD (MCSDCA underdamped Langevin dynamics) where Markov chains are developed based on the underdamped Langevin diffusion with a novel discretization scheme introduced in[START_REF] Cheng | Underdamped langevin mcmc: A non-asymptotic analysis[END_REF]. We also discuss the implementation aspect of the MCSDCA-udLD in the context of deep neural networks (which is more complicated than the MCSDCA-odLD). Numerical experiments on standard neural networks including feed-forward neural networks, LSTM (long short-term memory), and RNN (recurrent neural networks) are postponed to the next chapter where we will study time series prediction problems arising in predictive maintenance.

  y) h(z)P k+s k (y, dz)P k (x, dy) = h(y) h(z)P k+s k (y, z)P k (x, y)dzdy. (4.13) On the other hand, h(y) h(z)P k+s k (y, z)P k (x, y)dzdy -h(y) h(z)P k+s k (y, z)π(y)dzdy = h(y) h(z)P k+s k (y, z)(P k (x, y) -π(y))dzdy ≤ h(y) h(z) P k+s k (y, z) -π(z) P k (x, y) -π(y) dzdy A + h(y) h(z)π(z) P k (x, y) -π(y) dzdy B . (4.14) We prove B = 0. Indeed, h(y) h(z)π(z) P k (x, y) -π(y) dzdy = h(y) h(z)π(z)P k (x, y)dzdy -h(y) h(z)π(z)π(y)dzdy = h(y)P k (x, y) h(z)π(z)dz =0 dy -h(y)π(y) h(z)π(z)dz =0 dy = 0.

  )π(y) h(z)P k+s k (y, z)dz -h(z)π(z)dz dy ≤ | h(y)|π(y) h(z)P k+s k (y, z)dz -h(z)π(z)dz ⊠ dy.

4. 4

 4 Applications to PDEs regularization in Deep learning 4.4.1 Regularization based on PDEs and a DC structure of the regularized problem PDE-based regularization techniques have recently risen to prominence as a method for training deep neural networks with good generalization [21, 20]. Consider the deep learning optimization problem, min f (x),

Figure 4 . 1 :

 41 Figure 4.1: Data structure of trainable parameters of a recurrent neural network

Figure 4 .

 4 Figure 4.1 illustrates a typical data structure of trainable parameters of a recurrent neural network.

Figure 5 . 2 :

 52 Figure 5.2: Feed-forward neural network

Figure 5 .

 5 Figure 5.4: LSTM cell

Figure 5 . 5 :Figure 5 . 6 :Figure 5 . 7 : 1 Figure 5 . 8 :

 555657158 Figure 5.5: Sensor 7, dataset FD001

Tables 5 . 2 , 5 . 3 , 5 . 4 ,

 525354 and 5.5 report the average results on the datasets FD001, FD002, FD003, and FD004, respectively. The MCSDCA-odLD outperforms other baselines on the fully-connected NN in three criteria for the dataset FD001, while RMSprop outperforms other baselines on the LSTM model in terms of generalization errors. On the LSTM models for the datasets FD002 and FD003, MCSDCA-odLD achieves the best generalization performance. Meanwhile, Adagrad wins on the dataset FD003 (albeit the test error margin between MCSDCA-odLD and Adagrad is narrow: 14.95 and 14.94), MCSDCA-odLD wins on FD002, and Adagrad wins on FD003. MCSDCA-odLD produces the best train, validation, and test results on the fully-connected NN for the dataset FD004, which is the most complex (6 operating conditions and 2 fault modes) dataset among the four datasets. Meanwhile, Adam comes in second place, catching up to MCSDCA-odLD's performance on the validation and test sets. MCSDCA-odLD is the sole winner on the LSTM. The training and validation curves (the original objective, plotted against the number of backprop calls -agnostic to less relevant factors, and the training curves are produced by taking the average of all mini-batch training losses thus far) displayed in Figures 5.10

  ,

Figure 5 . 10 :Figure 5 . 11 :Figure 5 . 12 :

 510511512 Figure 5.10: Training and validation curves on the fully-connected NN, dataset FD001

Figure 5 . 13 :Figure 5 . 14 :Figure 5 . 15 :

 513514515 Figure 5.13: Training and validation curves on the LSTM, dataset FD002

Figure 5 . 16 :Figure 5 . 17 :Figure 5 . 18 :Figure 5 . 21 :Figure 5 . 20 :

 516517518521520 Figure 5.16: Training and validation curves on the fully-connected NN, dataset FD004

  , 5.23, 5.24, 5.25, 5.26, 5.27, 5.28, 5.29 show that discarding more samples seems to improve training convergence but also increases the risk of overfitting.

Figure 5 . 22 :

 522 Figure 5.22: Burn-in effect on the fully-connected NN, dataset FD001

Figure 5 .Figure 5 . 25 :

 5525 Figure 5.24: Burn-in effect on the fully-connected NN, dataset FD002

Figure 5 .Figure 5 . 28 :

 5528 Figure 5.27: Burn-in effect on the LSTM, dataset FD003

Figure 5 . 30 :

 530 Figure 5.30: One cycle of charge-discharge

  , 5.33, and 5.34.Figure 5.34, for example, shows that the temperature of an old battery reaches a peak when charged and discharged quicker than a fresh battery. Data normalization and extraction We use the following data fields: Voltage measured, Current measured, Current load, Voltage load, Temperature measured, Time. Data in each field is then scaled to the range (0, 1) by minmax scaler. As one cycle contains a large number of data points, we uniformly extract 15 points for each charge and discharge phase. For example, Figure 5.35 shows data points extracted from one discharge voltage profile.

Figure 5 Figure 5 . 32 :

 5532 Figure 5.31: Capacity degradation

Figure 5 . 33 :

 533 Figure 5.33: Voltage profiles change as the battery B005 ages

Figure 5 . 34 :

 534 Figure 5.34: Temperature profiles change as the battery B005 ages

Figures 5. 37

 37 Figures 5.37, 5.38, 5.39 show the training curves of all considered algorithms. As the dataset sizes

  Figures 5.37, 5.38, 5.39 show the training curves of all considered algorithms. As the dataset sizes are modest, these training curves are generated by computing the full-batch training loss at the present solution, which correctly captures the training process. It is observed that it is the only competition between the MCSDCA-udLD and Adam when other optimizers are left behind. In comparison between Adam and MCSDCA-udLD, Adam decreases the objective faster than MCSDCA-udLD at the beginning. Then, the MCSDCA-udLD, in most cases, can catch up with Adam at some point and then strongly surpasses Adam till the end of the training process.

Figure 5 . 37 :Figure 5 . 38 :Figure 5 . 39 :Figure 5 . 40 :

 537538539540 Figure 5.37: Training curves, charge data

RésuméL

  'optimisation stochastique revêt une importance majeure à l'ère du big data et de l'intelligence artificielle. Ceci est attribué à la prévalence de l'aléatoire/de l'incertitude ainsi qu'à la disponibilité toujours croissante des données, deux facteurs qui rendent l'approche déterministe infaisable. Cette thèse étudie l'optimisation stochastique non convexe et vise à résoudre les défis du monde réel, notamment l'extensibilité, variance élevée, l'incertitude endogène et le bruit corrélé. Le thème principal de la thèse est de concevoir et d'analyser de nouveaux algorithmes stochastiques basés sur la programmation DC (différence de fonctions convexes) et DCA (algorithme DC) pour répondre aux nouvelles problématiques émergeant dans l'apprentissage automatique, en particulier l'apprentissage profond. Comme application industrielle, nous appliquons les méthodes proposées à la maintenance prédictive où le problème central est essentiellement un problème de prévision de séries temporelles. La thèse se compose de six chapitres. Les préliminaires sur la programmation DC et le DCA sont présentés dans le chapitre 1. Le chapitre 2 étudie une classe de programmes DC dont les fonctions objectives contiennent une structure de somme importante. Nous proposons deux nouveaux schémas DCA stochastiques, DCA-SVRG et DCA-SAGA, qui combinent des techniques de réduction de la variance et étudient deux stratégies d'échantillonnage (avec et sans remplacement). La convergence presque sûre des algorithmes proposés vers les points critiques DC est établie, et la complexité des méthodes est examinée. Le chapitre 3 étudie les programmes DC stochastiques généraux (la distribution de la variable aléatoire associée est arbitraire) où un flux d'échantillons i.i.d. (indépendants et identiquement distribués) de la distribution intéressée est disponible. Nous concevons des schémas DCA stochastiques dans le cadre en ligne pour résoudre directement ce problème d'apprentissage théorique. Le chapitre 4 considère une classe de programmes DC stochastiques où l'incertitude endogène est en jeu et où les échantillons i.i.d. ne sont pas disponibles. Au lieu de cela, nous supposons que seules les chaînes de Markov qui sont ergodiques assez rapidement vers les distributions cibles peuvent être accédées. Nous concevons ensuite un algorithme stochastique appelé DCA stochastique à chaînes de Markov (MCS-DCA) et fournissons une analyse de convergence dans les sens asymptotique et non asymptotique.La méthode proposée est ensuite appliquée à l'apprentissage profond via la régularisation des EDP (équations différentielles partielles), ce qui donne deux réalisations de MCSDCA, MCSDCA-odLD et MCSDCA-udLD, respectivement, basées sur la dynamique de Langevin suramortie et sous-amortie.Les applications de maintenance prédictive sont abordées au chapitre 5. La prédiction de la durée de vie utile restante (RUL) et l'estimation de la capacité sont deux problèmes centraux étudiés, qui peuvent tous deux être formulés comme des problèmes de prédiction de séries temporelles utilisant l'approche guidée par les données. Les modèles MCSDCA-odLD et MCSDCA-udLD établis au chapitre 4 sont utilisés pour former ces modèles à l'aide de réseaux neuronaux profonds appropriés. En comparaison avec divers optimiseurs de base en apprentissage profond, les études numériques montrent que les deux techniques sont supérieures, et les résultats de prédiction correspondent presque aux vraies valeurs de RUL/capacité. Enfin, le chapitre 6 met un terme à la thèse.Mots-clés: Maintenance Prédictive, Apprentissage profond, Programmation DC et DCA

  Dinh Tao, puis développé par lui et Le Thi Hoai An, ainsi que quelques collaborateurs, est une technique efficace pour résoudre les programmes DC, en particulier ceux avec des décompositions DC adaptées. La programmation DC et la DCA ont été appliquées avec succès dans divers domaines scientifiques, par exemple, le transport et la logistique, la fouille de données et l'apprentissage automatique, la biologie et la chimie computationnelles, les réseaux de communica-

tion, la finance et l'économie, le traitement d'images (voir

[START_REF] Le Thi | DC programming and DCA: thirty years of developments[END_REF]

pour un aperçu détaillé). La principale force de DCA réside dans sa flexibilité, qui est due au fait qu'une fonction DC possède une infinité de décompositions DC, chacune d'entre elles donnant lieu à un DCA correspondant (c'est pourquoi DCA est généralement qualifié de philosophie plutôt que d'algorithme spécifique). L'objectif de cette thèse est d'étendre la programmation DC et la DCA du cadre déterministe au cadre stochastique. Dans ce dernier cadre, le principe DCA conseille comment traiter la non-convexité, alors que d'autres techniques stochastiques doivent être utilisées en plus pour traiter l'incertitude.

  problème de la maintenance prédictive, lorsqu'on utilise l'approche axée sur les données, est essentiellement un problème de prédiction de séries temporelles, où l'objectif est de prédire les quantités d'intérêt telles que la durée de vie utile restante d'un moteur. Les 'objectif principal de la thèse est de concevoir et d'étudier des algorithmes stochastiques innovants basés sur la programmation DC et DCA afin de relever les nouveaux défis de l'apprentissage automatique, en particulier l'apprentissage profond. D'une part, les algorithmes sont conçus pour plusieurs classes de programmes d'optimisation stochastique non convexes, avec des analyses de convergence fermes fournies. Ils sont, d'autre part, utilisés pour résoudre des problèmes d'apprentissage automatique tels que l'analyse en composantes principales, la sélection de variables de groupe dans la régression logistique multiclasse et l'entraînement de réseaux neuronaux profonds via la régularisation des EDP (équations différentielles partielles). En tant qu'application industrielle, les méthodes proposées seront appliquées à la maintenance prédictive via une approche d'apprentissage profond.Sur le plan théorique, nous concevons et étudions un certain nombre d'algorithmes stochastiques pour résoudre des programmes DC stochastiques. Tout d'abord, nous considérons un programme DC Étant donné que la taille du mini-batch augmente à un rythme approprié au fil des itérations, la convergence presque certaine vers les points critiques DC est assurée. Troisièmement, nous étudions une classe de programmes DC stochastiques dans lesquels l'incertitude endogène est présente et les échantillons i.i.d. sont indisponibles. Au lieu de cela, nous supposons que seules les chaînes de Markov ergodiques suffisamment rapides pour atteindre les distributions cibles sont accessibles. Ce contexte est valable car le bruit markovien se produit dans une variété de contextes, y compris Il s'agit d'une extension importante puisque certaines chaînes de Markov basées sur la diffusion qui jouent un rôle clé dans l'échantillonnage sont homogènes en temps. Enfin, nous montrons comment décomposer une large classe de structures de régularisation basées sur les EDP en décompositions DC qui peuvent être utilisées pour entraîner des réseaux neuronaux profonds. Grâce à la structure DC découverte, la MCSDCA est appliquée spécifiquement à l'apprentissage profond. Il existe deux réalisations de la MCSDCA dans cette application. La première est appelée MCSDCA-odLD (MCSDCA overdamped Langevin dynamics) où les chaînes de Markov sont construites sur la base de la diffusion de Langevin suramplifiée avec une discrétisation Euler-Maruyama. La seconde est appelée MCSDCA-udLD (MCSDCA underdamped Langevin dynamics) où les chaînes de Markov sont

réseaux neuronaux profonds sont des candidats intéressants pour les prédicteurs, car la relation entre les données du capteur et les quantités d'intérêt peut être hautement non linéaire. Compte tenu de la topologie d'un réseau neuronal, l'ajustement des paramètres internes du réseau pour capturer une telle relation est fondamentalement un problème d'optimisation (hautement) non convexe. xviii Nos contributions Lavec une structure à somme élevée. La structure à somme élevée se présente généralement comme le problème de minimisation du risque empirique (ERM) du programme stochastique (1) utilisant un grand ensemble d'échantillons i.i.d.. Autrement dit, l'espérance de (1) est approximée par la méthode de Monte-Carlo. Dans une telle situation, le nombre de sommets est équivalent au nombre de points de données utilisés, ce qui est énorme pour les ensembles de données modernes. Une approche naturelle est d'approximer itérativement les quantités pertinentes de grandes sommes (telles que le sous-gradient de la deuxième composante DC) par certaines contreparties stochastiques dans un style mini-batch. Cependant, la variance de cette approche est élevée, ce qui fait que la séquence générée fluctue à proximité des points critiques mais ne converge pas vers ces points critiques. Nous proposons donc deux nouveaux schémas DCA stochastiques, DCA-SVRG et DCA-SAGA, qui combinent des techniques de réduction de la variance et étudient les deux stratégies d'échantillonnage, à savoir l'échantillonnage avec et sans remplacement. La convergence presque sûre des algorithmes proposés vers les points critiques DC est établie, et la complexité des méthodes est examinée. Deuxièmement, nous étudions comment résoudre directement les programmes DC stochastiques avec une variable aléatoire arbitraire Z (sous la forme de (1) où f (•, z) sont des fonctions DC). En comparaison, la première approche résolvant le problème ERM aura quelque chose à voir avec une sorte d'écart de généralisation. Nous essayons maintenant -au moins théoriquement -de combler ce fossé. La situation d'intérêt est i.i.d. les échantillons de la distribution de Z arrivent séquentiellement (données en continu), donc on n'a pas un grand nombre d'échantillons au début et l'algorithme proposé devrait être capable d'apprendre en ligne. Nous concevons un algorithme appelé DCA stochastique en ligne et deux variantes pour résoudre ce problème d'apprentissage statistique. l'inférence bayésienne, l'apprentissage par renforcement et l'optimisation stochastique dans des espaces hautement dimensionnels ou combinatoires. Nous concevons ensuite un algorithme stochastique appelé DCA stochastique par chaîne de Markov (MCSDCA) et fournissons l'analyse de convergence au sens asymptotique et non asymptotique. De plus, nous étendons notre théorie pour inclure les chaînes de Markov temps-inhomogène.

  To obtain a (nearly) ϵcritical point, they need O(1/ϵ 2 ) stages in total[START_REF] Xu | Stochastic optimization for dc functions and non-smooth non-convex regularizers with non-asymptotic convergence[END_REF] Lemma 3 and Corollary 9]. Each stage consumes a full gradient of H and requires solving inexactly a convex subproblem, resulting in the complexity of O(N/ϵ 2 ) in terms of gradient evaluations and O(1/ϵ 2 ) in terms of the number of convex subproblems.

  .1) is a nonconvex (resp. nonsmooth) stochastic optimization problem. Similar to deterministic optimization, the transition from convex to nonconvex settings presents challenges. Nonsmooth and nonconvex stochastic optimization, which is significantly more difficult than convex and/or smooth optimization, plays a pivotal role in a variety of practical applications. Nevertheless, techniques for nonsmooth and nonconvex stochastic optimization are still limited. The material of this chapter is extracted from the following works: H. A. Le Thi, H. P. H. Luu, & T. Pham Dinh (2022). Online Stochastic DCA with applications to Principal Component Analysis. Accepted for publication, IEEE Transactions on Neural Networks and Learning Systems. arXiv:2108.02300. H. A. Le Thi, T. Pham Dinh, H. P. H. Luu, & H. M. Le, Deterministic and stochastic DCA for DC programming, Handbook of Engineering Statistics 2nd edition, In press, Springer.

  a nonempty, compact, and convex set, g(•, Z) and h(•, Z) are lower semi-continuous convex functions for all Z, satisfying some technical conditions described later. The framework of the problem (3.2) is very general in two respects. Firstly, the underlying distribution of Z is arbitrary, which makes it capable of treating any random variable involved. As a special case, when Z is uniformly distributed over a finite set {z 1 , z 2 , . . . , z N }, we obtain a large-sum problem which has been addressed

	in Chapter 2,
	min w∈S

Table 3 .

 3

		1: Datasets' information
	Dataset	Features	Train-Validation split	Test set
	letter	16	15000:2500	2500
	YearPredictionMSD	90	463715:25815	25815
	SensIT Vehicle	100	78823:9852	9853
	shuttle	9	43500:7250	7250
	covtype	54	500000:40506	40506
	SUSY	18	4900000:50000	50000
	codrna	8	59535:78706	78707
	madelon	500	2000:300	300
	mushrooms	112	6000:1000	1124
	satimage	36	4435:1000	1000
	a7a	123	12880:1610	1610
	a9a	123	26049:3256	3256

Table 3 .

 3

		2: Average reconstruction errors of all studied algorithms
		osDCA-1 osDCA-2 SDCA1 SDCA2 SDCA3 SDCA4 PSS-dimi PSS-const AdaOja MaxAP
	letter	0.4412 0.4412 0.4412 0.4413 0.4412 0.4412 0.4412 0.4413 0.4423 0.4490
	YearPredictionMSD 0.1782 0.1782 0.1782 0.1782 0.1782 0.1782 0.1782 0.1783 0.1783 0.1785
	SensIT Vehicle	0.7054 0.7054 0.7054 0.7059 0.7054 0.7054 0.7054 0.7054 0.7058 0.7212
	shuttle	0.1130 0.1130 0.1130 0.1130 0.1130 0.1130 0.1130 0.1130 0.1131 0.1140
	covtype	0.1096 0.1096 0.1096 0.1096 0.1096 0.1096 0.1096 0.1097 0.1097 0.1096
	SUSY	0.4550 0.4550 0.4550 0.4550 0.4550 0.4550 0.4550 0.4551 0.4550 0.4566
	codrna	0.0151 0.0151 0.0151 0.0151 0.0151 0.0151 0.0151 0.0152 0.0152 0.0157
	madelon	0.0037 0.0037 0.0037 0.0037 0.0037 0.0037 0.0037 0.0038 0.0038 0.0047
	mushrooms	0.6028 0.6030 0.6032 0.6043 0.6031 0.6031 0.6029 0.6044 0.6069 0.6082
	satimage	0.6271 0.6307 0.6337 0.6322 0.6319 0.6319 0.6313 0.6307 0.6277 0.6600
	a7a	0.5491 0.5490 0.5490 0.5491 0.5490 0.5490 0.5490 0.5496 0.5506 0.5582
	a9a	0.5478 0.5478 0.5478 0.5479 0.5478 0.5478 0.5478 0.5479 0.5489 0.5567
	which is the fastest algorithm among all SDCA schemes, is 2.8 ∼ 7.7 times slower than osDCA-1; PSS

with diminishing stepsize is 2.9 ∼ 7.1 times slower than osDCA-1.

  Luu, H. M. Le, & H. A. Le Thi. Markov Chain Stochastic DCA and Applications in Deep Learning with PDEs Regularization. To be submitted.

d. samples from P (ξ|x) are unavailable, but a Markov chain with an equilibrium distribution P (ξ|x) can be constructed. Departing from the classical settings where the objective is convex and/or L-smooth and the uncertainty is exogenous (not affected by the optimization variable) with i.i.d. samples at hand, our setting has (direct or indirect) implications in challenging The material of this chapter is developed based on the following work: H. P. H.

  Conditional distribution of a multivariate normal distribution is again multivariate normal. To be specific, let (X, Y ) be a pair of random vectors, whose joint distribution of multivariate normal

	with mean µ =	µ 1 µ 2	and covariance matrix Σ =	Σ 11 Σ 12 Σ 21 Σ 22	. Then, the conditional distribu-
	tion of X given Y = a is multivariate normal N (μ, Σ), where μ and Σ are given as follows (see,
	e.g., [41, Section 3.4])

  b o are parameters to be learned. Similarly, the candidate memory cell, ct , is computed by passing h t-1 and x t through another fully-connected layer with tanh activation, which ends up being scaled to (-1, 1),ct = tanh(W xc x t + W hc h t-1 + b c ),where W xc , W hc , b c are again parameters to be learned. Then, the LSTM calculates the actual memory c t (that will be transmitted to the next time step) based on the previous memory c t-1 forgotten some amount of old information addressed by the forget gate f t , and the candidate memory ct governed by the input gate i t to control how much new information is taken into consideration,

Table 5 .

 5 [START_REF] Aladag | A new time invariant fuzzy time series forecasting method based on particle swarm optimization[END_REF], where an operating condition is a combination of three settings (flight altitude, Mach number, and throttling parser angle), and the considered fault modes include high-pressure compressor (HPC) degradation and fan degradation.

	Dataset Operating conditions Fault modes Number of training engines Number of test engines
	FD001	1	1	100	100
	FD002	6	1	260	259
	FD003	1	2	100	100
	FD004	6	2	248	249

Table 5 .

 5 1: CMAPSS dataset informationSensorial data of each engine is presented in form of 21 time series, where the actual physical meaning of each time series is unknown. In the beginning, each engine operates normally, and then gradually develops fault over time, where there is one fault mode for the datasets FD001 and FD002, and two fault modes for FD003 and FD004. In the training sets, the engines are let to run to failure, i.e., the moment when the engines are considered as being unhealthy. On the other hand, the sensors recorded in the test sets terminated sometime before the failure, and the goal is to predict the RUL of each engine of the test set. Among 21 sensors, s 1 , s 5 , s 6 , s 10 , s 16 , s 18 , s 19 contain no useful trend for predicting the RUL. Hence, only 14 time series are used for the prediction. Figures 5.5 and 5.6 depict the sensor 7 and sensor 4 of all engines of dataset FD001. The trends of these sensors are clearly observed:

Table 5 . 5

 55 5.11, 5.12, 5.13, 5.14, 5.15, 5.16 and 5.17 show that MCSDCA-odLD exhibits a strong decrease like RMSprop and SGD at the very beginning of the training process, but is much better than SGD afterwards, and is less prone to underfitting and overfitting than RMSprop. On both types of networks, the MCSDCA-odLD prediction results are quite close to the true RUL on the test sets (Figures 5.18, 5.19, 5.20, and 5.21). Overall, MCSDCA-odLD obtains the lowest test errors in 5 of 8 case studies.

	Algorithm	Fully-connected NN Train Validation Test	LSTM Train Validation Test
	MCSDCA-odLD 14.61	11.54	12.75	17.35	15.14	14.76
	Adam	21.24	12.11	13.10	29.71	15.03	14.45
	Adagrad	20.60	12.81	13.60	33.92	23.98	18.92
	RMSprop	16.71	13.19	15.66	16.17	14.67	14.42
	SGD	25.82	19.07	21.73	15.95	15.59	15.40
		Table 5.2: Results on dataset FD001
	Algorithm	Fully-connected NN Train Validation Test	LSTM Train Validation Test
	MCSDCA-odLD 20.22	19.32	21.88	21.18	19.37	21.47
	Adam	21.13	19.61	21.87	25.22	20.67	23.35
	Adagrad	21.18	20.63	21.02	26.78	21.76	23.21
	RMSprop	35.65	27.16	29.04	19.89	20.05	22.41
	SGD	49.81	49.44	54.18	29.99	27.06	30.27
		Table 5.3: Results on dataset FD002

: Results on dataset FD004

  5.5.3 Experimental resultsTables 5.6, 5.7, and 5.8 report the average training and test results when using charge, discharge, and both charge-discharge profiles, respectively. When only charge data is used, the MCSDCA-udLD obtains the best train and test results on both neural networks for the test battery B0006, on the RNN with the test battery B0005, and the best test result on the LSTM with the test battery B0018. Meanwhile, Adam wins the LSTM for the test battery B0005 and achieves the best test error on the RNN for the test battery B0007. Adagrad and RMSprop gain the best test results on the LSTM, test battery B0007 and the RNN, test battery B0018, respectively. Regarding the results using only discharge data, the MCSDCA-udLD is the sole winner on both types of neural networks for the test batteries B0005 and B0007. Meanwhile, Adam achieves the best test results on both types of neural networks for the test batteries B0006 and B0018

The datasets can be downloaded from https://www.csie.ntu.edu.tw/~cjlin/libsvm/.

This figure is produced by using https://alexlenail.me/NN-SVG/.

This figure is produced by using https://app.diagrams.net/.

This figure is produced by using https://app.diagrams.net/.

Remerciements

We denote a set of events

A < ∞, ∥y k -E P (ξ|x k ) (v(x k , ξ))∥ → 0, ϵ k → 0, {x k }, {y k } are bounded .

Obviously, P(S) = 1. Now fix an event ω ∈ S, giving rise to realization sequences of {x k } and {y k }, in which -for simplicity -we still call these realizations {x k } and {y k }. Let x * be a limit point of {x k }, there exists a subsequence {x l k } converging to x * . Since ∥x l k +1 -x l k ∥ → 0, we also have

From the definition of x l k +1 ,

By passing to subsequence if necessary, we can assume that y l k → y * . Taking lim inf on two sides of the above inequality and noting that G is lower semicontinuous,

Hence,

which implies y * ∈ ∂G(x * ) + γA(x * -x * ) = ∂G(x * ).

On the other hand, from the definition of S, ∥y l k -E P (ξ|x l k ) (v(x l k , ξ))∥ → 0. As a result, E P (ξ|x l k ) (v(x l k , ξ)) → y * .

In combination with E P (ξ|x l k ) (v(x l k , ξ)) ∈ ∂H(x l k ), it holds y * ∈ ∂H(x * ) thanks to the closedness of the subdifferential map. Therefore, as ∂G(x * ) ∩ ∂H(x * ) ̸ = ∅, x * is a critical point of F = G -H.

Additional analysis for the time-inhomogeneous case

In this section, we consider the case where each Markov chain can be time-inhomogeneous. In practice, time-inhomogeneous Markov chains play an important role as they offer more flexibility and sometimes give acceleration to the equilibrium distribution in comparison with the time-homogeneous case.

Different from the time-homogeneous case, the transition law of a time-inhomogeneous Markov chain can vary over time, i.e., the chain is characterized by a sequence of Markov kernels. We adapt the notion of geometric ergodicity to the time-inhomogeneous case as follows.

Let X be a general state space and π be a distribution on X . Let {Z k : k ≥ 0} be a timeinhomogeneous X -valued Markov chain. We denote P k+s k (x, dy) the probability distribution of Z k+s given Z k = x. When k = 0, we simply write P s (x, dy). The Markov chain is said to be geometrically Chapter 5

Predictive Maintenance: a deep learning approach Abstract. In this chapter, we study predictive maintenance -one of the key components in prognostics and health management. Predictive maintenance, with its ability to foresee operation failures and estimate assets' health, is able to reduce risks, maintenance costs and downtime, while increasing productivity. We investigate two major predictive maintenance problems: remaining useful life prediction and capacity/state-of-health estimation. We adopt the data-driven approach where deep neural networks are used to develop prediction models thanks to their strength in learning highly nonlinear relations while minimizing human effort in extracting features. Feed-forward, recurrent, and long short-term memory neural networks are the structures being used in our research. We then train these neural networks using the MCSDCA-odLD and MCSDCA-udLD presented in Chapter 4. Numerical experiments show the virtue of the two methods in comparison with other baseline optimizers in deep learning. The prediction results closely match the true remaining useful life and capacity values.

Introduction

Today, maintenance is at the heart of every industrial operation to ensure companies' stability, efficiency, and productivity. The right choice of maintenance policy can make a huge difference in the age of smart industry, enhancing significantly the competitiveness of companies. Going through a long history, maintenance has evolved from its original form of fixing assets once broken down to integrating "intelligence" to predict the time that failures occur, hence optimizing the maintenance schedule. Generally, there are three main themes of maintenance: reactive, preventive, and predictive [START_REF] Lee | Big data analytics for predictive maintenance strategies[END_REF]. Reactive maintenance, also known as run-to-failure maintenance, is the classical approach which intervenes only when a machine exhibits abnormal behavior. The primal advantage of such a maintenance strategy is the lower beginning expenditures and manpower requirements. For simple devices such as cell phones that do not serve a critical function, it is sufficient to employ reactive maintenance. However, when it comes to sophisticated machinery systems such as industrial plants, breakdowns mean major production challenges and

Time series forecasting

A time series, denoted by {y 1 , y 2 , . . . , y k , . . .}, is a sequence of data points that show up in time order.

The data points are often temporally correlated to one another. Time series forecasting, in general, is the prediction of the time series values at some point in the future based on the data gathered up to the present time. There are several types of time series forecasting. To begin, it is preferable to utilize the time series' historical values to forecast its future value,

where s is the look-ahead timestamp we want to predict at (known as the forecast horizon), r is the number of look-back timestamps we want to use to produce the prediction, and f is a function to be chosen in order to make accurate predictions. Some research also considered making predictions using data observed from timestamps with unequal lengths [START_REF] Ramasso | Joint prediction of continuous and discrete states in time-series based on belief functions[END_REF][START_REF] Aladag | A new time invariant fuzzy time series forecasting method based on particle swarm optimization[END_REF]

where l 1 , l 2 , . . . , l r indicate varying intervals of time between observations.

When several exogenous time series {u 1 i } i∈N , {u 2 i } i∈N , . . . , {u T i } i∈N are available to aid the prediction, one has the following formulation

In some cases, the true value of the time series {y k } k∈N is unknown at the prediction time and only exogenous time series {u 1 i } i∈N , {u 2 i } i∈N , . . . , {u T i } i∈N are at disposal, the formula becomes

In this latter case, s can be 0: one uses exogenous time series to forecast the unknown value of the time series of interest at the current moment. Usually, the function f is chosen in a certain family of functions parameterized by w, {f w } w∈W . The problem then becomes choosing the best w that fits the above regression as well as possible.

In the context of predictive maintenance, {u 1 i } i∈N , {u 2 i } i∈N , . . . , {u T i } i∈N are often the time series collected by sensors, and the target time series {y k } k∈N is the variable of interest such as the remaining useful life or the state-of-health/capacity. Methods for time series forecasting can be classified into three categories: naïve forecasting methods, statistical forecasting methods, and machine learning forecasting methods [START_REF] Meisenbacher | Review of automated time series forecasting pipelines[END_REF].

Naïve forecasting methods As the name suggests, this approach is the simplest family of methods for forecasting. Regarding the averaging method, the forecasts of all future values are equal to the average historical data (see, e.g., [START_REF] Hyndman | Forecasting: principles and practice[END_REF]),

Another method is AutoRegressive integrated moving average (ARIMA), which combines the AR, MA and differencing. In time series analysis, differencing is a technique to make a time series stationary.

Given a time series {y t } t , the differenced time series {y ′ t } t is defined by

Differencing can remove trends, as illustrated in Figure 5.1.

Furthermore, one can apply differencing again to {y ′ t } t to obtain a twice-differenced series

and so on to obtain {y

t } t , etc. An ARIMA model takes the following form

where p, q are the orders of the autoregressive part and the moving average part, respectively, d is the degree of differencing used.

The mentioned methods mainly perform forecasting based on historical data of the same time series.

In case one has to predict based on multiple exogenous time series, the following linear regression model can be used,

where β 0 , β 1 , . . . , β T are the model parameters, and ϵ t is Gaussian noise. Given a sequence of training samples {x 1 , x 2 , . . . , x K } and a corresponding groundtruth label y, we want to build a prediction for y based on the last output of the RNN. To this end, we stack on top of o t a fully-connected layer with an output size compatible with y,

resulting the loss of L(y, ŷ). We can unroll the above recurrent structure to give rise to the analytical form of ŷ,

where h 0 is some starting hidden state, which is usually chosen to be a zero vector.

Given a training dataset consisting of a number of training sequences with their associated labels,

the optimization problem reads

) is the optimization variable.

Experimental setups

In this research, we focus on the optimization aspect of training deep neural networks. In other words, we want to study the learning capacity of the two proposed algorithms, MCSDCA-odLD and MCSDCA-udLD, in comparison with other benchmark optimizers in deep learning. When comparing optimizers, it's common to employ pre-existing network topologies that are known to have good overall (train, test) performance on the considered datasets. Then, with the same random initialization weight, all optimizers are utilized to train these specified networks. We follow the same spirit here, but instead of relying on pre-existing structures for the CMAPSS dataset, we first find satisfactory model configurations (number of neurons per layer, dropout probability [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF]). To that purpose, we use a random search to find the best standard fully-connected neural network and LSTM for each dataset, as well as certain other hyperparameters such as the RUL truncation level and the window length (of time series). On the acquired models, we then conduct a comparison study between MCSDCA, SGD, Adam, Adagrad, and RMSprop, with the training process stated below. Furthermore, in this experimental study, we found that MCSDCA-odLD is a good proxy for the MCSDCA as it performs very well. As the MCSDCA-odLD is already efficient, the MCSDCA-udLD, which is expected to accelerate the MCSDCA-odLD, does not show its advantage in this case. However, in the next experiment on battery capacity estimation when the MCSDCA-odLD is slow to converge, the MCSDCA-udLD significantly accelerates the learning speed.

Therefore, we preserve the MCSDCA-udLD for the next experiment, while the MCSDCA-odLD will be used as a good proxy for the MCSDCA in this experiment. Training procedure For the baseline algorithms (Adam, Adagrad, RMSprop, SGD), we design a staircase schedule for the learning rate: the initial learning rate is set as Adam(0.001), Adagrad(0.01), RMSprop(0.01), SGD(0.01) and the learning rate is dropped by a factor of 2 after each 10 epochs.

Model configurations

For the MCSDCA-odLD, motivated from [START_REF] Chaudhari | Entropy-sgd: Biasing gradient descent into wide valleys[END_REF], we set time 1/t = 10 -4 • 1.001 and ϵ = 10 -8 . One can also slowly decrease time as 1/t = 10 -4 • 1.001 k w.r.t. iteration k (known as "scoping" [START_REF] Chaudhari | Entropy-sgd: Biasing gradient descent into wide valleys[END_REF]).

However, in this work, we just set k = 1. Moreover, based on the non-asymptotic convergence analysis, the hyperparameters are chosen as follows. The number of Langevin steps is n k = n init + ⌊k λ ⌋ with n init = 20, λ = 0.1. We discard 10 samples from each Markov chain. Because 1/t shows up in both G and ∇H, it is conventional to set γ k = 1 t γk . We choose sequence {γ k } as γk = 10 -5 (k +1) 0.1 . The step size of the Langevin dynamics (4.20) is set at 0.001. For all algorithms, we set the maximum number of epochs to be 40, resulting in the backprop call budget of 40N , where N is number of training samples.

If an algorithm exhausts its computing budget, it will be terminated automatically. At each run, we randomly partition the training dataset into training (90%) and validation (10%), where the validation set is utilized for early stopping: if the validation loss does not improve after 4 consecutive epochs, the algorithm is stopped.

Experimental setups

We use standard RNN and LSTM with the following network configurations: RNN: input 6×y → rnn 256 → fc 1 LSTM: input 6×y → lstm 256 (tanh) → fc 1 where y = 15 if either charge or discharge data is used, while y = 30 if both charge and discharge phases are used.

In this experiment, we observed that the MCSDCA-odLD exhibits a slow convergence. On the contrary, the MCSDCA-udLD significantly speeds up the training process, which is exactly what we expect the MCSDCA-ulLD to behave. For example, we can see an impressive performance of the MCSDCA-udLD in comparison with the MCSDCA-odLD. Training procedure We have four batteries, B0005, B0006, B0007, and B0018, and we alternate between using three as the training set and one as the test set. This yielded four train-test pairings.

Moreover, we either use only the charge or discharge data, or both in our experiment. For each traintest pair, we execute all algorithms at the same random initialization, and the results are averaged over five random runs. A staircase schedule for the learning rate is designed for the baseline algorithms (Adam, Adagrad, RMSprop, SGD): the initial learning rate is set as Adam(0.001), Adagrad(0.01), RMSprop(0.01), SGD(0.01), and the learning rate is lowered by a factor of 2 after each 10 epochs.

For the MCSDCA-udLD, all parameters that are shared by the MCSDCA-odLD (ϵ, time t, number of Langevin steps, number of discarded samples, {γ k }) are set to the same values as in Subsection 5.4.2.

The MCSDCA-udLD has its own parameter, δ, which is now set to 0.1. We set the maximum number of training epochs for all methods to 1000 due to the modest size of the collected datasets, resulting in a backprop call budget of 1000N , where N is the number of training samples. If an algorithm exceeds its computing budget, it will be automatically terminated.