Züber Marine Michel

Antoine Paul

Laurent

Constance

Elisabeth Dominique

Vincent Marion

Keywords: Change-point detection, kernel methods, penalized least-squares, median heuristic

it is a shame we do not see each other more often. Sophie, you are the best roommate one could dream of.

Apart from jury members, a number of people kindly proof-read part of this

Je laisse Sisyphe au bas de la montagne ! On retrouve toujours son fardeau. Mais Sisyphe enseigne la fidélité supérieure qui nie les dieux et soulève les rochers. Lui aussi juge que tout est bien. Cet univers désormais sans maître ne lui paraît ni stérile ni futile. Chacun des grains de cette pierre, chaque éclat minéral de cette montagne pleine de nuit, à lui seul, forme un monde. La lutte elle-même vers les sommets suffit à remplir un coeur d'homme. Il faut imaginer Sisyphe heureux.

-Albert Camus, Le mythe de Sisyphe To my family.

First and foremost, I want to thank my advisors. Sylvain, you have been a tremendous mentor during these years. I value more than I can express the scientific dialogue that we had, this thesis would not exist without your ideas. I also realize how much I have learned as a researcher thanks to your careful reading of my drafts and your constant quest for improvement in any result. From the bottom of my heart, thank you.

Gérard, while your involvement was less scientific, it was nonetheless critical. I am forever indebted to you for the confidence you entrusted me with and your political insights. I must add that, as everyone else, I am quite impressed -and maybe slightly worried! -by your superhuman average response time to any email.

I also want to thank members of the thesis committee, whose presence made the defense possible. Stéphane, Bharath, I would like to express my sincere gratitude for accepting to read my manuscript during summer time, I am sure that was not an easy task in spite of all my efforts. Your insightful comments were much appreciated. Céline, Jean-Philippe, I am incredibly honored that you accepted to be part of my jury, thank you.

Inria has been an incredible research environment throughout the years, mainly thanks to the incredibly talented people who work here. Francis, thank you for making me part of the Sierra team and for letting me stay after Sylvain left! I am also grateful to all the members and former members of the Willow-Sierra consortium for making these years so pleasant. Jean-Baptiste, Loic, Piotr, Nicolas, Rémy, Vincent, Christophe, Guilhem, Théophile, Alexandre, Vadim, Sesh, Rémi, Pierre, Pascal, Gauthier, Robert, Edouard, Yana, Maxime, Julia, Anastasia, Antoine, Vincent, Guillaume, Damien, Nino, Gül: discussing everyday with you was a pleasure, whatever the subject.

At this point I want to address very special thanks to the amazing people who shared an office with me during this PhD. Rémi, I miss our countless controversial debates. I sincerely hope that you did not record anything I said. Aymeric and Nicolas, the end of this PhD would have been unbearable without the mixture of hard work, sense of humor and optimism you brought to the office everyday. I feel extremely lucky to have spent so much time in your company.

Of course I would not have been able to complete this thesis without the unwavering support of my friends. Thibault, Sarah, Andrea, Fanny, Pierre-Yves, QBi, Chatoune, Rémi, Hatch, Raph, Julie, Tonio, it is always a pleasure to spend some time with you. Adrien, Baptiste, David, Florent, Guillaume, Laurent, Matthieu, vii function is linear and the choice of the kernel. We show how the dimension jump heuristic can be a reasonable choice for the penalty constant in simulations. We also compute a key quantity depending on the kernel that appears in our theoretical results, and show how this quantity is linked to the performance of KCP in practice. Some of the computations that we present are novel, up to the best of our knowledge. Finally, we demonstrate experimentally the consistency results proved in Chapter 3. This chapter is based upon [START_REF] Garreau | Consistent change-point detection with kernels[END_REF] and additional experiments.

Chapter 5. This final chapter is devoted to the median heuristic choice, a popular tool to set the bandwidth of radial basis function kernels. For large sample size, we show that the median heuristic behaves approximately as the median of a distribution that we describe completely in the setting of kernel two-sample test and kernel changepoint detection. More precisely, we show that the median heuristic is asymptotically normal around this value. We illustrate these findings in a simple setting, where the underlying distributions are multivariate Gaussian. This chapter is based upon [START_REF] Garreau | Asymptotic normality of the median heuristic[END_REF].

Notations

We recall here some notations used throughout the manuscript. generic penalty function linear penalty, defined by Eq. (2.3) alternative penalty, defined by Eq. (2.4) defined by Eq. (3.11) quadratic part in the decomposition of 𝜓 𝜏 quadratic risk, defined by Eq. (2.12) set of all the segmentations of { 1, . . . , 𝑛 } squared median heuristic, defined by Eq. ( 5.2) a segmentation estimated segmentation true segmentation variance bound in Assumption 2.2 speed of convergence in Theorem 3.1 speed of convergence in Theorem 3.2 speed of convergence in Theorem 3.3 Chapter 1

, d (1) ∞ , d (2) ∞ , d (3)

Introduction

A large part of this manuscript is devoted to questions in regard to kernel changepoint detection, a method introduced by [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF]. The main idea underlying this procedure is to adapt a penalized least-squares change-point detection scheme to data belonging to a general set on which a positive semi-definite kernel is defined. Concepts from two distinct branches of statistics meet here: change-point detection and kernel methods. In this introduction, we aim to present both, with the goal of relating our work to the existing literature. A precise description of kernel changepoint detection is delayed to Chapter 2.

We begin with a description of the change-point problem in Section 1.1, setting some vocabulary extensively used throughout this manuscript. Before giving an overview of the literature pertaining to change-point detection in Section 1.3, we expose some real-world situations where this problem naturally arises in Section 1.2. We then briefly present the powerful machinery of kernel methods in Section 1.4.

The change-point problem

Change-point detection is a long-standing question in mathematical statistics, which has attracted a lot of attention since the 30s. Our goal in this section is not to present the vast literature associated with this problem in an exhaustive fashion, but rather to present the main ideas that shaped the field, and to relate them to this thesis.

In the study of time series, it is natural to assume stationarity, that is, time-shift invariance of the data probability distribution. Indeed, suppose that we are recording data coming from a natural phenomenon during a time period where there is no trend or shock in the background, then there is no reason for the distribution parameters to change. This assumption often fails in practice, where the environment does endure potentially large changes, and a more reasonable assumption is to consider that the observed phenomenon is stationary only on smaller time-units -see Fig. 1-1 for a typical situation. The goal of change-point detection is to recover these segments as accurately as possible.

If the signal we consider is multi-dimensional, we assume that changes in the distribution occur simultaneously in the different dimensions of the signal. It is not necessary that changes take place in all of them, though it should be clear that the problem becomes quite challenging if only a few coordinates undergo a change in a high-dimensional setting. We will not deal with the situation where each dimension is segmented differently, that is, joint segmentation, and we refer to [START_REF] Picard | Joint segmentation of multivariate gaussian processes using mixed linear models[END_REF] and references therein for an introduction.

Let us specify some vocabulary and concepts that are going to be used throughout this thesis.

Off-line vs on-line setting. When observations 𝑋 1 , . . . , 𝑋 𝑛 are obtained one at a time, we say that the setting is on-line, or sequential. The goal is to detect changes as quickly as possible, while keeping the number of false alarms as low as possible. We talk about off-line or a posteriori detection of changes when the data is obtained all at once. In this case there is no need for real-time processing of the data, and one can take advantage of this additional computation time. Note that on-line procedures can be applied in the off-line setting, just by running through the data-points as if they were being observed one by one. Less obvious is the possibility to use an off-line procedure in the on-line setting, handling batches of newly observed data-points with the off-line procedure. However, the batches' size is then a strict lower bound to the quickest detection time. Let us emphasize that this thesis is essentially concerned with off-line change-point detection. Nevertheless, we briefly account for the main ideas and essential results pertaining to on-line change-point detection in Section 1.3.1.

Single vs multiple change-point detection.

There is quite a difference between detecting a single change-point and multiple change-points in the off-line setting. It is important to understand that the second problem is much harder, since the number of possible outcomes for the procedure jumps from 𝑛 -1 to 2 𝑛-1 if the number of change-points is unknown 1 , where 𝑛 is the number of observations. This thesis is mainly concerned with the second problem. Note that methods designed for detecting multiple change-points can obviously be applied to the detection of a single changepoint. The converse is also possible, even though there is no guarantee it will yield the desired results. For instance, one can proceed as follows: (i) search the entire dataset for a change-point, (ii) if a change-point is found, separate the dataset in two parts (before and after), and (iii) iterate. This generic procedure is called binary segmentation, see [START_REF] Vostrikova | Detecting disorder in multidimensional random process[END_REF].

Parametric vs non-parametric. It is common to make parametric assumptions, i.e., to assume that the distribution of the observations belongs to a family of distributions that can be described using a finite number of parameters. Thus the changes occur in one (or more) of the parameters describing the distribution. These assumptions are important both for the design and the analysis of change-point procedures. ) ways for a signal of size 𝑛 to be segmented in 𝑘 segments, hence a total of ∑ 𝑛-1 𝑘=1 ( 𝑛-1 𝑘-1

) = 2 𝑛-1 possibilities.

We are more interested in non-parametric settings, where little is known about the underlying distributions.

In brief, we are interested in the off-line detection of multiple changes in potentially multi-dimensional time-series with simultaneous changes, without parametric assumptions. ) whose mean abruptly changes from 1 to -1 and then to 0.5 at position 500 and 750 (indicated by broken vertical red lines) and whose variance stays constant (𝜎 = 0.5). The goal of change-point detection is to recover these positions from the observations. We are here in an off-line setting, with multiple change-points, in a parametric model.

Examples

In this section we present some real situations where change-point problems arise. The historical example for the use of change-point detection comes from process quality control [START_REF] Shewhart | Economic control of quality of manufactured product[END_REF], that is, the monitoring of industrial processes which can be either in control or out of control. The goal is then the quickest detection of anomalous behaviors, with the least false-alarms (see Basseville and Nikiforov [1993, Chapter 1]). It is essentially an on-line problematic, that we do not develop here. As we have seen in Section 1.1, this thesis is focused on the off-line detection of changes, so we present in more details some examples coming from fields where off-line changepoint detection is relevant.

Array Comparative Genomic Hybridization. During cellular division, the duplication of the genome can go astray and a large number of base pairs can be deleted or copied more than once. This phenomenon is called copy-number variation and occurs frequently in the life of a cancerous cell. Comparative Genomic Hybridization (CGH) allows to estimate the copy-number variations in a particular genome with respect to normal. More precisely, for each locus of the genome with a precision of 5 to 10Mb,2 CGH provides an estimate of the ratio between the copy-number of a test subject DNA and a reference. Note that before the development of CGH in the early 90s [START_REF] Kallioniemi | Comparative genomic hybridization for molecular cytogenetic analysis of solid tumors[END_REF], it was extremely costly to obtain this global information.

For the sake of completeness, let us explain briefly how CGH data is obtained: the DNA of both samples is first extracted and colored with a fluorescent marker, generally green for the test sample and red for the reference. Both samples are then heated, which makes the DNA strands of the chromosomes separate. Next, they are dropped down on a microchip that contains ordered single DNA strands of the same genome as the reference. Locally, if there are more copies of the test DNA, it associates preferentially with the control DNA and green prevails -conversely, if there are less copies of the test DNA, red prevails. The final data is obtained by measuring the fluorescence intensities.

Array CGH (aCGH) is a technical refinement of CGH that allows to work at a much finer scale (5 to 10kb) and is now widely used instead of CGH. As CGH, it is not exempt from noise, which comes from experimental measurement imprecision and the log-ratio is generally assumed to be Gaussian. An example of aCGH data is depicted in Fig. 12. Figure 1-2 -Logarithm of the copy-number variation as a function of the position in the genome of cancer cell T47D [data from [START_REF] Snijders | Assembly of microarrays for genome-wide measurement of dna copy number[END_REF]. Positive (resp. negative) values correspond to genome positions where the copy number is higher (resp. lower) that normal. As can be seen on this example, the copy-number variation can be modeled well by a noisy piecewise constant function.

Interestingly, several types of tumors show a consistent pattern of such genetic aberrations, i.e., large connected portions of the genome are consistently over or under-replicated -in fact CGH was invented precisely to study the genetic anomalies of tumor cells. Hence, such patterns can serve as a signature of the tumor, and identifying precisely the genome segments on which the copy-number is consistently higher or lower than normal can be used to diagnose cancer [START_REF] Bejjani | Application of array-based comparative genomic hybridization to clinical diagnostics[END_REF]. When there are multiple tumors, it is also possible to use these signatures for identifying which tumor is metastasizing [START_REF] Weiss | Comparative genomic hybridisation as a supportive tool in diagnostic pathology[END_REF]. These patterns are collections of geographic segments of the genome, and both during the identification and diagnosis stage, we are faced with an off-line multiple change-point detection problem.

Modeling of financial time series. Proposing relevant models for financial time series, in particular share prices, is a subject of the utmost importance both in quantitative finance and econometrics. For a given stock, portfolio, or stock market index, investors are primarily concerned with the return, that is, the net gain or loss generated by an investment strategy. Let us focus on a specific financial asset, write 𝑠 𝑡 the value of this asset at time 𝑡 ∈ Z, and denote by 𝑋 𝑡 the logarithmic return of this asset, that is, 𝑋 𝑡 := log (𝑠 𝑡+1 /𝑠 𝑡 ) (see Fig. 1-3 for a plot of log-returns on a real dataset). It is common to assume that 𝑋 is a generalized autoregressive model (GARCH). There is tremendous literature on GARCH processes, we refer to [START_REF] Tsay | Analysis of financial time series[END_REF] for an introduction. It falls out of the scope of this thesis to present this topic rigorously. To set ideas straight, let us just say that 𝑋 can be decomposed as

{ 𝑋 𝑡 = 𝜎 𝑡 𝑍 𝑡 𝜎 2 𝑡 = 𝛼 0 + ∑ 𝑝 𝑖=1 𝛼 𝑖 𝑋 2 𝑡-𝑖 + ∑ 𝑞 𝑖=1 𝛽 𝑖 𝜎 2 𝑡-𝑖 (1.1)
where the 𝑍 𝑡 s are i.i.d. standard Gaussian random variables, 𝑝 and 𝑞 are positive integers, 𝛼 0 > 0, and

(𝛼 𝑖 ) 1≤𝑖≤𝑝 , (𝛽 𝑖 ) 1≤𝑖≤𝑞 ∈ R + .
There is empirical evidence to show that the above model is satisfying for short periods of time. Furthermore, it is practical to estimate the parameters of model (1.1) when it is assumed that the observations constitute a stationary sequence of random variables (see [START_REF] Straumann | Estimation in Conditionnally Heteroscedastic Time Series Models[END_REF] for a comprehensive monograph on the estimation of GARCH models). But for longer time periods, this assumption is not realistic, in particular due to abrupt changes in the variance process during times of economic turmoil [START_REF] Mikosch | Changes of structure in financial time series and the garch model[END_REF]. A simple way out is to consider that the observations are only stationary on short periods of time and to update the parameters estimates on each segment, hence the need for an off-line change-point detection procedure.

Let us mention that some applications in finance are also focused on the realtime problem, where the goal is to detect the apparition of structural breaks in "live" financial time series [START_REF] Pepelyshev | Real-time financial surveillance via quickest change-point detection methods[END_REF].

Video processing. In a movie, a shot is the longest continuous sequence that originates from a single camera, and shot detection is the problem of detecting the beginning and ending of each shot. The goal is to produce small homogeneous movie parts, that can be easily used for indexing or more involved movie processing tasks.

Even though this information is sometimes available as meta-data in recent standards such as MPEG-7, there is still an ongoing research effort to develop reliable techniques for shot detection; we refer to [START_REF] Cotsaces | Video shot boundary detection and condensed representation: a review[END_REF] for a review. Once the movie transformed into a vector-valued time series, shot detection is essentially an off-line, non-parametric, multiple change-point detection problem. Nevertheless, let us emphasize two major differences with the setting that we introduced in Section 1.1: (i) there is no definitive feature extraction procedure for this task, and (ii) not all changes are abrupt -if a clear transition between shots (a cut) is most frequently employed, a gradual transition is also a possibility (a dissolve). An example of clear transitions between shots is pictured in Fig. 1234.

An additional difficulty is the absence of prior information regarding the number of change-points. Indeed, if some movies contain hundreds of distinct shots, others have very few. 3 A relevant task in the latter case is scene detection, that is, segmenting the video according to the actions taking place. When appropriate features are used, this is once again an off-line multiple change-point problem [START_REF] Allen | Change-point detection methods for body-worn video[END_REF].

In all these previous examples, change-point detection can be performed "by hand." We think that automatic change-point detection methods can help the practitioner in many respects, as is the case with most statistical methods:

-to increase the reliability of the data analysis;

-to decrease the risk of unexplainable errors;

-to increase the speed of data processing by several orders of magnitude, allowing to handle much more data;

-to free some time for other tasks of more interest.

History

In this section, we present some important algorithms both for on-line and off-line change-point detection.

Sequential change-point detection

Historically, change-point problems have come from the sequential point of view. We present here some of the most important ideas in the on-line setting. This account is far from exhaustive. First, we omit Bayesian procedures, which are completely out of the scope of this manuscript. See [START_REF] Fearnhead | On-line inference for multiple changepoint problems[END_REF] and references therein for an introduction to this point of view on the on-line change-point problem. Second, we do not mention sequential testing procedures since we shall encounter them shortly hereafter in the context of off-line change-point detection. We refer to [START_REF] Basseville | Detection of abrupt changes: theory and application, Prentice Hall Information and System Sciences Series[END_REF] for an extensive overview of on-line procedures and [START_REF] Tartakovsky | Sequential Analysis: Hypothesis Testing and Changepoint Detection[END_REF] regarding sequential testing.

In the following, 𝑋 1 , . . . , 𝑋 𝑛 are sequential observations of a real-valued random process that undergoes a change in the mean and, for any 𝑖 ∈ { 1, . . . , 𝑛 } , we define

𝜇 ⋆ 𝑖 := E [𝑋 𝑖 ] .
Control charts. The most immediate idea for detecting changes in signal is certainly to set a threshold and to decide that there is a change if a certain statistic crosses the threshold. How does one choose the threshold value and the statistic?

The first rigorous attempt to answer this question was made by [START_REF] Shewhart | Economic control of quality of manufactured product[END_REF].

Suppose for now that the true mean of the observations 𝜇 ⋆ is known, as well as the variance 𝜎 2 . Choose a batch size 𝑁 and set

X(𝐾) = 1 𝑁 𝑁 𝐾 ∑ 𝑖=𝑁 (𝐾-1)+1 𝑋 𝑖 .
Then the proposed algorithm is to detect a change as soon as

⃒ ⃒ X(𝐾) -𝜇 ⋆ ⃒ ⃒ > 𝜅 𝜎 √ 𝑁 ,
where 𝜅 is a constant. Keep in mind that in the context of quality control in which control charts were introduced, 𝜇 ⋆ is a known value prescribed by the industrial necessities. Moreover, 𝜎 2 can be easily estimated if it is unknown, for instance by setting

̂ 𝜎 2 := 1 𝑚 -1 𝑚 ∑ 𝑖=1 ( 𝑋 𝑖 - X) 2 ,
when there is a strong belief that no change-point occurred in the first 𝑚 observations. Concentration of measure can help choosing 𝜅 depending on the hypothesis satisfied by the observations -for instance, a refinement of Chebyshev's inequality [START_REF] Meidell | Note sur quelques inégalités et formules d'approximation[END_REF][START_REF] Camp | A new generalization of Tchebycheff's statistical inequality[END_REF] guarantees that, for unimodal distributions, |𝑋 𝑖 -𝜇| > 3𝜎 with probability smaller than 0.05 hence the 𝜅 = 3 chosen by Shewhart.

While numerous versions of Shewhart's control chart exist, they all share the same common idea: as the data is released, compute a statistic on each batch and take action if it crosses predefined thresholds [START_REF] Page | Continuous inspection schemes[END_REF][START_REF] Lai | Control charts based on weighted sums[END_REF].

CUSUM.

It was not before the 50s that optimal procedures were developed, after the seminal work of [START_REF] Wald | Sequential analysis[END_REF]. The breakthrough of the CUSUM algorithm [START_REF] Page | Continuous inspection schemes[END_REF] is to take action not if the statistic crosses a threshold, but if it is far away from the historic minimum. In fact, this amounts to a control chart comparing against an adaptive threshold. Let us be more specific: as in the previous paragraph, we collect samples of size 𝑁 . We then assign a score 𝑌 (𝐾) to the 𝐾-th sample, set 𝑆 𝑚 = ∑ 𝑚 𝐾=1 𝑌 (𝐾), and take action if

𝑆 𝑚 -min 0≤𝑖<𝑚 𝑆 𝑖 ≥ ℎ , (1.2)
where ℎ is a positive constant. Even though it was not explicit in [START_REF] Page | Continuous inspection schemes[END_REF] how to choose the scores 𝑌 (𝐾), it is common to use the log-likelihood ratio. That is, if we assume that the 𝑋 𝑖 are drawn according to a distribution 𝑝 𝜃 with 𝜃 ∈ Θ some parameter, equal to 𝜃 0 before the break and 𝜃 1 after, we set

𝑌 (𝐾) = 𝑁 𝐾 ∑ 𝑖=𝑁 (𝐾-1)+1 log 𝑝 𝜃 1 (𝑋 𝑖 ) 𝑝 𝜃 0 (𝑋 𝑖 ) .
Of course, Eq. ( 1.2) only allows to detect positive changes in the mean of the score. Using two CUSUM algorithms together, for instance, is a way to fix this problem [START_REF] Page | Continuous inspection schemes[END_REF]. This line of work was later extended in a series of papers by [START_REF] Shiryaev | The problem of the most rapid detection of a disturbance in a stationary process[END_REF][START_REF] Shiryaev | On optimum methods in quickest detection problems[END_REF][START_REF] Shiryaev | Some exact formulas in a "disorder" problem[END_REF]. We refer to [START_REF] Lorden | Procedures for reacting to a change in distribution[END_REF]; [START_REF] Moustakides | Optimal stopping times for detecting changes in distributions[END_REF] for optimality results regarding this algorithm.

Filtered derivative.

The key idea behind the filtered derivative algorithm [START_REF] Basseville | Edge detection using sequential methods for change in level-Part II: Sequential detection of change in mean[END_REF]] is simple: if there is no noise, then changes in the mean translate into sharp jumps in the absolute value of the discrete derivatives of the signal. More precisely, define the moving average

𝑔 𝑘 = 𝑁 -1 ∑ 𝑖=0 𝛾 𝑖 log 𝑝 𝜃 1 (𝑋 𝑘-𝑖 ) 𝑝 𝜃 0 (𝑋 𝑘-𝑖 ) ,
where the 𝛾 𝑖 are positive weights. Then instead of deciding for a change whenever 𝑔 𝑘 ≥ ℎ (a finite moving average control chart), we consider the discrete derivative ∇𝑔 𝑘 := 𝑔 𝑘 -𝑔 𝑘-1 and consider that there is a change-point if a sufficient number of discrete derivatives are above a threshold ℎ, i.e.,

𝑁 -1 ∑ 𝑖=0 1 ∇𝑔 𝑘-𝑖 ≥ℎ ≥ 𝜂 .
The parameter 𝜂 is typically small, e.g., 𝜂 = 2. Similar ideas are used for edgedetection algorithm in image processing [START_REF] Roberts | Machine Perception of Three Dimensional Solids[END_REF].

We now turn to one of the main focuses of this manuscript, off-line change-point detection.

Off-line change-point detection

Even though on-line change-point methods predate off-line methods by more than fifteen years, the literature relating to the latter is no less spread-out. As in the online setting, we do not pretend to an exhaustive treatment of the off-line change-point literature and we refer to Brodsky and Darkhovsky [2013, Chapter 2] for a review of non-parametric off-line change-point detection methods. Rather, we want to present with a reasonable amount of detail two principles paramount to most of the off-line methods.

The first idea is to interpret the change-point detection problem as a statistical hypothesis test, deciding for a change-point if it is statistically significant. Another possibility is to cast the change-point problem as an estimation problem, where the change-points are parameters from the model that we wish to estimate. Among the estimation methods, we take a special interest in estimates obtained via the minimization of a least-squares criterion given that our object of interest, kernel change-point detection, is a natural extension of this line of thought.

Before presenting off-line methods in more detail, let us define a simplified a posteriori change-point setting. For any integer 𝑛, we call segmentation integers 1 ≤ 𝐷 ≤ 𝑛 and 𝜏 0 := 0 < 𝜏 1 < • • • < 𝜏 𝐷-1 < 𝜏 𝐷 = 𝑛. We call segments the sets {1, . . . , 𝜏 1 }, {𝜏 1 + 1, . . . , 𝜏 2 },. . . ,{𝜏 𝐷-1 + 1, . . . , 𝑛}. Given observations 𝑋 1 , . . . , 𝑋 𝑛 , there is always a segmentation 𝜏 ⋆ with 𝐷 ⋆ segments such that the distribution of 𝑋 𝑖 is constant on the segments, but distinct for consecutive segments. In particular, the mean 𝜇 ⋆ 𝑖 = E [𝑋 𝑖 ] is constant on these segments. This model is extended to a more general setting in Chapter 2. We may also make a parametric assumption on the observations, that is, assume that the 𝑋 𝑖 have a density 𝑓 𝜃 , with 𝜃 = 𝜃 ℓ on segment ℓ + 1.

Hypothesis testing

A first idea for detecting a single change-point is to cast this problem as a hypothesis test. More specifically, one can test the null hypothesis

ℒ (𝑋 1 ) = ℒ (𝑋 2 ) = • • • = ℒ (𝑋 𝑛 ) , (𝐻 0 )
versus the alternative hypotheses

∃𝑡 ∈ { 1, . . . , 𝑛 -1 } , ℒ (𝑋 1 ) = • • • = ℒ (𝑋 𝑡 ) ̸ = ℒ (𝑋 𝑡+1 ) = • • • = ℒ (𝑋 𝑛 ) . (𝐻 alt )
Note that (𝐻 alt ) is generally decomposed into the union of 𝑛 -1 hypotheses

ℒ (𝑋 1 ) = • • • = ℒ (𝑋 𝑡 ) ̸ = ℒ (𝑋 𝑡+1 ) = • • • = ℒ (𝑋 𝑛 ) . (𝐻 𝑡 )
In this section, we present some statistics for testing (𝐻 0 ) versus (𝐻 alt ) existing in the literature, or variations thereof, in particular a one-sided version of (𝐻 alt ) when ℒ (𝑋 𝑖 ) boils down to a single real parameter. For an exhaustive account of hypothesis testing in off-line change-point detection, we refer to [START_REF] Deshayes | Off-line statistical analysis of change-point models using non parametric and likelihood methods[END_REF] and to [START_REF] James | Tests for a change-point[END_REF] for a comparison of the test powers -which we will not discuss below.

CUSUM and extensions. Let us restrict ourselves to the parametric framework defined in Section 1.3.2, that is, ℒ (𝑋 𝑖 ) ∼ 𝑓 𝜃 for some 𝜃 ∈ Θ. Keep in mind that 𝜃 can depend on 𝑖 since the 𝑋 𝑖 s are not identically distributed. Let us start with a very simple situation where the initial and final parameters are known, say 𝜃 0 and 𝜃 1 . One of the simplest ideas for testing is then to build a test statistic from the likelihood function associated with the observations 𝑋 1 , . . . , 𝑋 𝑛 . In [START_REF] Page | On problems in which a change in a parameter occurs at an unknown point[END_REF], the first offline method that we know of, it is proposed to choose the (𝐻 𝑡 ) that maximizes the likelihood function of the hypothesis, that is, 𝜏 1 is estimated by

 𝜏 1 ∈ arg max 1≤𝑡≤𝑛 { 𝑡 ∑ 𝑗=1 log 𝑓 𝜃 0 (𝑥 𝑗 ) + 𝑛 ∑ 𝑗=𝑡+1 log 𝑓 𝜃 1 (𝑥 𝑗 )
} .

In the case of a one-sided change in the mean of Gaussian observations with a known variance 𝜎 2 , [START_REF] Page | On problems in which a change in a parameter occurs at an unknown point[END_REF] recovers the cumulative sum from the CUSUM algorithm introduced at the beginning of Section 1.3.2. More precisely, we define

𝑆 𝑡 := 𝑡 ∑ 𝑗=1 (𝑋 𝑗 -𝜃 0 + 𝛿𝜎) , then reject (𝐻 0 ) if 𝑆 𝑛 -max 𝑡<𝑛 𝑆 𝑡 < -ℎ,
where ℎ is a positive number, and in this case choose ̂ 𝜏 1 as the first 𝑡 such that 𝑆 𝑛max 𝑡<𝑛 𝑆 𝑡 ≥ 0. This line of ideas is further studied in a series of three papers by [START_REF] Hinkley | Inference about the intersection in two-phase regression[END_REF][START_REF] Hinkley | Inference about the change-point in a sequence of random variables[END_REF][START_REF] Hinkley | Inference about the change-point from cumulative sum tests[END_REF], which obtain the asymptotic distribution of the estimate under Gaussian assumption.

If the initial parameter 𝜃 0 is not known -but the variance 𝜎 2 is still known -, Sen and Srivastava [1975a] following [START_REF] Gardner | On detecting changes in the mean of normal variates[END_REF] propose the statistic

𝑈 ⋆ = 1 𝑛 2 𝑛-1 ∑ 𝑖=1 ( 𝑛-1 ∑ 𝑗=𝑖 (𝑋 𝑗+1 -X) ) 2 ,
and obtain the exact cumulative distribution function of 𝑈 ⋆ , leading to power computations. Sen and Srivastava [1975a] claim that the power obtained is better for the test built with 𝑈 ⋆ than for the maximum likelihood statistic test. This line of work was extended for unknown variance in Sen and Srivastava [1975b], following [START_REF] Chernoff | Estimating the current mean of a normal distribution which is subjected to changes in time[END_REF], proposing among others the statistic

𝑃 1 = 𝑈/𝑉 1/2 1 , where 𝑈 = 𝑛-1 ∑ 𝑖=1 𝑖(𝑋 𝑖+1 -X)
and

𝑉 1 = 1 2(𝑛 -1) 𝑛 ∑ 𝑖=1 (𝑋 𝑖+1 -𝑋 𝑖 ) 2 .
The limiting distribution is not obtained in closed-form, but simulations suggest that 𝑃 1 has superior power.

Likelihood ratio test. Another possibility as a test statistic is called the likelihood ratio. The idea is to use as a statistic (the log of) the ratio between the likelihood under (𝐻 alt ) and the likelihood under (𝐻 0 ). Suppose that 𝜎 2 is known, and set

𝑄 𝑘 = 𝑘 ∑ 𝑗=1 ( 𝑋 𝑗 -X1:𝑘 ) 2 + 𝑛 ∑ 𝑗=𝑘+1 ( 𝑋 𝑗 -X(𝑘+1):𝑛 ) 2 ,
where X𝑎:𝑏 denotes the sample mean of the 𝑏 -𝑎 observations 𝑋 𝑎+1 , . . . , 𝑋 𝑏 . Then the likelihood ratio test statistic for Gaussian observations of unknown mean before and after the change is given by 𝑄 𝑛 -𝑄 𝑘 ⋆ ; whenever 𝜎 2 is unknown, this test statistic becomes equivalent to (𝑄 𝑛 -𝑄 𝑘 ⋆ ) /𝑄 𝑘 ⋆ [START_REF] Hinkley | Inference about the change-point in a sequence of random variables[END_REF][START_REF] Hawkins | Testing a sequence of observations for a shift in location[END_REF]. The null distribution is given by [START_REF] Worsley | On the likelihood ratio test for a shift in location of normal populations[END_REF] when the variance is unknown, and this statistic was also adapted by [START_REF] Worsley | Confidence regions and tests for a change-point in a sequence of exponential family random variables[END_REF] in the exponential setting. It was later generalized in the multi-dimensional setting [START_REF] Srivastava | Likelihood ratio tests for a change in the multivariate normal mean[END_REF][START_REF] Arias-Castro | Detection of an anomalous cluster in a network[END_REF]. We will encounter 𝑄 𝑘 again, or rather a generalization of 𝑄 𝑘 to the multiple change-point setting, later in this section.

Estimation procedures

In this section, we turn to an estimation-formulated version of the change-point problem. Rather than testing the possibility for each 𝑖 to be a change-point, such methods aim to propose an estimator ̂ 𝜏 𝑛 = ̂ 𝜏 𝑛 (𝑋 1 , . . . , 𝑋 𝑛 ) of the true segmentation 𝜏 ⋆ . Assessing the quality of such an estimator is one of the central themes of Chapter 2, we thus recall some of the theoretical results associated with the methods presented in this section. Before being able to present these results, we want to be more precise on the meaning of quality in this context.

A first definition of consistency. As is often the case in statistics, a question of crucial importance to the practitioner is the adequacy between the estimator ̂ 𝜏 𝑛 and 𝜏 ⋆ , especially when the sample size 𝑛 grows to infinity. We call consistency the asymptotic adequacy between ̂ 𝜏 𝑛 and 𝜏 ⋆ . In general, consistency results take the form with high probability, d (̂ 𝜏 𝑛 , 𝜏 ⋆ ) → 0 , where d (•, •) is a measure of similarity between segmentations -see Section 3.2.2 for a more involved treatment of this notion. Quite often, it is possible to give a quantitative version of the previous display, that is,

with high probability, d (̂ 𝜏 𝑛 , 𝜏 ⋆ ) ≤ 𝑟 𝑛 ,
with 𝑟 𝑛 → 0. We call 𝑟 𝑛 a rate of convergence. We will talk about almost sure consistency if, in the above statements, "with high probability" is replaced by "almost surely".

In the on-line setting, the meaning of 𝑛 → ∞ is rather obvious: the algorithm collects more and more data sequentially. However, in the off-line setting, there are several ways to grow the sample size to infinity. The simplest asymptotic setting consists in observing for each 𝑛 a sequence 𝑋 𝑛,1 , . . . , 𝑋 𝑛,𝑛 of random variables, with the distribution of the 𝑋 𝑛,𝑖 being constant on the segments {1, . . . , 𝜏 ⋆ 𝑛,1 }, {𝜏 ⋆ 𝑛,1 + 1, . . . , 𝜏 ⋆ 𝑛,2 },. . . ,{𝜏 ⋆ 𝑛,𝐷 ⋆ -1 + 1, . . . , 𝑛}, and the segments depending on 𝑛 in the following fashion:

∃𝛼 1 , . . . , 𝛼 𝐷 ⋆ -1 ∈ (0, 1), ∀1 ≤ 𝑖 ≤ 𝐷 ⋆ -1, 𝜏 ⋆ 𝑛,𝑖 𝑛 -→ 𝑛→∞ 𝛼 𝑖 . (1.3)
We will often drop the 𝑛 subscripts whenever the dependency in 𝑛 is clear. An illustration of the asymptotic setting described here is given in Fig. 12345. Note that, in the setting (1.3), normalized segment sizes are bounded away from 0. In this case, it is known that the best possible rate achievable is 1/𝑛 [START_REF] Korostelev | On minimax estimation of a discontinuous signal[END_REF][START_REF] Korostelev | Minimax theory of image reconstruction[END_REF]. Namely,

sup 1≤𝑖≤𝑛 ⃒ ⃒ ̂ 𝜏 𝑛,𝑖 -𝜏 ⋆ 𝑛,𝑖 ⃒ ⃒ = O P ( 1 𝑛 ) .
Whenever this assumption is not satisfied, this rate degrades to log(𝑛)/𝑛 [START_REF] Brunel | Convex set detection[END_REF]. Maximum likelihood. Suppose that we are in a parametric setting and that we know the true number of change-points 𝐷 ⋆ . It is then possible to generalize the approach of [START_REF] Hinkley | Inference about the change-point in a sequence of random variables[END_REF] and to write down the maximum likelihood estimator of 𝜏 ⋆ as

̂ 𝜏 ∈ arg max 1≤𝜏 1 <•••<𝜏 𝐷-1 <𝑛 𝜃 1 ,...,𝜃 𝐷 ⋆ ∈Θ { 𝐷 ⋆ ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 log 𝑓 𝜃 ℓ (𝑥 𝑖 ) } . (1.4)
It is shown to be consistent in probability in [START_REF] He | Asymptotic properties of maximum likelihood estimators in models with multiple change points[END_REF] under a compactness hypothesis and technical assumptions on the behavior of the log-likelihood function.

Least-squares. Let us now assume furthermore that the observations are Gaussian with known variance 𝜎 2 , that is,

𝑓 𝜃 (𝑡) = 1 𝜎 √ 2𝜋 exp ( - (𝜃 -𝑡) 2 2𝜎 2
) ,

with 𝜃 ∈ R. Then simple algebra shows that (1.4) becomes

 𝜏 (𝐷) ∈ arg min 𝜏 s.t. 𝐷𝜏 =𝐷 { ̂ ℛ 𝑛 (𝜏 ) } , (1.5)
with

̂ ℛ 𝑛 (𝜏 ) := 1 𝑛 𝐷 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ( 𝑋 𝑖 -X𝜏 ℓ-1 :𝜏 ℓ ) 2 .
We call ̂ ℛ 𝑛 (𝜏 ) the sum of squares criterion or least-squares criterion, which is also an estimate of the variance assuming that the true segmentation is 𝜏 . It is very convenient to use the least-squares criterion rather than the likelihood function, since we do not need to know the distribution of the observations. [START_REF] Fisher | On grouping for maximum homogeneity[END_REF] is the first to apply the least-squares criterion for a change-point problem to the best of our knowledge -note that his approach does not come from likelihood maximization but rather from variance minimization. [START_REF] Yao | Least-squares estimation of a step function[END_REF] prove that ̂ 𝜏 (𝐷) is consistent in probability in the asymptotic setting (1.3) and under mild assumptions -namely the continuity of the cumulative distribution function of the observations and a moment hypothesis. These assumptions are weakened further in [START_REF] Bai | Estimating and testing linear models with multiple structural changes[END_REF] and the minimax convergence rate of 1/𝑛 is obtained. The least-squares estimation procedure (1.5) was also shown to be consistent in the case of dependent processes (ARMA) with a single change-point in [START_REF] Bai | Least squares estimation of a shift in linear processes[END_REF], a work later extended for weak dependent disturbance processes (mixingales) by [START_REF] Bai | Estimating and testing linear models with multiple structural changes[END_REF]. Regarding multiple break-points, [START_REF] Lavielle | Detection of multiple changes in a sequence of dependent variables[END_REF]; [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] show the consistency of the least-squares estimate when 𝐷 ⋆ is known for a large class of dependent processes.

More precisely, define 𝜀 𝑖 := 𝑋 𝑖 -𝜇 ⋆ 𝑖 and set 𝑆 𝑎:𝑏 := ∑ 𝑏 𝑖=𝑎 𝜀 𝑖 , then the main assumption of [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] on the dependency structure of the noise 𝜀 is, for some 𝜑 > 0,

∃𝐶 > 0, ∀1 ≤ 𝑖, 𝑗 ≤ 𝑛, E [ 𝑆 2 𝑖:𝑗 ] ≤ 𝐶 |𝑗 -𝑖 + 1| 𝜑 . (𝐻(𝜑))
This assumption is satisfied 𝜑 = 1 for stationary processes such that the autocovariance function 𝛾(𝑠) = E [𝜀 𝑡+𝑠 𝜀 𝑡 ] satisfies ∑ 𝑠≥0 |𝛾(𝑠)| < ∞, and a variety of linear processes, e.g., any ARMA process. Assuming (𝐻(𝜑)) with 𝜑 ∈ [1, 2), ̂ 𝜏 (𝐷 ⋆ ) is consistent with convergence rate 𝑛 𝜑-2 [Lavielle and Moulines, 2000].

It is interesting to see that, at first sight, the minimization problem (1.5) seems rather daunting. Indeed, as we noticed before, the total number of segmentations with 𝐷 segments is

( 𝑛-1 𝐷-1
) -too large a number for optimizing directly. But it turns out that (1.5) can be solved exactly in O (𝐷𝑛 2 ) thanks to dynamic programming [START_REF] Bellman | On the approximation of curves by line segments using dynamic programming[END_REF]]. We will discuss further this algorithmic question in Section 2.4.

Unknown number of change-points. Whenever the number of change-points is not known, the problem becomes far more compelling. Indeed, minimizing directly (1.5) without constraints on 𝐷 𝜏 systematically outputs the segmentation consisting of 𝑛 segments of unit size, which is definitely not insightful: the number of segments has to be chosen in another way. The idea of [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF] is to consider the choice of  𝐷 as a model selection problem and to choose the number of change-points according to Schwartz criterion [START_REF] Schwarz | Estimating the dimension of a model[END_REF]. More precisely, given a ̂ 𝜏 (𝐷) that minimizes  ℛ 𝑛 (𝜏 ) for each 𝐷, [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF] then chooses

 𝐷 Yao ∈ arg min 1≤𝐷≤𝐷max { 𝑛 2 log ̂ ℛ 𝑛 (̂ 𝜏 (𝐷)) + 𝐷 log(𝑛) } ,
where 𝐷 max is a user-defined upper bound on the number of segments, and set

 𝜏 Yao := ̂ 𝜏 ( ̂ 𝐷 Yao )
. For homoscedastic 4 independent Gaussian observations and under Assumption (1.3), [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF] shows that ̂ 𝜏 Yao is consistent in probability. This result was extended for identically distributed error terms 𝜀 𝑖 by [START_REF] Yao | Least-squares estimation of a step function[END_REF], with however a more restrictive condition on the growth of 𝛽 𝑛 . More precisely, let us assume, in addition to Assumption (1.3), that (i) the cumulative distribution function of the 𝜀 𝑖 is continuous, (ii) E [𝜀 2𝑚 𝑖 ] < +∞ with 𝑚 ≥ 3. Let us also define

 𝐷 YAu ∈ arg min 1≤𝐷≤𝐷max { 𝑛 log ̂ ℛ 𝑛 (̂ 𝜏 (𝐷)) + 𝐷𝛽 𝑛 } , (1.6)
with 𝛽 𝑛 satisfying 𝛽 𝑛 𝑛 -2/𝑚 → ∞ and 𝛽 𝑛 𝑛 -1 → 0. In particular, 𝛽 𝑛 → ∞. Then, [START_REF] Lee | Estimating the number of change points in a sequence of independent normal random variables[END_REF][START_REF] Lee | Estimating the number of change points in exponential families distributions[END_REF] uses this estimator for observations belonging to an independent exponential family, that is, the 𝑋 𝑖 ∈ R are independent and have density with respect to the Lebesgue measure

 𝜏 YAu := ̂ 𝜏 ( ̂ 𝐷 YAu ) is consistent in probability.
𝑓 𝜃 (𝑥) = exp (𝜃𝑥 + 𝜑(𝜃) + 𝑠(𝑥)) .
The consistency is also proven for 𝛽 𝑛 log(𝑛).

Let us mention that it is also possible to minimize the least-squares criterion for a given number of segments 𝐷, and then to use a testing procedure to decide if more segments ought to be added, rather than using a criterion similar to (1.6) [see, e.g., [START_REF] Bai | Estimating and testing linear models with multiple structural changes[END_REF]].

Penalized least-squares criterion

But the scheme that interests us the most is the minimization of the sum of the least-squares criterion and an additional term that increases with the number of segments. This term prevents us from choosing 𝜏 with too many segments. On the other side, if it is chosen too large, we risk selecting a segmentation with too few segments. More precisely, let us define 𝒯 𝑛 as the set of all segmentations of {1, . . . , 𝑛}.

In our setting, we call penalty function any mapping pen :

𝒯 𝑛 → R + . If 𝐷 𝜏 1 ≤ 𝐷 𝜏 2
implies pen (𝜏 1 ) ≤ pen (𝜏 2 ), we will say that the penalty is non-decreasing. We can now define the penalized least-squares procedure

 𝜏 pen ∈ arg min 𝜏 ∈𝒯𝑛 { ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ) } . (1.7)
Let us consider a penalty function proportional to the number of segments, that is,

pen ℓ (𝜏 ) := 𝛽 𝑛 𝐷 𝜏 𝑛 , (1.8)
with 𝛽 𝑛 any sequence such that 𝛽 𝑛 → ∞ and 𝛽 𝑛 /𝑛 → 0. Then, under (𝐻(𝜑)) for any 𝜑 ∈ [1, 2), the estimator ̂ 𝜏 pen LM is consistent in probability with rate 1/𝑛 [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF][START_REF] Lavielle | Using penalized contrasts for the change-point problem[END_REF]. It is remarkable to notice that the rate of convergence obtained is also minimax.

A linear penalty is not the only possibility. When the variance of the noise 𝜎 2 is known, following ideas coming from model selection, [START_REF] Lebarbier | Quelques approches pour la détection de ruptures à horizon fini[END_REF] proposes to replace the 𝛽 𝑛 𝐷 𝜏 term in (1.8) with

pen L (𝜏 ) := 𝐷 𝜏 𝜎 2 𝑛 ( 𝑐 1 log 𝑛 𝐷 𝜏 + 𝑐 2 ) , (1.9)
where 𝑐 1 and 𝑐 2 are positive constants. Calibrating these constants is of course a key question; [START_REF] Lebarbier | Quelques approches pour la détection de ruptures à horizon fini[END_REF] argues in favor of setting 𝑐 2 /𝑐 1 = 2.5 and using a slope heuristic [START_REF] Baudry | Slope heuristics: overview and implementation[END_REF] for choosing the constant in front of the penalty shape.

If the variance of the observations is unknown, [START_REF] Lebarbier | Quelques approches pour la détection de ruptures à horizon fini[END_REF] also advocates to replace 𝜎 2 by a Hall estimate [START_REF] Hall | Asymptotically optimal difference-based estimation of variance in nonparametric regression[END_REF]]. In the case of heteroscedastic noise, a very different approach is to use cross-validation instead of penalization [START_REF] Arlot | Segmentation of the mean of heteroscedastic data via cross-validation[END_REF]. Using a result from [START_REF] Birgé | Gaussian model selection[END_REF], it is proved in [START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF] that the estimator of 𝜇 ⋆ satisfies an oracle inequality. It is another way to look at the quality of a change-point method, that is, to look at the quadratic risk between the estimator of 𝜇 ⋆ naturally associated with ̂ 𝜏 Leb and the true piecewise constant function 𝜇 ⋆ . Before stating this result more precisely, we introduce some notations.

To each segmentation 𝜏 we associate 𝑋 𝜏 , defined as the empirical mean of 𝑋 on each segment of 𝜏 . Namely,

∀𝜏 ℓ-1 + 1 ≤ 𝑖 ≤ 𝜏 ℓ , (𝑋 𝜏 ) 𝑖 := 1 |𝜏 ℓ -𝜏 ℓ-1 | 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝑋 𝑖 . Let us also define ̂ 𝜇 = 𝑋 ̂ 𝜏 and ‖𝑥‖ 2 = 1 𝑛 ∑ 𝑛 𝑖=1 𝑥 2 𝑖 .
Then there exists some constants 𝐶(𝑐 1 , 𝑐 2 ) and 𝐶 ′ (𝑐 1 , 𝑐 2 ) such that

E [ ‖𝜇 ⋆ -̂ 𝜇‖ 2 ] ≤ 𝐶 inf 𝜏 ∈𝒯𝑛 { ‖𝜇 ⋆ -𝑋 𝜏 ‖ 2 + pen (𝜏 ) } + 𝐶 ′ 𝜎 2 𝑛 .
Informally, the previous display means that, in terms of quadratic risk, ̂ 𝜏 is as good as the best of the 𝜏 up to a log(𝑛) factor. Note that this result does not give any information about the quality of the segmentation with respect to 𝜏 ⋆ .

Minimizing the least-squares criterion with a linear penalty (1.7) can be seen as the following optimization problem:

Minimize 𝑢∈R 𝑛 ‖𝑋 -𝑢‖ 2 subject to ‖(𝑢 𝑖+1 -𝑢 𝑖 ) 1≤𝑖<𝑛 ‖ 0 = 𝐷 ⋆ ,
where ‖𝑥‖ 0 is the number of non-zero components of 𝑥. [START_REF] Harchaoui | Multiple change-point estimation with a total variation penalty[END_REF] propose to relax this ℓ 0 constraint into a ℓ 1 constraint, that is, to solve

Minimize 𝑢∈R 𝑛 ‖𝑋 -𝑢‖ 2 subject to ‖(𝑢 𝑖+1 -𝑢 𝑖 ) 1≤𝑖<𝑛 ‖ 1 ≤ 𝐷 ⋆ ∆ , where ∆ := max ⃒ ⃒ 𝜇 ⋆ 𝑖+1 -𝜇 ⋆ 𝑖 ⃒ ⃒ and ‖𝑥‖ 1 := ∑ 𝑛 𝑖=1 |𝑥 𝑖 | is the ℓ 1 norm.
It turns out that the previous display exactly corresponds to the Least Absolute Shrinkage eStimatOr (LASSO) in least-squares regression [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. A major feature of this approach is to decrease the computational cost from O (𝐷 max 𝑛 2 ) to O (𝐷 max 𝑛 log(𝑛)) -see Section 2.4 for more details on the implementation of penalized least-squares methods. Still, the estimated segmentation is consistent: for any segmentation 𝜏 , denote by Λ 𝜏 the normalized size of its smallest segment. Then, assuming sub-Gaussian noise5 , Λ 𝜏 ⋆ ≥ (log(𝑛)) 2 /𝑛 2 , and ∆ ≥ (log(𝑛)) 1/4 , Harchaoui and Lévy-Leduc [2010] prove, in particular, that the change-point locations are consistent conditionally to  𝐷 = 𝐷 ⋆ with high probability. Note that the convergence rate (log(𝑛)) 2 /𝑛 is optimal up to a logarithmic factor.

This approach has been successfully generalized to the multi-dimensional setting [START_REF] Bleakley | The group fused lasso for multiple change-point detection[END_REF]; in this case it can be shown that the problem is equivalent to the group LASSO [START_REF] Bakin | Adaptive regression and model selection in data mining problems[END_REF][START_REF] Lin | Component selection and smoothing in multivariate nonparametric regression[END_REF]. [START_REF] Bleakley | The group fused lasso for multiple change-point detection[END_REF] show that their procedure is consistent in probability for a single changepoint, provided that the noise level is smaller than a threshold depending essentially on the size and location of the jump. This concludes our introduction to change-point detection. We will complete our survey of the literature in Section 2.1.

Kernel methods

We now continue this general introduction with a brief presentation of kernel methods, the missing brick in the construction of kernel change-point detectionabbreviated KCP from now on. We do not intend to cover all of this vast topic, and we refer to [START_REF] Vert | A primer on kernel methods[END_REF] for an introduction and to the monograph of [START_REF] Schölkopf | Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Adaptive Computation and Machine Learning[END_REF] for an extensive overview. In the following short introduction, we rather focus on essential tools for the understanding of the manuscript.

Positive semi-definite kernels

A fundamental idea when dealing with data is that of similarity measure, that is, a real-valued function whose values quantify how close two objects are. In most cases, the data has a 𝑑-dimensional representation and the Euclidean structure of R 𝑑 naturally provides a satisfactory notion of similarity between observations inherited from the scalar product. For instance, the dot product between two data-points 𝐴 and 𝐵 lying in the unit sphere reduces to the cosine of the angle 𝐴𝑂𝐵, a quantity that is close to 1 if 𝐴 and 𝐵 are in the same neighborhood and smaller otherwise.

Consider data living in a space 𝒳 such that the Euclidean metric does not reflect the structure of the data or which is not equipped with a scalar product. Can we replace the inner product by a mapping 𝑘 : 𝒳 × 𝒳 → R such that 𝑘 (•, •) and ⟨•, •⟩ benefit from similar properties? Can we define a function 𝑘 : 𝒳 × 𝒳 → R which would mimic the behavior of a scalar product on 𝒳 if it existed?

When the answer is affirmative, we shall say that 𝑘 is a positive semi-definite kernel, a notion that dates back to [START_REF] Mercer | Functions of positive and negative type, and their connection with the theory of integral equations[END_REF], building on ideas from [START_REF] Hilbert | Grundzüge einer allgemeinen Theorie der linaren Integralrechnungen[END_REF]. We now give a precise definition. Definition 1.1. Consider a non-empty set 𝒳 . Given a function 𝑘 : 𝒳 × 𝒳 → R, 𝑚 ∈ N ⋆ and 𝑥 1 , . . . , 𝑥 𝑚 ∈ 𝒳 , then the matrix In what follows, we use this concept extensively, hence we will abbreviate "positive semi-definite kernel" into "p.s.d. kernel", and often into "kernel". Note that it is possible to replace "positive semi-definite" by "positive definite" in Definition 1.1or equivalently "non-negative" by "positive" in (1.11) -, we then speak of positive definite kernel. We will not use this slightly more restrictive notion. Neither are we concerned with extensions of positive semi-definite kernels, such as kernels with values in C.

K := (𝑘 (𝑥 𝑖 , 𝑥 𝑗 )) 1≤𝑖,𝑗≤𝑚 ∈ R 𝑚×𝑚 (1.

Examples of kernels

Let us give a few examples of kernels together with proofs of their positive semidefiniteness in the simplest cases.

Linear kernel. For data belonging to 𝒳 = R 𝑑 , the most basic example is the linear kernel 𝑘 ℓ (𝑥, 𝑦) := ⟨𝑥, 𝑦⟩. See Fig. 1-6 for a plot of 𝑘 ℓ (𝑥, 𝑦) when 𝑑 = 1. It is routine to verify Eq. (1.11) since

𝑚 ∑ 𝑖,𝑗=1 𝜆 𝑖 𝜆 𝑗 𝑘 ℓ (𝑥 𝑖 , 𝑥 𝑗 ) = ⃦ ⃦ ⃦ ⃦ ⃦ 𝑚 ∑ 𝑖=1 𝜆 𝑖 𝑥 𝑖 ⃦ ⃦ ⃦ ⃦ ⃦ 2 ≥ 0 ,
by linearity of the dot product.

Polynomial kernel. A natural extension of the linear kernel is the polynomial kernel, defined as 𝑘 P (𝑥, 𝑦) := (⟨𝑥, 𝑦⟩ + 𝑐) 𝛼 , for 𝑥, 𝑦 ∈ 𝒳 = R 𝑑 , 𝑐 ≥ 0 and 𝛼 ∈ N ⋆ . See Fig. 1-6 for a plot of 𝑘 P (𝑥, 𝑦) when 𝑑 = 1.

Let us show that 𝑘 P is positive semi-definite when 𝛼 = 2. In this case, according to the multinomial theorem, 𝑘 P (𝑥, 𝑦) can be written

𝑑 ∑ 𝑖=1 ( 𝑥 2 𝑖 ) ( 𝑦 2 𝑖 ) + 𝑑 ∑ 𝑖=2 𝑖-1 ∑ 𝑗=1 ( √ 2𝑥 𝑖 𝑥 𝑗 ) ( √ 2𝑦 𝑖 𝑦 𝑗 ) + 𝑑 ∑ 𝑖=1 ( √ 2𝑐𝑥 𝑖 ) ( √ 2𝑐𝑦 𝑖 ) + 𝑐 2 .
Therefore, 𝑘 P (𝑥, 𝑦) = ⟨Φ(𝑥), Φ(𝑦)⟩ with Φ(𝑥) defined as

( 𝑥 2 𝑛 , . . . , 𝑥 2 1 , √ 2𝑥 𝑛 𝑥 𝑛-1 , . . . , √ 2𝑥 𝑛 𝑥 1 , √ 2𝑥 𝑛-1 𝑥 𝑛-2 , . . . , √ 2𝑐𝑥 𝑛 . . . , 𝑐 ) ⊤ ∈ R (𝑛+1)(𝑛+2) 2
.

The proof we used for the linear kernel can then be immediately adapted to the polynomial kernel. We will see later that such a mapping Φ exists for any kernel 𝑘.

Of course, the reasoning above can be adapted to any 𝛼.

Gaussian kernel. A widely used kernel is the so-called Gaussian kernel, introduced by [START_REF] Boser | A training algorithm for optimal margin classifiers[END_REF]. It is defined by

𝑘 G (𝑥, 𝑦) := exp ( -‖𝑥 -𝑦‖ 2 2𝜈 2 ) ,
for 𝑥, 𝑦 ∈ 𝒳 = R 𝑑 and 𝜈 > 0. See Fig. 1-6 a plot of 𝑘 ℓ (𝑥, 𝑦) when 𝑑 = 1 and 𝜈 = 1.0.

Let 𝑍 be a standard Gaussian random variable 𝒩 (0, I 𝑑 ) and 𝜆 1 , . . . , 𝜆 𝑚 ∈ R.

Then 𝑚 ∑ 𝑖,𝑗=1 𝜆 𝑖 𝜆 𝑗 𝑘 G (𝑥 𝑖 , 𝑥 𝑗 ) = 𝑚 ∑ 𝑖,𝑗=1 𝜆 𝑖 𝜆 𝑗 E [ e i𝜈 -1 (𝑥 𝑖 -𝑥 𝑗 ) ⊤ 𝑍 ] = E ⎡ ⎣ ⃒ ⃒ ⃒ ⃒ ⃒ 𝑚 ∑ 𝑖=1 𝜆 𝑖 e i𝜈 -1 𝑥 ⊤ 𝑖 𝑍 ⃒ ⃒ ⃒ ⃒ ⃒ 2 ⎤ ⎦ ≥ 0 ,
hence 𝑘 G is a positive-definite kernel.

Laplace kernel. The Laplace kernel is very similar to the Gaussian kernel, defined by 𝑘 L (𝑥, 𝑦) := exp (-‖𝑥 -𝑦‖ /𝜈), for 𝑥, 𝑦 ∈ 𝒳 and 𝜈 ≥ 0. It is sometimes called the exponential kernel [START_REF] Genton | Classes of kernels for machine learning: a statistics perspective[END_REF]. Together, they belong to a larger class of kernels such that 𝑘(𝑥, 𝑦) = 𝜅(𝑥 -𝑦) for some function 𝜅. These kernels are called translationinvariant since they depend only on the difference between the input vectors. They are considered in more depth in Section 4.3.1, where we give a precise definition and additional properties. We refer to [START_REF] Berg | Harmonic analysis on semigroups[END_REF] for a systematic study of such kernels. In this example and the previous one, it is clear that 𝑘 (𝑥, 𝑦) is large whenever 𝑥 and 𝑦 are near, since 𝑘 (𝑥, 𝑦) is a decreasing function of ‖𝑥 -𝑦‖. See Fig. 1-6 a plot of 𝑘 L (𝑥, 𝑦) when 𝑑 = 1 and 𝜈 = 1.0.

Graph kernels. Given two finite graphs 𝐺 1 = (𝐸 1 , 𝑉 1 ) and 𝐺 2 = (𝐸 2 , 𝑉 2 ), one can define the direct product of 𝐺 1 and 𝐺 2 as the graph 𝐺 × = (𝑉 × , 𝐸 × ) that has vertex set 𝑉 1 × 𝑉 2 , and edges given by the rule: "(𝑣

1 , 𝑣 ′ 1 ) is connected to (𝑣 2 , 𝑣 ′ 2 ) if and only if 𝑣 1 is connected to 𝑣 2 and 𝑣 ′ 1 is connected to 𝑣 ′ 2 .
" Denote by 𝐴 × the adjacency matrix of 𝐺 × , and pick a sequence (𝑎 𝑝 ) 𝑝≥0 of positive weights. Then the direct product kernel [START_REF] Gärtner | On graph kernels: Hardness results and efficient alternatives[END_REF] is defined as

𝑘 × (𝐺 1 , 𝐺 2 ) := |𝑉 × | ∑ 𝑖,𝑗=1 ( +∞ ∑ 𝑝=0 𝑎 𝑝 𝐴 𝑝 × ) 𝑖,𝑗
if the limit exists -see also [START_REF] Vishwanathan | Graph kernels[END_REF]. Note that it is already more difficult than in the previous examples to see why 𝑘 × is a similarity measure on the set of finite graphs.

The kernel-trick

From Definition 1.1, clearly any scalar product on 𝒳 is a kernel. In this sense, kernels can be seen as a generalization of scalar product.6 But in fact this analogy between kernels and dot products runs deeper, as was hinted in the previous examples. Let us recall that a Hilbert space is a real or complex inner product space that is also a complete metric space with respect to the distance induced by the inner product. The fundamental property of kernels is the following: Theorem 1.1 (Moore-Aronszajn). The mapping 𝑘 is a positive semi-definite kernel on 𝒳 if, and only if, there exists a Hilbert space ℋ and a mapping Φ : 𝒳 → ℋ such that ∀𝑥, 𝑦 ∈ 𝒳 , 𝑘(𝑥, 𝑦) = ⟨Φ(𝑥), Φ(𝑦)⟩ ℋ .

(1.12)

Note that the reverse sense of Theorem 1.1 is clearly true; it follows directly by setting (1.12) as a definition for 𝑘 and using the properties of the dot product. The non-trivial part in Theorem 1.1 is the direct implication. It was first obtained by [START_REF] Mercer | Functions of positive and negative type, and their connection with the theory of integral equations[END_REF] for 𝒳 = [𝑎, 𝑏] ⊂ R and continuous 𝑘, and later proved by [START_REF] Kolmogorov | Stationary sequences in Hilbert space[END_REF] for countable 𝒳 . The general result is stated by [START_REF] Aronszajn | Theory of reproducing kernels[END_REF], which refers to [START_REF] Aronszajn | La théorie générale des noyaux reproduisants et ses applications[END_REF] and attributes the result to [START_REF] Moore | On properly positive Hermitian matrices[END_REF][START_REF] Moore | General analysis (part I)[END_REF][START_REF] Moore | General analysis (part II). Memoirs of the American Philosophical Society[END_REF]. Moreover the Hilbert space ℋ is essentially unique.

Before any further comment, let us give a short proof of Theorem 1.1 in the special case where 𝒳 is a finite set.

Proof. We only prove the direct implication. Let us assume that 𝒳 = { 𝑥 1 , . . . , 𝑥 𝑚 } . Then any kernel 𝑘 on 𝒳 is entirely defined by the matrix K = (𝑘 (𝑥 𝑖 , 𝑥 𝑗 )) 1≤𝑖,𝑗≤𝑚 , which is positive semi-definite. In particular, it can be diagonalized on an orthonormal basis of eigenvectors (𝑒 1 , . . . , 𝑒 𝑚 ) ∈ R 𝑚×𝑚 , with eigenvalues

0 ≤ 𝜁 1 ≤ • • • ≤ 𝜁 𝑚 . Define Φ(𝑥 𝑖 ) := (√ 𝜁 1 (𝑒 1 ) 𝑖 , . . . , √ 𝜁 𝑚 (𝑒 𝑚 ) 𝑖 ) ⊤ ∈ R 𝑚 . Then, for any 1 ≤ 𝑖, 𝑗 ≤ 𝑚, ⟨Φ(𝑥 𝑖 ), Φ(𝑥 𝑗 )⟩ = 𝑚 ∑ ℓ=1 𝜁 ℓ (𝑒 ℓ ) 𝑖 (𝑒 ℓ ) 𝑗 = ( 𝑚 ∑ ℓ=1 𝜁 ℓ 𝑒 ℓ 𝑒 ⊤ ℓ ) 𝑖,𝑗 = 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) .
This link between positive definite kernels and Hilbert spaces has a simple, yet crucial consequence. Often referred to as the kernel-trick, Equation (1.12) allows to run any algorithm that depends only on the scalar products ⟨Φ(𝑥 𝑖 ), Φ(𝑥 𝑗 )⟩ ℋ without actually doing any calculations in ℋ, replacing them by evaluations 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) of the kernel map.

This fact was first uncovered by the statistical community in the 60s for specific kernels, cf. [START_REF] Aizerman | Theoretical foundations of the potential function method in pattern recognition[END_REF] for instance. The full generality of the kernel trick was made clear in the work of [START_REF] Schölkopf | Kernel principal component analysis[END_REF][START_REF] Schölkopf | Nonlinear component analysis as a kernel eigenvalue problem[END_REF]. This principle has since been applied in countless settings, see [START_REF] Schölkopf | Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Adaptive Computation and Machine Learning[END_REF] for a full account. In particular, the possibility to extend classical methods to data living in general sets such as graphs or texts is a huge benefit from the use of kernels.

We now give an elementary example to illustrate this principle. Fix 𝒳 and 𝑘, and let ℋ be the associated Hilbert space and Φ the feature map. Take 𝑥, 𝑦 ∈ 𝒳 , and suppose that we want to compute the distance between Φ(𝑥) and Φ(𝑦) in ℋ. We write

‖Φ(𝑥) -Φ(𝑦)‖ 2 ℋ = ⟨Φ(𝑥) -Φ(𝑦), Φ(𝑥) -Φ(𝑦)⟩ ℋ = ⟨Φ(𝑥), Φ(𝑥)⟩ ℋ -2⟨Φ(𝑥), Φ(𝑦)⟩ ℋ + ⟨Φ(𝑦), Φ(𝑦)⟩ ℋ ‖Φ(𝑥) -Φ(𝑦)‖ 2 ℋ = 𝑘 (𝑥, 𝑥) -2𝑘 (𝑥, 𝑦) + 𝑘 (𝑦, 𝑦) , thus d (Φ(𝑥), Φ(𝑦)) = √ 𝑘 (𝑥, 𝑥) -2𝑘 (𝑥, 𝑦) + 𝑘 (𝑦, 𝑦) .
As promised, for points in 𝒳 , pairwise distance computations in ℋ require only the knowledge of the Gram matrix of 𝑘. We refer to Fig. 1-7 for an illustration.

𝒳 ℋ • • • • 𝑥 𝑦 Φ(𝑥) Φ(𝑦) ‖ Φ ( 𝑥 ) - Φ ( 𝑦 ) ‖ ℋ Φ Figure 1-7 -
The points 𝑥 and 𝑦 are mapped from 𝒳 to the Hilbert space ℋ associated to 𝑘 via the feature map Φ. The distance between Φ(𝑥) and Φ(𝑦) in ℋ can be computed without explicit computations in ℋ.

A remarkable fact is that the mapping Φ in Theorem 1.1 is explicit. Let us define a Reproducing Kernel Hilbert Space, latter abbreviated RKHS. Definition 1.2 (RKHS). Let 𝒳 be a non-empty set and ℋ be a Hilbert space of functions7 𝑓 : 𝒳 → R. Then ℋ is called a RKHS if there exists a function 𝑘 : 𝒳 × 𝒳 → R with the following properties:

(i) 𝑘 has the reproducing property ∀𝑓 ∈ ℋ, ∀𝑥 ∈ 𝒳 , ⟨𝑓, 𝑘 (𝑥, •)⟩ = 𝑓 (𝑥) . In particular, ⟨𝑘 (𝑥, •) , 𝑘 (𝑦, •)⟩ = 𝑘 (𝑥, 𝑦). (ii) ℋ is the completion of Span { 𝑘 (𝑥, •) | 𝑥 ∈ 𝒳 } .
We then say that 𝑘 is a reproducing kernel. Then the following holds:

Theorem 1.2. A function 𝑘 : 𝒳 × 𝒳 → R is a positive semi-definite kernel if, and only if, it is a reproducing kernel.

In other words, we can write Φ(𝑥) = 𝑘 (𝑥, •) in Theorem 1.1. We call Φ the feature map.

This concludes our short introduction to kernel methods. Positive semi-definite kernels come with other fascinating features, as the representer theorem [START_REF] Kimeldorf | Some results on Tchebycheffian spline functions[END_REF], a result essentially useful in the resolution of optimization problems. As we will see in Section 2.4, the optimization problem that we are to solve does not require using this result, and we chose to exclude the representer theorem from this introduction.

We now turn to the center of this manuscript, kernel change-point detection.

Introduction

In many situations, some properties of a time series change over time, such as the mean, the variance or higher-order moments. Change-point detection is the long standing question of finding both the number and the localization of such changes. This is an important front-end task in many applications. For instance, detecting changes occurring in comparative genomic hybridization array data (CGH arrays) is crucial to the early diagnosis of cancer [START_REF] Lai | Comparative analysis of algorithms for identifying amplifications and deletions in array CGH data[END_REF]. In finance, some intensively examined time series like the volatility process exhibit local homogeneity and it is useful to be able to segment these time series both for modeling and forecasting [START_REF] Lavielle | Detection of multiple change-points in multivariate time series[END_REF][START_REF] Spokoiny | Multiscale local change point detection with applications to value-at-risk[END_REF]. Change-point detection can also be used to detect changes in the activity of a cell [START_REF] Ritov | Detection of onset of neuronal activity by allowing for heterogeneity in the change points[END_REF], in the structure of random Markov fields [START_REF] Liu | Support consistency of direct sparse-change learning in markov networks[END_REF], or a sequence of images [START_REF] Kim | Using labeled data to evaluate change detectors in a multivariate streaming environment[END_REF][START_REF] Abou-Elailah | Detection of abrupt changes in spatial relationships in video sequences[END_REF]. Generally speaking, it is of interest to the practitioner to segment a time series in order to calibrate its model on homogeneous sets of data-points.

Addressing the change-point problem in practice requires to face several important challenges. First, the number of changes can not be assumed to be known in advance -in particular, it can not be assumed to be equal to 0 or 1 -, hence a practical change-point procedure must be able to infer the number of changes from the data. Second, changes do not always occur in the mean or the variance of the data, as assumed by most change-point procedures. We need to be able to detect changes in other features of the distribution. Third, parametric assumptions -which are often made for building or for analyzing change-point procedures -are often unrealistic, so that we need a fully non-parametric approach. Fourth, data points in the time series we want to segment can be high-dimensional and/or structured. If the dimensionality is larger than the number of observations, a non-asymptotic analysis is mandatory for theoretical results to be meaningful. When data are structured -for instance, histograms, graphs or strings -, taking their structure into account seems necessary for detecting efficiently the change-points.

We focus only on the off-line problem, that is, when all observations are given at once, as opposed to the situation where data come as a continuous stream. We refer to [START_REF] Tartakovsky | Sequential Analysis: Hypothesis Testing and Changepoint Detection[END_REF] for an extensive review of sequential methods, which are adapted to the latter situation. Numerous off-line change-point procedures have been proposed since the seminal works of [START_REF] Page | A test for a change in a parameter occurring at an unknown point[END_REF], [START_REF] Fisher | On grouping for maximum homogeneity[END_REF] and [START_REF] Bellman | On the approximation of curves by line segments using dynamic programming[END_REF], which are mostly parametric in essence. We refer to Brodsky and Darkhovsky [2013, Chapter 2] for a review of non-parametric off-line change-point detection methods. Among recent works in this direction, we can mention the Wild Binary Segmentation (WBS, [START_REF] Fryzlewicz | Wild binary segmentation for multiple change-point detection[END_REF]) and the non-parametric multiple change-point detection procedure (NMCD, [START_REF] Zou | Nonparametric maximum likelihood approach to multiple change-point problems[END_REF]). Some authors also consider the case of highdimensional data when only a few coordinates of the mean change at each changepoint [Wang and Samworth, 2016, and references therein], or the problem of detecting gradual changes [START_REF] Vogt | Detecting gradual changes in locally stationary processes[END_REF]; we do not address these slightly different problems.

To the best of our knowledge, no off-line change-point procedure addressed simultaneously the four challenges mentioned above, until KCP was proposed by [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF]. In short, KCP mixes the penalized least-squares approach to change-point detection [START_REF] Comte | A new algorithm for fixed design regression and denoising[END_REF][START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF] with positive semi-definite kernels [START_REF] Aronszajn | Theory of reproducing kernels[END_REF]. It is not the only procedure that uses positive semi-definite kernels to detect changes in a times series. Apart from [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF], who introduced KCP for a fixed number of change-points, and [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] who extended KCP to an unknown number of change-points, we are aware of several closely related work. Maximum Mean Discrepancy [MMD, [START_REF] Gretton | A kernel method for the two-sample-problem[END_REF] has been used for building two-sample tests; a block average version of the MMD, named the 𝑀 -statistic, has lead to an on-line change-point detection procedure [START_REF] Li | 𝑀 -statistic for kernel change-point detection[END_REF]. A kernel-based statistic, named kernel Fisher discriminant ratio, has been used by [START_REF] Harchaoui | Kernel change-point analysis[END_REF] for homogeneity testing and for detecting a single change-point. [START_REF] Sharipov | Sequential block bootstrap in a Hilbert space with application to change point analysis[END_REF] build an analogue of the CUSUM statistic for Hilbert-valued random variables in order to detect a single change in the mean, and could be applied in our setting to the images of the observations in the feature space. Kernel change detection [START_REF] Desobry | An online kernel change detection algorithm[END_REF] is an on-line procedure that uses a kernel to build a dissimilarity measure between the near past and future of a data-point.

We first introduce in Section 2.2 and 2.3 some notations and concepts that are necessary for the rest of the manuscript. In Section 2.4, we present the computational aspect of KCP. In brief, the KCP segmentation can be computed efficiently thanks to a dynamic programming algorithm [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF][START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF].

Section 2.5 is devoted to presenting the hypothesis that will be used in Chapter 3 for the theoretical study of KCP. Our framework is common to [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], that proved an oracle inequality for KCP. We recall this result in Section 2.6.

Kernel change-point detection

We first describe the change-point problem with our notations (Section 2.2.1) and the kernel change-point procedure (Section 2.2.2).

Change-point problem

Set 2 ≤ 𝑛 < +∞ and consider 𝑋 1 , . . . , 𝑋 𝑛 independent 𝒳 -valued random variables, where 𝒳 is an arbitrary (measurable) space. The goal of change-point detection is to detect abrupt changes in the distribution of the 𝑋 𝑖 s. For any 𝐷 ∈ { 1, . . . , 𝑛 } and any integers 0 An important example to have in mind is the following.

= 𝜏 0 < 𝜏 1 < • • • < 𝜏 𝐷 = 𝑛, we define the segmentation 𝜏 := [ 𝜏 0 , . . . , 𝜏 𝐷 ] of { 1, . . . , 𝑛 } as the collection of segments 𝜆 ℓ = { 𝜏 ℓ-1 + 1, . . . , 𝜏 ℓ } , ℓ ∈ { 1, . . . ,
• • • • • • • • • • 𝜏
Example 2.1 (Asymptotic setting). Let 𝐾 ≥ 1, 0 = 𝑏 0 < 𝑏 1 < • • • < 𝑏 𝐾 < 𝑏 𝐾+1 = 1
and 𝑃 1 , . . . , 𝑃 𝐾+1 some probability distributions on 𝒳 be fixed. Then, for any 𝑛 and 𝑖 ∈ {1, . . . , 𝑛}, we set 𝑡 𝑖 := 𝑖/𝑛 and the distribution of 𝑋 𝑖 is 𝑃 𝑗(𝑖) where 𝑗(𝑖) is such that 𝑡 𝑖 ∈ [𝑏 𝑗(𝑖) , 𝑏 𝑗(𝑖)+1 ). In other words, we have a fixed segmentation of [0, 1], given by the 𝑏 𝑗 , a fixed distribution over each segment, given by the 𝑃 𝑗 , and we observe independent realizations from the distributions at discrete times 𝑡 1 , . . . , 𝑡 𝑛 . The corresponding true change-points in {0, . . . , 𝑛} are the ⌊𝑛𝑏 𝑗 ⌋, 𝑗 = 1, . . . , 𝐾. For 𝑛 large enough, there are 𝐾 + 1 segments. Fig. 2-2 shows an example. Let us emphasize that in this setting, 𝑛 going to infinity does not mean that new observations are observed over time. Recall that we consider the change-point problem a posteriori : a larger 𝑛 means that we have been able to observe the phenomenon of interest with a finer time discretization. This is similar to the setting presented in Chapter 1. Also note that this asymptotic setting is restrictive in the sense that segments size asymptotically are of order 𝑛; we do not make this assumption in our analysis, which also covers asymptotic settings where some segments have a smaller size.

𝑓 : [0, 1] → R is a (fixed) piecewise constant function (shown in red).
The goal is to recover the number of abrupt changes of 𝑓 (here, 2) and their locations (𝑏 1 = 0.5 and 𝑏 2 = 0.7). Note that other kinds of changes in the distribution of the 𝑋 𝑖 can be considered, see Section 4.2.

Kernel change-point procedure

Let 𝑘 : 𝒳 ×𝒳 → R be a positive semi-definite kernel, that is, a measurable function such that the matrix (𝑘 (𝑥 𝑖 , 𝑥 𝑗 )) 1≤𝑖,𝑗≤𝑚 is positive semi-definite for any 𝑚 ≥ 1 and 𝑥 1 , . . . , 𝑥 𝑚 ∈ 𝒳 [START_REF] Schölkopf | Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Adaptive Computation and Machine Learning[END_REF]. Let us recall some classical examples of positive semi-definite kernels, some of them already encountered in the Introduction:

-the linear kernel :

𝑘 ℓ (𝑥, 𝑦) = ⟨𝑥, 𝑦⟩ R 𝑝 for 𝑥, 𝑦 ∈ 𝒳 = R 𝑝 .
-the polynomial kernel of order 𝛼 ≥ 1:

𝑘 P (𝑥, 𝑦) = ( ⟨𝑥, 𝑦⟩ R 𝑝 + 1) 𝛼 for 𝑥, 𝑦 ∈ 𝒳 = R 𝑝 . -the Gaussian kernel with bandwidth 𝜈 > 0: 𝑘 G (𝑥, 𝑦) = exp[-|𝑥 -𝑦| 2 /(2𝜈 2 )]
for 𝑥, 𝑦 ∈ 𝒳 = R 𝑝 .

-the Laplace kernel with bandwidth 𝜈 > 0:

𝑘 L (𝑥, 𝑦) = exp[-|𝑥 -𝑦| /(2𝜈)] for 𝑥, 𝑦 ∈ 𝒳 = R 𝑝 .
-the 𝜒 2 -kernel:

𝑘 𝜒 2 (𝑥, 𝑦) = exp ( -1 𝑝𝜈 ∑ 𝑝 𝑖=1 (𝑥 𝑖 -𝑦 𝑖 ) 2 𝑥 𝑖 +𝑦 𝑖 )
for 𝑥, 𝑦 ∈ 𝒳 = ∆ 𝑝 the 𝑝dimensional simplex, and the bandwidth 𝜈 is a positive constant.

As done by [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF] and [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], for a given segmentation 𝜏 ∈ 𝒯 𝐷 𝑛 , we assess the adequacy of 𝜏 with the kernel least-squares criterion

 ℛ 𝑛 (𝜏 ) := 1 𝑛 𝑛 ∑ 𝑖=1 𝑘 (𝑋 𝑖 , 𝑋 𝑖 ) - 1 𝑛 𝐷 ∑ ℓ=1 ⎡ ⎣ 1 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝑘 (𝑋 𝑖 , 𝑋 𝑗 ) ⎤ ⎦ . (2.1)
Elementary algebra shows that, when 𝒳 = R 𝑝 and 𝑘 = 𝑘 ℓ , ̂ ℛ 𝑛 is the usual leastsquares criterion. Indeed, if we denote by ̂ 𝑅 𝑛 (𝜏 ) the least-squares criterion in R 𝑝 and set 𝑘 (𝑥, 𝑦) = ⟨𝑥, 𝑦⟩,

𝑛 ̂ 𝑅 𝑛 (𝜏 ) = 𝐷 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 𝑋 𝑖 - 1 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝑋 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 = 𝐷 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ⎛ ⎝ ‖𝑋 𝑖 ‖ 2 - 2 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 ⟨𝑋 𝑖 , 𝑋 𝑗 ⟩ + 1 (𝜏 ℓ -𝜏 ℓ-1 ) 2 𝜏 ℓ ∑ 𝑗,𝑗 ′ =𝜏 ℓ-1 +1 ⟨𝑋 𝑗 , 𝑋 𝑗 ′ ⟩ ⎞ ⎠ = 𝑛 ∑ 𝑖=1 ‖𝑋 𝑖 ‖ 2 - 𝑑 ∑ ℓ=1 1 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑖,𝑗=𝜏 ℓ-1 +1 ⟨𝑋 𝑖 , 𝑋 𝑗 ⟩ 𝑛 ̂ 𝑅 𝑛 (𝜏 ) = 𝑛 ̂ ℛ 𝑛 (𝜏 ) .
Minimizing this criterion over the set of all segmentations always outputs the segmentation with 𝑛 segments reduced to a point, which can be seen as over-fitting. To counteract this, a classical idea [START_REF] Lavielle | Using penalized contrasts for the change-point problem[END_REF] for instance] is to minimize a penalized criterion crit (𝜏 ) := ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ), where pen : 𝒯 𝑛 → R + is called the penalty. Formally, the kernel change-point procedure of [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] selects the segmentation

 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 { crit (𝜏 ) } where crit (𝜏 ) = ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ) . (2.2)
In this manuscript, we will use two different classes of penalty functions which were already encountered in Section 1.3.2. Let us recall their definition. The first is a classical choice in model selection, similar to AIC, BIC and 𝐶 𝑝 criteria. It is proportional to the number of segments and is often called a linear penalty. Namely, we consider

pen (𝜏 ) = pen ℓ (𝜏 ) := 𝐶𝑀 2 𝐷 𝜏 𝑛 , (2.3)
where 𝐶 is a positive constant and 𝑀 is specified in Assumption 2.1 later on. This definition coincides with Eq. (1.8) with 𝛽 𝑛 = 𝐶𝑀2 . Note that, as mentioned in Chapter 1, different penalty shapes can be considered. For instance, as suggested by [START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF], it is also possible to use as a penalty shape

pen (𝜏 ) = pen L (𝜏 ) := 𝐷 𝜏 𝑛 ( 𝑐 1 log 𝑛 𝐷 𝜏 + 𝑐 2 ) , (2.4)
with 𝑐 1 and 𝑐 2 positive constants. This definition coincides with Eq. ( 1.9) up to a variance term. In a very similar fashion, [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] advocates for the use of

pen (𝜏 ) = pen ACH (𝜏 ) := 1 𝑛 ( 𝑐 1 log ( 𝑛 -1 𝐷 𝜏 -1 ) + 𝑐 2 𝐷 𝜏 ) .
We will see in Section 2.6 that the oracle inequality obtained in [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] is valid for pen ℓ when the penalty constant 𝐶 is of order log(𝑛), as well as pen L and pen ACH .

The reproducing kernel Hilbert space

Let ℋ be the RKHS associated to 𝑘, together with the canonical feature map

Φ : 𝒳 → ℋ Φ : 𝒳 → ℋ 𝑥 ↦ → Φ(𝑥) := 𝑘 (•, 𝑥) ,
as exposed in Section 1.4. In this section, we explain how to define a "mean element" of Φ(𝑋 𝑖 ) belonging to ℋ, and we rewrite the empirical risk.

Kernel mean embedding

Let us write ⟨•, •⟩ ℋ (resp. ‖•‖ ℋ ) for the inner product (resp. the norm) of ℋ. For any 𝑖 ∈ { 1, . . . , 𝑛 } , define 𝑌 𝑖 := Φ(𝑋 𝑖 ) ∈ ℋ. As we have seen, in the case where 𝑘 = 𝑘 ℓ , then 𝑌 𝑖 = ⟨•, 𝑋 𝑖 ⟩ and the empirical risk ̂ ℛ 𝑛 reduces to the least-squares criterion

̂ 𝑅 𝑛 (𝜏 ) = 1 𝑛 𝐷𝜏 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ⃦ ⃦ 𝑋 𝑖 -𝑋 ℓ ⃦ ⃦ 2 ,
where 𝑋 ℓ is the empirical mean of the 𝑋 𝑖 over the segment

{ 𝜏 ℓ-1 + 1, . . . , 𝜏 ℓ } .
It is well-known that penalized least-squares procedures detect changes in the mean of the observations 𝑋 𝑖 , see [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF]. Hence the kernelized version of this least-squares procedure, KCP, should detect changes in the "mean" of the 𝑌 𝑖 = Φ(𝑋 𝑖 ), which are a non-linear transformation of the 𝑋 𝑖 .

More precisely, assume that ℋ is separable. 2 Suppose that

∀𝑖 ∈ { 1, . . . , 𝑛 } , E [ √ 𝑘 (𝑋 𝑖 , 𝑋 𝑖 ) ] < +∞ . (2.5)
Then, for any 𝑖 ∈ { 1, . . . , 𝑛 } , we define 𝜇 ⋆ 𝑖 as the Bochner integral of 𝑌 𝑖 ,

𝜇 ⋆ 𝑖 := E [𝑘 (𝑋 𝑖 , •)] ∈ ℋ .
We refer to Diestel and Uhl [1977, Chapter 2] and [START_REF] Ledoux | Probability in Banach Spaces: isoperimetry and processes[END_REF] for the proper definition of the Bochner integral. Let us say that the mean element 𝜇 ⋆ 𝑖 can be seen as an embedding of 𝑃 𝑋 𝑖 , the distribution of 𝑋 𝑖 , in ℋ -for this reason, the mapping 𝑃 ↦ → E 𝑋∼𝑃 [𝑘 (𝑋, •)] is often referred to as the kernel embedding, or kernel mean embedding. The kernel embedding can be seen as a generalization of the notion of characteristic function [START_REF] Muandet | Kernel mean embedding of distributions: A review and beyond[END_REF].

Note that the condition (2.5) is satisfied in our setting (when either Assumption 2.1 or Assumption 2.2 holds true, see Section 2.5), and ℋ is separable in most standard cases [START_REF] Dieuleveut | Nonparametric stochastic approximation with large step-sizes[END_REF]. Hence the mean elements 𝜇 ⋆ 𝑖 will be well-defined in our setting. The Bochner integral commutes with continuous linear operators, hence the following property holds, which will be of use:

∀𝑔 ∈ ℋ, ⟨𝜇 ⋆ 𝑖 , 𝑔⟩ ℋ = E [ 𝑔(𝑋 𝑖 ) ] = E [ ⟨𝑌 𝑖 , 𝑔⟩ ℋ ] .
We now define the "true segmentation" 𝜏 ⋆ ∈ 𝒯 𝑛 by

𝜇 ⋆ 1 = • • • = 𝜇 ⋆ 𝜏 ⋆
From now on, we focus on the problem of detecting the changes of 𝜏 ⋆ only, whether the kernel is characteristic or not.

Rewriting the empirical risk

It is convenient to see the images of the observations by the feature map as an element of ℋ 𝑛 . To this extent, we define 𝑌 := (𝑌 1 , . . . , 𝑌 𝑛 ), as well as 𝜇 ⋆ := (𝜇 ⋆ 1 , . . . , 𝜇 ⋆ 𝑛 ) ∈ ℋ 𝑛 and 𝜀 := 𝑌 -𝜇 ⋆ ∈ ℋ 𝑛 . We identify the elements of ℋ 𝑛 with the set of applications {1, . . . , 𝑛} → ℋ, naturally embedded with the inner product and norm 4 given by

∀𝑥, 𝑦 ∈ ℋ 𝑛 , ⟨𝑥, 𝑦⟩ := 𝑛 ∑ 𝑖=1 ⟨𝑥 𝑖 , 𝑦 𝑖 ⟩ ℋ and ‖𝑥‖ 2 := 𝑛 ∑ 𝑗=1 ‖𝑥 𝑗 ‖ 2 ℋ .
We now rewrite the empirical risk as a function of 𝜏 and 𝑌 . For any segmentation 𝜏 ∈ 𝒯 𝑛 , define 𝐹 𝜏 the set of applications {1, . . . , 𝑛} → ℋ that are constant over the segments of 𝜏 . We see 𝐹 𝜏 as a subspace of ℋ 𝑛 as a vector space. Take 𝑓 ∈ ℋ 𝑛 , we define Π 𝜏 𝑓 the orthogonal projection of 𝑓 onto 𝐹 𝜏 with respect to ‖•‖:

Π 𝜏 𝑓 ∈ arg min 𝑔∈𝐹𝜏 ‖𝑓 -𝑔‖ .
It comes without much surprise that Π 𝜏 𝑓 can be computed as in the real case and, in addition, is also equal to the piece-wise constant function whose values are the empirical mean of 𝑓 on each segment of 𝜏 . Namely, for any 𝑓 ∈ ℋ 𝑛 and any ℓ ∈ {1, . . . , 𝐷 𝜏 },

∀𝑖 ∈ {𝜏 ℓ-1 + 1, . . . , 𝜏 ℓ }, (Π 𝜏 𝑓 ) 𝑖 = 1 |𝜏 ℓ -𝜏 ℓ-1 | 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝑓 𝑗 .
(2.7)

We give the proof of Eq. (2.7) as found in [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], which is another illustration of the kernel trick.

Proof. Define 𝜆 ℓ := { 𝜏 ℓ-1 + 1, 𝜏 ℓ } the ℓ-th segment of 𝜏 . For any 𝑔 ∈ 𝐹 𝜏 , denote by 𝑔 𝜆 ℓ the value of 𝑔 on 𝜆 ℓ and ̃ 𝑔 𝜆 ℓ := 1 𝜏 ℓ -𝜏 ℓ-1 ∑ 𝑖∈𝜆 ℓ 𝑔 𝑖 .
Then we can decompose ‖𝑓 -𝑔‖ as andEq. (2.7) is proved.

‖𝑓 -𝑔‖ 2 = 𝐷𝜏 ∑ ℓ=1 ∑ 𝑖∈𝜆 ℓ ‖𝑓 𝜆 ℓ -̃ 𝑔 𝜆 ℓ ‖ 2 ℋ + ‖𝑔 𝑖 -̃ 𝑔 𝜆 ℓ ‖ 2 ℋ + 2⟨𝑓 𝜆 ℓ -̃ 𝑔 𝜆 ℓ , 𝑔 𝑖 -̃ 𝑔 𝜆 ℓ ⟩ ℋ = 𝐷𝜏 ∑ ℓ=1 (𝜏 ℓ -𝜏 ℓ-1 ) ‖𝑓 𝜆 ℓ -̃ 𝑔 𝜆 ℓ ‖ 2 ℋ + 𝐷𝜏 ∑ ℓ=1 ∑ 𝑖∈𝜆 ℓ ‖𝑔 𝑖 -̃ 𝑔 𝜆 ℓ ‖ 2 ℋ , since ∑ 𝑖∈𝜆 ℓ (𝑔 𝑖 -̃ 𝑔 𝜆 ℓ ) = 0. Therefore, ‖𝑓 -𝑔‖ 2 is minimal over 𝑓 ∈ 𝐹 𝜏 in ̃ 𝑔,
Thanks to Eq. (2.7), we are now able to write the empirical risk as

 ℛ 𝑛 (𝜏 ) = 1 𝑛 ‖𝑌 -̂ 𝜇 𝜏 ‖ 2 = 1 𝑛 𝐷𝜏 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ‖𝑌 𝑖 -(̂ 𝜇 𝜏 ) 𝑖 ‖ 2 ℋ , (2.8)
where ̂ 𝜇 𝜏 = Π 𝜏 𝑌 , following [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF][START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF].

Algorithmic aspects of KCP

In this section we discuss the implementation of the minimization problem (2.2).

As we explained before, one of the consequences of the kernel trick is that the KCP algorithm needs only the Gram matrix K to run. Hence the first step in KCP is the computation of K, which typically needs O (𝑛 2 ) computations and O (𝑛 2 ) storage space. Assuming that we dispose of a function computeKernel which can compute 𝑘 (𝑥, 𝑦) for 𝑥, 𝑦 ∈ 𝒳 , the calculation of K is straightforward and is given by Algorithm 2.1. Note that some non-negligible constant factors can be hidden in the O (•) notation, in particular regarding the computational cost. For instance, consider 𝑘 = 𝑘 G the Gaussian kernel. Then each evaluation of 𝑘 (𝑥, 𝑦) requires to compute ‖𝑥 -𝑦‖ 2 , that has a cost proportional to the dimension of the data. Suppose that 𝒳 = R 𝑝 , then the true computational cost of Algorithm 2.1 is O (𝑝𝑛 2 ).

Algorithm 2.1 Computation of the Gram matrix

procedure ComputeGramMatrix(x) 𝑛 ← length of 𝑥 K ← zeros(𝑛, 𝑛) for 𝑖 = 1 : 𝑛 do for 𝑗 = 1 : 𝑛 do K(𝑖, 𝑗) ← computeKernel(𝑥(𝑖), 𝑥(𝑗)) end for end for return K end procedure

KCP without penalty term

We now explain how problem (2.2) can be solved thanks to dynamic programming [START_REF] Bellman | On the approximation of curves by line segments using dynamic programming[END_REF]. To begin with, we solve (2.2) for a pre-defined number of segments 𝐷, without adding a penalty term, as in [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF]. For any 1 ≤ 𝑎 < 𝑏 ≤ 𝑛, we define 𝑐 (𝑎 : 𝑏) the cost of segment

{ 𝑎 + 1, . . . , 𝑏 } , that is 𝑐 (𝑎 : 𝑏) := 𝑏 ∑ 𝑖=𝑎+1 ⃦ ⃦ 𝑌 𝑖 - ( Ȳ𝑎:𝑏 ) 𝑖 ⃦ ⃦ 2 ℋ .
We also define cost (𝐷, 𝑡) the optimal cost of segmenting our signal up to time 𝑡 in 𝐷 segments, i.e., cost (𝐷, 𝑡) := inf

𝜏 ∈𝒯 𝐷 𝑡 { 1 𝑛 𝐷 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1+1 ‖𝑌 𝑖 -(̂ 𝜇 𝜏 ) 𝑖 ‖ 2 ℋ } = inf 𝜏 ∈𝒯 𝐷 𝑡 { 1 𝑛 𝐷 ∑ ℓ=1 𝑐 (𝜏 ℓ-1 : 𝜏 ℓ ) } . Notice that cost (𝐷, 𝑛) = 𝑛 ̂ ℛ(̂ 𝜏 (𝐷))
is the quantity we want to obtain. Now the key observation is that cost (𝐷, 𝑡) is additive:

cost (𝐷, 𝑡) = inf 𝜏 ∈𝒯 𝐷 𝑡 { 𝐷-1 ∑ ℓ=1 𝑐 (𝜏 ℓ-1 : 𝜏 ℓ ) + 𝑐 (𝜏 𝐷-1 : 𝑡) } = inf 𝑠≤𝑡 { cost (𝐷 -1, 𝑠) + 𝑐 (𝑠 : 𝑡) } .
Assuming that we dispose of a matrix S of size 𝑛 × 𝑛 that contains 𝑐 (𝑠 : 𝑡) for every values of 𝑠, 𝑡 with 0 ≤ 𝑠 < 𝑡 ≤ 𝑛, this gives rise to Algorithm 2.2, a dynamic programming scheme for computing the optimal cost. Of course, a simple backtracking procedure allows to recover the corresponding segmentation. The computational cost of running CostMatrix is

𝐷 ∑ 𝑑=2 𝑛 ∑ 𝑡=𝑑 (𝑡 + 1) = O ( 𝑛 2 𝐷 ) .
As for the memory space, it is dominated by the cost matrix and the index matrix, of size O (𝑛 2 𝐷). The BackTracking routine, on the other hand, runs in O (𝐷).

It is important to understand that computing 𝑛 2 entries of the matrix S is in fact a byproduct of the computation of the Gram matrix. Indeed, for any 1 ≤ 𝑠 < 𝑡 ≤ 𝑛, 𝑐 (𝑠 : 𝑡) can be expressed as a function of the cumulative sum of K. More precisely, we already noticed that Notice that Γ and 𝑇 can be computed at the same time than K, in O (𝑛 2 ) operations. Then 𝑐 (𝑠 :

𝑐 (𝑠 : 𝑡) = 𝑡 ∑ 𝑖=𝑠+1 𝑘 (𝑋 𝑖 , 𝑋 𝑖 ) - 1 𝑡 -𝑠 𝑡 ∑ 𝑖=𝑠+1 𝑡 ∑ 𝑗=𝑠+1 𝑘 (𝑋 𝑖 , 𝑋 𝑗 ) . Algorithm 2.
𝑡) = 𝑇 𝑡 -𝑇 𝑠 - 1 𝑡 -𝑠 (Γ 𝑡,𝑡 -2Γ 𝑠,𝑡 + Γ 𝑠,𝑠 ) .
Since 𝑘 is a symmetric function, Γ is a symmetric matrix, and the previous computation requires only 5 reads in the matrices Γ and 𝑇 and simple arithmetic operations.

Let us define the following auxiliary procedure.

Algorithm 2.3 Computation of the segment costs

procedure SegmentCosts(x,K) 𝑛 ← length of 𝑥 Γ ← CumSum(K) ◁ cumulative sum of the Gram matrix 𝑇 ← CumSum(diag (K))
◁ cumulative sum of the trace matrix 𝑆 ← zeros(𝑛, 𝑛) for 𝑡 = 1 : 𝑛 do 𝑆(0, 𝑡) ← 𝑇 (𝑡) -Γ(𝑡, 𝑡)/𝑡 end for for 𝑡 = 2 : 𝑛 do for 𝑠 = 1 : 𝑛 -1 do 𝑆(𝑠, 𝑡) ← 𝑇 (𝑡) -𝑇 (𝑠) -1 𝑡-𝑠 (Γ(𝑡, 𝑡) -2Γ(𝑠, 𝑡) + Γ(𝑠, 𝑠)) end for end for return 𝑆 end procedure

We are now ready to write down the kernel change-point algorithm without penalty term, Algorithm 2.4.

Algorithm 2.4 KCP algorithm without a penalty term

procedure KCPNoPenalty(x,D) 𝑛 ← length of 𝑥 K ← ComputeGramMatrix(𝑥) 𝑆 ← SegmentCosts(𝑥, K) [𝐼, 𝐶] ← CostMatrix(𝑆, 𝐷)  𝜏 ← BackTracking(𝐶, 𝐼, 𝐷) return ̂ 𝜏 end procedure

Adding the penalty

Let us go back to the original problem (2.2), with a penalty function pen depending only on the number of segments. We assume that a function ComputePenalty is provided that takes care of the computation of pen (𝜏 ) = pen (𝐷 𝜏 ). We can decompose problem (2.2) as follows: First, we compute the optimal least-squares criterion for any

1 ≤ 𝐷 ≤ 𝐷 max , that is ∀1 ≤ 𝐷 ≤ 𝐷 max , ̂ 𝜏 (𝐷) ∈ arg min 𝜏 ∈𝒯 𝐷 𝑛 ‖𝑌 -̂ 𝜇 𝜏 ‖ 2 , (2.9)
which is solved with the same scheme used by Algorithm 2.4. Second, we choose

 𝐷 ∈ arg min 1≤𝐷≤𝐷max {⃦ ⃦ 𝑌 -̂ 𝜇 ̂ 𝜏 (𝐷) ⃦ ⃦ 2 + pen (̂ 𝜏 (𝐷)) } , (2.10)
and third we set ̂ 𝜏 = ̂ 𝜏 ( ̂ 𝐷). From the algorithmic point of view, the cost matrix and index matrix outputted by the CostMatrix function contain all the relevant information to solve (2.9). Hence we just have to add the penalty term to the cost matrix after it is computed, and then select the number of segments for ̂ 𝜏 by minimizing the new criteria as in (2.10). This modification of Algorithm 2.4 is called Algorithm 2.5 and has the same computational complexity as Algorithm 2.4 for 𝐷 max segments, that is O (𝐷 max 𝑛 2 ).

Algorithm 2.5 KCP algorithm with a penalty term

procedure KCPWithPenalty(x,𝐷 max ) 𝑛 ← length of 𝑥 K ← ComputeGramMatrix(𝑥) 𝑆 ← SegmentCosts(𝑥, K) [𝐼, 𝐶] ← CostMatrix(𝑆, 𝐷 max ) for 𝑑 = 1 : 𝐷 max do 𝐶(𝑑, 𝑛) ← 𝐶(𝑑, 𝑛) + ComputePenalty(𝑑) ◁ modifying the last column of 𝐶 end for [ ̂ 𝐷, ∼] ← min 𝐶(:, 𝑛)  𝜏 ← BackTracking(𝐶, 𝐼, ̂ 𝐷) return ̂ 𝜏 end procedure
In definitive, the total computational cost for solving Problem 2.2 is thus

O ( (𝑐 𝑘 + 𝐷 max ) 𝑛 2 ) ,
where 𝑐 𝑘 is the cost of an evaluation of 𝑘 (𝑥, 𝑦). We want to emphasize that this is prohibitive for sample size larger than 10 5 . A path worth exploring to reduce this computational burden is to use a low-rank approximation of the Gram matrix instead of K as suggested by [START_REF] Celisse | New efficient algorithms for multiple change-point detection with kernels[END_REF]. If we let 𝑟 denote the rank of the approximation, the complexity of the dynamic programming part of the algorithm then drops to O (𝑟 2 𝐷 max ). Note that, however, there is no theoretical guarantees for the approximation of the correct solution obtained with this method to the best of our knowledge.

Assumptions

We now precise the framework under which we are going to study the kernel change-point detection procedure in the next chapter.

A key ingredient of our analysis is the concentration of 𝜀. Intuitively, the performance of KCP is better when 𝜀 concentrates strongly around its mean, since without noise we are just given the task to segment a piecewise-constant signal. It is thus natural to make assumptions on 𝜀 in order to obtain concentration results. We actually formulate assumptions on the kernel 𝑘, which translate automatically onto 𝜀.

As in [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], the main hypothesis used in our analysis is the following.

E [ ‖𝜀 𝑖 ‖ 2 ℋ ] = E [𝑘 (𝑋 𝑖 , 𝑋 𝑖 )] -‖𝜇 ⋆ 𝑖 ‖ 2 ℋ ≥ 0 . Hence ‖𝜇 ⋆ 𝑖 ‖ 2 ℋ ≤ E [𝑘 (𝑋 𝑖 , 𝑋 𝑖 )] ≤ 𝑀 2
, and the triangle inequality yields

‖𝜀 𝑖 ‖ ℋ ≤ ‖𝑌 𝑖 ‖ ℋ + ‖𝜇 ⋆ 𝑖 ‖ ℋ ≤ 2𝑀 . (2.11)
This is the reason why we sometimes refer to Assumption 2.1 as a bounded noise rather than bounded kernel. Assumption 2.1 is satisfied for a large class of commonly used kernels, such as the Gaussian, Laplace and 𝜒 2 kernels; 𝑀 = 1 in these three examples.

Note that Assumption 2.1 is weaker than assuming 𝑘 to be bounded -that is, 𝑘(𝑥, 𝑥) ≤ 𝑀 for any 𝑥 ∈ 𝒳 , which is equivalent to 𝑘 (𝑥, 𝑥 ′ ) ≤ 𝑀 for any 𝑥, 𝑥 ′ ∈ 𝒳 since 𝑘 is positive semi-definite. For instance, if 𝒳 = R 𝑝 and the data 𝑋 𝑖 are bounded almost surely, Assumption 2.1 holds true for the linear kernel and all polynomial kernels, which are not bounded on R 𝑝 .

In the setting of Example 2.

1≤𝑖≤𝑛 E [ ‖𝜀 𝑖 ‖ 2 ℋ ] ≤ 𝑉. Since 𝑣 𝑖 := E [ ‖𝜀 𝑖 ‖ 2 ℋ ] = E [𝑘 (𝑋 𝑖 , 𝑋 𝑖 )] -‖𝜇 ⋆ 𝑖 ‖ 2 ℋ , Assumption 2.2 holds true when ∀𝑖 ∈ { 1, . . . , 𝑛 } , E [𝑘 (𝑋 𝑖 , 𝑋 𝑖 )] ≤ 𝑉 .
As a consequence, Assumption 2.1 implies Assumption 2.2 with 𝑉 = 𝑀 2 . Note that Assumption 2.2 is satisfied for the polynomial kernel of order 𝑑 provided that the observations satisfy a moment assumption, namely

∀𝑖 ∈ { 1, . . . , 𝑛 } , E [ ‖𝑋 𝑖 ‖ 2𝑑 ] < +∞ .
In the setting of Example 2.1, Assumption 2.2 holds true with

𝑉 = max 1≤ℓ≤𝐾+1 E 𝑋∼𝑃 ℓ [𝑘 (𝑋, 𝑋)] ,
provided this maximum is finite.

An oracle inequality for KCP

In this section, we recall briefly the oracle inequality obtained by [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF]. This is not exactly a result on change-point estimation, but a guarantee on estimation of the "mean" of the time series in the RKHS associated with the kernel chosen.

Let us define the quadratic risk of any 𝜇 ∈ ℋ 𝑛 as an estimator of 𝜇 ⋆ by

ℛ (𝜇) := 1 𝑛 ‖𝜇 -𝜇 ⋆ ‖ 2 = 1 𝑛 𝑛 ∑ 𝑖=1 ‖𝜇 𝑖 -𝜇 ⋆ 𝑖 ‖ 2 ℋ .
(2.12)

Then the following holds.

Theorem 2.1 [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], Theorem 2). Let 𝐶 be a non-negative constant. Assume that Assumption 2.1 holds true and that pen : 𝒯 𝑛 → R is some penalty function satisfying

∀𝜏 ∈ 𝒯 𝑛 , pen (𝜏 ) ≥ 𝐶𝑀 2 𝑛 [ 𝐷 𝜏 + log ( 𝑛 -1 𝐷 𝜏 -1 )] . (2.13)
Then, some numerical constant 𝐿 1 > 0 exists such that the following holds: if 𝐶 ≥ 𝐿 1 , for every 𝑦 ≥ 0, an event of probability at least 1e -𝑦 exists on which, for every

 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 { ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ) } ,
we have

ℛ (̂ 𝜇 ̂ 𝜏 ) ≤ 2 inf 𝜏 ∈𝒯𝑛 { ℛ (̂ 𝜇 𝜏 ) + pen (𝜏 ) } + 83𝑦𝑀 2 𝑛 .
(2.14)

Informally speaking, Eq. (2.14) means that ̂ 𝜇 ̂ 𝜏 estimates well the mean 𝜇 ⋆ ∈ ℋ 𝑛 of the time series 𝑌 1 , . . . , 𝑌 𝑛 . More precisely, Theorem 2.1 states that, with high probability, the quadratic risk of the least-squares estimator built on ̂ 𝜏 is of the same order than the quadratic risk of the estimator associated to any segmentation, up to a penalty term and a remainder term.

Clearly, Theorem 2.1 applies to ̂ 𝜏 given by (2.2) when pen = pen ACH . If both 𝑐 1 , 𝑐 2 are greater than 2𝐿 1 𝑀 2 , Theorem 2.1 also applies to ̂ 𝜏 given by (2.2) when pen = pen L . Indeed, for any 𝐷 ∈ { 1, . . . , 𝑛 } ,

( 𝑛 -1 𝐷 -1 ) ≤ ( 𝑛 𝐷 ) ≤ ( 𝑛 e 𝐷 ) 𝐷 ,
where the last inequality is standard [see Prop. 2.5 in Massart, 2007, for a stronger result].

A linear penalty pen ℓ also satisfies Eq. (2.13). Indeed, from the last display we deduce that ∀𝑛 ≥ 1, max

1≤𝐷≤𝑛 1 𝐷 ( 𝑛 -1 𝐷 -1 ) ≤ log(𝑛) + 1 . As a consequence, for 𝐶 ≫ (log(𝑛)) -1 , pen ℓ (𝜏 ) ≥ 𝐶𝑀 2 𝑛 ( 𝐷 𝜏 + log ( 𝑛-1 𝐷𝜏 -1
) ) and Theorem 2.1 holds for a linear penalty.

Chapter 3

Consistency of kernel change-point detection

Abstract

In this chapter, we show that, with high probability, KCP with a linear penalty function recovers the correct number of change-points, provided that the penalty constant is well-chosen. In addition, we prove that KCP estimates the change-points at the optimal rate. As a consequence, when using a characteristic kernel, KCP detects all kinds of change in the distribution (not only changes in the mean or the variance), and it is able to do so for complex structured data (not necessarily in R 𝑝 ).

A key point of the proof is a concentration inequality of a quadratic form in a Hilbert space, a refinement of a result obtained in the first version of [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF]. We also show an analogous statement for a different penalty function. Both these results are proved under a boundedness assumption; we prove slightly weaker results under a finite variance assumption. We also discuss in detail the various notions of distances between segmentations, and show that they are all equivalent when one of them is small enough compared to the size of the smallest segment. This chapter is based upon the article [START_REF] Garreau | Consistent change-point detection with kernels[END_REF], under submission to the Electronic Journal of Statistics.

Introduction

At this stage, some key theoretical questions remain open: does KCP estimate correctly the number of change-points and their locations with a large probability? If yes, can we prove a consistency result similar to those introduced in Section 1.3.2?

This chapter answers these questions, showing that KCP has good theoretical properties for change-point estimation with independent data, under a boundedness assumption. (Theorem 3.1, stated for a linear penalty, and Theorem 3.2, for pen L ). These results are non-asymptotic, hence meaningful for high-dimensional or complex data. In the asymptotic setting -with a fixed true segmentation and more and more data points observed within each segment -, Theorem 3.1 implies that KCP estimates consistently all changes in the "kernel mean" of the distribution of data, at speed log(𝑛)/𝑛 with respect to the sample size 𝑛. Since we make no assumptions on the minimal size of the true segments, this matches minimax lower bounds [START_REF] Brunel | Convex set detection[END_REF]. We also provide a partial result under a weaker finite variance assumption (Theorem 3.3 in Section 3.2.3) and explain in Section 3.3 how our proofs could be extended to other settings, including the dependent case. These findings are illustrated by numerical simulations in Section 4.2.

An important case is when KCP is used with a characteristic kernel [START_REF] Fukumizu | Dimensionality reduction for supervised learning with reproducing kernel hilbert spaces[END_REF][START_REF] Fukumizu | Kernel measures of conditional dependence[END_REF], such as the Gaussian or the Laplace kernel. Then, any change in the distribution of data induces a change in the "kernel mean". So, Theorem 3.1 implies that KCP then estimates consistently and at the minimax rate all changes in the distribution of the data, without any parametric assumption and without prior knowledge about the number of changes.

Our results also are interesting regarding to the theoretical understanding of leastsquares change-point procedures. Indeed, when KCP is used with the linear kernel, it reduces to previously known penalized least-squares change-point procedures [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF][START_REF] Comte | A new algorithm for fixed design regression and denoising[END_REF][START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF], for instance]. There are basically two kinds of results on such procedures in the change-point literature: (i) asymptotic statements on change-point estimation [START_REF] Yao | Estimating the number of change-points via Schwarz' criterion[END_REF][START_REF] Yao | Least-squares estimation of a step function[END_REF][START_REF] Bai | Estimating and testing linear models with multiple structural changes[END_REF][START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF]] and (ii) non-asymptotic oracle inequalities [START_REF] Comte | A new algorithm for fixed design regression and denoising[END_REF][START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF][START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], which are based upon concentration inequalities and model selection theory [START_REF] Birgé | Gaussian model selection[END_REF] but do not directly provide guarantees on the estimated change-point locations. Our results and their proofs show how to conciliate the two approaches when we are interested in change-point locations, which is already new for the case of the linear kernel, and also holds for a general kernel. Section 3.2 is dedicated to the exposition of these results. They are discussed in Section 3.3, and the proofs are collected in Section 3.4. Section 3.5 contains the proofs of technical lemmas needed in Section 3.4.

Theoretical guarantees for KCP

We state our main results in this section, which is divided as follows. In Section 3.2.1, we state Theorem 3.1, which provides simple conditions under which KCP recovers the correct number of segments and localizes the true change-points with high probability, under the bounded kernel Assumption 2.1. This is our main result. Theorem 3.1 concerns KCP with a linear penalty: we state in the same section a result for a different penalty, Theorem 3.2. Section 3.2.2 contains a review of the classical losses between segmentations which can be considered in addition to the one used in Theorem 3.1 and 3.2. Corollary 3.1 formulates a result on ̂ 𝜏 in terms of the Frobenius loss. Finally, Section 3.2.3 states a partial result on KCP -requiring the number of change-points 𝐷 ⋆ to be known -under the weaker Assumption 2.2.

Consistency under bounded kernel assumption

We first need to define some quantities. The size of the smallest jump of 𝜇 ⋆ in ℋ is defined by

∆ := min 𝑖 / 𝜇 ⋆ 𝑖 ̸ =𝜇 ⋆ 𝑖+1 ⃦ ⃦ 𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑖+1 ⃦ ⃦ ℋ . (3.1)
Intuitively, the higher ∆ is, the easier it is to detect the smallest jump with our procedure. The quantity

⃦ ⃦ 𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑖+1 ⃦ ⃦
ℋ coincides with the MMD between the distributions of 𝑋 𝑖 and 𝑋 𝑖+1 . In the scalar setting (with the linear kernel), the ratio ∆/𝜎 (where 𝜎 2 is the variance of the noise) is called the signal-to-noise ratio [START_REF] Basseville | Detection of abrupt changes: theory and application, Prentice Hall Information and System Sciences Series[END_REF] and is often used as a measure of the magnitude of a change in the signal. In Example 2.1,

∆ = min 1≤𝑗≤𝐾 ⃦ ⃦ ⃦𝜇 ⋆ 𝑃 𝑗 -𝜇 ⋆ 𝑃 𝑗+1 ⃦ ⃦ ⃦ ℋ ,
where 𝜇 ⋆ 𝑃 𝑗 denotes the (Bochner) expectation of Φ(𝑋) when 𝑋 ∼ 𝑃 𝑗 . For any 𝜏 ∈ 𝒯 𝑛 , we denote the (normalized) sizes of its smallest and of its largest segment by

Λ 𝜏 := 1 𝑛 min 1≤ℓ≤𝐷𝜏 |𝜏 ℓ -𝜏 ℓ-1 | and Λ 𝜏 := 1 𝑛 max 1≤ℓ≤𝐷𝜏 |𝜏 ℓ -𝜏 ℓ-1 | . (3.2)
It should be clear that the smaller Λ 𝜏 ⋆ is, the harder it is to detect the segment that achieves the minimum in (3.2). For instance, in the particular case of Example 2.1,

Λ 𝜏 ⋆ ----→ 𝑛→+∞ min 0≤𝑗≤𝐾 |𝑏 𝑗+1 -𝑏 𝑗 | and Λ 𝜏 ⋆ ----→ 𝑛→+∞ max 0≤𝑗≤𝐾 |𝑏 𝑗+1 -𝑏 𝑗 | .
Finally, for any 𝜏 1 and 𝜏 2 ∈ 𝒯 𝑛 , we define

d (1) ∞ ( 𝜏 1 , 𝜏 2 ) := max 1≤𝑖≤𝐷 𝜏 1 -1 { min 1≤𝑗≤𝐷 𝜏 2 -1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ } ,
which is a loss function (a measure of dissimilarity) between the segmentations 𝜏 1 and 𝜏 2 . Note that d (1) ∞ is not a distance; other possible losses between segmentations and their relationship with d (1) ∞ are discussed in Section 3.2.2. We are now able to state our main result. Theorem 3.1. Suppose that Assumption 2.1 holds true. For any 𝑦 > 0, an event Ω of probability at least 1e -𝑦 exists on which the following holds true. For any 𝐶 > 0, let ̂ 𝜏 be defined by

 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 { crit (𝜏 ) } where crit (𝜏 ) = ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ) , (2.2)
with pen = pen ℓ defined by

pen ℓ (𝜏 ) := 𝐶𝑀 2 𝐷 𝜏 𝑛 . (2.3) Set 𝐶 min := 74 3 (𝐷 ⋆ + 1)(𝑦 + log(𝑛) + 1) and 𝐶 max := ∆ 2 𝑀 2 Λ 𝜏 ⋆ 6𝐷 ⋆ 𝑛 . Then, if 𝐶 min < 𝐶 < 𝐶 max , (3.3)
on Ω, we have

 𝐷 = 𝐷 ⋆ and 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) ≤ 𝑣 1 (𝑦) := 148𝐷 ⋆ 𝑀 2 ∆ 2 • 𝑦 + log(𝑛) + 1 𝑛 .
We delay the proof of Theorem 3.1 to Section 3.4.4. Some remarks follow.

Theorem 3.1 is a non-asymptotic result: it is valid for any 𝑛 ≥ 1 and there is nothing hidden in o (1) remainder terms. The latter point is crucial for complex data -for instance, 𝒳 = R 𝑝 with 𝑝 > 𝑛 -since in this case, assuming 𝒳 fixed while 𝑛 → +∞ is not realistic.

Nevertheless, it is useful to write down what Theorem 3.1 becomes in the asymptotic setting of Example 2.1. As previously noticed, 𝐷 ⋆ , Λ 𝜏 ⋆ , ∆ 2 and 𝑀 2 then converge to positive constants as 𝑛 → +∞. Therefore, 𝐶 min is of order log(𝑛), 𝐶 max is of order 𝑛 and we always have 𝐶 min < 𝐶 max for 𝑛 large enough. The upper bound on 𝐶 matches classical asymptotic conditions for variable selection [START_REF] Shao | An asymptotic theory for linear model selection[END_REF]. The necessity of taking 𝐶 of order at least log(𝑛) is shown by [START_REF] Birgé | Minimal penalties for Gaussian model selection[END_REF] in a variable selection setting, which includes change-point detection as a particular example; [START_REF] Birgé | Minimal penalties for Gaussian model selection[END_REF]; [START_REF] Abramovich | Adapting to unknown sparsity by controlling the false discovery rate[END_REF] provide several arguments for the optimality of taking a constant 𝐶 of order log(𝑛). When 𝐶 satisfies Eq. (3.3), the result of Theorem 3.1 implies that P

( ̂ 𝐷 = 𝐷 ⋆ ) → 1. For the linear kernel in R 𝑑 ,
this is a well-known result when the distribution of the 𝑋 𝑖 changes only through its mean. The first result dates back to Yao [1988, Section 2] for a Gaussian noise, later extended by [START_REF] Liu | On segmented multivariate regression[END_REF] and Bai and Perron [1998, Section 3.1] under mixingale hypothesis on the error, and [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] under very mild assumptions satisfied for a large family of zero-mean processes [for the precise statement of the hypothesis, see Lavielle and Moulines, 2000, Section 2.1]. Theorem 3.1 also shows that the normalized estimated change-points of ̂ 𝜏 converge towards the normalized true change-points at speed at least log(𝑛)/𝑛.

Up to a logarithmic factor, this speed matches the minimax lower bound 𝑛 -1 which has been obtained previously for various change-point procedures [START_REF] Korostelev | On minimax estimation of a discontinuous signal[END_REF][START_REF] Boysen | Consistencies and rates of convergence of jump-penalized least squares estimators[END_REF][START_REF] Korostelev | Minimax theory of image reconstruction[END_REF] for instance] including leastsquares [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF], assuming that Λ 𝜏 ⋆ ≥ 𝜅 > 0. When 𝐷 ⋆ ≥ 3 and the assumption on Λ 𝜏 ⋆ is removed -that is, segments of length much smaller than 𝑛 are allowed, which is compatible with Theorem 3.1 since it is non-asymptotic-, Brunel [2014, Theorem 6] shows a minimax lower bound of order log(𝑛)/𝑛. Therefore, in this setting, KCP achieves the minimax rate. We do not know whether KCP remains minimax optimal (without the log factor) under the assumption Λ 𝜏 ⋆ ≥ 𝜅 > 0.

Note finally that KCP also performs well for finite samples, according to the simulation experiments of [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF]. Theorem 3.1 emphasizes the key role of ∆ 2 /𝑀 2 , which can be seen as a generalization of the signal-to-noise ratio, for the change-point detection performance of KCP. The larger is this ratio, the easier it is to have Eq. (3.3) satisfied and the smaller is 𝑣 1 (𝑦). This suggests to choose 𝑘 (theoretically at least) by maximizing ∆ 2 /𝑀 2 , as we discuss in Section 3.3, and later in Section 4.3. Note that ∆ 2 /𝑀 2 is invariant by a rescaling of 𝑘, hence the result of Theorem 3.1 is unchanged when 𝑘 is rescaled.

The hypothesis in Eq. ( 3.3) is actually three-fold. First, we use that 𝐶 > 𝐶 min to get ̂ 𝐷 ≤ 𝐷 ⋆ . We have to assume 𝐶 large enough since a too small penalty leads to selecting (with KCP or any other penalized least-squares procedure) the segmentation with 𝑛 segments, that is ̂ 𝐷 = 𝑛. Second, 𝐶 < 𝐶 max is used to get ̂ 𝐷 ≥ 𝐷 ⋆ . Such an assumption is required since taking a penalty function too large in Eq. (2.2) would result in selecting the segmentation with only one segment, that is, ̂ 𝐷 = 1. Third, 𝐶 max has to be greater than 𝐶 min for providing a non-empty interval of possible values for 𝐶. This inequality is also used in the proof of the upper bound on d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) when we already know that ̂ 𝐷 = 𝐷 ⋆ . In Example 2.1, the 𝐶 min < 𝐶 max hypothesis translates into Λ 𝜏 ⋆ ≻ log(𝑛)/𝑛. That is, the size of the smallest segment has to be of order log(𝑛)/𝑛. This is known to be a necessary condition to obtain the minimax rate in multiple change-point detection [Brunel, 2014, section 2]. Theorem 3.1 helps choosing 𝐶, which is a key parameter of KCP, as in any penalized model selection procedure. However, in practice, we do not recommend to directly use (3.3) for choosing 𝐶 for two reasons: 𝐶 min , 𝐶 max depend on unknown quantities 𝐷 ⋆ , Λ 𝜏 ⋆ , ∆, and the exact values of the constants in 𝐶 min , 𝐶 max might be pessimistic compared to what we can observe from simulation experiments. We rather suggest to use a data-driven method for choosing 𝐶, see Section 4.1.

Finally, note that if we know 𝐷 ⋆ , we can replace ̂ 𝜏 by

 𝜏 (𝐷 ⋆ ) ∈ arg min 𝜏 ∈𝒯 𝐷 ⋆ 𝑛 { ̂ ℛ 𝑛 (𝜏 ) } .
Then, assuming that Λ 𝜏 ⋆ > 𝑣 1 (𝑦) -which is weaker than assuming 𝐶 min < 𝐶 max -, the proof of Theorem 3.1 shows that, on Ω, we have

1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 (𝐷 ⋆ )) ≤ 𝑣 1 (𝑦) .
We now present an analogue of Theorem 3.1 for another penalty function, pen L .

Theorem 3.2. Suppose that Assumption 2.1 holds true. For any 𝑦 > 0, an event Ω of probability at least 1e -𝑦 exists on which the following holds true. For any 𝑐 1 , 𝑐 2 > 0, let ̂ 𝜏 be defined by

 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 { crit (𝜏 ) } where crit (𝜏 ) = ̂ ℛ 𝑛 (𝜏 ) + pen (𝜏 ) , (2.2)
with pen = pen L defined by

pen L (𝜏 ) := 𝐷 𝜏 𝑛 ( 𝑐 1 log 𝑛 𝐷 𝜏 + 𝑐 2 ) , (2.4) Define 𝑐 1,min := 74𝑀 2 3 (𝐷 ⋆ + 1) and 𝑐 1,max := 𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ (𝑦 + log(𝑛) + 4) - 74𝑀 2 3 (𝐷 ⋆ + 2) .
Set

𝑐 2 := (𝑦 + 4)𝑐 1 + 74𝑀 2 3 (𝐷 ⋆ + 1) (𝑦 + 4) . Then, if 𝑐 1,min < 𝑐 1 < 𝑐 1,max , (3.4)
on Ω, we have

 𝐷 = 𝐷 ⋆ and 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) ≤ 148𝐷 ⋆ 𝑀 2 ∆ 2 • 𝑦 + log(𝑛) + 4 𝑛 =: 𝑣 ′ 1 (𝑦) .
The proof of Theorem 3.2 is postponed to Section 3.4.5.

The remarks made after Theorem 3.1 remain valid up to minor changes. In particular, the bounds given for the penalty constants in Theorem 3.2 do not depend on 𝑛 in the same fashion: the minimal theoretical penalty constant 𝑐 1,min does not depend on 𝑛, as well as the associated 𝑐 2 . This should not come as a surprise, given that we "incorporated" the log(𝑛) factor into the penalty. In some sense, pen L is more natural in the context of change-point detection, as it captures the complexity of the model. However, we want to emphasize that calibrating both 𝑐 1 and 𝑐 2 is not an easy task (see Lebarbier [2002, Chapter 3] for a detailed discussion of the calibration of 𝑐 1 and 𝑐 2 in the real case). Since, again, the constants given by Theorem 3.2 may be very pessimistic, we recommend using pen ℓ for all practical purposes. Also note that the hypothesis (3.4) is threefold, as it was the case for Assumption (3.3) in Theorem 3.1. Indeed, for 𝑐 1,min < 𝑐 1,max to hold, in particular one must have 𝑐 1,min < 𝑐 1,max , which translates into

∆ 2 𝑀 2 > 148𝐷 ⋆ (2𝐷 ⋆ + 3) (𝑦 + log(𝑛) + 4) 𝑛Λ 𝜏 ⋆ ,
after some algebra. Since log(𝑛)/𝑛 → 0, this condition is satisfied for 𝑛 large enough. Finally, as in Theorem 3.1, we recover a speed of convergence of order log(𝑛)/𝑛, which matches the minimax rate, since 𝑣 ′ 1 is 𝑣 1 up to numerical constants.

Notice that all our results are given in terms of d (1) ∞ , which may seem as an arbitrary choice. We discuss in more depth the notion of distances between segmentations in the next section.

Loss functions between segmentations

Theorem 3.1 shows that ̂ 𝜏 is close to 𝜏 ⋆ in terms of d (1) ∞ . Several other loss functions (measures of dissimilarity) can be defined between segmentations [START_REF] Hubert | Comparing partitions[END_REF]. Our goal in this section is to consider a few of them, which are often used or natural for the change-point problem, and to study their relationships. The main result of this section is Lemma 3.1, that states rigorously that all these notions of distance between segmentations are equal, provided they are close enough.

Let us first consider losses related to the Hausdorff distance. For any 𝜏 1 and 𝜏 2 ∈ 𝒯 𝑛 , we define

d (1) ∞ ( 𝜏 1 , 𝜏 2 ) := max 1≤𝑖≤𝐷 𝜏 1 -1 { min 1≤𝑗≤𝐷 𝜏 2 -1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ } d (2) ∞ ( 𝜏 1 , 𝜏 2 ) := max 1≤𝑖≤𝐷 𝜏 1 -1 { min 0≤𝑗≤𝐷 𝜏 2 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ } d (𝑖) H (𝜏 1 , 𝜏 2 ) := max { d (𝑖) ∞ (𝜏 1 , 𝜏 2 ), d (𝑖) ∞ (𝜏 2 , 𝜏 1 ) } for 𝑖 ∈ { 1, 2 } .
The difference between d ( 1)

∞ (𝜏 1 , 𝜏 2 ) and d (2) ∞ (𝜏 1 , 𝜏 2 ) is that d (2)
∞ takes into account the length of 𝜏 1 's first and last segments. As to d

(𝑖) H , it is just a symmetrized version of d (𝑖) ∞ for 𝑖 = 1, 2. Whenever 𝐷 𝜏 1 = 𝐷 𝜏 2 , we define d (3) ∞ ( 𝜏 1 , 𝜏 2 ) := max 1≤𝑖≤𝐷 𝜏 1 -1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑖 ⃒ ⃒ .
Note that d (3) ∞ is symmetric thus there is no need to define d

H . One could also define d [START_REF]Illustration of the kernel-trick[END_REF] H as the Hausdorff distance between the point sets {𝜏 1 1 , . . . , 𝜏 1 𝐷 𝜏 1 -1 } and {𝜏 2 1 , . . . , 𝜏 2 𝐷 𝜏 2 -1 } with respect to the distance 𝛿(𝑥, 𝑦) = |𝑥 -𝑦| on R. These definitions are illustrated by Fig. 3-1.

• • • • • • • • • • • • • • • • • • • 𝜏 1 𝜏 2 d (2) ∞ (𝜏 2 , 𝜏 1 ) d (1) ∞ (𝜏 1 , 𝜏 2 ) d (2) ∞ (𝜏 1 , 𝜏 2 ) Figure 3-1 -Illustration of the definition of d (𝑖) ∞ , with 𝑛 = 19, 𝜏 1 = [ 0, 8, 17, 19 ] and 𝜏 2 = [ 0, 7, 14, 19 ] . In this example, 𝐷 𝜏 1 = 𝐷 𝜏 2 = 3. We can compute d (1) ∞ (𝜏 1 , 𝜏 2 ) = d (1) ∞ (𝜏 2 , 𝜏 1 ) = d (2) ∞ (𝜏 2 , 𝜏 1 ) = d (3) ∞ (𝜏 1 , 𝜏 2 ) = 3 and d (2) ∞ (𝜏 1 , 𝜏 2 ) = 2.
Interestingly, all these loss functions coincide whenever 𝑛 -1 d (1) ∞ (𝜏 1 , 𝜏 2 ) is small enough. The following lemma makes this claim rigorous. Lemma 3.1. We have the following two properties.

(i) For any 𝜏 1 , 𝜏 2 ∈ 𝒯 𝑛 such that

1 𝑛 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) < 1 2 min { Λ 𝜏 1 , Λ 𝜏 2 } , we have 𝐷 𝜏 1 = 𝐷 𝜏 2 and d (1) ∞ ( 𝜏 1 , 𝜏 2 ) = d (2) ∞ ( 𝜏 1 , 𝜏 2 ) = d (3) ∞ ( 𝜏 1 , 𝜏 2 ) = d (1) H ( 𝜏 1 , 𝜏 2 ) = d (2) H ( 𝜏 1 , 𝜏 2 ) .
(ii) For any 𝜏 1 , 𝜏 2 ∈ 𝒯 𝑛 such that

𝐷 𝜏 1 = 𝐷 𝜏 2 and 1 𝑛 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) < Λ 𝜏 1 2 , we have d (1) ∞ ( 𝜏 1 , 𝜏 2 ) = d (1) ∞ ( 𝜏 2 , 𝜏 1 ) = d (1) H ( 𝜏 1 , 𝜏 2 ) .
Lemma 3.1 is proved in Section 3.5.1. As a direct application of Lemma 3.1 we see that the statement of Theorem 3.1 holds true with d (1) ∞ replaced by any of the loss functions that we defined above, at least for 𝑛 large enough.

Another loss between segmentations is the Frobenius loss [START_REF] Lajugie | Large-margin metric learning for constrained partitioning problems[END_REF], which is defined as follows. For any

𝜏 1 , 𝜏 2 ∈ 𝒯 𝑛 , d F ( 𝜏 1 , 𝜏 2 ) := ‖Π 𝜏 1 -Π 𝜏 2 ‖ F ,
where Π 𝜏 is the orthogonal projection onto 𝐹 𝜏 , as defined in Section 2.3.2, and ‖•‖ F denotes the Frobenius norm of a matrix:

∀𝐴 ∈ R 𝑁 ×𝑀 , ‖𝐴‖ 2 F := 𝑁 ∑ 𝑖=1 𝑀 ∑ 𝑗=1 𝐴 2 𝑖𝑗 .
A closed-form formula for d F can be derived from the matrix representation of Π 𝜏 that is given by (2.7): for any 𝑖, 𝑗 ∈ {1, . . . , 𝑛},

(Π 𝜏 ) 𝑖,𝑗 = { 1 |𝜆|
if 𝑖 and 𝑗 belong to the same segment 𝜆 of 𝜏 0

otherwise.

An interesting feature of the Frobenius loss is that it is smaller than 1 only when 𝜏 1 and 𝜏 2 have the same number of segments, whereas Hausdorff distances can be small with very different numbers of segments. Indeed, we prove in Section 3.5.2 that

|𝐷 𝜏 1 -𝐷 𝜏 2 | ≤ d F ( 𝜏 1 , 𝜏 2 ) 2 ≤ 𝐷 𝜏 1 + 𝐷 𝜏 2 . (3.5)
The next proposition shows that there is an equivalence (up to constants) between the Hausdorff and Frobenius losses between segmentations, provided that they are close enough.

Proposition 3.1. Suppose that 𝐷 𝜏 1 = 𝐷 𝜏 2 and 1 𝑛 d (1) ∞ (𝜏 1 , 𝜏 2 ) < Λ 𝜏 1 /2, then ( d F ( 𝜏 1 , 𝜏 2 )) 2 ≤ 12𝐷 𝜏 1 Λ 𝜏 1 • 1 𝑛 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) . If in addition 1 𝑛 d (1) ∞ (𝜏 1 , 𝜏 2 ) < Λ 𝜏 1 /3, then 2 3Λ 𝜏 1 1 𝑛 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) ≤ ( d F ( 𝜏 1 , 𝜏 2 )) 2 .
Prop. 3.1 was first stated and proved by [Lajugie et al., 2014, Theorem B.2]. We prove it in Section 3.5.2 for completeness.

As a corollary of Theorem 3.1 and Prop. 3.1, we get the following guarantee on the Frobenius loss between 𝜏 ⋆ and the segmentation ̂ 𝜏 estimated by KCP.

Corollary 3.1. Under the assumptions of Theorem 3.1, on the event Ω defined by Theorem 3.1, for any ̂ 𝜏 satisfying (2.2) with pen defined by (2.3), we have:

d F (𝜏 ⋆ , ̂ 𝜏 ) ≤ 43𝐷 ⋆ √ Λ 𝜏 ⋆ • 𝑀 ∆ √ 𝑦 + log(𝑛) + 1 𝑛 .
Note that Corollary 3.1 gives a better result (at least for large 𝑛) than the obvious bound

d F (𝜏 ⋆ , ̂ 𝜏 ) ≤ 𝐷 ⋆ + ̂ 𝐷 -2 .
Proof. On the event Ω, we have

1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) < Λ 𝜏 ⋆ /(𝐷 ⋆ + 1)
and 𝐷 ⋆ = ̂ 𝐷. Therefore, according to Prop. 3.1,

( d F (𝜏 ⋆ , ̂ 𝜏 ) ) 2 ≤ 12𝐷 ⋆ Λ 𝜏 ⋆ • 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) ≤ 1776 (𝐷 ⋆ ) 2 (𝑦 + log(𝑛) + 1) 𝑛Λ 𝜏 ⋆ • 𝑀 2 ∆ 2 .
Up to this point, we assessed the quality of the segmentation 𝜏 by considering the proximity of 𝜏 with 𝜏 ⋆ . Another natural idea is to measure the distance between 𝜇 ⋆ and 𝜇 ⋆ 𝜏 in ℋ 𝑛 . It is closely related to the oracle inequality proved by [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF], which implies an upper bound on ‖𝜇 ⋆ -̂ 𝜇 ̂ 𝜏 ‖ 2 . We can also observe that there is a simple relationship between ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 and the Frobenius distance between 𝜏 and 𝜏 ⋆ . Indeed,

‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 = ‖(Π 𝜏 ⋆ -Π 𝜏 )𝜇 ⋆ ‖ 2 ≤ ‖Π 𝜏 ⋆ -Π 𝜏 ‖ 2 2 ‖𝜇 ⋆ ‖ 2 ≤ ( d F (𝜏 ⋆ , ̂ 𝜏 ) ) 2 ‖𝜇 ⋆ ‖ 2 . (3.6)
Eq. (3.17) in the proof of Theorem 3.1 shows that on Ω, under the assumptions of Theorem 3.1,

‖𝜇 ⋆ -𝜇 ⋆  𝜏 ‖ 2 ≤ 74 ( 𝑦 + log(𝑛) + 1 ) 𝐷 ⋆ 𝑀 2
which is slightly better (but similar) to what Corollary 3.1,(3.6) and the bound ‖𝜇 ⋆ ‖ 2 ≤ 𝑀 2 𝑛 imply together.

Extension to the finite variance case

Theorem 3.1 and 3.2 are valid under a boundedness assumption (Assumption 2.1). What happens under the weaker Assumption 2.2? As a first step, we provide a result for

̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) ∈ arg min 𝜏 ∈𝒯 𝐷 ⋆ 𝑛 / Λ 𝜏 ≥𝑛𝛿𝑛 { ̂ ℛ 𝑛 (𝜏 ) } (3.7)
for some 𝛿 𝑛 > 0. In other words, we restrict our search to segmentations 𝜏 of the correct size -hence 𝐷 ⋆ must be known a priori -and having no segment with less than 𝑛𝛿 𝑛 observations. We discuss how to relax this restriction right after the statement of Theorem 3.3. Note that, since we know 𝐷 ⋆ , there is no need for a penalty function in the new problem given by Eq. (3.7). Also notice that the dynamic programming algorithm of [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF] can be used for computing

 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) efficiently.
Similarly to ∆, we define

∆ := max 𝑖 ⃦ ⃦ 𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑖+1 ⃦ ⃦ ℋ . Theorem 3.3.
Suppose that Assumption 2.2 holds true. For any 𝛿 𝑛 , 𝑦 > 0, define:

𝑣 2 (𝑦, 𝛿 𝑛 ) := 24(𝐷 ⋆ ) 2 ∆ √ 𝑉 ∆ 2 𝑦 √ 𝑛 + 8𝐷 ⋆ 𝑉 ∆ 2 𝑦 2 𝑛𝛿 𝑛 .
For any 𝑦 > 0, an event Ω 2 exists such that

P (Ω 2 ) ≥ 1 - 1 𝑦 2
and, on Ω 2 , we have the following: for any 𝛿 𝑛 ∈ (0, Λ 𝜏 ⋆ ] and any ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) satisfying Eq.

(3.7), if 𝑣 2 (𝑦, 𝛿 𝑛 ) ≤ Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 )) ≤ 𝑣 2 (𝑦, 𝛿 𝑛 ) . (3.8)
We postpone the proof of Theorem 3.3 to Section 3.4.6. Let us make a few remarks.

As for Theorem 3.1, our result is non-asymptotic. However, it is interesting to write it down in the setting of Example 2.1. If 𝑛 goes to infinity, then the assumption Λ 𝜏 ⋆ ≥ 𝛿 𝑛 is satisfied whenever 𝛿 𝑛 → 0. If we furthermore require that 𝑛𝛿 𝑛 → ∞, then Eq. (3.8) implies that

1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 )) P ----→ 𝑛→+∞ 0 ,
by taking a well-chosen 𝑦 of order √ 𝑛 + √ 𝑛𝛿 𝑛 . In the particular case of the linear kernel, this result is known under various hypothesis [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] for instance]; it is new for a general kernel.

More precisely, if we take 𝛿 𝑛 = 𝑛 -1/2 , Theorem 3.3 implies that

1 𝑛 d (1) ∞ ( 𝜏 ⋆ , ̂ 𝜏 (𝐷 ⋆ , 𝑛 -1/2 ) )
goes to zero at least as fast as ℓ 𝑛 / √ 𝑛, where (ℓ 𝑛 ) 𝑛≥1 is any sequence tending to infinity, for instance ℓ 𝑛 = log(𝑛). This speed seems suboptimal compared to previous results [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] for instance] -which do not consider the case of a general kernel -, but we have not been able to prove tight enough deviation bounds for getting the localization rate log(𝑛)/𝑛 under Assumption 2.2. How does Theorem 3.3 compares to Theorem 3.1 and 3.2? First, as noticed in Remark 3.4 in Section 3.4.4, the result of Theorem 3.1 also holds true for ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) as long as Λ 𝜏 ⋆ ≥ 𝛿 𝑛 . Second, 𝑣 1 (𝑦) is usually smaller than 𝑣 2 (𝑦, 𝛿 𝑛 ) -its order of magnitude is smaller when 𝑛 → +∞ -, and the lower bound on the probability of Ω is better than the one for Ω 2 . There is no surprise here: the stronger Assumption 2.1 helps us proving a stronger result for ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ). Nevertheless, these only are upper bounds, so we do not know whether the performance of ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) actually changes much depending on the noise assumption. For instance, as already noticed, we do not believe that the localization speed log(𝑛)/𝑛 requires a boundedness assumption; in particular cases at least, it has been obtained for unbounded data [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF][START_REF] Boysen | Consistencies and rates of convergence of jump-penalized least squares estimators[END_REF].

The dependency in 𝑘 of the speed of convergence of ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ) is slightly less clear than in Theorem 3.1. The signal-to-noise ratio appears through ∆ 2 /𝑉 , as expected, but the size ∆ of the largest true jump also appears in 𝑣 2 . At the very least, it is clear that ∆ 2 /𝑉 should not be too small.

As noted by [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF], it may be possible to get rid of the minimal segment length 𝛿 𝑛 , either by imposing stronger conditions on 𝜀 -which are not met in our setting -or by constraining the values of ̂ 𝜇 to lie in a compact subset Θ ⊂ ℋ 𝐷 ⋆ +1 .

Discussion

Before proving our main results, let us discuss some of their consequences regarding the KCP procedure.

Fully non-parametric consistent change-point detection. We have proved that for any kernel satisfying some reasonably mild hypotheses, the KCP procedure outputs a segmentation close by the true segmentation with high probability.

An important particular example is the "asymptotic setting" of Example 2.1, where we have a fixed true segmentation 𝜏 ⋆ and fixed distributions 𝑃 1 , . . . , 𝑃 𝐾+1 from which more and more points are sampled. How fast can KCP recover 𝜏 ⋆ , without any prior information on the number of segments 𝐷 ⋆ or on the distributions 𝑃 1 , . . . , 𝑃 𝐾+1 ?

Let us take a bounded characteristic kernel -for instance the Gaussian or the Laplace kernel if 𝒳 = R 𝑑 -, so that Assumption 2.1 holds true. Then, Theorem 3.1 shows that KCP detects consistently all changes in the distribution of the 𝑋 𝑖 , and localizes them at speed log(𝑛)/𝑛. This speed also depends on the kernel 𝑘 and the size of the differences between the 𝑃 𝑗 , through the ratio ∆ 2 /𝑀 2 . Obtaining such a fully non-parametric result for multiple change-points with a general set 𝒳 -we only need to know a bounded characteristic kernel on 𝒳 -has never been obtained before.

To the best of our knowledge, non-parametric consistency results for the detection of arbitrary changes in the distribution of the data have only been obtained for realvalued data [START_REF] Zou | Nonparametric maximum likelihood approach to multiple change-point problems[END_REF] or for the case of a single change-point [START_REF] Carlstein | Nonparametric change-point estimation[END_REF][START_REF] Brodsky | Nonparametric methods in change point problems[END_REF].

Choice of 𝑘. An important question remains: how to choose the kernel 𝑘? In Theorem 3.1, 𝑘 only appears through the "signal-to-noise ratio" ∆ 2 /𝑀 2 , leading to better theoretical guarantees when this signal-to-noise ratio is larger: a larger value for 𝐶 max and a smaller bound 𝑣 1 on d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ). Therefore, a simple strategy for choosing the kernel is to pick 𝑘 that maximizes ∆ 2 /𝑀 2 , at least among a family of kernels, for instance Gaussian kernels. This first idea requires to know the distributions of the 𝑋 𝑖 , or at least to have prior information on them. As we have noticed, when the change-points locations are known, ∆ 2 corresponds to the minimum of the MMDs between the distributions of the 𝑋 𝑖 over contiguous segments. In this particular setting, it may be feasible to estimate and to maximize ∆ 2 with respect to the kernel 𝑘, as done by Gretton et al. [2012b]. This question is addressed in more details in Section 4.3.1. An interesting future development would be to build an estimator of ∆ 2 without knowing the change-point locations and to maximize this estimator with respect to the kernel 𝑘. We refer to Arlot et al. [2012, section 7.2] for a complementary discussion about the choice of 𝑘 for KCP.

Choice of 𝐶. Another important parameter of the KCP procedure is the constant 𝐶 that appears in the linear penalty function. As mentioned below Theorem 3.1, our theoretical guarantees provide some guidelines for choosing 𝐶, but these are not sufficient to choose precisely 𝐶 in practice. We recommend to follow the advice of [Arlot et al., 2012, section 6.2] on this point, which is to choose 𝐶 from data with the "dimension jump" heuristic [START_REF] Baudry | Slope heuristics: overview and implementation[END_REF]. The exact procedure is explained in details in Section 4.1.

Modularity of the proofs and possible extensions. Finally, we would like to emphasize what we believe to be an important contribution of this thesis. The structure of the proofs of Theorems 3.1 and 3.3 -which follow the same strategy -is modular, so that one can easily adapt it to different sets of assumptions. This is also the case for the proof of Theorem 3.2, to some extent: the behavior of the penalty function near 𝐷 ⋆ has to be controlled more precisely.

Our proof strategy is not fully new, since it is similar to the one of almost all previous papers analyzing the consistency of least-squares change-point detection procedures. In particular, we adapted some ideas of the proofs of [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] to the Hilbert space setting. Nevertheless, these papers formulate their main results in asymptotic terms, which can be seen as a limitation -especially when 𝑛 is small or 𝒳 is of large dimension. Another approach is the one of [START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF]; [START_REF] Comte | A new algorithm for fixed design regression and denoising[END_REF]; [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] where non-asymptotic oracle inequalities -using concentration inequalities and following the model selection results of [START_REF] Birgé | Gaussian model selection[END_REF] -are provided as theoretical guarantees on some penalized least-squares change-point procedures. Up to now, these two approaches seemed difficult to combine. The proofs of Theorems 3.1 and 3.3 show how they can be reconciled, which allows us to mix their strengths.

Indeed, assumptions on the distributions of the 𝑋 𝑖 -Assumptions 2.1 and 2.2 -are only used for proving bounds on two quantities depending on 𝜀 -a linear term 𝐿 𝜏 and a quadratic term 𝑄 𝜏 -, uniformly over 𝜏 ∈ 𝒯 𝑛 . Under Assumption 2.1, this is done thanks to concentration inequalities (Lemmas 3.9 and 3.8) which have been proved first by [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] in order to get an oracle inequality. Under Assumption 2.2, this is done by generalizing the method of [START_REF] Lavielle | Least-squares estimation of an unknown number of shifts in a time series[END_REF] to Hilbert-space valued data, through two deterministic bounds (Lemmas 3.6 and 3.7) and a deviation inequality for 3.11). The rest of the proofs does not use any information about the distribution of 𝑋 1 , . . . , 𝑋 𝑛 .

𝐵 𝑛 := max 1≤𝑘≤𝑛 ⃦ ⃦ ⃦ ⃦ ⃦ 𝑘 ∑ 𝑗=1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ (Lemma
As a consequence, if one can generalize these bounds to another setting, a straightforward consequence is that a result similar to Theorem 3.1, 3.2, or 3.3 holds true for the KCP procedure in this new setting. In particular, this could be used for dealing with the case of dependent data 𝑋 1 , . . . , 𝑋 𝑛 . We could also consider an intermediate assumption between Assumption 2.2 and Assumption 2.1, of the form:

max 1≤𝑖≤𝑛 E [ 𝑘(𝑋 𝑖 , 𝑋 𝑖 ) 𝛼 ] ≤ 𝐵 𝛼 < +∞ ,
for some 𝛼 ∈ (1, +∞).

Without further ado, we now turn to the proofs of Theorems 3.1, 3.2 and 3.3.

Proofs

Let us start by describing our general strategy for proving Theorems 3.1 and 3.3. Our goal is to build a large probability event on which any ̂ 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 crit (𝜏 ) belongs to some subset ℰ of 𝒯 𝑛 . For proving this, we use the key fact that crit (𝜏 ⋆ ) ≥ crit (̂ 𝜏 ), together with a lower bound on crit (𝜏 ) holding simultaneously for all 𝜏 ∈ 𝒯 𝑛 -hence for 𝜏 = ̂ 𝜏 . In order to get such a lower bound on the empirical penalized criterion, we start by decomposing it in Section 3.4.1 into terms that are simpler to control individually: two random terms -a linear function of 𝜀 and a quadratic function of 𝜀 -, and two deterministic terms -the approximation error and the penalty. Then, we control these terms thanks to deterministic bounds (Section 3.4.2) and deviation/concentration inequalities (Section 3.4.3). Finally, we prove Theorem 3.1 in Section 3.4.4 and Theorem 3.3 in Section 3.4.6.

The proof of Theorem 3.2 shares the same ideas, but is slightly different due to the more complicated form of the penalty. It can be found in Section 3.4.5.

Decomposition of the empirical risk

The first step in the proofs of Theorems 3.1, 3.2 and 3.3 is to decompose the empirical risk (2.8). Lemma 3.2. Let 𝜏 ∈ 𝒯 𝑛 be a segmentation. Define 𝜇 ⋆ 𝜏 = Π 𝜏 𝜇 ⋆ . Then we can write

𝑛 ̂ ℛ 𝑛 (𝜏 ) = ‖𝑌 -̂ 𝜇 𝜏 ‖ 2 = ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 + 2⟨𝜇 ⋆ -𝜇 ⋆ 𝜏 , 𝜀⟩ -‖Π 𝜏 𝜀‖ 2 + ‖𝜀‖ 2 . (3.9)
Proof. First, recall that ̂ 𝜇 𝜏 = Π 𝜏 𝑌 and that 𝑌 = 𝜇 ⋆ + 𝜀, hence

‖𝑌 -̂ 𝜇 𝜏 ‖ 2 = ‖𝑌 -Π 𝜏 𝑌 ‖ 2 = ‖𝜇 ⋆ + 𝜀 -Π 𝜏 (𝜇 ⋆ + 𝜀)‖ 2 = ‖𝜇 ⋆ -Π 𝜏 𝜇 ⋆ ‖ 2 + ‖𝜀 -Π 𝜏 𝜀‖ 2 + 2⟨𝜇 ⋆ -Π 𝜏 𝜇 ⋆ , 𝜀 -Π 𝜏 𝜀⟩.
Since Π 𝜏 is an orthogonal projection,

‖𝑌 -̂ 𝜇 𝜏 ‖ 2 = ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 + ‖𝜀‖ 2 -2⟨𝜀, Π 𝜏 𝜀⟩ + ‖Π 𝜏 𝜀‖ 2 + 2⟨(I -Π 𝜏 )𝜇 ⋆ , 𝜀⟩ = ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 + ‖𝜀‖ 2 -‖Π 𝜏 𝜀‖ 2 + 2⟨(I -Π 𝜏 )𝜇 ⋆ , 𝜀⟩ .
Since each term of Eq. (3.9) behaves differently and is controlled via different techniques depending on the result to be proven, we name each of these terms:

𝐿 𝜏 := ⟨𝜇 ⋆ -𝜇 ⋆ 𝜏 , 𝜀⟩ , 𝑄 𝜏 := ‖Π 𝜏 𝜀‖ 2 and 𝐴 𝜏 := ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 . (3.10)
It should be clear that 𝐿 stands for "linear", 𝑄 stands for "quadratic" and 𝐴 stands for "approximation error". We also define

𝜓 𝜏 := 2𝐿 𝜏 -𝑄 𝜏 + 𝐴 𝜏 . (3.11)
Therefore a reformulation of Lemma 3.2 is

𝑛 ̂ ℛ 𝑛 (𝜏 ) = 𝜓 𝜏 + ‖𝜀‖ 2 .
Notice that 𝐿 𝜏 ⋆ = 𝐴 𝜏 ⋆ = 0 and 𝑄 𝜏 ⋆ ≥ 0, hence 𝜓 𝜏 ⋆ ≤ 0. Also note that 𝜓, 𝐿 and 𝑄 are random quantities depending on 𝜀.

Deterministic bounds

In this section, we provide some deterministic bounds that are used in the proofs of Theorems 3.1, 3.2 and 3.3.

Approximation error 𝐴 𝜏

We begin by the following result, which is the reason for the Λ 𝜏 ⋆ ∆ 2 term in the minimal penalty constants in Theorem 3.1 and 3.2. Lemma 3.3. Let 𝜏 ∈ 𝒯 𝑛 be a segmentation such that 𝐷 := 𝐷 𝜏 < 𝐷 ⋆ . Then

1 𝑛 𝐴 𝜏 = 1 𝑛 ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ 1 2 Λ 𝜏 ⋆ ∆ 2 . (3.
12)

The proof of Lemma 3.3 can be found in Section 3.5.3.

Remark 3.1. The inequality in Lemma 3.3 is tight. Indeed, consider the simple case 𝐷 𝜏 = 1 and 𝐷 ⋆ = 2. Assume that 𝑛 = 2𝑚 is an even number, and let 𝜏 ⋆ 1 = 𝑚. It follows from definitions (3.1) and (3.2) that, in this case,

∆ = ‖𝜇 ⋆ 1 -𝜇 ⋆ 𝑛 ‖ ℋ and Λ 𝜏 ⋆ = 1 2 .
According to Eq. (2.7), (𝜇 ⋆ 𝜏 ) 𝑖 = 1 2 (𝜇 ⋆ 1 + 𝜇 ⋆ 𝑛 ), which yields

1 𝑛 𝐴 𝜏 = 1 4 ‖𝜇 ⋆ 1 -𝜇 ⋆ 𝑛 ‖ 2 ℋ = 1 2 Λ 𝜏 ⋆ ∆ 2 .
Thus, in this particular class of examples, equality holds in (3.12).

We next state an analogous result, valid for any 𝜏 ∈ 𝒯 𝑛 , which plays a key role in the proofs of Theorems 3.1, 3.2 and 3.3. Lemma 3.4. For any 𝜏 ∈ 𝒯 𝑛 ,

1 𝑛 𝐴 𝜏 ≥ 1 2 min { Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , 𝜏 ) } ∆ 2 . (3.13)
Lemma 3.4 is proved in Section 3.5.4.

Linear term 𝐿 𝜏 and quadratic term 𝑄 𝜏

The proof of Theorem 3.3 relies on some deterministic bounds on 𝐿 𝜏 and 𝑄 𝜏 . We start with a preliminary lemma. Lemma 3.5. For any 𝜀 1 , . . . , 𝜀 𝑛 ∈ ℋ,

1 2 max 1≤𝑎<𝑏≤𝑛 ⃦ ⃦ ⃦ ⃦ ⃦ 𝑏 ∑ 𝑗=𝑎 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≤ max 1≤𝑘≤𝑛 ⃦ ⃦ ⃦ ⃦ ⃦ 𝑘 ∑ 𝑗=1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ =: 𝐵 𝑛 . (3.14)
Proof. For every 𝑎 < 𝑏, we have:

⃦ ⃦ ⃦ ⃦ ⃦ 𝑏 ∑ 𝑗=𝑎 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ = ⃦ ⃦ ⃦ ⃦ ⃦ 𝑏 ∑ 𝑗=1 𝜀 𝑗 - 𝑎-1 ∑ 𝑗=1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≤ ⃦ ⃦ ⃦ ⃦ ⃦ 𝑏 ∑ 𝑗=1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ + ⃦ ⃦ ⃦ ⃦ ⃦ 𝑎-1 ∑ 𝑗=1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≤ 2𝐵 𝑛 .
The following result is a deterministic bound on 𝑄 𝜏 in terms of 𝐵 𝑛 .

Lemma 3.6. Let 𝜏 ∈ 𝒯 𝑛 be a segmentation. Then

𝑄 𝜏 ≤ 4𝐷 𝜏 𝐵 2 𝑛 𝑛Λ 𝜏 .
Proof. By Eq. (2.7), for any

𝜏 ℓ-1 + 1 ≤ 𝑖 ≤ 𝜏 ℓ , (Π 𝜏 𝜀) 𝑖 = 1 |𝜏 ℓ -𝜏 ℓ-1 | 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝜀 𝑗 . Since 𝑄 𝜏 = ‖Π 𝜏 𝜀‖ 2 , 𝑄 𝜏 = 𝑛 ∑ 𝑖=1 ‖(Π 𝜏 𝜀) 𝑖 ‖ 2 ℋ = 𝐷𝜏 ∑ ℓ=1 𝜏 ℓ ∑ 𝑖=𝜏 ℓ-1 +1 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 1 |𝜏 ℓ -𝜏 ℓ-1 | 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 = 𝐷𝜏 ∑ ℓ=1 1 |𝜏 ℓ -𝜏 ℓ-1 | ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 ℋ ≤ 𝐷 𝜏 max 1≤ℓ≤𝐷𝜏 ⎧ ⎨ ⎩ 1 |𝜏 ℓ -𝜏 ℓ-1 | ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 ℋ ⎫ ⎬ ⎭ ≤ 𝐷 𝜏 𝑛Λ 𝜏 max 1≤ℓ≤𝐷𝜏 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 ℋ 𝑄 𝜏 ≤ 4𝐷 𝜏 𝑛Λ 𝜏 𝐵 2 𝑛 ,
where we used Lemma 3.5 for the last inequality.

The following result is a deterministic bound on 𝐿 𝜏 .

Lemma 3.7. For any 𝜏 ∈ 𝒯 𝑛 ,

|𝐿 𝜏 | ≤ 6𝐷 ⋆ max {𝐷 ⋆ , 𝐷 𝜏 } ∆𝐵 𝑛 .
Lemma 3.7 is proved in Section 3.5.5.

Concentration

In this subsection, we present concentration results on 𝑄 𝜏 , 𝐿 𝜏 , and deviation bounds for 𝐵 𝑛 -which will imply deviation bounds on 𝑄 𝜏 and 𝐿 𝜏 by Lemmas 3.6 and 3.7). For any 𝑗 ∈ {1, . . . , 𝑛}, 𝜏 ∈ 𝒯 𝑛 and ℓ ∈ {1, . . . , 𝐷 𝜏 }, we define

𝑣 𝑗 := E [ ‖𝜀 𝑗 ‖ 2 ℋ ] 𝑣 𝜏,ℓ := 1 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 𝑣 𝑗 and 𝑣 𝜏 := 𝐷 ∑ ℓ=1 𝑣 𝜏,ℓ .
Concentration under Assumption 2.1. We present in this section concentration results that we already identified as essential to the proofs of Theorems 3.1 and 3.2.

The first result that we provide1 in this section concerns the quadratic term 𝑄 𝜏 when Assumption 2.1 is satisfied. Lemma 3.8. Suppose that Assumption 2.1 holds true. Then for any 𝑥 > 0, with probability at least 1e -𝑥 ,

𝑄 𝜏 -𝑣 𝜏 ≤ ( 𝑥 + 2 √ 2𝑥𝐷 𝜏 ) 14𝑀 2 3 . (3.15)
We want to emphasize that obtaining concentration results for 𝑄 𝜏 = ‖Π 𝜏 𝜀‖ 2 is not an easy task at first sight, since 𝜀 𝑗 belongs to ℋ, a potentially infinite-dimensional space. A way to tackle the challenge of concentrating 𝑄 𝜏 is to use concentration results for 𝑈 -statistics of order 2 with values in a general set [Giné and Nickl, 2015, Th. 3.4.8], but in this case a term of order 𝑀 2 𝑥 2 would appear in the right-hand side of Eq. (3.15). Another would be the use of Talagrand's inequality [Boucheron et al., 2013, Cor. 12.12], leading to the same deviation term of order 𝑀 2 𝑥 2 . But we need a deviation term of order 𝑀 2 𝑥 for our proof machinery to operate. We refer to Arlot et al. [2012, Section 5.4.3] for a detailed discussion of such matters and to [START_REF] Ledoux | Probability in Banach Spaces: isoperimetry and processes[END_REF] for related questions regarding concentration of random variables in Banach spaces. Before turning to the proof of Lemma 3.8, we recall Bernstein's inequality and Pinelis-Sakhanenko's inequality, two results that play a crucial role in this proof.

Bernstein's inequality is a cornerstone in concentration inequality theory. It provides sub-exponential concentration bounds for a sum of independent random variables under an hypothesis on the growth of the moments of this sum. First proved by Bernstein in the 20s [START_REF] Bernstein | On a modification of chebyshev's inequality and of the error formula of laplace[END_REF], it implies as a special cases standard tools in concentration, e.g., Bennett's inequality. We give here the version that can be found in [START_REF] Boucheron | Concentration inequalities: A nonasymptotic theory of independence[END_REF], which is a bit more general than the classical form. Proposition 3.2 (Bernstein's inequality). Let 𝑍 1 , . . . , 𝑍 𝑚 be independent real-valued random variables. Assume that there exist some positive constants 𝑣 and 𝑐 such that

∀𝑞 ≥ 2, 𝑚 ∑ 𝑖=1 E [|𝑍 𝑖 | 𝑞 ] ≤ 𝑞! 2 𝑣𝑐 𝑞-2 .
Then, for every 𝑥 > 0,

P ( 𝑚 ∑ 𝑖=1 (𝑍 𝑖 -E [𝑍 𝑖 ]) > √ 2𝑣𝑥 + 𝑐𝑥 ) ≤ e -𝑥 .
Pinelis-Sakhanenko's inequality [Pinelis and Sakhanenko, 1986, Cor. 1] is close in spirit to Bernstein's inequality, with the noteworthy difference that it concerns a sum of Hilbert-valued random variables. Proposition 3.3 (Pinelis-Sakhanenko's inequality). Let 𝑍 1 , . . . , 𝑍 𝑚 be independent random variables with values in some Hilbert space H. Assume that the 𝑍 𝑖 are centered and that there exists some positive constants 𝑣 and 𝑐 such that

∀𝑞 ≥ 2, 𝑚 ∑ 𝑖=1 E [ ‖𝑍 𝑖 ‖ 𝑞 H ] ≤ 𝑞! 2 𝑣𝑐 𝑞-2 .
Then, for every 𝑥 > 0,

P (⃦ ⃦ ⃦ ⃦ ⃦ 𝑚 ∑ 𝑖=1 𝑍 𝑖 ⃦ ⃦ ⃦ ⃦ ⃦ H > 𝑥 ) ≤ 2 exp ( -𝑥 2 2(𝑐𝑥 + 𝑣)
) .

We are now ready to prove Lemma 3.8.

Proof of Lemma 3.8. Let us define

𝑇 𝜆 := 1 |𝜆| ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 𝜀 𝑗 ⃦ ⃦ ⃦ 2 ℋ
. Remark that (𝑇 𝜆 ) 𝜆 is a sequence of independent real-valued random variables. Since 𝑄 𝜏 = ∑ 𝜆∈𝜏 𝑇 𝜆 , we can obtain a concentration inequality for 𝑄 𝜏 via Bernstein's inequality as long as 𝑇 𝜆 satisfies some moment conditions. We will use the Pinelis-Sakhanenko's deviation inequality to prove such bounds.

By the independence property of the 𝜀 𝑗 s, for any 𝜆 ∈ 𝜏 , we have E [𝑇 𝜆 ] = 𝑣 𝜆 . Then, for any 𝑞 ≥ 2,

E [𝑇 𝑞 𝜆 ] = 1 |𝜆| 𝑞 E ⎡ ⎣ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ 2𝑞 ℋ ⎤ ⎦ = 1 |𝜆| 𝑞 ∫ 2|𝜆|𝑀 0 2𝑞𝑥 2𝑞-1 P ⎛ ⎝ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≥ 𝑥 ⎞ ⎠ d 𝑥 ,
because for any 𝑗, ‖𝜀 𝑗 ‖ ℋ ≤ 2𝑀 almost surely -cf. Eq. 2.11. The boundedness of ‖𝜀 𝑗 ‖ ℋ also implies that, for any 𝑝 ≥ 2 and 𝜆 ∈ 𝜏 ,

∑ 𝑗∈𝜆 E [ ‖𝜀 𝑗 ‖ 𝑝 ℋ ] ≤ (2𝑀 ) 𝑝-2 ∑ 𝑗∈𝜆 𝑣 𝑗 ≤ 𝑝! 2 ( ∑ 𝑗∈𝜆 𝑣 𝑗 ) ( 2𝑀
3

) 𝑝-2 ≤ 𝑝! 2 × |𝜆| 𝑀 2 × ( 2𝑀 3 ) 𝑝-2 ,
that is, the assumptions of Pinelis-Sakhanenko's deviation inequality hold true with 𝑐 = 2𝑀/3 and 𝑣 = |𝜆| 𝑀 2 . Therefore, for any 𝑥 ∈ [0, 2 |𝜆| 𝑀 ],

P ⎛ ⎝ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≥ 𝑥 ⎞ ⎠ ≤ 2 exp ( -𝑥 2 2 ( |𝜆| 𝑀 2 + 2𝑀 𝑥 3 ) ) ≤ 2 exp ( -3𝑥 2 14 |𝜆| 𝑀 2
) .

We now make use of the change of variables 𝑢 = √ 3/(7 |𝜆|)𝑥/𝑀 , and write

E [𝑇 𝑞 𝜆 ] ≤ 4𝑞 |𝜆| 𝑞 ∫ 2|𝜆|𝑀 0 𝑥 2𝑞-1 exp ( -3𝑥 2 14 |𝜆| 𝑀 2 ) d 𝑥 ≤ 4𝑞 ( 7𝑀 2 3 ) 𝑞 ∫ +∞ 0 𝑢 2𝑞-1 e -𝑢 2 /2 d 𝑢 ≤ 2(𝑞!) ( 14𝑀 2 3 ) 𝑞 ,
where we used

∫ +∞ 0 𝑢 2𝑞-1 e -𝑢 2 /2 d 𝑢 = 2 𝑞-1 (𝑞 -1)! .
Summing over 𝜆 ∈ 𝜏 , it comes

∑ 𝜆∈𝜏 E [𝑇 𝑞 𝜆 ] ≤ 2(𝑞!) ( 14𝑀 2 3 ) 𝑞 𝐷 𝜏 ≤ 𝑞! 2 × 𝐷 𝜏 ( 28𝑀 2 3 ) 2 × ( 14𝑀 2 3 ) 𝑞-2 .
Thus the condition of Bernstein's inequality holds with

𝑣 = 𝐷 𝜏 ( 28𝑀 2 3 ) 2 and 𝑐 = 14𝑀 2 3 .
Hence with probability at least 1e -𝑥 ,

𝑄 𝜏 -E [𝑄 𝜏 ] ≤ √ 2𝑣𝑥 + 𝑐𝑥 = √ 2𝐷 𝜏 𝑥 28𝑀 2 3 + 14𝑀 2 3 𝑥 = 14𝑀 2 3 ( 2 √ 2𝐷 𝜏 𝑥 + 𝑥 ) .
The linear term 𝐿 𝜏 can be controlled directly via Bernstein's inequality. This is achieved in the next lemma, which is a specialization of Arlot et al. [2012, Prop. 3] under Assumption 2.1.

Lemma 3.9. Suppose that Assumption 2.1 holds true. Then for any 𝑥 > 0, with probability at least 1 -2 e -𝑥 , for any 𝜃 > 0,

|𝐿 𝜏 | ≤ 𝜃𝐴 𝜏 + ( 4 3 + 1 2𝜃
) 𝑀 2 𝑥 .

Proof. Let us define 𝑆 𝜏 := ⟨𝜇 ⋆ -𝜇 ⋆ 𝜏 , 𝜀⟩. We note that

𝑆 𝜏 = 𝑛 ∑ 𝑖=1 𝑍 𝑖 with 𝑍 𝑖 := ⟨(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 , 𝜀 𝑖 ⟩ ℋ .
The 𝑍 𝑖 s are independent centered real-valued random variables. Let us prove that they satisfy the hypothesis of Bernstein's inequality.

Set 𝑖 ∈ { 1, . . . , 𝑛 } , the Cauchy-Schwarz inequality yields

|𝑍 𝑖 | ≤ ‖(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 ‖ ℋ • ‖𝜀 𝑖 ‖ ℋ .
We have already proved that ‖𝜀 𝑖 ‖ ℋ ≤ 2𝑀 . According to Eq. (2.7),

(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 = 𝜇 ⋆ 𝑖 - 1 𝜏 ℓ -𝜏 ℓ-1 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1
𝜇 ⋆ 𝑗 for some 1 ≤ ℓ ≤ 𝐷 𝜏 , and thus we can write

‖(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 ‖ ℋ = 1 𝜏 ℓ -𝜏 ℓ-1 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 𝜏 ℓ ∑ 𝑗=𝜏 ℓ-1 +1 ( 𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑗 ) ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≤ sup 𝜏 ℓ-1 <𝑗≤𝜏 ℓ ⃦ ⃦ 𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑗 ⃦ ⃦ ℋ .
Triangle inequality together with Eq. (2.11) yields ‖(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 ‖ ℋ ≤ 2𝑀 , and we have proved that

|𝑍 𝑖 | ≤ 4𝑀 2 .
Furthermore, again due to Cauchy-Schwarz inequality,

E [ |𝑍 𝑖 | 2 ] ≤ E [ ‖(𝜇 ⋆ -𝜇 ⋆ 𝜏 ) 𝑖 ‖ 2 ℋ • ‖𝜀 𝑖 ‖ 2 ℋ ] . Recall that under Assumption 2.1, E [ ‖𝜀 𝑖 ‖ 2 ℋ ] ≤ 𝑀 2 . Thus 𝑛 ∑ 𝑖=1 E [ |𝑍 𝑖 | 2 ] ≤ ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 • max 𝑖 E [ ‖𝜀 𝑖 ‖ 2 ℋ ] ≤ ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 • 𝑀 2 .
We now show that the conditions of Bernstein's inequality are satisfied. Let 𝑞 ≥ 2,

then 𝑛 ∑ 𝑖=1 E [|𝑍 𝑖 | 𝑞 ] ≤ ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 𝑀 2 • ( 4𝑀 2 3 ) 𝑞-2 = ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 𝑀 2 • 3 𝑞-2 • ( 4𝑀 2 3 ) 𝑞-2 . Since 3 𝑞-2 ≤ 𝑞!/2 for any 𝑞 ≥ 2, 𝑛 ∑ 𝑖=1 E [|𝑍 𝑖 | 𝑞 ] ≤ 𝑞! 2 ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 𝑀 2 ( 4𝑀 2 3 ) 𝑞-2 ,
and the conditions of Bernstein's inequality are satisfied with 𝑣 = ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 𝑀 2 and 𝑐 = 4𝑀 2 /3. As a consequence, for any 𝑥 > 0, with probability higher than 1 -2 e -𝑥 ,

⃒ ⃒ ⃒ ⃒ ⃒ 𝑛 ∑ 𝑖=1 𝑍 𝑖 ⃒ ⃒ ⃒ ⃒ ⃒ ≤ √ 2 ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 𝑀 2 𝑥 + 4𝑀 2 3 𝑥 .
We conclude the proof by applying the inequality

2𝑎𝑏 ≤ 𝜃𝑎 2 + 𝜃 -1 𝑏 2 to 𝑎 = √ 𝐴 𝜏 and 𝑏 = √ 2 2 𝑀 √ 𝑥.
We merge Lemmas 3.8 and 3.9 for convenience.

Lemma 3.10. Suppose that Assumption 2.1 holds true. Take any 𝜆 > 1 and 𝜏 ∈ 𝒯 𝑛 be a segmentation. Then, there exists an event Ω (0)

𝜏,𝜆 of probability greater than 1-3 e -𝜆𝐷𝜏 on which:

𝜓 𝜏 ≥ 1 3 𝐴 𝜏 - 74 3 𝜆𝐷 𝜏 𝑀 2 .
Proof. According to Lemma 3.9 with 𝜃 = 1/3 and 𝑥 = 𝜆𝐷 𝜏 , there exists an event Ω

(

( 𝜆 + 2 √ 2𝜆 ) 𝐷 𝜏 𝑀 2 , with P ( Ω (2) 𝜏,𝜆 ) ≥ 1 -e -𝜆𝐷𝜏 . Then, Ω (0)
𝜏,𝜆 := Ω

(1)

𝜏,𝜆 ∩ Ω (2)
𝜏,𝜆 has a probability larger than 1 -3 e -𝜆𝐷𝜏 by the union bound. Since for any 1 ≤ ℓ ≤ 𝐷 𝜏 , 𝑣 𝜏,ℓ ≤ 𝑀 2 , we have 𝑣 𝜏 = ∑ 𝐷𝜏 ℓ=1 𝑣 𝜏,ℓ ≤ 𝐷 𝜏 𝑀 2 . Hence, by definition (3.11) of 𝜓 𝜏 and using that 𝜆 ≥ 1, on the event Ω (0) 𝜏,𝜆 , we have:

𝜓 𝜏 ≥ 1 3 𝐴 𝜏 - ( 31 3 𝜆 + 28 3 √ 2 √ 𝜆 + 1 ) 𝐷 𝜏 𝑀 2 ≥ 1 3 𝐴 𝜏 -𝜆 ( 31 3 + 28 3 √ 2 + 1 ) 𝐷 𝜏 𝑀 2 .
Remark 3.2. It is also possible to obtain an upper bound for 𝜓 𝜏 : by Lemma 3.9, for every 𝜆 ≥ 0, on the event Ω

(2)

𝜏,𝜆 ⊂ Ω (0) 𝜏,𝜆 , 𝜓 𝜏 ≤ 5 3 𝐴 𝜏 + 17 3 𝜆𝐷 𝜏 𝑀 2 .
However, we do not need this result thereafter.

Concentration under Assumption 2.2. Lemma 3.6 and 3.7 directly translate upper bounds on 𝐵 𝑛 into controls of 𝐿 𝜏 and 𝑄 𝜏 . Under Assumption 2.2, this is achieved via the following lemma, a Kolmogorov-like inequality for the noise in the RKHS. This result is a straightforward generalization of the inequality obtained by [START_REF] Kolmogorov | Über die Summen durch den Zufall bestimmten unabhängigen größen[END_REF] into the Hilbert setting. A more precise result (for real random variables only) can be found in [START_REF] Hájek | Generalization of an inequality of Kolmogorov[END_REF], of which we follow the proof. The scheme of [START_REF] Hájek | Generalization of an inequality of Kolmogorov[END_REF] adapts well in our setting even though we do not need the full result.

Lemma 3.11. If Assumption 2.2 holds true, then, for any 𝑥 > 0,

P (𝐵 𝑛 ≥ 𝑥) ≤ 1 𝑥 2 𝑛 ∑ 𝑗=1 𝑣 𝑗 . (3.16)
We prove Lemma 3.11 in Section 3.5.6.

Remark 3.3. We can reformulate Lemma 3.11 as follows. For any 𝑦 > 0, there exists an event of probability at least 1 -𝑦 -2 on which 𝐵 𝑛 < 𝑦 √ ∑ 𝑛 𝑖=𝑗 𝑣 𝑗 ≤ 𝑦 √ 𝑛𝑉 .

Equivalently, for any 𝑧 ≥ 0, there exists an event of probability at least 1e -𝑧 such that 𝐵 𝑛 < e 𝑧/2 √ ∑ 𝑛 𝑖=𝑗 𝑣 𝑗 ≤ e 𝑧/2 √ 𝑛𝑉 .

Proof of Theorem 3.1

We follow the strategy described at the beginning of Section 3.4.

Definition of Ω. Let us define

Ω := ⋂ 𝜏 ∈𝒯𝑛 Ω (0)
𝜏,𝜆 with 𝜆 = 𝑦 + log(𝑛) + 1 > 1, where we recall that Ω (0) 𝜏,𝜆 is defined in Lemma 3.10. By the union bound, and since the Ω (0) 𝜏,𝜆 have probability greater than 1 -3 e -𝜆𝐷𝜏 ,

P (Ω) ≥ 1 -3 ∑ 𝜏 ∈𝒯𝑛 e -𝜆𝐷𝜏 .
The inequality P (Ω) ≥ 1e -𝑦 follows since

∑ 𝜏 ∈𝒯𝑛 e -𝜆𝐷𝜏 = 𝑛 ∑ 𝑑=1 ( 𝑛 -1 𝑑 -1 ) e -𝜆𝑑 = e -𝜆 ( 1 + e -𝜆 ) 𝑛-1 ≤ e -𝜆 exp ( (𝑛 -1) e -𝜆 ) = e -𝑦 𝑛 e exp ( 𝑛 -1 𝑛 e -1-𝑦 ) ≤ e -𝑦 exp (e -1 ) 𝑛 e ≤ 0.27 e -𝑦 ,
where the last inequality uses that 𝑛 ≥ 2. From now on, we work exclusively on Ω.

Key argument. We now make the simple (but crucial) observation that crit (𝜏 ⋆ ) ≥ crit (̂ 𝜏 ), hence

𝑛 pen (̂ 𝜏 ) + 𝜓 ̂ 𝜏 ≤ 𝑛 pen (𝜏 ⋆ ) + 𝜓 𝜏 ⋆ ≤ 𝑛 pen (𝜏 ⋆ ) = 𝐶𝐷 ⋆ 𝑀 2 .
Since we work on Ω, by definition of Ω (0)

𝜏,𝜆 in Lemma 3.10, for any 𝜏 ∈ 𝒯 𝑛 , we have:

𝜓 𝜏 ≥ 1 3 𝐴 𝜏 - 74 3 𝜆𝐷 𝜏 𝑀 2 .
Therefore, we get:

𝐶𝐷 ⋆ 𝑀 2 ≥ 1 3 𝐴 ̂ 𝜏 + ( 𝐶 - 74 3 𝜆 ) ̂ 𝐷𝑀 2 .
(3.17)

Proof that ̂ 𝐷 ≤ 𝐷 ⋆ . Since 𝐶 > 74𝜆/3 (by the lower bound in assumption (3.3)), 𝑀 2 > 0 and 𝐴 ̂ 𝜏 ≥ 0, Eq. (3.17) implies that

 𝐷 ≤ 𝐶 𝐶 -74 3 𝜆 𝐷 ⋆ .
The lower bound in assumption (3.3) ensures that

𝐶 𝐶 -74 3 𝜆 < 𝐷 ⋆ + 1 𝐷 ⋆ , hence ̂ 𝐷 ≤ 𝐷 ⋆ on Ω.
Proof that ̂ 𝐷 ≥ 𝐷 ⋆ . Since 𝐶 > 74𝜆/3 (by the lower bound in assumption (3.3)), Eq. (3.17) implies that 𝐴

̂ 𝜏 ≤ 3𝐶𝐷 ⋆ 𝑀 2 . A direct consequence of (3.3) is that 𝐴 ̂ 𝜏 < 1 2 𝑛Λ 𝜏 ⋆ ∆ 2 , hence ̂ 𝐷 ≥ 𝐷 ⋆ by Lemma 3.3.
Loss between ̂ 𝜏 and 𝜏 ⋆ . We have proved that ̂ 𝐷 = 𝐷 ⋆ on Ω, therefore, Eq. ( 3.17) can be rewritten 𝐴 ̂ 𝜏 ≤ 74𝜆𝐷 ⋆ 𝑀 2 . By Lemma 3.4 and the definition of 𝜆, we get min

{ Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) } ≤ 148𝐷 ⋆ 𝑀 2 ∆ 2 • 𝑦 + log(𝑛) + 1 𝑛 = 𝑣 1 (𝑦) . (3.18)
Remark that Assumption (3.3) implies that

∆ 2 𝑀 2 Λ 𝜏 ⋆ 6𝐷 ⋆ 𝑛 > 74 3 (𝐷 ⋆ + 1)(𝑦 + log(𝑛) + 1)
hence

Λ 𝜏 ⋆ > (𝐷 ⋆ + 1) 148𝐷 ⋆ 𝑀 2 ∆ 2 • 𝑦 + log(𝑛) + 1 𝑛 > 𝑣 1 (𝑦) .
Therefore, Eq. ( 3.18) can be simplified into

1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) ≤ 𝑣 1 (𝑦) .
Remark 3.4. The proof of Theorem 3.1 generalizes to ̂ 𝜏 defined by

 𝜏 ∈ arg min 𝜏 ∈𝒯𝑛 / Λ 𝜏 ≥𝛿𝑛 { crit (𝜏 ) }
instead of (2.2), for any 𝛿 𝑛 ≥ 0 such that Λ 𝜏 ⋆ ≥ 𝛿 𝑛 . Indeed, this assumption allows to write crit (𝜏 ⋆ ) ≥ crit (̂ 𝜏 ) in the key argument, and the rest of the proof can stay unchanged (with the same event Ω). More generally, any constraint can be added in the minimization problem defining ̂ 𝜏 , provided that 𝜏 ⋆ satisfies this constraint.

Proof of Theorem 3.2

The structure of the proof is very similar to the proof of Theorem 3.1, the main difference being the need for a more precise control of the penalty function near 𝐷 ⋆ . We refer to Section 3.5.7 for all the technical results required for this control.

Definition of Ω. Set 𝑦 > 0. For any 𝜏 , we set 𝜆 𝜏 := 𝑦 + log 𝑛 𝐷𝜏 + 4 and Ω 𝜏,𝜆𝜏 as in Lemma 3.10. Define Ω := ⋂ 𝜏 ∈𝒯𝑛 Ω 𝜏,𝜆𝜏 . By the union bound, and since the Ω 𝜏,𝜆𝜏 have probability greater than 1 -3 e -𝜆𝜏 𝐷𝜏 ,

P (Ω) = P ( ⋂ 𝜏 ∈𝒯𝑛 Ω 𝜏,𝜆𝜏 ) ≥ 1 -3 ∑ 𝜏 ∈𝒯𝑛 e -𝜆𝜏 𝐷𝜏 .
Recall that, for any 1 ≤ 𝑑 ≤ 𝑛,

( 𝑛-1 𝑑-1 ) ≤ ( 𝑛 𝑑 ) ≤ (𝑛 e /𝑑) 𝑑 , hence ∑ 𝜏 ∈𝒯𝑛 e -𝜆𝜏 𝐷𝜏 = 𝑛 ∑ 𝑑=1 ( 𝑛 -1 𝑑 -1
) e -𝑑(𝑦+log(𝑛/𝑑)+4)

≤ 𝑛 ∑ 𝑑=1 exp (𝑑 + 𝑑 log(𝑛) -𝑑 log(𝑑) -4𝑑 -𝑑𝑦 -𝑑 log(𝑛) + 𝑑 log(𝑑)) = 𝑛 ∑ 𝑑=1 exp (𝑑(-3 -𝑦)) = 𝑛 ∑ 𝑑=1 ( e -𝑦 e 3 ) 𝑑 = e -3-𝑦 • 1 -(e -3-𝑦 ) 𝑛 1 -e -3-𝑦 ≤ e -𝑦 /3.
The last inequality holds because 3 e -3 ≤ 1 and e -3-𝑦 < 1, and we have proved that P (Ω) ≥ 1e -𝑦 . We now work solely on the event Ω.

Proof that ̂ 𝐷 ≤ 𝐷 ⋆ on Ω. Let 𝜏 be a segmentation such that 𝐷 𝜏 > 𝐷 ⋆ . Let us show that crit (𝜏 ) > crit (𝜏 ⋆ ). In this case, we do not have a better lower bound than 0 for the approximation error 𝐴 𝜏 . We still can use the fact that

𝜓 𝜏 ≥ -74 3 𝜆 𝜏 𝐷 𝜏 𝑀 2 ,
and we know that 𝜓 𝜏 ⋆ ≤ 0. Thus 4). Given that 𝑐 1 > 𝑐 1,min and 𝑐 2 = (𝑐 1 + 74𝑀 2 (𝐷 ⋆ + 1)/3)(𝑦 + 4), we have 𝑎 > 0 and 𝑏 > 0. Moreover, 𝑏 -𝑎 = 74𝑀 2 (𝐷 ⋆ (𝑦 + 4) + 1)/3 + 𝑐 1 (3 + 𝑦) is a positive quantity. Therefore, according to Lemma 3.13 together with 𝐷 𝜏 > 𝐷 ⋆ ,

𝑛 (crit (𝜏 ) -crit (𝜏 ⋆ )) = 𝜓 𝜏 + 𝑛 pen L (𝜏 ) -𝜓 𝜏 ⋆ -𝑛 pen L (𝜏 ⋆ ) ≥ -74 3
( 𝑦 + log 𝑛 𝐷 𝜏 + 4 ) 𝐷 𝜏 𝑀 2 + 𝐷 𝜏 ( 𝑐 1 log 𝑛 𝐷 𝜏 + 𝑐 2 ) -𝐷 ⋆ ( 𝑐 1 log 𝑛 𝐷 ⋆ + 𝑐 2 ) ≥ 𝐷 𝜏 (( 𝑐 1 - 74 3 𝑀 2 ) log 𝑛 𝐷 𝜏 + ( 𝑐 2 - 74𝑀 2 3 (𝑦 + 4) )) -𝐷 ⋆ ( 𝑐 1 log 𝑛 𝐷 ⋆ + 𝑐 2 ) . Let us set 𝑎 := ( 𝑐 1 -74 3 𝑀 2 ) and 𝑏 := 𝑐 2 -74𝑀 2 3 (𝑦 +
𝑛 (crit (𝜏 ) -crit (𝜏 ⋆ )) ≥ (𝐷 ⋆ + 1) [( 𝑐 1 -74 3 𝑀 2 ) log 𝑛 𝐷 ⋆ +1 + ( 𝑐 2 -74𝑀 2 3 (𝑦 + 4) )] -𝐷 ⋆ ( 𝑐 1 log 𝑛 𝐷 ⋆ + 𝑐 2 ) .
After rewriting the right-hand side of the last display, we obtain

𝑛 (crit (𝜏 ) -crit (𝜏 ⋆ )) ≥ 𝑐 1 ( log(𝑛) + 𝐷 ⋆ log 𝐷 ⋆ -(𝐷 ⋆ + 1) log (𝐷 ⋆ + 1) ) -74𝑀 2 3 (𝐷 ⋆ + 1) log 𝑛 𝐷 ⋆ +1 + ( 𝑐 2 -74𝑀 2 3 (𝐷 ⋆ + 1) (𝑦 + 4)
) .

We now use the relationship between 𝑐 2 and 𝑐 1 , together with Lemma 3.14 applied to 𝑥 = 𝐷 ⋆ to write

𝑛 (crit (𝜏 ) -crit (𝜏 ⋆ )) ≥ 𝑐 1 ( 2 + 𝑦 + log(𝑛) -log 𝐷 ⋆ ) - 74𝑀 2 3 (𝐷 ⋆ + 1) (log(𝑛) -log (𝐷 ⋆ + 1)) .
This last display is positive if

𝑐 1 > 74𝑀 2 3 (𝐷 ⋆ + 1) (log(𝑛) -log (𝐷 ⋆ + 1)) 2 + 𝑦 + log(𝑛) -log (𝐷 ⋆ ) .
This is indeed the case according to Lemma 3.15 applied to 𝑥 = 𝐷 ⋆ together with 𝑐 1 > 𝑐 1,min , and we can conclude.

Proof that ̂ 𝐷 ≥ 𝐷 ⋆ on Ω. Let 𝜏 ∈ 𝒯 𝑛 be such that 𝐷 𝜏 < 𝐷 ⋆ . We are going to show that crit (𝜏 ) > crit (𝜏 ⋆ ). By construction of Ω, it holds that

𝜓 𝜏 ≥ 1 3 𝐴 𝜏 - 74𝜆 𝜏 𝐷 𝜏 3 𝑀 2 .
Because of Lemma 3.3,this leads to

𝜓 𝜏 ≥ 𝑛Λ 𝜏 ⋆ ∆ 2 6 - 74𝜆 𝜏 𝐷 𝜏 3 𝑀 2 .
According to Lemma 3.13, and since 𝐷 𝜏 < 𝐷 ⋆ ,

𝜓 𝜏 > 𝑛Λ 𝜏 ⋆ ∆ 2 6 - 74𝐷 ⋆ 3 ( 𝑦 + log 𝑛 𝐷 ⋆ + 4 ) 𝑀 2 .
On the other side, 𝜓 𝜏 ⋆ ≤ 0, and

𝑛 pen L (𝜏 ⋆ ) -𝑛 pen L (𝜏 ) < 𝐷 ⋆ ( 𝑐 1 log 𝑛 𝐷 ⋆ + 𝑐 2 ) . Thus 𝑛 (crit (𝜏 ) -crit (𝜏 ⋆ )) = 𝜓 𝜏 + 𝑛 pen L (𝜏 ) -𝜓 𝜏 ⋆ -𝑛 pen L (𝜏 ⋆ ) > 𝑛Λ 𝜏 ⋆ ∆ 2 6 - 74𝐷 ⋆ 3 ( 𝑦 + log 𝑛 𝐷 ⋆ + 4 ) 𝑀 2 -𝐷 ⋆ ( 𝑐 1 log 𝑛 𝐷 ⋆ + 𝑐 2 ) .
Let us use the relationship between 𝑐 1 and 𝑐 2 . The last display is positive if

𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ > 𝑐 1 (𝑦 + log(𝑛) -log (𝐷 ⋆ ) + 4) + 74 3 𝑀 2 ( 𝑦 + log 𝑛 𝐷 ⋆ + 4 ) + 74 3 𝑀 2 (𝐷 ⋆ + 1) (𝑦 + 4) = 𝑐 1 ( 𝑦 + log 𝑛 𝐷 ⋆ + 4 ) + 74 3 𝑀 2 ( (𝑦 + 4) (𝐷 ⋆ + 2) + log 𝑛 𝐷 ⋆ ) ,
which is true by Lemma 3.16 applied to 𝑥 = 𝐷 ⋆ together with the definition of 𝑐 1,max .

At this point, we have proved the first part of Theorem 3.2: on the event Ω,  𝐷 = 𝐷 ⋆ . We now address the problem of finding an upper bound for 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ).

Loss between ̂ 𝜏 and 𝜏 ⋆ . Since ̂ 𝐷 = 𝐷 ⋆ , we have pen L (̂ 𝜏 ) = pen L (𝜏 ⋆ ). Therefore,

𝜓 ̂ 𝜏 ≤ 𝜓 𝜏 ⋆ . Since 𝜓 𝜏 ⋆ ≤ 0 and 𝜓 ̂ 𝜏 ≥ 1 3 𝐴 𝜏 -74 3 𝜆 ̂ 𝜏 ̂
𝐷𝑀 2 , we have

1 3 𝐴 ̂ 𝜏 ≤ 74 3 𝜆 ̂ 𝜏 ̂ 𝐷𝑀 2 ≤ 74𝐷 ⋆ 𝑀 2 3 (𝑦 + log(𝑛) + 4) . By Lemma 3.4, min { Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) } ≤ 148𝐷 ⋆ 𝑀 2 ∆ 2 • 𝑦 + log(𝑛) + 4 𝑛 = 𝑣 ′ 1 (𝑦) . (3.19)
Rewriting 𝑐 1,min < 𝑐 1,max as a condition on Λ 𝜏 ⋆ , it follows that

Λ 𝜏 ⋆ > 148𝑀 2 𝐷 ⋆ (2𝐷 ⋆ + 3) (𝑦 + log(𝑛) + 4) 𝑛∆ 2 = 𝑣 ′ 1 • (2𝐷 ⋆ + 3) .
From 𝐷 ⋆ ≥ 1,we deduce that Λ 𝜏 ⋆ > 𝑣 ′ 1 . Eq. (3.19) now yields

1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 ) < 𝑣 ′ 1 (𝑦) ,
which concludes the proof of Theorem 3.2.

Proof of Theorem 3.3

We follow the strategy described at the beginning of Section 3.4. Throughout the proof, we write ̂ 𝜏 2 as a shortcut for ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ).

Key argument. By definition (3.7) of ̂ 𝜏 2 = ̂ 𝜏 (𝐷 ⋆ , 𝛿 𝑛 ), since we assume

Λ 𝜏 ⋆ ≥ 𝛿 𝑛 ,  ℛ 𝑛 (𝜏 ⋆ ) ≥ ̂ ℛ 𝑛 (̂ 𝜏 2 ) hence 0 ≥ 𝜓 𝜏 ⋆ ≥ 𝜓 ̂ 𝜏 2 = 𝐴 ̂ 𝜏 2 + 2𝐿 ̂ 𝜏 2 -𝑄 ̂ 𝜏 2 .
By Lemma 3.6, Lemma 3.7 and the facts that

𝐷 ̂ 𝜏 2 = 𝐷 ⋆ and Λ ̂ 𝜏 2 ≥ 𝛿 𝑛 , we get 0 ≥ 𝜓 ̂ 𝜏 2 ≥ 𝐴 ̂ 𝜏 2 -12 (𝐷 ⋆ ) 2 ∆𝐵 𝑛 - 4𝐷 ⋆ 𝐵 2 𝑛 𝑛𝛿 𝑛 .
Hence, using Lemma 3.4,

min { Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 2 ) } ≤ 24 (𝐷 ⋆ ) 2 ∆ ∆ 2 𝐵 𝑛 𝑛 + 8𝐷 ⋆ ∆ 2 𝐵 2 𝑛 𝑛 2 𝛿 𝑛 . (3.20)
Definition of Ω 2 . We define

Ω 2 := {𝐵 𝑛 ≤ 𝑦 √ 𝑛𝑉 } .
By Lemma 3.11, under Assumption 2.2, P (Ω 2 ) ≥ 1 -𝑦 -2 .

Conclusion

. By definition of Ω 2 , Eq. ( 3.20) implies that on Ω 2 :

min { Λ 𝜏 ⋆ , 1 𝑛 d (1) ∞ (𝜏 ⋆ , ̂ 𝜏 2 ) } ≤ 24(𝐷 ⋆ ) 2 ∆ √ 𝑉 ∆ 2 𝑦 √ 𝑛 + 8𝐷 ⋆ 𝑉 ∆ 2 𝑦 2 𝑛𝛿 𝑛 = 𝑣 2 (𝑦, 𝛿 𝑛 ) .
Since we assume 𝑣 2 (𝑦, 𝛿 𝑛 ) < Λ 𝜏 ⋆ , the result follows.

Additional proofs

In this section are collected a large part of the technical details of the proofs that precede. Some additional notation used solely in this section are introduced below.

We denote by 𝜆 ⋆ 1 , . . . , 𝜆 ⋆ 𝐷 ⋆ the segments of 𝜏 ⋆ , that is,

𝜆 ⋆ 𝑖 = { 𝜏 ⋆ 𝑖-1 + 1, . . . , 𝜏 ⋆ 𝑖 } .
For any segment 𝜆 of 𝜏 ∈ 𝒯 𝑛 , we denote by 𝜇 ⋆ 𝜆 the value of 𝜇 ⋆ 𝜏 on 𝜆, which does not depend on 𝜏 and is given by (2.7):

𝜇 ⋆ 𝜆 = 1 |𝜆| ∑ 𝑗∈𝜆 𝜇 ⋆ 𝑗 . (3.21)
We will sometimes write ∑ 𝜆∈𝜏 instead of the more cumbersome ∑ 𝐷𝜏 ℓ=1 , when the dependency in 𝜏 • is not apparent in the summation. More generally, we will abuse the notation 𝜏 to denote the set of segments associated to the segmentation 𝜏 .

Proof of Lemma 3.1

Proof of (i). We set

𝐷 𝑖 := 𝐷 𝜏 𝑖 for 𝑖 ∈ {1, 2}. Let us show first that d (2) ∞ (𝜏 1 , 𝜏 2 ) = d (1) ∞ (𝜏 1 , 𝜏 2 ). Take any 𝑖 ∈ { 1, . . . , 𝐷 1 -1 } , by the definition of Λ 𝜏 1 , ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝐷 2 ⃒ ⃒ = ⃒ ⃒ 𝜏 1 𝑖 -𝑛 ⃒ ⃒ ≥ 𝑛Λ 𝜏 1 > 𝑛Λ 𝜏 1 /2 ≥ 𝑛 min { Λ 𝜏 1 , Λ 𝜏 2 } /2 ,
which is greater than d (1) ∞ (𝜏 1 , 𝜏 2 ) by assumption. In the same fashion we can prove that |𝜏 1 𝑖 -𝜏 2 0 | > d (1) ∞ (𝜏 1 , 𝜏 2 ). Hence, for any 𝑖 ∈ {1, . . . , 𝐷 1 -1}, min

0≤𝑗≤𝐷 2 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ = min 1≤𝑗≤𝐷 2 -1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ , which proves that d (2) ∞ (𝜏 1 , 𝜏 2 ) = d (1) ∞ (𝜏 1 , 𝜏 2 ). Next, we prove that 𝐷 1 = 𝐷 2 and d (3) ∞ (𝜏 1 , 𝜏 2 ) = d (1) ∞ (𝜏 1 , 𝜏 2 ). Define the mapping 𝜑 : { 1, . . . , 𝐷 1 -1 } → { 1, . . . , 𝐷 2 -1 } such that { 𝜑(𝑖) } = arg min 1≤𝑗≤𝐷 2 -1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒
for all 𝑖 ∈ {1, . . . , 𝐷 1 -1}. This mapping is well-defined: indeed, suppose that 𝑗, 𝑘 ∈ { 1, . . . , 𝐷 2 -1 } both realize the minimum for some

𝑖 ∈ { 1, . . . , 𝐷 1 -1 } . Since we assumed 1 𝑛 d (1) ∞ (𝜏 1 , 𝜏 2 ) < min { Λ 𝜏 1 , Λ 𝜏 2 } /2, ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒ = ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑘 ⃒ ⃒ ≤ d (1) ∞ ( 𝜏 1 , 𝜏 2 ) < 𝑛 min { Λ 𝜏 1 , Λ 𝜏 2 } /2.
By the triangle inequality,

⃒ ⃒ 𝜏 2 𝑗 -𝜏 2 𝑘 ⃒ ⃒ < 𝑛 min { Λ 𝜏 1 , Λ 𝜏 2 } ≤ 𝑛Λ 𝜏 2 , hence 𝑗 = 𝑘. Next, we show that 𝜑 is increasing. Take 𝑖, 𝑗 ∈ { 1, . . . , 𝐷 1 -1 } such that 𝑖 < 𝑗. Recall that 𝜏 𝑘 • is increasing (𝑘 = 1, 2). Then 𝜏 2 𝜑(𝑖) -𝜏 2 𝜑(𝑗) = 𝜏 2 𝜑(𝑖) -𝜏 1 𝑖 + 𝜏 1 𝑖 -𝜏 1 𝑗 + 𝜏 1 𝑗 -𝜏 2 𝜑(𝑗) = 𝜏 2 𝜑(𝑖) -𝜏 1 𝑖 - ⃒ ⃒ 𝜏 1 𝑖 -𝜏 1 𝑗 ⃒ ⃒ + 𝜏 1 𝑗 -𝜏 2 𝜑(𝑗) ≤ ⃒ ⃒ 𝜏 2 𝜑(𝑖) -𝜏 1 𝑖 ⃒ ⃒ - ⃒ ⃒ 𝜏 1 𝑖 -𝜏 1 𝑗 ⃒ ⃒ + ⃒ ⃒ 𝜏 1 𝑗 -𝜏 2 𝜑(𝑗) ⃒ ⃒ ≤ 2 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) - ⃒ ⃒ 𝜏 1 𝑖 -𝜏 1 𝑗 ⃒ ⃒ < 𝑛 min { Λ 𝜏 1 , Λ 𝜏 2 } -𝑛Λ 𝜏 1 ≤ 0 .
Hence 𝜑(𝑖) < 𝜑(𝑗), so 𝜑 is increasing. As a consequence, 𝜑 is injective and we get 𝐷 1 ≤ 𝐷 2 . The same argument, exchanging 𝜏 1 and 𝜏 2 , shows that 𝐷 2 ≤ 𝐷 1 . Therefore, 𝐷 1 = 𝐷 2 and 𝜑 is an increasing permutation of { 1, . . . , 𝐷 1 -1 } , hence it is the identity. Thus d (3) ∞ (𝜏 1 , 𝜏 2 ) = d (1) ∞ (𝜏 1 , 𝜏 2 ). Take 𝑖 and 𝑗 distinct elements of { 1, . . . , 𝐷 -1 } , and suppose that 𝜑(𝑖) ∩ 𝜑(𝑗) is non-empty. Let 𝑘 be any element of 𝜑(𝑖) ∩ 𝜑(𝑗). By the triangle inequality and the definition of d (1) ∞ ,

Finally, since d (3) ∞ is symmetric, d (𝑖) ∞ (𝜏 1 , 𝜏 2 ) = d (𝑖) H (𝜏
𝑛Λ 𝜏 1 ≤ ⃒ ⃒ 𝜏 1 𝑖 -𝜏 1 𝑗 ⃒ ⃒ ≤ ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑘 ⃒ ⃒ + ⃒ ⃒ 𝜏 2 𝑘 -𝜏 1 𝑗 ⃒ ⃒ ≤ 2 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) < 𝑛Λ 𝜏 1 .
Hence, the 𝜑(𝑖) are disjoint and we can write ∑ 𝐷-1 𝑖=1 𝐶 𝜑 (𝑖) = 𝐷 -1, which clearly implies that 𝐶 𝜑 (𝑖) = 1.

From now on, we identify 𝜑(𝑖) with its unique element. Let us show that 𝜑 is increasing similarly to what we have done for proving (i). Take 𝑖, 𝑗 ∈ { 1, . . . , 𝐷 -1 } such that 𝑖 < 𝑗. We showed that

𝜏 2 𝜑(𝑖) -𝜏 2 𝜑(𝑗) ≤ 2 d (1) ∞ ( 𝜏 1 , 𝜏 2 ) - ⃒ ⃒ 𝜏 1 𝑖 -𝜏 1 𝑗 ⃒ ⃒ ,
thus according to the definition of Λ 𝜏 1 , and our assumption,

𝜏 2 𝜑(𝑖) -𝜏 2 𝜑(𝑗) < 𝑛Λ 𝜏 1 -𝑛Λ 𝜏 1 ≤ 0 .
Hence 𝜑(𝑖) < 𝜑(𝑗): 𝜑 is increasing. As a consequence, d

∞ ( 𝜏 1 , 𝜏 2 ) = d (1) ∞ ( 𝜏 2 , 𝜏 1 ) = d (1) H ( 𝜏 1 , 𝜏 2 ) . (1)

The Frobenius loss

A formula for d 2

F

We start by proving a general formula for d F , which is stated by [START_REF] Lajugie | Large-margin metric learning for constrained partitioning problems[END_REF], we prove it here for completeness:

∀𝜏 1 , 𝜏 2 ∈ 𝒯 𝑛 , d F ( 𝜏 1 , 𝜏 2 ) 2 = 𝐷 𝜏 1 + 𝐷 𝜏 2 -2 𝐷 𝜏 1 ∑ 𝑘=1 𝐷 𝜏 2 ∑ ℓ=1 |𝜆 1 𝑘 ∩ 𝜆 2 ℓ | 2 |𝜆 1 𝑘 | × |𝜆 2 ℓ | . (3.22)
Indeed, by definition, we have

d F ( 𝜏 1 , 𝜏 2 ) 2 = Tr ( (Π 𝜏 1 -Π 𝜏 2 ) 2 ) = Tr(Π 𝜏 1 ) ⏟ ⏞ =𝐷 𝜏 1 + Tr(Π 𝜏 2 ) ⏟ ⏞ =𝐷 𝜏 2 -2 Tr(Π 𝜏 1 Π 𝜏 2 )
and Tr(Π

𝜏 1 Π 𝜏 2 ) = 𝑛 ∑ 𝑖=1 𝑛 ∑ 𝑗=1 1 𝜆 1 (𝑖)=𝜆 1 (𝑗) and 𝜆 2 (𝑖)=𝜆 2 (𝑗) |𝜆 1 (𝑖)| |𝜆 2 (𝑖)| = 𝐷 𝜏 1 ∑ 𝑘=1 𝐷 𝜏 2 ∑ ℓ=1 |𝜆 1 𝑘 ∩ 𝜆 2 ℓ | 2 |𝜆 1 𝑘 | × |𝜆 2 ℓ |
, where we denoted by 𝜆 𝑘 (𝑖) the segment of 𝜏 𝑘 to which 𝑖 ∈ {1, . . . , 𝑛} belongs.

Proof of Eq. (3.5) Eq. (3.5) is stated by [START_REF] Lajugie | Large-margin metric learning for constrained partitioning problems[END_REF]. The upper bound is a straightforward consequence of Eq. (3.22). We prove the lower bound here for completeness. We remark that

𝐷 𝜏 1 ∑ 𝑘=1 𝐷 𝜏 2 ∑ ℓ=1 |𝜆 1 𝑘 ∩ 𝜆 2 ℓ | 2 |𝜆 1 𝑘 | × |𝜆 2 ℓ | ≤ 𝐷 𝜏 1 ∑ 𝑘=1 𝐷 𝜏 2 ∑ ℓ=1 |𝜆 1 𝑘 ∩ 𝜆 2 ℓ | |𝜆 1 𝑘 | = 𝐷 𝜏 1 ,
hence Eq. (3.22) shows that

d F ( 𝜏 1 , 𝜏 2 ) 2 ≥ 𝐷 𝜏 2 -𝐷 𝜏 1 .
The lower bound follows since 𝜏 1 and 𝜏 2 play symmetric roles.

Proof of Prop. 3.1

Throughout the proof, we write

𝐷 = 𝐷 𝜏 1 = 𝐷 𝜏 2 , 𝜀 = 𝑛 -1 d (1)
∞ (𝜏 1 , 𝜏 2 ) and we denote by (𝜆 1 𝑘 ) 1≤𝑘≤𝐷 and (𝜆 2 𝑘 ) 1≤𝑘≤𝐷 the segments of 𝜏 1 and 𝜏 2 , respectively.

Preliminary remark. Since we assume that

𝐷 𝜏 1 = 𝐷 𝜏 2 and 1 𝑛 d (1) ∞ (𝜏 1 , 𝜏 2 ) = 𝜀 < Λ 𝜏 1 /2, point (ii) in Lemma 3.1 shows that d (1) ∞ (𝜏 1 , 𝜏 2 ) = d (1)
H (𝜏 1 , 𝜏 2 ) = d (3) ∞ (𝜏 1 , 𝜏 2 ).
In other words, for every 𝑘 ∈ {1, . . . , 𝐷 -1}, we have |𝜏 1 𝑘 -𝜏 2 𝑘 | ≤ 𝑛𝜀, and some

𝑘 0 ∈ {1, . . . , 𝐷 -1} exists such that ⃒ ⃒ 𝜏 1 𝑘 0 -𝜏 2 𝑘 0 ⃒ ⃒ = 𝑛𝜀. As a consequence, for every 𝑘 ∈ {1, . . . , 𝐷 -1}, ⃒ ⃒ ⃒ ⃒ 𝜆 1 𝑘 ⃒ ⃒ - ⃒ ⃒ 𝜆 2 𝑘 ⃒ ⃒ ⃒ ⃒ ≤ 2𝑛𝜀 and ⃒ ⃒ 𝜆 1 𝑘 ∩ 𝜆 2 𝑘 ⃒ ⃒ ≥ ⃒ ⃒ 𝜆 1 𝑘 ⃒ ⃒ -2𝑛𝜀 . (3.23)
Upper bound for d F (𝜏 1 , 𝜏 2 ) 2 . We focus on the sum appearing in the right-hand side of Eq. (3.22). Using Eq. ( 3.23), we get:

𝐷 ∑ 𝑘=1 𝐷 ∑ ℓ=1 |𝜆 1 𝑘 ∩ 𝜆 2 ℓ | 2 |𝜆 1 𝑘 | × |𝜆 2 ℓ | ≥ 𝐷 ∑ 𝑘=1 |𝜆 1 𝑘 ∩ 𝜆 2 𝑘 | 2 |𝜆 1 𝑘 | × |𝜆 2 𝑘 | ≥ 𝐷 ∑ 𝑘=1 [ (|𝜆 1 𝑘 | -2𝑛𝜀) 2 |𝜆 1 𝑘 | × (|𝜆 1 𝑘 | + 2𝑛𝜀) ] = 𝐷 ∑ 𝑘=1 ( 1 -2𝑛𝜀 |𝜆 1 𝑘 | ) 2 1 + 2𝑛𝜀 |𝜆 1 𝑘 | ≥ 𝐷 ∑ 𝑘=1 ( 1 - 6𝑛𝜀 |𝜆 1 𝑘 | ) ≥ 𝐷 - 6𝜀𝐷 Λ 𝜏 1 ,
since for any 𝑥 ≥ 0, (1-𝑥) 2 1+𝑥 ≥ 1 -3𝑥. The upper bound follows, using Eq. (3.22).

Lower bound for d F (𝜏 1 , 𝜏 2 ) 2 . As shown in the preliminary remark, there exists some

𝑘 0 ∈ {1, . . . , 𝐷 -1} such that ⃒ ⃒ 𝜏 1 𝑘 0 -𝜏 2 𝑘 0 ⃒ ⃒ = 𝑛𝜀.
First consider the case where

𝜏 1 𝑘 0 < 𝜏 2 𝑘 0 .
Then, by definition of d F and Π 𝜏 , we have:

d F ( 𝜏 1 , 𝜏 2 ) 2 := ∑ 1≤𝑖,𝑗≤𝑛 (Π 𝜏 1 -Π 𝜏 2 ) 2 𝑖,𝑗 ≥ ∑ 𝑖∈𝜆 1 𝑘 0 +1 ∩𝜆 2 𝑘 0 ∑ 𝑗∈𝜆 1 𝑘 0 +1 ∩𝜆 2 𝑘 0 +1 1 ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ 2 + ∑ 𝑖∈𝜆 1 𝑘 0 +1 ∩𝜆 2 𝑘 0 +1 ∑ 𝑗∈𝜆 1 𝑘 0 +1 ∩𝜆 2 𝑘 0 1 ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ 2 = 2 ⃒ ⃒ 𝜆 1 𝑘 0 +1 ∩ 𝜆 2 𝑘 0 ⃒ ⃒ • ⃒ ⃒ 𝜆 1 𝑘 0 +1 ∩ 𝜆 2 𝑘 0 +1 ⃒ ⃒ ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ 2 . Now, remark that ⃒ ⃒ 𝜆 1 𝑘 0 +1 ∩ 𝜆 2 𝑘 0 ⃒ ⃒ = 𝑛𝜀
, by the preliminary remark and our assumption 𝜏 2 𝑘 0 > 𝜏 1 𝑘 0 . Using also Eq. ( 3.23), we get:

d F ( 𝜏 1 , 𝜏 2 ) 2 ≥ 2𝑛𝜀 (⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ -2𝑛𝜀 ) ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ 2 ≥ 2𝑛𝜀 3Λ 𝜏 1 , since ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ -2𝑛𝜀 ≥ ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ /3 and ⃒ ⃒ 𝜆 1 𝑘 0 +1 ⃒ ⃒ ≤ Λ 𝜏 1 . When 𝜏 1 𝑘 0 > 𝜏 2
𝑘 0 , we apply the same reasoning, restricting the sum over 𝑖, 𝑗 in the definition of

d F to 𝑖 ∈ 𝜆 1 𝑘 0 ∩ 𝜆 2 𝑘 0 and 𝑗 ∈ 𝜆 1 𝑘 0 ∩ 𝜆 2 𝑘 0 +1
(plus its symmetric). We obtain the same lower bound, which concludes the proof.

Lower bounds on the approximation error

This section provides the proofs of Lemmas 3.3 and 3.4.

Preliminary lemma

We start with a lemma useful in the two proofs. Lemma 3.12. If a segment 𝜆 ⊂ {1, . . . , 𝑛} intersects only two segments of 𝜏 ⋆ , 𝜆 ⋆ 𝑖 and 𝜆 ⋆ 𝑖+1 , then we have:

∑ 𝑗∈𝜆 ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ⃦ ⃦ 2 ℋ = |𝜆 ∩ 𝜆 ⋆ 𝑖 | • ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ |𝜆 ∩ 𝜆 ⋆ 𝑖 | + ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ (3.24) ≥ ( |𝜆 ∩ 𝜆 ⋆ 𝑖 | |𝜆 ⋆ 𝑖 | ∧ ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ ) • |𝜆 ⋆ 𝑖 | • ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ |𝜆 ⋆ 𝑖 | + ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ . (3.25)
Proof. We first prove Eq. (3.24). Since 𝜆 only intersects 𝜆 ⋆ 𝑖 and 𝜆 ⋆ 𝑖+1 , we have:

∑ 𝑗∈𝜆 ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ⃦ ⃦ 2 ℋ = ∑ 𝑗∈𝜆∩𝜆 ⋆ 𝑖 ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ⃦ ⃦ 2 ℋ + ∑ 𝑗∈𝜆∩𝜆 ⋆ 𝑖+1 ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ⃦ ⃦ 2 ℋ = |𝜆 ∩ 𝜆 ⋆ 𝑖 | • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 ⃦ ⃦ ⃦ 2 ℋ + ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⃦ ⃦ ⃦ 2 ℋ . (3.26)
Since 𝜇 ⋆ 𝜆 is given by Eq. ( 3.21), we obtain

⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 ⃦ ⃦ ⃦ 2 ℋ = ⃦ ⃦ ⃦ ⃦ ⃦ 1 |𝜆| ∑ 𝑗∈𝜆 ( 𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝑗 ) ⃦ ⃦ ⃦ ⃦ ⃦ 2 ℋ = ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 1 |𝜆| ∑ 𝑗∈𝜆∩𝜆 ⋆ 𝑖+1 ( 𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 ⋆ 𝑖+1 ) ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ 2 ℋ = ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ 2 |𝜆| 2 ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ . The same computation on 𝜆 ∩ 𝜆 ⋆ 𝑖+1 yields ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⃦ ⃦ ⃦ 2 ℋ = |𝜆 ∩ 𝜆 ⋆ 𝑖 | 2 |𝜆| 2 ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ .
Therefore, Eq. ( 3.26) and the fact that .24). Now, we remark that for any 𝑎, 𝑏, 𝑐, 𝑑 > 0,

|𝜆| = |𝜆 ∩ 𝜆 ⋆ 𝑖 | + ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ yield Eq. ( 3
𝑎𝑏𝑐𝑑 𝑎𝑏 + 𝑐𝑑 = 1 𝑎𝑏 max(𝑎,𝑐) + 𝑐𝑑 max(𝑎,𝑐)
× min(𝑎, 𝑐) × 𝑏𝑑 ≥ min(𝑎, 𝑐) 𝑏𝑑 𝑏 + 𝑑 .

Taking 𝑎 = |𝜆 ∩ 𝜆 ⋆ 𝑖 | / |𝜆 ⋆ 𝑖 |, 𝑏 = |𝜆 ⋆ 𝑖 |, 𝑐 = ⃒ ⃒ 𝜆 ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ / ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ and 𝑑 = ⃒ ⃒ 𝜆 ⋆ 𝑖+1
⃒ ⃒ , we get Eq. (3.25).

Proof of Lemma 3.3 In fact, we prove a slightly stronger statement. We show that, for any 𝑛 ≥ 2, for any 𝐷 ⋆ ∈ {2, . . . , 𝑛}, for any 𝐷 ∈ {1, . . . , 𝐷 ⋆ -1} and any 𝜏 ∈ 𝒯 𝐷 𝑛 ,

‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ min 1≤𝑖≤𝐷 ⋆ -1 { |𝜆 ⋆ 𝑖 | • ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ |𝜆 ⋆ 𝑖 | + ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ } . (3.27) Then, ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ Γ • ∆ 2 where Γ = ( 𝑛 max 1≤𝑖≤𝐷 ⋆ -1 { 1 |𝜆 ⋆ 𝑖 | + 1 ⃒ ⃒ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ }) -1
.

Since we always have

Λ 𝜏 ⋆ ≥ Γ ≥ 1 2 Λ 𝜏 ⋆ ,
Eq. ( 3.12) follows.

Proof of Eq. (3.27) by induction. We show by strong induction on 𝐷 ⋆ that, for any 𝐷 ⋆ ≥ 2, for any 𝐷 ∈ {1, . . . , 𝐷 ⋆ -1}, any 𝑛 ≥ 𝐷 ⋆ and any 𝜏 ∈ 𝒯 𝐷 𝑛 , Eq. (3.27) holds true. First, if 𝐷 ⋆ = 2, the result follows by Eq. ( 3.25) in Lemma 3.12 since we then have 𝑖 = 1 and

|𝜆 ∩ 𝜆 ⋆ 1 | |𝜆 ⋆ 1 | = |𝜆 ∩ 𝜆 ⋆ 2 | |𝜆 ⋆ 2 | = 1 .
Suppose now that the result is proved for all 𝐷 ⋆ ∈ {2, . . . , 𝑝} and consider a change-point problem (𝜏 ⋆ , 𝜇 ⋆ ) with 𝐷 ⋆ = 𝐷 ⋆ = 𝑝 + 1 and 𝑛 ≥ 𝑝 + 1. Let 𝐷 < 𝑝 + 1 and some segmentation 𝜏 ∈ 𝒯 𝐷 𝑛 be fixed. Then one of these two scenarios occurs: (i) there exists 𝜆 ⋆ 𝑖 with 2 ≤ 𝑖 ≤ 𝐷 ⋆ -1 that does not contain any change-point of 𝜏 , or (ii) 𝜆 ⋆ 2 ,...,𝜆 ⋆ 𝐷 ⋆ -1 all contain a change-point of 𝜏 .

Case (i). Suppose that there exists an inner segment 𝜆 ⋆ 𝑖 of 𝜏 ⋆ , 2 ≤ 𝑖 ≤ 𝐷 ⋆ -1, that does not contain any change-point of 𝜏 (see Fig. 3-2). Therefore, there exists 𝑘 ∈ {1, . . . , 𝐷} such that 𝜆 ⋆ 𝑖 𝜆 𝑘 . By definition, there are 𝑖 -1 change-points of 𝜏 ⋆ to the left of 𝜆 ⋆ 𝑖 and 𝑘 -1 change-points of 𝜏 to the left of 𝜆 ⋆ 𝑖 . Suppose that 𝑘 < 𝑖. We define 𝜏 ∘ as the segmentation obtained by adding 𝜏 ⋆ 𝑖 to 𝜏 (see Fig. 3-2). Then

‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ∘ ‖ 2 because 𝜏 ∘ is
‖̃ 𝜇 ⋆ -̃ 𝜇 ⋆  𝜏 ∘ ‖ 2 ≥ inf 1≤𝑗≤𝑖-1 { ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ • ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ + ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ • ⃦ ⃦ ⃦̃ 𝜇 ⋆ 𝜆 ⋆ 𝑗+1 -̃ 𝜇 ⋆ 𝜆 ⋆ 𝑗 ⃦ ⃦ ⃦ 2 ℋ } ≥ inf 1≤𝑗≤𝐷 ⋆ -1 { ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ • ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ + ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑗+1 -𝜇 ⋆ 𝜆 ⋆ 𝑗 ⃦ ⃦ ⃦ 2 ℋ }
and we get the result since

‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ∘ ‖ 2 ≥ ‖̃ 𝜇 ⋆ -̃ 𝜇 ⋆  𝜏 ∘ ‖ 2 .
A symmetric reasoning can be applied if 𝑘 ≥ 𝑖, considering change-points to the right of 𝜆 ⋆ 𝑖 and using that

𝐷 -𝑘 + 1 < 𝐷 ⋆ -𝑖 + 1 since 𝐷 < 𝐷 ⋆ . 𝜆 ⋆ 𝑖 𝜆 𝑘  𝜏 ⋆ 𝜏 ⋆ 𝜏 𝜏 ∘  𝜏 ∘ • • • • • • • • • • • • • • • • • • • • • • • • Figure 3-2 -Proof of Lemma 3.3, Case (i): 𝜆 ⋆
𝑖 is a segment of 𝜏 ⋆ that is included in a segment of 𝜏 . The segmentation 𝜏 ∘ is obtained by joining 𝜏 ⋆ 𝑖 to the segmentation 𝜏 .

Case (ii). Suppose that each inner segment of 𝜏 ⋆ contains a change-point of 𝜏 . Since there are 𝐷 ⋆ -2 inner segments of 𝜏 ⋆ and 𝐷 -1 ≤ 𝐷 ⋆ -2 change-points of 𝜏 , there is at most (hence exactly) one change-point of 𝜏 in each inner segment of 𝜏 ⋆ .

Then 𝐷 = 𝐷 ⋆ -1 and we are in the situation depicted in Fig. 3-3. Lemma 3.3,Case (ii): 𝐷 = 𝐷 ⋆ -1 and each inner segment of 𝜏 ⋆ contains exactly one change-point of 𝜏 .

𝜏 ⋆ 𝜏 • • • • • • 𝜆 ⋆ 1 𝜆 ⋆ 2 𝜆 ⋆ 𝐷 𝜆 ⋆ 𝐷+1 𝜆 1 𝜆 2 𝜆 𝐷 𝛼 2 |𝜆 ⋆ 2 | (1 -𝛼 2 ) |𝜆 ⋆ 2 | 𝛼 𝐷 |𝜆 ⋆ 𝐷 | (1 -𝛼 𝐷 ) |𝜆 ⋆ 𝐷 | Figure 3-3 -Proof of
We can use Eq. (3.25) in Lemma 3.12 to lower bound the contribution of each

𝜆 ∈ 𝜏 to ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 . For 2 ≤ 𝑖 ≤ 𝐷 = 𝐷 ⋆ -1, define 𝛼 𝑖 := |𝜆 ⋆ 𝑖 ∩ 𝜆 𝑖-1 | / |𝜆 ⋆ 𝑖 |. Then, we have ‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ (1 ∧ 𝛼 2 ) |𝜆 ⋆ 1 | • |𝜆 ⋆ 2 | |𝜆 ⋆ 1 | + |𝜆 ⋆ 2 | • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 2 -𝜇 ⋆ 𝜆 ⋆ 1 ⃦ ⃦ ⃦ 2 ℋ + 𝐷-1 ∑ 𝑗=2 ( [ (1 -𝛼 𝑗 ) ∧ 𝛼 𝑗+1 ] • ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ • ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ + ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑗+1 -𝜇 ⋆ 𝜆 ⋆ 𝑗 ⃦ ⃦ ⃦ 2 ℋ ) + [ (1 -𝛼 𝐷 ) ∧ 1 ] |𝜆 ⋆ 𝐷 | • ⃒ ⃒ 𝜆 ⋆ 𝐷+1 ⃒ ⃒ |𝜆 ⋆ 𝐷 | + ⃒ ⃒ 𝜆 ⋆ 𝐷+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝐷+1 -𝜇 ⋆ 𝜆 ⋆ 𝐷 ⃦ ⃦ ⃦ 2 ℋ ≥ [1 ∧ 𝛼 2 +(1 -𝛼 2 ) ∧ 𝛼 3 + • • • + (1 -𝛼 𝐷-1 ) ∧ 𝛼 𝐷 + (1 -𝛼 𝐷 ) ∧ 1] × inf 1≤𝑗≤𝐷 ⋆ -1 { ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ • ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ ⃒ ⃒ 𝜆 ⋆ 𝑗 ⃒ ⃒ + ⃒ ⃒ 𝜆 ⋆ 𝑗+1 ⃒ ⃒ • ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑗+1 -𝜇 ⋆ 𝜆 ⋆ 𝑗 ⃦ ⃦ ⃦ 2 ℋ } .
Since 𝛼 𝑖 ≥ 0 for any 2 ≤ 𝑖 ≤ 𝐷 ⋆ -1, it is straightforward to show that

𝛼 2 + (1 -𝛼 2 ) ∧ 𝛼 3 + • • • + (1 -𝛼 𝐷 ) ≥ 1 ,
which concludes the proof.

Proof of Lemma 3.4

Let us define 𝛿 := min

{ 𝑛Λ 𝜏 ⋆ , d (1)
∞ (𝜏 ⋆ , 𝜏 )

}

. If 𝛿 = 0, then Eq. (3.13) holds true. We assume from now on that 𝛿 > 0.

Because 𝑛Λ 𝜏 ⋆ ≥ 𝛿, for any 1 ≤ 𝑖 ≤ 𝐷 ⋆ -1, we can write ⃒ ⃒ 𝜏 ⋆ 𝑖+1 -𝜏 ⋆ 𝑖 ⃒ ⃒ ≥ 𝛿.
On the other hand, because d (1) ∞ (𝜏 ⋆ , 𝜏 ) ≥ 𝛿, there exists 𝑖 ∈ {1, . . . , 𝐷 ⋆ -1} such that, for any 𝑗 ∈ {1, . . . , 𝐷 -1}, |𝜏 ⋆ 𝑖 -𝜏 𝑗 | ≥ 𝛿. Since 𝛿 ≤ 𝑛Λ 𝜏 ⋆ , this also holds true for 𝑗 = 0 and 𝑗 = 𝐷. Let us define, as illustrated by Fig. 34, Lemma 3.4. In this case, 𝛿 = 2 since Λ 𝜏 ⋆ = 2/10 (the rightmost segment of 𝜏 ⋆ is of size 2) and d (1) ∞ (𝜏 ⋆ , 𝜏 ) = 3 (achieved in 𝜏 ⋆ 𝑖 ).

𝜆 ∘ := { 𝜏 ⋆ 𝑖 -𝛿 + 1, . . . , 𝜏 ⋆ 𝑖 , 𝜏 ⋆ 𝑖 + 1, . . . , 𝜏 ⋆ 𝑖 + 𝛿 } . • × • × • • • • • × • • × • 𝜏 ⋆ 𝜏 𝜆 ∘ 𝜏 ⋆ 𝑖 𝜏 ⋆ 𝑖+1 Figure 3-4 -Construction of 𝜆 ∘ in the proof of
Since 𝜆 ∘ is included in a segment of 𝜏 ,

‖𝜇 ⋆ -𝜇 ⋆ 𝜏 ‖ 2 ≥ ∑ 𝑗∈𝜆 ∘ ⃦ ⃦ 𝜇 ⋆ 𝑗 -(𝜇 ⋆ 𝜏 ) 𝑗 ⃦ ⃦ 2 ℋ ≥ ∑ 𝑗∈𝜆 ∘ ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ∘ ⃦ ⃦ 2 ℋ .
Because of the hypothesis we made, 𝜆 ∘ only intersects 𝜆 ⋆ 𝑖 and 𝜆 ⋆ 𝑖+1 among the segments of 𝜏 ⋆ , so Eq. ( 3.24) in Lemma 3.12 shows that

∑ 𝑗∈𝜆 ∘ ⃦ ⃦ 𝜇 ⋆ 𝑗 -𝜇 ⋆ 𝜆 ∘ ⃦ ⃦ 2 ℋ = |𝜆 ∘ ∩ 𝜆 ⋆ 𝑖 | • ⃒ ⃒ 𝜆 ∘ ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ |𝜆 ∘ ∩ 𝜆 ⋆ 𝑖 | + ⃒ ⃒ 𝜆 ∘ ∩ 𝜆 ⋆ 𝑖+1 ⃒ ⃒ ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ = 𝛿 2 ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖+1 -𝜇 ⋆ 𝜆 ⋆ 𝑖 ⃦ ⃦ ⃦ 2 ℋ ≥ 𝛿 2 ∆ 2 ,
hence the result.

Proof of Lemma 3.7

In this proof, since 𝜏 is fixed, we denote by 𝜆 1 , . . . , 𝜆 𝐷 the segments of 𝜏 , that is,

𝜆 𝑖 = { 𝜏 𝑖-1 + 1, . . . , 𝜏 𝑖 } . First, notice that 𝐿 𝜏 = ⟨𝜇 ⋆ -𝜇 ⋆ 𝜏 , 𝜀⟩ = 𝐷 ⋆ ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 ⋆ 𝑖 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⟩ ℋ - 𝐷𝜏 ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 𝑖 , ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⟩ ℋ . (3.28) Now, if 𝐷 𝜏 < 𝐷 ⋆ we arbitrarily define 𝜆 𝐷𝜏 +1 = • • • = 𝜆 𝐷 ⋆ = ∅, so that ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 = 0 for every 𝑖 ∈ {𝐷 𝜏 + 1, . . . , 𝐷 ⋆ }. Similarly, if 𝐷 ⋆ < 𝐷 𝜏 , we define 𝜆 ⋆ 𝐷 ⋆ +1 = • • • = 𝜆 𝐷𝜏 = ∅. We also define 𝜇 ⋆ ∅ = 𝜇 ⋆ 𝑛 by convention. Then, defining 𝐷 + := max { 𝐷 ⋆ , 𝐷 𝜏 }
, we can rewrite Eq. (3.28) as follows:

𝐿 𝜏 = 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 ⋆ 𝑖 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⟩ ℋ - 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 𝑖 , ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⟩ ℋ = 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 𝑖 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⟩ ℋ + 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 𝑖 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 - ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⟩ ℋ = 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 𝑖 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⟩ ℋ + 𝐷 + ∑ 𝑖=1 ⟨𝜇 ⋆ 𝜆 𝑖 -𝜇 ⋆ 𝑛 , ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 - ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⟩ ℋ , since 𝐷 + ∑ 𝑖=1 ( ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 - ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ) = 0 .
Then, by the triangle inequality and Cauchy-Schwarz inequality,

|𝐿 𝜏 | ≤ 𝐷 + ∑ 𝑖=1 ⃦ ⃦ ⃦𝜇 ⋆ 𝜆 ⋆ 𝑖 -𝜇 ⋆ 𝜆 𝑖 ⃦ ⃦ ⃦ ℋ ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ℋ + 𝐷 + ∑ 𝑖=1 ⃦ ⃦ 𝜇 ⋆ 𝜆 𝑖 -𝜇 ⋆ 𝑛 ⃦ ⃦ ℋ ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 - ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ≤ diam conv { 𝜇 ⋆ 𝑗 / 𝑗 ∈ { 1, . . . , 𝑛 }} × ⎡ ⎣ 𝐷 + ∑ 𝑖=1 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ℋ + 𝐷 + ∑ 𝑖=1 ⎛ ⎝ ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 ⋆ 𝑖 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ⃦ ℋ + ⃦ ⃦ ⃦ ⃦ ⃦ ∑ 𝑗∈𝜆 𝑖 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ ⎞ ⎠ ⎤ ⎦ ≤ 3𝐷 + diam conv { 𝜇 ⋆ 𝑗 / 𝑗 ∈ { 1, . . . , 𝑛 }} × sup 1≤𝑎<𝑏≤𝑛 ⃦ ⃦ ⃦ ⃦ ⃦ 𝑏 ∑ 𝑗=𝑎 𝜀 𝑗 ⃦ ⃦ ⃦ ⃦ ⃦ ℋ where we used that 𝜇 ⋆ 𝜆 ∈ conv { 𝜇 ⋆ 𝑗 / 𝑗 ∈ { 1, . . . , 𝑛 }}
for any segment 𝜆. Since the diameter of the convex hull of a finite set of points is equal to the diameter of the set, we have

diam conv { 𝜇 ⋆ 𝑗 / 𝑗 ∈ { 1, . . . , 𝑛 }} = diam { 𝜇 ⋆ 𝑗 / 𝑗 ∈ { 1, . . . , 𝑛 }} ≤ (𝐷 ⋆ -1)∆ < 𝐷 ⋆ ∆ .
Using also Lemma 3.5, we get the result.

Proof of Lemma 3.11

Let us put

𝜁 := ‖𝜀 1 + • • • + 𝜀 𝑛 ‖ 2 ℋ . Since for any 𝑗 ̸ = 𝑘, E [⟨𝜀 𝑗 , 𝜀 𝑘 ⟩ ℋ ] = 0 (see Remark 3.5), by definition of 𝑣 𝑗 , E [𝜁] = E [ ‖𝜀 1 + • • • + 𝜀 𝑛 ‖ 2 ℋ ] = 𝑛 ∑ 𝑗=1 𝑣 𝑗 .
We recognize the right-hand side of (3.16) up to 1/𝑥 2 . For any 𝑟 > 1, let us denote by 𝐴 𝑟 the event

∀1 ≤ 𝑠 < 𝑟, ‖𝜀 1 + • • • + 𝜀 𝑠 ‖ ℋ < 𝑥 and ‖𝜀 1 + • • • + 𝜀 𝑟 ‖ ℋ ≥ 𝑥 ,
and by 𝐴 1 the event ‖𝜀 1 ‖ ℋ ≥ 𝑥. These events are disjoints, thus we can write

P ( max 1≤𝑘≤𝑛 ‖𝜀 1 + • • • + 𝜀 𝑘 ‖ ℋ ≥ 𝑥 ) = P ( 𝑛 ⋃ 𝑟=1 𝐴 𝑟 ) = 𝑛 ∑ 𝑟=1 P (𝐴 𝑟 ) . (3.29)
The law of total expectation and the positiveness of 𝜁 yield

E [𝜁] ≥ 𝑛 ∑ 𝑟=1 E [ 𝜁 ⃒ ⃒ 𝐴 𝑟 ] P (𝐴 𝑟 ) .
Finally, let ℓ ≤ 𝑟 < 𝑘 be integers. Since 𝜀 ℓ is independent from 𝜀 𝑘 conditionally to 𝜎(𝜀 1 , . . . , 𝜀 𝑟 ), 𝜀 ℓ is independent from 𝜀 𝑘 conditionally to 𝐴 𝑟 . Furthermore, 𝜀 𝑘 is independent from 𝐴 𝑟 and

E [ ⟨𝜀 𝑘 , 𝜀 ℓ ⟩ ℋ ⃒ ⃒ 𝐴 𝑟 ] = ⟨E [𝜀 𝑘 ] , E [ 𝜀 ℓ ⃒ ⃒ 𝐴 𝑟 ] ⟩ ℋ = 0 .
Because of this relation and the positivity of the (real) conditional expectation, for any integers 𝑟 ≤ 𝑘 ≤ 𝑗,

E [ 𝜁 ⃒ ⃒ 𝐴 𝑟 ] = E [ ‖𝜀 1 + • • • + 𝜀 𝑛 ‖ 2 ℋ ⃒ ⃒ 𝐴 𝑟 ] ≥ E [ ‖𝜀 1 + • • • + 𝜀 𝑟 ‖ 2 ℋ ⃒ ⃒ 𝐴 𝑟 ] ≥ 𝑥 2 . Therefore, E [ 𝜁 ⃒ ⃒ 𝐴 𝑟 ] ≥ 𝑥 2 , which gives E [𝜁] ≥ 𝑥 2 ∑ P (𝐴 𝑟 )
. This concludes the proof, thanks to Eq. (3.29).

Remark 3.5. The independence between 𝜀 𝑗 and 𝜀 𝑘 for 𝑗 ̸ = 𝑘 yields E [⟨𝜀 𝑗 , 𝜀 𝑘 ⟩ ℋ ] = 0. Indeed, we dispose of a conditional expectation on ℋ [Diestel and Uhl, 1977, chapter 5], which satisfies the same properties than the conditional expectation with real random variables. Hence we can write

E [⟨𝜀 𝑗 , 𝜀 𝑘 ⟩ ℋ ] = E [ E [ ⟨𝜀 𝑗 , 𝜀 𝑘 ⟩ ℋ ⃒ ⃒ 𝜀 𝑘 ]] = E [ ⟨E [ 𝜀 𝑗 ⃒ ⃒ 𝜀 𝑘 ] , 𝜀 𝑘 ⟩ ℋ ] = E [⟨E [𝜀 𝑗 ] , 𝜀 𝑘 ⟩ ℋ ] = 0.
Note that the 𝜀 𝑗 s expectation vanishes by hypothesis.

3.5.7 Technical lemmas for the proof of Theorem 3.2

In this we state and prove technical results used in the proof of Theorem 3.2 in Section 3.4.5.

Our first assert that the penalty shape pen L is increasing under some assumption on the penalty coefficients. Lemma 3.13. Take 𝑛 ∈ N ⋆ . Then, for any 𝑎, 𝑏 > 0 such that 𝑎 < 𝑏, the mapping

𝑓 𝑎,𝑏 : 𝑥 ↦ → 𝑥 ( 𝑎 log 𝑛 𝑥 + 𝑏 )
takes positive values and is increasing on [1, 𝑛].

Proof. Write 𝑓 ′ 𝑎,𝑏 (𝑥) = 𝑎 (log(𝑛)log(𝑥)) + 𝑏 -𝑎 > 0. The second result of this section is a simple inequality used in the proof of Theorem 3.2. Lemma 3.14. For any 𝑥 ≥ 1, 𝑥 log(𝑥) -(𝑥 + 1) log(𝑥 + 1) ≥log(𝑥) -2 .

Proof. Let 𝑥 ≥ 1. Since log (1 + 1/𝑥) ≤ 1/𝑥, we have

(𝑥 + 1) log(𝑥 + 1) -𝑥 log(𝑥) ≤ log(𝑥) + 1 + 1/𝑥 ,
and we can conclude.

The next result collects some computations that intervene at the end of the proof of Theorem 3.2. Lemma 3.15. For any 1 ≤ 𝑥 ≤ 𝑛 and 𝑦 > 0, it holds that

74𝑀 2 3 (𝑥 + 1) (log(𝑛) -log (𝑥 + 1)) 2 + 𝑦 + log(𝑛) -log(𝑥) ≤ 74𝑀 2 3 (𝑥 + 1) .
Proof. We write

74𝑀 2 3 (𝑥 + 1) (log(𝑛) -log (𝑥 + 1)) 2 + 𝑦 + log(𝑛) -log(𝑥) ≤ 74𝑀 2 3 (𝑥 + 1) (log(𝑛) -log(𝑥)) 2 + 𝑦 + log(𝑛) -log(𝑥) .
Since log(𝑛)log(𝑥) ≥ 0 and 𝑦 > 0, we have

log(𝑛) -log(𝑥) 2 + 𝑦 + log(𝑛) -log(𝑥) ≤ 1 ,
and we can conclude.

The following result is a computation regarding 𝑐 1,max .

Lemma 3.16. Under the Assumptions of Theorem 3.2,

𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ > 𝑐 1 ( 𝑦 + log 𝑛 𝐷 ⋆ + 4 ) + 74 3 𝑀 2 ( (𝑦 + 4) (𝐷 ⋆ + 2) + log 𝑛 𝐷 ⋆ ) .
Proof. We assumed that

𝑐 1 < 𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ (𝑦 + log(𝑛) + 4) - 74𝑀 2 3 (𝐷 ⋆ + 2) .
Simple algebra yields

𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ > ( 𝑐 1 + 74𝑀 2 3 (𝐷 ⋆ + 2)
) (𝑦 + log(𝑛) + 4) , and since 𝐷 ⋆ + 2 > 0,

𝑛Λ 𝜏 ⋆ ∆ 2 6𝐷 ⋆ > 𝑐 1 (𝑦 + log(𝑛) + 4) + 74𝑀 2 3 ( (𝑦 + 4) (𝐷 ⋆ + 2) + log(𝑛) ) .
We can conclude since 𝑥 ↦ → log(𝑥) is increasing and 𝑐 1 > 0.

Chapter 4

Experimental results

Abstract

In this chapter, we show how the dimension jump heuristic can be a reasonable choice for the linear penalty in simulations. We also provide empirical evidence supporting the claims of Chapter 3. Finally, we demonstrate how to compute the key quantity ∆ that appears in our theoretical results, for translation-invariant kernels. Thanks to these computations, some of them novel, we are able to study precisely the link between the maximal penalty constant and ∆. We show that, as suggested by Theorem 3.1, it is proportional to ∆ 2 all other things being equal. This chapter is partly based upon [START_REF] Garreau | Consistent change-point detection with kernels[END_REF].

Whereas Chapter 3 was dedicated to theoretical results regarding KCP, we turn to more practical issues in this chapter. The pertinence of the dimension jump heuristic for KCP is studied in Section 4.1. Section 4.2 is devoted to the demonstration of the consistency of KCP on synthetic data. Finally, in Section 4.3.1, we focus on translation-invariant kernels and study the connection between the maximal penalty constant and ∆ 2 .

Choice of the penalty constant

From now on, we focus exclusively on KCP with a penalty proportional to the number of segments, that is

pen (𝜏 ) = pen ℓ (𝜏 ) = 𝐶𝐷 𝜏 𝑛 for some 𝐶 > 0 ,
as defined in Eq. (2.3). A key practical question is the following: how do we choose a penalty constant 𝐶 such that KCP recovers the correct number of change-points? Let us denote by ̂ 𝜏 (𝐶) the segmentation estimated by KCP for a penalty constant 𝐶, and ̂ 𝐷(𝐶) the number of segments of ̂ 𝜏 (𝐶). We put 𝑐 min = 𝑐 min (𝑋 1 , . . . , 𝑋 𝑛 ) and 𝑐 max = 𝑐 max (𝑋 1 , . . . , 𝑋 𝑛 ) such that ̂ 𝐷(𝑐) = 𝐷 ⋆ for any 𝑐 ∈ [𝑐 min , 𝑐 max ], a potentially empty interval. We call 𝑐 min (resp. 𝑐 max ) the minimal (resp. maximal) penalty constant. Assuming that [𝑐 min , 𝑐 max ] is non-empty, we can reformulate the previous question as: is it possible to choose 𝑐 ∈ [𝑐 min , 𝑐 max ] in a data-driven way? Theorem 3.1 provides theoretical bounds for 𝑐 min and 𝑐 max . Namely, with high probability,

𝑐 min ≤ 𝐶 min ≈ 𝐷 ⋆ log(𝑛) and 𝑐 max ≥ 𝐶 max ≈ ∆ 2 𝑀 2 Λ 𝜏 ⋆ 𝐷 ⋆ 𝑛 , (4.1)
However, as discussed in Section 3.2.1 after Theorem 3.1, 𝐶 min and 𝐶 max are not of much use in practice for choosing an adequate penalty constant 𝑐. Indeed, they both depend on the unknown quantities 𝐷 ⋆ , Λ 𝜏 ⋆ and ∆. Furthermore, the numerical constants are such that 𝐶 min > 𝐶 max more often than not for small values of 𝑛.

In Section 4.1.1, we present the dimension jump heuristic, an empirical method that aims at choosing 𝑐 ∈ [𝑐 min , 𝑐 max ], giving a partial answer to our opening question. We demonstrate its pertinence on synthetic data. In Section 4.1.3, we show that 𝑐 min and 𝑐 max have the same dependency on 𝑛 than their theoretical counterparts 𝐶 min and 𝐶 max .

The dimension jump heuristic

A well-understood phenomenon in penalized model selection is the existence of a minimal penalty, that is a penalty function pen min such that if pen = 𝛼 pen min with 𝛼 < 1 then the dimension of the estimated model tends to be close to the dimension of the largest models. This question has been first addressed by Birgé andMassart [2001, 2007] in the fixed-design Gaussian regression framework. In particular, it is shown that the optimal penalty is twice the minimal penalty, which gives rise to the following heuristic: (i) identify the minimal penalty pen min , (ii) choose as a penalty function pen = 2 pen min [Birgé and Massart, 2007, Section 4].

Then the dimension jump heuristic for choosing 𝐶 from the data is (i) compute 𝑐 min such that ̂ 𝐷(𝑐 min ) is very large for 𝑐 < 𝑐 min and reasonable for 𝑐 > 𝑐 min , (ii) define

 𝜏 := ̂ 𝜏 (2𝑐 min ).
Given the data of ̂ 𝐷(𝑐) as a function of 𝑐, it is usually not too hard to identify 𝑐 min , that is when there is a clear jump in the values taken by ̂ 𝐷(𝑐). In this situation, it is customary to define

𝑐 min ∈ arg max 𝑐>0 { ̂ 𝐷(𝑐-) -̂ 𝐷(𝑐+) } , (4.2)
that is the penalty constant achieving the maximal dimension jump. In case of equality, we take the largest constant, in order to select preferentially segmentations with fewer segments. Let us emphasize that a clear jump is not always present. In this case, one can for instance restrict

{ 𝑐 > 0 } in Eq. (4.2) to { 𝑐 > 0 | ̂ 𝐷(𝑐) ≤ 𝐷 max }
, where 𝐷 max is a user-defined constant. We provide examples of both situations in Fig. 4-1.

Fortunately, when running the KCP algorithm, it is not necessary to compute  𝐷(𝑐) for every value of 𝑐 > 0 in order to find (4.2). The idea is to build recursively a sequence of critical constants 𝑐 𝑖 together with a sequence 𝐷 𝑖 for 𝑖 ≥ 0, such that

 𝐷(𝑐) = 𝐷 𝑖 for any 𝑐 ∈ [𝑐 𝑖 , 𝑐 𝑖+1 ].
The first terms of these sequences are 𝑐 0 = 0 and 𝐷 0 = 𝑛, since ̂ 𝐷(0) = 𝑛. Let 𝑖 ≥ 1, and suppose that we successfully computed 𝑐 𝑖-1 and 𝐷 𝑖-1 . At the end of the dynamic programming step, we obtained a vector 𝑅 ∈ R 𝑛 such that the 𝐷-th component of 𝑅 is ̂ 𝜏 (𝐷); it is the last column of the cost matrix C in the KCP algorithm, c.f. Section 2.4. Then, since 𝑅(•) is non-increasing and pen is increasing as a function of 𝐷 𝜏 , for any 𝑖 ≥ 1, 𝑐 𝑖 is such that

𝑅 ( 𝐷 𝑖 ) + 𝑐𝐷 𝑖 𝑛 = 𝑅 ( 𝐷 𝑖-1 ) + 𝑐𝐷 𝑖-1 𝑛 .
Algorithm 4.1 takes 𝑅 as an argument and output sequences 𝑐 𝑖 and 𝐷 𝑖 . It can be seen as a simplified1 version of Algorithm 3.2 in [START_REF] Arlot | Resampling and model selection[END_REF] in the case of a linear penalty.

The overall complexity of Algorithm 4.1 is O (𝑛 2 ) in the worst case, but usually a lot less steps are needed. Note also that it is possible to replace 𝑛 by 𝐷 max in Algorithm 4.1, which decreases the complexity as well.

Empirical performance of the dimension jump heuristic

We now present some empirical results on synthetic data, suggesting that the dimension jump heuristic is generally a reasonable choice of penalty constant for the linear penalty. In all the tables, 𝑥(𝜎) denotes a result 𝑥 with standard deviation 𝜎 Table 4.1 -Performances of KCP with Gaussian kernel on synthetic data with a single jump of size 𝛿 ∈ [START_REF]Illustration of the kernel-trick[END_REF]10], measured by d

(2) H . The "DJH" column corresponds to a penalty constant chosen with the dimension jump heuristic. 𝜈\𝑐 1.0 5.0 10 50 DJH 0.05 0.24 (0.02) 0.46 (0.07) 0.47 (0.02) 0.47 (0.02) 0.02 (0.08) 0.1 0.24 (0.02) 0.04 (0.13) 0.47 (0.02) 0.47 (0.02) 0.01 ⋆ (0.07) 0.5 0.24 (0.02) 0.01 ⋆ (0.01) 0.02 (0.09) 0.47 (0.02) 0.03 (0.06) 1.0 0.23 (0.03) 0.01 ⋆ (0.01) 0.01 ⋆ (0.01) 0.14 (0.22) 0.05 (0.08) Table 4.2 -Performances of KCP with Laplace kernel on synthetic data with a single jump of size 𝛿 ∈ [START_REF]Illustration of the kernel-trick[END_REF]10], measured by d

(2) H . The "DJH" column corresponds to a penalty constant chosen with the dimension jump heuristic. 𝜈\𝑐 1.0 5.0 10 50 DJH 0.05 0.24 (0.02) 0.08 (0.17) 0.47 (0.02) 0.47 (0.02) 0.01 (0.05) 0.1 0.24 (0.02) 0.02 (0.09) 0.12 (0.21) 0.47 (0.02) 0.01 (0.01) 0.5 0.24 (0.03) 0.01 ⋆ (0.01) 0.01 (0.08) 0.47 (0.02) 0.04 (0.08) 1.0 0.14 (0.09) 0.01 ⋆ (0.01) 0.02 (0.08) 0.17 (0.23) 0.03 (0.06) Table 4.3 -Performances of KCP with Gaussian kernel on synthetic data with 5 jumps of sizes in the range [START_REF]Illustration of the kernel-trick[END_REF]10], measured by d

(2) H . The "DJH" column corresponds to a penalty constant chosen with the dimension jump heuristic. 𝜈\𝑐 1.0 5.0 10 50 DJH 0.5 0.09 (0.01) 0.04 (0.07) 0.22 (0.09) 0.48 (0.02) 0.02 (0.03) 1.0 0.08 (0.02) 0.02 (0.05) 0.08 (0.08) 0.47 (0.02) 0.02 (0.03) 5.0 0.03 (0.06) 0.13 (0.09) 0.19 (0.08) 0.42 (0.10) 0.03 (0.03) 10 0.09 (0.09) 0.23 (0.10) 0.30 (0.11) 0.45 (0.08) 0.03 (0.03)

We can see in the results reported in Table 4.1 and 4.2 that the dimension jump heuristic provides a reasonable way to tune the penalty constant in the setting described at the beginning of this section. Indeed, the dimension jump heuristic generally manages to find a trade-off between (i) too small 𝑐, that yields an oversegmentation which translates into performances close to E ⌊ Λ 𝜏 ⌋ ≃ 0.25, and (ii) too large 𝑐, that yields an under-segmentation which translates into performances close to 0.5 in Table 4.1 and 4.2. Even though the dimension jump heuristic does not always finds a penalty constant in the correct region, we want to point out that the results obtained are nevertheless far better than those observed for ad hoc penalty constants chosen too small or too large.

We report in Table 4.3 further results that confirms the good behavior of the dimension jump heuristic in the same setting, this time for multiple change-points. We do not report results for the Laplace kernel, that are similar.

Finally, we also present some results for a change in the variance of a Gaussian sequence of observations. We consider as before 𝑛 = 200 data-points, with a single random break-point drawn uniformly (though the minimal segment size is fixed to 5). The standard deviation of the observations before the change-point is set to 1.0, and 5.0 after, while the mean stays at zero. This data generation procedure is repeated 100 times for each choice of 𝐶 and 𝜈. The results of this experiment are reported in Table 4.4. As before, we observe that the dimension jump heuristic provides a reliable choice for 𝐶, if not the best. We do not report results for the Laplace kernel, which are similar.

Note that the sampling of the change-point locations is slightly different from before. It is more probable to observe short segments with uniform sampling rather than binomial sampling -a multinomial with parameters (𝑛; 1/2, 1/2) is a binomial with parameters (𝑛, 1/2) -, hence it is a slightly more difficult problem.

multinomial distribution with parameters (𝑛; 𝑝 1 , . . . , 𝑝 𝑘 ). (2) H . The "DJH" column corresponds to a penalty constant chosen with the dimension jump heuristic. 𝜈\𝑐 1.0 5.0 10 50 DJH 0.5 0.36 (0.07) 0.22 (0.14) 0.22 (0.14) 0.22 (0.14) 0.15 (0.13) 1.0 0.35 (0.08) 0.24 (0.15) 0.24 (0.15) 0.24 (0.15) 0.06 (0.07) 5.0 0.35 (0.08) 0.01 (0.03) 0.16 (0.14) 0.24 (0.14) 0.03 (0.07) 10 0.34 (0.09) 0.01 (0.02) 0.03 (0.05) 0.24 (0.15) 0.03 (0.06)

Minimal and maximal penalty constant

In this section, we show how a modification of Algorithm 4.1 can give the values of 𝑐 min and 𝑐 max , when they exist, and we compare these values to the theoretical bounds 𝐶 min and 𝐶 max .

Computing 𝑐 min and 𝑐 max . Suppose that we have access to the true dimension 𝐷 ⋆ . Since Algorithm 4.1 provides all the critical constants, it is a straightforward modification to recover those corresponding to 𝐷 ⋆ when they exist. We call Algorithm 4.2 this modification. It outputs 𝑐 min and 𝑐 max such that ̂ 𝐷(𝑐) = 𝐷 ⋆ for any 𝑐 ∈ [𝑐 min , 𝑐 max ]. As a convention, if there is no penalty constant 𝑐 such that ̂ 𝐷(𝑐) = 𝐷 ⋆ , the algorithm outputs the null value for both 𝑐 min and 𝑐 𝑚𝑎𝑥 . The computational complexity of this procedure is at most O (𝐷 ⋆ ).

Experimental results. We present in Fig. 4-2 the results of the following experiment: for each 𝑛 ranging from 5 to 300, we generate a piecewise-constant function with 5 segments of equal length and values on each segment alternating between 0 and 5. We then add noise sampled from a standard Gaussian distribution, and find 𝑐 min and 𝑐 max according to Algorithm 4.2 for Gaussian kernel KCP (𝜈 = 1.0). The preceding process is repeated 100 time for each value of 𝑛.

In the present setting, 𝐷 ⋆ , ∆, 𝑀 and Λ 𝜏 ⋆ are fixed. Therefore, Theorem 3.1 states that, with high probability, 𝐶 min is proportional to log(𝑛) and 𝐶 max is proportional to 𝑛. We observe that the empirical evidence suggests that the dependency on 𝑛 of 𝑐 min and 𝑐 max is the same than the dependency on 𝑛 of 𝐶 min and 𝐶 max : 𝑐 min ∝ log(𝑛) and 𝑐 max ∝ 𝑛.

In the next section, we vary 𝑛 as well in our experiments, but we interest ourselves in d (2) ∞ (̂ 𝜏 𝑛 , 𝜏 ⋆ ). segmentation and the estimated segmentation for each value of 𝑛. The error bars are ±̂ 𝜎/ √ 𝑁 , where ̂ 𝜎 is the empirical standard deviation over 𝑁 = 10 3 repetitions. We want to emphasize that, though these experiments illustrate our main result Theorem 3.1, they are carried out in a slightly different setting since the penalty constant 𝐶 is not chosen according to (3.3), but using the dimension jump heuristic as explained in Section 4.1.

H (𝜏 ⋆ , ̂ 𝜏 𝑛 ) towards 0 when the number of data points 𝑛 is increasing. A linear regression between log(𝑛) and 1 𝑛 d

(2) H (𝜏 ⋆ , ̂ 𝜏 𝑛 ) for 𝑛 ≥ 300 yields slope estimates -0.97, -1.04 and -1.00, respectively.

The three segmentation problems considered here are quite different in nature, but all lead to a linear convergence rate (slopes close to -1 on the graphs of Fig. 4-4) with different constants (different values for the intercept on the graphs of Fig. 4-4). Recall that Theorem 3.1 combined with Lemma 3.1 states that, with high probability,

1 𝑛 d (2) H (𝜏 ⋆ , ̂ 𝜏 𝑛 )  𝑣 1 = 𝐷 ⋆ 𝑀 2 ∆ 2 • log(𝑛) 𝑛 .
Hence, whenever 𝐷 ⋆ , ∆ and 𝑀 are fixed, 1 𝑛 d

(2) H (𝜏 ⋆ , ̂ 𝜏 𝑛 ) converges to 0 at rate at least log(𝑛)/𝑛 when the number of data points increases. In our experimental setting, these quantities are fixed, and the observed convergence rate matches our theoretical upper bound. The performance of KCP still depends on the regression function 𝜇 𝑚 experimentally, by a constant multiplicative factor, like the theoretical bound ̃ 𝑣 1 . We want to emphasize that other choices of 𝜈 can lead to another ordering of the speeds of convergence observed in Fig. 4-4. To put it plainly, for another bandwidth, e.g., 𝜈 = 1.0, 1 𝑛 d (1) ∞ (̂ 𝜏 𝑛 , 𝜏 ⋆ ) can converge more quickly to zero for 𝜇 3 than for 𝜇 1 .

Detecting changes in the number of modes. Let us now consider observations 𝑋 1 , . . . , 𝑋 𝑛 ∈ R whose distribution vary only through the number of modes. Can we accurately detect such changes with the KCP procedure? The data are generated according to the following process for several 𝑛:

-Set 𝜏 ⋆ 1 = ⌊𝑛/3⌋ and 𝜏 ⋆ 2 = ⌊2𝑛/3⌋; -Draw 𝑋 1 , . . . , 𝑋 𝜏 ⋆ 1 , 𝑋 𝜏 ⋆ 2 +1 , . . . , 𝑋 𝑛 according to a standard Gaussian distribution, and 𝑋 𝜏 ⋆ 1 +1 , . . . , 𝑋 𝜏 ⋆ 2 according to a (1/2, 1/2)-mixture of Gaussian distributions 𝒩 (𝛿, 1 -𝛿 2 ) and 𝒩 (-𝛿, 1 -𝛿 2 ), with 𝛿 = 0.999; the 𝑋 𝑖 are independent.

We test KCP with various kernels assuming that the number of change-points (𝐷 ⋆ = 3) is known; this simplification avoids possible artifacts linked to the choice of the penalty constant. Results are shown on Fig. 4-5. The 𝑋 𝑖 all have zero mean and unit variance, hence a classical penalized least-squares procedure -KCP with the linear kernel-is expected to detect poorly the changes in the distribution of the 𝑋 𝑖 , as confirmed by Fig. 4-5 (for instance, according to the right panel, it is not consistent). On the contrary, a Gaussian kernel with well-chosen bandwidth yields much better performance according to the middle and right panels of Fig. 4-5 (with a rate of order 1/𝑛).

H (𝜏 ⋆ , ̂ 𝜏 𝑛 ) vs. 𝑛 in log scale, for KCP with a Gaussian kernel with bandwidth 0.01 (blue solid line; estimated slope -1.05) and with the linear kernel (red dashed line; estimated slope 0.16).

Translation-invariant kernels

Experiments for finite sample size in Arlot et al. [2012, Section 6] demonstrate how KCP can detect changes in the distribution of the data, even though the mean and variance are fixed. In this section, on the contrary, we focus on simple examples where the mean or variance of a sequence of Gaussian random variables vary, and we bring out the role of the kernel in KCP's ability to accurately detect changes.

Recall that Theorem 3.1 suggests that ∆ 2 is a quantity of interest regarding the performances of KCP. In particular, Theorem 3.1 states that, with high probability,

𝑐 max ≥ 𝐶 max ≈ ∆ 2 𝑀 2 ,
up to factors depending on 𝜏 ⋆ , 𝜇 ⋆ and 𝑛. We restrict our study to translation-invariant kernels, since it is possible to compute ∆ 2 in closed form for some kernels belonging to this class when the noise is Gaussian. In this setting, we show that 𝑐 max ∝ ∆ 2 /𝑀 2 holds experimentally.

We first introduce translation-invariant kernels in Section 4.3.1, and show how to compute ∆ 2 in this setting. In Section 4.3.2, we go through the details of theses computations when 𝑘 = 𝑘 G and 𝑘 = 𝑘 L , with Gaussian observations. Finally, we use the results of these calculations in Section 4.3.3, where we demonstrate experimentally that 𝑐 max ∝ ∆ 2 /𝑀 2 .

Introduction

We begin with a definition. In other words, 𝑘 (𝑥, 𝑦) only depends on the difference between the vectors 𝑥 and 𝑦. It is an attractive property, for instance when one wishes to detect changes in the mean of a signal: a priori, detecting a change between 0 and 5 is as important as detecting a change between 5 and 10.

If furthermore the function 𝜅 is continuous, then 𝜅 is called a positive-definite function [START_REF] Berg | Harmonic analysis on semigroups[END_REF]. We assume that 𝑘 denotes a translation-invariant kernel such that 𝑀 = 1 from now on. For instance, the Gaussian kernel and the Laplace kernel are both translation-invariant. [START_REF] Bochner | Vorlesungen über Fouriersche Integrale[END_REF][START_REF] Bochner | Monotone funktionen, stieltjessche integrale und harmonische analyse[END_REF] showed there is a deep correspondence between positive semidefinite functions and nonnegative measures on R. More precisely, let us define the Fourier transform of any integrable function 𝑓 as ℱ 𝑓 (𝜔) := ∫ e -i𝑥𝜔 𝑓 (𝑥) d 𝑥. Then Theorem 4.1 (Bochner). A continuous function 𝜅 : R 𝑑 → C is positive semi-definite if, and only if, it is the Fourier transform of a finite nonnegative Borel measure 𝜈 on R 𝑑 , that is

∀𝑥 ∈ R 𝑑 , 𝜅(𝑥) = ℱ 𝜈(𝑥) = ∫ e -i𝑥 ⊤ 𝜔 d 𝜈(𝜔) .
It is possible to be more precise than Theorem 4.1 if we assume that 𝜅 is realvalued and integrable. More precisely, we have the following description of the RKHS associated to 𝑘. Theorem 4.2 [START_REF] Wendland | Scattered data approximation[END_REF], Theorem 10.12). Let 𝑘 be a translation-invariant kernel such that 𝜅 is integrable on R 𝑑 as well as its Fourier transform ℱ 𝜅.

‖𝜇 ⋆ 𝑖 -𝜇 ⋆ 𝑗 ‖ 2 ℋ = 1 2𝜋 ∫ | e -i𝐿 𝑖 𝜔 ℱ 𝜌 𝑖 (𝜔) -e -i𝐿 𝑗 𝜔 ℱ 𝜌 𝑗 (𝜔)| 2 ℱ 𝜅(𝜔) d 𝜔 .
Proof. Recall that the mean elements 𝜇 ⋆ 𝑖 satisfy

∀𝑔 ∈ ℋ, ⟨𝜇 ⋆ 𝑗 , 𝑔⟩ ℋ = E [𝑔(𝑋 𝑗 )] = E [⟨𝑌 𝑗 , 𝑔⟩ ℋ ] .
Let 𝑦 be a real number. Take 𝑔 = 𝑘 (•, 𝑦) in the previous display, we obtain

𝜇 ⋆ 𝑖 (𝑦) = E [𝑘 (𝑋 𝑖 , 𝑦)] = ∫ 𝜅(𝐿 𝑖 + 𝑥 -𝑦)𝜌 𝑖 (𝑥) d 𝑥 .
Since 𝜅 is symmetric,

𝜇 ⋆ 𝑖 (𝑦) = ∫ 𝜅(𝑦 -𝐿 𝑖 -𝑥)𝜌 𝑖 (𝑥) d 𝑥 = ((𝜅 ⋆ 𝜌 𝑖 ) ∘ 𝜏 𝐿 𝑖 ) (𝑦) ,
where ⋆ denotes the convolution operation and for any 𝑎 ∈ R, 𝜏 𝑎 : 𝑥 ↦ → 𝑥 -𝑎.

Elementary properties of the Fourier transform yields

ℱ 𝜇 ⋆ 𝑖 (𝜔) = e -i𝐿 𝑖 𝜔 ℱ 𝜅(𝜔) • ℱ 𝜌 𝑖 (𝜔) .
We conclude the proof by a straightforward application of Theorem 4.2.

Computations

We now give a few concrete examples of ∆ 2 computations under a Gaussian noise hypothesis. In the following, we give results for a single change-point, with the convention that 𝐿 1 and 𝜌 1 are the mean and distribution before the jump, whereas 𝐿 2 and 𝜌 2 denote the mean and distribution after the jump. Note that stating these results can be straightforwardly adapted to an arbitrary number of change-points.

We first consider the Gaussian kernel, with a change in the mean.

Proposition 4.2 (Gaussian kernel, change in the mean). Set 𝑘 = 𝑘 G . Suppose that Assumption 4.1 holds true with 𝐿 2 -𝐿 1 = 𝛿 and 𝛽 1 , 𝛽 2 ∼ 𝒩 (0, 𝜎 2 ), where 𝜎 is a positive number. Then

∆ 2 = 2𝜈 √ 𝜈 2 + 2𝜎 2 ( 1 -exp ( -𝛿 2 2 (𝜈 2 + 2𝜎 2 ) )) .
Proof. The Gaussian kernel with bandwidth 𝜈 > 0 is a translation-invariant kernel with

𝜅(𝑧) = exp ( -𝑧 2 2𝜈 2 ) and ℱ 𝜅(𝜔) = 𝜈 √ 2𝜋 exp ( -𝜔 2 𝜈 2 2 ) . Moreover, ℱ 𝛽 1 (𝜔) = ℱ 𝛽 2 (𝜔) = exp (-𝜎 2 𝜔 2 /2). Prop. 4.1 then yields ∆ 2 = 4𝜈 √ 2𝜋 ∫ sin 2 𝛿𝜔 2 exp ( - ( 𝜎 2 + 𝜈 2 2 ) 𝜔 2 ) d 𝜔 ,
and a computation concludes the proof.

It is possible to prove an analogous result for a change in the variance.

∆ 2 = 𝜈 ( 1 √ 2𝜎 2 1 + 𝜈 2 + 1 √ 2𝜎 2 2 + 𝜈 2 - 2 √ 𝜎 2 1 + 𝜎 2 2 + 𝜈 2 ) .
Proof. As in the proof of Prop.

∆ 2 = 𝜈 √ 2𝜋 ∫ ( e -𝜎 2 1 𝜔 2 2 -e -𝜎 2 1 𝜔 2 2 ) 2 e -𝜔 2 𝜈 2 2 d 𝜔 ,
and a computation concludes the proof.

We now turn to the Laplace kernel. A technical lemma is needed. Then

𝑓 (𝜆) = 𝜋 e 𝑎/𝑏 2 √ 𝑏 [ e 𝜆/ √ 𝑏 ( 1 -erf ( 𝜆 2 √ 𝑎 + √ 𝑎 𝑏 )) + e -𝜆/ √ 𝑏 ( 1 + erf ( 𝜆 2 √ 𝑎 - √ 𝑎 𝑏 ))] . In particular, ∫ e -𝑎𝜔 2 1 + 𝑏𝜔 2 d 𝜔 = 𝜋 √ 𝑏 e 𝑎 𝑏 erfc (√ 𝑎 𝑏 ) .
The proof of Lemma 4.1 is postponed to the end of this section. We are now able to prove analogous statements to Prop. 4.2 and 4.3.

Proposition 4.4 (Laplace kernel, change in the mean). Set 𝑘 = 𝑘 L . Suppose that Assumption 4.1 holds true with 𝐿 2 -𝐿 1 = 𝛿, and 𝛽 1 , 𝛽 2 ∼ 𝒩 (0, 𝜎 2 ), where 𝜎 is a positive number. Then

∆ 2 = e 𝜎 2 4𝜈 2 ( 2 erfc ( 𝜎 2𝜈 ) -e 𝛿/2𝜈 erfc ( 𝛿 2𝜎 + 𝜎 2𝜈 ) -e -𝛿/2𝜈 ( 1 + erf ( 𝛿 2𝜎 -𝜎 2𝜈 ))) .
Proof. The Laplace kernel with bandwidth 𝜈 > 0 is a translation-invariant kernel with

𝜅(𝑧) = exp ( -|𝑧| 2𝜈 ) and ℱ 𝜅(𝜔) = 4𝜈 1 + 4𝜈 2 𝜔 2 . Moreover, ℱ 𝛽 1 (𝜔) = ℱ 𝛽 2 (𝜔) = exp (-𝜎 2 𝜔 2 /2). Prop. 4.1 then yields ∆ 2 = 4𝜈 𝜋 ∫ (1 -cos(𝛿𝜔)) 1 + 4𝜈 2 𝜔 2 e -𝜎 2 𝜔 2 d 𝜔 ,
and we conclude the proof with Lemma 4.1.

Proposition 4.5 (Laplace kernel, change in the variance). Set 𝑘 = 𝑘 L . Suppose that Assumption 4.1 holds true with 𝐿 1 = 𝐿 2 , 𝛽 1 ∼ 𝒩 (0, 𝜎 2 1 ), and 𝛽 2 ∼ 𝒩 (0, 𝜎 2 2 ), where 𝜎 1 and 𝜎 2 are positive numbers. Then

∆ 2 = e 𝜎 2 1 4𝜈 2 erf (𝜈) ( 1 √ 2𝜎 2 1 + 𝜈 2 + 1 √ 2𝜎 2 2 + 𝜈 2 - 2 √ 𝜎 2 1 + 𝜎 2 2 + 𝜈 2 ) 𝑐 𝜎 1 2𝜈 + e 𝜎 2 2 4𝜈 2 erfc ( 𝜎 2 2𝜈 ) -2 e 𝜎 2 1 +𝜎 2 2 8𝜈 2 erfc (√ 𝜎 2 1 + 𝜎 2 2 8𝜈 2 ) . ( 4
∆ 2 = 2𝜈 𝜋 ∫ e -𝜎 2 1 𝜔 2 + e -𝜎 2 2 𝜔 2 -2 e -𝜎 2 1 +𝜎 2 2 2 𝜔 2 1 + 4𝜈 2 𝜔 2 d 𝜔 ,
we can conclude with Lemma 4.1.

Remark 4.1. We have noted before that, in the single change-point setting, ∆ coincides with the MMD. As such, computations similar to Prop. 4.2, 4.3, 4.4 and 4.5 exist in the kernel two-sample test literature. In particular, Prop. 4.2 is consequence of Prop. 1 in [START_REF] Reddi | On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions[END_REF], and the content of Prop. 4.3 appears in the proof of Prop. 3 in Reddi et al. [2015]. Prop. 4.4 and 4.5,however, are new up to the best of our knowledge.

Proof of Lemma 4.1. We note that 𝑓 is well defined and twice differentiable. It is clear that

𝑓 ′′ (𝜆) = - ∫ 𝜔 2 e -𝑎𝜔 2 cos(𝜆𝜔) 1 + 𝑏𝜔 2 d 𝜔 ,
hence 𝑓 satisfies a second order linear differential equation, namely 4𝑎) .

𝑓 ′′ (𝜆) - 1 𝑏 𝑓 (𝜆) = -1 𝑏 √ 𝜋 𝑎 e -𝜆 2 /(
The solutions of this equation can be found with the variation of constants method, and we have, for 𝑐 1 and 𝑐 2 unknown constants,

𝑓 (𝜆) =𝑐 1 e -𝜆/ √ 𝑏 +𝑐 2 e 𝜆/ √ 𝑏 - 𝜋 e 𝑎/𝑏 2 √ 𝑏 [ e 𝜆/ √ 𝑏 erf ( 𝜆 2 √ 𝑎 + √ 𝑎 𝑏 ) -e -𝜆/ √ 𝑏 erf ( 𝜆 2 √ 𝑎 - √ 𝑎 𝑏 )] . (4.5)
Since

𝑓 (0) = 𝜋 e 𝑎/𝑏 √ 𝑏 ( 1 -erf (√ 𝑎 𝑏 )) ,
we get 𝑐 1 + 𝑐 2 = 𝜋 e 𝑎/𝑏 / √ 𝑏. On the other side, we know that 𝑓 is bounded for 𝜆 → ∞ thus 𝑐 2 = 𝜋 e 𝑎/𝑏 /(2 √ 𝑏), hence 𝑐 1 = 𝑐 2 = 𝜋 e 𝑎/𝑏 /(2 √ 𝑏), and we conclude the proof.

Empirical study

Let us use the theoretical results of Section 4.3.2 to study the connection between 𝑐 max and ∆ 2 .

The sample size is set to 𝑛 = 200. We consider two regression functions:

-𝜇 . In a nutshell, 𝑋 1 and 𝑋 2 are semi-deterministic versions of the signals we used so far in this chapter -there is still randomness in the noise. For 100 values of 𝜈 in [0, 20], we generate 100 samples of 𝑋 1 and 𝑋 2 . For each sample, we compute 𝑐 max via algorithm 4.2, for 𝑋 1 using both the Gaussian and the Laplace kernel, for 𝑋 2 using only the Laplace kernel. We also compute ∆ 2 (𝜈) thanks to Prop. 4.2,4.3 and 4.4. The results are reported in Fig. 456.

Again, the bound given in Theorem 3.1 is accurate, and ∆ 2 and 𝑐 max are linearly correlated -all 𝑅 2 coefficients we obtained for a linear regression of 𝑐 max versus ∆ 2 are > 0.999. We find this result remarkable for two reasons.

-First, it shows that the analysis conducted in Chapter 3 brings out a relevant quantity, ∆ 2 . Note however that we got rid of the influence of the segment sizes in the present experiments. We think that the true quantity of interest is a trade-off between size of the jump in the RKHS and size of the segments adjacent to the jump. We refer to Eq. (3.27), the point in our analysis where we begin working with de-correlated jump size and segment size.

-Second, this link between 𝑐 max and ∆ gives us a route to choosing the kernel for KCP. By taking a kernel that maximizes ∆, we have a shot at maximizing 𝑐 max .

Of course ∆ depends on unknown quantities -for instance, the size of the jump that we are trying to detect -, but it is conceivable to find an estimator of ∆ which could act as a proxy for this goal.

Finally, note that we also report in Fig. 4-6 the bandwidth proposed by the median heuristic. This simple heuristic consist to pick 2𝜈 2 = Med { |𝑋 𝑖 -𝑋 𝑗 | 2 } for the Gaussian kernel, where Med is the sample median. As we can see, it is generally close to the global maximum of ∆ 2 , and we recommend the use of this heuristic for choosing the bandwidth of the Gaussian and Laplace kernels.

We study the median heuristic in more depth in the next chapter. kernel -also called Laplace or Laplacian kernel-, whereas more exotic 𝑓 give rise to less common kernels such as the rational quadratic kernel, the wave kernel or the Matérn kernel [see Genton, 2001, and references therein].

It is well-known that the performances of kernel methods depend highly on the kernel choice, see for instance [START_REF] Sriperumbudur | Kernel choice and classifiability for RKHS embeddings of probability distributions[END_REF]. However, the calibration of the bandwidth 𝜈 is perhaps even more important as the choice of 𝑓 [Schölkopf and Smola, 2002, Sec. 4.4.5].

Since the Gram matrix depends only on the ‖𝑥 𝑖 -𝑥 𝑗 ‖ /𝜈 in this case, it is common sense to pick 𝜈 of the same order than the family of all pairwise distances (‖𝑥 𝑖 -𝑥 𝑗 ‖) 1≤𝑖,𝑗≤𝑛 . As an example, suppose that we settled for the Gaussian kernel 𝑘 G . Then when 𝜈 → 0, the Gram matrix K is the identity matrix, and when 𝜈 → ∞, the components of K are constant equal to 1. All relevant information about the data is lost in both these extreme cases. This is a general phenomenon, even though the values taken by K in the degenerate cases depend on the function 𝑓 . Hence a reasonable middle-ground for choosing 𝜈 is to pick a value "in the middle range" of the (‖𝑥 𝑖 -𝑥 𝑗 ‖) 1≤𝑖,𝑗≤𝑛 , that is, an empirical quantile, which is often set to be the median. This strategy is called the median heuristic.

As noted in [START_REF] Flaxman | Bayesian learning of kernel embeddings[END_REF], the origin of the median heuristic is quite unclear and does not appear in the monograph of [START_REF] Schölkopf | Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Adaptive Computation and Machine Learning[END_REF], while it has become the main reference for this heuristic. The earliest appearance of the median heuristic that we know of is in Sriperumbudur et al. [2009, Sec. 5]. Let us also mention Gretton et al. [2012a], which refers to [START_REF] Takeuchi | Nonparametric quantile estimation[END_REF] and [START_REF] Schölkopf | Kernel principal component analysis[END_REF] for similar heuristics.

Nevertheless, the median heuristic is extensively used in practice. In a supervised learning setting, e.g., kernel SVM [START_REF] Boser | A training algorithm for optimal margin classifiers[END_REF] or kernel ridge regression [START_REF] Hoerl | Ridge regression: Biased estimation for nonorthogonal problems[END_REF], one can always resort to a grid-search for the choice of 𝜈. The performance of each prescribed value for 𝜈 is then evaluated by cross-validation. But whenever cross-validation is too expensive, the median heuristic can provide a good alternative. It is for instance the default bandwidth choice in the kernel SVM implementation of the 𝑅 language kernlab package [START_REF] Karatzoglou | kernlab -an S4 package for kernel methods in R[END_REF].

In the context of hypothesis testing, a possibility is to try and find the bandwidth that maximizes the test power. This is for instance what is done in [Gretton et al., 2012a;[START_REF] Jitkrittum | An adaptive test of independence with analytic kernel embeddings[END_REF][START_REF] Sutherland | Generative models and model criticism via optimized maximum mean discrepancy[END_REF]. However, it can be challenging to compute the test power, and further to maximize it with respect to the bandwidth. In the unsupervised setting, the choice of the kernel and more specifically the choice of the bandwidth 𝜈 has no definitive answer. Hence heuristics as the median heuristic are among the first choice that comes to mind when one has to choose a bandwidth, and numerous authors report using the median heuristic in their experiments [START_REF] Sriperumbudur | Kernel choice and classifiability for RKHS embeddings of probability distributions[END_REF][START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF][START_REF] Reddi | On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions[END_REF][START_REF] Zhang | Large-scale kernel methods for independence testing[END_REF][START_REF] Jitkrittum | An adaptive test of independence with analytic kernel embeddings[END_REF][START_REF] Muandet | Kernel mean shrinkage estimators[END_REF][START_REF] Sutherland | Generative models and model criticism via optimized maximum mean discrepancy[END_REF]. Note that it is also possible to use other rule-of-thumbs, coming for instance from kernel density estimation where the problem of the bandwidth choice is also of the utmost importance -see, e.g., [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF] in the context of kernel change-point detection.

Related work

Despite its popularity, there is very little theoretical understanding of the median heuristic. To the best of our knowledge, the only work in this direction is contained in [START_REF] Reddi | On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions[END_REF]. They observe that the median of all the pairwise distances has to be close to the mean pairwise distance E ‖𝑋 𝑖 -𝑋 𝑗 ‖, the computation of which is then used to obtain the asymptotic of the median heuristic when the dimension of the data goes to infinity. This argument can be made rigorous by observing that, given a random variable 𝑋 with a second order moment, the following inequality holds [START_REF] Mallows | Another comment on O'Cinneide[END_REF]:

|E 𝑋 -med(𝑋)| ≤ √ Var (𝑋) .
Hence the observation of [START_REF] Reddi | On the decreasing power of kernel and distance based nonparametric hypothesis tests in high dimensions[END_REF] is correct, up to a variance term. We will see in Section 5.4 that our results make this insight more precise.

Outline

Our goal in this chapter is to obtain a precise understanding of the median heuristic for a large sample size in the setting of kernel two-sample test and off-line kernel change-point detection. Our setting is made explicit in Section 5.2, and we show in the same section how it is relevant for these applications. In Section 5.3, we claim our main result: the median heuristic is asymptotically normal when the number of observations goes to ∞. In particular, the median heuristic converges towards the theoretical median of a target distribution that we describe completely. This result is obtained thanks to an auxiliary proposition that we think has an interest of its own, namely a central limit theorem for a certain class of 𝑈 -statistics that we state and prove in the same section. We demonstrate with numerical experiments the validity of our claims.

Setting

Given any random variable 𝑍, the notation 𝑍 ′ will stand for an independent copy of 𝑍. Unless specified in subscript, the expected value is taken with respect to all the random variables that appear in the expression. For instance, E [ℎ(𝑋, 𝑌 )] means E 𝑋,𝑌 [ℎ(𝑋, 𝑌 )]. We also denote by 𝐿 2 (𝑃 ) the space of real functions such that E [𝑓 (𝑋) 2 ] < +∞ where 𝑋 ∼ 𝑃 .

In the following, we suppose that we are given a triangular array of independent R 𝑑 -valued random variables. Namely, for each 𝑛, we suppose that the observations are drawn from line 𝑛 of the following scheme:

𝑋 1,1 𝑋 2,1 𝑋 2,2 . . . . . . 𝑋 𝑛,1 • • • • • • 𝑋 𝑛,𝑛 . . . . . .
Let 𝑋 (resp. 𝑌 ) be a R 𝑑 -valued random variable following the law 𝑃 (resp. 𝑄). Our main hypothesis on the distribution of the 𝑋 𝑛,𝑖 is the following:

Assumption 5.1. There exists 𝛼 ∈ (0, 1) such that 𝑋 𝑛,𝑖 ∼ 𝑃 for any 𝑖 ≤ 𝛼𝑛 and 𝑋 𝑛,𝑖 ∼ 𝑄 otherwise.

We will assume from now on that 𝛼𝑛 is an integer. Everything that follows can be readily adapted by replacing 𝛼𝑛 with ⌊𝛼𝑛⌋ when it is needed. Assumption 5.1 means that our observations are split in two segments, {1, . . . , 𝛼𝑛} and {𝛼𝑛 + 1, . . . , 𝑛}. On the left segment, they follow 𝑃 and on the right segment they follow 𝑄, as illustrated below:

𝑋 𝑛,1 𝑋 𝑛,2 • • • 𝑋 𝑛,𝛼𝑛 ⏟ ⏞ ∼𝑃 ⃒ ⃒ ⃒ 𝑋 𝑛,𝛼𝑛+1 • • • 𝑋 𝑛,𝑛 ⏟ ⏞ ∼𝑄

Connection with kernel two-sample test

Let us briefly recall the modus operandi of kernel two-sample test. Suppose that, for a given 𝑛, we sample observations 𝑥 1 , . . . , 𝑥 𝑀 of 𝑋 and observations 𝑦 1 , . . . , 𝑦 𝑁 of 𝑌 . The goal of two-sample test is to decide whether 𝑃 = 𝑄 or 𝑃 ̸ = 𝑄 given these observations. [START_REF] Gretton | A kernel method for the two-sample-problem[END_REF] have proposed a kernel method for two-sample testing, that relies on the mean embeddings of 𝑃 and 𝑄 inside the reproducing kernel Hilbert space ℋ associated with 𝑘. Let us call 𝜇 𝑃 and 𝜇 𝑄 these embeddings, then a good measure of proximity between the distributions 𝑃 and 𝑄 is the so called maximum mean discrepancy (MMD), that can be written ‖𝜇 𝑃 -𝜇 𝑄 ‖ ℋ . It is also possible to write the (squared) MMD as

MMD 2 (𝑝, 𝑞) = E [𝑘 (𝑥, 𝑥 ′ )] -2 E [𝑘 (𝑥, 𝑦)] + E [𝑘 (𝑦, 𝑦 ′ )] .
It is proven in Gretton et al. [2012a] that an unbiased estimate of this quantity is

MMD 2 (𝑃, 𝑄) = 1 𝑀 (𝑀 -1) 𝑀 ∑ 𝑖=1 𝑀 ∑ 𝑗̸ =𝑖 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) + 1 𝑁 (𝑁 -1) 𝑁 ∑ 𝑖=1 𝑁 ∑ 𝑗̸ =𝑖 𝑘 (𝑦 𝑖 , 𝑦 𝑗 ) - 2 𝑀 𝑁 𝑀 ∑ 𝑖=1 𝑁 ∑ 𝑗=1 𝑘 (𝑥 𝑖 , 𝑦 𝑗 ) .
The setting that we described in Section 5.2 corresponds to the kernel two-sample test setting when we take 𝑀 = 𝛼𝑛, 𝑁 = (1 -𝛼)𝑛 and the 𝑥 𝑖 are realizations of the 𝑋 𝑛,𝑖 . Letting 𝑛 grow to infinity will correspond to let both 𝑀 and 𝑁 grow to infinity with the ratio 𝑀/𝑁 constant equal to 𝛼/(1 -𝛼).

Connection with kernel change-point detection

We restrict here to the case when there is a single change-point, thus both the approaches of [START_REF] Harchaoui | Retrospective multiple change-point estimation with kernels[END_REF] and [START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF] coincide and may be summarized as follow: suppose that we are given observations 𝑥 1 , . . . , 𝑥 𝑛 ∈ 𝒳 drawn from 𝑋 1 , . . . , 𝑋 𝑛 such that 𝑃 𝑋• is a step function. That is, there exists an unknown change-point 1 ≤ 𝜏 1 ≤ 𝑛 such that the 𝑋 𝑖 share a common distribution for 1 ≤ 𝑖 ≤ 𝜏 1 , say 𝑃 , and another for 𝜏 1 < 𝑖 ≤ 𝑛, say 𝑄 ̸ = 𝑃 . The kernel change-point detection procedures then consider the minimization of the kernel least-squares criterion

Minimize 1≤𝜏 1 ≤𝑛 { 1 𝑛 𝑛 ∑ 𝑖=1 𝑘 (𝑥 𝑖 , 𝑥 𝑖 ) - 1 𝑛 [ 1 𝜏 1 𝜏 1 ∑ 𝑖,𝑗=1 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) + 1 𝑛 -𝜏 1 𝑛 ∑ 𝑖,𝑗=𝜏 1 +1 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) ]}
to estimate 𝜏 1 . This corresponds to our setting when we let the 𝑥 𝑖 be realizations of 𝑋 𝑛,𝑖 for any 1 ≤ 𝑖 ≤ 𝑛 and set 𝜏 1 = 𝛼𝑛.

Remark 5.1. Our setting can be easily adapted to multiple change-points, that is, when there are more than one change-point. Indeed, set 𝐷 the -possibly unknownnumber of segments, and 𝛼 1 , . . . , 𝛼 𝐷 positive numbers such that ∑ ℓ 𝛼 ℓ = 1 the lengths of the segments. In addition, set 𝛼 0 = 𝛼 𝐷+1 = 0 and let 𝑋 𝑛,𝑖 follow the distribution

𝑃 ℓ if 𝛼 0 + • • • + 𝛼 ℓ-1 𝑛 < 𝑖 ≤ 𝛼 0 + • • • + 𝛼 ℓ 𝑛.
Then the single change-point case corresponds to 𝐷 = 2 and 𝛼 1 = 𝛼. Note that this is the asymptotic interpretation of off-line change-point detection presented in Chapter 1: taking 𝑛 → ∞ corresponds to letting the number of observations on each segments grow to infinity while the unknown change-points remains constant.

The median heuristic

Suppose that we use a kernel that has the form (5.1) for a fixed 𝑓 . Both for kernel two-sample test and kernel change-point detection, the choice of the kernel thus boils down to the choice of the bandwidth 𝜈. The power of the kernel two-sample test procedure that we recalled in Sec. 5.2.1 is known to have maximum power when the kernel maximizes the MMD divided by its standard deviation [Gretton et al., 2012b]. This is approximately true for the quadratic MMD as well [START_REF] Sutherland | Generative models and model criticism via optimized maximum mean discrepancy[END_REF]. A similar situation occurs in the kernel change-point setting of Sec. 5.2.2: as we have seen in Section 4.3.1, ∆ 2 is a key quantity in kernel change-point detection. Thus it would be very interesting to know whether the median heuristic picks a bandwidth that achieves these goals.

We first define where Med is the empirical median. That is (i) order the ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 in increasing order, (ii) output the central element if 𝑛(𝑛 -1)/2 is odd (𝑛 ≡ 2, 3 mod 4) and the mean of the two most central elements if 𝑛(𝑛 -1)/2 is even (𝑛 ≡ 0 or 1 mod 4).

𝐻 𝑛 = Med { ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 | 1 ≤ 𝑖 < 𝑗 ≤ 𝑛 } , ( 5
We call median heuristic the choice 𝜈 = √ 𝐻 𝑛 . Note that some authors choose 𝜈 = √ 𝐻 𝑛 /2. In order to investigate the asymptotic properties of 𝐻 𝑛 , rather than using (5.2), we are going to define 𝐻 𝑛 via the empirical cumulative distribution function of the ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 . Namely, for any 𝑡 ∈ R, we let

 𝐹 𝑛 (𝑡) = 2 𝑛(𝑛 -1) ∑ 1≤𝑖<𝑗≤𝑛 1 ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 ≤𝑡 (5.3)
For any 𝑝 ∈ (0, 1), we define the generalized inverse of ̂ 𝐹 𝑛 by

 𝐹 -1 𝑛 (𝑝) = inf { 𝑡 ∈ R | ̂ 𝐹 (𝑡) ≥ 𝑝 } .
We choose to define 𝐻 𝑛 as

𝐻 𝑛 = ̂ 𝐹 -1 𝑛 ( 1 2
) .

(5.4)

Notice that definitions (5.2) and (5.4) differ whenever 𝑛 ≡ 0 or 1 (mod 4).

Remark 5.2. It is tempting to define other empirical quantiles as ̂ 𝐹 -1 𝑛 (𝑝) for 𝑝 ∈ (0, 1), and indeed such quantiles are used in practice, often for 𝑝 = 0.1 and 𝑝 = 0.9. Though we are mainly concerned with 𝐻 𝑛 , we will see that our main result still holds for

𝜃 = 𝛼 2 E [ℎ(𝑋, 𝑋 ′ )] + 2𝛼(1 -𝛼) E [ℎ(𝑋, 𝑌 )] + (1 -𝛼) 2 E [ℎ(𝑌, 𝑌 ′ )] . Then √ 𝑛(𝑈 𝑛 -𝜃) ℒ -→ 𝒩 ( 0, 𝜎 2 ) , (5.6)
where 𝜎 = 𝜎(ℎ, 𝑃, 𝑄) is defined in Eq. (5.9).

We make the following remarks.

Central limit theorems for 𝑈 -statistics are known since the fundamental article of [START_REF] Hoeffding | A class of statistics with asymptotically normal distribution[END_REF]. Prop. 5.1 is in the line of such results. An asymptotic normality result also exists in the non-identically distributed case, see Hoeffding [1948, Th. 8.1]. However, this result was not applicable in our triangular array setting. The material in [START_REF] Jammalamadaka | Limit theorems for a triangular scheme of U-statistics with applications to inter-point distances[END_REF] covers the case of a triangular array scheme, but does not cover the non-identically distributed setting. Results regarding two-sample 𝑈 -statistics are closest in spirit but not directly applicable, see van der Vaart [1998, Sec. 12.2] for an introduction and [START_REF] Dehling | Asymptotic distribution of two-sample empirical U-quantiles with applications to robust tests for shifts in location[END_REF] for recent developments. With our notations, the two-sample statistic is written

1 𝛼𝑛(1 -𝛼)𝑛 𝛼𝑛 ∑ 𝑖=1 𝑛 ∑ 𝑗=𝛼𝑛+1 ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) .
The sole difference is the absence of "intra-segment" interactions: the previous display does not contain terms in ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) with 𝑖 and 𝑗 in the same segment. It is the appearance of these terms in our case which complicates the analysis.

Finally, suppose that ℎ is degenerate, that is, E ℎ(𝑋, 𝑦) = E ℎ(𝑥, 𝑌 ) = 0. Then the variance term in Eq. (5.6) is zero, and Prop. 5.1 remains true in the following sense:

√ 𝑛(𝑈 𝑛 -𝜃) converges towards the constant 0, which is a degenerate Gaussian distribution 𝒩 (0, 0). In this case, we believe that the convergence will be faster, but not toward a Gaussian distribution, c.f. Lee [1990, Section 3.2.2] for results in this direction. The idea of the proof is the following: (i) split 𝑈 𝑛 in three terms depending on the relative position of the indices; (ii) write down the Hoeffding decomposition of each of these terms; (iii) show that the remainders are negligible, and (iv) conclude thanks to the central limit theorem for triangular arrays.

We begin by decomposing 𝑈 𝑛 . To this extent, define

𝐴 𝑛 = ( 𝛼𝑛 2 ) -1 ∑ 1≤𝑖<𝑗≤𝛼𝑛 ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) , 𝐵 𝑛 = ( (1 -𝛼)𝑛 2 ) -1 ∑ 𝛼𝑛<𝑖<𝑗≤𝑛 ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) ,
and

𝐶 𝑛 = 1 𝛼𝑛(1 -𝛼)𝑛 ∑ 1≤𝑖≤𝛼𝑛 𝛼𝑛<𝑗≤𝑛 ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) .
Note that in 𝐶 𝑛 there are no terms ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) with 1 ≤ 𝑗 ≤ 𝛼𝑛 and 𝛼𝑛 < 𝑖 ≤ 𝑛, since the sum in 𝑈 𝑛 is prescribed to 𝑖 < 𝑗. Simple algebra shows that

𝑈 𝑛 = 𝛼 2 𝐴 𝑛 + (1 -𝛼) 2 𝐵 𝑛 + 2𝛼(1 -𝛼)𝐶 𝑛 (5.7) - 𝛼(1 -𝛼) 𝑛 -1 𝐴 𝑛 - 𝛼(1 -𝛼) 𝑛 -1 𝐵 𝑛 - 2𝛼(1 -𝛼) 𝑛 -1 𝐶 𝑛 .
According to Lemma 5.1 -see Section 5.6-, the variance of 𝐴 𝑛 , 𝐵 𝑛 and 𝐶 𝑛 is O (1/𝑛). Hence the second line of Eq. ( 5.7) converges in probability to 0 with speed at least √ 𝑛. Therefore we can focus on the first line of (5.7). The next step is to obtain the 𝐻-decomposition [Lee, 1990, Sec. 1.6] of 𝐴 𝑛 and 𝐵 𝑛 . Let us detail this process for 𝐴 𝑛 . We set 𝜃 𝐴 = E [ℎ(𝑋, 𝑋 ′ )], ℎ 𝐴 (𝑥) = E [ℎ(𝑥, 𝑋 ′ )] -𝜃 𝐴 and 𝑔 𝐴 (𝑥, 𝑦) = ℎ(𝑥, 𝑦)ℎ 𝐴 (𝑥)ℎ 𝐴 (𝑦) -𝜃 𝐴 . Then it is possible to write [Lee, 1990, Th. 1] 𝐴 𝑛 = 𝜃 𝐴 + 𝐿 𝐴 + 𝑅 𝐴 ,

where

𝐿 𝐴 = 2 𝛼𝑛 ∑ 1≤𝑖≤𝛼𝑛 ℎ 𝐴 (𝑋 𝑛,𝑖 ) , and
𝑅 𝐴 = ( 𝛼𝑛 2 ) -1 ∑ 1≤𝑖<𝑗≤𝛼𝑛 𝑔 𝐴 (𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) .
A totally analogous statement holds for 𝐵 𝑛 . We decompose 𝐶 𝑛 in the same fashion, the only difference is the appearance of a second term in the linear part. Namely, set

𝜃 𝐶 = E [ℎ(𝑋, 𝑌 )], ℎ 𝐶,1 (𝑥) = E [ℎ(𝑥, 𝑌 )], ℎ 𝐶,2 (𝑦) = E [ℎ(𝑋, 𝑦)] and 𝑔 𝐶 (𝑥, 𝑦) = ℎ(𝑥, 𝑦) -ℎ 𝐶,1 (𝑥) -ℎ 𝐶,2 (𝑦) -𝜃 𝐶 . Then 𝐶 𝑛 = 𝜃 𝐶 + 𝐿 𝐶 + 𝑅 𝐶 , with 𝐿 𝐶 = 1 𝛼𝑛 𝛼𝑛 ∑ 𝑖=1 ℎ 𝐶,1 (𝑋 𝑛,𝑖 ) + 1 (1 -𝛼)𝑛 𝑛 ∑ 𝑖=𝛼𝑛+1 ℎ 𝐶,2 (𝑋 𝑛,𝑖 ) , and
𝑅 𝐶 = 1 𝛼𝑛(1 -𝛼)𝑛 ∑ 1≤𝑖≤𝛼𝑛 𝛼𝑛<𝑗≤𝑛 𝑔 𝐶 (𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) .
Note that 𝜃 = 𝛼 2 𝜃 𝐴 +(1-𝛼) 2 𝜃 𝐵 +2𝛼(1-𝛼)𝜃 𝐶 . According to Lemma 5.2, the variance of 𝑅 𝐴 , 𝑅 𝐵 and 𝑅 𝐶 is of order 𝑛 -2 , thus

√ 𝑛(𝑈 𝑛 -𝜃) = √ 𝑛 [ 𝛼 2 𝐿 𝐴 + (1 -𝛼) 2 𝐿 𝐵 + 2𝛼(1 -𝛼)𝐿 𝐶 ] + 𝑟 𝑛 , (5.8)
with 𝑟 𝑛 P -→ 0. We now regroup the terms in (5.8) that belong to the same segment. For any 1 ≤ 𝑖 ≤ 𝛼𝑛, define 𝑍

𝑛,𝑖 = 𝛼ℎ 𝐴 (𝑋 𝑛,𝑖 ) + (1 -𝛼)ℎ 𝐶,1 (𝑋 𝑛,𝑖 ) .

Since ℎ ∈ 𝐿 2 (𝑃 ) ∩ 𝐿 2 (𝑄) × 𝐿 2 (𝑃 ) ∩ 𝐿 2 (𝑄) and the 𝑍 (1)
𝑛,𝑖 are identically distributed,

Var ( 𝑍 (1) 𝑛,𝑖 ) = Var (𝛼ℎ 𝐴 (𝑋) + (1 -𝛼)ℎ 𝐶,1 (𝑋))
is finite and does not depend on 𝑖. Let us put 𝜎 2 1 := Var

( 𝑍

(1) 𝑛,𝑖

) . The Lindeberg condition is satisfied, hence we can apply the central limit theorem for triangular arrays of independent random variables Billingsley [2012, Th. 27.2]. Thus

1 √ 𝛼𝑛 𝛼𝑛 ∑ 𝑖=1 𝑍 (1) 𝑛,𝑖 ℒ -→ 𝒩 ( 0, 𝜎 2 1
) .

In a similar fashion, set 𝑍

𝑛,𝑖 := 𝛼ℎ 𝐶,2 (𝑋 𝑛,𝑖 ) + (1 -𝛼)ℎ 𝐵 (𝑋 𝑛,𝑖 ) and 𝜎 2 2 := Var

( 𝑍 (2) 𝑛,𝑖 ) . Then 1 √ (1 -𝛼)𝑛 𝑛 ∑ 𝑖=𝛼𝑛+1 𝑍 (2) 𝑛,𝑖 ℒ -→ 𝒩 ( 0, 𝜎 2
2 ) .

The two previous sums are independent, thus by Lévy's theorem

√ 𝑛 [ 𝛼 2 𝐿 𝐴 + (1 -𝛼) 2 𝐿 𝐵 + 2𝛼(1 -𝛼)𝐿 𝐶 ] ℒ -→ 𝒩 ( 0, 𝜎 2 ) , with 𝜎 2 = 𝜎 2 1 + 𝜎 2 2 .
(5.9)

Since the remainder term converges in probability to 0, we can conclude via Slutsky's Lemma.

Asymptotic normality of 𝐻 𝑛

We now turn to the statement of our main result. In the previous section, we only obtained the convergence of the empirical distribution function. It is well-known that such a result implies the convergence of the empirical quantiles towards the and rewrite this event as

{ √ 𝑛 ( ̂ 𝐹 𝑛 ( 𝑚 + 𝑡 √ 𝑛 ) -𝐹 ( 𝑚 + 𝑡 √ 𝑛 )) ≥ √ 𝑛 ( 𝑝 -𝐹 ( 𝑚 + 𝑡 √ 𝑛 ))} . (5.11)
Since 𝐹 is differentiable in 𝑚, the right-hand side of (5.11) converges towards -𝑡𝐹 ′ (𝑚) by Taylor expansion. From Prop. 5.1, it is also true that

√ 𝑛 ( ̂ 𝐹 𝑛 (𝑚) -𝐹 (𝑚) ) ℒ -→ 𝒩 ( 0, 𝜎 2 ) .
Therefore, if we manage to prove that

√ 𝑛 [( ̂ 𝐹 𝑛 ( 𝑚 + 𝑡 √ 𝑛 ) -̂ 𝐹 𝑛 (𝑚) ) - ( 𝐹 ( 𝑚 + 𝑡 √ 𝑛 ) -𝐹 (𝑚)
)] P -→ 0 , (5.12)

Eq. ( 5.10) will follow by Slutsky's Lemma.

Define ℎ(𝑥, 𝑦) = 1 𝑚<‖𝑥-𝑦‖ 2 ≤𝑚+ 𝑡 √ 𝑛

. Then, with the notations used in the proof of Prop. 5.1, Eq. ( 5.12) reads √ 𝑛(𝑈 𝑛 -𝜃) P -→ 0 .

We dispose of the remainder terms as in the proof of Prop. 5.1, thus we are left to show that

√ 𝑛 [ 𝛼 2 (𝐴 𝑛 -𝜃 𝐴 ) + (1 -𝛼) 2 (𝐵 𝑛 -𝜃 𝐵 ) + 2𝛼(1 -𝛼)(𝐶 𝑛 -𝜃 𝐶 ) ] P -→ 0 . (5.13)
Let us focus on the first term of the previous display, which can be written

𝛼 2 √ 𝑛 (𝐴 𝑛 -𝜃 𝐴 ) = 𝛼 2 √ 𝑛 (𝐿 𝐴 + 𝑅 𝐴 ) .
Once again, we use Lemma 5.2 to get rid of 𝑅 𝐴 . The linear term is slightly more tedious to analyze. Recall that E [ℎ 𝐴 (𝑋)] = 0. Thanks to Jensen's inequality,

Var (ℎ 𝐴 (𝑋)) = E 𝑋 [( E 𝑋 ′ [ 1 𝑚<‖𝑋-𝑋 ′ ‖ 2 ≤𝑚+ 𝑡 √ 𝑛 ] 2 )] ≤ P ( 𝑚 < ‖𝑋 -𝑋 ′ ‖ 2 ≤ 𝑚 + 𝑡 √ 𝑛 ) .
We recognize

𝐹 ( 𝑚 + 𝑡 √ 𝑛 ) -𝐹 (𝑚) ,
which goes to 0 when 𝑛 → ∞, since we assumed 𝐹 to have a derivative in 𝑚. Furthermore, by independence of the 𝑋 𝑛,𝑖 ,

Var (√ 𝑛𝐿 𝐴 ) = 4 Var (ℎ 𝐴 (𝑋)) -→ 𝑛→∞ 0 .
A similar reasoning applies to the other terms in Eq. ( 5.13), and the proof is concluded. For each 𝑑, we repeated the experiment 10 2 times. The shaded area is within standard deviation from these experiments. We plot in red the approximation given by Eq. (5.14).

An example

In this section, we investigate the consequences of Prop. 5.2 when 𝑃 and 𝑄 are known to be multivariate Gaussian distributions.

Before, let us recall that a sum of 𝑑 independent squared standard Gaussian random variables is said to follow the chi-squared distribution with 𝑑 degrees of freedom, denoted by 𝜒 2 𝑑 . We also define the non-central chi-squared central distribution with the law of ∑ 𝑑 𝑖=1 𝒩 (𝜇 𝑖 , 1) 2 , with non-centrality parameter 𝜆 = ∑ 𝑑 𝑖=1 𝜇 2 𝑖 . Now suppose that 𝑋 ∼ 𝒩 (0, I 𝑑 ) and 𝑌 ∼ 𝒩 (𝜇, I 𝑑 ) with 𝜇 ∈ R 𝑑 . This is the situation depicted in Fig. 5-1. We choose to tackle the unit variance case for convenience, as everything that follows can be adapted for a covariance matrix Σ = 𝛾 I 𝑑 . With the notations introduced in Section 5.3, a quick computation shows that

⎧ ⎪ ⎨ ⎪ ⎩ 𝑇 𝑋𝑋 = 2𝜒 2 𝑑 , 𝑇 𝑋𝑌 = 2𝜒 2 𝑑 (𝜆), with 𝜆 = 1 2 ‖𝜇‖ 2 , 𝑇 𝑌 𝑌 = 2𝜒 2 𝑑 .
According to Prop. 5.2, 𝐻 𝑛 is close to 𝑚 = med(𝑇 ) for large 𝑛. Can we hope for some insights on the value of 𝑚? The following proposition shows that it is possible in the high-dimensional regime.

Proposition 5.3. Let 𝑇 be as before. Take 𝛼 ∈ (0, 1/2). Suppose that ‖𝜇‖ 2 = O (𝑑) when 𝑑 goes to infinity. Define

𝜅 𝛼 = 2 √ 2Φ -1 ( 1 2 (𝛼 2 + (1 -𝛼) 2 ) ) > 0 ,
where Φ is the repartition function of the standard Gaussian random variable. Then

𝑚 = 2𝑑 + 𝜅 𝛼 √ 𝑑 + o ( √ 𝑑 ) , ( 5

Conclusion and future directions

In this chapter, we partly explained the behavior of the median heuristic for a large sample size. We believe that it opens the door to more rigorous statements regarding the optimality of bandwidth choice in kernel two-sample test and kernel change-point detection, at least for some specific distributions.

As a future direction for research, we believe that it would be interesting to obtain a non-asymptotic version of Prop. 5.2. Indeed, as it is often the case in kernel methods, both kernel two-sample test and kernel change-point detection run in quadratic time -even though linear time approximations are available. Hence these methods, and consequently the median heuristic, are frequently used with a sample size that does not exceed a few hundreds.

We would also like to improve the results given in Prop. 5.3. Namely, a version of Prop. 5.3 with non-identity covariance matrices for 𝑃 and 𝑄 seems out of reach at the moment. Indeed, obtaining asymptotic behavior for 𝑇 𝑋𝑋 , 𝑇 𝑌 𝑌 and 𝑇 𝑋𝑌 is much harder. Extending Prop. 5.3 to the case where 𝑃 and 𝑄 are mixtures of Gaussian distributions with non-identity covariance matrices could yield some precious insights on situations where the median heuristic is known to fail empirically [Gretton et al., 2012b, Fig. 1].

Additional proofs

In this section, we state and prove the technical results that are needed in the proofs of this chapter. Recall that we denote by |𝐴| the cardinality of any finite set 𝐴. The proof is standard in 𝑈 -statistics [START_REF] Lee | U-statistics: Theory and Practice[END_REF]. where 𝑎, 𝑏, 𝑐 and 𝑑 range from 1 to 𝑚. There are three separate cases in the sum that we detail below.

-The indices 𝑎, 𝑏, 𝑐 and 𝑑 are all distincts. There are ( 𝑚 Since 𝑚 = 𝛼𝑛, O (𝑚 -1 ) = O (𝑛 -1 ) and we can conclude for 𝐴 𝑛 :

Var (𝐴 𝑛 ) = E [ 𝐴 2 𝑛 ] -E [𝐴 𝑛 ] 2 = O ( 𝑛 -1 ) .
The same proof transfers readily for 𝐵 𝑛 and 𝐶 𝑛 . where ⋆ denotes that we sum on indices such that |{𝑎, 𝑏} ∩ {𝑐, 𝑑}| ≥ 2. As we have seen in the proof of Lemma 5.1, there are only O (𝑚 2 ) such possibilities, and we can conclude.

Our last result is a central limit theorem for a non-central chi-squared distributed random variable. )

(1 -2i𝑡) Chapter 6

Conclusion and future work

Summary of the thesis

In this thesis, we mostly focused on a method for detecting abrupt changes in a sequence of independent observations belonging to an arbitrary set 𝒳 on which a positive semi-definite kernel 𝑘 is defined. That method, kernel change-point detection, is a kernelized version of a penalized least-squares procedure.

Our main contribution is to show that, for any kernel satisfying some reasonably mild hypotheses, the KCP procedure outputs a segmentation close to the true segmentation with high probability. This result is obtained under a bounded assumption on the kernel, in Theorem 3.1 for a linear penalty, and Theorem 3.2 for another penalty function, coming from model selection.

The proofs rely on a concentration result for bounded random variables in Hilbert spaces, and we prove a less powerful result under relaxed hypotheses -a finite variance assumption -in Theorem 3.3.

Up to now, it seemed difficult to combine the "change-point estimation / asymptotic" with the "model selection / non-asymptotic" point of view. The proofs of Theorems 3.1 and 3.3 show how they can be reconciled. Moreover, the structure of these proofs is modular, so that one can easily adapt them to different sets of assumptions.

In the asymptotic setting, we show that we recover the minimax rate log(𝑛)/𝑛 for the change-point locations without additional hypothesis on the segment sizes. We provide empirical evidence supporting these claims.

Another contribution of this thesis is the detailed presentation of the different notions of distances between segmentations. Additionally, we prove a result showing these different notions coincide for sufficiently close segmentations.

From a practical point of view, a contribution of this thesis is to demonstrate how the so-called dimension jump heuristic can be a reasonable choice of penalty constant when using kernel change-point detection with a linear penalty.

We also show how a key quantity depending on the kernel that appears in our theoretical results, ∆, influences the performance of KCP in the case of a single change-point. When the kernel is translation-invariant and parametric assumptions are made, it is possible to compute ∆ in closed-form. Thanks to these computations, some of them novel, we are able to study precisely the link between the maximal penalty constant and ∆. We show that, as suggested by Theorem 3.1, our theoretical upper bound on the penalty constant is proportional to ∆ 2 all other things being equal.

Our last main contribution is a study of the median heuristic, a popular tool to set the bandwidth of radial basis function kernels. For a large sample size, we show that the median heuristic behaves approximately as the median of a distribution that we describe completely in the setting of kernel two-sample test and kernel changepoint detection. More precisely, we show that the median heuristic is asymptotically normal around this value.

Perspectives

A number of questions remain unanswered at the end of this manuscript.

-In our theoretical study, we essentially provide an upper bound for 𝐶 max , that is, 𝐶 max ∆ 2 with high probability, up to constants that do not depend on the kernel. We believe that a lower bound exists. Proving this lower bound would achieve two goals. First, it would explain our empirical findings, and second, it would provide an important argument in favor of keeping up the study of ∆ as a criterion for kernel choice.

-In all our main results, we impose that ̂ 𝐷 = 𝐷 ⋆ . This may be a little too demanding, and as such it translates into very tight bounds on the possible constants for the penalty functions. Maybe it would be preferable not to work on an event such that ̂ 𝐷 = 𝐷 ⋆ , but rather such that ⃒ ⃒ ⃒ ̂ 𝐷 -𝐷 ⋆ ⃒ ⃒ ⃒ ≤ 𝜂, with 𝜂 a threshold. As another technical improvement, we believe that it is possible to take advantage of the structure of our proofs to account for dependency between observations -a very natural hypothesis when dealing with time series. The only missing piece is a concentration inequality for dependent Hilbert-valued random variables.

-In the main concentration result used in our proofs, the deviations are of order 𝑀 2 , which yields the ∆ 2 /𝑀 2 term that intervenes in both the expression of 𝐶 max and the speed of convergence. We believe that this result is not optimal. In the real case, for instance, we know for a fact that 𝑀 2 can be replaced by a variance term. To prove a concentration result where the kernel bound is replaced by a variance term would provide a much more general version of Theorem 3.1. In particular, this new version would hold in the archetypal change-point problem -linear kernel, change in the mean of Gaussian observations -, which is not covered by the present result.

-The penalty function pen L showed its superiority for detecting the changes in R [START_REF] Lebarbier | Detecting multiple change-points in the mean of a gaussian process by model selection[END_REF] and is a natural choice of penalty function for KCP as well.

So far, we have not managed to calibrate properly the constants of pen L . In particular, in the real case, both these constants are of order 𝜎 2 -the variance of the data -for which we have no natural substitute in the kernel setting. Is it possible to provide a method for the calibration of the constants in pen L in the setting of KCP?

-The link uncovered between ∆ 2 and the maximal penalty constant suggests a simple method of kernel choice for KCP: maximizing ∆ 2 with respect to the kernel. Unfortunately, even in the simplest case, ∆ 2 depends on unknown quantities, in particular the size of the jump we are trying to detect. Is it possible to estimate ∆ 2 accurately, and thus to build a data-driven method for choosing the kernel for KCP?

-The results we obtained regarding the asymptotic behavior of the median heuristic are more precise than what was already known. It would be interesting to use this new insight in simple situations. For instance, suppose that we want to detect a single change in the mean of Gaussian observations. In this situation, is the median heuristic close to the global maximum of ∆ 2 ?

-Finally, we think that the KCP algorithm can be applied successfully to real data in cases where 𝑑-dimensional features are not easily accessible but a positive semi-definite kernel is available. An example of such a situation is the study of granular material, for instance cereal grains in a silo. It is possible to summarize such an environment to the weighted graph of forces between particles, and even further to the persistence diagram associated to this graph -see [START_REF] Oudot | Persistence theory: from quiver representations to data analysis[END_REF] for an introduction to persistence. The time-evolution of the granular material is then described by a sequence of such diagrams. The study of abrupt changes in the distribution of the forces between particles, as for instance in [START_REF] Gutiérrez | Silo collapse under granular discharge[END_REF], needs a change-point detection method for persistence diagrams. They are complicated objects, but positive semi-definite kernels do exist on the space of persistence diagrams [START_REF] Kusano | Persistence weighted gaussian kernel for topological data analysis[END_REF]. Pour finir, nous traitons de l'heuristique de la médiane, un moyen courant de choisir la largeur de bande des noyaux à base de fonctions radiales. Dans un cadre asymptotique, nous montrons que l'heuristique de la médiane se comporte à la limite comme la médiane d'une distribution que nous décrivons complètement dans le cadre du test à deux échantillons à noyaux et de la détection de ruptures. Plus précisément, nous montrons que l'heuristique de la médiane est approximativement normale centrée en cette valeur.

Résumé

Mots Clés

Détection de ruptures, méthodes à noyaux, moindres carrés pénalisés, heuristique de la médiane.

Abstract

In this thesis, we focus on a method for detecting abrupt changes in a sequence of independent observations belonging to an arbitrary set on which a positive semidefinite kernel is defined. That method, kernel changepoint detection, is a kernelized version of a penalized least-squares procedure.

Our main contribution is to show that, for any kernel satisfying some reasonably mild hypotheses, this procedure outputs a segmentation close to the true segmentation with high probability. This result is obtained under a bounded assumption on the kernel for a linear penalty and for another penalty function, coming from model selection.

The proofs rely on a concentration result for bounded random variables in Hilbert spaces and we prove a less powerful result under relaxed hypotheses -a finite variance assumption.

In the asymptotic setting, we show that we recover the minimax rate for the change-point locations without additional hypothesis on the segment sizes. We provide empirical evidence supporting these claims. Another contribution of this thesis is the detailed presentation of the different notions of distances between segmentations. Additionally, we prove a result showing these different notions coincide for sufficiently close segmentations.

From a practical point of view, we demonstrate how the so-called dimension jump heuristic can be a reasonable choice of penalty constant when using kernel changepoint detection with a linear penalty.

We also show how a key quantity depending on the kernel that appears in our theoretical results influences the performance of kernel change-point detection in the case of a single change-point. When the kernel is translationinvariant and parametric assumptions are made, it is possible to compute this quantity in closed-form. Thanks to these computations, some of them novel, we are able to study precisely the behavior of the maximal penalty constant.

Finally, we study the median heuristic, a popular tool to set the bandwidth of radial basis function kernels. For a large sample size, we show that it behaves approximately as the median of a distribution that we describe completely in the setting of kernel two-sample test and kernel change-point detection. More precisely, we show that the median heuristic is asymptotically normal around this value.

  ∞ . . . . . . . .  ℛ . . . . . . . . . . . . . . . . . . . . . . . 𝑘 . . . . . . . . . . . . . . . . . . . . . . . K . . . . . . . . . . . . . . . . . . . . . . . Λ, Λ . . . . . . . . . . . . . . . . . . . . 𝐿 𝜏 . . . . . . . . . . . . . . . . . . . . . . 𝑀 . . . . . . . . . . . . . . . . . . . . . . 𝜇 ⋆ . . . . . . . . . . . . . . . . . . . . . .  𝜇 . . . . . . . . . . . . . . . . . . . . . . . 𝜈 . . . . . . . . . . . . . . . . . . . . . . . linear part in the decomposition of 𝜓 𝜏 penalty constant minimal and maximal theoretical penalty constant a model selection criterion smallest and largest jump size in the RKHS number of segments of 𝜏 number of segments of 𝜏 ⋆ number of segments of ̂ 𝜏 generic distance Frobenius distance between segmentations Hausdorff distance between segmentations custom distances, defined in Section 3.2.2 empirical risk, defined by Eq. (2.1) a positive semi-definite kernel the Gram matrix, defined by Eq. (1.10) minimum and maximum segment size linear part in the decomposition of 𝜓 𝜏 kernel bound in Assumption 2.1 true regression function, element of ℋ 𝑛 estimated regression function, element of ℋ 𝑛 (positive) bandwidth pen . . . . . . . . . . . . . . . . . . . . . pen ℓ . . . . . . . . . . . . . . . . . . . . pen L . . . . . . . . . . . . . . . . . . . . 𝜓 𝜏 . . . . . . . . . . . . . . . . . . . . . . 𝑄 𝜏 . . . . . . . . . . . . . . . . . . . . . . ℛ . . . . . . . . . . . . . . . . . . . . . . . 𝒯 𝑛 . . . . . . . . . . . . . . . . . . . . . . 𝐻 . . . . . . . . . . . . . . . . . . . . . . . 𝜏 . . . . . . . . . . . . . . . . . . . . . . .  𝜏 . . . . . . . . . . . . . . . . . . . . . . . 𝜏 ⋆ . . . . . . . . . . . . . . . . . . . . . . . 𝑉 . . . . . . . . . . . . . . . . . . . . . . . 𝑣 1 . . . . . . . . . . . . . . . . . . . . . . . 𝑣 ′ 1 . . . . . . . . . . . . . . . . . . . . . . . 𝑣 2 . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 - 1 -

 11 Figure1-1 -Here we observe 10 3 observations of a Gaussian distribution (marked as black dots) whose mean abruptly changes from 1 to -1 and then to 0.5 at position 500 and 750 (indicated by broken vertical red lines) and whose variance stays constant (𝜎 = 0.5). The goal of change-point detection is to recover these positions from the observations. We are here in an off-line setting, with multiple change-points, in a parametric model.

Figure 1 - 3 -

 13 Figure 1-3 -Log-returns of the Standard and Poors 500 Price index, closing prices, from June 1, 1995 to June 1, 2017. The S&P500 is an index reflecting the market capitalization of the 500 largest listed companies in the USA. As one can see, the 𝑋 𝑡 s have a fairly stationary behavior between structural changes that often corresponds with major financial crises. The most visible in this graph is the 2008 financial crisis.

Figure 1 - 4 -

 14 Figure 1-4 -An excerpt of the movie The Big Lebowski. Here we can see 800 frames -corresponding roughly to 30s -of the movie, and for each frame the corresponding color histogram (27 colors) in the bottom panel. In this scene, the Dude is talking to his landlord Marty while drinking a White Russian. The camera alternates between shots of the Dude on his doorway (upper right panel) and Marty in front of him (upper left panel). Abrupt changes in the color histogram of the frames match this alternation.

Figure 1 - 5 -

 15 Figure 1-5 -Three samples 𝑋 1 , . . . , 𝑋 𝑛 with several values of 𝑛. The distribution of the 𝑋 𝑖 is Gaussian with mean 1 (resp. -1, 0.5) on the first (resp. second, third) segment and standard deviation 𝜎 = 0.5. The increasing sample size leads to more observations on the segments, whose normalized sizes converge to constant numbers. Left panel : 𝑛 = 50, Middle panel : 𝑛 = 500, Right panel : 𝑛 = 5000.

Figure 1 -

 1 Figure 1-6 -In this figure, we plot the values of 𝑘 (𝑥, 𝑦) for different positive semidefinite kernels and 𝑥, 𝑦 ∈ R. Upper left: linear kernel; Upper right: polynomial kernel with 𝛼 = 2 and 𝑐 = 1; Bottom left: Gaussian kernel with 𝜈 = 1.0; Bottom right: Laplace kernel with 𝜈 = 1.0.

Figure 2 - 1 -

 21 Figure 2-1 -A typical graphical representation of a segmentation. The black solid disks stand for the ordered elements of { 1, . . . , 𝑛 } , and the vertical lines denote the changes. Here, as one can read, 𝑛 = 10, 𝐷 𝜏 = 3, 𝜏 0 = 0, 𝜏 1 = 3, 𝜏 2 = 7 and 𝜏 3 = 10 -thus 𝜏 = [0, 3, 7, 10].

Figure 2

 2 Figure 2-2 -Illustration of the asymptotic setting (Example 2.1) in the case of changes in the mean of the 𝑋 𝑖 . Here, 𝒳 = R, 𝑋 𝑖 = 𝑓 (𝑡 𝑖 ) + 𝜀 𝑖 with 𝜀 1 , . . . , 𝜀 𝑛 i.i.d. and centered, and 𝑓 : [0, 1] → R is a (fixed) piecewise constant function (shown in red).The goal is to recover the number of abrupt changes of 𝑓 (here, 2) and their locations (𝑏 1 = 0.5 and 𝑏 2 = 0.7). Note that other kinds of changes in the distribution of the 𝑋 𝑖 can be considered, see Section 4.2.

Figure 4 - 1 -

 41 Figure 4-1 -Two plots of the estimated dimension as a function of the penalty constant 𝑐. In both experiments, we considered 𝑛 = 50 data-points with a single jump in the mean of size 10, with unit variance Gaussian noise. Left panel : Gaussian kernel with bandwidth 𝜈 = 0.01. Right panel : Gaussian kernel with bandwidth 𝜈 = 0.1. The black dotted line denotes the maximal jump, located at 𝑐 = 0.93 in the left panel (resp. 𝑐 = 0.72 in the right panel); the red dashed line marks the penalty chosen by the dimension jump heuristic. We recover the true dimension 𝐷 ⋆ = 2 in the left panel, but not in the right panel, where ̂ 𝐷 = 4.

Figure 4 -

 4 Figure 4-2 -Minimum and maximum penalty constants leading to exact recovery of the true dimension as a function of the sample size. The solid lines inside the shaded areas are the empirical mean over 100 repetitions of the experiments; the error bars correspond to the standard deviation. Null values were removed (3.5% of the experiments).

3 Figure 4 - 3 -

 343 Figure 4-3 -In thick red lines, the three piecewise constant functions used in the simulations of Section 4.2. In lighter blue, a noisy version of these functions. Both 𝜇 1 and 𝜇 2 have 5 segments; 𝜇 3 has 10 jumps.

3 Figure 4 - 4 -

 344 Figure 4-4 -Convergence of 1 𝑛 d

Figure 4 - 5 -

 45 Figure 4-5 -Left: One sample 𝑋 1 , . . . , 𝑋 𝑛 for 𝑛 = 10 3 . Middle: Performance of KCP with various kernels (𝑛 = 200). Methods 1 to 8: Gaussian kernel with bandwidth set via the median heuristic (method 1), or fixed equal to 0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1 (methods 2, . . . , 9, respectively). Method 9: linear kernel. Right: Estimated values of 𝑛 -1 d

Definition 4 . 1 .

 41 A translation-invariant kernel 𝑘 : R 𝑑 × R 𝑑 → R is a positive definite kernel such that there exists a function 𝜅 : R 𝑑 → R that satisfies 𝑘 (𝑥, 𝑦) = 𝜅(𝑥 -𝑦) .

Proposition 4 . 3 (

 43 Gaussian kernel, change in the variance). Set 𝑘 = 𝑘 G . Suppose that Assumption 4.1 holds true with 𝐿 1 = 𝐿 2 , 𝛽 1 ∼ 𝒩 (0, 𝜎 2 1 ), and 𝛽 2 ∼ 𝒩 (0, 𝜎 2 2 ), where 𝜎 1 and 𝜎 2 are positive numbers. Then

Lemma 4 . 1 .

 41 Let 𝑎, 𝑏 and 𝜆 be positive numbers. Define 𝑓 (𝜆) := ∫ e -𝑎𝜔 2 cos(𝜆𝜔) 1 + 𝑏𝜔 2 d 𝜔 .

Figure 4 - 6 -

 46 Figure 4-6 -In this figure, we plot 𝑛 • ∆ 2 /𝑀 2 (in black) and 𝑐 max (in blue) as a function of 𝜈 in different scenarios. The error bars for 𝑐 max come from 100 repetitions of the experiment, the vertical red line is the average median heuristic bandwidth (standard deviation was very small and is not represented). Upper left: Gaussian kernel, change in the mean; Upper right: Gaussian kernel, change in the variance; Bottom left: Laplace kernel, change in the mean; Bottom right: Laplace kernel, change in the variance.

Figure 5 - 1 -

 51 Figure 5-1 -Histogram of the ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 for Gaussian distributions in dimension 𝑑 = 100. Namely, 𝑋 ∼ 𝒩 (0, I 𝑑 ), 𝑌 ∼ 𝒩 (10𝑑 1, I 𝑑 ), 𝛼 = .25 and 𝑛 = 400. We repeated the experiment 10 3 times, counting each time how many observations were into the bins, error bars are standard deviations over these repetitions; the vertical black line is the average sample median.

  that the 𝑋 𝑛,𝑖 satisfy Assumption 5.1. Define 𝑈 𝑛 = 2 𝑛(𝑛 -1) ∑ 1≤𝑖<𝑗≤𝑛 ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) , and set

Remark 5 . 3 .

 53 It is possible to prove a version of Prop. 5.1 for the multiple changepoint setting introduced in Rem. 5.1. The proof follows the lines of Sec. 5.3.2, with an additional technical difficulty due to the numerous inter-segment interactions -we only deal with one in the present work.

Figure 5 - 2 -

 52 Figure 5-2 -Empirical distribution of 𝐻 400 for different distributions 𝑃 and 𝑄 over 10 3 repetitions. Upper left: 𝑃 ∼ 𝒩 (0, 1) and 𝑄 ∼ 𝒩 (10, 1); Upper right: 𝑃 ∼ 𝒩 (0, I 100 ) and 𝑄 ∼ 𝒩 (10 1 100 , I 100 ); Bottom left: 𝑃 ∼ Γ(2, 2) and 𝑄 ∼ ℰ(1); Bottom right: both 𝑃 and 𝑄 are Gaussian mixtures 𝒩 (0, 1) and 𝒩 (5, 1) with equal weight. The results of Kolmogorov-Smirnov tests were non-significant, indicating that 𝐻 400 is normally distributed in each case.

Figure 5 - 3 -

 53 Figure 5-3 -A plot of 𝐻 𝑛 (𝑑) for 𝑋 ∼ 𝒩 (0, I 𝑑 ) and 𝑌 ∼ 𝒩 (10𝑑 1, I 𝑑 ) for 𝑛 ∈ {50, 100, 200}.For each 𝑑, we repeated the experiment 10 2 times. The shaded area is within standard deviation from these experiments. We plot in red the approximation given by Eq.(5.14).

Lemma 5 . 1 .

 51 Let 𝐴 𝑛 , 𝐵 𝑛 and 𝐶 𝑛 be defined as in the proof of Prop. 5.1. Then Var (𝐴 𝑛 ), Var (𝐵 𝑛 ) and Var (𝐶 𝑛 ) are O (𝑛 -1 ).

Proof.

  We set 𝑚 = 𝛼𝑛 in this proof. Recall that 𝑛,𝑖 , 𝑋 𝑛,𝑗 ) , Define ℎ 𝑖,𝑗 = ℎ(𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ), thus E ℎ 𝑖,𝑗 = 𝜃 𝐴 and E 𝐴 𝑛 = 𝜃 𝐴 . Let us turn to the computation of E [𝐴 2𝑛 ], that is, 𝑎,𝑏 ℎ 𝑐,𝑑 ] ,

4 ) 3 )E

 43 say, 𝑎 < 𝑏, and only one possibility left for 𝑐 < 𝑑.-One of the indices is common, that is, |{𝑎, 𝑏} ∩ {𝑐, 𝑑}| = 1. There are 6 ways to do so.-Both indices are equal, that is, 𝑎 = 𝑐 and 𝑏 = 𝑑. There are ( 𝑚2) = O (𝑚 2 ) ways to do so.Note that when 𝑎, 𝑏, 𝑐 and 𝑑 are all distinct,E [ℎ 𝑎,𝑏 ℎ 𝑐,𝑑 ] = E [ℎ 𝑎,𝑏 ] E [ℎ 𝑐,𝑑 ] by independence of the 𝑋 𝑛,𝑖 s. Thus [ℎ 𝑎,𝑏 ] E [ℎ 𝑐,𝑑 ] + O ( 𝑚 -1 ) ,where the summation is on distinct indices. On the other hand,E [𝐴 𝑛 ] ,𝑏 E ℎ 𝑐,𝑑 .By the same combinatorial argument, the terms corresponding to intersecting sets of indices are at most O (𝑛 3 ) and we have E [𝐴 𝑛 ] 2 = ℎ 𝑎,𝑏 E ℎ 𝑐,𝑑 + O ( 𝑚 -1 ) .

Lemma 5 . 2 .

 52 Let 𝑅 𝐴 , 𝑅 𝐵 and 𝑅 𝐶 be as in the proof of Prop. 5.2. Then Var (𝑅 𝐴 ), Var (𝑅 𝐵 ) and Var (𝑅 𝐶 ) are of order O (𝑛 -2 ). , with 𝑔 𝑖,𝑗 = 𝑔 𝐴 (𝑋 𝑛,𝑖 , 𝑋 𝑛,𝑗 ). By definition of 𝑔 𝐴 , it holds that E 𝑔 𝑖,𝑗 = 0 for any 𝑖 ̸ = 𝑗, thus E 𝑅 𝐴 = 0. Hence to control the variance of 𝑅 𝐴 , we just need to compute E [𝑅 2 𝐴 ]. As in the proof of Lemma 5𝑎,𝑏 𝑔 𝑐,𝑑 ] .Note that E [𝑔 𝑎,𝑏 𝑔 𝑐,𝑑 ] = 0 whenever 𝑎, 𝑏, 𝑐 and 𝑑 are all distinct. But a straightforward computation also shows that E [𝑔 𝑎,𝑏 𝑔 𝑎,𝑐 ] = 0 for any distinct 𝑎, 𝑏, 𝑐. 𝑎,𝑏 𝑔 𝑐,𝑑 ] ,

Lemma 5 . 3 .

 53 Let 𝑌 be a 𝜒 2 𝑑 (𝜆) distributed random variable, with 𝜆 > 0 possibly depending on 𝑑. Then 𝑌 satisfies a central limit theorem, namely𝑌 -(𝑑 + 𝜆) The characteristic function of 𝑌 is 𝑡 ↦ → 𝜙 𝑌 (𝑡)

  ℋ . . . . . . . . . . . . . . . . . . . . ‖•‖ F . . . . . . . . . . . . . . . . . . . . . arg max, arg min . . . . . . . . C . . . . . . . . . . . . . . . . . . . . . . . conv . . . . . . . . . . . . . . . . . . . . diag . . . . . . . . . . . . . . . . . . . . diam . . . . . . . . . . . . . . . . . . . . erf . . . . . . . . . . . . . . . . . . . . . . E . . . . . . . . . . . . . . . . . . . . . . . ℱ . . . . . . . . . . . . . . . . . . . . . . . ℋ . . . . . . . . . . . . . . . . . . . . . . . I 𝑑 . . . . . . . . . . . . . . . . . . . . . . . 1 •∈𝐴 . . . . . . . . . . . . . . . . . . . . Med . . . . . . . . . . . . . . . . . . . . med. . . . . . . . . . . . . . . . . . . . . o (•) , O (•) . . . . . . . . . . . . . . . N . . . . . . . . . . . . . . . . . . . . . . . N ⋆ . . . . . . . . . . . . . . . . . . . . . . P . . . . . . . . . . . . . . . . . . . . . . . R . . . . . . . . . . . . . . . . . . . . . . . ⟨•, •⟩. . . . . . . . . . . . . . . . . . . . . ⟨•, •⟩ ℋ . . . . . . . . . . . . . . . . . . . Span . . . . . . . . . . . . . . . . . . . .

		expectation of a random variable
		Fourier transform
		Hilbert space
		identity matrix of size 𝑑 × 𝑑 indicator function of the set 𝐴
		sample median
		theoretical median
		Landau notations
		non-negative integers
		positive integers
		probability of an event
		set of real numbers
		scalar product
		scalar product in ℋ span of a set of vectors exempli gratia trace of a matrix Equation transpose of the matrix 𝑥 et alii variance of a random variable et cetera Figure input space
		id est
		independent and identically distributed
	Other notations	kernel change-point detection Maximum mean discrepancy
		page
		positive semi-definite
		Proposition
		respectively
		reproducing kernel Hilbert space
		such that
	General mathematical notations
		define equals
		absolute value
		norm of a vector
		norm of a vector in ℋ Frobenius norm of a matrix
		argument of the maxima (resp. minima)
		set of complex numbers
		convex hull of a set
		diagonal of a matrix
		diameter of a set
		error function, defined by erf (𝑥) := 2

Abbreviations e.g. . . . . . . . . . . . . . . . . . . . . . Eq. . . . . . . . . . . . . . . . . . . . . . et al. . . . . . . . . . . . . . . . . . . . . etc. . . . . . . . . . . . . . . . . . . . . . Fig. . . . . . . . . . . . . . . . . . . . . . i.e.. . . . . . . . . . . . . . . . . . . . . . i.i.d. . . . . . . . . . . . . . . . . . . . . KCP . . . . . . . . . . . . . . . . . . . . MMD . . . . . . . . . . . . . . . . . . . p. . . . . . . . . . . . . . . . . . . . . . . . p.s.d. . . . . . . . . . . . . . . . . . . . Prop. . . . . . . . . . . . . . . . . . . . resp. . . . . . . . . . . . . . . . . . . . . RKHS . . . . . . . . . . . . . . . . . . s.t. . . . . . . . . . . . . . . . . . . . . . . := . . . . . . . . . . . . . . . . . . . . . . |•| . . . . . . . . . . . . . . . . . . . . . . . ‖•‖ . . . . . . . . . . . . . . . . . . . . . . ‖•‖ erfc . . . . . . . . . . . . . . . . . . . . . Tr . . . . . . . . . . . . . . . . . . . . . . 𝑥 ⊤ . . . . . . . . . . . . . . . . . . . . . . Var . . . . . . . . . . . . . . . . . . . . . 𝒳 . . . . . . . . . . . . . . . . . . . . . . . 𝐴 𝜏 . . . . . . . . . . . . . . . . . . . . . . 𝐶 . . . . . . . . . . . . . . . . . . . . . . . 𝐶 min , 𝐶 max . . . . . . . . . . . . . . crit . . . . . . . . . . . . . . . . . . . . . ∆, ∆ . . . . . . . . . . . . . . . . . . . . 𝐷 𝜏 . . . . . . . . . . . . . . . . . . . . . . 𝐷 ⋆ . . . . . . . . . . . . . . . . . . . . . .  𝐷 . . . . . . . . . . . . . . . . . . . . . . . d . . . . . . . . . . . . . . . . . . . . . . . d F . . . . . . . . . . . . . . . . . . . . . . d H . . . . . . . . . . . . . . . . . . . . . . d ∞

  10) is called the Gram matrix of 𝑘 with respect to 𝑥 1 , . . . , 𝑥 𝑚 . Any function 𝑘 that gives rise to a positive semi-definite Gram matrix for any 𝑥 1 , . . . , 𝑥 𝑚 ∈ 𝒳 is called a positive semi-definite kernel. Equivalently, we ask for 𝑘 to be a symmetric function such that, for any 𝑚 ∈ N ⋆ , 𝑥 1 , . . . , 𝑥 𝑚 ∈ 𝒳 and 𝜆 1 , . . . , 𝜆 𝑚 ∈ R,

	𝑚	𝑚
	∑ 𝑖=1	∑ 𝑗=1	𝜆 𝑖 𝜆 𝑗 𝑘 (𝑥 𝑖 , 𝑥 𝑗 ) ≥ 0 .	(1.11)

  𝐷 }. We call change-points the right-end of the segments, that is the 𝜏

ℓ , ℓ ∈ { 1, . . . , 𝐷 } . Let us denote by 𝒯 𝐷 𝑛 the set of segmentations with 𝐷 segments 1 and 𝒯 𝑛 := ⋃ 𝑛 𝐷=1 𝒯 𝐷 𝑛 the set of all segmentations of { 1, . . . , 𝑛 } . For any 𝜏 ∈ 𝒯 𝑛 , we write 𝐷 𝜏 for the number of segments of 𝜏 . Fig. 2-1 provides a visual example.

  2 KCP dynamic programming step with given segment costs Define Γ the cumulative sum and 𝑇 the cumulative trace of the matrix K, i.e., ∀1 ≤ 𝑠 < 𝑡 ≤ 𝑛, Γ 𝑠,𝑡 :=

	𝑠	𝑡
	∑	∑	𝑘 (𝑋 𝑖 , 𝑋 𝑗 )	and	𝑇 𝑠 :=
	𝑖=1	𝑗=1
	procedure CostMatrix(S,D)
	𝑛 ← length of 𝑆(:, 1) 𝐶 ← zeros(𝐷, 𝑛) 𝐼 ← zeros(𝐷, 𝑛) for 𝑡 = 1 : 𝑛 do				◁ cost matrix ◁ index matrix
	𝐶(1, 𝑡) ← 𝑆(0, 𝑡) end for
	for 𝑑 = 2 : 𝐷 do
				}
	end for
	end for
	output 𝐼, 𝐶
	end procedure
	procedure BackTracking(C,I,D)
	 𝜏 ← zeros(𝐷) 𝑝 ← 𝑛 for 𝑑 = 1 : 𝐷 do			◁ estimated segmentation ◁ current position
	𝑝 ← 𝐼(𝐷 -𝑑 + 1, 𝑝) end for
	𝜏 return ̂ end procedure

for 𝑡 = 𝑑 : 𝑛 do [𝐼(𝑑, 𝑡), 𝐶(𝑑, 𝑡)] ← min 𝑠≤𝑡 { 𝐶(𝑑 -1, 𝑠) + 𝑆(𝑠, 𝑡) 𝑠 ∑ 𝑖=1

𝑘 (𝑋 𝑖 , 𝑋 𝑖 ) .

  Assumption 2.2. A positive constant 𝑉 < +∞ exists such that max

		1, Assumption 2.1 holds true when
	∀𝑗 ∈ {1, . . . , 𝐾},	𝑘 (𝑥, 𝑥) ≤ 𝑀 2	for 𝑃 𝑗 -a.e. 𝑥 ∈ 𝒳 .
	It is sometimes possible to weaken Assumption 2.1 into a finite variance assump-
	tion.

  Proof of (ii). Since 𝐷𝜏 1 = 𝐷 𝜏 2 , we can set 𝐷 = 𝐷 𝜏 1 = 𝐷 𝜏 2 . Next, define 𝜑(𝑖) := arg min 1≤𝑗≤𝐷-1 ⃒ ⃒ 𝜏 1 𝑖 -𝜏 2 𝑗 ⃒ ⃒and 𝐶 𝜑 (𝑖) := |𝜑(𝑖)| for all 𝑖 ∈ {1, . . . , 𝐷 -1}. Clearly, 𝐶 𝜑 (𝑖) ≥ 1 for any 𝑖. Let us show that we actually have 𝐶 𝜑 (𝑖) = 1.

1 , 𝜏 2 ) for any 𝑖 ∈ { 1, 2, 3 } .

  finer than 𝜏 . Reducing 𝜏 ∘ to a segmentation  𝜏 ∘ of {1, 2, . . . , 𝜏 ⋆ 𝑖 } in 𝑘 segments and 𝜏 ⋆ to a segmentation ̃ 𝜏 ⋆ of {1, 2, . . . , 𝜏 ⋆ 𝑖 } in 𝑖 segments and defining ̃ 𝜇 ⋆ = (𝜇 ⋆ 1 , . . . , 𝜇 ⋆ 𝜏 ⋆ 𝑖 ) ∈ ℋ 𝑖 , we get back to a situation covered by the induction since 𝑖 ≤ 𝐷 ⋆ -1 and 𝑘 < 𝑖. So,

Table 4 .

 4 4 -Performances of KCP with Gaussian kernel on synthetic data with a single jump in the variance, measured by d

  Note that both the Gaussian kernel and the Laplace kernel satisfy the assumptions of Theorem 4.2.We now explain how Theorem 4.2 allows to compute ∆ 2 in certain cases. Let us restrict our study to real observations with changes in the mean and centered i.i.d. noise with a density on R, that is Assumption 4.1. There exist real numbers 𝐿 1 , . . . , 𝐿 𝑛 and i.i.d. centered real random variables 𝛽 𝑖 with densities 𝜌 𝑖 such that 𝐿 𝑖 + 𝛽 𝑖 . 𝜇 ⋆ 𝑖+1 . It is the content of the next proposition, which is equivalent to Lemma 13 in[START_REF] Sriperumbudur | Injective hilbert space embeddings of probability measures[END_REF].Proposition 4.1. Assume that 𝑘 is a translation-invariant kernel such that 𝜅 and ℱ 𝜅 are integrable. Suppose that Assumption 4.1 holds true. Let 𝑖, 𝑗 be distinct

	is the RKHS associated to 𝑘.
	∀𝑖 ∈ 𝑋 𝑖 = This setting corresponds to the experiments of Section 4.1 and 4.2. Recall that { 1, . . . , 𝑛 } ,
	∆ 2 = min 𝜇 ⋆ 𝑖 ̸ =𝜇 ⋆ 𝑖+1 such that 𝜇 ⋆ 𝑖 ̸ = elements of { 1, . . . , 𝑛 ⃦ ⃦ 𝜇 ⋆ 𝑖+1 -𝜇 ⋆ 𝑖 } . Then ⃦ ⃦ ℋ , thus we have to compute	⃦ ⃦ 𝜇 ⋆ 𝑖+1 -𝜇 ⋆ 𝑖	⃦ ⃦ ℋ for every 𝑖
						The subset
	ℋ of 𝐿 2	(	R 𝑑 )	that consists of integrable and continuous functions 𝑓 such that
				‖𝑓 ‖ 2 ℋ :=	1 (2𝜋) 𝑑	∫ |ℱ 𝑓 (𝜔)| 2 ℱ 𝜅(𝜔)	d 𝜔 < +∞ ,	(4.3)
	endowed with the inner product
				⟨𝑓, 𝑔⟩ ℋ :=	1 (2𝜋) 𝑑	∫ ℱ 𝑓 (𝜔)ℱ 𝑔(𝜔) ℱ 𝜅(𝜔)	d 𝜔 ,

  .14) First note that, by the definition of 𝑚, which is exactly 1/(2(𝛼 2 + (1 -𝛼) 2 )) according to the definition of 𝜅 𝛼 . Now we turn to Eq. (5.11). By the same manipulations, we obtain Remark 5.4. It is also possible to prove a version of Prop. 5.3 for an arbitrary number of segments. With the notations of Remark 5.1, suppose that 𝜇 𝑝 is the average on segment 𝑝. Assume ‖𝜇 𝑝 -𝜇 𝑞 ‖ = 𝜆 𝑝,𝑞 𝑑 and order the 𝜆 𝑝,𝑞 in increasing order, and set 𝑎 0 = ∑ 𝑝 𝛼 2 𝑝 and 𝑎 𝑖 = 2𝛼 𝑝 𝑖 𝛼 𝑞 𝑖 . The median will then be close to 𝜆 𝑝 𝑖 ⋆ ,𝑞 𝑖 ⋆ 𝑑, where 𝑖 ⋆ is such that 𝑎 0 + • • • + 𝑎 𝑖 ⋆ > 1/2.

		P	( 2𝜒 2 𝑑 ≤ 𝑚	)	= P	(	𝜒 2 𝑑 ≤ 𝑑 +	𝜅 𝛼 2	√	𝑑	)
									= P	(	𝜒 2 𝑑 -𝑑 √ 2𝑑	≤	𝜅 𝛼 2 √ 2	)	.
	A direct application of Lemma 5.3 (see Section 5.6) yields,
		P	(	𝜒 2 𝑑 -𝑑 √ 2𝑑	≤	𝜅 𝛼 2 √ 2	)	→ Φ	(	𝜅 𝛼 2 √ 2	)	,
	P	( 2𝜒 2 𝑑 (𝜆) ≤ 𝑚	)	= P	(	𝜒 2 𝑑 (𝜆) -(𝑑 + 𝜆) √ 2(𝑑 + 𝜆)	≤	𝜅𝛼 2 √ 2(𝑑 + 2𝜆) √ 𝑑 -𝜆	)	.
	Since 𝜆 = O (𝑑), clearly				𝜅𝛼 2 √ 2(𝑑 + 2𝜆) √ 𝑑 -𝜆	→ -∞ .
	According to Lemma 5.3,
				𝜒 2 𝑑 (𝜆) -(𝑑 + 𝜆) √ 2(𝑑 + 𝜆)

ℒ

-→ 𝒩 (0, 1) , and we can conclude.

  𝑑/2 . Define 𝑆 𝑑 := (𝑌 -(𝑑 + 𝜆))/ √ 2(𝑑 + 𝜆). We show that 𝜙 𝑆 𝑑 (𝑡) → exp (-𝑡 2 /2), and conclude with Lévy's continuity's theorem.

								= exp	( -𝑡 2 + o	( 𝑡 2 ))	.
	E	[	exp	(	i𝑡	𝑌 -(𝑑 + 𝜆) √ 2(𝑑 + 2𝜆)	)]	= E	[	exp	(	i	𝑡 2(𝑑 + 2𝜆) √	)]	exp	(	√ 2(𝑑 + 2𝜆) -i𝑡(𝑑 + 𝜆)	)
								= exp	⎛ ⎝	i𝜆 1 -√ √ 2(𝑑+2𝜆) 𝑡 2i𝑡 2(𝑑+2𝜆)	⎞ ⎠ • exp	(	√ 2(𝑑 + 2𝜆) -i𝑡(𝑑 + 𝜆)	)
												•	(	1 -2i	√ 2(𝑑 + 2𝜆) 𝑡	) -𝑑/2
											⎛
								= exp	⎝	i𝜆𝑡 2(𝑑 + 2𝜆) √	•	1 -	1 √ 2(𝑑+2𝜆) 2i𝑡	-	i𝑡(𝑑 + 𝜆) √ 2(𝑑 + 2𝜆)
												-	𝑑 2	log	( 1 -2i	√	2(𝑑 + 2𝜆) 𝑡	))
								= exp	(	i𝜆𝑡 2(𝑑 + 2𝜆) √	( 1 +	2i𝑡 2(𝑑 + 2𝜆) √	)	-	i𝑡(𝑑 + 𝜆) √ 2(𝑑 + 2𝜆)
												+	𝑑 ¡ 2	¡ 2i𝑡 2(𝑑 + 2𝜆) √	+	𝑑 ¡ 2	1 ¡ 2	-¡ 2(𝑑 + 2𝜆) 4𝑡 2	+ o	( 𝑡 2 ) )
								= exp	(	𝜆𝑡 2(𝑑 + 2𝜆) √	-	¡ 2𝜆𝑡 2 2(𝑑 + 2𝜆) ¡	-	𝑡𝜆 2(𝑑 + 2𝜆) √
											¨¨¨¨¨ï		¨¨¨¨¨ï
												-	𝑑𝑡 2 2(𝑑 + 2𝜆)	+ o	(	𝑡 2 ) )
																	123

  Dans cette thèse, nous nous intéressons à une méthode de détection des ruptures dans une suite d'observations appartenant à un ensemble muni d'un noyau semi-défini positif. Cette procédure est une version « à noyaux » d'une méthode des moindres carrés pénalisés.Notre principale contribution est de montrer que, pour tout noyau satisfaisant des hypothèses raisonnables, cette méthode fournit une segmentation proche de la véritable segmentation avec grande probabilité. Ce résultat est obtenu pour un noyau borné et une pénalité linéaire, ainsi qu'une autre pénalité venant de la sélection de modèles. Les preuves reposent sur un résultat de concentration pour des variables aléatoires bornées à valeurs dans un espace de Hilbert, et nous obtenons une version moins précise de ce résultat lorsque l'on suppose seulement que la variance des observations est finie. Dans un cadre asymptotique, nous retrouvons les taux minimax usuels en détection de ruptures lorsqu'aucune hypothèse n'est faite sur la taille des segments. Ces résultats théoriques sont confirmés par des simulations. Nous étudions également de manière détaillée les liens entre différentes notions de distances entre segmentations. En particulier, nous prouvons que toutes ces notions coïncident pour des segmentations suffisamment proches. D'un point de vue pratique, nous montrons que l'heuristique du « saut de dimension » pour choisir la constante de pénalisation est un choix raisonnable lorsque celle-ci est linéaire. Nous montrons également qu'une quantité clé dépendant du noyau et qui apparaît dans nos résultats théoriques influe sur les performances de cette méthode pour la détection d'une unique rupture. Dans un cadre paramétrique, et lorsque le noyau utilisé est invariant par translation, il est possible de calculer cette quantité explicitement. Grâce à ces calculs, nouveaux pour plusieurs d'entre eux, nous sommes capable d'étudier précisément le comportement de la constante de pénalité maximale.

A base pair (bp) consists of an 𝐴 -𝑇 or 𝐺 -𝐶 pair; it is approximately 340pm long. A megabase (Mb) denotes 10 6 base pairs. The size of the Human genome is approximately 3, 000Mb.

Victoria (2015), a 2 hours 18 minutes long movie, consists of a single continuous shot.

A sequence of random variables is homoscedastic if all the random variables in the sequence have the same variance. If not, we say that the random variables are heteroscedastic.

We say that a random variable 𝑆 is sub-Gaussian if there exists 𝐶, 𝑣 > 0 such that P (|𝑆| > 𝑡) ≤ 𝐶 e -𝑣𝑡 2 . Informally, the tails of the distribution of 𝑆 decay at least as fast as the tails of a Gaussian random variable.

Let us stress that some key properties of dot products are not satisfied by kernels, in particular linearity.

That is a Hilbert space whose elements are functions 𝑓 : 𝒳 → R and such that all the evaluation functionals are continuous.

In the context of model selection, we can see 𝐷 as the dimension of the model, hence the notation.

A topological space is called separable if it contains a countable dense subset.

, 𝜇 ⋆ 𝜏 ⋆ 1 +1 = • • • = 𝜇 ⋆ 𝜏 ⋆

, • • • 𝜇 ⋆ 𝜏 ⋆ 𝐷 ⋆ -1 +1 = • • • = 𝜇 ⋆ 𝑛 and ∀𝑖 ∈ {1, . . . , 𝐷 ⋆ -1}, 𝜇 ⋆ 𝜏 ⋆ 𝑖 ̸ = 𝜇 ⋆ 𝜏 ⋆ 𝑖+1 (2.6) with 1 ≤ 𝜏 ⋆ 1 < • • • < 𝜏 ⋆ 𝐷 ⋆ -1 ≤ 𝑛.We call the 𝜏 ⋆ 𝑖 s the true change-points. It should be clear that it is always possible to define 𝜏 ⋆ , and that the previous display is not an assumption we make.A kernel is said to be characteristic if the mapping 𝑃 ↦ → E 𝑋∼𝑃 [Φ(𝑋)] is injective, for 𝑃 belonging to the set of Borel probability measures on 𝒳[START_REF] Fukumizu | Dimensionality reduction for supervised learning with reproducing kernel hilbert spaces[END_REF][START_REF] Fukumizu | Kernel measures of conditional dependence[END_REF]. In simpler terms, when 𝑘 is a characteristic kernel, 𝑋 𝑖 and 𝑋 𝑖+1 have the same distribution if and only if 𝜇 ⋆ 𝑖 = 𝜇 ⋆ 𝑖+1 , and 𝜏 ⋆ indeed corresponds to the set of changes in the distribution of the 𝑋 𝑖 . For instance, all integrally

positive definite kernels are characteristic, including the Gaussian kernel, see[START_REF] Sriperumbudur | Hilbert space embeddings and metrics on probability measures[END_REF]. Therefore, in the setting of Example 2.1, for 𝑛 large enough, 𝐷 ⋆ = 𝐾 + 1 and 𝜏 ⋆ ℓ = ⌊𝑛𝑏 ℓ ⌋ for ℓ = 1, . . . , 𝐾.For a general kernel, some changes of 𝑃 𝑋 𝑖 , the distribution of 𝑋 𝑖 , might not appear in 𝜏 ⋆ . For instance, with the linear kernel, 𝜏 ⋆ only corresponds to changes of the mean of the 𝑋 𝑖 . In most cases, a characteristic kernel is known and we can choose to use KCP with a characteristic kernel; then, as we prove in Chapter 3, KCP eventually detects any change in the distribution of the observations. But one can also choose a non-characteristic kernel on purpose, hence focusing only on some changes in the distribution of the 𝑋 𝑖 . For instance, the polynomial kernel of order 𝑑 is not characteristic and leads to the detection of changes in the first 𝑑 moments of the distribution; with the linear kernel, KCP detects changes in the mean of the 𝑋 𝑖 . 3. A measurable, symmetric and bounded function 𝑘 is said to be integrally positive definite if ∫∫ 𝒳 𝑘 (𝑥, 𝑦) d 𝜇(𝑥) d 𝜇(𝑦) > 0 for any finite signed Borel measure 𝜇 on 𝒳 .

Note that we slightly abuse the notations ‖•‖ and ⟨•, •⟩, previously defined for elements of R 𝑛 . This should create no confusion.

Lemma 3.8 is a refinement of a result obtained by Arlot, Celisse and Harchaoui in the first version of[START_REF] Arlot | A kernel multiple change-point algorithm via model selection[END_REF].

Indeed, this procedure can be implemented for any non-decreasing penalty.

Acknowledgements

}

end if end for return 𝑐, 𝐷 end procedure on the experiments repetitions, 𝑥 ⋆ stands for a performance strictly better than 0.01, and x indicates the best performance on the line.

In the following experiments, we test KCP against a variety of generated data. These data are piecewise-constant in R, with standard independent Gaussian noise. The number of jumps is fixed, say 𝐷. The jump locations are then chosen so that the segment sizes follow a multinomial distribution 2 with parameter (100; 1/𝐷, . . . , 1/𝐷). Finally, the jump sizes are chosen uniformly at random in a pre-specified range. Experiments are repeated 10 2 times, and the randomness comes from the segments sizes, the breakpoints locations and the noise; standard deviation is computed over these repetitions.

We choose to focus on translation-invariant kernels, namely the Gaussian kernel 𝑘 G in Tables 4. 1, 4.3, and 4.4, and the Laplace kernel 𝑘 L in Table 4.2. Keeping in mind the quadratic complexity of KCP, experiments over more than 10 3 data-points are quite expensive thus we limit ourself to 𝑛 ≃ 10 2 . We choose to present results only for 𝑛 = 200, as we obtained similar results for values of 𝑛 in the same range. The maximum authorized number of segments is set to 𝐷 max = 20, which is much larger than the true number of segments in the simulations.

In our results, we choose to use d

H , whereas our theoretical results are all stated with d (1) ∞ -see Chapter 3. This should not confuse the reader, recall that for segmentations that are close enough, these functions coincide, as we proved in Lemma 3.1.

Note that d

(2) H is upper bounded by 0.5, and takes special values in the limit cases. For instance, if 𝜏 0 is the 𝑛-segments segmentation, then d

(2)

Consistency

A consequence of our main result, Theorem 3.1, is that for a bounded kernel, the KCP is consistent in the asymptotic setting presented in Example 2.1. In this section, we illustrate this by a simulation study.

Detecting changes in the mean with the Gaussian kernel. Let us consider the archetype change-point detection problem -finding changes in the mean of a sequence of independent random variables-and show how these changes are localized more precisely when more data are available.

We define three functions 𝜇 𝑚 : [0, 1] → R, 1 ≤ 𝑚 ≤ 3, previously used by [START_REF] Arlot | Segmentation of the mean of heteroscedastic data via cross-validation[END_REF], which cover a variety of situations (see Fig. 4-3). For each 𝑚 ∈ {1, 2, 3} and several values of 𝑛 between 10 2 and 10 3 , we repeat 10 3 times the following:

-Sample 𝑛 independent Gaussian random variables 𝑔 𝑖 ∼ 𝒩 (0, 1); -Set 𝑋 𝑖 = 𝜇 𝑚 (𝑖/𝑛) + 𝑔 𝑖 -Fig. 4-3 shows one sample for each 𝑚 ∈ {1, 2, 3}; -Perform KCP with Gaussian kernel and linear penalty on 𝑋 1 , . . . , 𝑋 𝑛 ; the penalty constant is chosen as indicated in Section 3.3, the bandwidth is set to 0.1, and the maximum number of change-points is set to 30; -Compute d H

(2) (𝜏 ⋆ , ̂ 𝜏 𝑛 ). The results are collected in Fig. 4-4, where each graph corresponds to a regression function 𝜇 𝑚 . We represent in logarithmic scale the mean distance between the true Chapter 5

The median heuristic

Abstract

The median heuristic is a popular tool to set the bandwidth of radial basis function kernels. While its empirical performances make it a safe choice under most circumstances, there is little theoretical understanding of why this is the case. For a large sample size, we show in this chapter that the median heuristic behaves approximately as the median of a distribution that we describe completely in the setting of kernel two-sample test and kernel change-point detection. More precisely, we show that the median heuristic is asymptotically normal around this value. We illustrate these findings when the underlying distributions are multivariate Gaussian distributions. This chapter is based upon the preprint [START_REF] Garreau | Asymptotic normality of the median heuristic[END_REF].

Introduction

Kernel methods form an important class of algorithms in machine learning and statistics. They make use of rich feature spaces that depend only on the kernel chosen by the user. Given a positive semi-definite kernel 𝑘 and observations 𝑥 1 , . . . , 𝑥 𝑛 , the first step of most kernel-based method is to compute the Gram matrix K = (𝑘 (𝑥 𝑖 , 𝑥 𝑗 )) 1≤𝑖,𝑗≤𝑛 . Thanks to the celebrated kernel trick , all ensuing computations need only the knowledge of K.

We are especially interested in data lying in a metric space (𝒳 , d). When this is the case, commonly used kernels are radial basis function kernels. They can be written

where 𝑓 : R + → R + and 𝜈 is a positive parameter called the bandwidth. In many applications, the space 𝒳 is R 𝑑 and the distance d is derived from the Euclidean norm ‖𝑥‖ = √ ∑ 𝑖 𝑥 2 𝑖 , that is, d (𝑥, 𝑦) = ‖𝑥 -𝑦‖. Among this class of kernel are found numerous kernels often used in practice. For instance, 𝑓 (𝑥) = exp (-𝑥 2 ) corresponds to the Gaussian kernel [START_REF] Aizerman | Theoretical foundations of the potential function method in pattern recognition[END_REF], arguably the most popular positive definite kernel used in applications (see, for instance, [START_REF] Vert | A primer on kernel methods[END_REF]). The function 𝑓 (𝑥) = exp (-𝑥) yields the exponential arbitrary 𝑝.

Main results

The empirical distribution function

Under Assumption 5.1, there are three possibilities for an element of the family There are 𝛼𝑛(𝛼𝑛 -1)/2 occurrences of case (i). Suppose that we make 𝑛 → ∞, then case (i) occurs with proportion 𝛼 2 . Similarly, case (ii) occurs with proportion (1 -𝛼) 2 and case (iii) with proportion 2𝛼(1 -𝛼). Define a mixture distribution 𝑇 ∼ 𝑇 𝑋𝑋 , 𝑇 ∼ 𝑇 𝑌 𝑌 and 𝑇 ∼ 𝑇 𝑋𝑌 with weights 𝛼 2 , (1 -𝛼) 2 and 2𝛼(1 -𝛼) respectively. Thereafter, we will call 𝑇 the target distribution and denote by 𝐹 its cumulative distribution function. The non-rigorous reasoning above suggests that when 𝑛 → ∞, 𝑇 𝑛 behaves like a 𝑛-sample of the target distribution 𝑇 . Indeed, a specialization of a result stated in the next paragraph shows that ∀𝑡 ∈ R, ̂ 𝐹 𝑛 (𝑡) P -→ 𝐹 (𝑡) .

(5.5) Fig. 5-1 illustrates this phenomenon. For large 𝑛, if we plot the histogram of the ‖𝑋 𝑛,𝑖 -𝑋 𝑛,𝑗 ‖ 2 for 1 ≤ 𝑖 < 𝑗 ≤ 𝑛, then the "two bumps" behavior depicted in Fig. 5-1 is typical. The left mode of the empirical distribution corresponds to 𝑇 𝑋𝑋 and 𝑇 𝑌 𝑌 , close to zero by definition, whereas the right mode corresponds to 𝑇 𝑋𝑌 that can be arbitrarily far from 0. Eq. (5.5) is already a step in the comprehension of the median heuristic, since we are now able to think about 𝐻 𝑛 "approximately" as the theoretical median of the target distribution 𝑇 . It turns out that (5.5) is a trivial consequence of a much stronger statement. Indeed, ̂ 𝐹 𝑛 (𝑡) can be seen as a sum of three dependent 𝑈 -statistic with kernel ℎ(𝑥, 𝑦) = 1 ‖𝑥-𝑦‖ 2 ≤𝑡 , and the following result shows that it follows a central limit theorem. We refer to classical textbooks [START_REF] Lee | U-statistics: Theory and Practice[END_REF][START_REF] Korolyuk | Theory of U-statistics[END_REF] for an introduction to the theory of 𝑈 -statistics.

Proposition 5.1 (CLT for non-identically distributed triangular array 𝑈 -statistic). Consider ℎ : R × R → R such that ℎ ∈ 𝐿 2 (𝑃 ) ∩ 𝐿 2 (𝑄) × 𝐿 2 (𝑃 ) ∩ 𝐿 2 (𝑄), and suppose theoretical quantiles of the target distribution if the convergence of the empirical distribution function is "strong enough" [van der Vaart, 1998, Chapter 21]. More precisely, if this convergence is uniform or follows a CLT -as in our case. Proposition 5.2 (Asymptotic normality of 𝐻 𝑛 ). Suppose that Assumption 5.1 holds, and define 𝑇 as in Sec. 5.3.1. Define 𝑚 = med(𝑇 ) the theoretical median of the target distribution. Suppose that 𝐹 has a non-zero derivative at 𝑚, and define 𝜎 = 𝜎(ℓ, 𝑃, 𝑄) as in Prop. 5.1, where ℓ(𝑥, 𝑦) = 1 ‖𝑥-𝑦‖ 2 ≤𝑚 .

.

(5.10)

We illustrate Prop. 5.2 in Fig. 5-2. Before we turn to the proof of Prop. 5.2, we make a few remarks.

Note that we made very few assumptions on the distribution of the 𝑋 𝑛,𝑖 s, hence Prop. 5.2 can be applied in a wide range of situations. Of course, when met with an experimental problem, one does not know 𝑃 and 𝑄. Nevertheless, Prop. 5.2 suggests that 𝐻 𝑛 is concentrated around some deterministic value depending on the data at hand for large 𝑛. If more information is available regarding 𝑃 and 𝑄, we will demonstrate in the next section that it is possible to transfer this information to med(𝑇 ), hence to 𝐻 𝑛 .

Once again, this result should not come as a shock to the knowledgeable reader since empirical 𝑈 -quantiles are known to satisfy asymptotic normality since [START_REF] Serfling | Approximation theorems of mathematical statistics[END_REF] in the i.i.d. case. Though a lot of work has been done to relax the independence assumption, to the best of our knowledge there is no result regarding the non-identically distributed case. In the two-sample setting, some results exist, both in the independent case [START_REF] Lehmann | Consistency and unbiasedness of certain nonparametric tests[END_REF] and with some dependence structure [START_REF] Dehling | Asymptotic distribution of two-sample empirical U-quantiles with applications to robust tests for shifts in location[END_REF]. However, as noted before, in our setting it is necessary to consider the intra-segment interactions.

Note that, as it can be seen for instance in Fig. 5-1, observations lying between max(E 𝑇 𝑋𝑋 , E 𝑇 𝑌 𝑌 ) and E 𝑇 𝑋𝑌 are quite scarce. Therefore, it is possible for 𝐹 ′ (𝑚) to be small, leading to a large variance term in (5.10). Note that 𝐹 ′ (𝑚) ̸ = 0 does not hold for arbitrary continuous distributions.

Proof of Prop. 5.2

Set 𝑡 ∈ R. The general idea of the proof is to rewrite statements about the event { √ 𝑛 (𝐻 𝑛 -𝑚) ≤ 𝑡 } as statements about a sum of 𝑈 -statistics. We will then control these 𝑈 -statistics with Prop. 5.1 for conveniently chosen ℎ, and conclude with Slutsky's Lemma. Throughout this proof, we only suppose that 𝑝 ∈ (0, 1) to emphasize that Prop. 5.2 can be extended to any quantile, not only the median.

We use the property of the generalized inverse to obtain

The accuracy of Eq. ( 5.14) is illustrated in Fig. 5-3. We observe that, even though Prop. 5.3 is an asymptotic statement, it is valid with good accuracy for small values of 𝑑. Furthermore, we see the variance (represented by the shaded area) thinning when the number of data-points increases, as predicted by Prop. 5.2.

We make a few comments.

At first sight, it appears that 𝑚 behaves as if only 𝑇 𝑋𝑋 and 𝑇 𝑌 𝑌 contribute, in the sense that Eq. (5.14) does not depend on 𝜇. There is a simple explanation to this observation. Indeed, a careful inspection of Fig. 5-1 shows that the left part of the histogram, corresponding to the contribution of 𝑇 𝑋𝑋 and 𝑇 𝑌 𝑌 , is much larger than the right part, corresponding to 𝑇 𝑋𝑌 . It holds since, for any 𝛼 ∈ (0, 1),

Thus if E [𝑇 𝑋𝑋 ] and E [𝑇 𝑌 𝑌 ] are both small with respect to E [𝑇 𝑋𝑌 ], this is a general phenomenon and 𝐻 𝑛 will be close to max (E [𝑇 𝑋𝑋 ] , E [𝑇 𝑌 𝑌 ]). Hence the median heuristic will select a bandwidth according to the maximum variance, since E [𝑇 𝑋𝑋 ] = 2 Var (𝑋) and E [𝑇 𝑌 𝑌 ] = 2 Var (𝑌 ). As noted in Gretton et al. [2012b, Sec. 5], if the variance of 𝑋 and 𝑌 is much higher than the scale of the changes one aims to detect, the median heuristic will thus fail completely to select an appropriate bandwidth.

In this special case, we recover the setting of Reddi et al. [2015, Sec. 4.1 (A)] as in Sec. 5.1.1, with the exception that ‖𝜇‖ 2 = O (𝑑). Following the reasoning in Sec. 5.1.1, we would write

which is no longer dominated by the first term as 𝑑 increases. Yet, according to Prop. 5.3, 𝐻 𝑛 ∼ 2𝑑. We have thus obtained a rigorous and more precise result, though we acknowledge that the order of magnitude of 𝐻 𝑛 stays the same, that is, 𝐻 𝑛 = O (𝑑). 5.4.1 Proof of Prop. 5.3 We are going to show that

and our claim will follow. We first note that P (𝑇 ≤ 𝑚) can be decomposed as .15) Let us show that, with the prescribed choice of 𝜅 𝛼 , the left-hand side of Eq. ( 5.15) converges to 1/2 and (5.11) converges to 0.

List of Figures