Conclusions et perspectives

Les méthodologies presentées dans ce manuscrit ont été utilisées pour caractériser des processus synthétiques ou des mesures de vitesse expérimentale de haute qualité. Seulement dans l'application biomedical (chapitre 9), ces méthodologies ont été testées sur des données de mauvaise qualité. Comme on a expliqué tout au long de la thèse, ces méthodologies peuvent être appliquées de manière général et ainsi être utilisées pour étudier des systèmes génériques. Par conséquent, quelques problèmes dans les méthodologies pourraient être révélés par des nouvelles applications sur des données du monde réel, et spécialement sur des données de mauvaise qualité. Ces applications pourraient aussi révéler des aspects qu'ont besoin d'être ameliorés afin que les mesures soient instructives quand elles sont utilisées sur ce type de données.

Tout au long du manuscrit, on a analysé des processus stationnaires et non-stationnaires, stochastiques ou chaotiques, et on a donné une interprétation à nos mesures pour chaque type de processus. Dans le cas des processus non-stationnaires, on l'a fait même dans deux cadres différents. La question qui émerge de manière naturelle est si on peut utiliser les méthodologies developpées pendant la thèse pour caractériser des processus déterministes non-chaotiques, et quelles sont les interprétations de nos mesures dans ce cas.

Dans l'analyse de flux d'information et relations de causalité est, peut être, où plus des questions restent ouvertes. On a conclu que les mesures de causalité entre échelles d'un processus temporel sont fortement influencées par la manière de générer ces échelles. Pourtant, pour toutes les fonctions d'échelle utilisées, on trouve que les petites échelles causent les grandes échelles. Ce résultat général, trouvé pour tous les systémes étudiés et pour toutes les fonctions d'échelle utilisées, doit être étudié plus attentivement. Un autre progrès important dans la caractérisation de la turbulence peut être fait en étudiant la causalité entre séries temporelles expérimentales avec une signification d'échelle. Ainsi, l'influence due à la manière de générer les échelles d'une série temporelle serait eliminée et la mesure de causalité dépendrait seulement des proprietés du système étudié.

First of all, I want to thank my PhD advisors Stèphane G. Roux and Nicolas B. Garnier for their inestimable help during the development of my thesis. They were always available to guide me with their great and innovative ideas, and to show me the research career. Without their help throughout these three years, this dissertation wouldn't have reached its final form.

I wish to thank, Alain Arneodo and Annick Lesne for being referees of my PhD dissertation, and Laurette Tuckerman, Bérengère Dubrulle and Laurent Chevillard for being members of the jury of my thesis.

Special thanks to Laurent Chevillard for helping me everytime I had questions on turbulence. His knowledge on the field enlightened some shadows on the theoretical aspects of turbulence, and his suggestions and remarks were very useful to understand them.

I want to thank Roberto Leonarduzzi for answering all my questions about multifractal analysis and for helping me to understand multifractality.

Finally, I wish to thank the two people who bring me in science research, when I was only an undergraduate student at the Universidad Autónoma de Madrid : Carlos Sánchez and José Ramón Ares. They, together with all the members of the Materials of Interest in Renewable Energies laboratory, show me what research was.

Résumé

Cette thèse est consacrée à l'étude statistique des systèmes complexes à travers l'analyse de signaux expérimentaux, de signaux synthétiques et de signaux générés à partir de modèles théoriques. On a choisi la turbulence comme paradigme d'étude en raison de ses propriétés: dynamique non linéaire, comportement multi-échelle, cascade d'énergie, intermittence ... Afin de faire une caractérisation statistique d'un système complexe on s'intéresse à l'étude de la distribution (fonction de densité de probabilité), des corrélations et dépendances, et des relations de causalité de Wiener, des signaux qui décrivent le système. La théorie de l'information apparaît comme un cadre idéal pour developer ce type d'analyse.

Dans le cadre de la théorie de l'information on a développé deux méthodologies, différentes mais reliées, pour analyser les propriétés d'auto similarité d'un système complexe, et plus précisément de la turbulence. La première méthodologie est basée sur l'analyse des incréments du processus étudié, avec l'entropie de Shannon et la divergence de Kullback-Leibler. La deuxième méthode, qui permet d'analyser des processus non stationnaires, est basée sur l'analyse du processus avec le taux d'entropie de Shannon. On a étudié la relation entre les deux méthodologies, lesquelles caractérisent la distribution d'information du système et la déformation de la distribution des incréments, à travers les échelles.

Introduction: buts et contributions

Cette thèse vise à développer de nouvelles méthodologies pour la caractérisation des systèmes complexes.

La définition de systèmes complexes reste encore vague, et en conséquence ses fontières sont imprécises [START_REF] Mitchell | Complexity: A guided tour[END_REF]. Par système complexe, nous comprenons celui composé de plusieurs parties qui interagissent entre elles de manière non linéaire. La plupart du temps ces systèmes sont multi-échelles, i.e., ils presentent des échelles différentes avec des comportements différents. Cette définition de système complexe inclue un grand nombre de systèmes comme: le marché financier [START_REF] Mauboussin | Revisiting market efficiency: the stock market as a complex adaptive system[END_REF], un ecosystème [START_REF] Mcgarigal | Multi-scale habitat selection modeling: a review and outlook[END_REF], le cerveau [START_REF] Siegelmann | Complex systems science and brain dynamics[END_REF], la Terre [START_REF] Donner | Understanding the Earth as a complex system -recent advances in data analysis and modelling Earth sciences[END_REF], une colonie de fourmis [START_REF] Miramontes | Order-disorder transitions in the behavior of ant societies[END_REF] ... Le prototype idéal de système complexe est la turbulence des fluides, car elle est hautement non linéaire et multi-échelles [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF]. À cause de la facilité de généralisation à n'importe quel système complexe des méthodologies capables de caractériser la turbulence, et aussi parce que la turbulence représente un sujet très intéressant, nous utilisons la turbulence pleinement developée comme notre objet d'étude tout au long de la thèse.

Nous proposons de décrire statistiquement un système complexe par l'étude de sa distribution, ses correlations et dependences, et les relations de causalité entre différentes observables du système. La théorie de l'information [START_REF] Shannon | A mathematical theory of communication[END_REF][START_REF] Cover | Elements of Information Theory[END_REF] fournit le cadre parfait pour accomplir ce type d'analyse [START_REF] Prokopenko | An information-theoretic primer on complexity, self-organization, and emergence[END_REF][START_REF] Lizier | An introduction to Transfer entropy: Information flows in complex systems[END_REF].

Chapitre 1: Aspects théoriques

La théorie de l'information cherche à décrire les lois mathématiques qui gouvernent la transmission, le stockage, le traitement, la mesure et la représentation de l'information. Claude E. Shannon a développé proprement les bases de la théorie de l'information en 1948, dans son article "A Mathematical theory of communication" [START_REF] Shannon | A mathematical theory of communication[END_REF] puis l'a étendu avec Weaver [START_REF] Shannon | The Mathematical Theory of Information[END_REF]. Comme le titre du premier article (de Shannon) le souligne, la théorie de l'information apparait initialement comme une théorie de la communication. Neanmoins, depuis les années cinquante, les applications ont commencé à se généraliser dans d'autres domaines comme les systèmes dynamiques [START_REF] Kolmogorov | On the shannon theory of information transmission in the case of continuous signals[END_REF][START_REF] Kolmogorov | New metric invariant of transitive dynamical systems and automorphism of lebesgue spaces[END_REF][START_REF] Sinai | On the notion of entropy of a dynamical system[END_REF][START_REF] Kolmogorov | Three approaches to the quantitative definition of information[END_REF]. Ce saut des processus stochastiques, l'objet naturel d'étude de la théorie de l'information, aux systèmes dynamiques a supposé un progrès très important dans le développement de la théorie de l'information, qu'a permis son propagation á une gamme très large de domaines.

Aujourd'hui la théorie de l'information est utilisée dans un spectre de domains très large: thermodynamique [START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Van Den Broeck | Stochastic thermodynamics: A brief introduction[END_REF][START_REF] Parrondo | Thermodynamics of information[END_REF], thermodynamique quantique [START_REF] Horowitz | Thermodynamics with continuous information flow[END_REF], neuroscience [START_REF] Vicente | Transfer entropy-a model-free measure of effective connectivity for the neurosciences[END_REF][START_REF] Wibral | Measuring information-transfer delays[END_REF][START_REF] Wollstadt | Efficient transfer entropy analysis of non-stationary neural time series[END_REF], etc. En plus, depuis le premier article de Shannon, le nombre de quantitées différentes en théorie de l'information a augmenté significativement. Parmi ces objects théoriques on peut trouver: taux d'entropie [START_REF] Shannon | The Mathematical Theory of Information[END_REF], divergence de Kullback-Leibler [START_REF] Kullback | On information and sufficiency[END_REF][START_REF] Kullback | Information Theory and Statistics[END_REF], entropie de transfert [START_REF] Schreiber | Measuring information transfer[END_REF], information dirigée [START_REF] Kramer | Directed Information for Channels with Feedback[END_REF] ... Récemment, l'accés à des ordinateurs de plus en plus puissants et le plus large nombre de domaines d'application de la théorie de l'information ont rendu cette théorie un sujet de recherche très interessant à la fois dans des aspects théoriques et appliqués.

Dans ce chapitre se presente le cadre de la théorie de l'information utilisé pendant la thèse, avec l'entropie de Shannon comme brique de base. Cependant le nombre de cadres differents est grand [START_REF] Chodrow | Divergence, entropy, information: An opinionated introduction to information theory[END_REF]. En plus, comme il a été montré que la théorie de l'information sert á la déscription des systèmes stochastiques ainsi qu'á la déscription des systèmes dynamiques [START_REF] Krieger | On entropy and generators of measure-preserving transformations[END_REF][START_REF] Krieger | On unique ergodicity[END_REF][START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF], tout au long de ce manuscrit on applique la théorie de l'information sur tous les deux sortes de processus.

Chapitre 2: Estimateurs non-paramétriques

Afin d'appliquer la théorie de l'information au traitement de signal, il est nécessaire de développer des codes numériques fiables, lesquels doivent estimer proprement les entropies. À notre connaissance, il y a principalement trois sortes d'algorithmes non paramétriques utilisés pour estimer des entropies: des algorithmes basés sur des histogrammes, algorithmes basés sur l'utilisation de noyaux, ou algorithmes basés sur la recherche des plus proches voisins. Chaque sorte d'algorithme suit differentes approximations pour estimer les fonctions de densité de probabilité, ou pour estimer les mesures d'entropie directement. Naturellement, d'autres sortes d'estimateurs des entropies existent, mais, tout au long de la thèse seulement des estimateurs non paramétriques ont été utilisés.

Tout au long de ce chapitre on utilise la littérature précédente et des nouveaux résultats pour choisir les estimateurs les plus fiables des mesures de théorie de l'information. Les résultats nouveaux obtenus pendant la thèse sont debillés et reliés à l'article correspondant. Le chapitre est organisé comme suit: la première section énumère les différents algorithmes pour estimer les fonctions de densité de probabilité, la deuxième présente l'algorithme de Kozachenko et Leonenko pour estimer directement l'entropie de Shannon [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF][START_REF] Singh | Nearest neighbor estimates of entropy[END_REF][START_REF] Leonenko | A class of rnyi information estimators for multidimensional densities[END_REF], la troisième est dediée à l'estimateur de Kraskov, Stögbauer et Grassberger pour l'information mutuelle [START_REF] Kraskov | Estimating mutual information[END_REF], la quatrième section présente une nouvelle méthode pour estimer le taux d'entropie d'ordre m [START_REF] Granero-Belinchón | Un estimateur du taux d'entropie basé sur l'information mutuelle[END_REF], basée sur les estimateurs de Kozachenko et Leonenko, et celui de Kraskov, Stögbauer et Grassberger. La cinquième section presente comment utiliser la prescription de Theiler pour sonder la dynamique et la complexité à une échelle λ donnée. Finalement, la dernière section présente l'estimateur de Kraskov, Stögbauer et Grassberger pour l'information mutuelle conditionnée.

Chapitre 3: Statistiques d'ordre supérieure de bruits non Gaussiens Afin de caractériser un processus ergodique X(t), qui évolue le long d'une dimension t, e.g., le temps, on a besoin de décrire simultanément: les statistiques à un point du processus à un temps donné t et les dépendances entre n points x t à temps differents. Pour analyser ces dépendances, il faut faire une hypothèse de stationnarité.

La fonction de correlation, ou sa transformé de Fourier, le spectre de puissance, sont conventionnellement utilisées pour caractériser les dépendances des processus stochastiques stationnaires, et des processus déterministes chaotiques. Cependant, l'étude des dépendances via l'analyse spectrale ou les correlations sonde seulement des interactions à deux points, et en conséquence ne prend en compte que des statistiques d'ordre deux seulement des processus analysés. Ces correlations d'ordre deux ensemble avec la moyenne et la variance de la fonction de densité de probabilité (PDF) du processus caractérisent completement les processus ayant une distribution Gaussienne. Neanmoins, pour des processus avec une PDF non Gaussienne il est nécessaire de prendre en compte des statistiques d'ordre supérieure de la PDF.

Les outils de théorie de l'information sont construits sur l'entropie de Shannon, et, comme l'entropie de Shannon, ils sont des fonctions de la PDF du processus analysé. En conséquence, ils dependent de tous les moments de la PDF et par voie de conséquence des statistiques d'ordre supérieur, comme detaillé dans la section 1.5.

L'auto information mutuelle, définie dans la section 1.4.2, et le taux d'entropie d'order m, défini dans la section 1.4.1, sont capables de mesurer des interactions à deux points au même temps qu'elles prennent en compte des statistiques de tout ordre de la PDF du processus analysé. Ces deux quantités permettent alors l'étude des dépendances, et non pas seulement des correlations, d'un processus. En plus, en augmentant la dimension d'embedding m, tous les deux peuvent mesurer des interactions á m + 1 points, et ils peuvent alors se concentrer sur les statistiques d'ordre superieur de la PDF.

L'auto information mutuelle, eq.(1.26), et le taux d'entropie d'ordre m, eq.(1.24) ou eq. (1.25), mesurent les dépendances à une échelle particulière d'interaction (section 1.5), et très peu d'articles analysent l'évolution de ces quantités à travers les échelles [START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF]. On propose de réaliser cette analyse à travers les échelles pour obtenir une analyse des dépendances à travers les échelles, laquelle peut être intéressante pour étudier des systèmes, comme par exemple, la turbulence. À titre d'exemple, cette méthodologie est appliquée sur deux processus synthétiques non Gaussiens avec une structure de dépendance différente, mais avec la même fonction de corrélation et la même PDF marginale. Notre méthodologie est alors capable de mesurer les différences qu'il y a dans les statistiques d'ordre supérieur des processus.

Chapitre 4: Étude de processus non stationnaires

La théorie de l'information a été initialement conçu pour travailler avec des processus stationnaires, où les mesures ont une interprétation claire en termes d'information (chapitre 1). Neanmoins, deux questions aparaissent immédiatement:

• Peut-on appliquer la théorie de l'information sur de processus non-stationnaires?

• Est-ce que les interprétations sur des processus stationnaires restent valides pour des processus non-stationnaires?

Dans la vie réel, un grand nombre de processus comme: la température environnemental, l'évolution des ressources d'eau, séries temporelles macroéconomiques ou le mouvement Brownien, sont non-stationnaires. Par conséquent, la caractérisation et l'analyse de ce type de processus deviennent très importants. Parce que l'étude des processus nonstationnaires est très difficile, sa caractérisation est normalement fait à travers l'étude des increments ou des dérivés temporelles. Pourtant, dans ce chapitre, nous sommes interessés par l'analyse direct des processus non-stationnaires.

Pour un processus temporel non-stationnaire M (t) dont sa PDF et ses dependences, évoluent en temps, son entropie H(M (t)) évolue avec le temps.

On peut mesurer l'entropie d'un processus à un temps donné H t (M (t)) en moyennant à travers les réalisations. Dans ce cadre, l'interprétation de l'entropie pour des processus stationnaires reste valide [START_REF] Gómez Herrero | Assessing coupling dynamics from an ensemble of time series[END_REF]. De la même manière, on peut mesurer les dépendances du processus à temps t. Réaliser ces analyses à un temps donné pour différents temps, permet de caracériser non seulement la distribution et les dépendances du processus au temps t, mais aussi son évolution en temps. Malhereusement, l'analyse à un temps donné t exige d'avoir nombreuses réalisations du processus afin de construire sa PDF sans avoir besoin de l'hypothèse d'ergodicité. Cette exigence n'est pas facilement soluble, spécialement quand on travaille avec des signaux expérimentaux.

Afin d'éviter cette exigence, nous voulons étudier un processus M (t) en analysant un signal expérimental de longueur T . On peut mesurer son entropie pendant la fenêtre temporelle [0, T ], H T (M (t)), mais l'interprétation de cette entropie n'est plus la même. We interprète H T (M (t)) comme la complexité, ou la quantité totale d'information requis pour caractériser une trajectoire temporelle M t ∀ t ∈ [0, T ] du processus M (t). De cette manière, la distribution doit prendre en compte ne seulement le space de phase du processus mais aussi son évolution temporelle [START_REF] Vu | Information In The Non-Stationary Case[END_REF]. Le principal inconvénient de ce point de vue est la nécesitée de définir une taille T de fenêtre, limitant l'évolution de la distribution en temps.

Même si la théorie de l'information a été conçu pour analyser des processus stationnaires, quelques travaux ont essayé de réaliser un analyse des processus non-stationnaires avec des outils de théorie de l'information [START_REF] Vu | Information In The Non-Stationary Case[END_REF][START_REF] Ray | On the characterization of non-stationary chaotic systems: Autonomous and non-autonomous cases[END_REF][START_REF] Gómez Herrero | Assessing coupling dynamics from an ensemble of time series[END_REF]. Dans ce chapitre, le but est de développer un cadre de théorie de l'information pour analyser processus nonstationnaires génériques. Ensuite, on développe dans ce cadre le cas spécifique des mouvements auto-similaires non-stationnaires avec des incréments stationnaires, dans lequel on est spécialement intéressés. Les mouvements auto-similaires avec décroissance en loi de puissances sont particulièrement intéresants parce qu'ils modélisent un grand nombre de systèmes physiques, biologiques ou sociaux. Une fois la théorie est développé, on la vérifie numériquement et on l'utilise pour caractériser differents mouvements.

Chapitre 5: Turbulence pleinement developpée

La théorie de Kolmogorov de 1941 (K41) pour la turbulence pleinement developpée, inspirée par la description multiéchelles de Richardson [START_REF] Richardson | Some measurements of atmospheric turbulence[END_REF], a secoué les piliers de la mécanique des fluides. Inspiré par la description de Richardson, Kolmogorov divise dans sa théorie, les échelles de la turbulence en trois domaines: le domaine integral qui comprend les grandes échelles où l'énergie est injectée, le domaine inertiel, constitué par les échelles plus petites que l'échelle integrale L et plus grandes que l'échelle dissipative η K , où l'énergie injectée dans les grandes échelles cascade vers les petites, et finalement le domaine dissipatif formé par les échelles plus petites que l'échelle dissipative et où l'énergie est dissipé. Kolmogorov a postulé que l'anisotropie des grandes échelles est perdue dans le transfert d'energie vers les petites échelles, et il existe donc une gamme d'échelles où la turbulence pleinement developpée a un comportement statistiquement isotrope et homogène. Cette gamme d'échelles définit le domaine inertiel et dissipative.

Kolmogorov identifie les échelles avec les increments de vitesse turbulente, et arrive à caractériser statistiquement la distribution d'énergie [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF][START_REF] Kolmogorov | On degeneration (decay) of isotropic turbulence in an incompressible viscous liquid[END_REF] et la cascade d'énergie à travers les échelles [START_REF] Kolmogorov | Dissipation of energy in locally isotropic turbulence[END_REF]. Pour y arriver, Kolmogorov étudie les fonctions de structure d'ordre deux et trois de la vitesse.

Des résultats expérimentaux postérieurs ainsi que des nouvelles interrogations théoriques ont suggeré le besoin d'ameliorer la théorie de Kolmogorov 1941. Un raffinement a été proposé par lui même et par Oboukhov dans la théorie connu aujourd'hui comme Kolmogorov-Oboukhov 1962 [112, 150], qui introduit le phénomene d'intermittence.

Dans ce chapitre on décrit brièvement les théories de Kolmogorov 1941 et Kolmogorov-Oboukhov 1962. On presente, ensuite la théorie multifractale de la turbulence [START_REF] Frisch | On the singularity structure of fully developed turbulence[END_REF] qui est capable de décrire les modèles de Kolmogorov 1941 et Kolmogorov-Oboukhov 1962 mais aussi des modèles postériurs comme celui de She and Leveque [START_REF] She | Universal scaling laws in fully developed turbulence[END_REF].

Chapitre 6: La théorie K41 du point de vue de la théorie de l'information

On propose dans ce chapitre un point de vue de la turbulence, et en particulier de la théorie K41 (section 5.1), basé sur la théorie de l'information. On décrit d'abord la distribution d'information (dans le sens de Shannon) à travers les échelles de la turbulence, puis on montre et explique la forte connexion avec la distribution d'énergie. On explore aussi l'existence d'une cascade d'énergie dans la turbulence avec la théorie de l'information. Afin d'étudier la turbulence, et retrouver les résultats de la théorie K41 dans le cadre de la théorie de l'information, on analyse des signaux expérimentaux de vitesse obtenus avec differents montages expérimentaux, ainsi qu'un mouvement Brownien fractionnaire, qui est un processus synthétique monofractal proposé par Kolmogorov pour modéliser la turbulence (section 5.2).

Dans un premier temps, on montre comment le taux d'entropie est capable de décrire la distribution d'information à travers les échelles, et on illustre la relation entre le taux d'entropie du processus analysé et l'entropie de ses increments (chapitre 4). Dans un deuxième temps, on designe une procédure de conditionnement du signal, et on montre que ce conditionnement permet une très fine exploration de l'assymétrie de la distribution des increments, ce qui permet d'inférer l'éxistence d'une skewness finie et d'une cascade d'energie. Les méthodes presentées ici sont d'intérêt général dans l'étude de signaux avec dynamiques complexes.

Chapitre 7: L'intermittence revisitée

Comme evoqué dans le chapitre 5, la turbulence présente un phénomène appelé intermitence. Cette intermittence se manifeste comme un comportement non linéaire des exposants des fonctions de structure ζ(p) et donc comme la déformation de la PDF à travers les échelles. Cette déformation de la PDF a souvent été quantifiée par l'evolution de sa flatness [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF], mesurée comme la kurtosis normalisée de la distribution: (δ l v) 4 / (δ l v) 2 2 = S 4 (l)/S 2 (l) 2 . Aux grandes échelles, au voisinage superieur de l'échelle integrale L, la PDF des incréments de vitesse est quasiment Gaussienne et a une flatness très proche de 3 (la valeur attendu pour une Gaussienne). Pour des échelles de plus en plus petites, la PDF est de moins en moins Gaussienne et en conséquence la flatness augmente. Des évolutions fines de la PDF, et donc l'intermitence, ont été étudiées avec la flatness, comme par exemple, l'augmentation rapide de l'intermittence quand l'échelle est réduite en dessous de l'échelle dissipative [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF]. Neanmoins, la flatness décrit la déviation du comportement linéaire de ζ(p) n'en prenant en compte que la fonction de structure d'ordre 4 (S 4 (l)). On propose, dans ce chapitre, une mesure de l'intermittence que implique toutes les fonctions de structure.

Nous proposons mesurer l'intermittence, interpretée comme la déformation de la PDF. Pour cela, on considère la divergence de Kullback-Leibler [START_REF] Kullback | Information Theory and Statistics[END_REF] (KL) (section 1.3.2) entre la PDF et la PDF Gaussienne définie avec la même variance. Comparer la PDF -définie par tous ses moments -et son approximation Gaussienne -définie par le moment d'ordre deux seulement -revient à mesurer l'évolution de tous les moments par rapport à la variance, i.e., à caractériser exhaustivement la déformation de la PDF. Mesurer l'intermittence avec la divergence de KL fournit une généralisation des mesures comme la flatness (p = 4), hyperflatness (p = 6), etc.

Même si l'on étudie ici la turbulence, nos définitions sont très générales et n'ont besoin que d'un signal pour sonder l'intermittence. Notre approche n'a pas besoin ni d'une connaisance a priori, ni d'aucun modèle du système qui a produit la signal. Il peut donc être un outil très puissant pour l'analyse des systèmes complexes qui présentent des comportements en loi de puissances ou des dépendances multiéchelles.

Ce chapitre est organisé comme suit. Dans la section 7.1, on definit notre mesure de l'intermittence basée sur la théorie de l'information, que implique l'entropie de Shannon et une divergence de KL bien choisie. Dans la section 7.2, on calcule cette quantité pour des mesures expérimentales du champ de vitesse Eulerienne provenant de différents montages expérimentaux avec des nombres de Reynolds differents. Enfin, la section 7.3 revient à quelques modèles phénoménologiques afin de mieux comprendre et décrire nos observations. Chapitre 8: Causalité de Wiener à travers les échelles de la turbulence

La caractérisation et la comprehension complètes de la cascade d'énergie décrit par Richardson (chapitre 5) sont loin d'être atteints, et les questions suivantes restent à l'heure actuelle sans reponse: Est-ce que cette cascade est due à la rupture des grands tourbillons en petits? Combien de temps met l'énergie pour aller d'une échelle à une autre? Est-ce que ce temps dépend des échelles? Avec les résultats de recherche actuels la discussion est ouverte, et les conclusions ne sont pas claires. On propose donc dans ce chapitre de creuser ces questions en utilisant la théorie de l'information. Plus précisement, nous proposons d'étudier les interactions entre les échelles de la turbulence avec l'entropie de transfert (section 1.6.2). L'entropie de transfert, definie dans la section 1.6.2, est une mesure de la causalité de Wiener entre deux séries temporelles. Elle a été souvent interpretée comme une mesure de flux d'information (section 1.6). Donc, la question émerge de manière naturelle: est-ce que des flux d'informations entre les échelles de la turbulence existent? Ou: y a-t'il des relations de causalité entre les différentes échelles de la turbulence? Et, si l'on peut les mesurer, aident-ils à la caractérisation de la cascade d'énergie? Afin de sonder l'existence de relation de causalité entre les échelles de la turbulence, nous étudions différentes séries temporelles: d'un coté des mesures expérimentales de vitesse turbulente (presentées dans la section 6.1), de l'autre coté différents mouvements Browniens fractionnaires (section 4.2) avec différents exposants de Hurst, et sans cascade d'énergie entre les échelles.

Les résultats obtenus dans ce chapitre sont non-concluants pour l'instant, car les mesures d'entropie de transfert melangent des informations dus à la physique du système étudie avec des informations dus à la manière de générer les échelles. Cet effet fait difficile l'interpretation des résultats.

Chapitre 9: Application à l'analyse du rythme cardiaque des foetus Parce que c'est important de fournir les obstétriciens avec information significative reliée à l'état de santé du fetus pendant l'accouchement, Le suivi du rythme cardiaque du foetus (fetal heart rate, FHR) pendant l'accouchement est une tâche quotidienne dans les hôpitaux, qui permet aux obstétriciens d'évaluer l'état de santé du foetus. En particulier il permet à un oeil expert d'anticiper l'acidose du foetus qui peut entrainer des conséquences sevères pour le foetus et la mere. Par consequence une decision pertinente et opportune est importante pour faciliter une intervention rapide et un accouchement opérationnel [START_REF] Chandraharan | Prevention of birth asphyxia: responding appropriately to cardiotocograph (CTG) traces[END_REF]. Dans l'activité clinique quotidienne, FHR est majoritairement inspecté visuellement, en suivant les lignes cliniques directrices formalisées par la International Federation of Gynecology and Obstetrics (FIGO) [START_REF] Ayres-De Campos | Figo consensus guidelines on intrapartum fetal monitoring: Cardiotocography[END_REF]194]. Neanmoins, cette inspection visuelle est très variable entre individus, et il y a même une importante variabilité pour un même individu [START_REF] Hruban | Agreement on intrapartum cardiotocogram recordings between expert obstetricians[END_REF]. La dynamique temporelle du FHR est en effet complexe et difficile à évaluer. Les critères FIGO conduisent à une evaluation exigeante, car ils combinent plusieurs aspects de la dynamique FHR: dérive de la baseline, décélérations, accélérations, variabilités de courte et longue durée.

L'analyse FHR automatisée est compliquée à cause de l'éxistence de deux étapes differentes de l'accouchement. L'étape de dilatation (Stage 1) consiste en une dilatation cervicale progressive et des contractions régulières. L'étape de poussée active (Stage 2) est caracterisée par un col de l'uterus completement dilaté et des contractions d'expulsion. Les approches les plus communes en analyse FHR, soit ne distinguent pas ces deux étapes et réalisent une analyse globale [START_REF] Costa | Prediction of neonatal acidemia by computer analysis of fetal heart rate and ST event signals[END_REF][START_REF] Warrick | Classification of normal and hypoxic fetuses from systems modeling of intrapartum cardiotocography[END_REF], ou bien se concentrent sur l'étape de dilatation seulement, puisque elle est plus documentée car plus fréquemment enregistrée et analysée [START_REF] Georgieva | Phase-rectified signal averaging for intrapartum electronic fetal heart rate monitoring is related to acidaemia at birth[END_REF][START_REF] Spilka | Sparse support vector machine for intrapartum fetal heart rate classification[END_REF]. Pour le moment, il n'a pas été étudie si la dynamique temporelle de chaque étape est differente ou pas (a contrario [START_REF] Spilka | Impacts of first and second labour stages on hurst parameter based intrapartum fetal heart rate analysis[END_REF][START_REF] Lim | Quantitative comparison of entropy analysis of fetal heart rate variability related to the different stages of labor[END_REF]). Pourtant, récemment, quelques contributions ont commencé à realiser des comparaisons systématiques [START_REF] Spilka | Fetal heart rate classification: First vs second stage of labor[END_REF][START_REF] Granero-Belinchon | Mutual information for intrapartum fetal heart rate analysis[END_REF].

Cette contribution reste dans la catégorie des travaux cherchant à développer de nouveaux outils pour l'analyse FHR, basés ici sur des concepts avancés de théorie de l'information. Ces outils, qui sont l'auto information mutuelle, l'entropie de Shannon, le taux d'entropie, la "Approximate entropy" et la "Sample entropy", sont appliqués sur une large base de données de grande qualité et bien documentée, rassemblée à travers des années dans un hôpital publique en France. La base de données est divisée en deux ensemble de données associés chacun avec une étape de l'accouchement. Cela permet en second lieu, d'évaluer et comparer la qualité de la détection de l'acidose de chaque outil indépendamment dans chaque étape, et troisièmement, de remarquer les différences de la dynamique temporelle du FHR entre les deux étapes.

via Shannon entropy rate. We studied the relationship between these two methodologies which are able to characterize the distribution of information across scales and the deformation of the probability density function of the increments across scales.

Notation and Terminology

General notation X (m,λ) ≡ process of dimension m and sampling distance λ (distance between consecutive samples).

X (m,λ) i ≡ random variable of X (m,λ) with index i, embedding dimension m and sampling distance λ. x (m,λ) i ≡ possible value of X (m,λ) i . x (m,λ) ≡ possible value of X (m,λ) . if m = 1 :          X (m,λ) ≡ X X (m,λ) i ≡ X i x (m,λ) i ≡ x i x (m,λ) ≡ x
The notation above is completely generic and can be applied to any kind of processes. Consequently, it will be used for general developments. Nevertheless, throughout this dissertation we mainly focus on temporal processes, and then we use the specific notation presented below when analysing them.

Time processes notation

X (m,τ ) ≡ process of dimension m and sampling τ . X (m,τ ) t ≡ random variable of X (m,τ ) at time t, embedding dimension m and sampling time τ .

x (m,τ ) t ≡ possible value of X (m,τ ) t . x (m,τ ) ≡ possible value of X (m,τ ) . if m = 1 :          X (m,τ ) ≡ X X (m,τ ) t ≡ X t x (m,τ ) t ≡ x t x (m,τ ) ≡ x

Introduction

Signal processing is omnipresent in almost any domain of science. From physics to chemistry and throughout engineering, experimental measures and records have to be processed in order to be useful and informative. What do we mean by signals? and by processing? We define a signal as a function bringing information from the properties or from the behaviour of a given phenomenon. So, the temperature of a nuclear reactor, population of a city, the luminosity of a star, a voice record or a photograph, are signals. By processing, we consider the analysis, modification, recording, transport or synthesis of a signal [START_REF] Priemer | Introductory Signal Processing[END_REF]. This processing can have multiple goals, such as the extraction of information from the signal or the reduction of its size, and it can be as basic as the conversion of volts into Kelvin for thermocouple measures, but also as challenging as the analysis of image textures using multifractal analysis and wavelets decomposition.

The correct processing and analysis of the high-quality (not always as high) measures obtained from experimental set-ups, is one of the most important parts of experimental research.

Throughout this thesis, which combines signal processing and physics, we use signal processing to progress in the understanding of physical systems and, at the same time, we use a physical approach to develop signal processing methods.

Objectives

The objectives of this PhD can be summarized as developing methodologies for the characterization of complex systems.

The definition of complex systems still remains vague, and so its frontiers are imprecise [START_REF] Mitchell | Complexity: A guided tour[END_REF][START_REF] Ladyman | What is a complex system?[END_REF]. We understand a complex system as being composed of several parts interacting together in a non-linear way. Most of the time they are multi-scale, i.e. they present a continuum of coupled scales, in which the different scales exhibit different behaviours. This definition of complex systems includes a huge number of systems such as: financial markets [START_REF] Borland | The dynamics of financial markets-mandelbrot's multifractal cascades and beyond[END_REF][START_REF] Bouchaud | Theory of Financial Risk and Derivative Pricing: From Statistical Physics to Risk Management[END_REF][START_REF] Mauboussin | Revisiting market efficiency: the stock market as a complex adaptive system[END_REF], ecosystems [START_REF] Mcgarigal | Multi-scale habitat selection modeling: a review and outlook[END_REF], the brain [START_REF] Siegelmann | Complex systems science and brain dynamics[END_REF], the Earth [START_REF] Donner | Understanding the Earth as a complex system -recent advances in data analysis and modelling Earth sciences[END_REF], colonies of ants [START_REF] Miramontes | Order-disorder transitions in the behavior of ant societies[END_REF], geographycal systems [2] ... A prototype of complex system is turbulence, which is highly non-linear and multi-scale [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF].

Most people when hearing the word "turbulence" think of the violent mouvements shaking an aeroplane, and indeed, these motions are due to turbulence, more precisely to the turbulent behaviour of the air around the plane. When a fluid flow is turbulent, many eddies of different sizes appear. But turbulence is not only present in the air around planes, we can find turbulence in the flows of rivers, the smoke from cigars, the storm clouds and even in breakfast coffee (when we stir the coffee to cool it, we are modifying the state of the fluid "coffee", and more precisely we are inducing turbulence). A turbulent flow is characterized by chaotic trajectories of its fluid particles, and we say that a flow is turbulent when two very close particles of the fluid at a given time t will be arbitrarily far at another time t + ∆t, for a small ∆t. Understanding turbulence is very important for many different fields such as meteorology and climate sciences, aeronautic, naval or vehicle engineering, or, mixing reactions in chemistry. Because of the easy generalization to any complex system of methodologies capable of characterizing turbulence, and also because of the very interesting topic it represents, we use fully developed turbulence as our object of study almost all along the PhD.

In order to accomplish the description of a complex system we are interested in the study of the distribution, the statistical dependences and the (Wiener) causality relationships of experimental measures of different observables of the system. Information theory [START_REF] Shannon | A mathematical theory of communication[END_REF][START_REF] Cover | Elements of Information Theory[END_REF] provides the perfect framework to perform this kind of analysis [START_REF] Prokopenko | An information-theoretic primer on complexity, self-organization, and emergence[END_REF][START_REF] Lizier | An introduction to Transfer entropy: Information flows in complex systems[END_REF].

Framework

Throughout this thesis, we propose to characterize complex systems and in particular turbulence, in terms of Shannon information [START_REF] Shannon | A mathematical theory of communication[END_REF]. We thus take a statistical viewpoint of complex systems. Experimental measures can be used to study the properties and behaviours of systems and phenomena, for example, the changes in temperature of a fluid during a chemical reaction. Once the experimental measures have been obtained, it is necessary to analyse them in order to grasp all the information they contain. We aim to characterize complex systems by analysing experimental measures with information theory tools.

Information theory characterizes a signal or process in terms of complexity, the amount of information needed to characterize the signal, and in terms of dependences, the amount of information shared by different parts of the signal. Furthermore, information theoretic measures are able to grasp non-linear effects, and high order dependences. This together with its applicability to stochastic as well as to deterministic processes [START_REF] Krieger | On entropy and generators of measure-preserving transformations[END_REF][START_REF] Krieger | On unique ergodicity[END_REF][START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF], and the consistency of its interpretation, makes information theory a very interesting approach in the analysis of complex systems [START_REF] Paluš | Multiscale atmospheric dynamics: Cross-frequency phase amplitude coupling in the air temperature[END_REF][START_REF] Vicente | Transfer entropy-a model-free measure of effective connectivity for the neurosciences[END_REF][START_REF] Wibral | Measuring information-transfer delays[END_REF] and more precisely, in the study of turbulence, where information theory has been very rarely used [START_REF] Brown | Information theory and the spectrum of isotropic turbulence[END_REF][START_REF] Ikeda | Information theoretical characterization of turbulence[END_REF][START_REF] Cerbus | Information content of turbulence[END_REF].

A very large number of systems in different domains such as physics, sociology, ecology etc, present multi-scale behaviour [START_REF] Bar-Yam | Multiscale variety in complex systems[END_REF][START_REF] Li | Multi-scale methodology for complex systems[END_REF][START_REF] Bar-Yam | Improving the effectiveness of health care and public health: a multiscale complex systems analysis[END_REF][START_REF] Bar-Yam | Engineering Complex Systems: Multiscale Analysis and Evolutionary Engineering[END_REF][START_REF] Paluš | Multiscale atmospheric dynamics: Cross-frequency phase amplitude coupling in the air temperature[END_REF][START_REF] Mcgarigal | Multi-scale habitat selection modeling: a review and outlook[END_REF]. Therefore, in these kind of systems the analysis at different scales (how the system behaviour evolves with the scale), and the analysis of possible interactions between scales (how the behaviour at a given scale can forecast the behaviour at another scale), become of utmost significance.

For this reason, during the PhD we developed an information theory approach to analyse the complexity and dependences of a process across scales. We also studied the possible (Wiener) causality relationships between different scales.

Contributions

Along the PhD we:

• Developed a multi-scale framework to analyse the distribution and dependences of a process and to characterize it in terms of complexity. We showed the ability of information theoretic measures to grasp high order statistics and the whole dependence structure of the studied processes.

-To illustrate this ability we performed the discrimination between two processes with identical marginal distribution and correlation function but with differences at high order statistics.

-We used this framework to analyse fully developed turbulence and recover Kolmogorov's 1941 theory [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF][START_REF] Kolmogorov | On degeneration (decay) of isotropic turbulence in an incompressible viscous liquid[END_REF][START_REF] Kolmogorov | Dissipation of energy in locally isotropic turbulence[END_REF]. We characterized fully developed turbulence in terms of complexity by studying experimental velocity signals and theoretical models, as well as synthetic processes modelling turbulence. Finally, we found the energy cascade of turbulence by performing a very precise analysis, also based on information theory, of the skewness of the velocity increment distributions.

-We used this framework to analyse fetal heart rate signals during delivery in order to detect acidosis and characterize the dynamics of its development.

• Developed a methodology which allows to analyse the scale invariant properties, mono and multi fractality, of a process across scales. This methodology characterizes the deformation of the distribution of a process across scales. This methodology is applicable to any multi-scale system.

-We applied this methodology to fully developed turbulence and found the multifractal nature of it. We compared the results for a set of experimental turbulent signals at different Reynolds and theoretical models, in order to corroborate the validity of the models.

Structure of this thesis

This thesis is composed of three main parts:

Information theory:

• In chapter 1 we present the main information quantities used during the PhD, we also show their properties and interpretations. We classify them in three different groups: static measures, dynamic measures and causality measures.

• In chapter 2 we enumerate and characterize the main non-parametric algorithms used to compute information theory quantities. Then, we choose the most robust estimators in order to use them along the PhD.

• In chapter 3 we show the ability of information theory to measure dependences and to probe high order statistics of a process. We present some information theory tools as more general versions of correlation function and power spectrum.

• In chapter 4, we generalize information theory to the analysis of non-stationary processes. In order to do that, we develop two different frameworks to study nonstationary processes. The obtained analytical results are supported by numerical computations.

Turbulence:

• Chapter 5 is a summary of the state of the art in three dimensional fully developed turbulence. We present Kolmogorov theories and the multifractal theory of turbulence first introduced by Frisch and Parisi.

• In chapter 6, we study turbulence using information theory and develop a new framework capable to explain Kolmogorov 1941 theory. This chapter is based on the article "Scaling of information in turbulence" published in "Europhysics letters" [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF].

• In chapter 7 we go one step further and develop an information theory framework capable of quantifying multifractality in turbulence across scales by measuring the deformation of the probability density functions with the scale. This framework can be generally applied to characterize the multifractal nature of any multi-scale system. This chapter is based on the article "Kullback Leibler divergence measure of intermittency: Application to turbulence" published in "Physical Review E " [START_REF] Granero-Belinchón | Kullback-leibler divergence measure of intermittency: Application to turbulence[END_REF].

• In chapter 8 we try to measure flows of information between scales in turbulence, understood as Wiener causality relations between the different scales. To do that we analyse interactions between scales on several different systems. The interactions are quantified using transfer entropy and directed information.

Part I Information Theory

CHAPTER 1

Theoretical aspects

Information Theory is a theoretical viewpoint aiming at describing the mathematical laws governing the transmission, storage, processing, measurement and representation of information. Claude E. Shannon developped the proper basis of information theory in 1948, in his article "A Mathematical theory of Communication" [START_REF] Shannon | A mathematical theory of communication[END_REF] and expanded it later together with Weaver in a book [START_REF] Shannon | The Mathematical Theory of Information[END_REF]. As the title of the first article (of Shannon) points out, Information Theory appeared initially as a theory of communication. Nevertheless since the fifties, applications started to generalize to other fields such as dynamical systems [START_REF] Kolmogorov | On the shannon theory of information transmission in the case of continuous signals[END_REF][START_REF] Kolmogorov | New metric invariant of transitive dynamical systems and automorphism of lebesgue spaces[END_REF][START_REF] Sinai | On the notion of entropy of a dynamical system[END_REF][START_REF] Kolmogorov | Three approaches to the quantitative definition of information[END_REF]. This leap from stochastic processes, the natural object of study of information theory, to dynamical processes suppose a very important advance in the development of information theory, allowing its spread to a very wide range of fields. Nowadays Information theory is used in an extraordinary large array of different fields: thermodynamics [START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Van Den Broeck | Stochastic thermodynamics: A brief introduction[END_REF][START_REF] Parrondo | Thermodynamics of information[END_REF], quantum thermodynamics [START_REF] Horowitz | Thermodynamics with continuous information flow[END_REF], neuroscience [START_REF] Vicente | Transfer entropy-a model-free measure of effective connectivity for the neurosciences[END_REF][START_REF] Wibral | Measuring information-transfer delays[END_REF][START_REF] Wollstadt | Efficient transfer entropy analysis of non-stationary neural time series[END_REF], etc. In addition, from the first article of Shannon until now, the number of different information theoretical objects increased significantly. Amongst these theoretical objects we can find: Entropy rate [START_REF] Shannon | The Mathematical Theory of Information[END_REF], Kullback Leibler divergence [START_REF] Kullback | On information and sufficiency[END_REF][START_REF] Kullback | Information Theory and Statistics[END_REF], Transfer Entropy [START_REF] Schreiber | Measuring information transfer[END_REF], Directed Information [START_REF] Kramer | Directed Information for Channels with Feedback[END_REF] ... Recently, access to increasingly powerful computers together with the broader fields of application of Information Theory render it a very interesting research topic both in theoretical aspects and in applied ones.

Here, we present the Information Theory framework used during the PhD for the analysis of stationnary processes, with Shannon entropy as the elementary brick. In addition, as information theory has been proved to be able to describe both stochastic and dynamical systems [START_REF] Krieger | On entropy and generators of measure-preserving transformations[END_REF][START_REF] Krieger | On unique ergodicity[END_REF][START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF], throughout this manuscript information theory is applied over both kinds of processes.

Stochastic and Dynamical processes

Random variables

A measurable space (Ω, B) consists of a sample space Ω together with a σ-algebra B. This σ-algebra B is a collection of subsets of Ω that includes the empty subset and is closed under any countable sequence of set theoretic operations such as union, intersection, complement ... A random variable, f , defined on a measurable space (Ω, B) and taking values on another measurable space (A,

B A ) is a function f : (Ω, B) → (A, B A ) such that: if F ∈ B A , then f -1 (F ) = {ω : f (ω) ∈ F } ∈ B
One then defines a probability space as (A, B A , P f ), where the measurable space (A, B A ) is called the alphabet of the random variable f , and P f is the probability measure, which assigns a real number to every member F of B A . The probability measure should satisfy the following conditions:

• Nonnegativity: P f (F ) ≥ 0 ∀ F ∈ B A . • Normalization: P f (A) = 1 • Countable additivity: If F i ∈ B A , i = 1, 2, • • • are disjoint, then P f (∪ ∞ i=1 F i ) = ∞ i=1 P f (F i )
The given definition of random variable ensures that the output of the random variable inherits its own probability measure, also called the probability density distribution of the random variable [START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF][START_REF] Papoulis | Probability, Random Variables, ans Stochastic Processes[END_REF]:

P f (F ) = P (f -1 (F ))
Two random variables defined on a common probability space are considered to be the same if their distributions are the same [START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF][START_REF] Papoulis | Probability, Random Variables, ans Stochastic Processes[END_REF].

Random processes

A random process X is a family of random variables {X i } i∈I defined on a common probability space (Ω, B, P ), where I is a d-dimensional index set. In general I is only required to be closed under addition. It is assumed that all the random variables X i share a common alphabet A [START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF][START_REF] Papoulis | Probability, Random Variables, ans Stochastic Processes[END_REF].

Dynamical processes

On the other hand, a dynamical system consists of a probability space (Ω, B, P ) together with a measurable transformation T : Ω → Ω [START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF][START_REF] Strogatz | Nonlinear Dynamics and Chaos: with applications to Physics[END_REF]. By measurable, one means that if F ∈ B, then also T -1 F ∈ B. (Ω, B, P, T ) is called a dynamical system in ergodic theory. Roughly speaking, in a dynamical system a function describes the time dependence of a point in a geometrical space.

This definition of a dynamical system leads to an alternative description of random processes. This description, based on dynamical systems, consists on considering a random variable together with a transformation defined on the underlying probability space [START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF]. With inverse reasoning, a chaotic (deterministic) dynamical system defined by a time function f t (x) can be seen as a random process where the input/state (x) of the dynamical system is a random variable.

The instability of chaotic systems with respect to initial conditions implies that time sequences generated by chaotic systems behave, from the point of view of information theory, as authentic stochastic processes [START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF]. This equivalence between time sequences of dynamical systems and time sequences of stochastic processes is shown by Jewett-Krieger theorem [START_REF] Krieger | On entropy and generators of measure-preserving transformations[END_REF][START_REF] Krieger | On unique ergodicity[END_REF][START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF], see section 1.4.1.

Multidimensional random variables: embedding

It is possible to show that collections of random variables,

X I = {X i ; i ∈ I} such as X n = X 1 , X 2 , • • • , X n ,
are also random variables [START_REF] Ash | Real Analysis and Probability[END_REF][START_REF] Gray | Probability, Random Processes, and Ergodic Properties[END_REF]. The measurable space (Ω, B) and the probability measure P of the new constructed random variables X I are different from those of the separate single random variables X i .

In order to properly characterize a stochastic process X taking into account both its static and dynamic properties, we need families or collections of random variables formed by sampling the process. With this purpose one uses a delay-embedding procedure [START_REF] Takens | Dynamical Systems and Turbulence[END_REF] to construct a m dimensional stochastic process X (m,λ) , sampled by λ, from the initial stochastic process X by defining m-dimensional random variables from the 1-dimensional ones:

X (m,λ) i = (X i , X i-λ , X i-2λ , ..., X i-(m-1)λ ) (1.1)
where m is the embedding dimension, λ is the sampling distance between random variables of X, and X (m,λ) i is a random variable of X (m,λ) . This development of embedding is independent of the base dimension d of the initial random variables X i . For a generic process X of dimension d = 1 the total dimension obtained for the embedded process X (m,λ) , is m × d. Throughout this dissertation, for simplicity and because of the perfect equivalence between dimensions d and embedding dimensions m, we fix d = 1 unless noted otherwise. Consequently the total dimension of the embedded process X (m,λ) is given by the embedding dimension m.

Static measures

In this section, we present several information theory objects capable of studying static properties of processes, i.e. properties that take into account neither interactions nor dependences between previous and next samples of the process. In the case of a onedimensional process X, these objects depend on the marginal distribution of X. For ndimensional processes, they depend on the n-dimensional joint distribution of the process.

The embedding procedure (section 1.2) allows the static measures to take also into account the dynamics of a process X: in the case of embedded processes X (m,τ ) , static measures are functions of the m-dimensional joint PDF, and consequently, they take into account interactions between samples inside the m-dimensional embedded vector.

Shannon Entropy

Discrete framework

Information theory started with the definition of Shannon Entropy [START_REF] Shannon | A mathematical theory of communication[END_REF]. The Shannon entropy of a discrete process X is defined as the uncertainty that one has of the process X i.e. the amount of information that characterizes the process. It is a function of the probability density function (pdf) of the process:

H(X) = - x∈A p X (x) log(p X (x)) (1.2)
where the sum is over all the possible values x of X and p X is the probability distribution of X.

Information and thus Shannon entropy should satisfy some important properties [START_REF] Shannon | A mathematical theory of communication[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF]:

• The entropy should be continuous in the probabilities: small changes in the PDF should imply small changes in the entropy.

• For a given number n of possible events defining the alphabet A, the entropy is a maximum and equal to log(n) when all the p X (x) are equal (p

X (x) = 1/n ∀ x ∈ A)
. This is the most uncertain situation. With equally likely events there is more uncertainty when there are more possible events n.

• Additivity: The entropy of a system Y can be expressed in function of the entropies of its subsystems X if the relations between the system and the subsystems and between the subsystems themselves are known. We define a system Y = {Y i } as:

Y 1 = {X 1 , • • • , X k 1 } Y 2 = {X k 1 +1 , • • • , X k 1 +k 2 } . . . Y m = {X n-km+1 , • • • , X n }
with the probability of Y i as:

p Y i = k 1 +•••+k i -1 l=k 1 +•••+k (i-1) p X l
where p X l is the probability of X l . We then define the entropy of X as:

H(p X 1 , • • • , p Xn ) = H(p Y 1 , • • • , p Ym ) + m i=1 p Y i H p X k 1 +•••+k (i-1) p Y i , • • • , p X k 1 +•••+k i -1 p Y i
• Concavity: Shannon entropy is concave in the probability density function.

H(ap

X 1 + (1 -a)p X 2 ) ≥ aH(p X 1 ) + (1 -a)H(p X 2 )
for any probability density functions and 0 ≤ a ≤ 1.

• The entropy is positive and vanishes if and only if all the p X (x) but one are zero, thus when there is no uncertainty.

• Given two different processes X and Y , let p X,Y (x, y) be the probability of the joint occurrence of x ∈ A X and y ∈ A Y . The joint entropy is the uncertainty of the joint occurrence:

H(X, Y ) = H(Y, X) = - x∈A X ,y∈A Y p X,Y (x, y) log(p X,Y (x, y)). (1.
3)

It should respect: H(X, Y ) ≤ H(X) + H(Y ) (1.4)
where the equality is obtained if and only if the two processes are independent.

• Given two processes X and Y , for any particular value x that X can take with probability p X (x) > 0, we can define the conditional probability, p Y |X (y|x), that Y has the value y:

p Y |X (y|x) = p X,Y (x, y) p X (x) .
We can define the conditional entropy of Y , H(Y |X) as the average of the entropy of Y for each value of X weighted according to the probability of getting that particular value of X:

H(Y |X) = - x∈A X ,y∈A Y p X,Y (x, y) log(p Y |X (y|x)). (1.5)
The conditional entropy measures how uncertain we are of Y when X is known. It satisfies:

H(Y |X) = H(X, Y ) -H(X). (1.6)
From eq.(1.4) and eq.(1.6) we see that:

H(Y ) ≥ H(Y |X)
which can be intuitively inferred by noting that knowing X can only help in the characterization of Y . In the worst-case scenario, if X and Y are independent, knowing X will not alter how uncertain we are of Y , and then

H(Y |X) = H(Y ).

Continuum framework

Generalizing the information measure to continuous processes is not straightforward. Differential entropy defined by Shannon [START_REF] Shannon | The Mathematical Theory of Information[END_REF] as:

H(X) = - A p X (x) log(p X (x))dx (1.7)
is not invariant under change of variables and can be negative. In addition, differential entropy is maximal for the multivariate Gaussian distribution instead of the uniform distribution. Thus differential entropy is not a good measure of information [START_REF] Jaynes | Information theory and statistical mechanics[END_REF]. Instead of the straightforward definition in eq.(1.7) intuited by Shannon, the correct measure of information in the continuous case is [START_REF] Jaynes | Information theory and statistical mechanics[END_REF]:

H(X) = - A p X (x) log p X (x) m(x) dx (1.8)
where m(x) is the measure function defined by: b a m(x)dx = lim Ne→∞ (# of events in a < x < b) N e (1.9)

In eq.(1.9) N e is the number of discrete events in the whole domain of A, and a and b define an infinitely small segment of A.

In the following we use the discrete formalism unless said otherwise.

Interpretation of Shannon entropy and relation with thermodynamical entropy

• Shannon entropy, as presented above, is a measure of the total uncertainty that one has of a process. Shannon entropy measures the amount of information that completely characterizes a processs . The more complex the process the larger the Shannon entropy.

• Thermodynamical entropy, was defined by Clausius [START_REF] Carnot | Reflexions sur la puissance motrice du feu et sur les machines propres a developper cette puissance[END_REF][START_REF] Clausius | Ueber eine veranderte form des zweiten haupsatzes der mechanischen wamertheoriein[END_REF] as:

∆S C (1 → 2) = 2 1 dQ T (1.10)
where T is the temperature of the system and dQ is the heat.

In his article, "Gibbs vs Boltzmann entropies" [START_REF] Jaynes | Gibbs vs Boltzmann entropies[END_REF], Jaynes showed that Gibbs entropy defined as:

H G = k B Ω W N log(W N )dΩ, (1.11)
where k B is the Boltzmann constant and W N is the N -particle distribution function, which gives the probability density in the full phase space Ω of the system, verifies in the case of thermal equilibrium (canonical distribution) and reversible path 1 → 2:

(H G ) 2 -(H G ) 1 = ∆H G (1 → 2) = 2 1 dQ T .
(1.12)

The fact that Shannon entropy and Gibbs entropy share the same mathematical expression, does not in itself establish any connection between this two entropies.

• Maximum-Entropy Principle

In order to properly relate Shannon entropy to thermodynamic entropy, Jaynes formulated the maximum-entropy principle [START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Jaynes | Information theory and statistical mechanics[END_REF]. Jaynes built maximum-entropy principle in the framework of subjectivist or Bayesian statistics, which understands probabilities as expressions of human ignorance. Thus, the probability of an event is a formal expression of our expectation that the event occur, based on the available information. The problem that Jaynes formulated was to find "... a probability assignment which avoids bias, while agreeing with whatever information is given" [START_REF] Jaynes | Information theory and statistical mechanics[END_REF]. Information theory provides a measure of the amount of uncertainty characterizing a discrete probability distribution. The only unbiased statement one can conclude with a given amount of information, is that one must use the probability distribution which maximizes the entropy subject to whatever is known. This is known as the maximum-entropy principle.

Then Jaynes interpreted the probability density in the full phase space of the system at time zero, W N (t = 0), and consequently its Gibbs entropy as measuring "our degree of ignorance as to the true unknown microstate, when the only information we have consists of the macroscopic thermodynamic parameters" [START_REF] Jaynes | Gibbs vs Boltzmann entropies[END_REF].

Kullback-Leibler Divergence

Kullback-Leibler divergence, also called Relative entropy or Kullback-Leibler distance, is defined as the measure of the distinguishibility between two probability density functions, p X and q X , of the same process X. Kullback-Leibler divergence measures the inefficiency of assuming probability density function q X when the true one is p X [START_REF] Kullback | Information Theory and Statistics[END_REF][START_REF] Cover | Elements of Information Theory[END_REF]:

K p X ||q X = x∈A p X (x) log p X (x) q X (x) (1.13)
Kullback-Leibler divergence measures how "similar" the distribution q X is to the distribution p X . Some Kullback-Leibler divergence properties are:

• It's always non-negative and only zero for p X = q X . • It's not a distance, since it is not symmetric:

K p X ||q X = K q X ||p X (1.14)
• It remains well-defined for continuous distributions:

K p X ||q X = +∞ -∞ p X (x) log p X (x) q X (x) dx (1.15)
Chernoff-Stein's lemma gives an easy interpretation of Kullback-Leibler divergence in terms of "surprise" [START_REF] Chodrow | Divergence, entropy, information: An opinionated introduction to information theory[END_REF][START_REF] Cover | Elements of Information Theory[END_REF]. After n realizations of X, with X distributed with p X (x), the probability to mistake pX,n (x) (observed distribution after n realizations) for q X (x) decays exponentially with K p X ||q X :

P (q X (x)|p X,n (x)) ≈ e -nK p X ||q X
(1.16)

Mutual Information

A specific but famous case of Kullback-Leibler divergence is mutual information. Given two different processes X and Y , mutual information measures the shared (common) information between both processes, i.e. the "distance" between independence and dependence of the processes. It is defined as the Kullback-Leibler divergence between the joint probability of both processes p X,Y (x, y) (general case) and the product of the marginals p X (x)p Y (y) (thus assuming the independence of X and Y ) [START_REF] Shannon | The Mathematical Theory of Information[END_REF]:

I(X, Y ) = x∈A X ,y∈A Y p X,Y (x, y) log p X,Y (x, y) p X (x)p Y (y) (1.17)
Mutual Information can be expressed in function of Shannon entropies:

I(X, Y ) = H(X) + H(Y ) -H(X, Y ) (1.18)
From eq.(1.18) it is easy to verify the symmetry of mutual information, I(X, Y ) = I(Y, X).

As for Shannon entropy it is possible to define a conditional version of Mutual information [START_REF] Wyner | A definition of conditional mutual information for arbitrary ensembles[END_REF]:

I(X, Y |Z) = z∈A Z p Z (z) x∈A X ,y∈A Y p X,Y |Z (x, y|z) log p X,Y |Z (x, y|z) p X|Z (x|z)p Y |Z (y|z) (1.19)
It can also be expressed in function of conditional entropies:

I(X, Y |Z) = H(X|Z) + H(Y |Z) -H(X, Y |Z) (1.20)
Conditional Mutual information in eq. (1.19) measures the shared information between X and Y that is not contained in Z.

Dynamic measures

Dynamic measures are able to characterize interactions and dependences between previous and next samples of a process X, and then, connections between the different samples of the process.

Entropy Rate

In order to measure how the entropy of a sequence of m random variables grows with m, Shannon presented the entropy rate [START_REF] Shannon | The Mathematical Theory of Information[END_REF]. Entropy rate of a process X = {X i } i∈I is defined as [START_REF] Cover | Elements of Information Theory[END_REF]:

h(X) = lim m→∞ 1 m H(X 1 , X 2 , X 3 , • • • , X m ) (1.21)
Entropy rate defined in eq.(1.21) is the entropy per symbol of the m-dimensional random variable characterizing the process. Another possible, definition of entropy rate is [START_REF] Cover | Elements of Information Theory[END_REF]:

h (X) = lim m→∞ H(X m+1 |X m , X m-1 , X m-2 , • • • , X 1 ) (1.22)
In this case (eq.(1.22)) entropy rate is defined as the entropy of the m + 1th random variable, knowing the previous ones, X i ∀ i ∈ [1 : m]. For a stationary process, these two definitions of entropy rate (eq.(1.21) and eq.(1.22)) are equivalent, both limits exist and are equal [START_REF] Cover | Elements of Information Theory[END_REF].

Entropy rate probes the dynamical evolution of the process. It measures the effect of the previous samples of the process on the next one (eq.(1.22)), or the contribution of each one-dimensional random variable X i to the total entropy of the process characterized by a m-dimensional random variable, eq. ( 1.21).

m-order entropy rate m-order entropy rate is an approximation of entropy rate from eq. (1.22), where m doesn't go to infinity but takes a finite value m, indicating the dimension of the analysed random variables (section 1.2) and hence the order of the entropy rate [START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF]:

h (m) (X) = H(X m+1 |X m , X m-1 , X m-2 , • • • , X 1 ) (1.23) = H(X m+1 , X m , • • • , X 1 ) -H(X m , X m-1 , • • • , X 1 )
Due to the impracticability of computing entropy rate as defined in eq.(1.21) or eq. (1.22), the approximation of entropy rate to its mth order (eq.(1.23)) has been extensively used [START_REF] Ebeling | Entropy and long-range correlations in literary english[END_REF][START_REF] Herzel | Entropies of biosequences: The role of repeats[END_REF][START_REF] Bollt | Validity of thresholdcrossing analysis of symbolic dynamics from chaotic time series[END_REF][START_REF] Lesne | Entropy estimation of very short symbolic sequences[END_REF]. In definitions (1.21), (1.22) and (1.23) of entropy rate the distance between consecutive samples of process X is fixed to λ = 1. In order to generalize the definition of m-order entropy rate the use of embedding notation presented in section 1.2 can be useful:

h (m,λ) (X) = H(X i+λ |X (m,λ) i ) = H(X (m+1,λ) ) -H(X (m,λ) ) (1.24)
This abreviated notation allows to easily indicate variations on the index i and the sampling distance λ. m-order entropy rate can be understood as a kind of derivative of the entropy, measuring how much the entropy varies between two consecutive dimensions m and m + 1. It can be seen also as a measure of the information of X i+λ not contained in the m previous random variables X (m,λ) i

, or, the new information about the process X brought by

X i+λ if X (m,λ) i is known.
A very important characteristic of entropy rate is that it can be applied in the analysis of data from both deterministic and stochastic sources [START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF].

Kolmogorov-Sinai Entropy and relation with entropy rate

In the 1960s, Kolmogorov and Sinai adapted Shannon's information theory to the study of dynamical systems. The divergence of trajectories starting from different, but undistinguishable initial conditions can be pictured as creating uncertainty, so creating information. Metric entropy, also known as the Kolmogorov-Sinai entropy h KS (ρ), measures the mean rate of creation of information by a dynamical system with ergodic probability measure ρ. Metric entropy is constructed exactly as the Shannon entropy rate from information theory, but using the density ρ of trajectories in phase space instead of the probability density p. Metric entropy is related to Shannon entropy rate by h KS = h if Shannon entropy rate is defined using natural logarithm [START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF]. A very important result relating dynamical system and stochastic processes is Jewett-Krieger theorem [START_REF] Krieger | On entropy and generators of measure-preserving transformations[END_REF][START_REF] Krieger | On unique ergodicity[END_REF]: A continuous valued deterministic system in discrete time and with finite entropy is equivalent to a stochastic process with a finite number of states. The minimum number of states, m s , required for the theorem to be valid, is given by e h KS < m s < 1 + e h KS .

The exact analogy between the Shannon entropy rate and the metric entropy supports the validation and meaning of Information Theory analysis over dynamical systems [START_REF] Falcioni | Kolmogorov's Legacy about Entropy, Chaos, and Complexity[END_REF][START_REF] Lesne | Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics[END_REF].

Auto-Mutual Information

Another way to express Shannon m-order entropy rate is using mutual information between two consecutive chunks of a process X i+λ and X (m,λ) i :

h (m,λ) (X) = H(X) -I(X i+λ , X (m,λ) i ) (1.25)
where H(X) is the total Shannon entropy of the process X.

From this expression of Shannon entropy rate, we can separate two different contributions: H(X), which only depends on the one-point distribution of X and is hence a static property; Conversely, I (m,1,λ) (X) gathers all information conveyed by linear and non linear dynamics, irrespective of the variance of X, but dependent on the shape of its distribution [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF]. We define auto-mutual information [START_REF] Cover | Elements of Information Theory[END_REF][START_REF] Gomez | Reduced predictable information in brain signals in autism spectrum disorder[END_REF][START_REF] Faes | Information decomposition in bivariate systems: Theory and application to cardiorespiratory dynamics[END_REF][START_REF] Xiong | Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations[END_REF], also called information storage, as the information shared between X i+λ and X (m,λ) i , i.e. the mutual information between them, I(X i+λ , X (m,λ) i

). Information storage can be understood as the part of information contained in sample i + λ of process X (X i+λ ) that can be predicted by the knowledge of previous samples (X (m,λ) i

). Information storage definition can be generalized to "the mutual information between X (p,λ) i+pλ and X (m,λ) i " (figure 1.1):

I(X (p,λ) i+pλ , X (m,λ) i ) = H(X (p,λ) i+pλ ) + H(X (m,λ) i ) -H(X (m+p,λ) i+pλ ) (1.26)
In order to simplify the expression of auto mutual information we denote I(X

(p,λ) i+pλ , X (m,λ) i ) = I (m,p,λ) (X)

Auto-mutual information Symmetry

Auto-mutual information of stationnary processes is symmetrical with respect to the embedding dimension:

I (m,p,λ) (X) = I (p,m,λ) (X) (1.27)
Proof. If we develop the MI expression in function of the entropies, for both cases, left and right sides of the above equation:

I(X (p,λ) i+pλ , X (m,λ) i ) = H(X (p,λ) i+pλ ) + H( (m,λ) i ) -H(X (p,λ) i+pλ , X (m,λ) i )
As we work with the same process X delayed:

H(X (p,λ) i+pλ , X (m,λ) i ) = H(X (p+m,λ) i+pλ ) (1.28)
and invoking stationarity:

H(X (p,λ) i+pλ ) = H(X (p,λ) ) H(X (m,λ) i ) = H(X (m,λ) ) H(X (p+m,λ) i+pλ ) = H(X (p+m,λ) )

Information dynamics across scales

In order to analyse the dynamics across scales of a 1-dimensional process X, we build m-embedded processes X (m,λ) , where λ indicates the scale at which the process is studied (section 1.2). Then, we estimate the entropy rate (section 1.4.1) and the auto mutual information (section 1.4.2) for different λ values.

The scale λ of the statistics:

At the scale λ, consecutive elements of the m-dimensional vectors are samples of the initial process X with a distance λ between them, eq.(1.1). Consequently we focus on the dynamics of the process at this scale λ. When λ → 0, so at the small scales, the dependences between two consecutive coordinates of the embedded vector increase. Conversely, when λ → +∞, then at the large scales of the process, all dependences disappear.

Of course the entropy of X (m,λ) (H(X (m,λ) )) and then, the entropy rate (h (m,λ) (X)) and auto mutual information (I (m,1,λ) (X)) probe the dynamics of X at this scale and behave accordingly to the dependences. When the dependences are null H(X (m,λ) ) = mH(X), I (m,1,λ) (X) = 0 and h (m,λ) (X) = H(X). However when the dependences increase, H(X (m,λ) ) < mH(X), I (m,1,λ) (X) increases and h (m,λ) (X) decreases.

I (m,1,λ) (X) measures the shared information between current m-points dynamics and future 1-point dynamics at the given scale λ.

The order m of the statistics:

The Shannon entropy of a 1-dimensional stationnary process X involves arbitrarily high order moments of the PDF, and therefore high order statistics, although it importantly does not depend on the first moment of the PDF and one can work with centered processes. In the case of no embedding, there is no dependence of the entropy on λ, so we need at least two samples of the 1-dimensional process to define a scale λ.

Theoretically, the embedding dimension m indicates the number of points directly involved in the statistics, so, for embedding dimension equals m, m-points statistics will be directly caught by IT tools, even if all IT tools work over the complete PDF, and then indirectly they work always on all order statistics. In this way, for the m-embedded version of the stationnary process X (m,λ) , the Shannon entropy focuses on the m-order statistics, and depends on the scale parameter λ.

The entropy rate of order m and the auto mutual information of order m, as the entropy, probe statistics of any order, but eq.(1.24) and eq. (1.25) suggest that for a given m, they focus on the extra information contained in order m + 1 statistics. Therefore, when the embedding dimension m increases, higher and higher order statistics are taken into account. As the Shannon entropy of the m-embedded process X (m,λ) , both Shannon entropy rate and auto mutual information depend on the scale λ of the embedding procedure.

The Gaussian and log-normal processes presented in section 1.7 will be used as illustration.

Causality measures

In this section we present three measures capable of quantifying causal relations between processes. The second and the third measures are based on Wiener causality [START_REF] Wiener | Modern Mathematics for engineers, volume 1, chapter 8 The theory of Prediction[END_REF][START_REF] Granger | Investigating causal relations by econometric models and crossspectral mehods[END_REF][START_REF] Geweke | Measurement of linear dependence and feedback between multiple time series[END_REF][START_REF] Rissanen | Measures of mutual and causal dependence between two time series[END_REF][START_REF] Bressler | Wiener-Granger causality: A well established methodology[END_REF][START_REF] Amblard | The relation between Granger causality and directed information theory: A review[END_REF][START_REF] Lizier | An introduction to Transfer entropy: Information flows in complex systems[END_REF], i.e. a process X causes another process Y if the prediction of Y i using the previous samples of both processes X and Y is more succesful than the prediction of Y i using only previous samples of Y . The first measure that we present is more basic and causality is here understood in a simpler way: a process X causes another process Y if Y i and previous samples of X share some information.

Delayed Mutual Information

As we show in section 1.3.3, Mutual information measures the shared information between two processes X and Y . By adding a lag δ, MI can measure the directional exchanged information from X to Y and vice versa. Delayed mutual information from X to Y is defined as:

I(X → Y ) δ = I(X i , Y i+λ ) (1.29)
We can define equivalently, the delayed mutual information from Y to X:

I(Y → X) δ = I(X i+λ , Y i ) (1.30)
Unfortunately delayed mutual information presents at least three main flaws [START_REF] Schreiber | Measuring information transfer[END_REF][START_REF] Paluš | Synchronization as adjustement of information rates: Detection from bivariate time series[END_REF]:

1 Common history of both processes can influence the measure.

2 Common input for both processes can influence the measure. Even if neither X causes Y nor Y causes X some causality relationships can be obtained between X and Y , if a third process Z causes both of them.

3 Exchanged information measured in one direction can be a reminiscent residue of exchanged information in the other direction. A positive measure of I(X → Y ) δ can be due to a residue of the Wiener causality from Y to X.

Transfer Entropy

In order to fix flaws 1 and 3, Schreiber developed Transfer Entropy (TE) [START_REF] Schreiber | Measuring information transfer[END_REF][START_REF] Paluš | Synchronization as adjustement of information rates: Detection from bivariate time series[END_REF][START_REF] Vicente | Transfer entropy-a model-free measure of effective connectivity for the neurosciences[END_REF]:

T E(X (p,λ) → Y (m,λ) ) δ = H(Y i |Y (m,λ) i-δ ) -H(Y i |Y (m,λ) i-δ , X (p,λ) i-δ ) (1.31) = M I(Y i , X (p,λ) i-δ |Y (m,λ) i-δ ) (1.32)
Transfer Entropy measures the causal relations, in Wiener-Granger terms, that X has over Y at a distance equals δ. It can be seen as the information contained in

Y i which is not contained in Y (m,λ) i-δ but in X (p,λ) i-δ .
Transfer entropy can be expressed as a conditional mutual information, see eq. (1.32). In order to give a dynamical meaning to CMI, the conditioning is done on the past of the target process, e.g. in eq. (1.32) on the past of Y .

As we can see in eq.(1.31), conditioning only by previous samples of Y in the first conditional entropy and previous samples of X and Y in the second conditional entropy makes Transfer Entropy not symmetrical, T E(X (p,λ) → Y (m,λ) ) δ = T E(Y (m,λ) → X (p,λ) ) δ . This non-symmetry allows to measure directionality and dynamics. In order for Transfer Entropy to be causal, δ must be larger than zero, in this way it does not take into account instantaneous interactions.

Later Wibral et al. state that, in order to be Granger-Wiener causal, another definition of TE is needed [START_REF] Wibral | Measuring information-transfer delays[END_REF][START_REF] Lizier | An introduction to Transfer entropy: Information flows in complex systems[END_REF]:

T E(X (p,λ) → Y (m,λ) ) δ = H(Y i |Y (m,λ) i-1 ) -H(Y i |Y (m,λ) i-1 , X (p,λ) i-δ ) (1.33) = M I(Y i , X (p,λ) i-δ |Y (m,λ) i-1 )
In this definition the knowledge of previous samples of Y come until the ith sample, and the distance δ only remains in the causation process X.

Directed Information

Another measure of influence of a system X to another one Y is Directed Information [START_REF] Kramer | Directed Information for Channels with Feedback[END_REF] (DI):

DI(X → Y ) M = M m=1 H(Y m |Y (m-1,λ) m-λ ) -H(Y m |Y (m-1,λ) m-λ , X (m,λ) ) (1.34) = M m=1 M I(X (m,λ) , Y m |Y (m-1,λ) m-λ ) (1.35)
where M is called the order and indicates the maximum embedding dimension of the measure.

We can compare Transfer Entropy to Directed Information, finding that the structure of the measures is very similar (compare eq.(1.31) to eq.(1.34)), both of them can be expressed as a conditional mutual information (see eq.(1.32) and eq. (1.35)). But contrary to Transfer entropy, Directed Information takes into account instantaneous interactions. These instantaneous interactions measured by DI are symmetric under directionality, i.e. are identical in both directions.

We present transfer entropy and directed information as measures of Wiener causality. Initially, both transfer entropy and directed information were presented as measures of directional information flows between two systems. Since their definitions they have been often used as measures of exchanged information, or measures of sent/received information. Some recent works have questioned this interpretation, pointing out some possible misconceptions and some possible problems in TE and DI [START_REF] Williams | Nonnegative decomposition of multivariate information[END_REF][START_REF] Williams | Generalized Measures of Information Transfer[END_REF][START_REF] James | Information flows? a critique of transfer entropies[END_REF][START_REF] Chicharro | Synergy and redundancy in dual decompositions of mutual information gain and information loss[END_REF]]. The most interesting example is the possibility of having a third system Z influencing both X and Y . A conditioning, not only on the past of X and the past of Y , but also on the rest of systems interacting, would solve, or at least reduce (It's imposible to take into account all the possible sources of influence), the problem. Some other objections, such as the possibility of adding synergetic information with the conditioning procedure (the conditioning on two, or more variables may provide information that is not available from either alone), or the possibility of being measuring influences not related to information exchanges, have to be studied. The interpretation of TE and DI as measures of influence and causality seems well accepted, and we adopt this point of view. Nevertheless we let the door open to other possible quantitities to measure information flows.

Specific distributions

In this section we express the Shannon entropy, the entropy rate and the auto mutual information for some specific distributions.

Gaussian distribution

If X is a stationary Gaussian process, hence fully defined by its variance σ 2 and normalized correlation function c(λ), its entropy is [START_REF] Zografos | Expressions for renyi and shannon entropies for multivariate distributions[END_REF]:

H(X (p,λ) ) = p 2 log 2πeσ 2 + 1 2 log |Σ (p) | (1.36)
where Σ (m) is the m × m correlation matrix of the process X; Σ i,j = c(|i -j|λ) and

|Σ (1) | = 1.
Using eq.(1.36) we can obtain the information storage of X:

I(X (p,λ) i+pλ , X (m,λ) i ) = 1 2 log |Σ (m) ||Σ (p) | |Σ (m+p) | (1.37)
For the particular case p = 1 we have:

H(X) = 1 2 log 2πeσ 2 (1.38) I(X i+λ , X (m,λ) i ) = 1 2 log |Σ (m) | |Σ (m+1) | (1.39)
so we directly obtain the m-order entropy rate of a Gaussian process:

h (m,λ) (X) = 1 2 log 2πeσ 2 - 1 2 log |Σ (m) | |Σ (m+1) | (1.40)
which clearly illustrates the decomposition of the entropy rate according to eq.(1.25): the first term H(X) depends only on the static (one-point) statistics (via σ 2 ), and the second term I(X i+λ , X (m,λ) i

) depends on the temporal dynamics (and in this simple case, only on the dynamics, via the auto-correlation function c(λ)).

Other distributions

Given a bijective transformation F that maps the Gaussian signal X into a signal Y = F (X), formulae (1.36,1.37) can be rewritten for the signal Y :

H(Y ) = H(X) + ln F X = 1 2 ln(2πeσ 2 x ) + ln F X (1.41) I (m,p,τ ) (Y ) = I (m,p,τ ) (X) = 1 2 ln |Σ (m) ||Σ (p) | |Σ (m+p) | (1.42)
where ln F X is the average of ln F (x) using the probability density of X, and hence is a constant, σ 2

x is the variance of the Gaussian variable X, and Σ (m) is the m-dimensional covariance square matrix of X. As previously observed in [START_REF] Kraskov | Estimating mutual information[END_REF], the MI is invariant under re-parametrization of its arguments. So, knowing the underlying Gaussian process X allows to estimate the theoretical entropy, auto-mutual information and then entropy rate of process Y .

Example: Log-normal distribution

If we consider as a particular case a stationary log-normal process Y , defined as Y = F (X) = e X with X Gaussian of mean µ and standard deviation σ, its entropy is [START_REF] Zografos | Expressions for renyi and shannon entropies for multivariate distributions[END_REF]:

H(Y (p,λ) ) = p 2 log 2πeσ 2 + 1 2 log |Σ (p) | + pµ (1.43)
where Σ (m) is the m × m correlation matrix of the process X; Σ i,j = c(|i -j|λ) and |Σ (1) | = 1, and µ is the mean of the process X.

Using eq.(1.43) we can obtain the information storage of Y :

I(Y (p,λ) i+pλ , Y (m,λ) i ) = 1 2 log |Σ (m) ||Σ (p) | |Σ (m+p) | (1.44)
For the particular case p = 1 of eq.(1.43) and eq.(1.44), we have:

H(Y ) = 1 2 log 2πeσ 2 + µ (1.45)
I(Y i+λ , Y (m,λ) i ) = 1 2 log |Σ (m) | |Σ (m+1) | (1.46)
The m-order entropy rate of a log-normal process is:

h (m,λ) (Y ) = 1 2 log 2πeσ 2 - 1 2 log |Σ (m) | |Σ (m+1) | + µ (1.47) CHAPTER 2
Non-parametric estimators Along this chapter we use previous literature and new original results to choose the most reliable estimates of Information theory measures. All the already known conclusions are supported with their respectives original papers. The new results obtained during the PhD, are properly explained (and related with their corresponding article). The chapter is organized as follows: the first section enumerates different algorithms to estimate PDF's, the second one presents the Kozachenko-Leonenko algorithm to compute Shannon entropy [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF][START_REF] Singh | Nearest neighbor estimates of entropy[END_REF][START_REF] Leonenko | A class of rnyi information estimators for multidimensional densities[END_REF], the third section is dedicated to Kraskov-Stögbauer-Grassberger estimate of Mutual Information [START_REF] Kraskov | Estimating mutual information[END_REF], the fourth section presents a new method to calculate the m-order Shannon entropy rate [START_REF] Granero-Belinchón | Un estimateur du taux d'entropie basé sur l'information mutuelle[END_REF] (based on those of Kozachenko-Leonenko and Kraskov-Stögbauer-Grassberger), the fifth section explains how to use Theiler prescription to probe dynamics and complexity at a given scale λ, and finally the last section presents the Kraskov-Stögbauer-Grassberger estimate for conditional mutual information.

Probability Density Function Estimates

Historically, the first entropy estimates were based on histogram estimation of probability density functions [START_REF] Scott | Kernel density estimation revisited[END_REF][START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF]. These estimates present two free parameters: the origin, and the size of the bins. Changes in both of them, especially the size of the bins, affect seriously the estimation of the PDF [START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF]. The bias of these algorithms is significant and renders the measure difficult to interpret [START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF]. In order to enhance estimations several variations of histograms alorithms were developed [START_REF] Scott | Averaged shifted histograms: Effective nonparametric density estimators in several dimensions[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF], which "eliminate" the origin as a parameter of the estimate and then "eliminate" this source of bias. Nevertheless, using bins still generates a high bias of the measure [START_REF] Scott | Kernel density estimation revisited[END_REF][START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF].

A very important improvement is achieved when the shape of the bin is made smoother. We call this a kernel estimate [START_REF] Rosenblatt | Remarks on some nonparametric estimates of a density function[END_REF][START_REF] Parzen | On estimation of a probability density function and mode[END_REF][START_REF] Cacoullos | Estimation of a multivariate density[END_REF]. The function defining the shape of the bins must obey some properties which make the estimation well behaving and reduce its bias [START_REF] Scott | Kernel density estimation revisited[END_REF][START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF]. However, as the size of the kernels is still fixed a bias remains [START_REF] Scott | Kernel density estimation revisited[END_REF][START_REF] Tarter | An introduction to the implementation and theory of nonparametric density estimation[END_REF] specially for long tailed PDFs [START_REF] Silverman | Monographs on statistics and applied probability[END_REF]. Various improvements on kernel estimations have been done along the last fifty years [START_REF] Terrel | On improving convergence rates for nonnegative kernel density estimator[END_REF][START_REF] Abramson | On bandwidth variation in kernel estimates-a square root law[END_REF][START_REF] Terrel | Variable kernel density estimation[END_REF].

Nearest Neighbor Search algorithms were then developed to deal with constant-length problems of kernels [START_REF] Loftsgaarden | A nonparametric estimate of a multivariate density function[END_REF]. The idea is to vary the size of the kernel according to the density of samples in the area [START_REF] Loftsgaarden | A nonparametric estimate of a multivariate density function[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF]. Bias due to fixed size of the kernels disappears but we recover the bias due to their "square" shape [START_REF] Mack | Multivariate k-nearest neighbor density estimates[END_REF][START_REF] Moore | Consistency properties of nearest neighbor density function estimators[END_REF]. Some mixtures of kernel and NNS estimates of PDF have been later developed [START_REF] Breiman | variable kernel estimates of multivariate densities[END_REF], improving the estimation properties. Finally, Walter and Blum, and, Terrel and Scott showed that many different nonparametric PDF estimators, among them NNS, can be understood as kernel estimators [START_REF] Walter | Probability density estimation using delta sequences[END_REF][START_REF] Terrel | Variable kernel density estimation[END_REF].

Once the PDF has been estimated, it should be inserted in the specific formula of the corresponding information theory measure. Calculating the PDF to later compute the information theory measure presents an important bias. For this reason, algorithms performing straightforward estimations of information theory objects are usually better than those that first estimate the PDF. These "direct" algorithms are normally based on the same strategies: histograms, KDE or NNS, and they search for bias reductions or compensations.

Histogram algorithms

Histogram algorithms base the estimation of the probability density function of a random variable on a simple box counting method. The domain of possible values that the random variable can take is divided in boxes of a chosen size with a given origin, and then the number of times the random variable X i takes a value in each box is counted. Given an origin x 0 and a bin width ∆x we define the bins as [x 0 +m∆x, x 0 +(m+1)∆x) for positive and negative integers m. The estimate of the distribution is defined by [START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Izenman | Recent developments in nonparametric density estimation[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF]:

f (x) = 1 N × nb. of x i in same bin as x ∆x (2.1)
where x i ∈ A i represents the possible values of X i , N is the number of realizations of the random variable, and the bin width ∆x controls the amount of smoothing in the procedure.

One can generalize the histogram algorithm by allowing the bin width to vary [START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF]. Nevertheless, histogram estimators depend on both origin x 0 and width of the bins ∆x. The existence of two parameters affecting the distribution estimation together with the discontinuity of the estimated PDF lead to the necessity of more accurate estimators [START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Izenman | Recent developments in nonparametric density estimation[END_REF].

Kernel algorithms

Kernel estimator is a generalization of histogram estimator in which every observation x i is the center of a bin. In this way, kernel estimator does not depend anymore on the origin x 0 that disappear as a parameter of the estimation. On the other hand, kernel estimator allows a wide range of different shapes K for the bins. Then, kernel estimator with kernel K is defined by [START_REF] Parzen | On estimation of a probability density function and mode[END_REF][START_REF] Rosenblatt | Remarks on some nonparametric estimates of a density function[END_REF][START_REF] Cacoullos | Estimation of a multivariate density[END_REF][START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF][START_REF] Moon | Estimation of mutual information using kernel density estimators[END_REF]:

f (x) = 1 N ∆x x i ∈A i K x -x i ∆x (2.2)
where ∆x is the window width or band width. The kernel estimator is a sum of bump functions K, centered at the observations x i . For f defined in eq.( 2.2) to tend to the real distribution f , the kernel function should be a Borel function satisfying [START_REF] Parzen | On estimation of a probability density function and mode[END_REF][START_REF] Rosenblatt | Remarks on some nonparametric estimates of a density function[END_REF][START_REF] Cacoullos | Estimation of a multivariate density[END_REF][START_REF] Izenman | Recent developments in nonparametric density estimation[END_REF]:

sup -∞<x<∞ |K(x)| < ∞ (2.3) ∞ -∞ |K(x)|dx < ∞ (2.4) lim x→∞ |xK(x)| = 0 (2.5) ∞ -∞ K(x)dx = 1 (2.6)
and the window width ∆x should satisfy:

lim

N →∞ ∆x| N = 0 (2.7)
For the most basic kernel estimator, called the naive estimator, the kernel function K is a weight function, K(x) = 1/2 if |x| < 1 and zero otherwise [START_REF] Parzen | On estimation of a probability density function and mode[END_REF][START_REF] Cacoullos | Estimation of a multivariate density[END_REF][START_REF] Steuer | The mutual information: Detecting and evaluating dependencies between variables[END_REF]. The choice of bin width still remains and controls the amount by which the data are smoothed to produce the estimate. As the basic histogram estimator the naive one is not a continuous function of x [START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Izenman | Recent developments in nonparametric density estimation[END_REF].

More complex kernel estimators can be obtained by replacing the weight function by a smoother kernel function K as for example Gaussian. A table containing some kernel functions satisfying the above properties was presented by Parzen [START_REF] Parzen | On estimation of a probability density function and mode[END_REF]. From the definition of kernel estimates one obtains that if K is a probability density function, then f is a probability density function which inherits all the continuity and the differentiability properties of the kernel [START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Izenman | Recent developments in nonparametric density estimation[END_REF].

The main drawback of this kind of estimators appears when applied to data from long-tailed distributions. The density of this kind of distributions strongly varies from high values in the region near the mode of the distribution to small densities in the tails. Nevertheless the band width is fixed across the entire sample, and then there is a tendency for spurious noise to appear in the tails of the estimates, where density is small [START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Markovitch | Nonparametric estimation of long-tailed density functions and its application to the analysis of World Wide Web traffic[END_REF].

Nearest Neighbors Search algorithms

In NNS estimators the degree of smoothing is determined by the distance to the kth nearest neighbor d k (x) . The kth nearest neighbor density estimate is defined by [START_REF] Mack | Multivariate k-nearest neighbor density estimates[END_REF][START_REF] Moore | Consistency properties of nearest neighbor density function estimators[END_REF][START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF]:

f (x) = k 2N d k (x) (2.8)
The nearest neighbor estimate is inversely proportional to the size of the box needed to contain k number of observations. Then, in the tails of the distribution, the distance d k will be larger than in the center, and the problem of undersmoothing in the tails should be reduced.

The nearest neighbor estimate f , is continuous and positive everywhere but have discontinuous derivatives [START_REF] Silverman | Monographs on statistics and applied probability[END_REF].

Combinations of KDE and NNS

Kernel generalization of nearest neighbor estimate

With K(x) a normalized kernel function, the generalized nearest neighbor estimate reads [START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF]:

f (x) = 1 N d k (x) x i ∈A i K x -x i d k (t) (2.9)
where f (x) is precisely the kernel estimate with window width d k (x). However, the derivative of the generalized nearest neighbor estimate will remain discontinuous when d k (x) has discontinuous derivative [START_REF] Silverman | Monographs on statistics and applied probability[END_REF].

The variable kernel method

This estimate is constructed similarly to the classical kernel estimate but the scale parameter of the bumps is allowed to vary from one data point to another. We define K as the kernel function as above, and d i,k as the distance from x i to the kth nearest observation.

The variable kernel estimate with smoothing parameter ∆x is [START_REF] Devroye | Nonparametric Density Estimation: The L1 view[END_REF][START_REF] Silverman | Monographs on statistics and applied probability[END_REF][START_REF] Scott | Multivariate Density Estimation: Theory, practice and Visualization[END_REF]:

f (x) = 1 N x i ∈A i 1 ∆xd i,k K x -x i ∆xd i,k (2.10)
where the band width of the estimate placed on the point X i is proportional to d i,k , i.e. data points in regions where the data are sparse will have flatter kernels associated with them. Finally for fixed k the degree of smoothing depends on ∆x.

As the basic kernel estimate, the variable kernel estimate is a probability density function by definition.

In the above sections we have presented three different strategies of computing the PDF of a process: histograms, KDE and NNS. In addition, for each strategy we presented different algorithms. Nevertheless, estimating the entropy of a process by computing its PDF and using eq.(1.2) is not the less biased approach. In sections 2.2, 2.3 and 2.4 we present a straightforward methodology based on NNS to compute respectively the entropy, the mutual information and the m-order entropy rate. These three straightforwards estimators based on NNS have been proved to be less biased than the strategies presented above and other straightforward strategies [START_REF] Victor | Binless strategies for estimation of information from neural data[END_REF][START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Papana | Evaluation of mutual information estimators on nonlinear dynamic systems[END_REF][START_REF] Walters-Williams | Estimation of mutual information: A survey[END_REF][START_REF] Tobin | A kernel-based calculation of information on a metric space[END_REF].

Kozachenko-Leonenko estimate for Shannon Entropy

The nearest neighbor search estimator described by Kozachenko and Leonenko [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF][START_REF] Singh | Nearest neighbor estimates of entropy[END_REF][START_REF] Leonenko | A class of rnyi information estimators for multidimensional densities[END_REF] is based on the measure of the density µ(x) of the distribution, by calculating so that there are k points inside a ball of radius r = /2 whose center is the point x i of the process X. The expression of the entropy becomes:

H(X) = - A dxµ(x) log(µ(x)) = -log(µ(x)) = - 1 N N i=1 log µ(x i ) (2.11)
where N is the number of realizations of the process X.

It's important to remark that in the Kozachenko estimator the number of neighbors k is fixed and varies in function of the density µ(x) around x i .

The recipe to compute the Kozachenko-Leonenko estimate of entropy is: i) For a process X, with values in a metric space, and whose density is µ(x), we define its Shannon entropy in eq. (2.11). Unbiased estimator of the density log(µ(x)) = log(µ(x)) implies unbiased estimator of the entropy H(X) = H(X).

ii) We want to calculate log(µ(x i )). In order to do this, we consider:

1. P k ( ) ≡ Probability distribution for the distance between x i and its kth nearest neighbour.

2. P k ( )d ≡ Probability of having one point with distance r ∈ [ /2, /2 + d( /2)] from x i , having k -1 points with smaller distances, and Nk -1 with larger distances.

3. p i ≡ Mass of the ball centered at x i .

iii) We can obtain (using the trinomial formula):

P k ( )d = (N -1)! 1!(k -1)!(N -k -1)! dp i ( ) d d × p k-1 i (1 -p i ) N -k-1 (2.12) with d P k ( ) = 1 (2.13) iv)
We can therefore calculate the expectation value of log(p i ( )):

E(log(p i )) = ∞ 0 d P k ( ) log(p i ( )) = ψ(k) -ψ(N ) (2.14)
with ψ(x) ≡ Digamma function and E the average over the positions of the other N -1 points, with x i kept fixed.

v) We suppose µ(x) ≡ cte within the ball (This is the only approximation).

p i ( ) ≈ c d d µ(x i ) (2.15)
where d is the dimension of X, c d is the volume of the d-dimensional unit ball (it depends on the space metric).

vi) We take the logarithm on both sides of eq.( 2.15) and then its expectation value, so we arrive at:

log µ(x i ) ≈ ψ(k) -ψ(N ) -dE(log( )) -log(c d ) (2.16) vii)
We finally obtain:

H(X) = -ψ(k) + ψ(N ) + log(c d ) + d N N i log( (i)) (2.17)
with (i) twice the distance from x i and its kth nearest neighbour.

The only approximation made in eq.(2.15) implies that log(µ(x)) and then H(X) are unbiased only if the density µ(x) is strictly constant. In the following, unless said otherwise, we use the Kozachenko-Leonenko estimator for entropy calculations and use infinite norm when computing it. We reserve the notation H(X) for the Kozachenko-Leonenko estimate of entropy.

Rate of Convergence: Bias and Variance

Gao et al. studied the rate of convergence of the Kozachenko-Leonenko estimator over probability density functions with bounded support [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF]. They showed that under some assumptions for the PDF's and for finite k = O(1) and dimension m = O(1) the bias of the Kozachenko-Leonenko estimator is Õ(N -1 d ), its variance is Õ(1/N ) and hence the l 2 error of the Kozachenko-Leonenko estimator is Õ( 1

√ N + N -1 d ).
Comparisons between Kozachenko-Leonenko algorithm and kernel density estimation algorithms for Shannon entropy estimates have shown the advantages of the first one, especially for high dimensions [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF][START_REF] Xiong | Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations[END_REF]. The Kozachenko-Leonenko Shannon entropy estimate presents also better properties, as smaller bias or faster convergence, than bin strategies for entropy estimation [START_REF] Victor | Binless strategies for estimation of information from neural data[END_REF].

Kraskov-Stögbauer-Grassberger estimate for Mutual Information

As MI can be expressed as a combination of entropies, a direct way to calculate MI is to use the Kozachenko-Leonenko estimator for the three entropies involved in MI definition. Procceding this way, the bias of the measure is large because the different bias provided of the different entropy estimations have no reason to compensate and add up into a huge bias for the MI measure [START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF].

To get a better measure of the Mutual Information, Kraskov et al. [START_REF] Kraskov | Estimating mutual information[END_REF] developed two methods based on the Kozachenko-Leonenko estimate for Shannon entropy. The Kraskov-Stögbauer-Grassberger estimators fix a k neighbors value for the joint space (X ∪ Y ), then, in this space they obtains the distance to the kth neighbor, . Once is obtained in the joint space, it remains fixed for the marginal spaces (X and Y ), where the number of neighbors can vary. Fixing for all the entropy calculations produces a bias compensation between them. As such, we can understand the Kraskov-Stögbauer-Grassberger estimate as a mixture between NNS and KDE.

Next we detail the recipe to compute the Kraskov-Stögbauer-Grassberger estimates for mutual information.

First Mutual Information estimator. We consider maximum norm and the 2D random variable Z(X, Y ), defined as Z = X ∪ Y . We take one point z i and we consider the distance /2 to its kth nearest neighbour. We find eq.(2.12) and eq.( 2.14) and we can write eq.(2.15) in 2D:

p i ( ) ≈ c dx c dy dz µ(z i ) (2.18)
where d z = d x + d y . And we obtain:

H(X, Y ) = -ψ(k) + ψ(N ) + log(c dx c dy ) + d x + d y N N i log( (i)) (2.19)
With this method H(X), H(Y ), H(X, Y ) could be given, and we could find the MI using eq. (1.18). But if we use the same number of neighbors k for H(X), H(Y ) and H(X, Y ) their bias is so different that they do not cancel.

It's therefore better to have different values of k in the calculation of H(X), H(Y ) and H(X, Y ). To obtain adequate values for the number of neighbors of each different distribution, we fix k for the joint distribution Z(X, Y ) and we obtain the distance to the kth nearest neighbour (i). Then using this distance (i), we calculate the number of neighbours n x (i) + 1 lying on a segment

x = x i ± (i) 2 of the X dimension, i.e. lying on -∞ ≤ Y ≤ ∞, x i -(i)/2 ≤ X ≤ x i + (i)/2.
Consequently, (i)/2 is the distance to the n x (i) + 1st neighbour of x i . We use the same developement for Y and we obtain that (i)/2 is the distance to the n y (i) + 1st neighbour of y i . That's not really true, but is not a bad approximation. The errors in H(X), H(Y ), H(X, Y ) will not cancel exactly in general, but the chances that they will do approximately are bigger with this last procedure. We arrive to:

1) I(X, Y ) = ψ(k) + ψ(N ) -ψ(n x ) + ψ(n y ) (2.

20)

Second Mutual Information estimator. In this development we don't use hypercubes in the joint space but hyper-rectangles. Now we don't have a single but two, x and y . We therefore replace P k ( ) by:

P k ( x , y ) = N -1 k d 2 [q k i ] d x d y (1 -p i ) N -k-1 + (k -1) N -1 k d 2 [q k i ] d x d y (1 -p i ) N -k-1
(2.21) where q i ≡ q i ( x , y ) is the mass of the rectangle of size x × y centered at (x i , y i ) and p i is the mass of the square of size = max x , y . Using the maximum norm we guarantee that there are no points in the square that are not in the rectangle. We can calculate the expectation value of the logarithm of the mass of the hyperrectangles:

E(log q i ) = ∞ 0 d x d y P k ( x , y ) log(q i ( x , y )) = ψ(k) - 1 k -ψ(N ) (2.22)
If we say now that n x (i) and n y (i) are the number of points with distance less than or equal to x /2 and y /2 respectively, and following the above procedure we arrive to:

2) I(X, Y ) = ψ(k) + ψ(N ) -ψ(n x ) + ψ(n y ) - 1 k (2.23)
From here till the end of the PhD manuscript, even if both mutual information estimates are equally robust, when computing mutual information we will use the first mutual information estimator with infinite norm, unless said otherwise.

Rate of Convergence: Bias and Variance

In their initial article, Kraskov et al. derived and showed a bias behaviour of the Kraskov-Stögbauer-Grassberger estimate in k N [START_REF] Kraskov | Estimating mutual information[END_REF]. This result was later studied and supported by Gao et al. [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF] who showed that the bias of the estimator is Õ(

k N 1 dx+dy ) and the variance Õ(1/N ). Its l 2 error is Õ( 1 √ N + k N 1 dx+dy ).
Since the apparition of Kraskov-Stögbauer-Grassberger estimator of mutual information numerous comparisons between this NNS based estimator and others based on KDE or bins have been done [START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF][START_REF] Xiong | Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations[END_REF][START_REF] Tobin | A kernel-based calculation of information on a metric space[END_REF]. While Kraskov et al. [START_REF] Kraskov | Estimating mutual information[END_REF], Gao et al. [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], and Xiong et al. [START_REF] Xiong | Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations[END_REF], support the use of the Kraskov-Stögbauer-Grassberger estimator over KDE and bins, Tobin et al. [START_REF] Tobin | A kernel-based calculation of information on a metric space[END_REF] show that their KDE estimator produce very similar results. Kraskov-Stögbauer-Grassberger estimator appears always as better than bins estimators.

m-order entropy rate estimate

The simplest entropy rate estimation would use eq.(1.24) with two Kozachenko-Leonenko entropy estimators: ĥ(m,λ)

1 (X) = Ĥ(X (m+1,λ) ) -Ĥ(X (m,λ) ).
(2.24)

We propose instead to use eq.1.25 with Kozachenko-Leonenko estimate for the entropy and the Kraskov-Stögbauer-Grassberger estimate for the mutual information I. ĥ(m,λ)

2 (X) = Ĥ(X) -Î(X i+λ , X (m,λ) i ) (2.25)
As we showed above, both estimators Ĥ and Î are consistent with a variance O(1/N ) and a bias O(( k N )

1 d total )
where N is the number of samples of the analysed process and k the number of neighbors in the search.

From here till the end we estimate the m-order entropy rate by computing the Kozachenko-Leonenko entropy and the Kraskov-Stögbauer-Grassberger mutual information ( ĥ(m,λ) In order to compare both estimators of entropy rate, we study their bias and standard deviation [START_REF] Granero-Belinchón | Un estimateur du taux d'entropie basé sur l'information mutuelle[END_REF] when computed for two kinds of temporal processes (for temporal processes the sampling distance λ is noted τ ) with different marginals and correlation functions but with the same entropy rate [START_REF] Zografos | Expressions for renyi and shannon entropies for multivariate distributions[END_REF]. The first process X is obtained by low-pass filtering a Gaussian white noise. The used filter is defined as:

2 (X)), unless said otherwise. bias (a) m = 2 m = 3 (b) m = 1 m = 6 m = 4 m = 5 10 -2 10 -1 10 0 (c) (d) ĥ(m) 1 = H (m+1) -H (m) log 2 (N ) variance log 2 (N ) ĥ(m) 2 = H -I (m,1)
m = 5 m = 4 m = 6 (a) m = 1 (b) m = 2 m = 3 log 2 (N) ĥ(m,τ) 1 = H (m+1) -H (m) bias ĥ(m,τ) 2 = H -I (m,1,τ ) (c) (d)
F (ω) = 1 1 + i f fc (2.26)
where f c represents the cut-off frequency of the filter and i = √ -1. Next we normalize the signal by its variance, so that σ 2 = 1, and we obtain a Gaussian process whose correlation function is:

c(t -t ) = e -2πfc|t-t | .
(2.27)

The entropy rate of process X is deduced from eq.(1.40).

The second process is defined as Y = e X with X being the above defined Gaussian process. We define thus a process with log-normal marginal and correlation function:

c LN (t -t ) = e c(t-t ) -1 e 1 -1 . (2.28)
If the mean of the Gaussian process X is zero, the entropy and entropy rate of the log-normal process Y defined in eq.(1.43) and eq.(1.47) have the same expressions as the entropy and entropy rate of the Gaussian process X [START_REF] Zografos | Expressions for renyi and shannon entropies for multivariate distributions[END_REF], see eq.(1.36) and eq. (1.40).

In figures 2.1 and 2.2 we analyse the evolution of the bias and standard deviation in function of the size N of the analysed process, the scale parameter is fixed to τ = 3 and the number of neighbors k is fixed to 5. Indeed the different estimators depend very slightly on this parameter if k N . The new entropy rate estimator defined as the difference between the entropy and the mutual information appears as better than the estimator defined as the difference of entropies. Indeed, this new estimator is not only less biased and less dependent on the embedding dimension but also its bias is much less dependent on the process statistics (see figures 2.1 a) and b) and figures 2.2 a) and b)). This is an important feature as the statistics of the processes are a priori unknown in applications. for the new entropy rate estimator. The scale τ is fixed at (τ = 3) and chosen in order for the correlation coefficient between two succesive points to be c(τ ) = 0.94. The number of neighbors is 5 ≤ k ≤ 10, the sizes of the processes are comprised between 6 ≤ log 2 (N ) ≤ 12 and the embedding dimensions are 1 ≤ m ≤ 6. ) and supports the results already shown by Kraskov et al. [START_REF] Kraskov | Estimating mutual information[END_REF] and by Gao et al. [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], but this time for our estimator of entropy rate. In figure 2.3 the number of neighbors in the search varies between 5 and 10, k = [START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Arneodo | Revealing a lognormal cascading process in turbulent velocity statistics with wavelet analysis[END_REF]. The number of samples increases from a minimum value of N = 64 to a largest value of N = 4096. The embedding dimension varies from m = 1 to m = 6. In both figures 2.3(a) and (b) the bias goes to zero when k/N → 0. In addition there is another intermediate value of k/N for which the bias is zero, k/N | 0 . This intermediate value depends on the analyzed process. When k/N is between zero and k/N | 0 , the theoretical value h (m,λ) (X) is underestimated. However when k/N is higher than k/N | 0 the theoretical value is overestimated. These results are in agreeement with the Kraskov et al. study [START_REF] Kraskov | Estimating mutual information[END_REF].

log-normal k N 1/(m+1) Gaussian bias k N 1/(m+1) m = 1 m = 4 (a) m = 2 m = 3 m = 6 m = 5 (b) 0 0.2 0.4 0.6 0 0.2 0.4 0.6 -0.1 0 0.1 0.2 0.3
We also checked that the variance of the new entropy rate estimator behaves as O(1/N ) (see figures 2.1 d) and 2.2 d)).

Dynamics at a given scale: Adapted Theiler prescription

Takens embedding procedure (section 1.2) inherently defines a distance λ between samples of the analysed process X. In this way, we generate a process X (m,λ) , for which each sample is an m-dimensional vector, and each vector is composed of consecutive elements of X separated by λ. The distance between consecutive m-dimensional vectors defining the embedded process, X (m,λ) , is another important sampling distance λ th . Then, from a process X, we reconstruct an embedded one X (m,λ) , composed by m-dimensional vectors with a separation between consecutive vectors of λ th . The classical Theiler prescription consists on choosing λ th higher or equal to the correlation distance of the process, in order to eliminate all the dependencies between consecutive vectors of the embedded process. Therefore, only the static properties of the embedded process are taken into account. This correction, known as Theiler prescription [START_REF] Theiler | Spurious dimension from correlation algorithms applied to limited time-series data[END_REF], is usually used in order to avoid the effect of spurious correlations on the PDF of the studied signal [START_REF] Kantz | Nonlinear time series analysis[END_REF].

We are interested in the study of the dynamics of the process at a given scale λ. With this purpose we modify the classical Theiler prescription to take into account the statistics and dynamics of the process at this scale, and we choose λ th = λ. In this way, we force the distance between consecutive vectors to be exactly λ, the distance between consecutive samples inside the m-dimensional vectors, in order to eliminate the dependencies until the analysed scale λ. This is very important in order to take into account only desired interactions. We show, by analysing some temporal processes that, varying λ in the embedding procedure with an adapted Theiler prescription allows an efficient analysis of high order statistics across scales. We study the effect of performing an adapted Theiler prescription in four different temporal processes: a Gaussian filtered noise (the filter is defined in eq.(2.26)), a fractional Gaussian noise (section 3.1), a fractional Brownian motion (section 4.2) and Modane experimental turbulent velocity signal (section 6.1).

x(t + τ ) x(t) τ th ∈ [1, 30] h (m,τ ) ln(τ ) (b) τ th = τ τ th = 30 τ th = 1 (a) -2 -1 0 1 2 0 1 2 3 -2 -1 0
h (m,τ ) ln(τ ) x(t + τ ) x(t) τ th = 30 (b) τ th ∈ [1, 30] τ th = 1 τ th = τ (a) -4 -2 0 2 4 0 1 2 3 -4 -2 0
ln(τ ) x(t + τ ) τ th = τ τ th = 1 τ th = 30 τ th ∈ [1, 30] x(t) (b) h (m,τ ) (a
We can see in figures 2.4,2.5,2.6 and 2.8 that the adapted Theiler prescription is not equally important for every process. It depends on the importance of the spurious correlations remaining: the more correlated the process the more important the application of a Theiler correction to only take into account the desired statistics. In order to illustrate the importance of the spurious correlations on each analysed process, we plot in figures 2.4 b),2.5 b),2.6 b) and 2.8 b) their corresponding phase space. The more scattered the points in the phase space, the more uncorrelated the process.

We have analysed two noises with different dependence structure (figures 2.4 and 2.5), (a) but for noises, where correlations are usually weak (figures 2.4 b),2.5 b)), Theiler prescription is not very important. Only for pathological cases where the noises are specially correlated Theiler correction starts to be necessary. We find the opposite situation when analysing motions (figure 2.6). Motions are defined as processes where each step is characterized by a corresponding noise, see section 4.2. Motions are more correlated than a noise (figure 2.6 b)), and Theiler prescription is therefore very important. Figure 2.7 shows the effect of the size N of the analyzed process: increasing the size of the process it is possible to reduce the effect of spurious correlations. We can see in figure 2.7 that both limits τ th = 1, τ = 30 and τ th = 30, τ = 1 tend to their spected values, those at τ th = τ , when N increases. Nevertheless the convergence towards the Theiler corrected measure when N increases is very slow.

h (m,τ ) τ th = τ = 1 N τ th = τ = 30
τ th = 1 τ th = τ x(t + τ ) x(t) τ th ∈ [1, 30] τ th = 30 (b) ln(τ ) h (m,τ ) -3 -2 -1 0 1 2 0 1 2 3 4 5 6 7 -3 -2 -1 0
We also present the results obtained when analysing an experimental signal of turbulent velocity 2.8. The importance of Theiler prescription in this case is not surprising as the turbulent velocity field is modeled in terms of correlations by a fractional Brownian motion, see section 5.2.1.

Kraskov-Stögbauer-Grassberger estimate for Conditional Mutual Information

Kraskov-Stögbauer-Grassberger estimates for mutual information, see section 2.3, can be generalized for conditional mutual information [START_REF] Frenzel | Partial mutual information for coupling analysis of multivariate time series[END_REF][START_REF] Vicente | Transfer entropy-a model-free measure of effective connectivity for the neurosciences[END_REF][START_REF] Gómez Herrero | Assessing coupling dynamics from an ensemble of time series[END_REF][START_REF] Lizier | An introduction to Transfer entropy: Information flows in complex systems[END_REF]. As for the mutual information, this procedures allows to reduce the bias of the estimate by a bias compensation of the terms. The conditional mutual information of eq.( 1. [START_REF] Bar-Yam | Improving the effectiveness of health care and public health: a multiscale complex systems analysis[END_REF]) can be expressed as:

I(X, Y |Z) = H(X, Z) + H(Y, Z) -H(X, Y, Z) -H(Z)
To extend the first Kraskov-Stögbauer-Grassberger estimate we calculate in the joint space (X ∪ Y ∪ Z) the distance (using max norm) to the kth nearest neighbor. In the same way as for the mutual information, n z is the number of neighbors lying strictly within the ball of diameter in the marginal space Z, and n xz and n yz are the number of neighbors strictly within a ball of diameter in the joint spaces X ∪ Z and Y ∪ Z respectively. We obtain:

1) I(X, Y |Z) = ψ(k) -ψ(n xz ) -ψ(n yz ) + ψ(n z ) (2.29)
The second estimate of Kraskov-Stögbauer-Grassberger for mutual information can be also extended to conditional mutual information, leading to:

2) I(X, Y |Z) = ψ(k) + ψ(n z ) -ψ(n xz ) -ψ(n yz ) + 1 n xz + 1 n yz - 2 k (2.30)
As transfer entropy can be expressed as a conditional mutual information we can exploit these algorithms to compute it, and from here till the end of the manuscript every transfer entropy is computed following these algorithms.

CHAPTER 3 High order Statistics on non-Gaussian noises

The analysis of correlations and dependences covers an important fraction of the signal processing research. In order to characterize a process X(t), evolving along a given dimension t, e.g., the time, one needs to describe simultaneously: the one-point statistics of the process at a given time t and the dependences between n samples x t at different times. The one-point statistics are given by the probability distribution, and amounts to the knowledge of all the moments. The dependences between n samples, or n-point dependences, are usually hard to assess and very often limited to 2 points.

To analyse one-point statistics and dependences, we assume, unless said otherwise, that the process is stationary. One-point statistics are then independent of the time t and n-point dependences depend on the n -1 time delays separating the n samples.

The correlation function in direct space, or the power spectrum in Fourier space, are commonly used to characterize dependences in stationary processes. These measures give insight into the underlying dynamics; they are essential tools in statistics and data analysis and have been widely used in many different fields. In neurosciences, the analysis of correlations between neurons [START_REF] Moore | Statistical signs of synaptic interaction in neurons[END_REF][START_REF] Cohen | Measuring and interpreting neuronal correlations[END_REF] or between stimulus and brain activity [START_REF] Bair | Power spectrum analysis of bursting cellss in area mt in the behaving monkey[END_REF][START_REF] Gray | Chattering cells: Superficial pyramidal neurons contributing to the generation of synchronous oscillations in the visual cortex[END_REF][START_REF] Miller | Decoupling the cortical power spectrum reveals real-time representation of individual finger movements in humans[END_REF] is essential. In fluid mechanics, Kolmogorov's empirical theory of turbulence [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF] characterizes the multiscale distribution of energy with the power spectrum of the velocity field [START_REF] Anselmet | High-order velocity structure functions in turbulent shear flow[END_REF]. The analysis of dependences also supports useful descriptions in terms of complex networks for, e.g., ecology [START_REF] Zhang | Constructing ecological interaction networks by correlation analysis: hints from community sampling[END_REF], climatology [START_REF] Yamasaki | Climate networks around the globe are significantly affected by el niño[END_REF] or traffic [START_REF] Ou | Power-law strength-degree correlation from resource-allocation dynamics on weighted networks[END_REF].

Correlation analysis and power spectrum probe two-point interactions and involve only the statistics of order two of the one point disctribution. This is satisfying for processes with Gaussian distribution: these are completely characterized by their two-point interactions together with the mean and variance of the distribution. On the contrary, the complete characterization of non-Gaussian processes requires higher order statistics.

A first way to explore higher order statistics is to generalize the two point correlation function and consider n different points. This increase of the dimensionality naturally involves the n first moments of the one-point distribution, and hence n-order statistics. Three-point correlation functions, defined as an extension of the classical two-point correlation function, have been used in cosmology, where they were related to cosmological parameters [START_REF] Takada | The three-point correlation function in cosmology[END_REF][START_REF] Slepian | Computing the three-point correlation function of galaxies in O(n 2 )[END_REF]. Unfortunately the definition of three-point correlation functions is not unique, which may lead to ambiguities.

Another way to probe higher order statistics is to generalize the power spectrum: the bi-spectrum [START_REF] Greb | The interpretation of the bispectrum and bicoherence for non-linear interactions of continuous spectra[END_REF][START_REF] Collis | Higher-order spectra: The bispectrum and trispectrum[END_REF] combines Fourier transforms at two different frequencies and therefore takes into account up to the third order statistics of the distribution. In the same spirit, the tri-spectrum [START_REF] Collis | Higher-order spectra: The bispectrum and trispectrum[END_REF][START_REF] Chandran | Statistics of tricoherence[END_REF] probes the distribution up to the fourth order moment. Numerous applications of both bi-spectrum and tri-spectrum were proposed in cosmology [START_REF] Boubekeur | Detecting a small perturbation through its non-gaussianity[END_REF][START_REF] Matsubara | Analytic minkowski functionals of the cosmic microwave background: Second-order non-gaussianity with bispectrum and trispectrum[END_REF][START_REF] Trivedi | Cosmic microwave background trispectrum and primordial magnetic field limits[END_REF], in medicine [START_REF] Ning | Autoregressive and bispectral analysis techniques: Eeg applications[END_REF][START_REF] Husar | Bispectrum analusis of visually evoked potentials[END_REF][START_REF] Kertai | Brain monitoring with electroencephalography and the electroencephalogram-derived bispectral index during cardiac surgery[END_REF] and other fields [START_REF] Astola | Application of bispectrum estimation for time-frequency analysis of ground surveillance doppler radar echo signals[END_REF][START_REF] Courtney | Application of the bispectrum for detection of small nonlinearities excited sinusoidally[END_REF]. It is important to note that in order to probe higher order statistics with n-point correlations or multispectra, it is necessary to increase the dimension of the analysis.

An efficient way to probe high order statistics without increasing the dimensionality of the analysis is to use Shannon's information theory, see chapter 1. Shannon entropy characterizes the one-point distribution of the process and as such it naturally depends on all the moments of this probability density function, see section 1.3.1. Besides entropy, we are interested in the entropy rate, see section 1.4.1 and the auto mutual information, see section 1.4.2. Applications of the entropy rate have been reported in a wide range of domains, such as biology [START_REF] Herzel | Entropies of biosequences: The role of repeats[END_REF], dynamical systems [START_REF] Bollt | Validity of thresholdcrossing analysis of symbolic dynamics from chaotic time series[END_REF][START_REF] Lesne | Entropy estimation of very short symbolic sequences[END_REF], fluid turbulence [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF] or analysis of languages [START_REF] Ebeling | Entropy and long-range correlations in literary english[END_REF][START_REF] Takahira | Entropy rate estimates for natural language -a new extrapolation of compressed large-scale corpora[END_REF]. Mutual information is also very popular, especially in computer sciences, e.g., in machine learning [START_REF] Tourassi | Application of the mutual information criterion for feature selection in computer-aided diagnosis[END_REF]. AMI has been used in very different domains [4,[START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Granero-Belinchon | Mutual information for intrapartum fetal heart rate analysis[END_REF][START_REF] Gomez | Reduced predictable information in brain signals in autism spectrum disorder[END_REF][START_REF] Faes | Information decomposition in bivariate systems: Theory and application to cardiorespiratory dynamics[END_REF].

Entropy rate and AMI probe dependences at a particular scale or delay, and until recently, very few papers explore their evolution with the scale [START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF]. Finally, these quantities probe arbitrarily high order dependences when considering only two points, but they can be straigthforwardly generalized to consider interactions between more than two points.

The main goal of the chapter is to demonstrate the ability of information theory to describe high order statistics. To this end, we illustrate the use of entropy rate and auto-mutual information to characterize non Gaussian processes across scales in the same way as power spectrum and correlation function characterize Gaussian processes. As an example, we consider two synthetic log-normal processes with different dependence structure but with identical correlation function and identical marginal statistics. We show that we are able to measure differences in the high order statistics of the processes. In section 3.1, we present the synthetic processes that we study. In section 3.2, we report the convergence and the performance of the estimators of AMI and entropy rate. In section 3.3, we show their ability to measure high order dependences. We present their evolution across scales and emphasized that, contrary to classical correlation analysis, they allow a fine characterization of non Gaussian processes.

Data and procedure

Synthetic processes

In order to show the ability of entropy rate and auto mutual information in characterizing high order statistics of a signal, we synthesize two log-normal processes which have a different dependence structure, but identical one-point and two-point statistics. To show the generality of the method we use two frequently encountered covariance structures: exponential decay and long range dependence.

To synthesize a process Y , we need to impose the PDF and the correlation function. To obtain a log-normal PDF, we apply a transformation F on a Gaussian process X , which has the cumulative functionF G and the correlation function c x (τ ). This allows to map the Gaussian statistics of X to the prescribed log-normal statistics of Y using

y = F -1 LN • F (x ) where F -1
LN is the inverse of the cumulative log-normal distribution. The correlation function c y (τ ) of the process Y can be related to c x (τ ) using Hermite polynomial expansions. In some case, this relation can be inverted and c x (τ ) can be obtained from the targeted correlation function c y (τ ) of the log-normal process.

The methodology proposed in [START_REF] Helgason | Synthesis of multivariate stationary series with prescribed marginal distributions and covariance using circulant matrix embedding[END_REF] allows the synthesis of two log-normal processes by using two different transformation functions F . They lead to two log-normal processes with same marginal and correlation function but with different dependence structure.

Standard transformation

We consider the bijective transformation

F (x ) = F G (x ) (3.1)
Applied to a white Gaussian variable, this commonly used transformation leads naturally to the desired log-normal marginal distribution for Y . Due to the specific dependence structure of the log-normal process generated in this way, the correlation function of the Gaussian distribution c x (τ ) is obtained by inversion of the targeted correlation function c y (τ ) [START_REF] Helgason | Synthesis of multivariate stationary series with prescribed marginal distributions and covariance using circulant matrix embedding[END_REF].

c x (τ ) = 1 σ 2 x ln((e σ 2 x -1)c y (τ ) + 1) (3.2)
Even transformation Now, we consider the transformation

F (x ) = 2(F G (|x |) -1/2) (3.3)
which is even (∀x , F (-x ) = F (x )). In that case, the transformation that relates the correlation function c x (τ ) of the underlying Gaussian process to the correlation of the log-normal process c y (τ ) is not known. However, it can be approximated using circulant embedding, as presented in [START_REF] Helgason | Synthesis of multivariate stationary series with prescribed marginal distributions and covariance using circulant matrix embedding[END_REF].

Correlation structures

We impose for both log-normal processes, two different correlation structures with either short or long range dependences.

Exponential decay

We impose the following covariance of the log-normal processes:

c y (τ ) = σ 2 y e -fc|τ | , f c > 0.
where σ y is the standard deviation of the process. The corresponding power spectrum is Lorentzian.

Power law decay A long range dependence process is often modelled by a fractional Gaussian noise (fGn) indexed by a coefficient 0.5 < H < 1, called Hurst exponent. We impose the covariance of the log-normal processes to be the one of the fGn, which is [START_REF] Qian | Fractional Brownian Motion and Fractional Gaussian Noise[END_REF]:

c y (τ ) = σ 2 y 2 (τ -1) 2H -2τ 2H + (τ + 1) 2H , (3.4)
The corresponding power spectrum behaves as a power law with exponent -(2H -1).

For each correlation structure, we synthesize two log-normal processes and a Gaussian one with the toolbox provided at www.hermir.org. We normalize these processes such that the standard deviation, σ y , is equal to one.

In order to express analytically the auto mutual information defined in eq.(1.44) and the entropy rate in eq.(1.47), it's necessary to know the correlation of the process X = log Y . The standard transformation amounts to choose X = log Y = X, so eq.(3.2) can be directly used to express Σ appearing in eq.(1.44) and eq.(1.47). However, for the even transformation, the complex dependence structure of Y doesn't allow to express explicitly c x (τ ). As a consequence, there is no analytical expression of the auto mutual information and the entropy rate, although the entropy, which only depend on the PDF, is identical to the one obtained with the standard transformation.

Throughout this chapter, we work only with temporal processes, and then we note the sampling distance λ as τ .

Estimation Setup

For two given correlation structures, exponential decay or power law decay, we consider two log-normal and a Gaussian process. For each process, we generate a set of realizations of size N . We choose f c = 0.1 for the exponential decay correlation and a Hurst exponent H = 0.7 for the power law decay.

First, we characterize the three estimators Ĥ, Î(m,p,τ) and ĥ(m,τ) at a fixed scale τ , by measuring their bias and variance when varying both the signal size N and the number of nearest neighbors k. We also analyse their bias and variance when varying the correlation coefficient c(τ ). We do so with m = p = 1 for the Gaussian process and the standard lognormal, for which we have the analytical expressions, see section 1.7. Then, we present a new log-normal process with the identical marginal and correlation function of the standard log-normal one, but with different dependence structure. For each correlation structure (exponential decay, power law decay) we present the behavior of our quantities across scales τ for all the processes. Finally we show how the AMI behaves when the embedding dimension m and p are increased. For all the estimations we compute the average of 100 independent realizations and use the standard deviation over realizations as error bars in the different graphs.

Characterization of the estimates

The study performed in this section allows to understand and characterize the effect that the non-physical parameters, such as the size of the analysed process N (section 3.2.1), or the number of neighbors used in the nearest neighbors search k (section 3.2.2), have on the measures. In section 3.2.3 we characterize the effect of the correlation strength on the estimations. Figures 3.1, 3.2 and 3.3 present biases and standard deviations of the three different estimators Ĥ (upper row), Î(1,1,1) (middle row) and ĥ(1,1) (bottom row). Figure 3.1 shows the behaviour of the estimators with N , figure 3.2 with k, and 3.3 with c(τ ). Troughout this chapter, the results for the standard log-normal process are depicted in blue, while the ones for the Gaussian are depicted in black. 16 , for a fixed k = 5. For the entropy estimator, figures 3.1(a) and (b) show that the bias B Ĥ = E[ Ĥ -H] goes to zero when the signal size N increases. The standard deviation of the estimate, represented as the error bar of the bias, also tends to zero when N increases. The auto mutual information and entropy rate estimators behave similarly, see figure 3.1(c),(d) and (e),(f), respectively. So, as a first conclusion, our three estimates are asymptotically unbiased with a vanishing variance, for any dependence structure (exponential or power law decay) and for any one-point distribution (Gaussian or log-normal). They are therefore robust [START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], and have (very) satisfying performance as soon as N ≥ 2 10 .

Dependence on the data size

N . (e) B Î(1,1,1) (c) (f) log 2 (N) B ĥ(1,1)

Short range correlations

In the following, we use N = 2 16 to discard finite size effects. 16 . For any estimate, and any dependence structure, we barely see any evolution with k, although we have observed a small increase of bias when k increases. Because the choice of k does not impact significatively our estimates, we set k = 5 in the following to reduce the computation time.

Dependence on the number of neighbors

k. k B Î(1,1,1) (a)

Influence of the correlation strength

We now briefly study the behaviour of the AMI estimator when the correlation varies. With our choice of (f c , H), we have a correlation c(τ = 1) = 0.90 for the exponential decay and c(τ = 1) = 0.32 for the power law, when τ = 1.

In figure 3.3, we observe that both the bias and the standard deviation of the estimator increase when the correlation increases. For N = 2 16 the bias doesn't increase significantly when c(τ ) increases. Nevertheless, the standard deviation still depends on c(τ ) for large sizes.

For both correlation structures (exponential decay and power law decay) the lognormal statistics lead to a larger standard deviation than the normal ones do, but a very similar bias. In addition, for a given correlation c(τ ) the power law correlation structure shows a larger bias and standard deviation than the exponetial decay.

B Ĥ(1,τ) (a)
Long range correlations

(f) (d) (e) (b) B ĥ(1,τ) c Y (τ ) Short range correlations c Y (τ ) (c) B Î(1,1,τ) 0 0.

Dependence structure across scales

Once characterized our estimates, we explore their evolution on the scale τ , as well as on the embedding dimension. We show how these estimations can be used to discriminate processes that only differ on high-order dependences.

Exponential decay

For processes with the prescribed correlation function c(τ ) = e -fcτ , the power spectrum decreases as 1/(1 + (f /f c ) 2 ). The three synthetic processes have identical power spectrum -and hence identical correlation functions -so they cannot be distinguished with these estimators, see figures 3.4(b)(c). Classical analysis: histogram (a), power spectrum (b) and correlation function (c). Information theory analysis for m = 1, p = 1: entropy (d), entropy rate (e) and auto mutual information (f). The color indicates the process: blue for standard log-normal, red for even log-normal and black for Gaussian. The continuous lines in (e,d,f) correspond to analytical expressions. We used N = 2 16 and k = 5.

On the contrary, AMI and entropy rate behave differently across scales for each of the three processes. Although AMI vanishes for larger τ , its value at τ = 1 is different in each case. The observation is identical for the entropy rate: its value for smaller τ is also different for the three processes, although it converges to the entropy H for τ larger than the cut-off scale 1/f c . Analytical expressions (1.44 and 1.47) are presented as straight lines in 3.4(e) and 3.4(f) and they perfectly describe the scale-evolution of our estimations for the standard log-normal signal. The even (red) log-normal deviates significantly from the "standard" log-normal for smaller values of τ , where not only correlations are stronger, but also the complete dependence structure is different. We also report different behavior of the AMI of Gaussian and log-normal signals, although they all have the same correlation function (figure 3.4(c)). Again, this illustrates the sensitivity of AMI on higher order dependences. Studying the evolution of either AMI or entropy rate across scales reveals subtle differences in the dependence structures of the signals.

Power law decay

For scale invariant processes, there is no cut-off scale and both the power spectrum and the correlation function are power laws of the scale. As for the exponential correlation structure, classical measures are identical for both log-normal processes (figures 3.5(a,b,c)) but AMI and entropy rate evolve differently for each process ((figures 3.5(e,f)). Again, AMI always converges to zero for large τ but takes different values for τ = 1. Again, entropy rate tends to the entropy for large τ while evolving differently from its value at τ = 1, which depends noticeably on the process.

For stronger dependences (here the exponential decay, due to our choice of (f c = 0.1, H = 0.7), see section 3.2.3), AMI is larger and so are the differences between the processes. For scale invariant processes, AMI and entropy rate are again able to probe details of the dependence structure that originate from higher orders. In this case, the difference between both log-normal processes is smaller but still significant. show how information theory probes high order dependences, which allow to discriminate two very similar processes when performing an analysis across scales τ . We interpret AMI as a generalization of correlation function.

Dependence on embedding dimensions for fixed τ .

We now want to probe explicitly dependences between more than two points. To do so, we can increase the number of points that are involved in the definition of AMI, as this quantity provides a direct measure of dependences between its arguments, see eq.(1.17).

A straightforward generalization is to increase any embedding dimension m or p appearing in eq. (1.26). This can be interpreted as probing how additional points increase the shared information, or in other words as measuring multiple-point dependences. We note that this shared information is a non decreasing function of m or p, i.e. it can be constant or increase. This increase of AMI hints at the evolution of multiple-point dependences when considering additional points, and we expect it to be larger for long-range than for short-range dependence processes.

Î(m,p,

Î(m,p,1) For the exponential decay, the analytical curves (straight lines) in figure 3.6(a,c) show that AMI is independent of the total embedding dimension m + p. The estimations agree with the analytical curve, even if the existence of a bias unfortunately impairs an arbitrary increase of the embedding dimensions. This is known as the curse of dimensionality [START_REF] Bellman | Adaptive Control Processes: A Guided Tour[END_REF].

Exponential Decay m + p Î(m,p,1) m + p Scale Invariant (c) (e) (a) m=1 (d) m=3 m=2 (f
For the power law decay, figure 3.6(b,d,f) shows a small increase of analytical and estimated AMI when the total embedding dimension increases. Indeed, the estimations closely follow the analytical prediction and the increase is very similar for the three processes. In this case the bias is almost negligeable, nevertheless it slightly increases when increasing m + p.

A comparison between exponential and power law decay confirms that the increase of AMI with m and p indicates long range dependences. For long range dependence structures, any additional point contains new information. In that case, we can make another observation from figure 3.6: it is not equivalent to increase m or p. For a fixed m+p, AMI is larger when m and p are closer. This can be understood as a closer similarity between the two embedded vectors appearing as the arguments of the AMI in eq. (1.26).

All these observations indicate the more complex dependence structure of the longrange correlation processes.

Conclusion 2-point interactions:

We proposed to explore the evolution of AMI and entropy rate on the scale parameter. AMI and entropy rate measure the strenght of the dependences between two points of a process. These quantities probe statistics of any order, and hence non-linear interactions, and as such appear as unambiguous generalizations of the correlation function and the power spectrum. As an illustration, we analyzed two log-normal processes with identical 1-point distribution and identical correlation function. We were able to discriminate these two processes, which appear as identical for classical analysis: so our measures enlight differences in high order dependences of the two processes. The behavior of our quantities along scales reveal the existence of stronger high-order dependences at smaller scale, which allows an easier separation of processes.

More than 2-point interactions: In addition, AMI generalizes easily to consider explicitly interactions between more than two points. We thus probe the complexity of the dependence structure above linear order, i.e., the additional information measured by AMI when considering an extra point should indicate the existence and relevance of next-order interactions. As an example, comparing the effect of increasing the order of interactions for two dependence structures showed that while exponential decayshort range -dependences do not involve next order interactions, power law -long range -dependences do. This result could show that increasing the embedding allows to discriminate between short and long range dependences.

The same qualitative results were obtained when considering an exponential one-point distribution, which supports the validity of the methodology for any distribution and dependence structure. This generality should make the methodology very interesting to perform non-Gaussian processes characterization in several different applications.

CHAPTER 4 Information theory of non-stationary processes

In real life, a high number of processes such as: weather temperature, water ressources evolution, macroeconomic time series or Brownian motion, are non-stationnary. Consequently, the characterization and analysis of such processes become very important. However, because of the difficulty of studying non-stationnary processes, their characterization is usually done by studying their increments or time derivatives. During this chapter, we are interested in the direct analysis of non-stationnary processes.

Information theory was initially conceived to work with stationnary processes, where the measures have a clear interpretation in terms of information, see chapter 1. Nevertheless, two questions appear immediately:

• Can we apply information theory on non-stationnary processes?

• Do the interpretations on stationnary processes remain valid for non-stationnary ones?

For a non-stationnary time process M (t) whose PDF and dependences evolve in time, its entropy H(M (t)) evolves with time.

We can measure the entropy of a process at a given time H t (M (t)) by averaging over the realizations. In this framework, the interpretation of the entropy for stationnary processes remains valid [START_REF] Gómez Herrero | Assessing coupling dynamics from an ensemble of time series[END_REF]. In the same way, we can measure the dependences of the process at time t. Performing these analysis at fixed time for different times allows to characterize not only the distribution and dependences of the process at time t but also their evolution in time. Unfortunately, the analysis at a given time t requires having numerous realizations of the process to build its PDF without requiring ergodic hypothesis. This requirement is not easily resolve, specially when dealing with experimental signals.

In oder to avoid this requirement, we want to study a process M (t) by analysing a experimental signal of length T . We can measure its entropy during the time window [0, T ], H T (M (t)), but the interpretation of this entropy is not anymore the same. We interpret H T (M (t)) as the complexity, or the total amount of information needed to characterize a temporal trajectory M t ∀ t ∈ [0, T ] of the process M (t). In this way, we consider the temporal dimension as a new dimension of the process and then, the distribution of the process should take into account not only the distribution of the process in its phase space but also in the temporal dimension [START_REF] Vu | Information In The Non-Stationary Case[END_REF]. The main disadvantage of this viewpoint is the necessity of defining a window size T , restricting the evolution of the distribution in time.

Even if the natural environment of information theory is the analysis of stationnary processes, some scarce works have tried to perform an information theory analysis of non-stationnary processes [START_REF] Vu | Information In The Non-Stationary Case[END_REF][START_REF] Ray | On the characterization of non-stationary chaotic systems: Autonomous and non-autonomous cases[END_REF][START_REF] Gómez Herrero | Assessing coupling dynamics from an ensemble of time series[END_REF]. In this chapter, the goal is to develop an information theory framework to analyse generic non-stationary processes. Once it is done we develop inside this framework the specific case of non-stationary self-similar motions with stationary increments, on which we are specially interested. Self similar power law decay motions are particularly interesting due to the high number of physical, biological or social systems (between others) that they can model. Once the theory is developed we verify it numerically and we use it to characterize some different motions.

Information Theory for Non-Stationary processes 4.1.1 General framework

Shannon entropy

We note M (t) a continuous non-stationnary stochastic time process, and p Mt (x) = p(M (t) = x) its probability density function at time t. We can naturally build the time embedded process M (m,τ ) (t) [START_REF] Takens | Dynamical Systems and Turbulence[END_REF].

The entropy of a non-stationnary time embedded process M (m,τ ) (t) at time t is defined as:

H(M (m,τ ) (t)) = H t (M (m,τ ) ) = - R m p M (m,τ ) t (x) log(p M (m,τ ) t (x))dx (4.1)

Entropy rate

The entropy rate of order m at time t measures the variation of Shannon entropy between M (m+1,τ ) t and M (m,τ ) t-τ , two successive time-embedded versions of the non-stationnary stochastic process M (t):

h (m,τ ) (M (t)) = h (m,τ ) t (M ) = H t (M (m+1,τ ) ) -H t-τ (M (m,τ ) ). (4.2)

Mutual Information

The mutual information measures the amount of information shared by two processes L (m,τ ) t and M (p,τ ) t at time t and t , and is defined by :

I(M (p,τ ) t , L (m,τ ) t ) = (4.3) = R m+p p M (p,τ ) t ,L (m,τ ) t (y, x) log p M (p,τ ) t ,L (m,τ ) t (y, x) p M (p,τ ) t (y)p L (m,τ ) t (x) dydx (4.4)

Auto Mutual Information

In order to analyse the nonlinear temporal dynamics of process M , we use auto mutual Information, defined in [START_REF] Granero-Belinchon | Information theory to probe intrapartum fetal heart rate dynamics[END_REF].

I (m,p,τ ) (M (t)) = I (m,p,τ ) t (M ) = I(M (p,τ ) t , M (m,τ ) t-pτ ) .
(4.5)

I (m,p,τ ) t
(M ) measures the shared information at time t between current p-points dynamics and past m-points dynamics.

Following eq.(1.25), the entropy rate can then be expressed as:

h (m,τ ) t (M ) = H t (M ) -I (m,1,τ ) t (M ) . (4.6)
Relation between the entropy rate of a process and the entropy of its increments

For any process M , we have the following relation:

h (m,τ ) (M (t)) = h (m-1,τ ) (δ τ M (t)) -I M (t -τ ), δ τ M (t)|δ τ M (m-1,τ ) (t) . (4.7)
In the special case m = 1, we have:

h (1,τ ) (M (t)) = H(δ τ M (t)) -I(M (t -τ ), δ τ M (t)) . (4.8)
Proof. The demonstration does not involve any hypothesis on the process, which may not be scale invariant. The stationarity of the increments is not required.

Step 1 : We say that:

H(M (m,τ ) (t)) ≡ H (M (t), M (t -τ ), M (t -2τ ), ..., M (t -(m -1)τ )) = H (M (t), δ τ M (t), δ τ M (t -τ ), ..., δ τ M (t -(m -2)τ )) . (4.9)
This follows from expressing the movement M as a sum of its "recent" increments, and using chained conditioned probabilities.

Another way to derive eq.(4.9) is to transform the embedded vector M (m,τ ) (t), using a linear transformation:

M (m,τ ) → M (m,τ ) (t) = Q m .M (m,τ ) (t),
where Q is the band matrix defined as:

Q m ≡          1 0 0 • • • 0 0 1 -1 0 • • • 0 0 0 1 -1 • • • 0 0 . . . . . . . . . . . . . . . . . . 0 0 0 • • • -1 0 0 0 0 • • • 1 -1          . (4.10)
Note that only the diagonal and the subdiagonal are populated. Note also that Q m is not orthogonal, and not unitary, but its determinant is

det(Q m ) = (-1) m-1 , (4.11)
so |det(Q m )| = 1, which is enough to write H(M (m,τ ) (t)) = H(M (m,τ ) (t)), which proves eq.(4.9).

Step 2 : We combine the entropies according to eq.( 4.2) and get:

h (m,τ ) (M (t)) = H(δ τ M (m,τ ) (t)) -I M (t -τ ), δ τ M (m-1,τ ) (t -τ ) , δ τ M (m,τ ) (t) (4.12)
which can straightforwardly be rewritten as eq.(4.7).

Self Similar motions: Non-Stationary processes with Stationary increments

Along the above section we have setted a framework where applying information theory to non-stationnary processes. In this section we focus on self similar motions with stationnary increments and we deduce the behaviour of the above tools in function of the time t and the scale τ of analysis.

We suppose stationary increments For stationary increments, the statistics of δ τ M (t) do not depend on time t and we can write p δτ M (t) = p δτ .

We suppose scale invariance

For scale-invariant processes there exists a value H ∈ R, 0 < H < 1 such that for all a ∈ R + , x ∈ R we have an equality in law between the statistics of M (at) and a H M (t).

p Mat (x) = 1 a H p Mt ( x a H
). This is also true for the embedded signal.

p M (m,aτ ) at (x) = 1 a mH p M (m,τ ) t x a H .
With the stationarity of the increments p δτ M (t) = p δτ and setting M (t = 0) = 0, M (t) can be seen as an increment, M (t) = M (t) -M (0) = δ t (t), which is itself scale invariant.

p δaτ (x) = 1 a H p δτ
x a H . It is easy to show that a sum of scale invariant processes with parameter H is also scale invariant with the same parameter

H [].
When there is scale invariance, the special time dependence can be recast to unittime (t=1). This simply amounts to rescaling each dimension. This is easy to do with M (m,τ ) (t). Unfortunately, all the results above do not give explicit formulas for the "constants" that appear.

Shannon entropy

The Shannon entropy defined in eq.(4.16), of the set of points M (m,τ ) (t) at time t is:

H(M (m,τ ) (t)) = H 0 (M (m,τ /t) (1)) + H ln t + (m -1)H ln τ (4.13)
Entropy rate Once we know the Shannon entropy of a process M for any embedding m we can calculate the entropy rate using eq.(4.17) or eq.(4.19) arriving to:

h (m,τ ) (M (t)) = h (m,τ /t) 0 (M (1)) + H ln τ (4.14)
The "constants" H 0 or h 0 are given by the dependence structure of either the movement or its increments (i.e., the noise) between times (1, 1τ /t, ..., 1 -(m -1)τ /t). They have no reason to be independent on τ /t, but the dependence on τ /t is weak, hence the naming "constants".

T -length window framework

In order to work within this framework, the stationarity of the increments δ τ (M (t)) of the analysed process is required. In this framework we study non-stationnary processes by fixing a window size T of analysis, and defining the PDF of apparition of a value M (m,τ ) during a time interval of length T .

pM (m,τ ) ,T (x) = 1 T t i +T t i p M (m,τ ) t (x)dt (4.15)

Shannon entropy

The entropy of this PDF is:

HT (M (m,τ ) ) = - R m pM (m,τ ) ,T (x) log(p M (m,τ ) ,T (x))dx (4.16)
As such, it involves arbitrarily high order moments of the PDF pM (m,τ ) ,T (x), and therefore high order statistics. Nevertheless, as the entropy doesn't depend on the first moment of the PDF and the increments δ τ (M (t)) are stationnary, HT (M (m,τ ) ) only depend on T and not on the initial time value t i defining pM (m,τ ) ,T (x).

This entropy HT (M (m,τ ) ) represents the entropy of the set of values of the process M (m,τ ) (t) during a time interval of size T . As for stationnary processes, in the case of m = 1, there is no dependence of the entropy on τ , and then, from here till the end we will write HT (M ) = HT (M (1,τ ) ).

Entropy rate

As for the Shannon entropy we can define the entropy rate during a time interval of size T : h(m,τ)

T (M ) = HT (M (m+1,τ ) ) -HT (M (m,τ ) ). ( 4

Analytical results for fractional Brownian motion

Fractional Brownian motion (hereafter fBm) is a continuous-time random process proposed by Mandelbrot and Van Ness [START_REF] Mandelbrot | Fractional brownian motions fractional noises and applications[END_REF] in 1968, which quickly became a major tool in various fields where concepts of self-similarity and long-range dependence are relevant. Although this signal is non stationary, its increments are Gaussian and stationary. The non-stationary covariance structure E{B(t)B(t + τ )} = σ 2 0 c(t, τ ) is given by

σ 2 0 c(t, τ ) = σ 2 0 2 t 2H + (t + τ ) 2H -|t -(t + τ )| 2H (4.20)
The pre-factor σ 0 is a normalization constant.

The fBm has Gaussian statistics, we can therefore write its entropy following eq.(1.36). In addition, the fBm is non stationary, but if we consider a signal of finite temporal extension T τ and Hölder exponent H, if we note H 0 FBM ≡ 1 2 ln (2πeσ 2 0 ), we find: Entropy

H(1,τ) T (B) = H 0 FBM + H ln T (4.21)
Auto-mutual information

Ī(1,1,τ) T (B) = -H ln τ T + C τ T (4.22)
with

C τ T = 1 2 ln τ 2T + 1 2H+1 ( τ T + 1 2H+1
) which has been numerically shown to be very few dependent on the embedding. Entropy rate

h(m,τ) T (B) H 0 FBM + H ln τ -C τ T (4.23)
which is independent of T up to corrections in τ /T , which are negligible if T is large enough compared to the range of τ used. The entropy rate of the fBm is therefore linear in ln(τ ), with a constant slope H, independent of the temporal extension. This analitycal result has been numerically probed to be very few dependent of the embedding. We can observe how the main dependance on ln(T ), appearing in the expressions of HT (B) and Ī(m,p,τ)

T (B) dissapears in the entropy rate h(m,τ) T (B)
, just because of a compensation of the dependances in the entropy and the auto mutual information.

Relation between the entropy rate of fBm and the entropy of its increments

We can write the entropy of the increments of a fBm, which are stationnary and Gaussian, as:

H(δ τ B) = H 0 FBM + H ln(τ ) (4.24)
Looking at eq.(4.23) and eq.( 4.24), we can straightforwardly write:

h(m,τ) T (B) H(δ τ B) -C τ T max (4.25)
To obtain these relations for a fBm, we have calculated the different coefficients of the covariance matrix of eq. (1.36). Then, we can calculate the determinant of this matrix and the mean over time of the determinant between t min = 0 and t max = T . With the determinant and eq.(1.36), we can obtain the different entropies involved in the different equations of HT , ĪT and hT respectively. It's important to suppose that T τ .

Data and procedure

In order to build different self-similar non-stationary processes M (t) with stationary increments x(t), we use eq.(4.26) with the three different processes defined in section 3.1 as increments.

M (t) = t i=1 x(i) (4.26)
Using the fractional Gaussian noise of section 3.1, we generate a fractional Brownian motion (fBm), which is the only Gaussian self-similar process with stationary increments [START_REF] Mandelbrot | Fractional brownian motions fractional noises and applications[END_REF]. Using the two log-normal noises of section 3.1, we generate two different self-similar motions, that we call "standard" and "even" log-normal motions depending on the respective transformation used to generate them (section 3.1).

We characterize each motion by computing its entropy, auto mutual information and entropy rate, using the Kozachenko-Leonenko and Kraskov-Stögbauer-Grassberger algorithms for entropy and mutual information (see chapter 2).

Numerical analysis: T -length window framework 4.3.1 Characterization of the estimates

Standard deviation characterization

We present the variation of the standard deviation (std) of h(m,τ)

T
when N is varied, and in particular we compare it to the std of the two separate terms HT and Ī(m,τ) T in eq. (4.19). We observe in figure 4.1 that the std of each contribution separately is much higher than the std of h(m,τ)

T : There is a partial cancellation of the standard deviations in the estimation of the entropy and the mutual information, respectively, such that the standard deviation of h(m,τ)

T is much smaller.

Bias characterization

We have tried to quantify how the bias depends on both N and k and observed that the convergence seems to be in k/N 1 m+1 -in agreement with the results of [START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF] for the mutual information estimator that we use -as can be seen in figure 4. T (stars), as functions of N , in a) for the fBm and in b) for the two log-normal processes. In blue the standard log-normal process; in red the even log-normal process.

k/(N 1/2 ) k/(N 1/2 ) k/(N 1/2 ) h (1,2) (a) (c)

Conclusions

The standard deviation of h(m,τ)

T can be made arbitrarily small, especially compared to the ones of the entropy or the auto mutual information. Both the bias and the standard deviation of our estimator increase when k is increased or N is reduced. We choose k = 5 in the remainder of this chapter, and use datasets of N = 2 16 points. . The small dependence of the entropy rate on T can be due to bias effects, whose importance increases with the embedding, but it can also be due to a different behavior of the entropy rate in the different frameworks when the embedding increases. Returning to the general framework, figure 4.5 shows the behaviour of I(M t-τ , δ τ M t ) in function of τ /t which varies from 0 in both limits to a maximum value of 0.08 when τ /t = 0.5. In figure 4.5, the red line correspond to the analytical prediction, while the black lines correspond to multiple curves for different t and different τ . Consequently, the only important parameter defining the magnitude of I(M t-τ , δ τ M t ) is τ /t and the importance of this term in eq.(4.8) is very few for any rate τ /t.

Figure 4.6 a) shows that eq.(4.8) is valid in the T -window framework for scale invariant motions with stationnary increments. The black line indicates the entropy of the different increments of size τ of a fBm M . The blue line is the entropy rate of the fBm at scale τ . The red line is the entropy rate of the fBm computed as the right hand side of eq.(4.8). Finally, the magenta line is the mutual information between the increments of the motion and the motion itself. This mutual information term is negligeable for τ T . In addition, the behaviour of ĪT (M t-τ , δ τ M t ) in τ /T is expected to be similar to that of I(M t-τ , δ τ M t ) (figure 4.6), and numerical computations show that both I(M t-τ , δ τ M t ) and ĪT (M t-τ , δ τ M t ) are of the same maginitude for τ /t = τ /T . Figure 4.6 b) shows how the entropy rate of the fBm match almost perfectly (in the errorbars) with the entropy of the increments of the fBm. T (M t ), entropy of the increments HT (δ τ M ), mutual information between the motion and the increments ĪT (M t-τ , δ τ M t ), and the difference between the two HT (δ τ M ) -ĪT (M t-τ , δ τ M t ). b) Entropy rate of the motion h(1,τ) T (M t ) minus entropy of the increments HT (δ τ M ), and mutual information between the motion and the increments. The motion is a fBm with H = 0.7. The time window size is T = 1024.

h(1,τ) T (M ) HT (δ τ M ) - h(1,τ) T (M ) h(1,τ) T (M t ) HT (δ τ M) -ĪT (M t-τ , δ τ M t ) ĪT (M t-τ , δ τ M t ) HT (δ τ M) (b) HT (δ τ M) - h(1,τ) T (M t ) ln(τ ) (a) ĪT (M t-τ , δ τ M t ) 0 1 2

Application: Discrimination between processes

h (1,τ ) -H ln(τ ) (b)
h (1,τ ) ln(τ )

We now study the dependence of h(m,τ)

T on the time lag τ for the three motions presented in section 4.2. Varying τ is analogous to varying the time scale at which the signal is sampled [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF], and then, we can analyse the self-similarity properties of the processes and look for fine differences between them.

Results for fixed size, N = 2 16 , k = 5 neighbors and embedding dimension m = 1, are reported in figure 4.7 (a). The behaviors accross scales of the three motions are linear in ln(τ ). To get a more precise insight, we substract, for the three motions, the dependence H ln(τ ) obtained numerically for self-similar processes in the precedent section. Results are presented in figure 4.7 (b).

We obtain for the fBm a constant value very close to the theoretical value 1 2 log(2πe) 1.42. The red, even log-normal motion does not depend on τ anymore. So, just like in the fBm case, the slope is exactly H, as expected for a self-similar process. On the contrary, the blue, standard log-normal motion still depends slightly on τ , sharing the same value with the other log-normal process for τ = 1, and the same value with the fBm for large τ . We can state that the fBm and the even log-normal (red) motions are self-similar with Holder exponent H = 0.7. Their values at τ = 1 are directly related to the probability density function of the noises defining the motions, see eq.(4.8) for the general framework. We can state also that the standard log normal process (blue) is not self-similar with an evolution of its pdf across scales.

P (M(t + τ ) -M(t)) M(t + τ ) -M(t)
The different behaviors of h(m,τ)

T observed for the two log-normal motions should be therefore related to different evolutions of the pdfs of the increments when τ is increased.

To check this, we plot in figure 4.8 the normalized pdfs (std=1) of the increments δ τ M (t) = M (t) -M (tτ ), for various values of τ .

While the pdf of the increments of the, even log-normal process (red) does not change when τ is increased, this is not the case for the increments of the standard log-normal motion (blue): the shape of the pdf changes from log-normal for small τ to Gaussian for large τ . Note that for τ = 1, the pdf of the increments of the motion is nothing but the pdf of the noise, which is the same for both log-normal processes. Indeed, the entropy rate has very similar values for the two log-normal motions when τ = 1. While for the red, even log-normal process, h(m,τ)

T remains constant across scales τ , this is not the case for the blue, standard process, for which the entropy rate converges to the value obtained for the Gaussian process, see figure 4.7 (b).

Discussion and Conclusions

In the first part of the chapter, we develop analytically the expression of entropy rate h (m,τ ) t (M ) at a given time t and scale τ for a generic non-stationary process M (t), and we obtain the analytical relationship between this entropy rate and the entropy of its increments H(δ τ M ), which are very strongly connected (eq.(4.8)). This connection allows us to argue that the self-similarity properties of any motion can be studied by measuring its entropy rate across scales. In the case of monofractal motions, we show analytically that the entropy rate across scales gives a measure of the Hurst exponent. However, we don't show the specific analytical expressions for multifractal processes.

In the second part of the chapter, we propose a new framework where analysing non-stationnary processes using information theory. In this framework, we analyse time windows of size T of the processes. In order to do that, we define the probability density function of a process in a time window of size T , as the mean of the different PDF's of the process at the different times inside the window. Then, the information theory quantities are computed using this averaged PDF. The interpretation of these information theory quantities is not anymore the same as in the general framework. We interpret HT (M (t)) as the complexity, or the total amount of information needed to characterize a temporal trajectory M t ∀ t ∈ [0, T ] of a process M (t).

In this framework, we develop the analytical expressions of entropy, auto mutual information and entropy rate of a fractional Brownian motion, for which the covariance is known. We also show numerically the behaviours of entropy, and entropy rate in function of T and τ for self similar processes and conclude that these behaviours are in agreement with the ones predicted for H (m,τ ) t (M ) and h (m,τ ) t (M ). We show numerically the validity in this framework of the analytical relationship between the entropy rate of a process and the entropy of its increments, obtained in the generic framework. The validity, in this new framework, of this analytical relationship is shown for a fBm. However, it is valid for any process, either monofractal or multifractal. This methodology, based on the measure of the entropy rate computed in a window size of time T , is strongly related to a new methodology that we develop in chapter 7. This new methodology works on the increments of the analysed processes, and both are able to describe the self-similarity properties of any process.

Finally, we use h(m,τ) T (M ) to analyse three different a priori self similar non-stationnary processes, and obtain fine differences in their self similarity properties (figure 4.7). We observe that while the fBm and the even log-normal motions are self-similar with h(m,τ) T (M ) = H 0 + H ln(τ ) , the standard log-normal motion presents a non-linear entropy rate across scales. This application allows to corroborate the close relationship between the entropy rate across scales of a process and the entropy, and consequently the PDF, of its increments. In this application, the three analysed motions have been built with the noises of chapter 3 as smaller increment. Comparing the results of chapter 3 with the results obtained in section 4.4, we can conclude that analysing the motion is not equivalent to analysing the generator noise. More precisely, the entropy rate of the motions characterize their self-similarity properties and the entropy rate of the noise doesn't.

Part II Turbulence CHAPTER 5

Fully developed turbulence

Two different descriptions of a fluid flow are usually adopted to study fluid mechanics: the Eulerian framework and the Lagrangian one. In both, one supposes the fluid to be composed of particles of size larger than molecular scales but smaller than dissipative ones (defined below).

• In the Eulerian framework, the observables of the fluid particles: velocity, pressure, temperature, etc depend on space r and time t.

• In the Lagrangian framework, one follows the particles and studies the observables of each particle over time.

During my PhD, I focused on Eulerian turbulence and so, in the following, we develop the theory using the Eulerian framework. In this framework a flow of a fluid of viscosity ν is characterised by:

• Conservation of momentum or Navier-Stokes equations:

ρ ∂v ∂t + (v • ∇)v = -∇p + ∇ • τ (5.1)
where p is the pressure, ρ the density of the fluid, v the velocity and τ is the viscous stress tensor.

• Conservation of mass or continuity equation:

∂ρ ∂t + ∇ • (ρv) = 0 (5.2)
• Conservation of energy:

ρc p ∂T ∂t + (v • ∇)T = k∇ 2 T + φ (5.3)
where c p is the specific heat of the fluid, φ is the dissipation function representing the work done against viscous forces, and T is the temperature.

In the following we develop the theory only for incompressible and isothermal flows. For an incompressible flow (ρ ≡ cte) with constant temperature (T ≡ cte):

• Conservation of momentum or Navier-Stokes equations becomes:

∂v ∂t + (v.∇)v = - ∇p ρ + ν∇ 2 v (5.4)
• Conservation of mass or continuity equation becomes:

∇.v = 0 (5.5)
• Conservation of energy becomes:

φ = 0 (5.6)
Eq.( 5.4) describes the dynamics of the velocity v(r, t) of an incompressible fluid of viscosity ν and density ρ. It shows the competition between different forces acting on the fluid:

• ∂v ∂t + (v.∇)v are the inertial terms. The first one indicates the variation in time of the velocity, while the second one, which is non-linear, describes advection.

• -∇p ρ is the internal force.

• ν∇ 2 v is the diffusion term due to the viscosity of the fluid.

The competition between the non-linear advective and the linear diffusion terms defines the Reynolds number:

R = (v.∇)v ν∇ 2 v (5.7)
The Reynolds number, initially introduced by Stokes and later generalized by Reynolds, characterises the behaviour of the flow. A low Reynolds number corresponds to a laminar flow and a high Reynolds number indicates a turbulent flow. Fully developped turbulence is defined in the limit of infinite Reynolds number. With dimensional analysis one can approximate the Reynolds number as [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF]:

R ≈ σL ν (5.8)
where the characteristic velocity σ is the r.m.s velocity fluctuation of the fluid at the characteristic scale L of the flow, defined as σ = (δ L v(x)) 2 . In 1921 Richardson presented an account of turbulence, which still prevails. In Richardson's description, turbulence is presented as a hierarchy of whirls of different sizes, with an energy cascade from larger eddies down to smaller ones [START_REF] Richardson | Some measurements of atmospheric turbulence[END_REF], see figure 5.1. This description defines inherently two important domains: the integral (large scales) domain where energy is injected and the dissipative (small scales) domain where energy is mostly dissipated. So, from Richardson's picture, we note a very important characteristic of turbulence: it is multi-scale, see figure 5

Kolmogorov's theory

Influenced by Richardson's description of turbulence, Kolmogorov differentiated three different domains of scales in turbulence:

• The integral region: large scales, equal to or larger than the integral scale L. The integral domain is defined as the domain where energy is injected in the system.

• The inertial range: scales smaller than L but larger than the Kolmogorov dissipative scale η K . In the inertial domain, energy cascades from large to small scales.

• The dissipative domain: scales smaller than η K . In the dissipative domain, energy is dissipated.

The inertial region is characterized by a clear predominance, in the Navier-Stokes equation, of the advective term over the diffusive one, while in the dissipative domain both terms are equally important.

In order to develop his theory, Kolmogorov made two hypotheses [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF]: • The first similarity hypothesis: "At very high, but not infinite, Reynolds number, all the small scale statistical properties are uniquely and universally determined by the scale l, the mean energy dissipation rate and the viscosity ν."

• The second similarity hypothesis: "In the limit of infinite Reynolds number, all the small-scale statistical properties are uniquelly and universally determined by the scale l and the mean energy dissipation rate ."

which we can summarize as: for Reynolds number going to infinity, the small scales (scales smaller than the integral scale L) of turbulence are statistically isotropic and independent of the large scales [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF][START_REF] Chevillard | Description multifractale unifieé du phénomène d'intermittence en turbulence Eulérienne et Lagrangienne[END_REF].

The mean energy dissipation rate is defined as [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF]:

= 1 2 ν i,j ∂v i ∂x j + ∂v j ∂x i 2 (5.9)
Using only his first similarity hypothesis, Kolmogorov shows in [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF] that the dissipation scale η K , below which diffusion is relevant, can be expressed as:

η K ≈ ν 3/4 1/4
(5.10)

Using the two universality assumptions and phenomenological analysis [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF], Kolmogorov statistically characterized the behaviour of the velocity increments in the inertial region [START_REF] Kolmogorov | On degeneration (decay) of isotropic turbulence in an incompressible viscous liquid[END_REF]:

δ l v(x) = v x (x + l) -v x (x) ≈ 1/3 l 1/3
(5.11)

Eq.(5.11) leads to specific scaling behaviours of the moments of the velocity increments, called structure functions [START_REF] Kolmogorov | On degeneration (decay) of isotropic turbulence in an incompressible viscous liquid[END_REF]:

S p (l) ≡ (δ l v) p = C p (l ) p/3
(5.12)

where p is the order of the structure function and C p are the universality constants that only depend on the order of the structure function.

From the scaling behaviour of the second order structure function S 2 (l) one obtains the spectral energy distribution of Eulerian fully developed turbulence (see figure 5.2):

E(k) = v x (k) 2 ∝ k -5/3
(5.13)

where l = 1/k.

The relations above (eq.(5.11), eq.( 5.12) and eq.(5.13)) are valid for any scale l in the inertial region, where there is neither direct energy injection nor direct energy dissipation, but a flux of energy Π from integral scale L to dissipative scale η K . Consequently with the eq.( 5.11), the flux of energy Π should be independent of the scale and equal to the mean energy dissipation rate:

Π ≈ (δ l v(x)) 3 l ≈ (5.14)
In particular, very close to the integral scale L we arrive to

≈ (δ L v(x)) 3 L , (5.15)
showing the finiteness of the mean energy dissipation rate.

Four-fifths law

The a priori phenomenological definition of the mean energy dissipation rate of eq.(5.15) was directly derived by Kolmogorov from the Navier-Stokes equations. Kolmogorov obtained an exact relation for the third order structure function S 3 (l) [START_REF] Kolmogorov | Dissipation of energy in locally isotropic turbulence[END_REF], which should be respected for any model of turbulence:

In the limit of infinite Reynolds number, the third order structure function of homogeneous isotropic turbulence, evaluated for increments l small compared to the integral scale, is given in terms of the mean energy dissipation rate by

S 3 (l) = - 4 5 l (5.16)
The four-fifths law shows the existence of an energy cascade in fully developed turbulence from large scales to smaller ones.

From the estimation of the Kolmogorov scale in eq.(5.10) and the approximation of the mean energy dissipation rate of eq.(5.15) it is possible to show that for large enough Reynolds number the inertial range is sufficiently large [START_REF] Landau | Fluid Mechanics: Volume 6 of course of theoretical physics[END_REF][START_REF] Kolmogorov | A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high reynolds number[END_REF].

η K L ≈ R -3/4
(5.17)

This relation shows that for Reynolds number going to infinity the distance between L and η K becomes infinite (infinite inertial range) and one is in the framework of fully developped turbulence.

In order to avoid anisotropic effects of integral scales, L, in the definition of the Reynolds number, it is useful to define the Reynolds number at a smaller scale, called the Taylor scale λ. This Taylor scale is defined through the relation:

1 λ 2 = (v -v ) 2 (∂ x v(x)) 2 (5.18)
The associated Reynolds number is defined as:

R λ ≈ σ λ λ ν (5.19)
where σ λ is the r.m.s velocity fluctuation of the fluid at the scale λ.

Using a phenomenological approach and eq.(5.15) (result obatined directly from the finiteness of the energy dissipation) one can shows [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF]:

R λ ≈ √ R (5.20)
This definition of the Reynolds number is specially advantageous for experimentalists, who can measure the Reynolds number taking advantage of the isotropy of the flow at the Taylor scale, which is located in the inertial range.

Kolmogorov-Oboukhov 1962 correction

Kolmogorov's 1941 theory assumes that "Since is almost constant in regions which are small in comparison with the external scale L, when l L it may be supposed that l = ". This assumption was quickly constested by Landau, as it does not take into account that with the increase of the ratio L/l the variation σ 2 of the dissipation of energy defined in the K41 theory (see eq.(5.9)) would increase without limit [START_REF] Landau | Fluid Mechanics: Volume 6 of course of theoretical physics[END_REF][START_REF] Kolmogorov | A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high reynolds number[END_REF][START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF].

Experimental support for the criticism of K41 theory came later with observations of turbulent intensity variations when analysing spectral fluctuations [START_REF] Gurvitch | Eksperimentalnye issledovania chastotnykh spektrov[END_REF][START_REF] Oboukhov | Some specific features of atmospheric turbulence[END_REF]. These observations indicated the inhomogeneity of the dissipation rate. In the seventies, experiments analysing the deformation of the PDFs of velocity increments across scales [START_REF] Tennekes | The intermittent small-scale structure of turbulence: data-processing hazards[END_REF] (see figure 5.4) and later the scalings of the structure functions of order higher than 3 [START_REF] Anselmet | High-order velocity structure functions in turbulent shear flow[END_REF] corroborate the innacuracy of K41 theory.

Kolmogorov and Oboukhov corrected the previous K41 theory by supposing an asymptotic behaviour for the dispersion of the logarithm of , σ log( ) . The main correction made by Oboukhov and Kolmogorov to the K41 theory was the definition of a local energy dissipation l instead of a global one [START_REF] Kolmogorov | A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high reynolds number[END_REF][START_REF] Oboukhov | Some specific features of atmospheric turbulence[END_REF]. They defined l distributed with log-normal statistics and they found:

δ l v(x) = v x (x + l) -v x (x) ≈ ( l l) 1/3
(5.21)

S p (l) = (δ l v) p = C p (l l ) p/3 (5.22)
The assumption of a non-constant dissipation rate leads to different scalings of the increments, and hence of the structure functions, across scales. This phenomenon is called intermittency.

Multifractal approach of Kolmogorov's theory

Frisch and Parisi [START_REF] Frisch | On the singularity structure of fully developed turbulence[END_REF], recovered all of the Kolmogorov theory of turbulence using the multifractal approach [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF][START_REF] Chevillard | Description multifractale unifieé du phénomène d'intermittence en turbulence Eulérienne et Lagrangienne[END_REF][START_REF] Staicu | Intermittency in turbulence[END_REF]. This is done by reformulating the scaling behaviour of increments and structure functions across scales: In multifractal theory h is a random variable with probabilty density: In order to compute D(h) we use the wavelet leaders methodology [START_REF] Wendt | Bootstrap for empirical multifractal analysis[END_REF].

δ l v(x) ≈ l h (5.
P l (h) ≈ l 3-D(h) (5.
Another way to describe the scaling exponents is to use the following expansion [START_REF] Delour | Intermittency of 1d velocity spatial profiles in turbulence: a magnitude cumulant analysis[END_REF]:

ζ(p) = c 1 p -c 2 p 2 2! + c 3 p 3 3! -• • • (5.27)
where the coefficients c p of the expansion are called log-cumulants.

A linear behavior of the scaling exponents with p characterizes a monofractal process. In the case of a monofractal process c 1 = h is called the Hurst exponent, and noted H. In contrast, the existence of non-zero log-cumulants c p of order p ≥ 2, and hence a non linear behavior of the scaling exponents, reveals multifractality, see figure 5.3 a).

Monofractal model

Kolmogorov's 1941 theory states h ≡ H = 1/3 which places turbulence in the monofractal regime. This assumption can be formulated as:

• D(h) = δ(h -1/3
) is a single-valued function. For a monofractal process there is only one possible value for the Hölder exponent h, the Hurst exponent H, see figure 5.3 b).

• ζ(p) = pH = p 3 . The scaling exponent are linear in p of slope 1/3 (see figure 5.3) a), so only the first log-cumulant c 1 is non zero:

c 1 = H = 1/3.
• S p (l) ≈ l p/3 normal power laws.

• Self similarity: statistical properties of the increments remain unchanged across scales. The scale invariance implies the following relation between the probability distributions p δ l X and p δ l 0 X of the increments of scales l and l 0 :

p δ l X (δ l X) = l 0 l H p δ l 0 X l 0 l H δ l X , (5.28)
This relation is valid for all pairs of scales (l, l 0 ).

Figure 5.3 a) shows that, when p increases, the scaling exponents of experimental turbulent velocity deviate from the linear behaviour predicted by the monofractal model. Therefore, following K41 to model turbulence is not very satisfying for larger p.

Multifractal models

Intermittency is manifested in different ways, which are related. First, one can understand intermittency as the multivaluation of h which leads to the non-linearity of the scaling exponents ζ(p) [START_REF] Atta | Statistical self-similarity and inertial subrange turbulence[END_REF][START_REF] Anselmet | High-order velocity structure functions in turbulent shear flow[END_REF]. This non-linearity directly implies the abnormal power law behaviour of the structure functions of the velocity increments [START_REF] Anselmet | High-order velocity structure functions in turbulent shear flow[END_REF].

A related manifestation of intermittency is the evolution of the PDF of the velocity increments when the size of the increment varies, see figure 5.4. The PDF of the velocity increments, close to Gaussian for scales higher or equal to the integral scale L, deforms across the inertial domain: it develops larger tails and a dissymetry [START_REF] Tennekes | The intermittent small-scale structure of turbulence: data-processing hazards[END_REF][START_REF] Atta | Statistical self-similarity and inertial subrange turbulence[END_REF][START_REF] Noullez | Transverse velocity increments in turbulent flow using the relief technique[END_REF]. The deformation of the PDF is faster when entering the dissipative range [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF].

Intermittency corrections give a multifractal status to turbulence and then:

• h takes different values. Consequently, the singularity spectrum D(h) is multivalued, see figure 5.3 b). In the case of KO62 theory the singularity spectrum is parabolic.

• ζ(p) = p 3 non linear. • S p (l) ≈ l ζ(p) modified power laws.

• No self similarity: statistical properties of the increments change across scales. Their pdfs deform across scales.

δ l v(x)/σ l P (δ l v(x)/σ l )
Figure 5.4: Deformation of the PDF of velocity increments across scales, from Gaussian at large scales to strongly non Gaussian at small scales.

Different multifractal models of turbulence exist. Each one of them takes into account intermittency in a slightly different way. Between the most famous models are the lognormal model and the log-Poisson one.

Intermittent log-normal model

This model for turbulence was introduced by Kolmogorov and Oboukhov in 1962 [START_REF] Kolmogorov | A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high reynolds number[END_REF][START_REF] Oboukhov | Some specific features of atmospheric turbulence[END_REF] and extensively studied by Chevillard et al. [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF]. It was the first intermittent model of turbulence, with the following scaling exponents:

ζ(p) = c 1 p -c 2 p 2 2
The non-linear dependence of the scaling exponents in p indicates the multifractal nature of the model, which is quantified by c 2 . All log-cumulants c p of order p ≥ 2 are zero. Its singularity spectrum is:

D(h) = 1 - (h -c 1 ) 2 2c 2 (5.29)
This multifractal process offers a satisfying representation of the scaling exponents of turbulence if c 2 = 0.025 and c 1 = 1/3 + 3c 2 /2 = 0.371. The c 2 value for a correct log-normal modeling of turbulence was obtained experimentally. Once c 2 is fixed, the four-fifths law imposes the c 1 value for the model.

Intermittent log-Poisson model

This model was introduced by She and Leveque [START_REF] She | Universal scaling laws in fully developed turbulence[END_REF]. This heuristic model leads to scaling exponents of the form:

ζ(p) = -γp -λ(β p -1)
The four-fifths law, together with the assumption of a finite support of dissipation when the viscosity tends to zero [START_REF] She | Universal scaling laws in fully developed turbulence[END_REF], imposes λ = 2, β = 2 3 (1/3) and γ = -1/9. For these values the model describes the scaling exponents ζ(p) as well as the log-normal model does.

It has later been interpreted as a log-Poisson model with a singularity spectrum

D(h) = 1 -λ + h + γ ln(β) ln h + γ -λ ln(β) -1 (5.30)
The corresponding log-cumulants are:

c 1 = -γ -λ ln(β) (5.31) c m = λ ln(β) m , m ≥ 2
(5.32)

Conclusion

This multifractal formalism of turbulence developed by Frisch and Parisi can describe both Kolmogorov 1941 andKolmogorov-Oboukhov 1962 theories. This formalism is also able to characterize modern physical theories such as the β-model [START_REF] Frisch | A simple dynamical model of intermittent fully developed turbulence[END_REF][START_REF] Benzi | Characterisation of intermittency in chaotic systems[END_REF][START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF] or She and Leveque model of turbulence [START_REF] She | Universal scaling laws in fully developed turbulence[END_REF].

A possible objection to the formalism is that it moves away from physical phenomena. Nevertheless, it provides an ideal framework to statistically characterize processes and signals. In part II, we characterize turbulence by analysing experimental and synthetic processes together with theoretical models, using information theory. We rely on multifractal formalism to describe turbulence and to give an interpretation to our measures.

CHAPTER 6 An information theory viewpoint of K41 theory

We propose in this chapter an information theory perspective on turbulence, and especially on K41 theory (section 5.1). We first describe the distribution of information (in Shannon's sense) across scales in turbulence and then show and explain its close connexion with the energy scaling. We also explore the existence of an energy cascade in turbulence via information theory.

In order to study turbulence, and recover the K41 theory in the information theory framework, we analyse experimental velocity signals obtained from different experimental setups and a fractional Brownian motion, which is the monofractal synthetic process proposed by Kolmogorov to model turbulence (section 5.2).

In a first stage, we report how the entropy rate is able to describe the distribution of information amongst scales, and the relation between the entropy rate of the analysed process and the entropy of its increments (chapter 4). In a second stage, we design a conditioning procedure of the signal and we show that this allows for a very fine exploration of the asymmetry of the increments distribution and hence the inference of the existence of a finite skewness and an energy cascade.

Turbulent experimental systems and signals

We analyze two different sets of experimental turbulent data, in order to show the ability of our measures to grasp inherent properties of turbulence.

The first system consists of a temporal measurement of the longitudinal velocity (V ) at one location in a grid turbulence setup in the wind tunnel of ONERA at Modane [START_REF] Kahalerras | Intermittency and Reynolds number[END_REF]. The Taylor-scale based Reynolds number R λ is about 2500, with a turbulence intensity of about 8%. The inertial region spans approximately three decades. The sampling frequency is f s = 25 kHz and the mean velocity of the wind in the tunnel is v = 20.5 m/s. The probability density function of the data is almost Gaussian although there is some visible asymmetry: the skewness is about 0.175 ± 0.001.

The second system is a set of temporal velocity measurements at different Reynolds numbers in a jet turbulence experiment with Helium [START_REF] Chanal | Intermittency in a turbulent low temperature gaseous helium jet[END_REF]. The Taylor-scale based Reynolds number R λ is respectively 89, 208, 463, 703, 929, with a turbulence intensity about 23%.

Using the Taylor hypothesis [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF] and the mean velocity v of the flow, we interpret these time series as the spatial evolution of the longitudinal velocity. The time scale τ and the spatial scale l are related by l = v τ . We note the integral time scale T and the integral spatial scale L, and we have L = v T . During this chapter, we represent our results in function of ln 1 τ . We define the temporal variable τ being multiple of dt where dt = 1/f s with f s the sampling frequency in the case of experimental signals. This representation together with Batchelor model presented in section 6.2.1 allows to estimate the integral and Kolmogorov scales of the studied flow. In the case of the generated fBm we analysed in this section, we can suppose f s = 1. In chapter 7 we present all our results as functions of the ratio τ /T = l/L between the scale of the increment and the integral scale.

Scaling of information in turbulence

In order to study the dynamics of the signals, we compute the entropy rate h (m,λ) across scales (see sections 1.5 and 2.5).

The size of the analysed processes N = 2 17 is kept constant for all the scales τ . The entropy rate is computed, and then averaged, over 195 independent realizations in the case of the Modane turbulent velocity data and the fractional Brownian motion and over 10 realizations in the case of the jet turbulence.

Grid turbulence

Our results are presented in figure 6.1 as a function of log(1/τ ). Three regions are observed: above 36ms are the integral scales, below ∼ 0.18ms is the dissipative range, and in-between them is the inertial range. At larger (integral) scales, the auto-correlation function vanishes and the entropy rate is equal to the entropy H of the signal (m = 1) which depends only on one-point statistics. We can interpret this first result as indicating that the integral scales, where energy is injected and turbulence is generated, are the most disorganized. In the inertial range, the entropy rate decreases almost linearly with a slope close to -1/3, represented by a straight line in figure 6.1. As the time -or space -scale is decreased, the flow appears more and more organized in the sense that the amount of "new" information brought to one point by another point at a distance τ decreases with τ . In the dissipative range, the entropy rate decreases faster and faster, as the dissipation become stronger and stronger. We interpret this as a consequence of the flow being more and more organized.

The entropy rate displays differently from the power spectrum the separation between the different domains, as can be seen in figure 6.1. Remaining in the real space we avoid some perturbing efects of the Fourier transform over the spectrum [START_REF] Lohse | Bottleneck effects in turbulence: Scaling phenomena in r versus p space[END_REF]. The integral and Kolmogorov scales shown in this figure have been obtained using the Batchelor model [START_REF] Batchelor | Pressure fluctuations in isotropic turbulence[END_REF] for fully developed turbulence, as detailed below.

We checked that our estimation of the entropy rate does not depend on the number of neighbors used in the k-nn search algorithm (k = 5 and 10) nor on the sample size N (N = 2 16 , 2 17 and 2 19 ). The standard deviation of the entropy rate estimation is of the order of 0.03 for small τ and increase to 0.08 for large τ . We varied the embedding dimension m and observed a dependance of the entropy rate in the dissipative range only, for scales τ smaller than 0.18ms. Going deeper and deeper in this range, the signal is more and more continuous so the knowledge of an increasing number m of points in the past (separated by the scale τ ) decreases significantly the "new" information brought by a (m + 1)th point in the future (figure 6.1). But for m > 2, there is no measurable evolution anymore, even in the dissipative range.

Batchelor Model: Entropy rate and autocorrelation

Here we present the Batchelor model for fully developed turbulence [START_REF] Batchelor | Pressure fluctuations in isotropic turbulence[END_REF]. Batchelor provided a model for the inertial and dissipative regions of S 2 (l), the second-order structure function:

S 2 (l) = (l/L) 2/3 1 + (η k /l) 2 2/3 . (6.1)
L is the integral scale and η k the dissipation scale, l is the current scale. Eq.( 6.1) imposes on the second-order structure function to have a slope 2/3 in the inertial range and 2 in the dissipative range, with a smooth transition between these two regions. Noting that S 2 (l) = 1-c(l) and assuming Gaussian distribution for the turbulent velocity, we can use eq.(1.40) and obtain:

h (1,l) (X) = H(X) + 1 2 log(S 2 2 (l) -2S 2 (l)) (6.2)
Substituting the expression of the second order structure function of eq.( 6.1) into eq.( 6.2) one derives for the entropy rate h (1,l) two linear behaviors in log(1/l): one with a slope -1/3 in the inertial region and another one with a slope of -1 in the dissipative range. This analytical development shows the narrow connection between the traditional Kolmogorov 1941 scaling law (section 5.1) and our scaling law for the entropy rate.

Fitting S 2 , the structure function, or fitting the entropy rate (with the Batchelor model) can be used to estimate the scales L and η k . Thus, for a serie of measures where L is kept constant and the Reynolds number (Re) varied, Batchelor model allows to verify, the well known relation between the Kolmogorov scale (η k ) and the integral one (L) showed in eq. (5.17).

This can be understood as another test (and application) for our entropy rate estimator, in order to check if the physics of turbulence is well grasped by the entropy rate. Theoretically, being directly related with the second-order structure function by eq.( 6.2), the entropy rate can also be used to corroborate eq.(5.17).

Fractional Brownian motion

As the turbulent velocity distribution is almost Gaussian, in the K41 theory Kolmogorov proposed fractional Brownian motion (see section 4.2) with Hurst exponent H = 1/3 to model the dependency structures of turbulent velocity in the inertial range, see section 5.2.1. Fractional Brownian motion with Hurst exponent H = 1/3 presents a power spectral density with a power law of exponent -5/3, like that of the turbulent velocity, see figure 5.2.

The entropy rate of the fBm is therefore linear in ln(τ ), with a constant slope H, independent of the temporal extension, as we can see in figure 4.3. Figure 6.5 shows the entropy rate h (1,τ ) of a synthesized fBM with H = 1/3. The measured slope of the entropy rate in function of ln(τ ) is 0.32 ± 0.01, in agreement with the theoretical value H = 1/3. We have also computed h (m,τ ) for 2 ≤ m ≤ 4 and observed a small deviation from the linear behavior for large τ , while the slope slightly increases to reach 0.34±0.01 for m = 4. We attribute this to finite size effects, especially in the corrections to eq.(4.23).

The fBm, being monofractal (see section 5.2.1), has no characteristic scales (neither integral nor Kolmogorov scale), so the slope of its entropy rate is unperturbed. Therefore, analysing the behaviour of such a process with Hurst exponent H = 1/3 allowed us to explain the scaling behaviour of the entropy rate of our turbulence data in the inertial range.

Entropy rate of the velocity and the entropy of its increments

The turbulent velocity is stationary, and eq.(4.8) can be used to show the analytical relationship between the entropy rate of the velocity and the entropy of its increments. Figure 6.2 shows numerically the correctness of eq.(4.8). Figure 6.2 b) shows the border effects in both limits: when τ is small, under the dissipative scale, the filtering effects of the experimental set-up affects differently the velocity signal and its increments; for large scales, the effects of the available statistics are also different. Nevertheless, figures 6.2 a) and b) show the close relationship between the entropy rate of the velocity and the entropy of its increments. .2: a) Entropy rate of the velocity h (1,τ ) (V ), entropy of its increments H(δ τ V ), mutual information between the velocity and its increments I(V t-τ , δ τ V t ), and the difference between the two last H(δ τ V )-I(V t-τ , δ τ V t ). b) Entropy rate of the velocity h (1,τ ) (V ) minus entropy of its increments H(δ τ V ), and mutual information between the velocity and its increments. The velocity signal is the one from the wind tunnel at Modane . The time window size is T = 2 16 .

I(V t-τ , δ τ V t ) H(δ τ V ) h (1,τ ) (V ) (a) H(δ τ V ) -h (1,τ ) (V ) ln(1/τ ) H(δ τ V ) -h (1,τ ) (V ) (b) I(V t-τ , δ τ V t ) H(δ τ V ) -I(V t-τ , δ τ V t ) h (1,τ ) (V )
The case of fractional Brownian motion, being non-stationary, is completely studied in chapter 4, sections 4.1.2 and 4.3. (1,τ ) for different Reynolds numbers, as a function of the scale (f = 1/τ ). The entropy has been normalised to put all the curves over the lowest Reynolds number measure. The thin straight line is a line of slope -1/3, the thick straight line has a slope -1. Bottom: entropy rates h (1,τ ) , as a function of the scale (f = 1/τ ) for different Reynolds numbers, and the Batchelor fits for each of them (points).

ln(1/τ )

h (1,τ ) (X) h (1,τ ) (X) R λ = 463 R λ = 703 R λ = 929 R λ = 703 R λ = 463 R λ = 929

Jet turbulence in Helium

As in the grid turbulent system, in the jet turbulence on Helium we find three different regions, figure 6.3: integral, inertial and dissipative, with three different behaviors. As predicted by the theory developed by Batchelor we find a slope -1/3 in the inertial range, a slope tending to -1 in the dissipative region and a convergence to the entropy value of the velocity signal in the integral domain. The interpretation is the same than in the first system. We can see in figure 6.3 how the Batchelor Model is able to fit the behavior of the entropy rate in the inertial and dissipative domains, and the transition between them. Another important result is the maximum value of the entropy rate increasing with the Reynolds number. That can be understood as the total information contained in the system increasing with the Reynolds number, and then, increasing with the development of turbulence. The higher the Reynolds the more developed the turbulence, the larger the value of information contained in the system. It's important to remark that the maximum value of the entropy rate is the value of the entropy of the signal and mainly depends on the standard deviation of the signal.

In this system we have access to different Reynolds number flows, and thus we can analyse the variation of the inertial region length of turbulence as measured by the entropy rate in function of R λ . We can observe in figure 6.3 how qualitatively the inertial region length of turbulence measured by the entropy rate decreases when the Reynolds number decreases, as expected. Using Batchelor model we can fit the different Reynolds number curves and obtain the integral and Kolmogorov scales predicted by the model for each curve. The integral scale L is the same for the different analysed signals as they come from the same experimental set-up, in which R λ was changed without modifying L.

When the embedding dimension increases we find similar results to the obtained in the first system. The estimation doesn't vary, except in the dissipation domain.

Conditioned Entropy rate

In a general way we can define the conditioned entropy rate h (m,λ) (X cond ) as the entropy rate applied over a subsampled signal X cond obtained by conditioning the mother signal X.

In chapters 3 and 4 we showed that the entropy rate of a random variable X is able to show the distribution of information along the scales. Even if the entropy rate, as defined in eq. (1.25), is related to all order statistics, it cannot probe finely the existence of a non zero skewness because of the effect of a very large 2-point correlation hiding the higher order statistics of the random variable distribution.

To probe more accurately the symmetry of the two points PDF of a signal X, we propose a conditioning procedure over its increments. Given a random variable X composed by a set of points x(i) we can define the increment signal δ λ X composed by the points δ λ (i) = x(i)x(iλ).

We define the signal X + , resp. X -, as the subset of points x(i) from X such that δ λ (i) = x(i)x(iλ) > 0, resp. δ λ (i) < 0. We then define the conditioned PDF p + , resp. p -, of the signal X + , resp. X -. Although signals X + and X -are one-dimensional and take their values in the same vector space S as X does, they contain some information provided by the increments, namely the sign of the local increment δ λ (i) associated with x(i). And then the PDF's of the signals X + and X -will be, generically, different from the PDF of X.

It is important to note here that if the statistics of the increments δ λ (i) of the signal X are skewed, then the joint PDF of (x(i), x(iλ)) is not symmetrical with respect to the origin. The reciprocal may not hold.

We then define the increment conditioned entropy rate h +,(m,λ) , resp. h -,(m,λ) , of the signal X as the entropy rate h (m,λ) of the conditioned signal X + , resp. X -. In practice, we compute the same quantity as before, defined by eq.(1.25), but using only a subset of all data points; this subset is obtained by retaining points x(i) with a given sign of δ λ (i). The conditioning is performed at the single date i, whatever the embedding dimension m is, so not looking at the (sign of) increments

δ λ (i -kλ), 1 ≤ k ≤ m -1.
The entropy rate considers an embedded signal of dimension m + 1, so even for the smallest m = 1, the entropy rate probes 2-points dependences (see chapter 3), between x(i) and x(iλ). In that case, the additional conditioning on the sign of the increment allows the conditioned entropy rate to probe the asymmetry of the joint PDF p(x(i), x(iλ)), which is related to the skewness of the increments.

If the joint PDF is symmetrical with respect to the origin, it is easy to check that p + (x) = p -(-x) and therefore h +,(m,λ) (X) = h -,(m,λ) (X).

On the contrary, if the statistics of the increments are skewed, then the joint pdf does not have the central symmetry and we may have h +,(m,λ) (X) = h -,(m,λ) (X).

To check the robustness of our results, we perform two different tests using skewed signals. First, because of the skewness of the increments, the fraction of points of X in subsets X + and X -can be quite different. As this may cause a difference in conditioned entropy rates, we recomputed h -,(m,λ) when imposing that X -has the same number of points as X + . To do so, we simply discarded the extra points from X -(randomly chosen). This procedure does not change the correlations of X -, nor the statistics and hence the pdf. Again, we will find the same significant difference between h +,(m,λ) and h -,(m,λ) . Second, we replaced the conditioning on the sign of the increments by a random sub-sampling of X, in order to obtain a subsampled signal X rand . In that case we will obtain h (m,λ) (X rand ) = h (m,λ) (X), as expected, although the number of points in X and X rand differ by a factor 2.

v(t -τ ) v(t)

Skewness of turbulent velocity increments

We compute the conditioned entropy rate, h +,(m,τ ) , resp. h -,(m,τ ) (section 6.3) with the embedding dimension always kept as one (m = 1). The size of the analysed signals is N = 2 19 , constant. The conditioned entropy rate is averaged over over 195 independent (1,τ ) (red) and h -,(1,τ ) (blue) as function of scale for turbulence data for embedding m = 1. The entropy rate h (1,τ ) (black) is reported for comparison. Bottom: difference (h +,(1,τ )h -,(1,τ ) ) (black) and standard deviation of h +,(1,τ ) (blue) and h -,(1,τ ) (red).

h (1,τ ) (X) h +,(1,τ ) -h -,(1,τ ) /σ h ln(1/τ ) ln(1/τ ) h h + Turbulence h - fBm h . h + h - std(h + ) std(h -) h + -h - std(h -) h + -h - std(h + )
realizations in the case of the Modane turbulent velocity data and the fractional Brownian motion and over 10 realizations in the case of the jet turbulence.

Grid turbulence:

Above we showed that the entropy rate of a turbulent velocity signal can be well described using only the autocorrelation function (blue line in figure 6.1), because the signal statistics are very close to Gaussian. Nevertheless, the statistics of the increments δ τ (t) ≡ x(t)x(tτ ) of a turbulent velocity signal can be far from Gaussian, especially for smaller scales τ . In particular, the PDF of increments is skewed (see eq.(5.16)). A non-vanishing skewness of the increments or a non-vanishing order-3 structure function, (1,τ ) (red) and h -,(1,τ ) (blue) as function of scale for turbulence data (Increasing Reynolds form left to right) for embedding m = 1. The entropy rate h (1,τ ) (black) is reported for comparison. Bottom: difference (h +,(1,τ )h -,(1,τ ) ) (black) and standard deviation of h +, (1,τ ) (blue) and h -,(1,τ ) (red). 3 } results in an energy cascade [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF][START_REF] Kolmogorov | On degeneration (decay) of isotropic turbulence in an incompressible viscous liquid[END_REF][START_REF] Kolmogorov | Dissipation of energy in locally isotropic turbulence[END_REF] (see section 5.1). The PDFs of the subsampled conditioned signals and the joint PDF of the signal are shown in figure 6.4. In order to probe this energy cascade, measurements of conditioned entropy rates are reported in figure 6.5.

R e = 463 h (1,τ ) (X) h +,(1,τ ) -h -,(1,τ ) /σ h ln(1/τ ) R e = 703 ln(1/τ ) R e = 929 ln(1/τ ) h - h h + h + -h - std(h + ) std(h -) h h + h - std(h + ) h + -h - std(h -) h + h h - std(h + ) std(h -) h + -h -
S 3 (l) ≡ E{(x(t) -x(t -τ ))
For the turbulent velocity signal, the two conditioned entropy rate give significant different results, with an almost constant difference around 0.1 ± 0.01 (compared to the standard deviation of h (m,τ ) (X ± ) which is around 0.03 ± 0.01), see figure 6.5.

Jet turbulence:

As in the precedent system, the difference between the two conditioned entropy rates, h +, (1,τ ) and h -, (1,τ ) , is significant for all the different reynolds numbers, see figure 6.6. For all the different Reynolds numbers, the behaviour of the entropy rate and the conditioned entropy rates is the same, being the constant value at large scales, and then the entropy of the velocity signal, the only variation.

Fractional Brownian motion Figure 6.5 shows how for the fBm, the conditioned entropy rates are indistinguishable and follow the same linear behavior as the entropy rate h (m,τ ) , as expected for a Gaussian (no-skewed) distribution.

Conclusions

We measured the information content of several turbulent velocity signal by computing the entropy rate as a function of the scale at which the signal is considered. We found that the distribution of information is reminiscent of the energy scaling, and we related it to the second order structure function. The entropy rate is able to separate properly the different domains as the second order structure function does. We argued that the entropy rate is more sensitive to dependences than the Power Spectrum (chapter 3), in particular because it can take into account higher order correlations, especially for large embeddings. For this reason, Information Theory perspective may give some new insight on turbulence.

In the entropy rate measure of the longitudinal velocity, the distinction between the contribution of the second order and the higher order moments (concretely the skewness) is unclear. As a consequence, we are not able to probe either the weak skewness of the turbulence signal or the larger skewness of its increments via entropy rate. We then designed a conditioning procedure of the data, based on the sign of the increments. Applying the entropy rate to this conditioned data, we were able to illustrate an effect of the skewness of the velocity increments.

The procedures described here are of general interest for the study of complex systems, especially those having multiscale dynamics, as can be found in, e.g., economy, ecology, neuroscience, and of course fluid dynamics. Given the plethora of laws governing such different systems, the model-free and nonlinear nature of information theory makes it a very interesting approach. In the particular case of turbulence, we showed not only that the entropy rate allows one to measure in all generality the information distribution amongst scales, in perfect agreement with known models, but also that a well-chosen conditioning of the data allows one to prove the existence of the energy cascade. Using information theory only, we recovered all classical characteristics of second order moment of turbulence, as well as the existence of an energy cascade via the third order moment.

CHAPTER 7 Rediscovering intermittency

As we explain in chapter 5, sections 5.1 and 5.2, turbulence displays intermittency which manifests in a non linear behaviour of the scaling exponents ζ(p) and also in the deformation of the PDF across scales. This deformation of the PDF has been conventionally quantified by the evolution of its flatness [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF], measured as the normalized kurtosis of the distribution: (δ l v) 4 / (δ l v) 2 2 = S 4 (l)/S 2 (l) 2 . At larger scales, about or above the integral scale L, the PDF of the velocity increments is almost Gaussian and has a flatness very close to 3. For smaller and smaller scales, the PDF is less and less Gaussian as the PDF of the normalized increments becomes wider and wider; therefore the flatness increases. Fine evolutions of the PDF, and hence intermittency, have been studied with the flatness, such as a rapid increase of intermittency when the scale is reduced down or below the Kolmogorov dissipative scale [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF]. S 2 evolves according to the 2/3 law predicted by K41 theory (see section 5.1), so, the kurtosis only involves one higher-order structure function, namely S 4 , and as such it does not describe the deviation of all the scaling exponents ζ(p) from their linear behavior in p. This is why, we propose in this chapter a measure of intermittency that involves all structure functions.

We propose to measure intermittency, interpreted as the deformation of a PDF which is Gaussian at large scales. To do so, we consider the Kullback-Leibler (KL) divergence [START_REF] Kullback | Information Theory and Statistics[END_REF] (see section 1.3.2) between the PDF and the Gaussian PDF defined with the same standard deviation. By comparing the PDF -defined by all its moments -and the Gaussian approximation of this PDF -defined by the second order moment only -we measure not only the growth of the pth order moment with respect to the variance, but also the evolution of all the moments with respect to the variance, i.e., we exhaustively characterize the deformation of the PDF. Measuring the intermittency with a KL divergence provides a generalization of measures such as flatness (p = 4), hyperflatness (p = 6), etc.

Although we propose to study turbulence as an application of our framework, our methodology is very general, and it does not require any a priori knowledge of the signal, neither any underlying model of the system that produced the signal. As such, it can prove a very powerful tool to analyze complex systems exhibiting power law behaviors or multiscale dependencies. We apply it to characterize intermittency in turbulence. This chapter is organized as follows. In section 7.1, we define our information theoretical measure of intermittency that involves Shanon entropy and a well chosen Kullback-Leibler divergence. In section 7.2, we compute this quantity for experimental measurements of the Eulerian velocity field in several setups and several Reynolds numbers. We then turn in section 7.3 to some phenomenological modelings in order to better understand and describe our observations.

Definitions

KL divergence from Gaussianity

Shannon entropy, H(X), of a process X of PDF p X (x) is the total information that defines the process, see section 1.3.1. Eq.(1.2) shows that entropy depends on all the moments of the PDF p X (x) except the first order one.

We know that a Gaussian process, X G , is uniquely defined by the prescription of its mean, variance, and two-point correlation function. Therefore, its Shannon entropy only depends on its variance σ X G , as eq. (1.38) shows. For a generic process X which is a priori non-Gaussian and has the variance σ 2 X , we define the "entropy under Gaussian hypothesis" H G (X) as the entropy that one would get assuming the process is Gaussian and using eq.(1.38):

H G (X) = 1 2 ln(2πeσ 2 X ) , (7.1)
where σ X is the standard deviation (std) of the generic process X. So, the "entropy under Gaussian hypothesis" of X is a measure of the entropy of a Gaussian PDF with same std as the real PDF of X.

If X is Gaussian, obviously H G (X G ) = H(X G ).
For any process X with probability density function p X (x), we can measure the difference between the "real" PDF p X (x) of X and the Gaussian approximation p G (x) using the Kullback-Leibler divergence defined in section 1.3.2:

K p X ||p G (X) = R p X (x) ln p X (x) p G (x) dx . (7.2)
Using the definitions of H(X) and H G (X), we have:

K p X ||p G (X) = H G (X) -H(X) ≥ 0 . (7.3) K p X ||p G (X)
is a measure of the distance from Gaussianity of the process X, i.e., the distance between the PDF p X (x) of X, and a Gaussian PDF p G (x) which has the same std. The maximum entropy principle [START_REF] Jaynes | Information theory and statistical mechanics[END_REF][START_REF] Jaynes | Information theory and statistical mechanics[END_REF] states that for a given standard deviation, the Gaussian PDF maximizes the entropy, see also [START_REF] Cover | Elements of Information Theory[END_REF]. So this distance is also a comparison between the total information needed to define the process and the total information defining the most ambiguous process with same std. The maximization of the entropy for the Gaussian case ensures that the difference H G (X) -H(X) is always positive, as expected for a KL divergence (see section 1.3.2) and vanishes only when X has a Gaussian distribution.

Distance from Gaussianity across scales

We analyze the process X at scale τ by studying its increments of size τ :

δ τ X(t) = X(t + τ ) -X(t) (7.4)
We note D τ (X) the KL divergence K p X ||p G (δ τ X) which measures the distance from Gaussianity of the increments at scale τ of a process X:

D τ (X) = K p X ||p G (δ τ X) = H G (δ τ X) -H(δ τ X) (7.5)
This quantity measures the deformation of the PDF of the increment as a function of the size τ of the increment: it quantifies the evolution of the shape of the PDF, which depends on all the moments of the process, except its mean.

Indeed, at each scale τ , the increment δ τ X has a different standard deviation. The larger the scale τ the higher the standard deviation. So, changing τ changes quantitatively the entropies H(δ τ X) and H G (δ τ X), which both depend strongly on the standard deviation. Subtracting the two entropies eliminates most of this quantitative variation because the std is by construction the same in both expressions H G (δ τ X) and H(δ τ X)). D τ (X) thus only measures subtle and delicate evolutions of the shape of the PDF than the trivial rescaling induces by the std.

In the specific case of turbulence, the PDFs of the increments of size equal or larger than the integral scale L are almost Gaussian. As a consequence, we expect that the distance from Gaussianity D τ (X) tends to zero when τ approaches the integral scale. Conversely, it is expected to increase in the inertial range down to the dissipative scale where it should increase faster [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF]. Our distance from Gaussianity should therefore be able to probe intermittency of turbulence by measuring the deformation of the PDF of velocity increments.

Methodology

In order to compute accurately H from experimental data, we use the estimator described in section 2.2. We chose the usual value k = 5 which is large enough to estimate the Shannon entropy correctly within a reasonable computational time.

Following Theiler [START_REF] Theiler | Spurious dimension from correlation algorithms applied to limited time-series data[END_REF], when computing the entropy of δ τ X(t), we only retain data points separated in time by a delay time τ max , defined as the size of the largest increment that we compute. This prescription has two benefits. Firstly, two successive points of the subsampled dataset are uncorrelated, because the increments of size τ are typically correlated over a time τ ≤ τ max . Secondly, the number N of points used in the computation of the entropy of δ τ X is independent of τ , so the bias due to finite size effects is constant when τ is varied.

To compute the entropy under Gaussian hypothesis H G , we compute the standard deviation of X and then use eq. (1.38).

In the remainder of this chapter, all quantities are computed using N = 512 points (Bias Ĥ 0.01), τ max = 4096, so signals with a total of N τ max = 2 21 points. We also average our results over independent realizations, in order to compute the standard deviation of the quantities and provide error bars to the estimations. We use 12 realizations for experimental signals and 8 realizations for synthetic processes.

ln(E) (a) ln(τ /T ) ( (δ l v) 4 / (δ l v) 2 2 )/3 (b) H(δτ V ) (c) H(δτ V ) H G (δτ V ) ln(τ /T ) Dτ (V ) (d) -8 -6 -4 -2 0 -8 -6 -4 -2 0

Application to turbulent velocity signals

In figure 7.1, we present the analysis of the Modane data (section 6.1). In the left column, we report the classical viewpoint and compare it to the information theory viewpoint in the right column. We first plot the power spectrum of the velocity signal V as a function of the inverse frequency 1/f = τ in figure 7.1(a) : it shows the distribution of energy across scales following the well known 5/3 Kolmogorov law. In order to measure the deformation of the PDF of the velocity increment when the scale τ is varied, we follow Frisch [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF], and compute the flatness as the kurtosis (δ τ v) 4 / (δ τ v) 2 2 of the velocity increments normalized by 3, the kurtosis of a Gaussian PDF. Results are reported in figure 7.1(b). For τ T , i.e., l L, the flatness is 1, as expected for a Gaussian PDF. Reducing τ , the flatness increases. When τ is smaller than the dissipative scale [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF], the increase of the flatness is sharper. Three different regions can be distinguished in both figures: integral, inertial and dissipative.

The right column of figure 7.1 is devoted to the information theory viewpoint. We first plot the entropy of the increments in figure 7.1(c) and compare it with the PSD in Fig. a. We then plot D τ (V ) in figure 7.1(d) and compare its behavior in τ with the flatness.

In figure 7.1(c) we see that the entropy of the increments (H(δ τ V )) increases with τ . The larger the scale, the higher the total Shannon information needed to completely characterize the increments. We can distinguish three different ranges with different dependence of the entropy on the scale. For the large scales, larger than the integral scale, the entropy reaches its highest value and is then constant. So, the most disorganized or complex scales -the ones requiring more information to be completely characterized -are the scales in the integral domain. Within this region, the characterization of the scale does not require more entropy when the size of the increment increases . A linear behavior of the entropy in ln(τ /T ) is found in the inertial region, τ ∈ [10, 400] samples i.e. τ ∈ [0.0004, 0.016] seconds, l ∈ [0.0082, 0.328] meters. The complexity of the scales, as measured by H(δ τ V ), decreases linearly in ln(τ /T ) between the integral and the Kolmogorov scales. For the smallest scales, below the Kolmogorov scale -which we can measure at ln(τ /T ) ≈ -5 -we observe a steeper decrease of the disorganization when the scale decreases. So, using the entropy of the increments, we recover the same qualitative results as using the entropy rate of the velocity measures (chapter Chap7).

Both entropies H(δ τ V ) and H G (δ τ V ) in figure 7.1,c) are indistinguishable in the integral domain. They start to separate when they enter into the inertial region. In figure 7.1,d, we plot the difference between these two entropies, which, according to eq.(7.5) is the distance from Gaussianty D τ (V ). Starting from 0 at scales larger than the integral one, it increases when the scale decreases. The vanishing of D τ (V ) for scales larger than the integral scale implies that the PDF of the velocity increments is almost Gaussian, as expected. Below this integral scale the PDF starts to deform, and becomes less and less Gaussian when the scale decreases. The evolution of D τ (V ) is almost linear between the integral and the Kolmogorov scales. Finally, in the dissipative range, we observe an abrupt deformation of the PDF, in perfect agreement with the rapid increase of the flatness in figure 7.1,b). [START_REF] Chevillard | On the rapid increase of intermittency in the near-dissipation range of fully developed turbulence[END_REF] In the four sub-plots of figure 7.1, the three different domains of turbulence are distinguishable: integral, inertial and dissipative. Figure 7.1c) allows us to interpret these three domains in terms of organization and complexity of velocity increments. Figure 7. 1,d) shows that the KL divergence allows us to quantify the evolution of intermittency amongst scales τ . We not only recover the three different ranges with our measures based on information theory, but the qualitative behavior of intermittency in each domain is also in perfect agreement with previous studies. Moreover, our measure of intermittency doesn't depend on a specific ratio between selected moments of the PDFs like the kurtosis. D τ (V ) takes into account all the moments defining the PDFs: this makes our KL distance from Gaussianity across scales a good candidate for a quantitative measure of intermittency.

In figure 7.2, we have compiled information theory results for all the experimental signals presented in section 6.1, in order to study the influence of the Reynolds number. The entropy as a function of the scale is reported in figure 7.2(a); we observe how the size of the inertial range varies with the Reynolds number, with the Kolmogorov scale increasing when the Reynolds number decreases. This classical behavior of the Kolmogorov scale is also recovered with the KL divergence, represented in figure 7.2(b). The steeper slope -that indicates the dissipative domain -appears at higher scales when the Reynolds number is lower; we recover the dependence of the Kolmogorov scales with the Reynolds number.

The behaviors of both the entropy and the distance from Gaussianity are qualitatively the same for different experimental setups and for any Reynolds number. The dependence of the entropy H(δ τ V ) of the increments is, at first order, in agreement with the K41 theory: we recover the scaling law in the inertial domain [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF]. The KL divergence D τ (V ) then enlightens the deformation of the PDF across scales, which is qualitatively compatible with the K062 theory and hence the intermittency in turbulence.

D τ (V ) R λ = 208 (b) R λ = 2500 (a) R λ = 703 ln(τ /T ) R λ = 929 R λ = 463 R λ = 89 H(δ τ V ) -8 -4 0
(τ /T ) = ln(l/L). b) KL divergence D τ (V ) = H G (δ τ V ) -H(δ τ V ).
Different experimental signals with various Reynolds numbers have been used.

Modeling

In order to get some insight on the quantitative results obtained with our Kullback-Leibler divergence D τ , we now turn to some theoretical descriptions of the inertial domain of fully developed turbulence.

First, we study different processes generated to imitate the behavior of turbulence in the inertial region according to a corresponding model. Amongst the simplest, popular and most important is fractional Brownian motion (fBm) [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF][START_REF] Mandelbrot | Fractional brownian motions fractional noises and applications[END_REF] which, as a monofractal process, doesn't display intermittency. We also explore multifractal processes, which exhibit intermittency: Multifractal Random Walk (MRW) [START_REF] Bacry | Multifractal random walk[END_REF][START_REF] Leonarduzzi | Análisis multifractal basado en coeficientes ondita líderes: selección automática del rango de escalamiento, formalismo multifractal basado en plíderes y aplicación a señales biomedicas[END_REF], Random Wavelet Cascade (RWC) with log-normal [START_REF] Leonarduzzi | Análisis multifractal basado en coeficientes ondita líderes: selección automática del rango de escalamiento, formalismo multifractal basado en plíderes y aplicación a señales biomedicas[END_REF][START_REF] Arneodo | Random cascades on wavelet dyadic trees[END_REF] or log-Poisson distribution of multipliers [START_REF] Leonarduzzi | Análisis multifractal basado en coeficientes ondita líderes: selección automática del rango de escalamiento, formalismo multifractal basado en plíderes y aplicación a señales biomedicas[END_REF][START_REF] Arneodo | Random cascades on wavelet dyadic trees[END_REF][START_REF] She | Universal scaling laws in fully developed turbulence[END_REF]. We then examine the propagator formalism [START_REF] Castaing | Velocity probability density functions of higher reynolds number turbulence[END_REF], a phenomenological model that provides an analytical expression of the PDF of the velocity increments [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF].

Synthetic processes

We now briefly present the different processes that we numerically generated, according to the prescriptions of section 5.2.

Monofractal process

We use the spectral procedure presented by Helgason to synthesize a fractional Brownian motion (fBm) [START_REF] Helgason | Fast and exact synthesis of stationary multivariate gaussian time series using circulant embedding[END_REF] with Hurst exponent H = 1/3, as introduced by Kolmogorov [START_REF] Kolmogorov | The local structure of turbulence incompressible viscous fluid for very large reynolds number[END_REF].

Log-normal multifractal processes We use two different synthetic processes with log-normal statistics: a Random Wavelet Cascade (RWC) [START_REF] Leonarduzzi | Análisis multifractal basado en coeficientes ondita líderes: selección automática del rango de escalamiento, formalismo multifractal basado en plíderes y aplicación a señales biomedicas[END_REF][START_REF] Arneodo | Random cascades on wavelet dyadic trees[END_REF] and a Multifractal Random Walk (MRW) [START_REF] Bacry | Multifractal random walk[END_REF]. Multifractality requires the existence of an integral scale T , from or towards which the PDF evolves. For both processes, we impose the integral scale T to be equal to the size of the signal.

Log-Poisson multifractal process

We use a RWC with log-Poisson statistics [START_REF] Arneodo | Random cascades on wavelet dyadic trees[END_REF]. Again, our synthesis fixes the integral scale T to the size of the generated signal.

Results

Classical multifractal analysis offers a way to estimate the log-cumulants c 1 and c 2 , but fails to estimate c 3 and higher order log-cumulants. It can therefore be interpreted as projecting the different models onto their log-normal approximation, with varying (c 1 , c 2 ). For example, the multifractal analysis of a realistic log-Poisson model of turbulence leads the couple of values given in table 7.1, and no additional higher order log-cumulant. As a consequence, such an analysis is not able to discriminate which process -lognormal or log-Poisson -better represents turbulence. For this reason, we compute the KL divergence D τ which takes into account all moments of the PDF of increments, and hence higher order log-cumulants [START_REF] Venugopal | Revisiting multifractality of high-resolution temporal rainfall using a wavelet-based formalism[END_REF], in order to obtain a finer analysis of the inertial domain of turbulence. In figure 7.3(a) we plot for the four synthetic signals the entropy H(δ τ X) as a function of ln(τ /T ), the logarithm of the scale. We also plot the entropy under Gaussian hypothesis, H G (δ τ X), but it is indistinguishable from H(δ τ X).

From eq.(5.28), it is possible to show that for a monofractal, and then non-intermittent (K41 theory), process:

H(δ τ X) = H(δ τ 0 X) + H ln(τ /τ 0 ) . (7.6)
For any process, the entropy under Gaussian hypothesis H G is computed using eq.(7.1). It involves the second order moment S 2 (τ ) only, which we express using eq. (5.24) as

S 2 (τ ) = σ 2 τ = σ 2 T τ T ζ (2)
. 2) are obtained by classical multifractal analysis. Last line reports the slopes ∆ ln(τ ) H G (δ τ X) of the entropy H G (δ τ X) as a function of ln(τ /T ), for the four different models, which according to eq.( 7.7) provides another estimate of ζ(2)/2.

H(δ τ X) (a) H(δ τ X) H G (δ τ X) ln(τ /T ) D τ (X) (b)
We then obtain the dependence of H G on the scale τ :

H G (δ τ X) = H G (δ T X) + ζ(2) 2 ln (τ /T ) . (7.7)
In figure 7.3(a), we observe that the slope of the curves, which should be ζ(2) 2 is very similar for all processes: we report in table 7.1 the different values we measured, and compare them to the prescribed value (1/3 for fBM and 0.345 for all three multifractal processes). The distribution of information along the scales for the four different models is in agreement with the prescribed Kolmogorov K41 scaling [START_REF] Granero-Belinchon | Scaling of information in turbulence[END_REF].

Up to this point, looking at the entropies, the four models cannot be distinguished in the inertial domain. In figure 7.3(b) we plot the Kullback-Leibler divergence D τ (X) as a function of ln(τ /T ) = ln(l/L), for scales ranging from τ /T = 1/2 24 to τ /T = 4096/2 24 where the integral scale is T = 2 24 .

For a monofractal process, the entropy is given by eq.(7.6), and the entropy under Gaussian hypothesis is given by eq.(7.7) with

H = ζ(2)/2, so D τ (X) = H G (δ τ X) - H(δ τ X
) is constant and does not depend on the scale τ . If the monofractal process has Gaussian statistics -which defines the fBm -D τ (X) = 0 by construction. Looking at figure 7.3(b), D τ for the fBm is not exactly zero; this is due to the bias in the estimation of H(δ τ X) and H G (δ τ X). This bias is constant across scales, because our procedure was built to use a constant number of points in the range of τ we use.

For the three multifractal processes, D τ (X) decreases monotonically when τ increases, and tends to zero when the scale tends to the integral scale. So in the three multifractal models, the PDF of the increments deforms into a Gaussian PDF when approaching the integral scale. Moreover, in figure 7.3(b), we observe that the three processes, which indeed have different statistics, do not converge to zero in the same way. The distance from Gaussianity D τ , by involving all the moments of the probability distributions, is able to reveal fine differences between processes. The synthetic processes used above are good representations of the inertial range only. They do not properly take into account either the dissipative nor the integral scales. Nevertheless, the synthesis imposes an effective integral scale that corresponds to the size of the generated signal. In order to study more precisely the deformation of the PDFs at large scale, we now turn to descriptions that explicitly involve the integral scale.

Phenomenological model : the propagator formalism

First introduced by Castaing [START_REF] Castaing | Velocity probability density functions of higher reynolds number turbulence[END_REF], the propagator formalism describes the statistics of the Eulerian velocity increment δ l v as identical, in the probabilistic sense, to the statistics of the product of two random variables: the large scale fluctuations σ L δ and the propagator (l/L) h . The large scale fluctuations are supposed Gaussian, with standard deviation σ L , and δ is therefore a Gaussian variable with unit variance. The propagator deforms the large-scale statistics when the scale l is reduced below the integral scale L. In the simple situation where no dissipative scale is taken into account, and where the propagator is supposed independent of large scale statistics, one can write formally the PDF of the Eulerian velocity increments δ l v = σ L (l/L) h δ as [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF]:

p δ l v (δ l v) = ∞ -∞ 1 σ L l L -h P δ δ l v σ L l L -h P h [h]dh (7.8)
where h is the Hölder exponent. We have noted P δ (δ) and P h (h) the probabilities of the independent random variables δ and h. The PDF P h (h) depends only on the singularity spectrum D(h). See [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF] for a detailed explanation. We integrate numerically eq.(7.8) to get the PDF of the increments δ l v, and then compute the KL divergence D τ by introducing this PDF in eq.(7.5) for several singularity spectra, either log-normal or log-Poisson. log-normal model . We varied the value of the log-cumulant c 1 and didn't observe any dependence of D τ on c 1 . On the contrary, varying c 2 strongly changes the convergence. Results are presented in figure 7.4(a). We observe and report in figure 7.4(b) that curves for different values of c 2 can be collapsed into a single curve when plotted as a function of c 2 ln(τ /T ) = c 2 ln(l/L).

To understand this scaling behavior, we performed a saddle-node expansion of expression (7.8) in the log-normal case, and obtained the following simplified expression for the PDF of the normalized increments y = δ l v/σ l at scale l:

p y (y) = e 3 2 c 2 x √ 2π e -2W +W 2 8c 2 x √ 1 + W (7.9)
where we have noted x ≡ln(l/L) the logarithmic scale, and W the value of the Lambert W-function of argument 2c 2 xy 2 e 4c 2 x . Eq.(7.9) is a non-Gaussian PDF which converges to the Gaussian PDF of variance σ 2 L when x → 0. From eq.(7.9), the PDF of the increments only depends on ln(l/L) = ln(τ /T ) and c 2 via the product c 2 ln(l/L). As a consequence, the entropy of the increments depends on the scale l as c 2 ln l/L only. This implies that the KL divergence D τ for the log-normal process has the scaling observed in figure 7.4(b).

log-Poisson model

We varied independently γ, λ and β. We didn't observe any change of D τ when γ was varied. This can be understood as γ only changes the value of c 1 (see eq.(5.31)), which does not impact D τ . Varying λ changes the convergence, as this amounts to change c 2 (see eq.(5.32)), but we observe again that D τ depends only on c 2 ln(τ /T ), see figure 7.5(a). This can be understood by noting that all log-cumulants are linear in λ; thus varying λ amounts to a change of c 2 while keeping higher order cumulants within the same ratio. On the contrary, varying β has more impact on the convergence, and the re-scaling in c 2 ln(τ /T ) is then not perfect, albeit still relevant, see figure 7.5(b). This can be understood by noting that changing β does not only change c 2 , but also the ratio of all higher order cumulants.

Comparison between models The rescaling in c 2 ln(τ /T ) -which absorbs most, if not all, the dependence of D τ on c 2 -allows a direct comparison of models. As can be seen in figure 7.5, all curves obtained with the log-Poisson propagator model clearly differs from the ones obtained with the log-normal propagator, especially for smaller scales. This probably results from the presence of higher order log-cumulants c p , p > 2 in the log-Poisson propagator. As a consequence, whatever the choices of the propagators parameters, the KL divergence behaves very distinctly in the log-normal and log-Poisson models.

Comparison with turbulence data Amongst open questions regarding statistical descriptions of Eulerian turbulence is the choice of a log-normal or log-Poisson modeling of its multifractal nature. We of course want to address this issue, and we explore both the flatness and the KL divergence to compare the two propagator models with experimental data in figure 7.6. In both models, parameters are set to the values acknowledged for turbulence (see section 5.2). For experimental data, we remove the bias from the KL divergence estimation by subtracting the small constant value that we measured for a fBm signal, see figure 7.3(b). As the c 2 value for turbulence is a priori unknown, we do not rescale the x-axis with c 2 . Let us remark though that the c 2 value we have used in the log-normal propagator (c 2 = 0.025) is exactly the one measured in experimental data, using multifractal analysis [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF].

In the inertial range, although the flatness behaves differently for the two propagator models, the difference is small and remains within the error bars of the estimation performed on experimental data. On the contrary, when looking at the KL divergence, our results show a much better agreement of the log-normal model with the experimental data. Although this may be due to the very appropriate choice of c 2 in the log-normal model, the log-Poisson model does not allow such a choice and fixes all the log-cumulants [START_REF] She | Universal scaling laws in fully developed turbulence[END_REF]. As a consequence, we can state that the deformation of the experimental velocity increments PDF in the inertial range is better modelled by a multiplicative cascade with log-normal multipliers.

Similar results are obtained by using wavelet multifractal analysis, which allows to probe both sides of the singularity spectrum, see figure 5.3 b). Measures of D(h) for experimental turbulent velocity together with log-normal and log-Poisson models suggests that log-normal can model turbulence better than log-Poisson does [START_REF] Wendt | Wavelets leaders and bootstrap for multifractal analysis of images[END_REF][START_REF] Arneodo | Revealing a lognormal cascading process in turbulent velocity statistics with wavelet analysis[END_REF]. Nevertheless, the results obtained by performing this analysis depend a lot on the chosen parameters in the estimation, and then the interpretation is difficult.

In the dissipative range, i.e., for smaller scales ln(τ /T ) -5, we observe a rapid increase of D τ for the experimental data, which the two models (continuous lines in figure 7.6) fail to reproduce. This is expected, as both propagator models were proposed to describe the inertial range only, and as such do not incorporate any modeling of the dissipative scales. To probe the dissipative range, we use the extension described in [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF], and indicate the results for both propagators with dashed lines in figure 7.6. The rapid increase of intermittency in the dissipation range is captured by the flatness and the KL divergence. Although great care should be taken when commenting on the lowest ln(τ /T ) The dashed (blue) and dotted (red) lines represent the modified models that take into account the dissipative range [START_REF] Chevillard | A phenomenological theory of eulerian and lagrangian velocity fluctuations in turbulent flows[END_REF].

( (δ l v) 4 / (δ l v) 2 2
accessible scales in the experimental data (mainly due to the acquisition process), we observe once again that the KL divergence indicates -much more clearly than the flatness does -that the log-normal propagator model is closer to the experimental observations.

Discussion and Conclusions

We have measured the Shannon entropy of the Eulerian turbulent velocity increments, and studied its dependence on the scale. We have recovered the same qualitative results as with the entropy rate of the velocity (chapter 6), in perfect agreement with K41 theory. A closer look at the entropy of the increments, and especially a comparison with its Gaussian approximation, which only takes into account the variance of the signal -exactly as the PSD does -allows a much finer description and in particular, a measure of intermittency, as introduced in KO62.

We have proposed a quantitative measure of intermittency. Although some quantities were already used as an intermittency coefficient, most, if not all, were ratios of structure functions [START_REF] Frisch | Turbulence: The Legacy of A.N. Kolmogorov[END_REF], and as such, they were depending on the chosen ratio: flatness, hyper-flatness [START_REF] Anselmet | High-order velocity structure functions in turbulent shear flow[END_REF] or higher order ratios. We interpret intermittency as the distance from Gaussianity, and measure it as D τ , the Kullback-Leibler divergence between the complete PDF p(δ τ V ) and its Gaussian approximation p G (δ τ V ); the first involves all the statistical moments while the second one only depends on the variance. Our measure of intermittency, by comparing complete PDFs, takes into account all the moments of the distributions, which leaves no room for ambiguity on the choice of the moments.

We have checked the robustness of our approach by analyzing several experimental datasets, from two different experimental setups, and with varying Reynolds numbers.

The quantity D τ is not only able to measure intermittency in turbulence, but also to discriminate very easily monofractal from multifractal processes. Furthermore, the evolution of D τ with the scale depends on the process: this provides a much more precise characterization of the process than the bare set of log-cumulant values (c 1 , c 2 ) given by a regular multifractal analysis. This may be exploited to discriminate log-Poisson from log-normal models of intermittency in turbulence.

We have investigated the dependence of D τ on the log-cumulants. D τ does not depend on c 1 and we have captured its dependence on c 2 , and especially how it affects the convergence to 0 at large scales. Because D τ appears to mainly depend on c 2 ln(τ /T ), we can state that the speed of the deformation of the PDF, starting from a Gaussian at large scale L, depends on c 2 . For a given scale l/L, or equivalently τ /T , the deformation of the PDF, and hence the intermittency, is an increasing function of c 2 . Conversely, for a fixed value of c 2 , the influence -or reminiscence-of the integral scale persists down to scales l/L smaller and smaller when c 2 is reduced. Because the typical c 2 of turbulence is small, the influence of the integral scale persists in the inertial domain, down to the dissipative domain, unless the Reynolds number tends to arbitrarily large values. We have shown that D τ depends on higher order log-cumulants c p , for p > 2, by looking at the special case of log-Poisson statistics (figure 7.5). The dependence seems weak, but is nevertheless present, and could be exploited.

Our measure of intermittency substantially differs from the existing measure involving the flatness. KL divergence gives a sharper contrast between propagator models; this is not surprising as KL divergence offers a complete perspective on the PDF and all its moments. In addition, the errorbars of the KL divergence of experimental data are sufficiently small to allow a quantitative comparison of experiments with model predictions. This comparison was not instructive using previously existing tools. Our results show that KL divergence offers quantitative arguments in favor of a log-normal description of the propagator for modelling the distribution of Eulerian velocity increments across scales.

ln(τ ) (b) D τ (M ) h (1,τ ) (M ) -H ln(τ ) (c) (d) (a) H(δ τ M ) h (1,τ ) (M ) ln(τ )
Although we have put a strong emphasis on turbulence, we want to point out that our approach is extremely general and should find successful applications in many other fields. It should prove particularly interesting for non-Gaussian processes, the most common in Nature and Society. Figure 7.7 shows the performance of this approach (right column), together with the entropy rate across scales h(m,τ)

T presented in chapter 4 (left column), when analysing the non-stationnary processes presented in chapter 4. Both methodologies describe the processes in a very similar way. However, while the methodology presented in chapter 4 requires the previous knowledge of H to obtain figure 7.7 b), the approach presented in this chapter doesn't require any a priori knowledge of the process.

We can conclude that, the self-similarity properties of any process can be analyzed combining H(δ τ ) and D τ . The local intermittency measure that D τ provides can be used to characterize the process at any scale.

CHAPTER 8 Wiener Causality across scales of turbulence

The complete characterization and understanding of the energy cascade in turbulence is far from being reached. Is this energy cascade due to the splitting of large eddies into smaller ones? How much time does the energy take to go from one scale to another? Does this time depend on the range of scales? Within the current research frameworks the discussion is open, and the conclusions are not always clear. So, we propose in this chapter to look for conclusive results using information theory, and more precisely to use transfer entropy (section 1.6.2), to analyse turbulent velocity measures and synthetic processes modelling turbulence. We then propose to study interactions between scales in turbulence from this new viewpoint.

Transfer entropy, defined in eq. (1.31), is a measure of Wiener causality between time series, and has been frequently interpreted as a measure of information flows, (section 1.6). Then, the question naturally arises, do information flows between scales of turbulence exist? or, at least, are there Wiener causality relationships between different scales of turbulence? And if we can measure them, do they bring information about the energy cascade?

In order to probe the existence of Wiener causality relationships between scales of turbulence, we study different time series: on one hand, the experimental grid turbulent velocity measures presented in section 6.1, on the other hand fractional Brownian motions, (section 4.2), with different Hurst exponents. The third order structure function of turbulent velocity data is non-zero and thus it shows the existence of an energy cascade. This is not the case for fBm, where no energy cascade takes place.

Definitions of the scale

In order to measure causality relationships between scales of a time series, first, we have to define these scales. This step is very delicate, as the definition of the scales can influence the transfer entropy measures between them, and so, distort the results. We define the scales of a time-series X(t) by applying a transformation f τ on it. Then, we define the diferent scales of X(t) as f τ (X(t)) where τ indicates the size of the scale. Therefore, the scales are also time-series. However, the transformation f τ can itself introduce Wiener causality relationships between its different versions f τ , f τ . Relationships that are unconnected to the physical system.

We can generate the different scales of a time-series by using different transformations: increments, wavelets, filtering, embedding ... Each of these transformations lead to different time series defining the scales of the initial time series X(t). As a consequence, even if the transfer entropy interpretation remain constant, concerning Wiener causality between time series, the question is: how much part of the measure is due to the way of generating the scales? Can we really talk about Wiener causality between scales or just between these transformed time series?

To answer these questions, we perform a complete study of transfer entropy between scales of a time series by using three different scale definitions: increments, filtering and embedding. Having seen the wide range of possibilities in the definition of scales, we have decided to use increments, completely localized in the direct space, filtered signals, completely localized in the Fourier space, and embedded signals, in which the scale is defined by a couple of points of the initial time series X(t) separated by τ .

Increments

From an initial time-series, we generate different scales by generating increments of different sizes. The length of the increments indicates the size of the scale. The corresponding transformation f τ (X(t)) is defined as:

f τ (X(t)) = δ τ X(t) = X(t) -X(t -τ ) (8.1)
Then, for two different scales we have two increments δ τ 1 X(t) = X(t) -X(tτ 1 ) and δ τ 2 X(t) = X(t) -X(tτ 2 ) defined in order to fix the same present t, for both scales, see figure 8.1 (left).

In the example of figure 8.1 (right), transfer entropy definition of eq.(1.31) is a calculation of the information of the red increment (δ τ l X(t + lag)) that is contained in the small increment (δ τs X(t)) but not in the large one (δ τ l X(t)).

We define lag as the time delay which separates a future sample at t + lag from the present one at t. We interpret it as a time of interaction. With the above definition of transfer entropy, the delay lag is the same between the caused increment (δ τ l X(t + lag)) and both, the condition (δ τ l X(t)) and the causal increment (δ τs X(t)). We can see in figure 8.1(right) the effect of coincidence of increments when lag = τ lτ s and lag = τ l , that will define the emplacement of the peaks of transfer entropy between scales.

In the definition of transfer entropy of eq.(1.33), lag only indicates the distance between the caused (δ τ l X(t + lag)) and the causal increment (δ τs X(t)). With this definition, the distance between the caused (δ τ l X(t + lag)) and the conditioned increment is always the smallest possible. The effect of coincidence of increments, when lag = τ lτ s and lag = τ l , still defines the emplacement of the peaks of transfer entropy between scales.

This definition of scale allow to precisely localize the position and size of the scale in the direct space, nevertheless the localization of the scale in the Fourier space is unclear.

Filtering

Another way to define a scale is to filter the initial time series. In order to do that, three different kind of causal filter transformations have been used: Low pass (LP), defined in eq.( 8.2), High Pass (HP), defined in eq.( 8.3) and Band Pass (BP) filter, defined as the addition of a High and a Low Pass. The idea behind this generation of scales is to completely localize the scales in frequency, but not at all in space, so, the contrary to the increments case.

F (f ) LP = 1 1 + i f f c (8.2) F (f ) HP = i f fc 1 + i f f c (8.3)
where f c is the cutoff frequency, indicating the frequencies contained in the filtered signal.

In the case of the LP filter the frequencies contained in the signal are the frequencies smaller than fc, in the case of HP filter the frequencies higher than f c, and in the case of a BP filter the frequencies between f c HP and f c LP . For all the above filters, LP, HP and BP, we consider that the higher the cutoff frequency the smaller the scale generated.

Embedding

Increasing the dimension of the initial time series also allows to define a scale. We use the embedding procedure proposed by Takens:

f τ (X(t)) = X (2,τ ) (t) = (X(t), X(t -τ )) (8.4)
Thus, we define a scale τ as a 2-dimensional vector composed by two samples of the initial time series separated by τ . As in the case of the increments, the effect of coincidence of samples, will define the emplacement of the peaks of transfer entropy between scales (figure 8.1 (right)).

Results for different scale functions

In this section, we characterize the information flow between scales for three different systems: Modane velocity data, a fBm with H = 0.3 and a fBm with H = 0.5 and for the three different ways of generating the scales presented before.In order to analyse the dynamics of information in a given system, we choose an observable of the system, in this section velocity, and we build the scales by applying the corresponding function f τ to the observable. Then, we calculate the TE between two scales, from the larger to the smaller and vice versa.

Increments

We build the scales of V (t) by generating the increments of the observable. We calculate the TE, defined in eq. (1.31), where X = δ τ 1 V (t) and Y = δ τ 2 V (t). Figure 8.2 shows that TE is different from zero only from small scales to large ones, and is zero from large to small scales. This can be seen for turbulent velocity in figure 8.2 a), but it's general for any system, see figures 8.2 b) ,8.2 c). For all the studied systems: turbulence and fBm with different Hurst exponents the small scales Wiener cause the large ones.

Figures 8.2 a) b) and c) also show the behavior of the time of interaction between scales (lag). We find that the maximum intensity of the interaction between two scales appears at time delay lag = |τ lτ s |. The time of interaction between two scales depends directly on the difference of the length of the scales; the larger the diference of length the longer the time of interaction. This result is directly related to the definition of the scales as increments.

Another general result is, the closer the scales, the strong the interaction, i.e. the interaction between two scales is stronger when the sizes of the scales interacting are We now consider separately each system. It's well known that a fBm with Hurst exponent H = 1/3 models the distribution of energy of turbulence in the inertial range. We compare the results obtained for two interacting scales of turbulent velocity, both of them in the inertial region, and the same interacting scales of a fBm with Hurst exponent H = 1/3. We find the same time of interaction for both systems, as we expected from the coincidence of increments at lag = τ l -τ s . In addition, the maximum of the intensity of the interaction is the same for both systems. However, the way of reaching the maximum value of the interaction is different, being smoother for the turbulent system, see figure 8.2 b). This smoothness can be due to the dissipation existing in turbulence but not in the fBm, or it can be due to intermittency or even to the energy cascade. Another observation from figure 8.2 is the non-symmetry of the interaction between the two sides of the maximum of the interaction, i.e. the way of reaching the maximum is different from the left and from the right, but it seems to be symmetric when we are very close to the maximum for both systems turbulence and fBm.

Max(T E (1,1) (δ τs v → δ τl v) lag ) |τ l -τ s | 0 20
If we compare two fBm with H = 1/3 and H = 1/2 we see that there are differences in the measures, see figure 8.2 c). The Wiener causality relationships between scales depend on the self-similarity exponent of the system.

Scales obtained with filtering

In this section we generate scales by filtering signals. We filter the same turbulent velocity signal analysed above, and the same fBm with H = 0.3.

We generate the scales using three different filters: low pass, band pass and high pass. Using LP and BP filtering we obtain that the small scales cause the large ones for both In this case, the maximum of the TE between scales is always at lag = 0 and the amplitude of the maximum decreases when the scales are closer, see figure 8.6. When scales are obtained by filtering, the cut-off frequency associated with the large scale defines the decay length of the TE measure, see figure 8.4 and figure 8.5.

(b) lag T E (1,1) (F fc 1 (v) → F fc 2 (v))
ln(1/f c) ln(1/f c) T E (1,1) (F fc 1 (v) → F fc 2 (v))

Scales obtained by embedding

The third way of generating the scales is by embedding the data. As in the case of the increments, the maximum amplitudes of TE measures are localized for lag values where a sample of the caused vector coincide with a sample of the causal vector. We find qualitatively similar results, with an information flow from small to large scales, (figure 8.7). Even if this time a new peak appears in the TE from large to small scales, this new peak is much smaller than the peak from small to large (figure 8.7). The qualitative results are the same for turbulence and both fBm, but we can observe some differences in the amplitudes and the shapes of the peaks.

Results for different observables of turbulence

In this section, we study the causality relationships between scales of different observables of turbulence. We use increments as scales and we analyse: the turbulent velocity increments δ τ v(t) analysed above, the increments of the square of the velocity δ τ v 2 (t), and the square of the velocity increments (δ τ v(t)) 2 .

x τ = δ τ v lag (a) Figure 8.8, shows the TE between scales for these three observables of turbulence. The qualitative behavior of the causality is common for the three analysed observables: we find that small scales cause large scales, causality from large scales to small scales is null, the maximum amplitude of the causality appears at lag = τ lτ s , the causality is more important when the scales are closer. Figure 8.8, shows that the TE between the increments of the velocity δ τ v(t) and the TE between the increments of the square of the velocity δ τ v 2 (t) are almost identical. However, the TE between the square of the velocity increments (δ τ v(t)) 2 presents lower values. We find the same results for fBm with both Hurst exponents.

x τ = δ τ (v 2 ) (c) T E (1,1) (x τ1 → x τ2 ) lag (b) lag x τ = (δ τ v) 2

Conclusions and perspectives

In this chapter, we study several signals: velocity and square of the velocity, obtained from various complex systems: turbulence and fBm with two different Hurst exponents H. We generate the scales of the studied processes using three different methods: computing the increments of the process, filtering the process, or embedding the process. Next, we compute the transfer entropy between scales of the time process, and we find for any system and any scale function the same qualitative result: transfer entropy is larger from small to large scales than from large to small scales. We interpret transfer entropy as a measure of the Wiener causality between two time series, and eventually as a measure of information flows. With this interpretation of TE, our results indicate that the small scales of a complex system cause the large scales.

We show the existence of an information flow from small to large scales, independently of the system, the definition of the scales and the observable. Nevertheless, generating the scales of a process by applying a transformation to the initial process influences the causality measure and we cannot distinguish if this causation comes from the physics of the initial process or from the transformation used to generate the scales. Consequently, our search of a flow of information between scales is not meaningful for the moment. Nevertheless, the measures are not meaningless, and the shapes of the obtained peaks can be used to study the dependences of the process.

In order to confirm the causality from small to large scales, we have tested different causality measures: transfer entropy of eq.(1.32), transfer entropy of eq.( 1.33) and directed information. The qualitative results obtained with the different measures are the same: small scales Wiener cause large scales. We propose as a next step, to use experimental time series allowing to generate the scales without the above transformations, or even experimental time series with scale meaning. These experimental processes would allow to isolate the origin of the Wiener causality measures to the physics of the studied processes. Intrapartum fetal heart rate monitoring: Because it is likely to provide obstetricians with significant information related to the health status of the fetus during delivery, intrapartum fetal heart rate (FHR) monitoring is a routine procedure in hospitals. Notably, it is expected to permit detection of fetal acidosis, which may induce severe consequences for both the baby and the mother and thus requires a timely and relevant decision for rapid intervention and operative delivery [START_REF] Chandraharan | Prevention of birth asphyxia: responding appropriately to cardiotocograph (CTG) traces[END_REF]. In daily clinical practice, FHR is mostly inspected visually, through training by clinical guidelines formalized by the International Federation of Gynecology and Obstetrics (FIGO) [START_REF] Ayres-De Campos | Figo consensus guidelines on intrapartum fetal monitoring: Cardiotocography[END_REF]194]. However, it has been well documented that such visual inspection is prone to severe inter-individual variability and even shows a substantial intra-individual variability [START_REF] Hruban | Agreement on intrapartum cardiotocogram recordings between expert obstetricians[END_REF]. This reflects both that FHR temporal dynamics are complex and hard to assess and that FIGO criteria lead to a demanding evaluation, as they mix several aspects of FHR dynamics (baseline drift, decelerations, accelerations, long-and short-term variabilities). Difficulties in performing objective assessment of these criteria has led to a substantial number of unnecessary Caesarean sections [START_REF] Alfirevic | Continuous cardiotocography (CTG) as a form of electronic fetal monitoring (EFM) for fetal assessment during labour[END_REF]. This has triggered a large amount of research world wide aiming both to compute in a reproducible and objective way the FIGO criteria [START_REF] Ayres-De Campos | Figo consensus guidelines on intrapartum fetal monitoring: Cardiotocography[END_REF] and to devise new signal processing-inspired features to characterize FHR temporal dynamics (cf. [START_REF] Spilka | Using nonlinear features for fetal heart rate classification[END_REF][START_REF] Haritopoulos | Survey on Cardiotocography Feature Extraction Algorithms for Fetal Welfare Assessment[END_REF] for reviews).

Part III Biomedical Signal Processing

Related works: After the seminal contribution in the analysis of heart rate variability (HRV) in adults [3], spectrum estimation has been amongst the first signal processing tools that has been considered for computerized analysis of FHR, either constructed on models driven by characteristic time scales [START_REF] Gonçalves | Linear and nonlinear fetal heart rate analysis of normal and acidemic fetuses in the minutes preceding delivery[END_REF][START_REF] Van Laar | Spectral analysis of fetal heart rate variability for fetal surveillance: Review of the literature[END_REF][START_REF] Siira | Do spectral bands of fetal heart rate variability associate with concomitant fetal scalp pH?[END_REF] or the scale-free paradigm [START_REF] Francis | Physiological basis of fractal complexity properties of heart rate variability in man[END_REF][START_REF] Doret | Multifractal analysis of fetal heart rate variability in fetuses with and without severe acidosis during labor[END_REF][START_REF] Doret | Fractal Analysis and Hurst Parameter for intrapartum fetal heart rate variability analysis: A versatile alternative to Frequency bands and LF/HF ratio[END_REF]. Further, aiming to explore temporal dynamics beyond the mere temporal correlations, several variations of nonlinear analysis have been envisaged both for antepartum and intrapartum FHR [START_REF] Echeverria | Detrended fluctuation analysis: a suitable method for studying fetal heart rate variability?[END_REF], based, e.g., on multifractal analysis [START_REF] Doret | Multifractal analysis of fetal heart rate variability in fetuses with and without severe acidosis during labor[END_REF], scattering transforms [START_REF] Chudáček | Scattering transform for intrapartum fetal heart rate variability fractal analysis: A case-control study[END_REF],

phase-driven synchronous pattern averages [START_REF] Magenes | Classification of cardiotocographic records by neural networks[END_REF] or complexity and entropy measures [START_REF] Pincus | Approximate entropy: a regularity measure for fetal heart rate analysis[END_REF][START_REF] Pincus | Approximate entropy (ApEn) as a complexity measure[END_REF][START_REF] Lake | Sample entropy analysis of neonatal heart rate variability[END_REF]. Interested readers are referred to e.g., [START_REF] Spilka | Using nonlinear features for fetal heart rate classification[END_REF][START_REF] Haritopoulos | Survey on Cardiotocography Feature Extraction Algorithms for Fetal Welfare Assessment[END_REF] for overviews. There have also been several attempts to combine features different in nature by doing multivariate classification using supervised machine learning strategies (cf. e.g., [START_REF] Georgieva | Artificial neural networks applied to fetal monitoring in labour[END_REF][START_REF] Spilka | Using nonlinear features for fetal heart rate classification[END_REF][START_REF] Czabanski | Computerized analysis of fetal heart rate signals as the predictor of neonatal acidemia[END_REF][START_REF] Warrick | Classification of normal and hypoxic fetuses from systems modeling of intrapartum cardiotocography[END_REF][START_REF] Spilka | Sparse support vector machine for intrapartum fetal heart rate classification[END_REF]).

Measures from complexity theory or information theory remain, however, amongst the most used tools to construct HRV characterization. They are defined independently from (deterministic) dynamical systems or from (random) stochastic process frameworks. The former led to standard references, both for adult and for antepartum and intrapartum fetal heart rate analysis: approximate entropy (ApEn) [START_REF] Pincus | Approximate entropy: a regularity measure for fetal heart rate analysis[END_REF][START_REF] Dawes | Approximate entropy, a statistic of regularity, applied to fetal heart rate data before and during labor[END_REF] and sample entropy (Sam-pEn) [START_REF] Richman | Physiological time-series analysis using approximate entropy and sample entropy[END_REF], which can be regarded as practical approximations to Kolmogorov-Sinai or Eckmann-Ruelle complexity measures. The stochastic process framework leads to the definitions of Shannon and Rényi entropies and entropy rates. Both worlds are connected by several relations, cf., e.g., [START_REF] Grünwald | Kolmogorov complexity and information theory. with an interpretation in terms of questions and answers[END_REF][START_REF] Lake | Renyi entropy measures of heart rate gaussianity[END_REF] for reviews. Implementations of ApEn and Sam-pEn rely on the correlation integral-based algorithm (CI) [START_REF] Pincus | Approximate entropy: a regularity measure for fetal heart rate analysis[END_REF][START_REF] Grassberger | Estimation of the kolmogorov-entropy from a chaotic signal[END_REF], while that of Shannon entropy rates may instead benefit from the k-nearest neighbor (k-NN) algorithm [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF], which brings robustness and improved performance to entropy estimation [START_REF] Porta | Entropy-based complexity of the cardiovascular control in Parkinson disease: Comparison between binning and k-nearest-neighbor approaches[END_REF][START_REF] Spilka | Nearest-neighbor based wavelet entropy rate measures for intrapartum fetal heart rate variability[END_REF][START_REF] Xiong | Entropy measures, entropy estimators, and their performance in quantifying complex dynamics: Effects of artifacts, nonstationarity, and long-range correlations[END_REF].

Labor stages: Automated FHR analysis is complicated by the existence of two distinct stages during labor. The dilatation stage (Stage 1) consists of progressive cervical dilatation and regular contractions. The active pushing stage (Stage 2) is characterized by a fully-dilated cervix and expulsive contractions. The most common approaches in FHR analysis consist of not distinguishing stages and performing a global analysis [START_REF] Costa | Prediction of neonatal acidemia by computer analysis of fetal heart rate and ST event signals[END_REF][START_REF] Warrick | Classification of normal and hypoxic fetuses from systems modeling of intrapartum cardiotocography[END_REF] or of focusing on Stage 1 only, as it is better documented and usually shows data with better quality, cf., e.g., [START_REF] Georgieva | Phase-rectified signal averaging for intrapartum electronic fetal heart rate monitoring is related to acidaemia at birth[END_REF][START_REF] Spilka | Sparse support vector machine for intrapartum fetal heart rate classification[END_REF]. Whether or not temporal dynamics associated with each stage are different has not been intensively explored yet (see a contrario [START_REF] Spilka | Impacts of first and second labour stages on hurst parameter based intrapartum fetal heart rate analysis[END_REF][START_REF] Lim | Quantitative comparison of entropy analysis of fetal heart rate variability related to the different stages of labor[END_REF]). However, recently, some contributions have started to conduct systematic comparisons [START_REF] Spilka | Fetal heart rate classification: First vs second stage of labor[END_REF][START_REF] Granero-Belinchon | Mutual information for intrapartum fetal heart rate analysis[END_REF].

Goals, contributions and outline:

The present contribution remains in the category of works aiming to design new efficient features for FHR, here based on advanced information theoretic concepts. These new tools are applied to a high quality, large (1404 subjects) and documented FHR database collected across years in an academic hospital in France and described in section 9.1. The database is split into two datasets associated each with one stage of labor, which enables us first to assess and compare acidosis detection performance achieved by the proposed features independently at each stage and second to address differences in FHR temporal dynamics between the two stages. Reexamining formally the definitions of entropy rates in information theory, section 1.4 first establishes that they can be split into two components: Shannon entropy, which quantifies static data properties, and auto-mutual information (AMI), which characterizes temporal dynamics in data, combining both linear and nonlinear (or higher order) statistics. ApEn and SampEn, defined from complexity theory, are then explicitly related to entropy rates and, hence, to AMI (cf. section 9.2.2). Estimation procedures for Shannon entropy, entropy rate and AMI, based on k-nearest neighbor (k-NN) algorithms [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF], are compared to those of ApEn and SampEn, constructed on correlation integral algorithms [START_REF] Pincus | Approximate entropy: a regularity measure for fetal heart rate analysis[END_REF][START_REF] Costa | Multiscale entropy analysis of complex physiologic time series[END_REF][START_REF] Grassberger | Estimation of the kolmogorov-entropy from a chaotic signal[END_REF]. Acidosis detection performances are reported in section 9.3.1. Results are discussed in terms of quality versus analysis window size, k-NN or correlation integralbased procedures and differences between Stages 1 and 2. Further, a longitudinal study consisting of sliding analysis in overlapping windows across the delivery process permits showing that processes characterizing Stages 1 and 2 are different (section 9.3.4).

Datasets: Intrapartum Fetal Heart Rate Times

Series and Labor Stages Data collection: Intrapartum fetal heart rate data were collected at the academic Femme-Mère-Enfant hospital, in Lyon, France, during daily routine monitoring across the years 2000 to 2010. They were recorded using STAN S21 or S31 devices with internal scalp electrodes at 12-bit resolution, 500-Hz sampling rate (STAN, Neoventa Medical, Sweden). Clinical information was provided by the obstetrician in charge, reporting delivery conditions, as well as the health status of the baby, notably the umbilical artery pH after delivery and the decision for intervention due to suspected acidosis [START_REF] Doret | Use of peripartum ST analysis of fetal electrocardiogram without blood sampling: a large prospective cohort study[END_REF].

Datasets: For the present study, subjects were included using criteria detailed in [START_REF] Doret | Use of peripartum ST analysis of fetal electrocardiogram without blood sampling: a large prospective cohort study[END_REF][START_REF] Spilka | Sparse support vector machine for intrapartum fetal heart rate classification[END_REF], leading to a total of 1404 tracings, lasting from 30 min to several hours. These criteria essentially aim to reject subjects with too low quality recording (e.g., too many missing data, too large gaps, too short recordings, etc.). As a result, for subjects in the database, the average fraction of data missing in the last 20 min before delivery is less than 5%. The first goal of the present work is to assess the relevance of new information theoretic measures; their robustness to poor quality data is postponed for future work. The measurement of pH, performed by blood test immediately after delivery, is systematically documented and used as the ground-truth: When pH ≤ 7.05, the newborn is considered has having suffered from acidosis and is referred to as acidotic (A, pH ≤ 7.05). Conversely, when pH > 7.05, the newborn is considered not having suffered from acidosis during delivery and is termed normal (N, pH > 7.05). In order to have a meaningful pH indication, we retain only subjects for which the time between end of recording and birth is lower than or equal to 10 min. Beats-per-minute time series and preprocessing: For each subject, the collection device provides us with a digitalized list of RR-interarrivals ∆ k in ms. In reference to common practice in FHR analysis and for the ease of comparisons amongst subjects, RRinterarrivals are converted into regularly-sampled beats-per-minute times series, by linear interpolation of the samples {. . . , 36,000/∆ k , . . .}. The sampling frequency has been set to f s = 10 Hz as FHR do not contain any relevant information above 3 Hz.

Methods

Outline: We describe in this section the five features that we use to analyze heart rate signals. We propose to apply information theory, as defined by Shannon, to the analysis of cardiac signals. We do so by computing three features from information theory, the Shannon entropy, see section 1.3.1, the Shannon entropy rate, see section 1.4.1 and the auto-mutual information, see section 1.4.2. The first section reports the definitions of two features rooted in complexity theory: approximate entropy (ApEn) and sample entropy (SampEn), which are classically used in cardiac signal analysis. Although we use them in practice only as benchmarks, we devote the second section to their relation with the new features we propose.

Information theory and complexity theory only differ in the nature of the objects under study. Information theory, on the one hand, aims to analyze random processes and defines functionals of probability densities. Complexity theory, on the other hand, aims to analyze signals produced by dynamical systems and assumes the existence of ergodic probability measures to describe the density of trajectories in phase space, so that they can be manipulated as probability densities. In this spirit, we consider throughout this chapter the signals to analyze as random processes, although they indeed originate from a dynamical system.

Assumptions: For the sake of simplicity in the description of the features and for practical use, we assume that signals are monovariate (unidimensional) and centered (zero mean) because the five features we use are independent of the first moment of the probability density function. We also assume that signals are stationary. Although this may seem at first a very strong assumption, it is very reasonable when examining time windows smaller than the natural time scale of the evolution between Stages 1 and 2, as we discuss in section 9.3.4, and larger than events such as contractions. Finally, we also assume that the signals contain N points, sampled at a constant frequency. All estimates depend on N via finite size effects. In the following, we do not mention this dependence explicitly in the notations and only compare features computed over the same window size. Time-embedding: Because we are interested in the dynamics of the signal, we use the delay-embedding procedure introduced by Takens [START_REF] Takens | Dynamical Systems and Turbulence[END_REF] in the context of dynamical systems, see section 1.2.

Features from Complexity Theory

In the 1960s, Kolmogorov and Sinai adapted Shannon's information theory to the study of dynamical systems. The divergence of trajectories starting from different, but undistinguishable initial conditions can be pictured as creating uncertainty, so creating information. Kolmogorov complexity (KC), also known as the Kolmogorov-Sinai entropy and denoted h KS (ρ) in the following, measures the mean rate of creation of information by a dynamical system with ergodic probability measure ρ. KC is constructed exactly as the Shannon entropy rate from information theory, using eq.(1.21) and the same functional form as in eq.(1.7), but using the density ρ of trajectories in phase space instead of the probability density p. In the early 1980s, the Eckmann-Ruelle entropy K 2 (ρ) [START_REF] Grassberger | Estimation of the kolmogorov-entropy from a chaotic signal[END_REF][START_REF] Eckmann | Ergodic-theory of chaos and strange attractors[END_REF] was introduced following the same steps, but using the functional form of the Rényi order-two entropy [START_REF] Rényi | On measures of information and entropy[END_REF]. The interest of K 2 relies in its easier and hence faster computation from experimental time series, which was at the time a challenging issue.

Kolmogorov-Sinai and Eckmann-Ruelle entropies: The ergodic theory of chaos provides a powerful framework to estimate the density of trajectories in the phase space of a chaotic dynamical system [START_REF] Eckmann | Ergodic-theory of chaos and strange attractors[END_REF]. For an experimental or numerical signal, it amounts to assimilating the phase space average to the time average. Given a distance d(., .), usually defined with the L 2 or the L ∞ norm, in the m-dimensional embedded space, the local quantity:

C m i ( ) = number of j such that d x (m,τ ) i , x (m,τ ) j ≤ N -m + 1 , (9.1)
provides, up to a factor , an estimate of the local density ρ in the m-dimensional phase space around the point x (m,τ ) i

. The following averages:

Φ m ( ) = 1 N -m + 1 N -m+1 i=1 ln C m i ( ) , (9.2)
C m ( ) = 1 N -m + 1 N -m+1 i=1 C m i ( ) , (9.3)
are then used to provide the following equivalent definitions of the complexity measures [START_REF] Eckmann | Ergodic-theory of chaos and strange attractors[END_REF]:

h KS (ρ) = lim →0 lim m→∞ lim N →∞ Φ m ( ) -Φ m+1 ( ) , (9.4)
K 2 (ρ) = lim →0 lim m→∞ lim N →∞ ln C m ( ) C m+1 ( ) . (9.5)

Approximate Entropy

Approximate entropy (ApEn) was introduced by Pincus in 1991 for the analysis of babies' heart rate [START_REF] Pincus | Approximate entropy as a measure of system-complexity[END_REF]. It is obtained by relaxing the definition (9.4) of h KS and working with a fixed embedding dimension m and a fixed box size , often expressed in units of the standard deviation σ of the signal as = rσ. ApEn is defined as:

ApEn(m, ) = Φ m ( ) -Φ m+1 ( ) . (9.6)
On the practical side, and in order to have a well-defined Φ m ( ) in eq.( 9.2), the counting of neighbors in the definition (9.1) allows self-matches j = i. This ensures that C m i ( ) > 0, which is required by eq.(9.2). ApEn depends on the number of points N in the time series. Assuming N is large enough, we have: We interpret ApEn as an estimate of the m-order Kolmogorov-Sinai entropy h KS at finite resolution . The larger N , the better the estimate. More interesting is that the nonvanishing value of in its definition makes ApEn insensitive to details at scales lower than . On the one hand, this is very interesting when considering an experimental (therefore noisy) signal: choosing larger than the rms of the noise (if known) filters the noise, and ApEn is then expected to measure only the complexity of the underlying dynamics. This was the main motivation of Pincus and explains the success of ApEn. On the other hand, not taking the limits → 0 and m → ∞ makes ApEn an ill-defined quantity that has no reason to behave like h KS . In addition, only very few analytical results have been reported on the bias and the variance of ApEn. Although m should in theory be larger than the dimension of the underlying dynamical system, ApEN is defined and used for any possible value of m, and most applications reported in the literature use small m (1 or 2) without any analytical support, but with great success [START_REF] Pincus | Approximate entropy as a measure of system-complexity[END_REF][START_REF] Krstacic | Non-linear analysis of heart rate variability in patients with coronary heart disease[END_REF].

Sample Entropy

A decade after Pincus's seminal paper, Richman and Moorman pointed out that ApEn contains in its very definition a source of bias and was lacking in some cases "relative consistency". They defined sample entropy (SampEn) on the same grounds as ApEn:

SampEn(m, ) = ln C m ( ) C m+1 ( ) . (9.8)
So that: lim

→0 lim m→∞ SampEn = K 2 . (9.9)
On the practical side, the counting of neighbors in eq.(9.1) does not allow self-matches. C m i ( ) may vanish, but when averaging over all points in eq.( 9.3), the correlation integral C m ( ) > 0. In practice, SampEn is considered to improve on ApEn as it shows lower sensitivity to parameter tuning and data sample size than ApEn [START_REF] Richman | Time series analysis using approximate entropy and sample entropy[END_REF][START_REF] Lake | Improved entropy rate estimation in physiological data[END_REF].

We interpret SampEn as an estimate of the m-order Eckmann-Ruelle entropy K 2 at finite resolution .

Estimation

We note by the following ApEn (m) and SampEn (m) the estimated values of ApEn and SampEn using our own MATLAB implementation, based on Physio-Net packages. We used the commonly-accepted value, = 0.2σ, with σ the standard deviation of X, and m = 2. For all quantities, we used τ = 5 = f s /f max with f max = 2 Hz the cutoff frequency above which FHR times series essentially contain no relevant information [START_REF] Doret | Multifractal analysis of fetal heart rate variability in fetuses with and without severe acidosis during labor[END_REF]; this time delay corresponds to 0.5 s.

Connecting Complexity Theory and Information Theory

We consider here for clarity only the relation between ApEn and m-order Shannon entropy rate, although the very same relation holds between SampEn and the m-order Rényi ordertwo entropy rate. In information theory terms, ApEn appears as a particular estimator of the m-order Shannon entropy rate that computes the probability density by counting, in the m-dimensional embedded space, the number of neighbors in a hypersphere of radius , which can be interpreted as a particular kernel estimation of the probability density.

Limit of Large Datasets and Vanishing : Exact Relation

When the size of the spheres tends to 0, the expected value of ApEn for a stochastic signal X with any smooth probability density is related, in the limit N → ∞, to the m-order Shannon entropy rate [START_REF] Lake | Renyi entropy measures of heart rate gaussianity[END_REF]:

ApEn(m, ) →0 h (m,τ ) (X) -log(2 ) . (9.10)
Both terms involve m-points correlations of the process X. This relation allows a quantitative comparison of ApEn with the m-order Shannon entropy rate h (m,τ ) . The log(2 ) difference corresponds to the paving, with hyperspheres of radius , of the continuous mdimensional space over which the probability p(x (m,τ ) t

) involved in eq.(1.7) is defined and, thus, h (m,τ ) . This paving defines a discrete phase space, over which eq.(9.1), eq.( 9.2) and eq.(9.6) operate to define ApEn [START_REF] Jaynes | Information theory and statistical mechanics[END_REF]. This illustrates that, for a stochastic signal, ApEn diverges logarithmically as the size approaches 0, as expected for h KS . Fortunately, is fixed in the definition of ApEn, which allows one in practice to compute it for any signal/process.

New Features

Having recognized the success of ApEn and remembering its relation to h (m,τ ) , it seems interesting to probe other m-order Shannon entropy rate estimators. A straightforward improvement would be to consider a smooth, e.g., Gaussian, kernel of width instead of the step function used in eq.(9.1). We prefer to reverse the perspective and use a k-nearest-neighbor (k-NN) estimate. Instead of counting the number of neighbors in a sphere of size , this approach searches for the size of the sphere that accommodates k neighbors. In practice, we compute the entropy H with the Kozachenko and Leonenko estimator [START_REF] Kozachenko | Sample estimate of entropy of a random vector[END_REF][START_REF] Singh | Nearest neighbor estimates of entropy[END_REF], which we denote Ĥ. We compute the auto-mutual information I (m,p,τ ) with the Kraskov et al. estimator [START_REF] Kraskov | Estimating mutual information[END_REF], which we denote Î(m,p) . We then combine the two according to eq.(1.25) to get an estimator ĥ(m) of the m-order Shannon entropy rate. We use k = 5 neighbors and set τ = 5 (see section 9.2.1).

We report in the next section our results for the five features when setting m = 2, p = 1 and compare their performances in detecting acidosis. The dependance of the m-order entropy rate (and its estimators) on m is expected to give some insight into the dimension of the attractor of the underlying dynamical system, but as we have pointed out, the dynamics is indeed contained in the AMI part of the entropy rate. This is why we further explore the effect of varying the embedding dimensions m and p on the AMI estimator Î(m,p) . . AE ĥ SE Ĥ ĥ the most crucial. We use the classical values m = 2 and p = 1 for embedding dimensions.

Results: Acidosis Detection Performance

N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A N A -4 -2 0
To compare the performance, we present the box plots of the five normalized (zero-mean, unit-variance) estimates in the left column of figure 9.1. For Dataset I, the average of ApEn and SampEn for acidotic subjects is smaller than for normal subjects, while the average of the Shannon entropy rate does not show any tendency. This is surprising as one might have expected for ĥ(m) a behavior similar to ApEn and SampEn (see section Features performance: Fetal acidosis detection performance is assessed with the pvalue given by the classical Wilcoxon rank sum test. This non-parametric test of the null hypothesis, which corresponds to identical medians of the distributions of estimates in the normal and abnormal classes, is reported in table 9.1. We have added one symbol when the p-value is less than 0.05, two when less than 0.01. We see that for Dataset I, ApEn (m) , SampEn (m) , Ĥ and Î(m,p) for m = 2 discriminate normal and acidotic subjects, while ĥ(m) does not. Out of the three estimates (ApEn (m) , SampEn (m) , ĥ(m) ) based on entropy rates, the nearest-neighbors one for Shannon entropy rate is the poorest, although its decomposition into Shannon entropy (static one-point information) and AMI (which SampEn (2) Î( 2

Effect of Window Size on Performance

We investigate the robustness of the detection performance when the window size T is reduced, using data from T = 20, 10 and five minutes. Results are reported in figure 9.1 and table 9.2. p-values and AUC both indicate that Î(2,1) and SampEn (2) provide robust discrimination in Dataset I even when the observation length is reduced. Again, Î(2,1) performs better: its AUC is reduced from 0.84 to 0.83 when T is reduced from 20 mn to 5 mn, where the AUC of SampEn is reduced from 0.79 to 0.72. In Dataset II, once again, performance degrades, but AMI is still better at discriminating acidotic from normal subjects. In the following, we focus on AMI estimates only.

Effect of the Embedding Dimensions on the (Fetal Acidosis Detection) Performance of AMI

In order to improve the acidosis detection performance of the AMI, especially in Dataset II, we increase the embedding dimensions m and p used in computing Î(m,p) . This way, we probe a higher order dependence structure in the dynamics. Because of the symmetry of AMI (eq.(1.27)) and aiming at probing the effect of increasing either m or p, we plot the AUC of ROC as a function of m + p only. The dependence of AMI on mp is much smaller and not reported here. These computations have been done with a larger value k = 15 in the k-NN algorithms, in order to accommodate the possibly large embedding dimensions (m + p up to 12). Results are presented in figure 9.3. For a fixed window size, the AUC increases when m + p increases and reaches a maximum; it then remains constant or decreases slightly. This behavior is observed in both Datasets I and II and for any window size T ∈ [START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Arneodo | Revealing a lognormal cascading process in turbulent velocity statistics with wavelet analysis[END_REF][START_REF] Batchelor | Pressure fluctuations in isotropic turbulence[END_REF] mn. Varying T dos not seem to change the location of the maximum of the AUC in a given dataset. The optimal embedding dimension is m + p = 6 in Dataset I and m + p = 10 in Dataset II. This hints at a difference in the dynamics of the FHR in the two datasets. Because both bias and computation time increase with the total dimensionality [START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], the maximal embedding is restricted to m = p = 3. A reduction of the AUC is observed when the analysis window is reduced, but this is only significant for Dataset II.

We reported in table 9.2 the AUC and p-values of the AMI for two embedding dimensions: Î(2,1) and Î(3,3) , for Datasets I and II and several window sizes. The best performing estimator is indicated in bold. For all observation windows and for the two datasets, Î(3,3) achieves the best performance. Their AUC is always larger than the one obtained using SampEn or AMI with m = 2 and p = 1.

Dynamical Analysis

We now explore how long before delivery the AMI can diagnose fetal acidosis on an FHR signal. To do so, we do not restrict our analysis to the last data points before delivery, but we apply it to an ensemble of windows scanning the first and second stages of labor. We examine the dynamics of Î(3,3) , the best performing feature, for both normal and abnormal subjects.

Dataset I: Rapid Delivery

In this first section, we focus on Dataset I and probe Stage 1, including early labor, active labor and transition. Using the time at which Stage 1 ends as a reference (setting it at t 0 = 0), we compute for each subject Î(3,3) in a set of time windows [t i -T, t i ], 0 ≤ i ≤ 50 of fixed size T ending at t i = t 0i * 2 mn, so separated by 2 mn. We perform this analysis for three window sizes T ∈ [START_REF] Batchelor | Pressure fluctuations in isotropic turbulence[END_REF][START_REF] Arneodo | Revealing a lognormal cascading process in turbulent velocity statistics with wavelet analysis[END_REF][START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF] mn. The value of AMI computed in the i-th window is then assigned to time t i -T /2, at the center of the interval. By construction of Dataset I, delivery occurs less than 15 min after pushing started and can be as short as 1 mn, so we completely discard data from Stage 2. We then average the values of AMI over the population of normal subjects and over the population of acidotic subjects, respectively. Results, including p-values, are presented in figure 9 A first observation is that AMI is always larger for acidotic subjects than for normal subjects. As labor progresses, AMI increases in both populations, but the increase is stronger for acidotic subjects. As a consequence, the p-value of the test decreases clearly, so the feature performs better and better when approaching delivery. Detection of acidosis using the AMI feature and T = 20 mn can be obtained in Dataset I as early as 80 min before entering the second stage. Using shorter windows, T = 10 mn or 5 mn, detection is still reliable as early as one hour before Stage 2. We interpret this reduced forecast of acidosis detection in Dataset I as a direct consequence of the reduction of the statistics when the window size T is reduced.

Dataset II: Delivery after Pushing More Than 15 mn For Dataset II, we performed the same dynamical analysis as in the previous section, using the end of Stage 1 as the reference time (t 0 = 0). Because there is now enough data in the pushing stage, we also perform the analysis of this stage using the delivery time (t 0 = D) as the reference. All results are presented in figure 9 Stage 1, we observe again that AMI is larger for acidotic subjects than for normal ones, but the difference is not significant in this group (see the corresponding p-value on the right of figure 9.5). The situation is identical at the end of Stage 2, although we obtain a lower p-value in some windows. The p-value does not decrease clearly when approaching delivery time, as it was in Dataset I, see figure 9.4. For subjects in Dataset II, it is very difficult to make an early detection of acidosis. However, we observe in figure 9.5 that the average AMI is significantly larger at the end of Stage 2 than at the end of Stage 1. The increase of AMI is larger for abnormal subjects.

To examine more precisely the dynamical increase of AMI, especially when entering Stage 2, we computed Î(3,3) over an ensemble of windows of size T = 20 mn spanning continuously a large time interval that includes the end of the active labor and the beginning of the pushing stage. Results are reported in figure 9.6. We see a continuous increase of AMI values when evolving from Stage 1 to Stage 2. The increase is more important for abnormal subjects, which corroborates the findings in figure 9.5. For smaller window sizes, the situation is less clear.

We also studied the dynamical evolution of the Shannon entropy estimate Ĥ, which, together with the AMI, combines into the Shannon entropy rate (see eq.(1.25)). Changes in the Shannon entropy H indicate changes in the probability density of the signal. Results are presented in figure 9.6, side by side with the AMI. We observe a dramatic rise of the value of Ĥ when subjects evolve from Stage 1 to Stage 2. This increase is clearly observed for normal and abnormal subjects. No significant difference between normal and acidotic subjects is observed for this static quantity. The start of pushing implies a strong deformation of the probability density of the FHR, indicating strong perturbations of the FHR, for both normal and acidotic subjects.

Discussion, Conclusions and Perspectives

We now discuss the interpretation of Shannon entropy and AMI measurements in different stages of labor. The fetuses are classified as normal or acidotic depending on a postdelivery pH measurement, which gives a diagnosis of acidosis at delivery only. There is no information on the health of the fetuses during labor.

The physiological interpretation of a feature, and especially its relation to specific FHR patterns, e.g., like those detailed in [START_REF] Freeman | Problems with intrapartum fetal heart rate monitoring interpretation and patient management[END_REF][START_REF] Ayres-De Campos | Figo consensus guidelines on intrapartum fetal monitoring: Cardiotocography[END_REF], is a difficult task that is only scarcely reported in the literature [START_REF] Signorini | Linear and nonlinear parameters for the analysis of fetal heart rate signal from cardiotocographic recordings[END_REF][START_REF] Gonçalves | Linear and nonlinear heart-rate analysis in a rat model of acute anoxia[END_REF]. In this chapter, we have averaged our results over large numbers of (normal or acidotic) subjects, which jeopardizes any precise interpretation in terms of a specific FHR pattern that may appear only intermittently.

Acidosis Detection in the First Stage

We can nevertheless suggest that the value of the Shannon entropy H is related to the frequency of decelerations in the FHR signals. Indeed, Shannon entropy strongly depends on the standard deviation of the signal (e.g., see eq.(1.38)), which in turn depends on the variability in the observation window. A larger number of decelerations in the observation window deforms the PDF of the FHR signal by increasing its lower tail; in particular, this increases the width of the PDF and hence increases the standard deviation and the Shannon entropy. This explains our findings in figure 9.1 (for Dataset I).

When acidosis develops in the first stage of labor, the Shannon auto-mutual information estimator Î(m,p) significantly outperforms all other quantities both in terms of p-value and AUC. The performance of AMI is robust when tuning either the size of the observation, and hence the number of points in the data, and the embedding dimensions (m, p). In addition, the performance slightly increases when the total embedding dimension m+p increases; although one has to care about the curse of dimensionality.

For abnormal subjects from Dataset II, AMI is not able to detect acidosis using data from Stage 1. This suggests that acidosis develops later, in the second stage of labor.

For all datasets, AMI computed with τ = 0.5 s is always larger for acidotic subjects than for normal subjects. This is in agreement with results obtained with ApEn and SampEn, which are both lower for acidotic subjects. This shows that FHR classified as abnormal have a stronger dependence structure at a small scale than normal ones. We can relate this increase of the dependence structure of acidotic FHR to the shortterm variability and to its coupling with particular large-scale patterns. For example, a sinusoidal FHR pattern [START_REF] Ayres-De Campos | Figo consensus guidelines on intrapartum fetal monitoring: Cardiotocography[END_REF], especially if its duration is long, should give a larger value of the AMI, because its large-scale dynamics is highly predictable. As another example, we expect variable decelerations (with an asymmetrical V-shape) and late decelerations (with a symmetrical U-shape and/or reduced variability) to impact AMI differently. Of course, the choice of the embedding parameter τ is then crucial, and this is currently under investigation.

AMI and entropy rates depend on the dynamics as they operate on time-embedded vectors. AMI focuses on nonlinear temporal dynamics, while being insensitive to the dominant static information. AMI is independent of the standard deviation, which on the contrary contributes strongly to the Shannon entropy. This explains why AMI performs better than entropy rate estimates, such as ApEn (m) , SampEn (m) and ĥ(m) , which depend also on the standard deviation.

Acidosis Detection in Second Stage

The results reported for Stage 2 show a severe decrease in the performance of the five estimated quantities. Analyzing Stage 2 is far more challenging than analyzing Stage 1, which suggests that temporal dynamics in Stage 2 differ notably from those of Stage 1 [START_REF] Spilka | Fetal heart rate classification: First vs second stage of labor[END_REF] or simply that our database does not contain enough acidotic subjects in that case. Î(m,p) achieves the best performance in terms of p-value and AUC; this clearly underlines that the analysis of nonlinear temporal dynamics is critical for fetal acidosis detection in Stage 2. As in Stage 1, the AMI is always larger in Stage 2 for acidotic subjects than for normal subjects.

Although the Shannon entropy computed from the last 20 mn of Stage 2 before delivery does not show a clear tendency in figure 9.1 for Dataset II, looking at figure 9.6 clearly shows that Ĥ increases as labor progresses: this is probably related to the average increase of the number of decelerations, which is expected in both the normal and acidotic population.

SampEn (2) is also able to perform discrimination in Stage 2. From these observations, one may envision the definition of a new estimator that would measure the auto-mutual information using the Rényi order-two entropy by applying eq. (1.25). Nevertheless, it should be emphasized that Rényi order-q entropy is lacking the chain rule of conditional probabilities as soon as q = 1, which may jeopardize any practical use of such an estimator.

Probing the Dynamics

Increasing the total embedding dimensions in AMI improves the performance in the detection of acidotic subjects, in both the first and second stages. The best performance is found for different total embedding dimension in the two datasets. This suggests that FHR dynamics is different in each stage.

As seen in eq.(1.25), the Shannon entropy rate can be split into two contributions: one that depends only on static properties (the Shannon entropy, estimated by Ĥ) and one that involves the signal dynamics (the auto-mutual information, estimated by Î(m,1) ). By following the time evolution of these two parts, we were able to relate Shannon entropy Ĥ with the evolution of the labor and AMI not only with the evolution of the labor, but also with possible acidosis. Looking at subjects for which the pushing phase is longer than 15 mn, it clearly appears that all fetuses are affected by the pushing, as evidenced by a large increase of the Shannon entropy Ĥ and a small increase of AMI. Additionally, the increase of AMI is steeper for abnormal subjects than for normal ones, which may indicate different reactions to the pushing and can be related to specific pathological FHR patterns. When the pushing stage is long (Dataset II), fetuses reported as acidotic do not show any sign of acidosis until prolonged pushing. These fetuses appear as normal until delivery is near.

When acidosis develops during the first stage of labor, in Dataset I, we observe clearly that while AMI increases steadily till delivery for healthy fetuses, it increases faster for acidotic ones. This suggests that acidotic fetuses in Dataset I react to early labor, as early as one hour before pushing starts. This could not only indicate that some fetuses are prone to acidosis, but also may pave the way for an early detection of acidosis in this case.

Most of the time when studying a system, scientists face processes whose properties are a priori unknown. Characterising these processes is a major task to describe the studied system. During this thesis, which combines signal processing and physics, we were mainly motivated by the study of complex systems and turbulence, and consequently, we were interested in the study of regularity and self-similarity properties, long range dependence structures and multi-scale behavior. In order to perform this kind of study, we use information theory quantities, which are functions of the probability density function of the analysed process, and so depend on any order statistics of its PDF (chapter 3).

We developed different, but related, data analysis methodologies, based on information theory, to analyse a process across scales τ . These scales are usually identified with the sampling parameter of Takens embedding procedure (section 1.2), but also with the size of the increments of the process (chapters 7 and 8). The methodologies developed during this thesis, can be used to characterize stationnary and non-stationnary processes by analysing time windows of length T of the studied signal (chapter 4).

Characterization of a process Regularity

In order to characterize the regularity of an unknown process X, we combine entropy rate h (m,τ ) (X) (section 1.4.1) and an adapted Theiler prescription (section 2.5) to perform an analysis across scales. The entropy rate across scales, that quantifies the unknown information of a sample knowing the m previous ones, can be expressed following eq. (1.25). Consequently, entropy rate across scales presents two different contributions: the entropy of the process, which only depends on its PDF, and the auto-mutual information across scales (section 1.4.2) that quantifies the shared information between two consecutive chunks of the signal.

Both, auto-mutual information and entropy rate across scales perform very similar statistical characterizations of a process (see eq. (1.25)). However, entropy rate presents some estimation advantages: it is less biased and has a smaller standard deviation than auto-mutual information, when analysing non-stationnary processes (section 4.3.1). The local slope of entropy rate across scales in a given range of scales, can be related to a measure of the exponent of "global regularity" of the process in this range (chapter 4):

• If the local slope, in a range of scales, is 0, the process, sampled at these scales, corresponds to a white noise. The analysed process can be seen as discontinuous at these scales (figure 9.7, a)). Furthermore, when the scale decreases the slope of entropy rate can only increase (or remains constant), due to the increase of dependences between samples of the process at these scales. This effect indicates the increase of the regularity of the process for the small scales.

• A local slope between 0 and 1 (∈ (0, 1)), in a range of scales, indicates the continuity but non-differentiability of the process at these scales. The value of the slope characterizes the exponent of "global regularity" of the process in this range (figure 9.7, b)).

• A local slope of 1, in a range of scales, indicates that the process is differentiable with a continuous derivative at these scales (figure 9.7, c)).

h (1,τ )

Scale invariance

A rigorous linear behavior of entropy rate in function of the logarithm of the scale indicates the scale invariance (monofractality) of the analysed process, and therefore the non-existence of integral scale. In this case the exponent of global regularity is equal to the Hurst exponent of the process. The entropy rate across scales of a process and the entropy of the increments of the process are strongly connected (eq.(4.8)). Their relationship is completely general: it is valid for both stationnary and non-stationnary processes and it is independent of the PDF and the regularity properties of the process. In addition, for scale-invariant processes, we show that both measures are almost identical, and both characterize the Hurst exponent of the process (figures 9.7, b) and c)). This equality shows the strong relation between entropy rate across scales of a process and the PDFs of its increments.

Self-similarity: monofractality and multifractality

While for scale-invariant processes the entropy rate is linear in function of the logarithm of the scales, for multifractal processes, the non-linear behaviour of entropy rate across scales allows to describe the distance from monofractality of the process (figure 7.7), and indicates the existence of an integral scale. This non-linear behavior for multifractal processes is explained since the regularity is not defined by a single exponent but by a distribution of exponents (Hölder exponent) characterized by the singularity spectrum of the process (section 5.2). Nevertheless, the entropy rate across scales mainly depends on the second order moment of the analysed process and deviations from linearity, due to higher order moments, are in general very weak, and difficult to study. For multifractal processes, the dominant linear behavior of entropy rate across scales characterizes the exponent of global regularity of the process defined as the Hölder exponent of order 2, h(p) = ∂ζ (p) ∂p with p = 2 (see section 5.2). In order to properly characterize self-similarity and discriminate between monofractal and multifractal processes, we used the concept of Kullback-Leibler distance between two PDFs. We define a measure of distance from Gaussianity of a PDF (section 7.1) as the Kullback-Leibler divergence between this PDF and its Gaussian approximation (Gaussian PDF with same standard deviation). Then, we measure the distance from Gaussianity of the increments of the analysed process to measure the deformation of their PDFs across scales and hence the intermittency of the process (chapter 7).

Long-range dependences

Combining Takens embedding procedure with information theory allows to measure dependences by taking into account n-points interactions, with n being defined by the total embedding dimension of the measure (section 3.3.2). This provides the framework to test Markov properties and long-range dependences. In order to identify long range dependences in a process, we can derive it until obtaining a discontinuous process (order of exponent of global regularity less than or equal to 0). Then, we measure auto-mutual information for different embedding dimensions m and p in order to vary the number of points interacting in the measure. For long range dependences, the auto-mutual information measure should increase when the embedding increases, while for short-range dependences, the auto mutual information remains constant when the embedding increases (figure 3.6).

Information flows

We also looked for causality relationships between the different scales of a process (chapter 8). We used transfer entropy (section 1.6.2) as a Wiener causality measure between two time processes. However, the generation of the scales of a time process has a strong influence on the causality measures, and renders their interpretation very difficult, i.e. we are not able to distinguish which contribution of the measure is due to the properties of the system and which one is due to the way of generating the scales (section 8.2). The results we obtained are therefore negative results, considering that no knowledge from the analysed process was obtained. However, all the studied systems presented a flow of information from small to large scales for any way of generating the scales, which can suggest the generality of this result for dynamical systems. We used the above methodologies mainly in the study of turbulence (chapters 6, 7 and 8), but also in a bio-medical application (chapter 9), and discrimination between processes (chapters 3 and 4).

Perspectives

The methodologies presented in this manuscript have been used to characterize synthetic processes or high quality experimental velocity measures. Only in the biomedical applications, (chapter 9), these methodologies have been tested on low quality data. As we explained throughout the thesis, these methodologies are of general application and can be used to study generic systems. New applications of the above methodologies on real world datasets, and specially on low quality datasets could reveal some flaws of the methods, or some aspects that need to be improved in order to keep the measures instructive when dealing with this kind of data.

Throughout the manuscript, we analysed stationnary and non-stationnary, stochastic or chaotic processes and we give an interpretation to our measures for all these kinds of processes. In the case of non-stationnary processes, we did that in two different frameworks. The question that naturally arises is if we can use the methodologies developed during the thesis to characterize non-chaotic deterministic processes (for exemple, a sine wave), and what are the interpretation of the measures in this case. This study can be interesting to analyse independently the stochastic and deterministic components of a process; for exemple, a noise on a sine wave.

In the analysis of information flows and causality relationships is, perhaps, where more questions remain opened. We concluded that the causality measures between the scales of a time series are strongly influenced by the way of generating the scales. However, for any scale function we found that the small scales cause the large ones. This general result for any process and any scale function needs to be studied carefully. On the other hand, to avoid the generation of the scales of a time series, we propose, as future research, to analyse the causality between experimental signals with a meaning of scale, i.e. not generating the different scales from an only signal, but experimentally measuring several signals associated with different scales. This way, with the scales measured experimentally and turned into different time series, the causality measures would be only due to the properties of the system.

Figure 1 . 1 :

 11 Figure 1.1: Embedded vectors X (p,λ) i+pλ and X (m,λ) i of process X.

Figure 2 . 1 : 1 N

 211 Figure 2.1: Entropy rate performances (at fixed τ ) on the Gaussian process. Bias and standard variation of the estimates in function of the number of samples N for both entropy rate estimators. The scale τ is fixed (τ = 3) and chosen such that the correlation coefficient between two succesive points is c(τ ) = 0.94. In c) and d) the straight line correspond to f ∝

Figure 2 . 2 :

 22 Figure 2.2: Entropy rate performances (at fixed τ ) on the log-normal process. Bias and standard variation of the estimates in function of the number of samples N for both entropy rate estimators. The scale τ is fixed (τ = 3) and chosen such that the correlation coefficient between two succesive points is c(τ ) = 0.94. In c) and d) the straight line correspond to f ∝

Figure 2 . 3 :

 23 Figure 2.3: Analysis of the Bias of the Gaussian and the log-normal processes in function of k N

Figure 2 . 1 m+1

 21 Figure 2.3 verifies that the bias of the new entropy rate estimator behaves as O(( k N ) 1 m+1) and supports the results already shown by Kraskov et al.[START_REF] Kraskov | Estimating mutual information[END_REF] and by Gao et al.[START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], but this time for our estimator of entropy rate. In figure2.3 the number of neighbors in the search varies between 5 and 10, k =[START_REF] Albers | Using time-delayed mutual information to discover and interpret temporal correlation structure in complex populations[END_REF][START_REF] Arneodo | Revealing a lognormal cascading process in turbulent velocity statistics with wavelet analysis[END_REF]. The number of samples increases from a minimum value of N = 64 to a largest value of N = 4096. The embedding dimension varies from m = 1 to m = 6. In both figures 2.3(a) and (b) the bias goes to zero when k/N → 0. In addition there is another intermediate value of k/N for which the bias is zero, k/N | 0 . This intermediate value depends on the analyzed process. When k/N

Figure 2 . 4 :

 24 Figure 2.4: a) Entropy rate of a low-pass filtered Gaussian noise (f c = 0.1) b) Phase space of a low-pass filtered Gaussian noise. The entropy rates have been computed with a constant number of N = 2000 samples and averaged over 50 realizations.

Figure 2 . 5 :

 25 Figure 2.5: a) Entropy rate of fractional Gaussian noise (H = 0.7) b) Phase space of fractional Gaussian noise. The entropy rates have been computed with a constant number of N = 2000 samples and averaged over 50 realizations.

Figure 2 . 6 :

 26 Figure 2.6: a) Entropy rate of fBm (H = 0.7) b) Phase space of fractional Brownian motion. The entropy rates have been computed with a constant number of N = 2000 samples and averaged over 50 realizations.

τ th = 1 and τ = 30 τFigure 2 . 7 :

 3027 Figure 2.7: Entropy rate of fBm (H = 0.7) in function of the effective size of the analyzed process. The entropy rates have been computed with a constant number of N = 2000 samples and averaged over 50 realizations.

Figure 2

 2 Figure 2.8: a) Entropy rate of a turbulent velocity signal b) Phase space of a turbulent velocity signal. The entropy rates have been computed with a constant number of N = 1300 samples and averaged over 20 realizations.

Figure 3 . 1 :

 31 Figure 3.1: Bias of entropy, auto mutual information and entropy rate for m = 1, p = 1, c(τ ) = 0.32, k = 5 and varying N , for the standard log-normal (blue) and the Gaussian (black) processes. Left column: exponential decay f c = 0.1. Right column: power law decay with H = 0.7. The errorbars correspond to the standard deviation.

Figures 3 .

 3 Figures 3.1 shows the behaviour of the estimates when N varies from N = 28 to N = 216 , for a fixed k = 5. For the entropy estimator, figures 3.1(a) and (b) show that the bias B Ĥ = E[ Ĥ -H] goes to zero when the signal size N increases. The standard deviation of the estimate, represented as the error bar of the bias, also tends to zero when N increases. The auto mutual information and entropy rate estimators behave similarly, see figure3.1(c),(d) and (e),(f), respectively. So, as a first conclusion, our three estimates are asymptotically unbiased with a vanishing variance, for any dependence structure (exponential or power law decay) and for any one-point distribution (Gaussian or log-normal). They are therefore robust[START_REF] Kraskov | Estimating mutual information[END_REF][START_REF] Gao | Demystifying fixed k-nearest neighbor information estimators[END_REF], and have (very) satisfying performance as soon as N ≥ 210 .In the following, we use N = 216 to discard finite size effects.

Figure 3 . 2 :

 32 Figure 3.2: Bias of entropy, auto mutual information and entropy rate for m = 1, p = 1, c(τ ) = 0.32, N = 2 16 and varying k, for the standard log-normal (blue) and the Gaussian (black) processes. Left column: exponential decay f c = 0.1. Right column: power law decay with H = 0.7. The errorbars correspond to the standard deviation.

Figure 3 .

 3 Figure3.2 shows how our three estimators depend on k for a signal size N = 216 . For any estimate, and any dependence structure, we barely see any evolution with k, although we have observed a small increase of bias when k increases. Because the choice of k does not impact significatively our estimates, we set k = 5 in the following to reduce the computation time.

Figure 3 . 3 :

 33 Figure 3.3: Bias of entropy (a,b), auto mutual information (c,d) and entropy rate (e,f) for m = 1, p = 1, k = 5, N = 2 12 and varying c(τ ), for the standard log-normal (blue) and the Gaussian (black) processes. Left column: exponential decay f c = 0.1. Right column: power law decay with H = 0.7. The errorbars correspond to the standard deviation.

3. 3 . 1

 31 Dependence on the scale τ for fixed embedding dimensions.

Figures 3 .

 3 Figures 3.4 and 3.5 compare the classical estimators (histogram, power spectrum and correlation function) with our estimators (entropy, entropy rate and AMI) for m = p = 1 when varying the scale τ . The results for the even log-normal process are reported in red.Classical estimators cannot distinguish the two log-normal processes. Prescribing the PDF (figure (3.4,a)) univocally prescribe the entropy (figure(3.4,d)). It is therefore not surprising that the Gaussian and the log-normal processes have different entropies and that the entropy of the two log-normal processes are undistinguishable.

Chapter 3 :Figure 3 . 4 :

 334 Figure 3.4: Exponential decay f c = 0.1.Classical analysis: histogram (a), power spectrum (b) and correlation function (c). Information theory analysis for m = 1, p = 1: entropy (d), entropy rate (e) and auto mutual information (f). The color indicates the process: blue for standard log-normal, red for even log-normal and black for Gaussian. The continuous lines in (e,d,f) correspond to analytical expressions. We used N = 216 and k = 5.

Figure 3 . 5 :

 35 Figure 3.5: Power law H = 0.7. Classical analysis: histogram (a), power spectrum (b) and correlation function (c). Information theory analysis for m = 1, p = 1: entropy (d), entropy rate (e) and auto mutual information (f). The color indicates the process: blue for standard log-normal, red for even log-normal and black for Gaussian. The continuous lines in (e,d,f) correspond to analytical expressions. We used N = 2 16 and k = 5.

Figure 3 . 6 :

 36 Figure 3.6: Auto mutual information vs total embedding m + p. Left column: exponential decay f c = 0.1. Right column: power law H = 0.7. The color indicates the process: blue for standard log-normal (c,d), red for even log-normal (e,f) and black for Gaussian (a,b). The continuous lines in (a-d) correspond to analytical expressions. We used N = 2 16 , k = 5, c(τ ) = 0.32 (τ = 1 for the scale invariant and τ = 11 for the exponential decay).

. 17 )

 17 Auto Mutual InformationWe can define the auto mutual information during a time interval of size T as: Ī(m,p,τ)T (M ) = HT (M (m,τ ) ) + HT (M (p,τ ) ) -HT (M (m+p,τ ) ) (4.18)Following eq.(1.25), the entropy rate during a time interval of size T can then be expressed as:h(m,τ) T (M ) = HT (M ) -Ī(m,1,τ) T (M ) .(4.19)All the above time-averaged measures, HT (M (m,τ ) ), Ī(m,p,τ)T (M ) and h(m,τ)T (M ) depend on the window size T and the scale τ .

Figure 4

 4 Figure 4.1: Standard deviations of H(1,τ) T

Figure 4 1 2

 41 Figure 4.2: Dependence of h(1,τ=2) T

TFigure 4 . 3 :

 43 Figure 4.3: a) Entropy HT (M (m,τ ) ) and b) entropy rate h(m,τ) T (M ) in function of the logarithm of the window size ln(T ) of the analysis for a fixed scale τ . c) Entropy HT (M (m,τ ) ) and d) entropy rate h(m,τ)T (M ) in function of the logarithm of the scale of analysis ln(τ ) for a fixed T = 216 . All the estimators were applied on a fBm with H = 0.7. In a) and d) the black line represents a straight line of slope H = 0.7. In c) the different measures follow a linear behaviour of slope (m -1)H.

Figures 4 .

 4 3 c) and d) show a behaviour of H(m,τ) τ according to eq.(4.13) and eq.(4.14) for H m -1)H ln(τ ) while entropy rate behaves as H ln(τ ) for any embedding m.

Figure 4 . 4 :

 44 Figure 4.4: Entropy rate h(m,τ) T (M ) -H ln(τ ) for a fBm with H = 0.7 . The embedding dimension m varies from 1 to 4. The time window size is fixed to T = 2 16 .

4. 3 . 3 Figure 4 . 5 :

 3345 Figure 4.5: Mutual information between the motion and its increments I(M t-τ , δ τ M t ) in function of the rate τ /t. The motion is a fBm with H = 0.7.

Figure 4 . 6 :

 46 Figure 4.6: a) Entropy rate of the motion h(1,τ)

T

  Figure 4.7: h(1,τ) T value for an standard (blue) and even (red) log normal signals and a fBm (black) in function of the logarithm of the τ . Number of neighbors k = 5 and data size N = 2 16 . The standard deviation is obtained with 100 realisations. The straight lines in (b) indicate the theoretical values of the entropy of the processes.

Figure 4 . 8 :

 48 Figure 4.8: PDF of the normalized increments with τ = 2 j , from j = 0 (bottom) up to j = 6 (up) of (a) Hermitian and (b) even Hermitian log-normal motions. Curves have been arbitrarily shifted on the Y-axis for clarity.

Figure 5 . 1 :

 51 Figure 5.1: Richardson cascade description of turbulence [66].

Figure 5 . 2 :

 52 Figure 5.2: P.S.D. of turbulence.

23 )

 23 S p (l) ≈ l ζ(p) (5.24) where h, called the Hölder exponent, indicates the order of the singularities characterising the turbulent velocity and ζ(p) defines the scaling exponents of turbulent velocity. Any model is characterized by the set of their scaling exponents, ζ(p), as they appear in eq.(5.24). The four-fifths law imposes ζ(3) = 1 and this should be respected by any model or process representing turbulence, see figure 5.3 a).

Figure 5 . 3 :

 53 Figure 5.3: a) Scaling exponents ζ(p) versus order p and b) Singularity spectrum D(h) versus Hölder exponent h, both for three different models of turbulence in the inertial domain, together with an experimental Eulerian velocity measure (Modane, black symbols). Models are: monofractal fractional Brownian motion (cyan, continuous straight line), multifractal log-normal (blue, dashed line) and multifractal log-Poisson (red, dotted line). The scaling exponents ζ(p) are computed from the partition functions of the increments. In order to compute D(h) we use the wavelet leaders methodology [215].

Figure 6 . 1 :

 61 Figure 6.1: Top: Power Spectral Density of the experimental signal. The straight line corresponds to the K41 scaling. Bottom: entropy rate h (m,τ ) , as a function of the scale (f = 1/τ ) for different embedding dimensions m ∈ {1, 2, 3}. The blue curve corresponds to eq.(1.40) using the autocorrelation function. The thin straight line is a line of slope -1/3, the thick straight line has a slope -1.

Figure 6

 6 Figure 6.2: a) Entropy rate of the velocity h (1,τ ) (V ), entropy of its increments H(δ τ V ), mutual information between the velocity and its increments I(V t-τ , δ τ V t ), and the difference between the two last H(δ τ V )-I(V t-τ , δ τ V t ). b) Entropy rate of the velocity h (1,τ ) (V ) minus entropy of its increments H(δ τ V ), and mutual information between the velocity and its increments. The velocity signal is the one from the wind tunnel at Modane . The time window size is T = 216 .

Figure 6 . 3 :

 63 Figure 6.3: Top: entropy rates h(1,τ ) for different Reynolds numbers, as a function of the scale (f = 1/τ ). The entropy has been normalised to put all the curves over the lowest Reynolds number measure. The thin straight line is a line of slope -1/3, the thick straight line has a slope -1. Bottom: entropy rates h(1,τ ) , as a function of the scale (f = 1/τ ) for different Reynolds numbers, and the Batchelor fits for each of them (points).

Figure 6 . 4 :

 64 Figure 6.4: Left: bi-variate PDF of (v(t), v(tτ )), the velocity field. Right: p(X) (in red), p + (X) (blue) and p -(X) (black).

Figure 6 . 5 :

 65 Figure 6.5: Top: conditioned entropy rates h +,(1,τ ) (red) and h -,(1,τ ) (blue) as function of scale for turbulence data for embedding m = 1. The entropy rate h(1,τ ) (black) is reported for comparison. Bottom: difference (h +,(1,τ )h -,(1,τ ) ) (black) and standard deviation of h +,(1,τ ) (blue) and h -,(1,τ ) (red).

Figure 6 . 6 :

 66 Figure 6.6: Top: conditioned entropy rates h +,(1,τ ) (red) and h -,(1,τ ) (blue) as function of scale for turbulence data (Increasing Reynolds form left to right) for embedding m = 1. The entropy rate h (1,τ ) (black) is reported for comparison. Bottom: difference (h +,(1,τ )h -,(1,τ ) ) (black) and standard deviation of h +,(1,τ ) (blue) and h -,(1,τ ) (red).

3 Figure 7 . 1 :

 371 Figure 7.1: (a) Power spectrum (b) Flatness (c) Entropies (d) KL distance from Gaussianity, for the Modane experimental data, as functions of ln(τ /T ) = ln(l/L), the logarithm of scale normalized by the integral scale. In (a) and (c), the straight lines indicate the theoretical scaling in the inertial region predicted by Kolmogorov K41 theory.

Figure 7

 7 Figure 7.2: a) Entropy H(δ τ V ) of the Eulerian velocity increments as a function of ln(τ /T ) = ln(l/L). b) KL divergence D τ (V ) = H G (δ τ V ) -H(δ τ V ). Different experimental signals with various Reynolds numbers have been used.

Figure 7 . 3 :

 73 Figure 7.3: a) Entropy H (continuous curve) and entropy under Gaussian hypothesis H G (symbols) for mono-and multifractal processes. Black lines represent the theoretical slope (1/3) for information expected for turbulence within K41 theory. b) Distance D τ (X) = H G (δ τ X) -H(δ τ X) from Gaussianity. Four different models are used: fBm (cyan ), MRW (magenta * ), log-normal RWC (blue ) and log-Poisson RWC ( red).

c 2 Figure 7 . 4 :

 274 Figure 7.4: Kullback-Leibler divergence D τ for the log-normal propagator model, for varying values of the log-cumulant c 2 , as a function of ln(τ /T ) = ln(l/L) (a) or as a function of c 2 ln(τ /T ) (b).

Figure 7 . 5 :

 75 Figure 7.5: Kullback-Leibler divergence D τ as a function of c 2 ln(τ /T ) for the log-Poisson propagator model, for varying values of λ (a) and β (b). The dotted line represents the single curve obtained for the log-normal propagator (figure 7.4(b)).

Figure 7 . 6 :

 76 Figure 7.6: Comparison of the two propagator models of turbulence (log-normal (continuous blue line) and log-Poisson (red dotted line)) with Modane experimental data (in black with errorbars), as a function of ln(τ /T ): (a) normalized flatness, (b) KL divergence. The dashed (blue) and dotted (red) lines represent the modified models that take into account the dissipative range [38].

  Figure 7.7: a) h(1,τ) T (M ) b) H(δ τ M ) c) h(1,τ) T (M ) -H ln(τ ) and d) D τ (M ) for an standard (blue) and even (red) log normal signals and a fBm (black) in function of the logarithm of τ . Number of neighbors k = 5 and data size N = 2 16 . The standard deviation is obtained with 100 realizations. The straight lines in (b) indicate the theoretical values of the entropy of the processes.

Figure 8 . 1 :

 81 Figure 8.1: Left:Increments initial emplacement. Right: Increments coincidence. The red increment is the lagged one.

Figure 8 . 2 :

 82 Figure 8.2: Transfer entropy between scales δ τ l and δ τs , with τ l = 92 and τ s = 66m for turbulence (red), fBm with H = 0.3 (black), and fBm with H = 0.5 (blue). Circles indicate the TE from the small to the large scale, and triangles indicate the TE from the large to the small scale. a) Turbulence b) Comparison between turbulence and fBm with H = 0.3 c) Comparison between fBm with H = 0.3 and fBm with H = 0.5

Figure 8 . 4 :

 84 Figure 8.4: Transfer entropy between scales of turbulent velocity. Red circles indicate the TE from the small scale ( 1 f c = 10) to the large one ( 1 f c = 100). Red triangles indicate the TE from the large scale to the small one. The scales are generated using three different filters: a) Low pass filter, b) High Pass filter, c) Band Pass filter.

Figure 8 . 5 :

 85 Figure 8.5: Transfer entropy between scales of fBm with H = 0.3. Black circles indicate the TE from the small scale ( 1 f c = 10) to the large one ( 1 f c = 100). Black triangles indicate the TE from the large scale to the small one. The scales are generated using three different filters: a) Low pass filter, b) High Pass filter, c) Band Pass filter.

Figure 8 . 6 :

 86 Figure 8.6: Maximum amplitude of TE between the scales generated at 1/f c ref = 10 (indicated by the blue line) and 1/f c variable (x axis). FBm H = 0.3 in black and turbulent velocity in red. For both plots, triangles indicates TE from large to small scales, and circles TE from small to large scales.

Figure 8 . 7 :

 87 Figure 8.7: Transfer entropy between scales v (2,τ l ) and v(2,τs) , with τ l = 92 and τ s = 66m for turbulence (red), fBm with H = 0.3 (black), and fBm with H = 0.5 (blue). Circles indicate the TE from the small to the large scale, and triangles indicate the TE from the large to the small scale. a) Turbulence b) Comparison between turbulence and fBm with H = 0.3 c) Comparison between fBm with H = 0.3 and fBm with H = 0.5

Figure 8 . 8 :

 88 Figure 8.8: Transfer entropy between scales for different observables of turbulence. Circles indicate TE from small to large scales. Triangles indicate TE from large to small scales. a) TE for turbulent velocity increments δ τ (v(t)). b) TE for the square of turbulent velocity increments (δ τ v(t)) 2 . c) TE for the increments of the square of the turbulent velocity δ τ (v(t)) 2 .

CHAPTER 9 Information

 9 Theory to Probe Intrapartum Fetal Heart Rate Dynamics

  Following the discussion above on labor stages, subjects are split into two different datasets. Dataset I consists of subjects for which delivery took place after a short Stage 2 (less than 15 min) or during Stage 1 (Stage 2 was absent). It contains 913 normal and 26 acidotic subjects. Dataset II gathers FHR for delivery that took place after more than 15 min of Stage 2. It contains 450 normal and 15 acidotic subjects.

9. 3 . 1

 31 Comparison of Features' Performance, Using a Single Time Window, Just Before Delivery Average features value for normal and abnormal subjects: We compute the five features: ApEn (m) , SampEn(m) , ĥ(m) , Ĥ and Î(m,p) for normal and acidotic (abnormal) subjects in Datasets I and II using data from the last T = 20 mn before delivery, which are Î

Figure 9 . 1 :

 91 Figure 9.1: Box plot-based comparisons of the five different (normalized) estimates (ApEn (m) (AE), SampEn (m) (SE), Shannon entropy rate h (m) ( ĥ), Shannon entropy ( Ĥ) and auto-mutual information (AMI) I (m,p) ( Î)) for normal (N) and pathological (A for "abnormal") subjects, for Dataset I (top) and Dataset II (bottom). Each column corresponds to a different window size: T = 20 mn, 10 mn and 5 mn. All features are computed with m = 2, p = 1.

Figure 9 . 3 :

 93 Figure 9.3: AUC of ROC for Î(m,p) as a function of the total embedding dimension m + p, with m ≥ p, for time windows of size T = 20, 10, 5 mn, for data in Dataset I (first line) and Dataset II (second line). Black circles indicate the special case p = 1 corresponding to the classical definition of Shannon entropy rate (see eq.(1.25)). Red squares correspond to p ≥ 2.

Figure 9 . 4 :

 94 Figure 9.4: Left: Average AMI Î(3,3) for normal (black) and abnormal (red) subjects in Dataset I (delivery occurring less then 15 mn after pushing started). AMI is computed in windows of size T = 20 mn (first line), 10 mn (second line) and 5 mn (third line) shifted by 2 mn. The vertical magenta line indicates the beginning of Stage 2 (pushing). Right: Corresponding p-value. A single black + symbol in the AMI plot indicates a p-value lower than 0.05, two ++ indicate a p-value lower than 0.01. The gray-shaded region represents the time window used to compute the last value of AMI.

. 5 .Figure 9 . 5 :

 595 Figure 9.5: Left: Average AMI Î(3,3) for normal (black) and abnormal (red) subjects in Dataset II (delivery occurring more then 15 mn after pushing started). AMI is computed in windows of size T = 20 mn (first line), 10 mn (second line) and 5 mn (third line) shifted by 2 mn. The vertical magenta line indicates the beginning of Stage 2 (pushing), and the vertical blue line indicates delivery. Right: Corresponding p-value. Black + symbols in the AMI plot indicate a p-value lower than 5% (+) or lower than 1% (++). The gray-shaded region represents the time window used to compute the last value of AMI.

Figure 9 . 6 :

 96 Figure 9.6: Average behavior of AMI Î(3,3) (left) and Shannon entropy Ĥ (right) for normal (black) and acidotic (red) subjects in Dataset II (delivery occurring more than 15 mn after pushing started).Quantities are computed in windows of size T = 20 mn. The vertical magenta line indicates the beginning of Stage 2 (pushing). We have used a different color code for windows spanning both Stages 1 and 2: blue for normal subjects and magenta for acidotic ones.

Figure 9 . 7 :

 97 Figure 9.7: Entropy rate across scales (black) and entropy of the increments (blue) for: a) fractional Gaussian noise with H = -0.3, the red line indicates the slope 0, b) fractional Brownian motion with H = 0.7, the red line indicates the slope 0.7 c) process with H = 1.7 generated by integrating a fractional Brownian motion with H = 0.7, the red line indicates the slope 1.

6

  An information theory viewpoint of K41 theory 6.1 Turbulent experimental systems and signals . . . . . . . . . . . . . . . . . 6.2 Scaling of information in turbulence . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Grid turbulence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.2 Jet turbulence in Helium . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Conditioned Entropy rate . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Skewness of turbulent velocity increments . . . . . . . . . . . . . . . . . . 6.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1.1 KL divergence from Gaussianity . . . . . . . . . . . . . . . . . . . . 7.1.2 Distance from Gaussianity across scales . . . . . . . . . . . . . . . . 7.1.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Application to turbulent velocity signals . . . . . . . . . . . . . . . . . . . 7.3 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results for different observables of turbulence . . . . . . . . . . . . . . . . 8.4 Conclusions and perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . Information Theory to Probe Intrapartum Fetal Heart Rate Dynamics159 9.1 Datasets: Intrapartum Fetal Heart Rate Times Series and Labor Stages . 9.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.2.1 Features from Complexity Theory . . . . . . . . . . . . . . . . . . . 9.2.2 Connecting Complexity Theory and Information Theory . . . . . . 9.3 Results: Acidosis Detection Performance . . . . . . . . . . . . . . . . . . . Dynamical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Discussion, Conclusions and Perspectives . . . . . . . . . . . . . . . . . . . 9.4.1 Acidosis Detection in the First Stage . . . . . . . . . . . . . . . . . 9.4.2 Acidosis Detection in Second Stage . . . . . . . . . . . . . . . . . . 9.4.3 Probing the Dynamics . . . . . . . . . . . . . . . . . . . . . . . . .

	7 Rediscovering intermittency 7.1 III Biomedical Signal Processing 9 Contents 9.3.4 Discussion, Conclusions and Perspectives

7.3.1 Synthetic processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3.3 Phenomenological model : the propagator formalism . . . . . . . . 7.4 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Wiener Causality across scales of turbulence 8.1 Definitions of the scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Increments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.3 Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Results for different scale functions . . . . . . . . . . . . . . . . . . . . . . 8.2.1 Increments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Scales obtained with filtering . . . . . . . . . . . . . . . . . . . . . 8.2.3 Scales obtained by embedding . . . . . . . . . . . . . . . . . . . . . 8.3 9.3.1 Comparison of Features' Performance, Using a Single Time Window, Just Before Delivery . . . . . . . . . . . . . . . . . . . . . . . 9.3.2 Effect of Window Size on Performance . . . . . . . . . . . . . . . . 9.3.3 Effect of the Embedding Dimensions on the (Fetal Acidosis Detection) Performance of AMI . . . . . . . . . . . . . . . . . . . . . . .

Table 7 .

 7 1: The first three lines indicate the values of parameters (c 1 and c 2 and hence ζ(2)) used in the generation. Estimates ĉ1 , ĉ2 and ζ(

		fBm	MRW	log-N	log-P
	c 1	1/3	0.371	0.371	0.381
	c 2	0	0.025	0.025	0.036
	ζ(2)/2	1/3	0.345	0.345	0.345
	ĉ1	0.333	0.42	0.372	0.382
	ĉ2	1e -4	0.038	0.026	0.035
	ζ(2)/2	0.332	0.363	0.353	0.356
	∆ ln(τ ) H G (δ τ X) 0.33±0.01 0.37±0.01 0.35±0.01 0.35±0.01

  Figure 8.3: Maximum amplitude of the interaction between two scales, τ large and τ small , in function of the time of interaction (≈ τ largeτ small ). τ large is fixed to 66 and τ small varies. Black is for fBm H = 1/3 blue is for fBm H = 1/2 and red is for 3D-Turbulence. The magenta line is guideline of slope 0.0055.

	0.3
	0.2
	0.1
	0
	40	60	80

closer, see figure 8.3.

  9.2.2). Average values of Ĥ and Î(2,1) are larger for acidotic subjects. The larger value of Shannon entropy H indicates that the acidotic FHR signals contain more information. The larger value of AMI indicates a stronger dependence structure in the dynamics of abnormal subjects.For subjects in Dataset II, it is harder to find any tendency by looking at the average values.

Table 9 .

 9 2: AUC and p-value of Wilcoxon test of SampEn and AMI in datasets I and II using data from the last 20, 10 or 5mn before delivery.

	,1)	Î(3,3)

Acknowledgements

Dataset I

Dataset II AUC p-Value AUC p-Value ApEn (2) 0.76 4.08e-06 0.61 1.33e-01 SampEn (2) 0. [START_REF] Granero-Belinchon | Mutual information for intrapartum fetal heart rate analysis[END_REF]

Dataset II

False positive rate Î(

Î( includes dynamic information) leads to two satisfying estimates figure 9.1 and table 9.1 both show that the best performing estimators are SampEn (m) and Î(m,p) . In Dataset II, although all features performs more poorly than in Dataset I, SampEn and AMI are again the best ones, with a p-value lower than 0.05. We focus on these two features in the following.

Receiver operating characteristics: To compare the two best performing features, SampEn and AMI, we plot receiver operating characteristics (ROC) curves in figure 9.2, both for Datasets I and II, using data from the last T = 20 mn before delivery. For Dataset I, AMI better discriminates acidotic subjects from normal ones. For Dataset II, AMI discrimination is only slightly better than SampEn. The area under the curve (AUC) of the ROC of the features is reported in table 9.1, with bold font indicating the estimator with the largest AUC. Performance is much worse in Dataset II than in Dataset I: the AUC is reduced. Nevertheless, AMI is always the better performing estimator (its AUC reduces from 0.84 to 0.68), followed by SampEn (AUC reducing from 0.79 to 0.67).