Keywords: interestingness measures, Algorithmic properties

2.3.1.1 A prolongation of the association rule model . . 2.3.1.2 Finding the right interesting measure . . . . . . Subjective and objective interestingness measures.

Objective interestingness measures: a definition. . Objective interestingness measures for rules between itemsets? . . . . . .

the same object: data consisting of a vector x of input variables and a vector y of output variables. Nature operates in a certain manner to associate to each input variable x an output variable y but, to the data scientist, this is a black box. Both cultures also share the same two goals: prediction (what will be the outputs to future inputs?) and information (how does nature associate outputs to inputs?). However, their approaches differ. On one side of the spectrum, members of the data modeling culture which, he argued, was the mainstream culture of the statistics community at that time, would try to fill in the black box with a statistical model such as linear regression or logistic regression. On the other side, members of the algorithmic modeling culture would simply substitute nature's black box by another black box consisting of objects such as neural nets, forests or support vectors.

Professor Breiman was a strong proponent of the algorithmic modeling culture. According to this statistician, the data modeling culture belonged to the past as it could not answer the problems of the twenty first century and its huge data flow. Conversely, the algorithmic modeling culture was the way of the future and was supported by a younger generation of computer scientists, physicists and engineers. He argued that the gains in the accuracy of the predictions brought forth by the methods within this developing culture trumped the losses that this gain induced in terms of interpretability. This article gave rise to a certain number of critical comments, notably from the statisticians community, some of which were published along with the article [START_REF] David | statistical modeling: The two cultures]: Comment[END_REF][START_REF] Efron | statistical modeling: The two cultures]: Comment[END_REF].

However, the vision it defended also appealed to a certain number of actors in the data mining and machine learning communities who could identify to this algorithmic modeling culture. Such opinions became increasingly popular and trending in these communities and eventually mainstream, thus making room for even more radical visions.

The end of theory

In June 2008, WIRED magazine released an issue containing an editorial by Chris Anderson, its editor-in-chief at the time, entitled "The end of theory :

The data deluge makes the scientific method obsolete" [START_REF] Anderson | The end of theory: The data deluge makes the scientific method obsolete[END_REF]. In this article,

Anderson wrote:

Petabytes allow us to say: "correlation is enough". We can stop looking for models. [...] Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all. There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?

The point developed by Chris Anderson was that the vast amounts of data now available, a.k.a. Big Data, could be processed and mined to reveal information (in this case, scientific knowledge) without the need to understand where this information came from and if it could be explained. Once more, this article brought forth some strong opposition from members of the scientific community [START_REF] Pigliucci | The end of theory in science?[END_REF][START_REF] Manin | Kolmogorov complexity as a hidden factor of scientific discourse: from newton's law to data mining[END_REF][START_REF] Mazzocchi | Could big data be the end of theory in science?[END_REF][START_REF] Liu | Rethinking big data: A review on the data quality and usage issues[END_REF][START_REF] Gotterbarn | The creation of facts in the cloud: a fiction in the making[END_REF] but it also obtained a large adherence within the data mining and machine learning communities and industries. If a system works, why would you need to understand how or why it works? From a commercial and industrial perspective, this makes perfect sense: if you only need a system to work, and you trust this system to work, trying to understand how and why it works would simply represent an additional and superfluous cost. As cost efficiency is a main drive for commercial and industrial actors, meaningfulness is often absent in data mining and machine learning algorithms. The opinion that meaningfulness and interpretability could be sacrificed to the benefit of higher accuracy in predictions, or simply economic efficiency, has continued its expansion up until today both within the computer science community and more broadly, to businesses and society in general [

The right to an explanation

The general development of this opinion and particularly the consequences of its influence, ranging from the production of scientific research to the implementations of everyday algorithms, has also led many to oppose and reject it These include of course statisticians and members of the scientific community, but also men and women within the civil society thus inciting policy makers to react [RS17, O'N16, CGM14, ZBB + 17, VBB + 18a, EU216]. In her 2016 bestseller, Weapons of Math Destruction : How Big Data Increases Inequality and Threatens Democracy, Cathy O'Neil advocates for transparency, fairness and accountability in algorithmic models [O'N16]. "Opaque and invisible models are the rule," she says, "and clear ones very much the exception", but if a decision is made based on the conclusions of a black box algorithm, how can this decision be contested? The accountability of algorithmic models depends therefore on their explainability. Such accountability became mandatory in the European Union after the implementation of the General Data Protection Regulation in May 2018 [EU216] which also introduced a new "right to an explanation" of automated decision-making [START_REF] Edwards | Enslaving the algorithm: From a "right to an explanatio" to a "right to better decisions[END_REF]. In 2018, Cédric Villani, Fields medalist and member of the French parliament, produced a six part report on artificial intelligence for the French government, one of which deals solely with the ethics of artificial intelligence [VBB + 18a]. In this section, he defends that priority should be given to increasing the transparency of automated processes and the possibility to audit them, by massively funding research on explainability and aiming at "opening the black boxes" within machine learning algorithms. Explaining artificial intelligence is also a priority for the United States military which started funding the "Explainable Artificial Intelligence (XAI)" program through its Defense Advanced Research Projects Agency in 2016 [START_REF] Gunning | Explainable artificial intelligence (xai)[END_REF]. This ongoing program funds thirteen research projects which aim to "produce more explainable models, while maintaining a high level of learning performance (prediction accuracy)" and "enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners".

A culture shock

Coming from a background in fundamental mathematics, specifically algebraic geometry, I was myself very much surprised by the algorithmic model culture which I discovered in the field of machine learning and data mining. Indeed, the fundamental mathematics culture is one based on the notion of deductive reasoning, while the algorithmic model culture is based on inductive reasoning. In a sense, the scientific culture is located somewhere in between the two because it is a subtle mix between logical and empirical approaches [Pop59, [START_REF] Lakatos | Philosophical papers. volume i: The methodology of scientific research programmes; volume ii: Mathematics, science and epistemology[END_REF][START_REF] Chalmers | What Is This Thing Called Science?[END_REF][START_REF] Potochnik | Idealization and the Aims of Science[END_REF]. That is why pure mathematics is generally not considered a science but rather an art ([Rus07, Har40, Dev00, Cel15]), a language ([Gal23, FP88, CC89, Sch71, O'H04, RSHF15]) or a philosophy ([Kör60, GV01, Jac01, Bro08]). Conversely, pure empiricism is no more than observation.

From my perspective as a mathematician, good mathematical modeling is an inextricable part of high quality scientific research and an indispensable means to achieve explainability, meaningfulness and accountability. I therefore needed to understand why the mainstream culture of an entire scientific community (i.e. the machine learning and data mining communities) would be adverse to mathematical modeling. Many aspects of this thesis may be better understood in light of this culture shock and the subsequent will to create a bridge between different cultures. In a sense, this can be seen as the underlying premise of this thesis as it is founded on the coming together of three different persons representing three different cultures: my supervisor, Philippe Lenca, who is a computer scientist, my co-adviser, Stéphane Lallich, who is a statistician, and myself, a mathematician. How my mathematical knowledge and mathematical approach could ben-efit the fields of data mining, machine learning and artificial intelligence was a question I kept in mind during the entire duration of my doctoral research.

But I also came to understand much more about my own culture, about mathematics and mathematical modeling; how and why they bring meaningfulness to scientific research. I also came to understand their limitations and the need for compromise in order to reach and satisfy the goals and expectations of the field in which I was working.

Objective frequency-based interesting pattern mining

The specific domain in which I conducted my research was frequency-based 1 pattern mining with a particular focus on rule and itemset mining as well as the search for objectively interesting patterns. I present a brief review of the history and state-of-the-art of these topics in chapter 2. One of the first goals which we set up together with my supervisor was to try to understand if and how meaningfulness was a criteria for defining interestingness in current itemset and rule mining procedures. This exercise helped me establish a link between meaningfulness and mathematical modeling. I therefore undertook the task of characterizing mathematical modelings in frequency-based pattern mining while keeping in mind their propensity to be explained and carry meaning. The product of this research constitutes chapter 3 of this thesis. Its contents pertain to the fields of pattern mining, applied mathematics and the philosophy of science. The contributions of the thesis within chapter 3 include:

• The presentation of a novel framework for the qualitative analysis of modeling processes and their meaningfulness comprising a new formulation for the notions of model and modeling and the definition of new concepts for the characterization of modeling processes such as phenotypic and genotypic modeling, pragmatic modeling, and patchwork and holistic modeling. 1 We use the terminology frequency-based pattern mining rather than the more common frequent pattern mining terminology because this last term is used ambiguously to refer both to the process of mining patterns that are frequent in the data (frequent pattern mining) or to the process of mining patterns based on their frequency in the data (what we call frequency-based pattern mining).

• An analysis of the impact in terms of meaningfulness of various modeling choices in frequency-based pattern mining based on these tools.

• The presentation of a Boolean lattice type structure fr representing the patterns to be mined and the demonstration that there are objective arguments for considering itemsets rather than other types of patterns (including association rules) in frequency-based pattern mining.

• A new approach towards reconciling the hypothetico-deductive model of the scientific method and frequency-based pattern mining, based on a notion of confidence in the empirical data.

While chapter 3 focuses on establishing a number of general recommendations for the definition of meaningful mathematical modelings in objective frequency-based interesting pattern mining, chapter 4 concentrates on the definition of specific mathematical models following some of these recommendations. The main objects in chapter 4 are mutual constrained independence (MCI) models which are particular cases of MaxEnt models. The definition of these models is the result of a two phase process which emerged during my doctoral research: first, a more specific model (easier to formalize and compute); second, a generalization of this previous model whose definition and computation rely on much more elaborate mathematical tools, notably based on modern algebraic geometry. The presentation of the mathematical modelings which lead to these models, together with their mathematical definitions and properties, as well as novel algorithms for computing them, make up for chapter 4 of this thesis. The contributions of the thesis within chapter 4 include:

• The mathematical proofs for the existence and the characterization of MCI models, as well as their relationship to MaxEnt models.

• The algebraic expressions and mathematical properties for MCI models in which the constraints are defined on all proper subitemsets of an itemset.

• The algebraic expressions for all MCI models in which the number of items m is equal to 4 or less.

• The presentation of an algorithm based on tools from algebraic geometry to determine the algebraic expression for any MCI model.

The last chapter of this doctoral thesis concentrates on mining algorithms which allow to extract interesting patterns, following the principles for mathematical modeling elaborated in chapter 3 and the mathematical tools developed in 4. As the limits of a direct implementation of the principles previously defined are made apparent, further paths in research are suggested. The contributions of the thesis within chapter 5 include:

• The reduction of objective frequency-based interesting pattern mining to a specific mathematical optimization problem.

• The presentation of an operational algorithm, based on a greedy strategy, for solving this problem.

Acknowledgments

This thesis is dedicated to my son Sacha Ali Karl Delacroix Sadighiyan who came into this world halfway through the completion of my doctoral research.

Thank you for giving me the strength to go through with this task. I hope that you will be proud of me for this accomplishment as you grow up. I thank my wife Saena for her continuing support and love, as well as the inspiration she has provided to me during all these years. I thank my parents for their support without which I would not have managed to pursue this work. I thank my supervisor, Philippe Lenca, and co-adviser, Stéphane Lallich, for accompanying me and advising me all along the way. I thank Jean-Paul Haton and Gilbert Saporta for their meticulous work in reviewing my thesis. I also thank the further members of the jury Jérome Azé, Pascale Kuntz and Franck Vermet for assessing my research. I thank Alain Hillion and Cédric Wemmert for their participation in the Follow-up Committee for this thesis. I would like to thank all my former colleagues, particularly those at IMT Atlantique and the University of Strasbourg, who have helped me develop my understanding of the various fields of study that I have approached in this thesis. I would also like to thank the researchers that I have met and discussed with at all the seminars and conferences which I have attended during these years. I thank all the researchers whom I have not met and whose research has nourished my own. I thank all the friends that have followed me through this endeavor and who have helped my ideas mature through our discussions. Last, I thank the open science community and particularly Alexandra Elbakyan for their invaluable contribution to scientific progress.

CHAPTER 2

Mining objectively interesting itemsets and rules -History and state-of-the-art

Early developments

For more than a century now, quite a few decades before the dawn of the Information Age, mathematicians and scientists alike have tried to define methods for exhibiting interesting associations and relations between nominal attributes based on observed frequencies in data. As such, Karl Pearson, who was already working "on the correlation of characters not quantitatively measurable" at the end of the nineteenth century by analyzing contingency tables [START_REF] Pearson | mathematical contributions to the theory of evolution.-vii. on the correlation of characters not quantitatively measurable[END_REF], can be seen as a pioneer. By developing the first chi-squared test in 1900 and later adapting it to be able to test for independence in contingency tables in 1904 [START_REF] Stephen | Statistics on the table: The history of statistical concepts and methods[END_REF], he set the foundations to the first formal mathematical method for analyzing such associations. His method was specified and corrected by Ronald A. Fisher in the 1920s to obtain what is now taught as Pearson's chisquared test for statistical independence [START_REF] Fisher | On the interpretation of χ 2 from contingency tables, and the calculation of p[END_REF][START_REF] Fisher | The conditions under which χ2 measures the discrepancey between observation and hypothesis[END_REF][START_REF] Jaynes | Probability theory: The logic of science[END_REF]. This method, as well as most early methods on nominal attribute association analysis, were only aimed at rejecting (or not) hypotheses individually defined by humans and which mostly involved only two different attributes.

The development of the computer and the beginning of the digital revolution allowed for much more possibilities. New approaches started to develop aiming both at considering relations and associations between much more than simply two attributes and at using automated systems to discover these associations. In 1966, a team of researchers at the University of Prague, Peter Hàjek, Ivan Havel and Metoděj Chytil, published a paper entitled "The GUHA Method of Automatic Hypotheses Determination" [START_REF] Hájek | The GUHA method of automatic hypotheses determination[END_REF] which presented the foundations of a theory for extracting logical relations between nominal attributes of significant interest. This theory, deeply rooted in fundamental logics and statistical analysis, continued its development with a number of puplished scientific articles and a book "Mechanizing Hypothesis Formation (mathematical foundations for general theory)", published by Springer-Verlag in 1978, but its impact remained quite local, mostly at the University of Prague [START_REF] Hájek | Mechanizing hypothesis formation: Mathematical foundations for a general theory[END_REF][START_REF] Hájek | The GUHA method and mining association rules[END_REF][START_REF] Holeňa | Fuzzy hypotheses for GUHA implications[END_REF][START_REF] Hájek | The GUHA method and its meaning for data mining[END_REF]. It seems that the combined barriers represented by the language (a lot of publications on the GUHA were in Czech), the geopolitical situation at the time (Prague was located behind the Iron Curtain), cultural differences in terms of scientific culture (the literature on the GUHA method, even in English, can seem exceedingly cryptic for someone who is not accustomed with the formal theory of mathematical logic) and simply bad timing (the GUHA method developed at the early beginning of the digital revolution) did not allow for a large dissemination and global recognition of the historical precedence in the field of itemset and rule mining of the work conducted on the GUHA method in Prague.

In the 1980s, numerous studies for characterizing rule type patterns and extracting them from datasets started to develop [Cen87, CN89, Qui87, LGR81a, PS91]. While some notions and approaches stayed quite confined to a restricted circle (see, for example, statistical implicative analysis which continues its mostly separate development up to this day [GL92, GBPP98, GRMG13]), a few dominant trends and key notions started to emerge. These included a differentiation between rules that are always correct (exact rules) and rules that are almost always correct (strong rules), together with the idea that the strength (or interest) of a rule could be characterized by a "rule-interest measure" (the terminology later settling on "interestingness measure").

Enjoying now a much more favorable conjuncture and building on the principles for characterizing strong rules suggested by Gregory Piatetsky-Shapiro [START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF], the framework for mining frequent itemsets and association rules which was developed at the IBM Almaden Research Center in the 1990s became quickly widespread. This framework was first presented in Mining Association Rules between Sets of Items in Large Databases, a 1993 article by Rakesh Agrawal, Tomasz Imieliński and Arun Swami [START_REF] Rakesh Agrawal | Mining association rules between sets of items in large databases[END_REF] and quickly followed in 1994 by Fast Algortihms for Mining Association Rules by Rakesh Agrawal and Ramakrishnan Srikant [START_REF] Agrawal | Fast algorithms for mining association rules[END_REF] which presented the now famous Apriori algorithm. By contrast with the GUHA method, which contained descriptions of objects equivalent to association rules in its 1966 version, the great simplicity of both the framework and the Apriori algorithm made it easily accessible to a large number. Furthermore, the presentation of the framework towards its direct application to market basket analysis, in both articles, undoubtedly appealed to many more. Through the general adherence it had gained, this framework was rapidly established as the reference for mining associations and relations between nominal attributes in data. The Apriori algorithm figured in the famous list of the "Top 10 algorithms in Data Mining"

[WKRQ + 08] and the current most popular textbooks in data mining all confer an important part to frequent itemset mining and association rule mining [START_REF] Han | Data mining: concepts and techniques[END_REF][START_REF] Zaki | Data mining and analysis: fundamental concepts and algorithms[END_REF][START_REF] Charu | Data mining: the textbook[END_REF][START_REF] Hastie | The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition[END_REF][START_REF] Tan | Introduction to Data Mining (2Nd Edition)[END_REF]. As for the two articles figuring above, they now rank amongst the most cited articles in the field of data mining, or even within the more general field of computer science, with respectively 21,285 and 24,654 referenced citations by Google Scholar as of August 2019.

When conducting research in the field of itemset mining, a recollection of its genealogy as presented above is far from superfluous. The canons in terms of terminology and representations have been defined and structured by this history. It is quite notable that the presentations of modern itemset and rule mining have strongly inherited from their early ties to market basket analysis. Such examples as the common storytelling of the discovery of a relation between beer and diaper sales by a large retail store company, even though it is mainly mythology (see [START_REF] Daniel | [END_REF] for a detailed explanation), helped to forge many of the representations of the people who study and work with itemset mining. Understanding the history allows one to perceive the reasons behind certain of the constraints and limitations of the mainstream framework and question their necessity. By acknowledging that not all choices by a given scientific community are motivated by scientific reasons, it is easier to depart from this mainstream framework when necessary.

Frequent itemsets and association rules

2.2.1 The models 2.2.1.1 Frequent itemsets

The canonical example of a dataset in itemset mining is a dataset consisting of information on the purchases of customers at a retail store which sells items such as beer, diapers, eggs, milk and bread. The information in the data is very basic. For each individual purchase, called a transaction, we know which set of items were bought from the store. We do not know which quantity of each item was purchased nor their price, neither do we possess any further information regarding the transactions (such as a customer ID or a time of purchase), we only know that it corresponds to a unique transaction. This gives rise to the following data representation: a table of transaction IDs together with the corresponding transactions. The transaction IDs (often generally shortened to TID) can be any unique identifier and, although they are usually numerical, their ordering is irrelevant to the model. The transactions themselves can be represented as sets of items also known as itemsets. The corresponding mathematical model can be formalized as follows. Let I = {a 1 , ..., a m } be a set of m elements hereinafter referred to as items. Define an itemset X to be a subset of the set of items I, i.e. X ⊂ I. Define a database of transactions to be a set T = {T 1 , ..., T n } of n itemsets hereinafter referred to as transactions. The support of an itemset X in a database of transactions T is the proportion supp T (X) (or simply supp(X) if there is no ambiguity) of transactions in T that contain X:

supp(X) = card ({T ∈ T | X ⊂ T }) card (T )
The frequent itemset mining problem is defined as determining the set of itemsets FI T ,minsupp (or simply FI if there is no ambiguity) for which the support in the database T is greater than a given threshold minsupp.

Itemset Support {a 3 } 0.7 {a 1 } 0.5 {a 1 , a 3 } 0.4 {a 2 } 0.4 {a 4 } 0.4 {a 3 , a 4 } 0.3 Table 2.2: The frequent itemsets ordered by support for minsupp = 0.3 for the database in Table 2.1.

There is much to say about the adequacy and the meaningfulness of the modeling choices resulting in the definition of the transaction/itemset model and the frequent itemset problem. This will be addressed specifically in the next chapter.

Association rules

Historically, the mining of rule type patterns was the main focus of frequent pattern mining because rules offered greater interpretability than most other patterns. Itemsets, which are conjunction type patterns, were originally only designed for the purpose of defining association rules [START_REF] Rakesh Agrawal | Mining association rules between sets of items in large databases[END_REF][START_REF] Agrawal | Fast algorithms for mining association rules[END_REF]. An association rule is a rule between two disjoint itemsets X and Y , noted X → Y , which is characterized by two measures. The support of the rule, noted supp(X → Y ), measures the observed frequency of the rule, that is the proportion of transactions that contain both itemsets. The confidence of the rule, noted conf(X → Y ), measures the observed frequency of Y when X occurs.

Formally this is expressed as follows.

Consider a set of items I and a database of transactions T as defined previously. Let X, Y ⊂ I. Then X → Y is an association rule if X ∩ Y = ∅ and supp(X ∪ Y ) = 0. The antecedent and the consequent of the rule are X and Y respectively. The support and confidence of the rule are defined by

supp(X → Y ) = supp(X ∪ Y ) and conf(X → Y ) = supp(X ∪ Y ) supp(X)
The association rule mining problem is defined as determining the set of association rules AR T ,minsupp,minconf (or simply AR if there is no ambiguity) for which the support in the database is greater than a given threshold minsupp and the confidence is greater than a given threshold minconf.

Association rule Support Confidence {a 1 } → {a 3 } 0.5 1 {a 3 } → {a 1 } 0.5 0.71 {a 4 } → {a 3 } 0.3 0.75 Table 2.3: Association rules for minsupp = 0.3 and minconf = 0.5 for the database in Table 2.1.

Algorithms

Solving both the frequent pattern mining problem and the association rule mining problem has been an important focus of research in this field and, as a result, quite a few algorithms for solving these problems have been suggested.

As this would lead us astray from the scope of this thesis, we do not dwell much on the specifics of these different mining methods. We will briefly present the general principles behind the Apriori algorithm, as these are intrinsically tied to the elaboration and development of the frequent itemset and association rule models, and simply recall the subsequent development of alternative approaches.

The Apriori algorithm

The Apriori algorithm holds a special place in the history of frequent itemset mining because the development of the field itself can be partly attributed to the popular success of this algorithm. Furthermore, even though it was presented in the second of the two founding articles previously cited, its underlying principles were already present in the first article. As we will discuss in the following chapter, this raises issues regarding the rationale behind the definition of the frequent pattern and association rule models.

The Apriori algorithm relies on two aspects which are related to the representation and definition of these models. The first aspect on which it relies is the underlying lattice structure of the set of itemsets ordered by inclusion. The second aspect is the monotonicity properties of the support and confidence measures which can be expressed as follows.

∅
Proposition 2.2.1 (Support anti-monotone property). Let X 1 , X 2 be two itemsets such that X 1 ⊂ X 2 , then supp(X 1 ) ≥ supp(X 2 ).

This anti-monotone property leads directly to the following downward closure property also known as the Apriori principle.

Corollary 2.2.0.1 (Apriori principle). The subsets of a frequent itemset are also frequent.

And it also leads easily to the confidence monotone property.

Proposition 2.2.2 (Confidence monotone property). Let X 1 → Y 1 and X 2 → Y 2 be two association rules such that X 1 ⊂ X 2 and

X 1 ∪ Y 1 = X 2 ∪ Y 2 , then conf(X 1 → Y 1 ) ≤ conf(X 2 → Y 2 ).
These properties are essential in the process defined by the algorithm. Indeed, for discovering frequent itemsets, the Apriori algorithm follows a bottomup, level wise approach. This means it scans the itemset lattice to find frequent itemsets starting from the bottom and moving upwards layer by layer.

Of course, a complete scan is practically infeasible because the number of nonempty itemsets is equal to 2 m -1 where m is the number of items. To tackle this issue, the algorithm uses the Apriori principle to prune off entire branches of the lattice which it knows does not contain frequent itemsets: it will only scan the supersets of itemsets which are frequent by generating candidates (i.e. potential frequent itemsets) in a given layer from the frequent itemsets already discovered in the previous layer. The algorithm also makes use of the natural lexicographic ordering between itemsets within the lattice to avoid multiple scans of a single itemset. Similarly, a brute force approach towards association rule mining is technically infeasible because the number of potential association rules is equal to 3 m -2 m+1 + 1. The Apriori algorithm uses then a top-down, level wise approach on each lattice of potential association rules corresponding to a frequent itemset of at least two items (as represented in Table 2.2) while using the confidence monotone property for pruning.1

{a 1 , a 2 , a 3 , a 4 } ↓ ∅ {a 2 , a 3 , a 4 } ↓ {a 1 } {a 1 , a 3 , a 4 } ↓ {a 2 } {a 1 , a 2 , a 4 } ↓ {a 3 } {a 1 , a 2 , a 3 } ↓ {a 4 } {a 3 , a 4 } ↓ {a 1 , a 2 } {a 2 , a 4 } ↓ {a 1 , a 3 } {a 2 , a 3 } ↓ {a 1 , a 4 } {a 1 , a 4 } ↓ {a 2 , a 3 } {a 1 , a 3 } ↓ {a 2 , a 4 } {a 1 , a 2 } ↓ {a 3 , a 4 } {a 4 } ↓ {a 1 , a 2 , a 3 } {a 3 } ↓ {a 1 , a 2 , a 4 } {a 2 } ↓ {a 1 , a 3 , a 4 } {a 1 } ↓ {a 2 , a 3 , a 4 }

Other mining methods

As interest grew for frequent pattern mining and the research that had been developed at the IBM Almaden Research Center became widespread in the data mining community, many researchers took upon them to solve the frequent itemset mining problem as efficiently as possible. The Apriori algorithm was indeed relatively easy to apprehend and implement but it was still generally quite slow and would not function for lower values of minimal support. Defining the most efficient algorithm became a computational challenge for researchers and the main focus of early research in itemset mining [START_REF] Charu | Frequent Pattern Mining Algorithms: A Survey[END_REF][START_REF] Goethals | Survey on frequent pattern mining[END_REF][START_REF] Han | Frequent pattern mining: current status and future directions[END_REF], reaching a peak with the workshops on Frequent Itemset Mining Implementations in the early 2000s [START_REF] Fimi | Proceedings of the IEEE ICDM workshop on frequent itemset mining implementations (FIMI '03)[END_REF][START_REF] Fimi | Proceedings of the IEEE ICDM workshop on frequent itemset mining implementations (FIMI '04)[END_REF].

The majority of the different algorithms which were proposed can be characterized by their differences to Apriori with respect to three different aspects. Firstly, many algorithms focused primarily on improving the cost of the support-counting process while retaining an Apriori-like structure. The Direct Hashing and Pruning algorithm accomplished this by trimming items from transactions [START_REF] Soo | An effective hash-based algorithm for mining association rules[END_REF]; other algorithms, such as Apriori LB [START_REF] Bayardo | Efficiently mining long patterns from databases[END_REF], used mathematical properties of the support function to avoid this process altogether; vertical algorithms, such as Monet [START_REF] Holsheimer | A perspective on databases and data mining[END_REF], Partition [START_REF] Savasere | An efficient algorithm for mining association rules in large databases[END_REF],

Eclat [START_REF] Mj Zaki | New algorithms for fast discovery of association rules[END_REF][START_REF] Zaki | Scalable algorithms for association mining[END_REF] or VIPER [SHS + 00], used the transposed of the sparse matrix representation of the transaction database to perform efficient support-counting; and projection-based algorithms, such as TreeProjection [START_REF] Agarwal | Depth first generation of long patterns[END_REF][START_REF] Agarwal | A tree projection algorithm for generation of frequent item sets[END_REF] or FP-Growth [START_REF] Han | Mining frequent patterns by patterngrowth: Methodology and implications[END_REF], used local projected databases when considering an itemset. Secondly, some algorithms explored other approaches for scanning the itemset lattice than the breadth-first (i.e. level wise) approach of Apriori. This included depth-first algorithms such as the TreeProjection algorithm in [START_REF] Agarwal | Depth first generation of long patterns[END_REF], FP-Growth or dEclat [START_REF] Zaki | Fast vertical mining using diffsets[END_REF], or combined breadth-first and depth-first approaches such as the TreeProjection algorithm in [START_REF] Agarwal | A tree projection algorithm for generation of frequent item sets[END_REF].

Lastly, some algorithms, such as FP-Growth, introduced specific data structures for a compressed representation of the databases. Pattern growth algorithms, following the example set by FP-Growth, are now considered to be the state-of-the-art of frequent pattern mining algorithms [START_REF] Charu | An Introduction to Frequent Pattern Mining[END_REF][START_REF] Han | Pattern-Growth Methods[END_REF].

Interestingness

Having reached its peak a decade or so following their introduction, the enthusiasm for the original frequent pattern mining and association rule problems started to dwindle. The state-of-the-art algorithms were generally capable of mining large databases for frequent itemsets and association rules but the usefulness of the results they produced was put into question. Indeed, in many applications, the patterns extracted for high values of support and confidence were quite obvious. Conversely, when lowering the values for support and confidence, the number of patterns extracted grew exceedingly large. The end user was stuck between gathering a small amount of information which brought little insight about the data on the one hand, and dealing with an information overload on the other hand. Neither one of these represented interesting pieces of information to the user. The promise of the discovery of a small number of interpretable yet unexpected patterns, supported by the beers and diapers mythology, failed to deliver. We will refer to this general issue as the interestingness issue. From a chronological perspective, it is important to note that the interestingness issue did not appear in research specifically during the first decade of the twenty-first century: it had been addressed explicitly right after the development of the itemset and association rule models and even before that. However, this period does correspond to a much larger development of this topic in research.

The notion of interest, as it is commonly understood, is of course essentially subjective (though some of the research focuses on objective qualifications of interestingness while others deal with specifically subjective aspects). Therefore a large variety of research focusing on finding interesting patterns has been conducted within the field of frequent pattern mining (or more generally frequency-based pattern mining) some of them holding radically different views of what it means for patterns to be interesting, leading to entirely diverging branches in the field. For example, research that focuses on rare patterns suggest that such outliers are the most interesting patterns in many contexts, often using the extraction of gold nuggets from a mine as an analogy. This view point is entirely antithetical to the frequent pattern mining approach which suggests that the most frequent patterns are the most interesting ones. This led to the development of an entire branch of frequent pattern mining research which actually focuses on infrequent or negative frequent patterns [SON98, KR05, AWF07, SNV07, TSB09, SBK12, MSG14, ALZ14, Agg17]. This shows to tell that the notion of interestingness may have a great number of possible definitions and, though we will try to present the most significant approaches towards interestingness, we cannot address them all.

Interestingness measures

One of the first approaches towards the issue of interestingness was to consider alternative or additional interestingness measures to determine the patterns that were truly interesting. This approach, which focused much more on rules than itemsets, was both a prolongation of the association rule model and a renewal with previous methods.

A prolongation of the association rule model

Indeed, the association rule model identifies interestingness with high support and high confidence: two measures of interestingness. Hence, for someone who accepts the general idea behind the association rule model, the strategy for solving the interestingness issue is quite simple. This depends on how the issue itself is perceived. Either the association rules mined are deemed uninteresting. In this case, one can believe it is simply because the measures of interestingness were not the right ones and they should be replaced by other measures. Or the rules are indeed considered interesting but not interesting enough, leading to an excessive number of patterns. In this case, using additional measures for interestingness might lead to the extraction of a fewer number of genuinely interesting rules. The Apriori algorithm already consisted of two successive steps each corresponding to a different interestingness measure: first, find the rules with high support, and second, find the rules with high confidence amongst the rules with high support. Adding a third step, based on a third measure, may seem like a natural progression, and such a process could be iterated as many times as needed until gaining satisfaction.

In both cases, the solutions are compatible with the general principles which justify the association rule model: interesting rules are those whose measure of interestingness is above a given threshold. As such these approaches are prolongations of the association rule model.

Finding the right interesting measure

In order to adopt these strategies, one must have alternative interestingness measures at ones disposal. A number of researchers therefore undertook the task of proposing a large number of interestingness measures, as well as different principles and strategies to choose an interesting measure wisely.

Subjective and objective interestingness measures. To accomplish such a task, one must first define what it means to be an interestingness measure.

One of the main debates relative to this issue, which resulted in the development of two very separate branches of research, focused on considering either subjective interestingness measures or objective interestingness measures. The advocates for subjective interestingness measures suggested that the main reason explaining that patterns extracted through standard mining procedures were not considered interesting was because they conformed to the preconceptions on the data held by the users. For the proponents of subjective interestingness, if the beer and diapers association was a good example of an interesting pattern, it was because it went against our common subjective belief that beer and diapers should not be purchased simultaneously. Methods relying on interestingness measures which integrated the user's subjective belief system were proposed to extract rules which did not conform to those beliefs.

In such methods, the user's belief system could be directly specified beforehand by the user [LHC97, LHML99, ST95, ST96] or learned from the user's feedback on the interestingness of proposed patterns [START_REF] Sahar | Interestingness via what is not interesting[END_REF][START_REF] Adomavicius | Expertdriven validation of rule-based user models in personalization applications[END_REF]. However, for the proponents of objective interestingness measures, one of the main aims of data mining is to relieve the user from having to analyze the data. For them, even though a mining process may be defined by user specifications, the process itself should not rely on the user but simply on the data. As the research in this thesis focuses on objective interestingness (although, as made explicit in the following chapter, we adopt a much stricter view on the definition of objectivity), we will focus slightly more on such objective measures.

Objective interestingness measures: a definition. The general consensus regarding the definition of an objective interestingness measure for a rule of type X → Y (up to a few exceptions as will be explicited further in this section) is that it is a function of four parameters n, n X , n Y and n X∪Y , corresponding respectively to the size of the dataset and the observed absolute frequencies in the dataset of X, Y and X ∪ Y2 , which models the interestingness of the rule by returning a real value [OKO + 04, TKS04, McG05, GH06, Vai06, LMVL08, BGK09, LB11, Hah15]. In other words, interestingness is quantified by a function with real values defined on the set of all possible contingency tables for two nominal variables X and Y , as there is a one-to-one correspondence between such contingency tables and the four parameters previously stated. In most cases, this definition can be restricted to three parameters corresponding to the relative frequencies f X , f Y and f X∪Y of observing respectively X, Y and X ∪ Y in the data. Note that, in the vast majority of research papers concerning objective interesting measures there is no distinction between observed frequencies (f X , f Y and f X∪Y ) and probabilities (p X , p Y and p X∪Y ) and the latter notation is often preferred together with the term probability (with a few rare exceptions such as [START_REF] Greco | Properties of rule interestingness measures and alternative ap-proaches to normalization of measures[END_REF] which makes this quite explicit). This leads to modeling issues as we will discuss in the following chapter.

X ¬X Y n X∪Y n Y -n X∪Y ¬Y n X -n X∪Y n + n X∪Y -n X -n Y X ¬X Y f X∪Y f Y -f X∪Y ¬Y f X -f X∪Y 1 + f X∪Y - f X -f Y
Regarding the specifics of the function or even which values should model low or high interestingness (such as 0, 1 or +∞) there is no consensus. Therefore a great number of functions can be considered. To be entirely precise, out of all the objective interestingness measures proposed in the scientific literature which we have scrutinized, only five of these utilized the itemset structure within the patterns to measure interestingness.

Out of these five measures, one of them, the cross-support ratio, was initially not intended to be an interestingness measure but rather described the upper bound of another interestingness measure in [START_REF] Xiong | Mining strong affinity association patterns in data sets with skewed support distribu-tion[END_REF] (this "interestingness measure" was likely classified as such by mistake in [START_REF] Hahsler | A probabilistic comparison of commonly used interest measures for association rules[END_REF]). Three other measures only addressed the interestingness of itemsets and not rules: the all-confidence ([Omi03] or h-confidence in [START_REF] Xiong | Mining strong affinity association patterns in data sets with skewed support distribu-tion[END_REF]); the collective strength [START_REF] Charu | A new framework for itemset generation[END_REF]; and lift as defined in [START_REF] Vreeken | Interesting Patterns[END_REF]. Therefore, the only remaining measure on that list: improvement [START_REF] Bayardo | Efficiently mining long patterns from databases[END_REF], was the one and only objective interestingness measures for rules proposed, out of more than fifty, that utilized the itemset structures of the antecedents and consequents of the rules. itself is left to the user. The database is queried for frequent patterns which satisfy a user-defined constraint formulated using a typical data query language syntax. As this branch of pattern mining has strong ties to the data management communities, it shares its traditions which explains why the focus is set on the algorithmic properties of interestingness measures. Indeed, database management algorithms rely strongly on the algorithmic properties of the functions defined within the query languages to provide fast responses.

Good modeling properties. In a defining article [START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF] for interestingness measures, Gregory Piatetsky-Shapiro suggested "several intuitive principles that all rule-interest functions should satisfy". These principles corre-sponded to three mathematical properties that an objective interestingness measure µ of parameters n, n X , n Y and n X∧Y3 should satisfy to be a good modeling of interestingness, namely that:

(P 1 ) µ = 0 if n X∧Y = n X n Y n (i.e.
if X and Y are statistically independent, the rule is not interesting).

(P 2 ) µ increases with n X∧Y when all other parameters remain unchanged.

(P 3 ) µ decreases with n X (or n Y ) when all other parameters remain unchanged.

Relying on the pragmatic algorithmic argument that the simplest function satisfying these principles would be more easily computed, this led him to suggest the following objective interestingness measure:

µ PS (n, n X , n Y , n X∧Y ) = n X∧Y - n X n Y n
Following Piatetsky-Shapiro's lead, researchers suggested a growing number of properties that an objective interestingness should satisfy in order to be a good model for interestingness. A fourth principle, similar in its mathematical formulation, was added to the list in [START_REF] Major | Selecting among rules induced from a hurricane database[END_REF]. [START_REF] Tan | Selecting the right objective measure for association analysis[END_REF] added five more properties to the list, which were presented as mathematical properties of the matrix operators that are interestingness measures, such as symmetry under variable permutation, row and column scaling invariance or antisymmetry under row or column permutation. These five properties were not, however, presented as necessary for a good mathematical model of interestingness but rather as potentially relevant depending on the specific context and the specific view towards interestingness which was adopted by the user. Furthermore, [START_REF] Tan | Selecting the right objective measure for association analysis[END_REF] suggested that certain mathematical properties which relied on arbitrary choices (such as the choice of 0 in (P 1 )) could be made less strict by accepting measures which would satisfy the property if conveniently normalized. The measures were classified with respect to the properties presented.

Furthermore, this paper analyzed how these measures ranked a number of contingency tables, comparing this to rankings suggested by experts.

Expanding on previous work in French [Lal02, LMP + 03, LMV + 04, LT04, GCB + 04] in which interestingness measures had been described using at least five additional properties, [START_REF] Lenca | On selecting interestingness measures for association rules: User oriented description and multiple criteria decision aid[END_REF] selected eight properties of objective interestingness measures to be used in a multicriteria decision aid (MCDA) process to help a user choose the most adapted interestingness measures. The MCDA process described required the intervention of two different experts, an expert analyst (expert in MCDA and KDD) and an expert user (expert in the data). Out of the eight properties presented in the paper, five were attached to the expert analyst with the three remaining attached to the expert user.

Though the majority of these criteria, six to be precise, were clearly mathematically defined, the definitions of the two remaining were more subjective.

These were defined as the "easiness to fix a threshold " and the "intelligibility" of the interestingness measures and were both evaluated using a nominal score:

easy or hard for the first criteria; a,b or c for the second criteria.

A total of fifteen properties of objective interestingness measures were described in a thorough survey on both subjective and objective interestingness by L. Geng and Howard J. Hamilton [START_REF] Geng | Interestingness measures for data mining: A survey[END_REF], two of which were presented as novel with the thirteen remaining associated to the papers referenced above. Later, the study of the robustness (i.e. the ability to tolerate noise in data) of the various interestingness measures was accomplished in [START_REF] Yannick | Mesure de la robustesse de règles d'association[END_REF][START_REF] Yannick | A robustness measure of association rules[END_REF][START_REF] Yannick | Contribution à l'étude des mesures de l'intérêt des règles d'association et à leurs propriétés algorihmiques[END_REF], adding an extra criteria for the choice of the perfect interestingness measure.

Meaningfulness. We note that the meaningfulness of interestingness measures, though not explicitly defined as a property, was nevertheless identified as a possible criteria for choosing an objective interestingness measure.

In [OKO + 04], the authors identified five general factors for categorizing inter- and D (Distance of a rule from the others based on rule attributes). How-ever, this classification was quite loose. Indeed, many of the measures that were classified in the categories N, P or I did not actually represent a specific number of instances, probability or quantity of information but rather used such quantities in their definition. Furthermore, this classification was not put to use for the evaluation of interestingness measures proposed which relied rather on the subjective evaluation by an expert of both interestingness and understandability of a number of given rules.

The two criteria in [START_REF] Lenca | On selecting interestingness measures for association rules: User oriented description and multiple criteria decision aid[END_REF], relative to the easiness to fix a threshold for interestingness and the intelligibility of the interestingness measures, can also be seen as related to the meaningfulness of the interestingness measure. However, both properties related more to the notion of interpretability than the notion of meaningfulness. As we will discuss in the following chapter, the first pertains to the concept of giving meaning whereas the second pertains to the idea of finding meaning. Another paper [START_REF] Blanchard | Semantics-based classification of rule interestingness measures[END_REF], suggested a classification of interestingness measures based on three aspects: subject, scope and nature.

These aspects, defined respectively as "the notion measured by the index ", "the entity concerned by the result of the measure" and "the descriptive or statistical feature of the index ", were explicitly presented as a means to "grasp the meaning of rule interestingness measures". However, the classification was based on a list of nine mathematical properties which were reformulated versions of some of the mathematical properties previously presented in the literature.

The perspective they offered towards understanding the meaning of the interest measures was indeed closer to a "grasp" than to a full understanding of the meaning of the measures.

Measuring the interestingness of interestingness measures. Between the work presented by Gregory Piatetsky-Shapiro in 1991 [START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF] and the research conducted in the following two decades presented in the previous paragraphs, there is a quite notable evolution regarding the approach towards choosing an interestingness measure. On the one hand, Piatetsky-Shapiro defines a number of principles for interestingness measures and defines a single measure from these principles. On the other hand, a number of measures are analyzed, using semi-automated methods, to determine the most interesting interestingness measure for a specific user based on a number of different criteria. Interestingly, the robustness index [LBMLL10a, LBMLL10b, LB11], for example, can naturally be considered as a measure of the interestingness of interestingness measures! In a sense, data analysis is used to define the notion of interestingness. This meta-analysis of interestingness encourages to consider as many interestingness measures as possible. However, determining the best model for measuring rule interestingness within a long list of interestingness measures is not necessarily the most appropriate way to go. Setting aside the fact that a number of the objective interestingness measures considered in the literature have double or more counterparts after normalization [START_REF] Greco | Properties of rule interestingness measures and alternative ap-proaches to normalization of measures[END_REF], such an approach does lead to theoretical modeling issues which will be explored in the following chapter.

Exact summarizations of itemsets

An entirely different approach towards the interestingness issue focuses on exact summarizations (also often referred to as lossless condensed representations) of the mined patterns. Though first introduced in 1996 [MT96], research on exact summarizations emerged mostly after the development of frequent itemset mining algorithms. This approach views the interestingness issue as an issue of information overload which is due to the high redundancy of the patterns extracted rather than them being uninteresting. That is the patterns extracted might be each individually interesting but many carry the same information and they are therefore collectively redundant. To deal with this issue, a straightforward approach is to consider a subset of non-redundant patterns which describes perfectfully the set of all individually interesting patterns.

Such a subset of non-redundant patterns is called an exact summarization.

By contrast with the research on interestingness measures, the research conducted on exact summarization focused primarily on itemsets rather than rules. This is because such approaches rely strongly on the underlying mathematical structure of the search space which is particularly easy to formalize with lattices when dealing with itemsets. As a matter of fact, some of the approaches towards exact summarization, including minimal generators [BTP + 00] or closed itemsets [START_REF] Pasquier | Discovering frequent closed itemsets for association rules[END_REF], find their roots in formal concept analysis, a mathematical theoretical framework in lattice theory [Wil82, GW12, GSW05, GORS16].

Maximal frequent itemsets

Introduced in [AMS + 96], the idea behind maximal frequent itemsets is no more than the Apriori principle (see corollary 2.2.0.1). If the subsets of a frequent itemset are also frequent and one knows that a given itemset is frequent then the information that a subset of this itemset is frequent represents redundant information with regards to the initial knowledge. Hence, the set of all frequent itemsets is entirely defined by the set of all maximal frequent itemsets, were a maximal frequent itemset is a frequent itemset that has no frequent supersets.

Within the itemset lattice, the set of maximal frequent itemsets can be seen as a border which splits the lattice into two areas: frequent itemsets on one side and infrequent itemsets on the other. In a sense, the focus switches from a 2-dimensional area to a 1-dimensional border. Therefore, when using the set of maximal frequent itemsets as a condensed representation for the set of all frequent itemsets, the reduction in terms of number of itemsets can be theoretically significant if the dataset contains long frequent itemsets.

∅ {a 1 } {a 2 } {a 3 } {a 4 } {a 1 , a 2 } {a 1 , a 3 } {a 1 , a 4 } {a 2 , a 3 } {a 2 , a 4 } {a 3 , a 4 } {a 1 , a 2 , a 3 } {a 1 , a 2 , a 4 } {a 1 , a 3 , a 4 } {a 2 , a 3 , a 4 } {a 1 , a 2 , a 3 , a 4 } Figure 2
.4: The border of maximal itemsets within the lattice of itemsets for the toy database in Table 2.1.

The theoretical gain was confirmed in practice with various datasets and researchers suggested different algorithms for specifically mining maximal frequent itemsets [AMS + 96, LK98, AAP00, BCG01, GZ05]. However, although this approach leads to a lossless representation of the set of frequent itemsets, it represents a lossy representation of the set of frequent itemsets together with their frequencies. This represents an issue when mining for association rules as the frequencies of the subsets of a given frequent itemset are needed to determine the confidence of the rules whose scope is defined by that given itemset.

This also represents a modeling issue which is whether itemsets themselves can be interesting or if it is the itemsets together with their frequencies which can be interesting.

Closed itemsets and minimal generators

Closed itemsets. Introduced in [START_REF] Pasquier | Discovering frequent closed itemsets for association rules[END_REF], closed itemsets were a response to the issues described above as they provide a lossless representation of the set of frequent itemsets together with their frequencies. The notion of closed itemsets relies on the scope of an itemset, which is the set of transactions (or tidset) that contains a given itemset. An itemset is said to be closed, if all of the scopes of its supersets are strictly contained in its scope. closure). This is also true when considering only frequent itemsets and closed frequent itemsets. Therefore, the set of closed frequent itemsets together with their frequencies allows for a lossless compression of the set of frequent itemsets together with their frequencies.

As a maximal frequent itemset is necessarily a closed itemset, the set of closed frequent itemsets contains the set of maximal itemsets. Therefore, the set of closed frequent itemsets is at least as big as the set of maximal frequent itemsets (and generally significantly larger). This is quite expected as the summarization of frequent sets using closed itemsets contains more information than the one using maximal itemsets. Although the reduction in terms of itemsets provided by closed itemsets is not as important as with maximal frequent itemsets, it can be significant nevertheless and numerous algorithms for specifically mining closed frequent itemsets have been suggested in the literature [PBTL99, ZH + 99, Zak00a, PHM + 00, WHP03, ZH05].

It is worth noting that the redundancy reduction obtained through closed itemsets for frequent itemset mining also transposes to association rule mining.

Indeed, the confidence of an association rule is equal to the confidence of the rule between the closures of its antecedent and consequent [ZH + 99]. Therefore the set of association rules between closed itemsets, together with their frequencies and confidences, is an exact summarization of the set of association rules, together with their frequencies and confidences.

Minimal generators. Minimal generators [SNK06, VL09, GORS16] (also referred to in previous research as key patterns [BTP + 00] or free itemsets [START_REF] Boulicaut | Free-sets: a condensed representation of boolean data for the approximation of frequency queries[END_REF]) are closely related to closed itemsets. An itemset is a minimal generator if all of the scopes of its subsets strictly contain its scope. While there is only one closed itemset for a given scope, there can be several minimal generators that share the same scope. Therefore, the number of minimal generators is at least as big as the number of closed itemsets and they do not provide a more condensed representation than closed itemsets for the set of frequent itemsets. However, they have been shown to be quite useful for defining exact summarizations of association rules [START_REF] Szathmary | Zart: A multifunctional itemset mining algorithm[END_REF][START_REF] Vo | Fast algorithm for mining minimal generators of frequent closed itemsets and their applications[END_REF]. Furthermore, more elaborate representations based on minimal generators have been shown to provide condensed representations (such as the generalized disjunction-free generators representation [START_REF] Kryszkiewicz | Concise representation of frequent patterns based on generalized disjunction-free generators[END_REF]) which can be more concise than the condensed representation given by frequent closed itemsets in some cases [START_REF] Kryszkiewicz | Why to apply generalized disjunction-free generators representation of frequent patterns?[END_REF].

Non-derivable itemsets

The notion of non-derivable itemsets was introduced by Toon Calders and Bart

Goethals in [START_REF] Calders | Mining all non-derivable frequent itemsets[END_REF]. This approach towards exact summarization relies principally on a generalization of the itemset model together with the centuries-old exclusion-inclusion principle from combinatorial mathematics. The generalized itemset model defined in the context of non-derivable itemsets considers both items x and their negations x. The model defines a generalized itemset X as any subset of I ∪ I (where I = {x 1 , ..., x m } is the set of items and I = {x 1 , ..., x m } is the set of their negations) which does not contain both an item and its negation. The set of generalized itemsets GI can therefore be expressed as such:

GI = X ⊂ I ∪ I ∀i ∈ 1, m , ¬ (x i ∈ X ∧ x i ∈ X)
A transaction in the database is said to contain a generalized itemset if it contains all of the items within the generalized itemset and none of those whose negation is in the generalized itemset.

For k given items corresponding to an itemset I (a regular itemset), there are 2 k generalized itemsets of size k. As the support of each of these generalized itemsets cannot be less than zero, this leads to 2 k inequalities. For each subset X ⊂ I ⊂ I, define δ X (I) as:

δ X (I) = X⊆J I (-1) |I\J|+1 supp(J)
Then, using the inclusion-exclusion principle, one can show that the inequalities described above are equivalent to the set of inequalities defined, for all X ⊂ I ⊂ I, by: This defines an interval for the support of I based on the knowledge of the supports for all of its proper subsets. Furthermore, this is the best possible interval given this knowledge as one can construct possible datasets to fit each of the values within this interval [START_REF] Calders | Deducing bounds on the support of itemsets[END_REF].

supp (I) ≤ δ X (I) if |I \ X|
An itemset (i.e. a regular itemset) whose support can be inferred from the knowledge of the support on its subsets is called a derivable itemset. From what precedes, the set of derivable itemsets is exactly the set of itemsets I such that LB(I) = UB(I). Itemsets for which LB(I) = UB(I) are defined as non-derivable itemsets.

Derivable and non-derivable itemsets have interesting mathematical properties which can be used effectively in data mining processes. This includes a monotonicity property.

Proposition 2.3.1. The supersets of a derivable itemset are also derivable.

Which can be slightly generalized as follows.

Proposition 2.3.2. If I is an itemset such that supp (I) ∈ {LB(I), UB(I)} then all supersets of I are derivable.

Another important property gives a bound on the size of non-derivable itemsets, depending on the size n of the database (i.e. the number of transactions).

Proposition 2.3.3. Let I be an itemset. If |I| > log 2 (n)+1 then I is derivable.

This last property shows that, for a fixed n, the number of non-derivable itemsets is at most polynomial in the number of items m. Furthermore, the set of frequent non-derivable items together with their frequencies gives a lossless condensed representation of the set of frequent itemsets together with their frequencies. This representation has been shown to be theoretically more concise than other representations such as those based on minimal generators [START_REF] Calders | Deducing bounds on the support of itemsets[END_REF][START_REF] Calders | Non-derivable itemset mining[END_REF]. With regards to the frequent closed itemsets representation, it is neither more nor less concise as this can vary depending on the dataset, and empirical studies have shown that they are relatively comparable [START_REF] Calders | Non-derivable itemset mining[END_REF][START_REF] Vreeken | Interesting Patterns[END_REF].

Local and global models for mining informative and significant patterns

One of the key aspects of the interestingness issue in frequent pattern mining is redundancy: the idea that some patterns can carry redundant information, whether individually or collectively. As made explicit in the previous section, this aspect is central in pattern mining approaches which use exact summarizations. However, in the context of exact summarizations, redundancy of information is only addressed exactly. That is, a given piece of information is considered redundant with other elements of information only if it can be inferred entirely and certainly from these elements. This vision is quite restrictive because sometimes such elements of information can tell us mostly (albeit not exactly) what there is to know about this piece of information.

In order to consider how a piece of information can be mostly inferred from other elements of information, various frameworks have been envisaged rooted in either statistics or information theory. These frameworks allow for the definition of either local or global models for considering pattern interestingness and identifying redundancy either in individual patterns or collectively.

Note that the emphasis put on the difference between local and global models for classifying different pattern mining approaches corresponds to our own general understanding of the various approaches in the literature and the issues, in terms of model consistency, related to the use of local models, which we will address in detail in the following chapter. This classification is the central theme of one of our publications [START_REF] Delacroix | Du local au global: un nouveau défi pour l'analyse statistique implicative[END_REF].

Local data models for identifying local redundancy within individual patterns

Given a single pattern, we use the term local data model to refer to a data model based on the observation of its proper components in the data (i.e. based on partial descriptions of the pattern). If the local data model mostly predicts the pattern, then the pattern is locally redundant. As such, many objective interestingness measures rely on (implicit) local models to identify local redundancy.

The fundamental idea behind this approach is to define a redundancy score for each individual pattern, or at least to incorporate this aspect in a more general interestingness measure. A redundancy score can be used to rank patterns and define a set of non-redundant patterns to be presented to the user (either the top-k less redundant patterns or all those whose redundancy score is beyond a given threshold).

Local redundancy in rules. Most objective interestingness measures for rules do not take into account the mathematical structures of the antecedents and consequents of the rules which they consider. They rely solely on a simple rule structure between two attributes (an antecedent X and a consequent Y ) whose parameters are entirely defined by the absolute frequencies in the 2 × 2 contingency table for these attributes (see section 2.3.1.2). Therefore, identifying redundancy within a rule is equivalent to identifying redundancy within the contingency table.

Statistical models. Throughout the past century and more, such contingency tables have been extensively studied in the field of statistics [START_REF] Pearson | mathematical contributions to the theory of evolution.-vii. on the correlation of characters not quantitatively measurable[END_REF][START_REF] Fisher | On the interpretation of χ 2 from contingency tables, and the calculation of p[END_REF][START_REF] Fisher | The conditions under which χ2 measures the discrepancey between observation and hypothesis[END_REF][START_REF] Jaynes | Probability theory: The logic of science[END_REF]. The likelihood of observing such a table in a given context may be modeled by using a number of various probability distributions [START_REF] Lerman | Élaboration et évaluation d'un indice d'implication pour des données binaires[END_REF][START_REF] Lerman | Élaboration et évaluation d'un indice d'implication pour des données binaires[END_REF] and different tests have been designed to measure the surprisingness of having observed one given table .   If there is no background knowledge about the data, the simplest model for the database is that it is a random sample of n independent identically distributed random variables (x, y) with values in {0, 1} 2 . The probability distribution for this random variable can be entirely defined by three values p X = Prob(x = 1), p Y = Prob(y = 1) and p X∧Y = Prob(x = y = 1).

Information on these values is provided by the contingency table of observed frequencies.

If there is nothing to say about the relationship between the antecedent and the consequent (i.e. there is no rule between the two), then the safest assumption to make is that x and y are independent random variables. In this case, p X∧Y = p X p Y . The local independence model for a rule is the model defined by p X = f X , p Y = f Y and the independence of x and y.

Although generally not based on proper statistical tests of independence, many objective interestingness measures are designed to discriminate against rules whose antecedent and consequent appear to be independent and it is widely defended that this is a necessary property for defining an objective interestingness measure (see section 2.3.1.2). In table 2.4, we make explicit the manner in which some objective interestingness measures in the literature correspond to a naive comparison between a given statistic and its predicted value given the independence model.

Measure

Regular formula Comparative formula

Added value max f X|Y -f X , f Y |X -f Y max f X|Y -p X|Y , f Y |X -p Y |X Bayes factor f X|Y f X|¬Y f X|Y f X|¬Y ÷ p X|Y p X|¬Y Centered confidence f Y |X -f Y f Y |X -p Y |X Collective strength f X∧Y +f ¬X∧¬Y f X f Y +f ¬X f ¬Y × f ¬X f Y +f X f ¬Y f ¬X∧Y +f X∧¬Y f X∧Y +f ¬X∧¬Y f ¬X∧Y +f X∧¬Y ÷ p X∧Y +p ¬X∧¬Y p ¬X∧Y +p X∧¬Y Conviction f X f ¬Y f X∧¬Y p X∧¬Y f X∧¬Y Interest |f X∧Y -f X f Y | |f X∧Y -p X∧Y | Lift f X∧Y f X f Y f X∧Y p X∧Y Loevinger 1 -f X f ¬Y f X∧¬Y 1 -p X∧¬Y f X∧¬Y Piatetsky- Shapiro1 f X∧Y -f X f Y f X∧Y -p X∧Y Piatetsky- Shapiro2 n (f X∧Y -f X f Y ) n (f X∧Y -p X∧Y )
Relative risk • S is the random variable associated to the statistic, ∼ indicates its distribution and its convergence in distribution;

f Y |X f Y |¬X f Y |X f Y |¬X ÷ p Y |X p Y |¬X Relative specificity f ¬X|¬Y -f ¬X f ¬X|¬Y -p ¬X|¬Y Varying rates liaison 1 -f X∧Y f X f Y 1 -f X∧Y p X∧Y
• B(n, p) for the binomial distribution with parameters n and p;

• P(λ) for the Poisson distribution with parameter λ;

• H(n, n 1 , n 2 ) for the hypergeometric distribution with parameters n, n 1 and n 2 ;

• N (µ, σ 2 ) for the normal distribution with parameters µ and σ 2 ;

• χ 2 (d) for the chi-square distribution with d degrees of freedom;

• T (d) for Student's t-distribution with d degrees of freedom.

Measure

Regular formula Probabilistic behavior 5 Model Support

n X∧Y S ∼ B(n, f X f Y ) Standard S ∼ P(nf X f Y ) Poisson S ∼ H(n, n X , n Y ) Fixed column margins Correlation coefficient f X∧Y -f X f Y √ f X f Y f ¬X f ¬Y S n-2 1-S 2 T (n -2) Standard Normalized support √ n f X∧Y -f X f Y √ f X f Y (1-f X f Y ) S N (0, 1) Standard √ n f X∧Y -f X f Y √ f X f Y Poisson √ n -1 f X∧Y -f X f Y √ f X f Y f ¬X f ¬Y Fixed column margins χ 2 statistic n f 2 X∧Y p X∧Y + f 2 ¬X∧Y p ¬X∧Y + f 2 X∧¬Y p X∧¬Y + f 2 ¬X∧¬Y p ¬X∧¬Y -1 S χ 2 (1) Standard S χ 2 (2)

Poisson

Fixed column margins

Table 2.5: Statistics and their behavior.

Information theory models. Another framework which allows to identify local redundancy in rules is information theory. Information theory provides a complete modeling for information [Sha48, Kul59, Mac03, CT12, Jay03, Bor11] which comprises:

• a model for the amount of information gained from the observation of a random event: the information content of the event;

• a model for the average amount of information required to describe a single sample of a random variable: the information entropy of a probability mass function ;

• a model for the amount of information lost when a probability mass function p is used to approximate a probability mass function f : the Information gain

D KL (f p) = f X∧Y log f X∧Y f X f Y + f X∧¬Y log f X∧¬Y f X f ¬Y + f ¬X∧Y log f ¬X∧Y f ¬X f Y + f X∧¬Y log f ¬X∧¬Y f ¬X f ¬Y If the
log f X∧Y f X f Y I p (X ∧ Y ) - I f (X ∧ Y ) J-Measure f X∧Y log f X∧Y f X f Y + f X∧¬Y log f X∧¬Y f X f ¬Y D KL (f p, X)
Normalized mutual information -

f X∧Y log f X∧Y f X f Y +f X∧¬Y log f X∧¬Y f X f ¬Y + f ¬X∧Y log f ¬X∧Y f ¬X f Y +f X∧¬Y log f ¬X∧¬Y f ¬X f ¬Y f X log f X +f ¬X log ¬X DKL(f p) H((f X ,f ¬X )) One way support f Y |X log f Y |X f Y D KL (f p, Y |X) Two way support f X∧Y log f X∧Y f X f Y D KL (f p, X ∧ Y )
Two way support variation

f X∧Y log f X∧Y f X f Y + f X∧¬Y log f X∧¬Y f X f ¬Y + f ¬X∧Y log f ¬X∧Y f ¬X f Y + f X∧¬Y log f ¬X∧¬Y f ¬X f ¬Y D KL (f p)
Table 2.6: Objective interestingness measures based on information theory that discriminate against locally redundant rules given the independence model.

structures, the improvement measure [START_REF] Bayardo | Efficiently mining long patterns from databases[END_REF]:

imp(X → Y ) = min X X (conf(X → Y ) -conf(X → Y ))
This measure does discriminate against locally redundant rules as it compares the confidence of the rule to the confidence of any proper subrule with the same consequent. In terms of local data models, this can be seen as the difference between the observed frequency f Y |X and its predicted value

p Y |X = max X ⊆X p Y |X .
However, this measure was constructed as an intuitive indicator for local redundancy and any justification for why a local data model would predict such a value would be quite ad hoc.

Local redundancy in itemsets. In the case of itemsets, the proper components of the pattern which usually define local data models are either its items or, more generally, its subsets (both together with their frequencies).

As in the case of rules, the statistical data models described in this section comply with the following classical structure for data models. Consider an itemset X = {x 1 , ..., x k }. The database is modeled as a random sample of n independent identically distributed random variables X = (x 1 , ..., x k ) with values in {0, 1} k . The probability distribution for this random variable can be entirely described by the 2 k -1 values p X for all ∅ X ⊂ X.

To simplify notations in this section, we will use f i and p i rather than f {x i } and p {x i } for the frequencies and probabilities associated to a single item x i .

The independence model The independence model is the simplest local statistical model for an itemset. It is based on the frequencies of the items that compose the item and the mutual independence of their associated random variables:

∀i ∈ 1, k , p i = f i and x i are all mutually independent

In this model, the value of the probability of any subset X ⊂ X is equal to the product of the frequencies of the items that compose X :

p X = x i ∈X f i
Different tests and measures for identifying redundancy using the independence model have been suggested in the literature. These can rely on a specific statistic. The most common statistic used is the frequency of the itemset f X which is compared to p X = 1≤i≤k f i . The lift measure for itemsets ([VT14])

compares the two through a simple ratio:

lift(X) = f X p X
and evaluation of p-values allow for stronger statistical testing ([VT14]).

Another statistic has been compared to its expected value given the independence model in the literature through the collective strength measure ( [START_REF] Charu | A new framework for itemset generation[END_REF]). This measure considers the ratio between the number of agreements to X (transactions that contain X or contain none of the items of X) and the number of violations to X (the remaining transactions) and divides by its expected value given the independence model:

cs(X) = 1-fv fv 1-pv pv where f v = f X∧ 1≤i≤k {x i } and p v = 1 - 1≤i≤k p i + 1≤i≤k (1 -p i )
However, testing a single statistic against its value given by the indepen- between models. This does not, however, prevent other issues with the use of multiple local models as we will discuss in the next chapter.

dence
MaxEnt models. Maximum entropy models (or MaxEnt models) are models which maximize the information entropy of the probability mass function of the underlying statistical model given a set of constraints. As we will discuss further in this thesis, the rationale for using MaxEnt models can be founded on three different approaches towards the meaning of entropy: Shannon's approach [Sha48], Jayne's approach [START_REF] Jaynes | On the rationale of maximum-entropy methods[END_REF][START_REF] Jaynes | Probability theory: The logic of science[END_REF], and the constrained independence approach which we have developed. In any case, MaxEnt models may be seen as a generalization of independence models. They are the least binding models, in terms of model hypotheses, given the set of constraints on which they are defined. As this thesis proposes a detailed study of such models, we will limit this section to a brief description of the classical representation of MaxEnt models in itemset mining.

Let X = {x 1 , ..., x k } be the itemset for which a local MaxEnt model is defined. Consider Ω = {0, 1} k the set of all possible transactions (which can also be seen as the set of all generalized itemsets of size k). These represent the 2 k different possible values which can be taken by the random variable X.

Hence, we can define the information entropy for the associated probability mass function as:

H = - ω∈Ω p ω ln(p ω )
where p ω is the probability that X = ω.

Now consider a set C of non empty subsets of X (or more generally a set of generalized itemsets based on X). We can define a MaxEnt model by considering the probability function that maximizes H while satisfying the constraints that p X = f X for all X ∈ C.

If C = ∅, then we obtain the model for k independent random coin tosses.

If C = {{x 1 }, ..., {x k }}, then we obtain the independence model. If C = P(X)\ {∅} (where P(X) represents the partition set of X), then we obtain the local empirical data model.

In addition to these simple cases, a few other specific cases are known to be solved using specific procedures.

The oldest example is the Chow-Liu tree model which describes the MaxEnt model were the constraints are given by the set of all itemsets of size both one and two [START_REF] Chow | Approximating discrete probability distributions with dependence trees[END_REF]. A generalization of this method, known as k-width junction cherry trees, has been suggested to define models based on all the itemsets of size k or less [KS10, SK12] 7 . However, to our knowledge, the models provided have only been proven to maximize entropy among a certain class of probabilities defined by k-width junction trees [START_REF] Szántai | Hypergraphs as a mean of discovering the dependence structure of a discrete multivariate probability distribution[END_REF]. Chow-Liu tree models, as well as k-width junction cherry trees have both been used in the context of itemset mining.

Our own constrained independence model describes, in its first version, a specific fast computable method for defining the MaxEnt model when the set of constraints contains all proper subsets of the itemset [START_REF] Delacroix | Constrained independence for detecting interesting patterns[END_REF].

Our further work on mutual constrained independence model, presented in 7 The itemset framework is, however, not mentioned in these articles. This contrasts with the vision that these methods aim at gaining knowledge, in other words learning, about the underlying mechanisms that generate the data. Learning, be it human or machine learning, is usually described as a never-ending incremental process8 . We present here the main approaches in the literature towards implementing the incremental aspect of learning. The endless aspect of learning implies learning knowledge from infinite or dynamic databases and involves an entirely different branch of research which is beyond the scope of this thesis.

Consider the process of determining the redundancy of a pattern with regards to background knowledge as described in the previous section. If a pattern (for example an itemset together with its frequency) is non-redundant with that background knowledge, then it can be added to that background knowledge and the process may be repeated. Hence, the background knowledge of any given step represents the entire knowledge, previously known and acquired up to that step during the learning process9 .

Different variations to this principle have been suggested in the itemset literature.

In [START_REF] Wang | Summarizing itemset patterns using probabilistic models[END_REF][START_REF] Wang | Summarizing itemset patterns using probabilistic models[END_REF], Markov random fields (MRF) are used to iteratively construct global models for the data. The algorithm proposed is a level-bylevel approach. First, the frequencies of all itemsets of size 1 are used to build the MRF model for predicting the frequencies of all itemsets of size 2.

These predictions are compared to the corresponding empirical frequencies. If the prediction is too far away from the empirical value for a given itemset, the empirical frequency of that itemset is added as a constraint to the MRF model. Once all itemsets of size 2 have been scanned, the additional knowledge is used to recalculate the MRF model using a junction tree algorithm or an approximation through an MCMC method. The process then goes on until a given itemset size is reached. Note that the model given by the junction tree algorithm, let alone its approximation obtained via an MCMC method, is not equal to the MaxEnt model based on the same constraints. Indeed as stated previously, junction tree algorithms are only known to converge towards the MaxEnt model in a specific class of probabilities.

In an article entitled "Tell me something I don't know " [HOV + 09], the authors use randomization methods applied to an iterative learning process.

Using the p-value given by the randomization model for all itemsets, the algorithm suggested finds the most surprising itemset at each step of the iteration and adds it to the randomization model. Given the exponential number of itemsets, the method proposed only considers itemsets of size 2 or 3. Furthermore, the complexity of the randomization task being to great when adding exact constraints on the frequencies of itemsets in the randomized data, the constraints are softened (the frequencies of the constrained itemsets in a randomized dataset can be different from the empirical frequencies in the original dataset but the probability of obtaining a randomized dataset exponentially decreases with its distance from the exact constraints). It is also important to note that the fixed rows and column margins constraint is used as an additional constraint in all the randomization processes described in [HOV + 09].

In [START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF] and [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF], the practical approaches are similar in the sense that the mining algorithms focus, at each step, on finding the itemset whose frequency diverges the most from the current data model (a randomization model in [START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF] and a MaxEnt model in [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF]). However, the theoretical approach is different in the sense that the method is presented as a greedy heuristic in order to determine the most interesting set of patterns. We address this perspective in the next section.

There are a number of other possible variations to the general iterative learning process and we will present a few of these in chapter 5, some based on previous suggestions we made in [START_REF] Delacroix | Du local au global: un nouveau défi pour l'analyse statistique implicative[END_REF].

Global models defined by interesting and non-redundant sets of patterns

In the approaches which we have described previously, redundancy and, more generally, interestingness are always regarded as properties of a single pattern with respect to a certain amount of knowledge: the interestingness of a pattern with respect to its components, the interestingness of a pattern with respect to predefined background knowledge, or the interestingness of a pattern with respect to acquired background knowledge based on other patterns.

Another approach regards interestingness as a property of sets of patterns.

In fact, the focus is not so much aimed towards patterns but towards the models they define. An interesting set of patterns is one that defines an interesting model.

Model evaluation. As the focus is set on models, the issue of evaluating a model becomes central with this approach. For defining a good model, most of the literature focuses on two aspects of the model: its ability to predict the data, on the one hand, and its simplicity, on the second hand. The first aspect is quite easy to define. Measures using likelihood or distances characterize this reasonably well. However, the best model to predict the data is the empirical data model itself, so relying simply on such an aspect would defeat the whole point of pattern mining.

One of the simplest approaches towards this issue is to fix the number of patterns which define the model [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF][START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF]. In this approach, the model with the best prediction defined by k patterns is the best model. However, the value of k is difficult to determine and this results generally in a quite ad hoc choice.

This is not an issue if the measure for ranking models decreases with regards to the complexity of the models (for a given precision in prediction). This is the case while considering statistical testing, such as To transpose to pattern mining, a pattern language and a model class must be chosen. The pattern language defines which patterns may be mined (for example, in the case of itemsets, the pattern language is the set of all itemsets). The model class defines how the language transposes to a data model (for example, MaxEnt models in [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF] or code tables in [VVLS11, SK11, SV12, TV12]). The MDL is computed as:

L(D, M ) = L(M ) + L(D|M )
where L(M ) is the length of the description of the model in the pattern language and L(D|M ) is the length of the description of the deviation of the data with regards to the model. The model which minimizes L(D, M ) is considered the best model. In other words, the strongest lossless compression of the data is considered to be the best explanation of the data. The set of patterns which define this compression is therefore considered to be the most interesting set of patterns.

Although this approach is one of the most solidly theoretically founded in pattern mining, it does come with certain limitations. We address some of these limitations in the following chapter and suggest an alternative general theoretical framework for pattern mining.

A necessary resort to heuristics. When considering the issue of maximizing interestingness with regards to sets of patterns rather than simply patterns (regardless of the measure for model interestingness chosen among the ones previously cited), the search space for an optimal solution is doubly exponential rather than simply exponential. In the case of itemsets, for m items, there are 2 m itemsets and 2 2 m sets of itemsets. Considering only 8 items, the order of magnitude is close to the estimated number of atoms in the Universe and for any value above 5, an exhaustive search within this space is technically infeasible. There are no known results that allow to reduce the search space significantly and sufficiently. Some results show that this may not be the case altogether as the search for an optimal solution has been shown to be NP-hard in some cases [START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF].

As such, the search of the optimal solution must resort to various heuristics and is limited to the search of locally optimal solutions. Greedy algorithms have been utilized to this effect for models based on randomization methods [START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF] and MaxEnt methods [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF]. We suggest a few possible improvements to such greedy algorithms in chapter 5.

Note that such approaches only bring down the complexity of the method from a double to a single exponential which remains insufficient in many cases.

In order to bring this down further still, the problem may be simplified by partitioning the set of items and searching for an optimal solution within the scope of each block of items. This is similar to the approach in [PNS + 07], in which clustering is used to partition the items before mining for association rules. However, the models of interestingness considered for mining association rules within each cluster in [PNS + 07] are local even within the scope of cluster.

Such approaches are also discussed in chapter 5.

Conclusion

In this chapter, we have presented the history and state-of-the-art of frequent rule and itemset mining while focusing on the approaches that mine for objectively interesting patterns. Of course, there are many other topics covered in the scientific literature on rule and itemset mining and we could not address these all. Other notable areas of research include, for example, the issue of huge data [RU11, AISK14, GLCZ17 In the following chapters, we will present our contributions to this field. In chapter 3, we analyze various theoretical issues in current approaches and the theoretical boundaries of the field. From this analysis, we suggest a number of recommendations for defining a general theoretical framework for pattern mining. In chapter 4, we present our novel approach towards MaxEnt models through mutual constrained independence (MCI). We show that MCI offers both further insight on the rationale behind MaxEnt models and novel techniques for computing such models. Theses techniques, based on tools from algebraic geometry, allow us to provide for direct exact solutions to a class of MaxEnt models which had not been previously solved. In chapter 5, we present algorithms for extracting objectively interesting patterns from data based on the principles which are defined in chapter 3 and the tools which are presented in chapter 4. We discuss the issues that arise from a direct implementation of these principles and give guidelines for future research that could allow to tackle these issues, while suggesting current solutions based on compromise.

CHAPTER 3

Meaningful mathematical modeling of the objective interestingness of patterns

Introduction

In the previous chapter, we have presented the history, development and stateof-the-art of frequency-based 1 itemset and rule mining, with a specific focus on the issue of extracting objectively interesting patterns. Each and every approach which we have described relies on mathematical models. The mathematical definitions of these models are usually quite explicitly laid out in the literature. However, the modeling processes themselves are not always made apparent. In fact, more often than not, they are brushed aside as if they were irrelevant or simply and totally ignored. As we have discussed in our foreword, this tendency to be oblivious to the mathematical modeling process can be seen as a cultural trend which goes far beyond the specific field of frequency-based pattern mining.

Mathematical modeling is often described as the act of establishing a correspondence between a system (usually corresponding to some aspect of the real world) and a mathematical model. When applied to a system, every mathematical model relies, explicitly or implicitly, on a mathematical modeling.

The mathematical modeling is what provides meaning to the mathematical model. It is the link between mathematics and reality. Without this link, a

1 Recall that we use the less ambiguous frequency-based itemset mining terminology or, more generally, frequency-based pattern mining rather than the more common frequent pattern mining terminology which is used ambiguously to refer both to the process of mining patterns that are frequent in the data or to the process of mining patterns based on their frequency in the data. mathematical model is just an abstract construction and the results that we can obtain through the model are meaningless with regards to reality. Therefore, questioning the meaningfulness of the answers provided by a method based on mathematical models implies a necessary inspection of the corresponding underlying mathematical modeling processes. If the modeling processes are implicit, they need to be made explicit in order to do so. As one of the main aims in this doctoral thesis is to provide both efficient and meaningful methods for extracting objectively interesting patterns, we undertook the task of identifying the various underlying modeling processes inherent to the different approaches in the literature. This led us to analyzing different mechanisms involved in these modeling processes and their impact, in terms of meaningfulness, on the general modeling process. We came to understand that there are certain number of recurrent issues within the research in the literature directly related to the modeling processes described or the absence of explicit descriptions of these modeling processes. Hence, we suggest a number of recommendations for a meaningful mathematical modeling of objective interestingness of patterns based on our analysis.

Modeling and mathematical modeling

Modeling as translation

Before we address any specific issue related to modeling processes, we must start by clearly defining the notion of mathematical modeling and, more generally, of modeling. The issue of defining the notions of models and modeling has been extensively debated by researchers studying the philosophy of science and definitions vary from one author to the other [Apo61, Sup61, Hes65, Min65, Sup69, Sta73, McM85, Gie88, RWLN89, BJ99, C + 99, Gie99, BJ02, GS06, TJ06, Sup07, Wei07, G + 09, BJ09, Con10, Cha13, Pot17]. We build upon some of these definitions to suggest a new description of models and modeling. The definition we propose here follows in the tradition of philosophers who have adopted a rather broad view towards the definitions of models and modeling. These include the definitions given by Leo Apostel in [Apo61]:

This will be our final and most general hint towards the definition of model: any subject using a system A that is neither directly nor indirectly interacting with a system B to obtain information about the system B, is using A as a model for B.

Marvin Minsky, whose definition of a model was given in [Min65]:

To an observer B, an object A * is a model of an object A to the extent that B can use A * to answer questions that interest him about A.

as well as the definition by Jeff Rothenberg in [RWLN89]:

Modeling in its broadest sense is the cost-effective use of something in place of something else for some cognitive purpose. We defend the idea that modeling is a translation. It is the act of establishing a correspondence between two languages each of which allow a representation of the world. The model corresponds to the representation of the world in the modeling language. The aim is to use the second language in order to answer questions about the world which cannot be satisfactorily answered in the first language as illustrated by Figure 3.1. Therefore the second language must allow for inference. It should also generally be more structured than the initial language, allowing for more acceptable forms of inference, in order to satisfy the aim of the modeling process. Our approach of modeling and models intentionally differs from that of a number of philosophers who see modeling exclusively as a representation of the real world (such as Giere [G + 09], Magnani [START_REF] Magnani | Scientific models are not fictions[END_REF] or Portides [Por14]) or based on formal models (such as Suppes [Sup61, Sup64, Sup68, Sup69], van Fraassen [VF67, [START_REF] Bas | On the extension of beth's semantics of physical theories[END_REF][START_REF] Bas | Scientific representation: Paradoxes of perspective[END_REF] or da Costa and French [START_REF] Newton | The modeltheoretic approach in the philosophy of science[END_REF][START_REF] Newton | Science and partial truth: A unitary approach to models and scientific reasoning[END_REF]). Firstly, we support the views of Stachowiak and Podnieks that the world is always accessed through models [Sta73,[START_REF] Podnieks | Philosophy of modeling: Neglected pages of history[END_REF]. Hence our starting point is not the world per se, but some representation of the world in an initial language (even though we might refer to this representation as the world for further simplicity in the other sections of this chapter). Secondly, our approach allows to consider mathematical modelings as well as a wide variety of other modelings, as long as we accept a broad definition of the notion of language. For example, we can easily include computer models if we consider programming languages as possible modeling languages3 . Indeed, in such a case, we can model our initial representation through a computer program implemented in a given programming language together with a set of input variables. A question about the world can then be represented by a subroutine of the program and the answer to this question by the output of this subroutine. The inference step of the modeling process corresponds here to the execution of the subroutine on a computer. Note here that the ability to provide for answers in the modeling language depends on the systems that perform the inference step of the modeling process: computers in the previous example; mathematicians in the case of mathematical models; etc.

We must add that the use of the term translation can also be seen as a relevant choice for describing the idea, emphasized as an important aspect of models by authors such as Cartwright [CM84, C + 99], Strevens [START_REF] Strevens | Depth: An account of scientific explanation[END_REF], Wimsatt [START_REF] William | False models as means to truer theories[END_REF] or Potochnik [START_REF] Potochnik | Idealization and the Aims of Science[END_REF], that models are necessarily idealized and therefore false representations of the reality they describe (see also [START_REF] Weisberg | Three kinds of idealization[END_REF] on idealizations). Similarly, a translation can never perfectly transpose every aspect of an original text, a characteristic often referred to by the Italian expression traduttore, traditore meaning translator, traitor. As translations betray original texts, models lie about the world. The analogy may even be pushed slightly further as, in both contexts, authors argue that a good translation or a good model is necessarily unfaithful or a lie (see [Mou76] for translations and [CM84, Wim87, Str08] for models).

Mathematical modeling

Through this approach, mathematical modeling is simply modeling into mathematical language. A mathematical modeling process establishes a correspondence between the representation of the world in the initial language (usually a technical form of a natural language) and a mathematical model. It also establishes a correspondence between questions about the world in the initial language and mathematical problems. Most importantly, it allows to establish a correspondence between mathematical solutions to the mathematical problems and answers to the questions in the initial language.

The significant advantage of using a mathematical modeling compared to other types of modeling is that the mathematical language is exact and infallible. Indeed, once a mathematical problem, or a class of mathematical problems, has been solved mathematically (i.e. there is a mathematically correct proof of the solution to the problem) it has been solved definitely. Moreover, the solution is independent from the proof: even if the proof may be extremely tedious and complex, the formulation of the problem and its solution may be quite simple, allowing for it be used easily, over and over again, as many times as needed. Hence, provided that we can trust mathematicians to deliver correct mathematical solutions to mathematical problems, we can entirely trust the problem solving step in the modeling process. Hence, the only debatable step in a mathematical modeling process is the establishment of the mathematical modeling correspondence. This allows to circumscribe issues related to cal modeling correspondence is meaningful. In particular, if an answer given by a mathematical modeling is to be contested, then the only thing there is to contest is the adequacy of the mathematical modeling with regards to the simplifications, approximations and representation choices which were made.

Another argument in favor of the use of mathematical models is based on the idea that mathematics is the fundamental language of the Universe. This theory, defended by many mathematicians and scientists every since Galileo Galilei expressed it in Il Saggiatore [START_REF] Galilei | English: The Assayer, Translation from the Italian by Stillman Drake in The Controversy on the Comets of 1618[END_REF], implies that there is a mathematical modeling which exactly represents the world as it is. Hence, the only way to truly access the essence of reality is through mathematical models. In contrast with the idea that models lie about reality, this theory suggests that the truth about reality can only be described by a mathematical model. Note that this is not incompatible with the idea that models, as translations, move away from the initial representation of the world which is modeled. Indeed, as we have stated, this initial representation is already at a distance from the world per se. Hence moving away from this representation may in some cases lead us towards reality rather than away from reality.

Complex modeling processes

Although the description of modeling and models which we have given in this section allows for a broad and elegant approach towards these notions, we understand that it falls short when it comes to analyzing the complex structure of actual modeling processes if it is not refined. Indeed, more often than not, multiple modelings of various aspects of the world are used, expressed in multiple modeling languages, and combined with modelings of modelings, thus creating a complex structure for the general modeling process. This does not imply that Figure 3.1 is not valid for such cases: it stays valid if we consider that the modeling language is a complex aggregation of all the modeling languages used within the general process. However, it does not inform much about such complex modelings and we need to be able to describe these complex processes.

Though are main focus in the rest of this chapter is mathematical modeling, we also describe a type of such complex modeling approaches which we name patchwork modelings in section 3.5.

Furthermore, in the pattern mining modeling processes that we address in this thesis, at least two modeling languages are used within the general modeling processes: the mathematical language; and a programming language. In some cases the modeling in the programming language will describe most of the mathematical objects from the mathematical model (this is usually the case for randomization methods), while in other cases the modeling in the programming language will simply be used to compute the mathematical solutions defined through the mathematical modeling but without any regards towards the other aspects of this mathematical modeling. However, in all of the cases we address, the computer modelings follow in sequence after the mathematical modelings in the general modeling processes: they model aspects of the mathematical modeling. Hence, the meaningfulness of the general modeling process lies foremost in the mathematical modeling. Of course, the computer modeling plays a distinctive role in the definition of the general modeling process and even of the mathematical modeling. In particular, the ability to infer an answer (in this case to reach an output) in the computer modeling sets the restrictions, in terms of complexity, for the usefulness of a mathematical modeling. However, the meaningfulness of the computer modeling is, in the cases we address, not a significant issue.

Chapter outline

Following the definition of the notion of mathematical modeling in this section, we address a number of aspects related to mathematical modeling in objective frequency-based interesting pattern mining. We define novel terminology for describing general features of mathematical modeling processes which we consider to be quite relevant in this context but also in most other contexts in which mathematical models are applied. The necessity for defining this new terminology stems from the fact that we have not found any preexisting terminology allowing to precisely describe the issues which we pinpoint in any of the literature which we have knowledge of, whether in the fields of computer science, mathematics, or the philosophy of science. As such, this chapter can be viewed as much as a contribution to the philosophy of science and mathematics as a contribution to the field of pattern mining.

In section 3.2, we compare modeling processes in which the data is the main subject of the modeling process to those in which it is only a deriving object and the main subject is the mechanism that generated such data. In section 3.3, we compare two general approaches towards the definition of a mathematical model: phenotypic and genotypic modeling. We show how different modelings in pattern mining relate to these approaches and discuss some issues of phenotypic modeling. In section 3.4, we address the issue of pragmatic modeling (i.e. modeling which is based foremost on pragmatic considerations). In section 3.5, we introduce the notions of patchwork and holistic mathematical modelings and address specific issues related to patchwork modeling in pattern mining. In section 3.6, we consider the modeling of patterns within the general modeling process and particularly the use of a mathematical model based on measure spaces and Boolean lattices. In section 3.7, we focus on the modeling of objectivity within the general modeling process. Finally, we conclude in section 3.8 on the definition of a mathematical modeling satisfying all the various recommendations previously defined in this chapter.

Note that the length of each of these sections may vary significantly. How-ever, this only indicates how much detail we felt was necessary to present each of these aspects and does not reflect in any way the importance of the related recommendations.

The data: subject or object of the modeling process

One important aspect that allows to categorize most of the different modeling approaches in the literature is related to the position held by the modeling of the data within the general modeling process. We discern two main categories. On the one hand, we consider the modeling processes in which the data is regarded as a main subject of the modeling process, and on the other hand, those in which the data is regarded as an object deriving from a main subject of the modeling process. By subject, we mean that the data is modeled using a mathematical model which does not conceptually derive from any another mathematical model within the general modeling process. Conversely, we use the term object to signify that the data is modeled as the result of a mechanism which is itself described in the modeling process through a mathematical model. In the first case, the main subject of the modeling process is the data and, in the second case, it is the mechanism that generates the data.

Classical statistical approaches fall into the second category. Consider, for example, the modeling process described in section 2.3.3.1 in which the database is modeled as a random sample of n independent identically distributed random variables X = (x 1 , ..., x k ) with values in {0, 1} k . In this case, the subject of the modeling process is the random process which generated the data. The data is only seen as a means to gain information regarding that random process. This is essential because the random process corresponds to a lower level of modeling than the database which has a much larger scope:

the modeling process considers any other transaction on the same items to be a result of that single random process. This in turn provides a basis for justifying the generalization of the results obtained on one database to other databases.

Conversely, any modeling process that considers the database as a subject per se does not provide any basis for utilizing the results obtained on one database to other databases. Note that this is not necessarily an issue. If the aim of a particular data mining method is to provide the best compression for one very large video file or the best summary for one given book, it is not a necessary condition to consider the mechanisms that generate video files or books in the mathematical modeling for the data mining method. In both examples, there is no need to generalize the results.

In many other applications of pattern mining, generalizing is nevertheless necessary. Take classification, for example. In order to be able to justify why a classification method learned on a given database can be used to classify elements from another database, it is necessary that the modeling process for this classification method include descriptions not only for the initial database, but also for other potential databases, and for the relationship between them that justifies that knowledge about this one database is knowledge about all databases. Modeling the underlying mechanisms that generate the data through a random distribution is one way to go but there could be many other possible manners to accomplish this.

Note that, for a statistician, what we describe here may seem quite obvious.

But recall from our foreword that the field of our study is at the junction of different scientific cultures and that what may be perceived as general knowledge in one culture may not be perceived as such in the other. The examples in the literature show that this is indeed not the case.

We present here two general cases of ambiguous mathematical modeling with regards to the position held by the modeling of the data within the general modeling process. In both cases, this leads to theoretical issues.

The data modeling process in the case of objective interestingness measures for rules

Consider the case of objective interestingness measures for rules as presented in section 2.3.1.2. In the vast majority of the literature dealing with such measures, there is no explicit reference to a random variable that generates the data. If this were the case, there would be a distinction between the probabilities p X , p Y and p X∧Y which define the probability distribution for the random variable defined on contingency tables and the frequencies f X , f Y and f X∧Y which correspond to the distribution in the empirical data. As we have stated previously, we have found only a few cases in the literature pertaining specifically to objective interestingness measures of rules were this distinction is made explicit, and none of them correspond to the most cited papers (see [START_REF] Greco | Properties of rule interestingness measures and alternative ap-proaches to normalization of measures[END_REF], for example). In most cases, it is the probabilistic notation, however, which is preferred rather than one using frequencies. There are three possible explanations for this.

The first possibility is that this corresponds to the case were the data is modeled as a subject and the probabilities that are described are simply referring to the probability distribution of the empirical data. In this case, as explained in the beginning of this section, there is no theoretical ground for generalizing the results from this dataset to any other dataset.

The second possibility is the case in which the data is modeled implicitly as an object deriving from a distinguishable subject. Even if this implies that a same notation is used alternatively to describe the probability distribution for a random variable and the frequencies in the empirical distribution, this would make sense for a certain number of interestingness measures as we have described in section 2.3.3.1. This is not an issue if a single rule is considered.

However, when considering multiple rules on itemsets, this could generate a conflict within the general modeling process, in the sense that a single aspect of reality can be modeled through two different and incompatible mathematical models. We will address such issues in detail in section 3.5.

A third option is that the data is modeled as an object deriving from an indistinguishable subject. In this case, the data is considered to be generated by a random variable. However, based on a frequentist approach towards probability, the distribution of this random variable is exactly equated with the empirical distribution in the data. In this sense, summarizing the dataset is equivalent to summarizing the mechanisms that generate such datasets.

Regretfully, this is mostly an artifice for presenting the first case with a means to justify that the results of the mining process may be generalized from one dataset to other datasets. Indeed, while there is sufficient ground for justifying that the distribution of the random variable be equated to the distribution of the empirical in the modeling process when considering a few items as long as the number of transactions n is large enough, there is no way that the number of transactions considered will ever be large enough to justify such an approximation when considering a hundred items, let alone a hundred thousand items (see section 3.7 for more details).

In all these cases, the modeling process raises issues. Moreover, the fact that the modeling process for the data is not made explicit makes it complicated to address these issues.

3.2.2

The data modeling process when considering the fixed row and column margins constraint

The second example we present here is related to the fixed row and column margin model that is one of the few global models which is considered in itemset mining. In the itemset mining literature, this model is especially present in research papers focusing on swap-randomization methods When modeling data using a fixed row and column margins model, the idea is that these margins correspond to defining characteristics in the system that is described by the data. Hence, the data is modeled as a single random sample

of a random variable D =     d 1,1 • • • d 1,m . . . . . . . . . d n,1 • • • d n,m   
 following a uniform distribution on all n × m matrices satisfying the fixed row and column margins constraint.

This makes sense for a certain number of applications in ecology, such as the case of the classical study of the distribution of bird species among islands in an archipelago [START_REF] Sanderson | Selfridge. Null matrices and the analysis of species co-occurrences[END_REF][START_REF] Chen | Sequential monte carlo methods for statistical analysis of tables[END_REF]. In this case, the datasets indicate the presence of a given species of bird (among m species) on a given island (among n islands).

As specialists defend that the natural characteristics of each island only allow for a given number of species to cohabit while the natural characteristics of each species define their widespreadness, the model can be justified. Any divergence from the model indicates information about the specific distribution which is studied (i.e. how these species of birds are distributed within this archipelago)

and the modeling process does not provide for a basis to justify the generalizing of the results obtained to any other system (at best a specialist with further knowledge about such systems might rely on these results to formulate general hypotheses).

This could also make sense in the case of evaluation datasets that indicate which questions (among m questions in an examination) were correctly answered by a given examinee (among n examinees). Indeed, both the general levels of the examinees (represented by the number of correct answers given by each examinee) and the general difficulty of the questions (represented by the number of correct answers at a given question) may be seen as defining characteristics of the system. This explains the popularity of fixed row and column margins model in psychometrics, whether based on swap-randomization Similarly, in the classical itemset example of market basket analysis, it makes sense to consider that the size of a consumer's basket (i.e. the number of items in a transaction) is a defining characteristic of that consumer's purchasing habits. However, one of the main goals of market basket analysis is to gain knowledge about general consumer habits rather than those simply related to the pool of consumers within the dataset.

Let us examine more closely what it means in terms of data modeling when these fixed row and column margin models are combined with itemset mining models as in [START_REF] Gionis | Assessing data mining results via swap randomization[END_REF] or in [START_REF] Lijffijt | A statistical significance testing approach to mining the most informative set of patterns[END_REF]. If the aim is to generalize the results obtained to other transactions there are necessarily theoretical issues with the general modeling process. Indeed, either we consider that the frequencies of an itemset I can inform us about the distribution of a random variable X = (x 1 , ..., x m ) which generated the data. In this case, we have two competing and incompatible models for the dataset: as a random sample of n independent identically distributed random variables X, on the one hand, and as single sample of the random variable D, on the other hand. Alternatively, if we only consider the data modeling process associated with the fixed row and column margins model, then there is little basis for justifying that the observed value of 1≤i≤n j∈I d i,j (corresponding to the frequency of itemset I) for a single sample gives any substantial information about the distribution defining the random variable D. Once more, note that if the aim is simply data compression and not generalizing there are no issues with this modeling process.

In contrast, the model suggested in [START_REF] Tatti | Using background knowledge to rank itemsets[END_REF] does not suffer from these theoretical issues. Indeed, the authors of this paper suggest that rather than considering the fixed row and column margins constraint similar constraints could be defined with regards to the random variable X. First, the fixed column constraint is easily replaced by the constraint based on the frequencies of an item:

p i = f i
where p i is the probability that x i = 1 and f i is the observed frequency of item a i . Second, the fixed row constrain is replaced by a constraint based on the frequencies of transactions of size k:

∀k ∈ 0, m , p |X|=k = f |X|=k
where |X| is the number of ones in X. Note that these are, in fact, two well defined linear constraints with regards to the elementary probabilities p ω where ω ∈ Ω = {0, 1} m :

p i = ω i =1 ω∈Ω p ω and p |X|=k = |ω|=k ω∈Ω p ω
Of course, it could be argued that if n and m are large enough the two approaches described in this section give similar results. However, if we are concerned by the meaningfulness of the process, theoretical rigor is necessary and a distinction between the two must be made. If a swap-randomization method, based on a fixed row and column margins constraint, is the easier option in terms of computability (which is rarely the case) and that the method is aimed at prediction, then it should be specified that the swap-randomization method is only used to approximate the probability distribution defined in the second approach and the appropriateness of the approximation should be justified. Otherwise, the use of fixed row and column margins models should be limited to exhaustive datasets, for describing other inherent properties of a given system in which they justifiably correspond to background knowledge (such as in the previous example in ecology), or for summarization.

Recommendation

We have shown in this section that, in order to provide for a meaningful justification of the utilization of a data mining method for predictive tasks, the mathematical model for the empirical dataset must derive from a more general mathematical model encompassing all potential datasets. The model must also allow for a justification to why the observation of one dataset can provide information about the general mathematical model for all potential datasets.

These elements are, however, not compulsory for compression tasks.

In any case, it is important that the modeling process leading to the definition of the mathematical model for the data may be made explicit in order to provide for a meaningful explanation of the mining method and detect possible inconsistencies within the modeling process. This is not a superfluous task, as such inconsistencies are not rare in the literature.

Phenotypic modeling and genotypic modeling of interestingness

Phenotypic and genotypic modeling: a definition

In the previous section, we addressed a particular aspect of the general modeling process related to the modeling of data. In this section, we focus on another aspect of the modeling process related to the modeling of interestingness. To describe this aspect, we will borrow elements from the terminology of the field of biology which illustrate quite adequately this notion: phenotypes and genotypes. In biology, the phenotype of an organism is its observable characteristics while its genotype is its genetic makeup. The phenotype is an expression of the genotype within a certain environment. Before the discovery of the role of DNA in genetic transmission and the development of DNA sequencing technologies, the only way of describing an organism was through its phenotype. Nowadays, accessing the organism's genotype provides for deeper understanding of both the organism and its phenotype.

Similarly, in order to model an object or a concept, one can rely on its traits and characteristics (i.e. its phenotype) or on its core code from which these characteristics derive (i.e. its genotype). These correspond to two different modeling approaches, which we shall refer to as phenotypic modeling and genotypic modeling. While the first approach provides for a method to model an object whose intrinsic nature is unknown or unclear, the second approach generally provides for a more meaningful explanation of the properties of the object.

Indeed, if a mathematical model is defined so that some of its mathematical properties correspond to some of the characteristics of the modeled object, then it is still hard to justify why the other mathematical properties of the model should correspond to actual characteristics of the modeled object. This is an issue because there is no reason a priori that these other mathematical properties do not influence the way the model behaves in a given context.

Note that the concepts of genotypic and phenotypic modelings are close to the concepts of white-box and black-box models as described in [START_REF] Kalmykov | A white-box model of s-shaped and double s-shaped single-species population growth[END_REF].

However, this is not the standard use of the term black box model which corresponds usually to the inability to fully understand the mechanisms of the model itself and is an entirely different issue. This motivated our use of this novel terminology.

Phenotypic approaches, on the one hand, are commonly used to describe systems or concepts for which there is more consensus on their characteristics than on their intrinsic nature. This is often the case for notions in social sciences like economics where the use of indicators relies mostly on phenotypic modeling approaches. This can also be the case for the notion of interestingness as we will discuss further in this section.

On the other hand, genotypic approaches can be used for systems or concepts which have been clearly defined as resulting from some underlying mechanism. In this case, the modeled concept is represented by a mathematical object which is defined by other mathematical objects such as a quantity, a distance, a probability, or any other more complex mathematical object. as specific traits of interestingness. He then suggests the use of a measure that he defines as the most easily computable function which satisfies these properties. In a sense, the traits he considers are not defining characteristics of interestingness because there is an infinite number of possible functions which satisfy these properties while exhibiting very different behaviors otherwise.

However, he still relies on these properties to propose a mathematical model for interestingness. This type of modeling approach falls completely within the phenotypic modeling category which we have described.

In the later scientific contributions involving the definition of additional properties of objective interestingness measures and the classification of a number of such measures with regards to these properties [OKO + 04, GH06, LMVL08, BGK09, LB11, GSS12] 4 , the modeling processes for interestingness are slightly different but still fall into the same general category of phenotypic modeling approaches. Indeed, in these papers the properties suggested are meant to model possible traits of interestingness. The idea is to choose a model for interestingness, i.e. an objective interestingness measure, based on which properties the measure satisfies, that is based on which traits are chosen to characterize interestingness. The exact list of traits which are intended to be utilized in order to characterize interestingness in a given modeling process is not defined a priori, contrarily to what was the case in [START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF], but rather left to the user or an expert to specify. In any case, as in [START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF], the lists of traits chosen and their associated properties do not entirey characterize the measures that are picked to model interestingness. Indeed, the measures are only chosen from a relatively short list of measures defined explicitly in the literature, short with respect to the infinite number of substantially different potential measures which exist for any given combination of these supposedly characteristic properties. Therefore, such approaches fully register as phenotypic modeling approaches and carry the related issues in terms of meaningfulness.

It can be argued that, what these methods lose in terms of meaningfulness of the modeling process for interestingness, they gain in terms of flexibility towards the definition of interestingness. Indeed, for any combination of supposedly characteristic traits of interestingness5 , one can always either find an objective interestingness measure in the literature that fits the corresponding properties or suggest a novel objective interestingness measure which would fit. For example, in [START_REF] Blanchard | Semantics-based classification of rule interestingness measures[END_REF], the authors suggest two new measures in order to satisfy two combinations of properties which they believe could correspond to some users' view towards interestingness and for which they did not find any corresponding measure in the literature. However, the fact that no prior existing objective measure corresponded to these two visions of interestingness could also suggest that not all possible ways of describing interestingness correspond to what is seen as objective interestingness by most researchers.

It is worth noting, at this point, that there can be a semantic confusion regarding the expression objective interestingness measure. The term objective can be seen as qualifying the measure, interestingness, or both. If the term only applies to the measure (see paragraph 2.3.1.2 for more details concerning subjective and objective measures), then the modeling process described above qualifies as modeling interestingness through an objective interestingness measure. However, if the term applies to interestingness, as in mining for objectively interesting patterns, then this process appears as highly subjective. If the essence of interestingness is fundamentally subjective, then this is inevitable. Nevertheless, as we will expose in section 3.7, a notion of objective interestingness can very well be defined, and it certainly cannot be described by using alternative opposing views. Hence, these phenotypic ap-proaches carry inherent issues regarding the two main aspects of our goal to provide for meaningful modeling of objective interestingness.

Genotypic approaches for modeling interestingness

In the case of genotypic modeling, the modeled concept (in this case interestingness) is modeled as a mathematical object which is defined by other mathematical objects within the model. This is the case for the models described in sections 2.3.3.1, 2.3.3.2 and 2.3.3.3 in which interestingness of a pattern is defined as its statistical surprisingness or informativeness (in the sense of information theory) relative to some data model. This is also the case for the various approaches described in section 2.3.3.4 in which sets of patterns are considered interesting if the data model they define is a good model for the dataset and where the specifics of the modeling process depends on the choice of the measure for evaluating the different data models. Notably, this is the case for the data mining as summarization approach which uses the minimum description length principle. Indeed, in this approach, the interestingness of a given set of patterns is equated with the length (or more exactly the shortness)

of the description of the data given by the data model defined by the given set of patterns with the correction corresponding to the error between the data model and the empirical data.

In all cases described above, the modeling of interestingness is associated to a clear definition of interestingness. Although one might question the adequacy of the specific definition chosen for interestingness, the modeling processes are clearly meaningful with regards to this choice.

When phenotypic modeling meets genotypic modeling: modeling information

We set aside the issue of modeling interestingness here to focus on the modeling of informativeness and, more generally, information through entropy. As we have mentioned previously, maximum entropy (MaxEnt) models are commonly used in frequency-based interesting pattern mining. A study of the rationale towards the use of entropy as a mathematical model for information is therefore essential for understanding the meaningfulness of a modeling process based on MaxEnt models. We will show here that there exists both phenotypic and genotypic modeling approaches of information which lead to the same mathematical model of entropy.

Phenotypic approaches for modeling entropy

Information entropy was first presented by Claude E. Shannon in 1948 [START_REF] Elwood | A mathematical theory of communication[END_REF] as a function H defined for any probability distribution p = (p i ) 1≤i≤n as below.

H(p) = - n i=0 p i log p i
In this founding paper, Shannon asks the following question:

Can we find a measure of how much "choice" is involved in the selection of the event or of how uncertain we are of the outcome?

He then pursues immediately by asserting that:

If there is such a measure, say

H(p 1 , p 2 , • • • , p n ), it is reasonable
to require of it the following properties:

followed by a list of three fundamental properties. The paper then continues by stating as a theorem that entropy is the only function satisfying these properties up to a multiplicative constant. However, this theorem is not presented as the main justification for the use of entropy as a model for information:

This theorem, and the assumptions required for its proof, are in no way necessary for the present theory. It is given chiefly to lend a certain plausibility to some of our later definitions. The real justification of these definitions, however, will reside in their implications. [...] The quantity H has a number of interesting properties which further substantiate it as a reasonable measure of choice or information.

which is followed by six important properties of entropy.

Hence, Shannon establishes a distinction between axiomatic properties which uniquely define entropy and inferred properties. However, the rationale for the use of entropy as a model for information is based mostly on the properties of entropy as a whole, axiomatic and inferred. As such, this is clearly a phenotypic modeling approach. Later in the literature, some debate arose over whether the rationale for entropy relied on its definition through axiomatic properties or its properties as a whole [START_REF] Csiszár | Review of "on measures of information and their characterizationś[END_REF][START_REF] Shore | Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy[END_REF]. A number of different axiomatizations of entropy were suggested [Fad56, CM60, AFN74, AD75, For75, SJ80]. While it was acknowledged that these axiomatizations supported the justification of entropy as the unique natural model for information (as it is clearly stated in W. Weaver's introductory notes to the 1949 republication of Shannon's paper [START_REF] Elwood | The Mathematical Theory of Communication[END_REF]), the rationale for the entropy model for information was mostly based on its properties as a whole [START_REF] Csiszar | Information theory: coding theorems for discrete memoryless systems[END_REF][START_REF] Cover | Elements of information theory[END_REF].

In any case, both approaches are phenotypic in the sense that the modeling is based on the properties which are expected of a mathematical model for information. However, the uniqueness of such a measure obtained through the axiomatic approach suggests that a genotypic approach should yield the same model. Indeed, if this were not the case, it would imply either that some of the axiomatic properties chosen to define the entropy model do not correspond to characteristics of information or that information simply cannot be modeled mathematically. As we show in the next paragraph, genotypic approaches for modeling information do indeed yield the same entropy model.

Genotypic approaches for modeling information

A first genotypic approach towards entropy was described by Edwin T. Jaynes in [START_REF] Jaynes | On the rationale of maximum-entropy methods[END_REF]. For Jaynes, the rationale behind the use of the entropy model and, more specifically, MaxEnt models before his work was based on a number of intuitive principles. However, he stated that:

While each of these intuitions doubtless expresses an element of truth, none seems explicit enough to lend itself to a "hard" quantitative demonstration of the kind we are accustomed to having in other areas of applied mathematics. Accordingly, many approaching this field are disconcerted by what they sense as a kind of vagueness, the underlying theory lacking solid content.

The opposition that Jaynes placed in prior approaches towards entropy based on intuitions and an approach based on a "hard" quantitative demonstration corresponds here exactly to the opposition between phenotypic and genotypic modeling approaches which we have described in this section.

Consider a partition of N elements into n different categories. Let N 1 , ..., N n be the number of elements in each category. There are

W = N ! N 1 !N 2 !...N n !
partitions corresponding to the values N 1 , ..., N n . Hence, if we consider a random variable X = (X 1 , ..., X n ) for the partition of N elements in n categories given by the uniform distribution on the set of all such partitions (which is isomorphic to 1, n N ), we have:

Prob (X 1 = N 1 , ..., X n = N n ) = W n N
More generally, if we limit the set of possible partitions by constraining them such that (N 1 , ..., N n ) belongs to a non empty subset S of P N,n as defined below:

∅ S ⊂ P N,m = (N i ) 1≤i≤n ∈ 0, N n n i=0 N i = N Then: Prob (X 1 = N 1 , ..., X n = N n ) = W |S|
In both cases, the probabilities are proportional to W , so we can compare two probabilities associated to two cardinalities W and W by comparing these two cardinalities.

Furthermore, consider a probability distribution p = (p i ) 1≤i≤n and a sequence

N (k) i 1≤i≤n k∈N
such that:

N (k) = n i=1 N (k) i ----→ k→+∞ +∞ and: ∀i ∈ 1, n , N (k) i N (k) ----→ k→+∞ +∞ Then, if W (k)
k∈N is the corresponding sequence of cardinalities:

1

N (k) log W (k) ----→ k→+∞ H(p)
which is obtained through Stirling's approximation. Now, let S (k) k∈N be a sequence of non empty subsets of P N (k) ,n defined by:

S (k) = N (k) i 1≤i≤n ∈ P N (k) ,n f N (k) 1 N (k) , ..., N (k) n N (k) = v (k)
where f : [0, 1] n → R m is a continuous function and v (k) ----→ k→+∞ v so that f (p) = v defines a valid constraint on p. Then the probabilities defined previously go towards zero:

W (k) |S (k) | ----→ k→+∞ 0
and the cardinalities alone go towards infinity

W (k) ----→ k→+∞ 0
so that it is necessary to consider an asymptotic behavior such as described through entropy to compare the likelihood of two probability distributions p and q satisfying the same constraint given by f (p) = f (q) = v. In particular, arg max

f (p)=v H(p) = lim k→+∞    1 N (k) arg max N (k) i 1≤i≤n ∈S (k) W (N (k) 1 , ..., N (k) n ) |S (k) |    .
The previous expression means that maximizing entropy corresponds to maximizing the likelihood of the associated partition while considering a uniform distribution on all partitions of N elements into n categories in the limit case when N goes to infinity. 6 This allows for a genotypic approach towards the use of MaxEnt models based on an argument of maximum likelihood.

Jaynes' work also explores another aspect of this asymptotic behavior through his entropy concentration theorem. The theorem states that, in the case where f is a linear function of rank m < n and v (k) = v, then we have asymptotically

2N (k) max f (p)=v H(p) -H 1 N (k) X (k) ∼ χ 2 (n -m -1).
One of the corollaries to this theorem is that the average partition for a given N and f , divided by N , converges towards the MaxEnt model when N goes to infinity:

E X (k) N (k) ----→ k→+∞ arg max f (p)=v H(p)
This last expression is equivalent to our own genotypic modeling towards Max-Ent models through mutual constrained independence. In this modeling approach, which we focus on in the following chapter of this thesis, we consider that the distribution which is the least binding in terms of model hypothesis other than defined by a set of linear constraints is precisely given by

lim k→+∞ E(X (k) ) N (k)
. However, the formulation in chapter 4 of the results relative to the existence and convergence of this limit (Theorem 4.1.3) and its relationship to MaxEnt models (Theorem 4.1.4), as well as their proofs, differ significantly from the presentation above and provide deeper insight on the nature of this limit.

Through this analysis, we have shown that some phenotypic approaches can lead to the same mathematical models as genotypic approaches. This should be the case if the model is uniquely defined by the model properties defined by the phenotypic modeling. Nevertheless, the genotypic approach always adds further understanding and meaningfulness to the modeling process.

Recommendation

In this section, we have introduced two new concepts that describe two different approaches towards modeling: phenotypic and genotypic approaches. We have shown that the modeling processes of the latter type carry much more meaning than those of the prior type. Therefore, if meaningfulness is desired, they should be preferred when applicable. Although phenotypic approaches are the only way to proceed in certain cases, such as when dealing with highly complex notions, we do not believe this is the case for objective interestingness.

Hence, we recommend that a genotypic approach towards the modeling of interestingness be adopted to provide for higher meaningfulness.

Pragmatic modeling

One important aspect when opting for a modeling process, is that the mathematical model which is used in the end must be computable, at least to a certain extent, in order to provide for answers to the questions initially formulated about the system which is modeled. In a sense, one must be certain that the modeling intention meets with the practical capacities. While there is no inherent issue with this idea, we would like to underline the fact that focusing primarily on pragmatic aspects for defining modeling processes leads to issues in terms of meaningfulness.

We use the term pragmatic modeling to describe modeling processes for which the question what can we do? seems to come before the question what should we do? when justifying their utilization. If pragmatic modeling can provide answers, these are not necessarily useful because they are answers to a question which was formulated in order to be able to provide for such answers. This is a widely known issue in statistics, often phrased simply as the right answer to the wrong question. A linear regression model, for example, is always the right answer to the question of finding the linear model which fits the data best but this question might be completely irrelevant and meaningless with regards to the data. John Tukey, a statistician who is considered to have been one of the pioneers in the development of exploratory data analysis and the establishment of principles for statistical practice [Tuk62, Tuk77, Tuk80], summarized this particular principle in [START_REF] Tukey | The future of data analysis[END_REF] as such:

Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.

Pragmatic modeling of interestingness

The frequent itemset mining problem and the association rule mining problem presented in [START_REF] Rakesh Agrawal | Mining association rules between sets of items in large databases[END_REF] (see sections 2.2.1.1 and 2.2.1.2 for more details) represent two typical cases of pragmatic modeling. In both cases, the seemingly vague question of finding interesting associations or interesting rules between associations in data are modeled by the very precise questions of finding all frequent items or all strong association rules. In both cases, the main motivation behind the modeling is the monotonicity properties of the support and confidence measures which provide a basis for the efficient mining of frequent itemsets and strong association rules. In other words, the Apriori algorithm is the main motivation for the definition of the itemset and association rule models. The meaningfulness and the adequacy of these modeling choices are secondary considerations at best. With retrospect, the fact that pragmatic modeling and its consequences were not systematically and explicitly pinpointed as an issue here had a larger impact than one might imagine at first. Indeed, the popularity of frequent itemset and association rule mining led many researchers to focus on efficiently computing (see section 2.2.2) or perfectly summarizing (see section 2.3.2) what was simply the right answer to the wrong question in most applied cases.

Pragmatic modeling in the field of pattern mining is not limited to frequent itemset mining and association rule mining. As described in section 2.3.1.2, measures can be chosen to model interestingness based on the facility to compute them or their algorithmic properties with little regard to their meaningfulness.

Meaningfulness first, computability second

It is important to note that trying to define computable models is not in itself an issue. The issue only arises when meaningfulness comes after computability. Following John Tukey's principle, if a meaningful model is defined and that it is not directly computable, then it is better to try to approximate the model than to entirely redefine the problem statement. Examples of such an approach may be found in the scientific literature on interesting pattern mining specifically in papers related to the mining through compression paradigm (see section 2.3.3.4). Indeed, the proponents of this approach defend that the most interesting set of patterns in a dataset are those that can be used to provide the most concise lossless compression of the data, the size of such an optimal compression being modeled through Kolmogorov complexity. As such, the modeling of interestingness is highly meaningful. However, there is a serious computability issue with this modeling. As a matter of fact, it is a proven result in algorithmic information theory that Kolmogorov complexity is uncomputable: no program can compute the Kolmogorov complexity for all possible datasets [START_REF] Li | An introduction to Kolmogorov complexity and its applications[END_REF]. Hence, Kolmogorov complexity is approximated using other notions such as the minimum description length [START_REF] Peter | Advances in minimum description length: Theory and applications[END_REF].

Recommendation

Similarly to the popular saying that states that just because you can, doesn't mean you should, just because a model is computable doesn't mean it carries any meaning. If meaningfulness is a criteria, a modeling process for objective interestingness should start by considering a truly meaningful approach towards modeling objective interestingness regardless of any computational aspects. This might lead to the case in which computing the solutions to the mathematical problems associated with the model is technically or theoretically infeasible. In such a case, means for approximating these solutions in a computable way should be envisaged.

Patchwork and holistic modeling processes

In this section we address the issue of meaningfully and consistently connecting the various parts that compose a general modeling process. As was the case for the issues described in the previous sections, we feel that there is no adequate preexisting terminology in order to describe this particular modeling issue. Therefore, we introduce two concepts which describe two opposing approaches towards mathematical modeling. On the one hand, we consider holistic modeling and, on the other hand, patchwork modeling. Holistic modeling describes a general modeling process in which a single mathematical model englobes every particular aspect of the world considered within the general modeling process. Patchwork modeling describes a general modeling process in which multiple mathematical models are used for modeling the different aspects of the world which is considered within the general modeling process. The use of the terminology holistic is based on the notion conveyed in holism that the whole is not equivalent to the sum of its parts. By contrast, a patchwork is constructed precisely as the sum of different parts.

Patchwork modelings can be decomposed into two steps. The first step consists in modeling the world by its different aspects. We call this step the projection modeling as, in a sense, it corresponds to the projection of the world onto each of the different aspects considered. Note that this step can be lossy (if not all aspects of the world are modeled) as well as redundant (if some elements of the world are described in several aspects of the projection modeling). The second step consists in the mathematical modelings for each of the local projections considered through the projection modeling. Together these two steps make up a general patchwork modeling process as illustrated in Figure 3 If a truly meaningful explanation of a complex modeling process can only be provided for mathematical modeling processes, then holistic modeling is the only approach that allows for a meaningful explanation of the general modeling process. Indeed, in a patchwork modeling approach the general modeling process is not a mathematical modeling process. Even if each part of the world is modeled by mathematical models, the world itself is modeled by this patchwork of mathematical models, which is not a priori a mathematical model.

Nevertheless, the fact that a patchwork model is not described through a mathematical modeling process does not necessarily imply that such a description cannot be given.

In some cases, a general mathematical layer is omitted simply because defining it feels unnecessarily tedious. We note this category of modeling processes PW0 (for patchwork type 0). However, in many cases, it is impossible to define a general mathematical modeling that is both meaningful and consistent with each local mathematical modeling within the general modeling process.

We note PW1 the category of modeling processes for which a consistent yet meaningless general mathematical modeling may be defined, at least meaningless in the sense that it carries no further meaning than being simply the junction of the different models from the projection step in the modeling process. Conversely, we note PW2 the category in which consistency is an issue.

PW2 corresponds to cases in which the different aspects in the projection are not entirely disjoint and one of the elements of the world, associated to two different aspects in the projection, is modeled by two different mathematical objects rendering a general mathematical model inconsistent. In other cases, this general mathematical modeling layer is absent because the current status of mathematical knowledge does not allow for its definition. We note PW3 this last category.

To each of these different categories correspond different modeling issues.

For a modeling process in the PW0 category, this may simply correspond to a case of implicit holistic modeling and the general mathematical modeling layer is not described because its definition is obvious. While this may well be the case, we still recommend that this be made explicit because what may seem obvious at first is not necessarily obvious when formalized. Omitting the general mathematical modeling may in fact hide issues related to the other PW1, PW2 and PW3 categories described above. Modeling processes pertaining to the PW1 and PW2 categories present irresolvable issues towards the definition of a meaningful holistic mathematical modeling. We will exhibit how some of the modeling processes used in frequency-based interesting pattern mining belong to these categories.

The PW3 category corresponds to modeling processes for which we do not know yet how or even if it is possible to define a holistic mathematical model that is proven to be consistent with the local mathematical models within the general modeling process. In such a case, the modeling process cannot be meaningfully explained because we do not know exactly why it should be a good modeling process or not. Although we did not encounter modeling processes in the frequency-based itemset mining literature which belonged to this category, we believe it is still worth mentioning here because it is linked to the main debate surrounding the notion of meaningfulness in the field of artificial intelligence: the possibility to meaningfully explain deep learning algorithms. Indeed, the local mathematical models that compose deep learning algorithms such as artificial neurons are easily understood and explained. Furthermore, in the case of single layer neural networks, the relationship between the artificial neurons and the general network can be meaningfully explained because we have sufficient mathematical knowledge on the convergence of such models [START_REF] Novikoff | On convergence proofs for perceptrons[END_REF]. However, there are no sufficiently general mathematical results known as of today which prove that multilayer neural networks converge towards mathematical solutions to the mathematical problems which model the real life questions that these algorithms aim at answering (even though it must be admitted that a number of intermediary theoretical results have been proven quite recently [APVZ14, LY17, ACGH18, AZLS18, BHL19, CJRR21]).

This means that we can observe that deep learning algorithms work but we cannot really explain why.

Note that holistic modeling cannot be reduced to a vain pursuit of math- Phenotypic modeling approaches, as those we have described in section 3.3, may often be associated to the PW1 category. Indeed, a phenotypic modeling approach is an attempt to define a holistic mathematical modeling which matches with all the local mathematical models arising from the projection step and that correspond to characteristics of the world. As we have described previously, this process does not necessarily allow for the general modeling process to carry any larger meaning than the description of the individual characteristics of the subject of the modeling. In fact, the only case in which phenotypic modelings are not akin to PW1 modelings is when the characteris-tics fully and uniquely characterize the subject of the modeling as is the case with the modelings involving entropy described in section 3.3.3.

Other examples may include the use of multiple objective interestingness measures. For example, considering that interesting rules are defined by a support greater than 25%, a confidence greater than 95% and a lift larger than 1.5 corresponds to finding a rule a → b such that the vector (f a∧b , f a∧b fa , f a∧b faf b ) is greater than (.25, .95, 1.5) for the partial product ordering and this carries no more meaning than the sum of the individual models given by f a∧b ≥ .25, f a∧b fa ≥ .95 and f a∧b faf b ≥ 1.5.

Type 2 patchwork modelings (PW2)

We have noted a high frequency of issues related to PW2 modelings when considering local models for redundancy (see section 2.3.3.1). To illustrate this we give two examples: one based on local independence models for rule mining; and one based on local MaxEnt models for itemset mining.

Consider two rules between itemsets a 1 → a 2 and a 1 ∧ a 2 → a 3 and a modeling for interestingness based on the local independence model between the antecedent and the consequent of rules. This means that we consider implicitly at least that the data corresponding to items a and b is generated by a random variable X = (X 1 , X 2 ) such that p a 1 = f a 1 , p a 2 = f a 2 and p a 1 ∧a 2 = p a 1 p a 2 = f a 1 f a 2 for a 1 → a 2 ; and that the data corresponding to items a 1 , a 2 and a 3 is generated by another random variable X = (X 1 , X 2 , X 3 )

such that p a 1 ∧a 2 = f a 1 ∧a 2 , p a 3 = f a 3 and p a 1 ∧a 2 ∧a 3 = p a 1 ∧a 2 p a 3 = f a 1 ∧a 2 f a 3 for a 1 ∧ a 2 → a 3 . If f a 1 ∧a 2 = f a 1 f a 2
, which is generally the case, these two models are not consistent. They cannot be seen as local projections of a global model in which the data is considered to be generated by a random variable X = (X 1 , ..., X m ). Note that an approach based on mutual local independence models for itemsets does not, however, share this issue as such local models are local projections of the global mutual independence model.

We will now consider an example in which the use of local MaxEnt models also leads to a case of global inconsistency. Consider four items a 1 , a 2 , a 3 , a 4

and the four local MaxEnt models for each of the itemsets of size 3, defined by the frequencies of all itemsets of size 2 or less. Then these models are not necessarily globally consistent. Indeed, not only are they generally not a projection of the global MaxEnt model for four items defined by all itemsets of size 2 or less (unless there is a specific pattern of independence), but they might not even be the projection of any possible probability distribution model. We

give one specific counter-example supporting this last statement.

Consider an empirical distribution satisfying the conditions given by the itemsets of size 3 defined by the frequencies of the itemsets of size 2 or less we obtain the probabilities in table 3.3.

Itemset Probability a 1 ∧ a 2 ∧ a 3 .0048564 a 1 ∧ a 2 ∧ a 4 .0083261 a 1 ∧ a 3 ∧ a 4
.0030280 a 2 ∧ a 3 ∧ a 4

.0054097

Table 3.3: Probabilities defined by the local MaxEnt models However, we see that the following condition given by the inclusion-exclusion principle:

p a 1 ∧a 2 ∧a 3 ∧a 4 ≤ p a 1 -p a 1 ∧a 2 -p a 1 ∧a 3 -p a 1 ∧a 4 + p a 1 ∧a 2 ∧a 3 + p a 1 ∧a 2 ∧a 4 + p a 1 ∧a 3 ∧a 4
does not allow to define a globally consistent model as we have:

p a 1 -p a 1 ∧a 2 -p a 1 ∧a 3 -p a 1 ∧a 4 + p a 1 ∧a 2 ∧a 3 + p a 1 ∧a 2 ∧a 4 + p a 1 ∧a 3 ∧a 4 = -.0019895
which would imply a negative value for p a 1 ∧a 2 ∧a 3 ∧a 4 .

Note that such counter-examples correspond to skewed distributions. Indeed, in order to present such a case here, we randomly generated one million hypothetical empirical distributions f = (f i ) 0≤i≤15 by randomly picking 16 integers between 1 and 100 and dividing them by there sum. For each of these distributions, we determined the values for the probabilities of the itemsets of size 3 given by the local MaxEnt models defined by the frequencies of itemsets of size 2 or less. We then proceeded by checking whether the values of the frequencies of itemsets of size 2 or less together with the probabilities of the itemsets of size 3 defined by the local MaxEnt models where globally consistent using the necessary and sufficient conditions given by the inclusionexclusion principle (see for example Figure 1 in [START_REF] Calders | Non-derivable itemset mining[END_REF]). Out of the million distributions that were generated, only forty were not globally consistent, each of which corresponded to skewed distributions. As such distributions correspond to structured data, they are the most relevant in interesting pattern mining. Therefore, this reveals one of the important limits to a naive use of local MaxEnt models for interesting pattern mining.

From the example which precedes, we can say that local MaxEnt models based on a global set of constraints do not necessarily provide for a globally consistent model. However, this does not mean that the result holds for every set of constraints. Indeed, the MaxEnt model given by the set of constraints defined by the frequencies of items is the mutual independence model and we know that, in this case, the local models are projections of the global model.

Hence, there are sets of constraints that do not always allow for consistency between local models and sets of constraints that do. Nevertheless, and this is simply a conjecture, we believe that the latter are quite the exception and that only a negligible fraction of the 2 d constraints defined by sets of itemsets always allow for global consistency between local models as in the case of mutual independence.

Recommendation

In this section, we have introduced two new concepts to qualify two opposing modeling approaches: patchwork and holistic modeling. We have shown that patchwork modeling can lead to meaningless or inconsistent mathematical models. Even when this is not the case, meaningfulness arises when making explicit an implicit holistic model. If meaningfulness is a main criteria, holistic modelings should be preferred. This does not imply that local projections of a holistic model should never be considered, but only that they should be considered as such. In other words, the definition of any local model should stem from the prior definition of a global model in order to be able to justify the meaningfulness of the general modeling process.

Mathematical modeling of patterns

In this section, we focus on the structure of the mathematical model for describing the patterns which are extracted in the mining process. In the standard itemset model, the patterns are sets of items (a.k.a. itemsets) and the set of all possible patterns is simply described as the set of all itemsets with its natural lattice structure (see section 2.2.1.1). This model, still currently very much in use in the data mining community, is a heritage from the early presentation of itemset mining as a tool for market basket analysis by Agrawal et al. in [START_REF] Rakesh Agrawal | Mining association rules between sets of items in large databases[END_REF]. If it is quite simple to apprehend, it is also quite limited and limiting with regards to the possibilities that it allows in terms of mathematical modeling. In fact, it can be considered a rather poor modeling choice for studying the fundamental measure in itemset mining: frequency8 . Indeed, some of the most basic properties of frequency, for example those based on the inclusionexclusion principle, are much more naturally understood if a more general structure is considered, which is why generalized itemsets were introduced in this specific example (see section 2.3.2.3).

At this point, it is important to recall that, in this thesis, we are addressing specific issues in frequency-based pattern mining. This means that frequency is at the core of any mining process, which might not necessarily be the case in other domains of pattern mining such as pattern recognition.

In fact, the patterns which are considered are usually objects together with their frequencies. If we sometimes use the term pattern to refer to an object regardless of its frequency, as in an itemset is a pattern, then it is generally a misnomer. Note that this can also be seen as part of Agrawal's legacy. Indeed, the phrasing of the frequent itemset mining problem suggests that the set of frequent itemsets alone, rather than the set of frequent itemsets together with their frequencies, is interesting. However, it is now widely acknowledged in the literature, whether explicitly or implicitly, that an object must always be considered together with its frequency.

Measure spaces and Boolean lattices

The natural mathematical structure for considering both itemsets and their frequencies is the measurable space generated by the set of all itemsets fitted with the empirical distribution in the dataset. For this reason and those we will present in section 3.6.2, this model is by far a more satisfying choice than the simple set model.

In order to consider frequency as a measure (in the sense of mathematical measure theory), terminology must first be explicitly specified as the standard usage in both itemset mining and measure theory may reflect opposite ideas.

Indeed, in itemset mining, focus is set on itemsets. Hence, the frequency of a given itemset X is noted as defined by X, for example f X . The frequency of a union of itemsets X = X 1 ∪ X 2 can therefore be noted f X 1 ∪X 2 . By contrast, in measure theory the focus is set on measurable sets (i.e. the equivalent of events in probability theory), which can be equated to tidsets in the itemset context.

Hence, if A, A 1 and A 2 are the measurable sets associated to X, X 1 and X 2 respectively and X = X 1 ∪ X 2 , then A = A 1 ∩ A 2 . Therefore, if the same notations are used for itemsets and their associated measurable sets (which is not uncommon), the notation f X 1 ∩X 2 in the context of measure theory might very well be used to designate the same notion as the notation f X 1 ∪X 2 in the context of itemset mining.

One approach which allows to remove all ambiguity is to consider the natural Boolean lattice structure of the measure space considered here. In this case, we can associate an itemset X = {x 1 , ..., x k } to a propositional logic formula X = x 1 ∧ ... ∧ x k using the same notations for itemsets and their corresponding formulas, as well as items and their corresponding atomic formulas, without cause for concern. Using this representation, the notation for the frequency of the unioned itemset X 1 ∪ X 2 becomes f X 1 ∧X 2 . The Boolean lattice model corresponds to identifying the measurable sets in the measure space model by their corresponding propositions in the Boolean lattice. This additional modeling layer helps for removing ambiguities such as described above and is also generally quite suited to the context of frequency-based itemset mining as we show in section 3.6.2.

Benefits of the measure space and Boolean lattice models

We present here some aspects in which using the measure space and Boolean lattice models for describing the pattern space can be quite beneficial to the general modeling process. The first focuses on the measure space model, the next three focus on the Boolean lattice model, and the last addresses benefits of the combined approach. We also question the relevance of the rule mining paradigm in light of these explanations.

Modeling the dataset using a random variable

Previously in this thesis, we have established that the dataset can be adequately modeled as the result of the random sampling of n independent identically distributed random variables with values in {0, 1} m . Modeling the patterns in the data as measurable sets from a measure space provides a context for transferring information on the patterns in order to define the probability distribution of the random variables because measure spaces and probability spaces have exactly the same structure (a probability space is simply a measure space such that the measure of the entire space equals one). Indeed, a distribution for the random variable can be defined by a certain number of constraints corresponding to observed patterns in the data together with a method for defining a unique probability distribution based on these constraints (using the MaxEnt principle or junction trees for example).

One may then wonder why we do not simply choose to consider two probability spaces rather than a measure space for the patterns to extract and a probability space for the random variable. This is because there is nothing probabilistic in the observation of the patterns in the data, probabilities are only used here to describe our ignorance about the general process that generated the data.

Pattern diversity

One of the major advantages of the Boolean lattice is that it allows to consider much more patterns than just itemsets, without the need to add an extra layer of modeling each and every time a new type of pattern needs to be considered. The Boolean lattice contains propositions which correspond to itemsets and negative itemsets. It also contains propositions corresponding to the 3 m generalized itemsets, but more generally still it contains exactly all of the 2 2 m disjunctions of the 2 m minimal generators of the Boolean lattice (which correspond to the generalized itemsets of size m). Of course this is too large to hope to ever consider all possible types of patterns. We will explain further in this section how we can choose to constrain the mining process to a smaller family of patterns. However, considering this huge space is necessary if we want to include the majority of the great diversity of patterns which we have encountered in the literature.

For example, the Boolean lattice contains all logical implications between itemsets which are the core patterns of statistical implicative analysis [START_REF] Gras | L'analyse statistique implicative Méthode exploratoire et confirmatoire à la recherche de causalités[END_REF].

Another example is given by the four different constraints used as possible background knowledge in [START_REF] Tatti | Using background knowledge to rank itemsets[END_REF]: column margins, row margins, lazarus counts and transaction bounds. Indeed, each of these constraints can be associated to a specific set of elements of the Boolean lattice. While column margins correspond quite trivially to the set of items, which are the simplest propositions in the Boolean lattice, the other constraints are described using much more complex propositions. The row margins constraint is associated to the following set of the m + 1 propositions:

I∈( 1,m k ) i∈I x i ∧ i / ∈I ¬x i
such that k ∈ 0, m . Similarly, the lazarus counts constraint, where a lazarus count gives the number of zeros in between two ones in a transaction, is associated to the set of all propositions:

1≤a≤b≤m    i<a ¬x i ∧ x a ∧    I∈( a,b k ) i∈I x i ∧ i / ∈I ¬x i    ∧ x b ∧ b<i ¬x i   
where k ∈ 0, m -2 . And finally, the transaction bounds constraint, where bounds correspond to the first and last positions for a one in a transaction, is associated to the set of propositions:

i<a ¬x i ∧ x a ∧ x b ∧ b<i ¬x i
where 1 ≤ a ≤ b ≤ m or a = b = 0.

Pattern complexity

One of the important issues when considering the interestingness of a pattern is its complexity. Indeed, if a single pattern allows to define a model that closely fits the data but the description of the pattern itself is impossible to apprehend then the pattern should not be considered interesting. While this issue is widely acknowledged in the field of pattern mining, it has been rarely addressed both theoretically and objectively, with the notable exclusion of the research conducted within the mining as compression paradigm. Indeed, the fact that a large portion of the literature focuses on rule mining is mainly motivated by the fact humans, among other intelligent systems, easily interpret rules. This is however a very subjective approach and, to define the notion of complexity objectively, one requires an objective description of the language in which the patterns are expressed.

The Boolean lattice model provides for a good basis for such a description.

In fact, the first-order languages which derive from such structures have been extensively studied in formal language theory [START_REF] Rautenberg | A concise introduction to mathematical logic[END_REF]. Moreover, the study of the synonyms in such languages, that is logically equivalent propositions, is by itself an important field of research [START_REF] Hachtel | Logic synthesis and verification algorithms[END_REF][START_REF] Meinel | Algorithms and Data Structures in VLSI Design: OBDD-foundations and applications[END_REF][START_REF] Peter | Switching theory: Insight through predicate logic[END_REF]. Indeed, each individual element in the Boolean lattice corresponds to an infinite number of propositional logic formulas each of which have their own complexity. For a given propositional logic formula, the task of transforming it into a synonym of a given type to reduce its complexity is a well known problem in computer science often referred to as the minimum equivalent expression problem

[Uma01, BU11].
Note that this is, in itself, a highly complex issue and beyond the scope of this thesis. However, I have conducted research in this specific area during the time of my doctoral thesis and published a paper analyzing the complexity of different conjunctive normal form encodings of cardinality constraints for Boolean satisfiability problems [START_REF] Delacroix | Choisir un encodage cnf de contraintes de cardinalité performant pour sat[END_REF]. Such cardinality constraints are involved in particular in the definition of the row margins constraint as an element of the Boolean lattice. While the row margins constraint is easily apprehensible by a human, its corresponding proposition is too long (and irreducibly so in the language deriving from the Boolean lattice) for it to be effectively used by a computer program. To be more precise, the constraint is concisely expressed in second-order logic but has a lengthy expression in first-order logic. As the state-of-the-art of propositional satisfiability solvers operate in first-order logic (even though there have been some recent attempts at defining second-order SAT solvers [START_REF] David | Second-order propositional satisfiability[END_REF]), problems expressed in secondorder logic are still generally transposed into first-order logic when considered by a computer program. In this case, one must rely on an extension of the language (i.e. the addition of new Boolean variables together with a characterizing propositional formula known as the encoding) in order to be able to reduce the complexity of the proposition. This example, among other aspects which will be addressed further in this chapter, supports the idea which we defend that no artificial intelligence may be developed to gain substantial objective and interesting information about the world unless it is capable of complexifying the native language which it uses to describe the world (by resorting to new variables and encodings in first-order logic or defining propo-sitions in second-order logic). This is exactly what allows humans to consider that certain patterns, which are inherently more complex than other patterns as is the case with the row margins constraint, are still more interpretable than simpler patterns.

Type diversity

Up to this point, we have only considered that the variables in the data are binary. Though we will maintain this view in the rest of this thesis, we briefly present here possible generalizations to include other types of variables which can be easily integrated within the Boolean lattice model. The first type consists of categorical variables. A categorical variable has a natural representation using Boolean variables and a constraint. Indeed, if a is a variable which can take any one of p values v 1 , ..., v p , then we can consider p variables a 1 , ..., a p together with the cardinality constraint that exactly one of these variables is true. This cardinality constraint can be given optimally by its naive conjunctive normal form:

1≤i≤p a i ∧ 1≤i<j≤p (¬a i ∨ ¬a j )
for p ≤ 5 or using a more elaborate encoding for larger values of p (see [START_REF] Delacroix | Choisir un encodage cnf de contraintes de cardinalité performant pour sat[END_REF]).

Similarly, an ordinal variable a taking values in 0, p can be replaced by p + 1 binary variables a 0 , ..., a p together with the CNF constraint that:

0≤i≤p a i ∧ 1≤i≤p (a i-1 ∨ ¬a i )
On another level, fuzzy logic may be used as a generalization of Boolean logic to consider numeric variables which correspond to fuzzy data. Fuzzy approaches have been considered in itemset mining [DMSV03, DHP06] and, in the general modeling process which we describe, a fuzzy modeling layer may be considered without raising any theoretical issues.

Sound and complete families of patterns

As we have mentioned previously, the size of the Boolean lattice is extremely large and the mining process must therefore be constrained to certain types of propositions within the lattice. In order to determine which type of patterns to mine for, one possible criteria is that the family of patterns considered allows for a complete and sound description of the frequency measure. This means that any frequency measure is uniquely defined on the entire measure space by the frequencies of the patterns in the family (completeness) and that no subfamily of the family holds this property (soundness). We will consider three families of patterns which satisfy these conditions: minimal generators;

itemsets; and implications between complementary itemsets. In order to prove this result, we introduce a few elements of notation and definitions.

Consider m items a 1 , ..., a m and the associated Boolean lattice B. We give the following definitions for completeness and soundness.

Definition 3.6.1 (Completeness). Let F be an ordered subset of B. Then F is a complete family of patterns if, and only if, for any two measures µ 1 and

µ 2 defined on B, (∀P ∈ F, µ 1 (P ) = µ 2 (P )) =⇒ (µ 1 = µ 2 ).
Definition 3.6.2 (Soundness). F is a sound family of patterns if, and only if, for every subset F F, there exists two measures µ and µ defined on B such that ∀P ∈ F , µ (P ) = µ(P ) and ∃P ∈ F, µ (P ) = µ(P ).

We note Ω the set of minimal generators of B defined by:

Ω = i∈A a i ∧ i / ∈A ¬a i ∈ B A ⊂ 1, m
For conciseness, we note d = 2 m -1. We consider the natural lexicographic order on Ω given by the following sequence of increasing bijections:

0, d → {0, 1} m → Ω k → k → ω k
where k is the binary representation of k as a tuple in {0, 1} m and

ω k = i∈A k a i ∧ i / ∈A k ¬a i such that A k = {i ∈ 1, m | k i = 1}
. Which gives, for example:

ω 0 = 1≤i≤m ¬a i , ω 1 = 1≤i<m a i ∧ a m and ω d = 1≤i≤m a i
Furthermore, we note I the set of all itemsets of B:

I = i∈A a i ∈ B A ⊂ 1, m
Similarly, we consider the natural lexicographic order on I given by the following sequence of increasing bijections:

0, d → {0, 1} m → I k → k → I k where I k = i∈A k
a i and both k and A k are defined as previously. Which gives here:

I 0 = , I 1 = a m and I d = 1≤i≤m a i
Lastly, we note R the set of all implications between complementary itemsets defined by:

R = i∈A a i =⇒ i / ∈A a i ∈ B A ⊂ 1, m
Again, we consider the natural lexicographic order on R given by the following sequence of increasing bijections:

0, d → {0, 1} m → R k → k → R k where R k = i∈A k a i =⇒ i / ∈A k
a i with the same notations as above. In which case, we have:

R 0 = 1≤i≤m a i , R 1 = 1≤i<m a i =⇒ a m and R d =
Notice that, for any pattern P ∈ B, there is a unique minimal decomposition as a disjunction of elements in Ω and we can therefore associate this decomposition to a unique vector in {0, 1} d+1 representing P in the basis Ω (given an order on Ω). Hence, any family of patterns This result is trivial for Ω by definition as they are minimal disjoint generators of B. It is less trivial but still common knowledge for I and can be demonstrated using the exclusion-inclusion principle for defining the matrix representation of I in Ω. We give the proof for R that is based on the following lemma (which incidentally also provides a proof for I without having to define the corresponding matrix).

Lemma 3.6.1. Consider S = (S k ) 0≤k≤d a family of patterns in B defined by

S 0 = R d , S k = ¬R k for all k ∈ 1, d -1 and S d = R 0 . Then, ∀k ∈ 0, d , k≤i≤d ω i = k≤i≤d S i = k≤i≤d I i
Proof of the lemma. For all k ∈ 1, d -1 ,

S k = ¬ i∈A k a i =⇒ i / ∈A k a i = ¬ ¬ i∈A k a i ∨ i / ∈A k a i = i∈A k a i ∧ i / ∈A k ¬a i . ω k = i∈A k a i ∧ i / ∈A k ¬a i and I k = i∈A k a i , this gives: ∀j ∈ 1, d -1 , ω k =⇒ S k =⇒ I k Moreover, ω d = S d = I d = 1≤i≤m a i . Therefore, ∀k ∈ 1, d , k≤i≤d ω i =⇒ k≤i≤d S i =⇒ k≤i≤d I i

Furthermore, as

0≤i≤d ω i = S 0 = I 0 = , the previous relationship is still true for all j ∈ 0, d . Now consider the sequences of implications:

ω d =⇒ (ω d-1 ∨ ω d ) =⇒ ... =⇒ k<i≤d ω i =⇒ k≤i≤d ω i =⇒ ... =⇒ and
I d =⇒ (I d-1 ∨ I d ) =⇒ ... =⇒ k<i≤d I i =⇒ k≤i≤d I i =⇒ ... =⇒
Both are strictly monotonic. This is trivial for the first sequence and we can see this for the second by noticing that ω k ∧ k<i≤d

I i = 0 whereas ω k ∧ k≤i≤d I i = ω k .
Finally, we see that both sequences have maximal length d + 1 (which corresponds to the number of possible interpretations in B). Hence, they are equal, as well as the third sequence which lies in between the two.

Proof of the proposition. From the previous lemma, we can see that the matrix representations for S and I in Ω are invertible triangular and hence they are both complete and sound families of patterns. The definition of S from R implies that the latter is also complete and, given its cardinality, it is necessarily sound. This concludes the proof of the proposition.

Now that we have presented the notions of complete and sound families of patterns and given a few examples of such families, we address two questions:

first the relationship of these families with regards to a random variable model for the data; and second the issue of deciding which family to choose for the mining process.

In a sense, complete and sound families allow for a full and minimal description of any measure on the measure space. In the theoretical case in which the dataset corresponds to a random sample of an infinite number of independent identically distributed random variables with values in {0, 1} m (which implies a dataset with infinitely many transactions) then a complete and sound family of patterns allows for a full description of the probability dis-tribution for the random variables with but a single redundancy corresponding to the fact that probability of the entire space (corresponding to the truth value in the Boolean lattice) is always equal to 1. An analogous notion of complete and sound families of patterns for defining the probability distribution can be given by considering, for all complete and sound family of patterns F = (P i ) 0≤i≤d (with regards to the definition of a measure), the subsets of size

d, F j = (P i ) 0≤i≤d i =j
, such that i =j P i is not a tautology. In the case of itemsets, this corresponds necessarily to removing the empty itemset I 0 , in the case of R, to removing R d , and in the case of Ω, to removing any one of its elements. Now, considering the issue of picking one particular family of patterns for the mining process, what criteria can we base this decision on, knowing that they are all already minimal? First, notice that they are only minimal in the sense that they contain a minimal number of patterns, not in the sense of the complexity of the patterns themselves. For example, the description of any pattern in Ω contains m literals while an itemset can contain but a single literal.

This may seem to be a large difference but it can be moderated by the fact that the total number of literals for every pattern in Ω is equal to m2 m while it is equal to m i=0 m i i = m2 m-1 for itemsets. Even though the number is smaller in the case of itemsets, it is only twice as small as for minimal generators.

As it is usually easier to represent any given pattern in B as a disjunction of minimal generators rather than as combination of itemsets, it is possible that a complete description of a measure is better given by its values on Ω than on I.

Note however, that the value m2 m-1 obtained for the total number of literals in I is the lowest possible bound for a complete and sound family of patterns in B. Furthermore, complete descriptions are usually not what we are aiming for. Indeed, we do not have infinite datasets so a complete description of the data would not give a complete description of the random variable but rather suffer from data over-fitting. Hence considering itemsets, particularly with a level-wise approach makes sense because it allows to consider the least complex patterns first. Another approach is to consider that certain patterns are more interpretable than others, such as rule type patterns of which implications are the standard example in Boolean logic. This motivated our search for a complete and sound family of implications which lead us to the definition of R.

We must also insist on the fact that it is not necessarily important to constrain mining processes to patterns within a sound family of patterns as long as the family of patterns mined during the process stays sound (which depends mostly on the size of m). For example, when mining for itemsets given the row margins constraint, we can consider that the we are mining for patterns in a family of 2 m + m + 1 patterns, m + 1 of which are already known. While this family is complete, because it contains the complete family of itemsets, it is not sound. However, if m is very large, the mining process will never reach a point in which the itemsets mined, together with the m + 1 row margins constraint, do not make up for a sound family of patterns. Nevertheless, this issue may easily arise in the case of small values of m.

Rule mining

Note that throughout this section, we have hardly addressed the issue of rule mining and not, in any case, considered rules such as they are defined in association rule mining or, more generally, rules characterized by any objective interestingness measure. This is due to the fact that such rules do not correspond to any element in the Boolean lattice. To be more precise, they correspond to tuples of elements from the Boolean lattice together with their associated frequencies in the measure space. Indeed, recall that in the standard itemset model an association rule X → Y can be defined for any two itemsets X = {x 1 , ..., x k } and Y = {y 1 , ..., y l } such that X ∩ Y = ∅ and is characterized by its support and confidence measures which are equal to f X∪Y and f X∪Y f X respectively. Hence, in the measure space and Boolean lattice model, an association rule can be identified by a tuple (X, Z) such that X, Z ∈ I ⊂ B and Z =⇒ X. In this representation, if

X = i∈A a i then Z = i∈B a i with A ⊂ B and Y = i∈B\A a i so that Z = X ∧ Y .
This means that if we consider three distinct itemsets X, Y, Z ∈ B such that Z =⇒ Y =⇒ X then the tuples (X, Y ), (Y, Z) and (X, Z) correspond to three different association rules, but the information on the measure given by (X, Z) is contained in the information on the measure given by (X, Y ) and (Y, Z).

More generally, a rule X → Y between any two patterns, characterized by one or several interestingness measures defined on the values for the frequencies in the contingency tables for X and Y , corresponds to a triple (X, Y, Z), if the rules are defined by the relative frequencies f X , f Y and f X∧Y , or a quadruple (1, X, Y, Z), if the rules are defined by the absolute frequencies n, n X , n Y and n X∧Y , where Z = X ∧ Y in both cases. Hence, association rules, and any other type of rule defined on contingency tables even more so, correspond to naturally redundant families of patterns with regards to the definition of a measure.

This brings further insight on the issue discussed in section 3.5. Indeed, either considering rules leads to redundancy or, if there is no redundancy, it means the model is necessarily inconsistent. As meaningfulness cannot be justified in an inconsistent model, an additional modeling layer for objectively interesting rules would be at best superfluous. Therefore, one simple question is to ask whether rule mining should be included in a meaningful frequencybased objectively interesting pattern mining process. Based on Occam's razor, our simple answer is no. Focus has been set on rule mining mainly because rules are more interpretable by humans. This is a highly subjective criteria and, though it might be very useful in a number of contexts, it does not apply to the specific goal we have defined. In any case, if rule type patterns are required, a mining process can be defined to mine for patterns within a complete and sound family of implications in the Boolean lattice such as R.

Recommendation

In this section, we have addressed the issue of the mathematical modeling of the patterns within the general mathematical modeling process used in objective frequency-based interesting pattern mining. From our analysis, we strongly recommend that the set of minable patterns be modeled by a measure space in which the measurable sets are identified with the elements of a Boolean lattice. Furthermore, we recommend that the mining process be restricted to a complete and sound family of patterns, such as itemsets which also constitute a reasonable option in other regards, and we disqualify the mining of rules defined as particular tuples of patterns9 .

Modeling objectivity

In this section, we address one last aspect of mathematical modeling for objective frequency-based interesting pattern mining: the issue of including the notion of objectivity in the general modeling process. Though some may argue that interestingness is inherently subjective, we defend the idea that the aim of science is precisely to objectively provide for interesting information.

Objectivity in science is obtained by conforming to the process known as the scientific method. Though there is much debate on the exact definition of the scientific method and to what extent it allows to reach scientific objectivity, we will set these issues aside in this thesis and focus mainly on one description of the scientific method: the hypothetico-deductive model. Hence, our question here is whether and how it is possible to include a modeling of scientific objectivity and, specifically, the hypothetico-deductive model within the general mathematical modeling which we use to give meaning to a pattern mining process.

The scientific method as described by the hypothetico-deductive model is seen as a dynamic process involving a number of different steps. While the number and the nature of these steps vary from author to author, most agree on a fundamental basis for the hypothetico-deductive model consisting in the four following steps as illustrated in Figure 3 Observation of the world leads to formulating hypotheses about the world.

We use these hypotheses to predict aspects of the world. The hypotheses are evaluated by comparing the predictions to new empirical observations. If the evaluation is positive (the hypotheses have not been falsified), we can continue evaluating them. Otherwise, we try to formulate new hypotheses based on further empirical observations. The hypothetico-deductive model is usually understood as a never-ending dynamic process because hypotheses may never Our aim is to include this representation of the scientific method into a mathematical modeling process for pattern mining. In a sense, this brings us back to the very beginning of itemset mining which can be seen to have originated in Prague in the 1960s with the GUHA method [HHC66, HH12]. Indeed, GUHA stands for General Unary Hypotheses Automaton and it was clearly intended as a tool for assisting researchers in the hypotheses formulation and evaluation steps of the hypothetico-deductive model. However, the GUHA method is essentially based on local models and, as such, suffers from a number of modeling issues which are described in this chapter (see section 3.5).

In the following, we will present all the issues regarding the modeling of the hypothetico-deductive model which we have identified, as well as corresponding solutions.

A static finite model for a dynamic never-ending process?

One of the first issues which arises when considering the modeling of the hypothetico-deductive model is that it is described as a dynamic never-ending process while the data which we consider is static and finite.

In the hypothetico-model, it is often argued that the empirical observations which are used to formulate hypotheses should not be the same as those which are used to evaluate those hypotheses. Indeed, if one formulates hypotheses on the basis of an observation and uses that same observation to evaluate these hypotheses, what is to say that the formulation of the hypotheses does not depend on the same aspects of the observation as the evaluation step? This issue is, in a sense, akin to the overfitting bias. As such, many of the standard methods that are used to prevent overfitting, particularly in supervised learning, could be envisaged in this case. Indeed, machine learning methods in supervised learning will often dissociate training data from validation data in a dataset, which corresponds in our case to dissociating the empirical observations on which the formulation step is based from those on which the evaluation step is based. However, this only allows to consider one cycle of the hypothetico-deductive model while the process involves many cycles.

One option is to conclude that this is simply the wrong type of data for modeling such a process and that we should use data streams (as humans do with their senses). We do acknowledge that this is a serious theoretical issue and that it is indeed impossible to consider a data mining modeling based on finite data for the hypothetico-deductive model strictly speaking. More precisely, it is impossible if the variables considered in the process (i.e. the items in itemset mining) have not been previously designated, which is the case if we consider the scientific method to produce science in general. Indeed, if the hypotheses formulated are rejected after evaluation, then the new hypotheses defined may characterize entirely new variables which had not necessarily been observed before. In fact, the formulation step involves both the definition of a set of variables and the definition of the hypotheses about these variables. As it is not possible to observe every possible variable simultaneously (no system is all-perceiving) and that we cannot know in advance which variables should be chosen (because choosing the variables is part of the scientific method itself), an objective method for designating the variables to observe between different observations must be elaborated if we hope to conceive a strong objective artificial intelligence (i.e. an artificial scientist). Though such an endeavor is eminently interesting it goes far beyond the scope of this thesis and we set it aside for now to concentrate on a more achievable goal. This leaves us with the following question, is it possible to include the hypothetico-deductive model in a mathematical modeling based on a finite dataset when the variables considered in the process are previously designated? This last condition corresponds to the idea that we are constraining the scientific method to a certain aspect of the world which is defined and described by the variables in the data. To this question, we answer by the positive and we shall detail this throughout this section. Intuitively, this seems quite normal.

Indeed, consider all the data that has been used in scientific research up to this day, it is a large yet finite amount of data and it is defined on a large yet finite number of variables. Now, consider this data as a single dataset defined on a given set of variables. It should be possible for a system based on a mathematical modeling to derive scientific knowledge from this dataset because we, as humans, have managed to do so.

Note also that, even though we acknowledge the idea that the production of science is a never-ending process, this does not imply that scientific knowledge may not be produced in a finite amount of time and, hence, with finite data (which is exactly what scientists do). In fact, this idea implies rather that scientific knowledge is produced with a certain degree of confidence 10 .

Hypotheses are considered as scientific knowledge if we are sufficiently confident that they are correct. Further data may increase our confidence in the hypotheses (or falsify the hypotheses altogether), but we can never reach full confidence with a finite amount of data.

Throughout the rest of this section, we will consider the classical statistical model for the data in which the data was generated by a random sampling of n independent identically distributed random variables following an unknown distribution p = (p i ) 0≤i≤d . The aim is to formulate hypotheses about this distribution and evaluate them, in order to extract those in which we are confident (akin to scientific knowledge about the distribution). Our modeling represents a self-contained process, in the sense that no knowledge is given outside from the data. As such, the hypotheses which we will consider are necessarily data-driven data models. In order to be able to produce hypotheses in which we might be confident, we must first define some conditions on the data. We address this issue in section 3.7.2. We then specify the type of hypotheses which may be formulated in section 3.7.3 and present means to evaluate them in section 3.7.4. We then discuss the limits of our approach before concluding with some recommendations in section 3.7.5. 10 We purposefully avoid the use of the term probability here.

Prerequisites to considering data-driven data models

Consider the following dataset in Table 3.4 which has been obtained by throwing two dice ten times and recording whenever one of the dice landed on a figure equal to five or six. As we have knowledge about how the world works, a 1 a 2 0 0 1 0 1 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 Table 3.4: Example dataset and particularly about how dice work, we will consider the distribution in which p a 1 = 1 /6, p a 2 = 1 /6 and the two tosses are considered independent, regardless of the data. Moreover, the hypothesis that the data follows such a distribution would not be rejected if we were to conduct a statistical test for this hypothesis.

However, if we have no knowledge about anything else than the dataset itself, let alone about dice, then we have no reason to consider this distribution at all. The distributions considered will be data-driven. If we test for independence between the two variables for example, we will consider the hypothesis based on the distribution in which p a 1 = f a 1 = 0.3, p a 2 = f a 2 = 0.2 and p a 1 ∧a 2 = p a 1 p a 2 = 0.06 while considering that it has one degree of freedom. Similarly, this hypothesis would likely not be rejected by a statistical test. Implicitly, it means however that we accept the constraints given by the values f a 1 and f a 2 in the empirical distribution as reasonable and this is not necessarily justified. In fact, this entirely depends on the size of the dataset and we see, in this example that we are considering data driven hypotheses that are quite far from the distribution which was used to generate the data.

If we do not have enough data to be certain that the empirical distribution is close enough to the unknown theoretical distribution which we are trying to describe, we cannot justify the use of data-driven models. This means that we must be able to say that we do not have enough data to suggest any knowledge of scientific value concerning the theoretical distribution. In this respect, we depart significantly from the pattern mining as compression paradigm for which our approach has otherwise a number of similarities. Indeed, mining as compression approaches compress the empirical distribution regardless of the size of the dataset which defines the distribution.

It remains then to define what it means to be certain that the empirical distribution is close enough to the theoretical distribution. We suggest that we can equate this to the following condition: any hypothesis stating that the data was generated following a distribution which is not close to the empirical distribution would be rejected if a statistical test was to be performed. We present a formalization of this condition and discuss its properties in sections 3.7.2.1 and 3.7.2.2.

Confidence in the empirical distribution

The notion of confidence in the empirical distribution which we wish to define is based on the following idea. If we know that any potential hypothesis, in which we consider that the data was generated by a probability distribution that is far away from the empirical distribution, would be rejected by a statistical test, then we know that the only hypotheses which would not be rejected correspond to probability distributions which are close to the empirical distribution. Hence, we can say that we are confident that the empirical distribution is a good approximation of the theoretical distribution. In order to give a formal definition of this notion, we must start by specifying the set of potential probability distributions which can be considered and define a distance among these probability distributions. In the following, we consider an empirical distribution f = (f i ) 0≤i≤d on {0, 1} m , corresponding to a dataset D of n transactions (where d = 2 m -1).

Potential distributions.

Regarding the set of potential probability distributions, a first idea would be to consider all probability distributions that could have generated the dataset D. This is the set S f of probability distributions p = (p i ) 0≤i≤d on {0, 1} m such that p i > 0 whenever f i > 0:

S f = p = (p i ) 0≤i≤d ∀i ∈ 0, d , f i > 0 =⇒ p i > 0
We may also consider including information provided by background knowledge (i.e. not inferred from the dataset D) about a more specific class of potential probability distributions C ⊂ S f . This could be the case if categorical or ordinal variables are considered as described in section 3.6.2.4. However, as no statistical test can be performed for a hypothesis that a probability is null for a given event, we must necessarily assume that the class of distributions C only contains probability distributions such that p i = 0 only when f i = 0. This means that C must be contained in the set Z f defined by:

Z f = p = (p i ) 0≤i≤d ∀i ∈ 0, d , f i > 0 ⇐⇒ p i > 0
Note that, if we may infer from the data that the theoretical distribution p satisfies p i > 0 whenever f i > 0, we may not infer the converse from the data and, therefore, this must correspond to background knowledge. In other words, if f i = 0 for some i ∈ 0, d , we must be able to infer this from background knowledge if we wish to proceed rigorously with the process which we describe here.

Distance. Considering now a set of potential distributions C, we wish to define a distance between the elements of this set. A simple choice would be to consider the Chebyshev distance defined, for two distributions p and q, by:

δ Cheb (p, q) = max 0≤i≤d |p i -q i |
However, this choice fails to take into account the fact that p and q are taken in C. Hence, we prefer the normalized Chebyshev distance defined by:

δ c (p, q) = max 0≤i≤d c i =0 |p i -q i | c i
where c is the probability distribution with maximal entropy in C. Indeed, c holds a centered position within C (see previous section 3.3.3 and further developments in chapter 4) so that the distance between two elements in C

given by δ c expresses the skewness of the distribution of the elements within C. Note that, in the case in which no background knowledge is assumed (which is the case that we will study more thoroughly), c is the uniform distribution given by c i = 1 2 m for all i ∈ 0, d so that δ c is equal to the standard Chebyshev distance multiplied by 2 m . In this case, we write δ rather than δ c .

Statistical test. For all p ∈ C, let H p be the hypothesis that the dataset D was generated by p and:

χ 2 p,f = n 0≤i≤d c i =0 (f i -p i ) 2 p i the corresponding χ 2 statistic.
The hypothesis H p is rejected by the χ 2 test of goodness of fit for the threshold α if:

χ 2 p,f > χ 2 α (d c )
where

d c + 1 = |{i ∈ 0, d | c i > 0}| is the number of non-zero values c i of c (which correspond also to the non-zero values f i of f ), χ 2 α (d c ) is the value such that Prob (Z > χ 2 α (d c )) = α if Z ∼ χ 2 (d c
) and α is a fixed probability threshold (typically .95 or .99).

Confidence. Rejecting the hypothesis means that it is considered unreasonable to assume that the data was generated by the corresponding distribution.

Conversely, the hypothesis cannot be rejected if it is not highly unlikely that the data was generated by p. As we want to consider the empirical distribution as an approximation for the theoretical distribution, we cannot accept a situation in which the dataset can reasonably be considered to be generated by a distribution which is too far away from the empirical distribution. This gives rise to the following definition which means that we are confident in the empirical distribution if any hypothesis which is not close to the distribution is rejected by a χ 2 test of goodness of fit. Definition 3.7.1. We say that we are confident in the empirical distribution f to the precision ε and to the degree given by the probability threshold α if, for all distributions p ∈ C, we have:

δ c (p, f ) > ε =⇒ χ 2 p,f > χ 2 α (d c )
Precise values of χ 2 α (d c ) are known for low values of d c . However, if d c = d (which is the case if no background knowledge is considered), d increases exponentially with m and we use the normal approximation for the χ 2 distribution given by χ 2 (k)-k √ k d -→ N (0, 1) in order to approximate χ 2 α (d) for values of m larger than 5.11 This gives:

χ 2 α (d) ≈ d + N α √ 2d
with N .95 ≈ 1.645 and N .99 ≈ 2.33. Note that we obtain χ 2 α (d) ∼ 2 m so that the increase in m is asymptotically exponential.

How many transactions are needed?

We now try to determine bounds on the number of transactions n which allow us to be confident or not in the empirical distribution.

Background knowledge. For this section, we will consider only the specific case in which no background knowledge is assumed, that is d c = d, c i = 1 2 m for all i ∈ 0, d and δ c = δ is the standard Chebyshev distance multiplied by 2 m . Note that all of the results presented in the following paragraphs can easily be generalized when c describes a uniform distribution on its non null space (i.e. c i = 1 dc+1 for d c + 1 values of i in 0, d and c i = 0 otherwise). This corresponds to the case where the only background knowledge given is that some events are impossible. However, we do not cover more general types of background knowledge which would likely lead to much more complicated results.

Precision. We discuss, in this paragraph, the necessity to fix some bounds on the values of ε in order to preserve the meaningfulness of the process. For this, consider e such that f = p + e, which represents the error between a potential theoretical distribution p ∈ C and the empirical distribution f . If • is the norm associated to the distance δ (i.e. x = 2 m max 0≤i≤d |x i |), then:

δ(f , p) ≥ ε ⇐⇒ e ≥ ε
Now suppose that we allow all distributions p such that |e i | ≥ f i for some i ∈ 0, d , then we could have p i equal to zero and which would mean that we are considering distributions outside of C. Therefore, in order to ensure that we stay strictly within the interior of C, we must necessarily have max

0≤i≤d |e i | < min 0≤i≤d
f i or, equivalently, e < 2 m min 0≤i≤d f i . Hence, it would not be meaningful to consider any precision greater than 2 m min 0≤i≤d f i . In other words, we must have:

ε < 2 m min 0≤i≤d f i
Furthermore, it is also meaningless to consider a better precision for e i than that given by the minimal quantity of information in the empirical distribution corresponding to a single transaction which is 1 n . Hence, necessarily:

1 n ≤ ε 2 m < min 0≤i≤d f i
As the frequencies in the empirical distribution are necessarily whole fractions over n, this gives:

2 n ≤ min 0≤i≤d f i
Which implies that confidence in the precision of an empirical distribution cannot be estimated meaningfully if there are less than two transactions for each element in {0, 1} m , setting a first lower bound on the number of transactions:

2 m+1 ≤ n
Conversely, if n < 2 m+1 , a more restricted set of distributions C must necessarily be considered through some background knowledge (i.e. we must have c i = 0 for some i ∈ 0, d ) if we want to say that we are confident in the data.

Minimizing the χ 2 statistic. We now study the χ 2 statistic as a function on (0, 1) d+1 with values in R:

χ 2 •,f : (0, 1) d+1 -→ R p -→ χ 2 p,f
in order to determine bounds for n. Indeed, if we can determine for which value n α,f we have:

min δ(p,f )≥ε χ 2 p,f = χ 2 α (d),
then we can say that we can be confident in the empirical distribution f to the precision ε if and only if n ≥ n α,f .

Determining n α,f is not an easy task unless the empirical distribution f is extremely simple (as is the case for the uniform distribution studied later on).

However, we have determined lower and upper bounds for n α,f , as presented in the following paragraphs.

In order to bound n α,f , we start by noticing that, as:

χ 2 p,f = n d i=0 f 2 i p i -n
we have:

∂ ∂p i χ 2 p,f = -n f i p i 2 ∀i ∈ 0, d and: ∂ 2 ∂p i ∂p j χ 2 p,f = 0 if i = j 2nf 2 i p 3 i if i = j
Hence, χ 2 •,f is strictly convex positive on (0, 1) d+1 with its unique minimum reached for p = f . Therefore:

min δ(p,f )≥ε χ 2 p,f = min δ(p,f )=ε χ 2 p,f
and, more generally:

min δ(p,f )=ε χ 2 p,f ≥ min δ(p,f )=ε χ 2 p,f if ε ≥ ε
Lower bounds for n α,f . Now consider a distribution p and 0 < η 1 ≤ η 2 < 1 such that:

∀i ∈ 0, d , η 1 f i ≤ |p i -f i | ≤ η 2 f i
Then:

f i (1 -η 2 ) ≤ p i ≤ f i (1 -η 1 )
and:

η 2 1 1 -η 2 f i ≤ (p i -f i ) 2 p i ≤ η 2 2 1 -η 2 f i
Hence, by summation:

η 2 1 1 -η 2 ≤ 1 n χ 2 p,f ≤ η 2 2 1 -η 2
Therefore, we can say that that the

H p hypothesis is rejected if n ≥ 1-η 2 η 2 1 χ 2 α (d) and that is not rejected if n ≤ 1-η 2 η 2 2 χ 2 α (d).
Moreover, 2 m η 1 min

0≤i≤d f i ≤ e ≤ 2 m η 2 max 0≤i≤d f i .
Hence we can say that for

any precision ε ≤ 2 m η 1 min 0≤i≤d f i , we must at least have n ≥ 1-η 2 η 2 1 χ 2 α (d)
n α,f ≥ a α,f where a α,f = 2 m min 0≤i≤d f i 2 m min 0≤i≤d f i -ε ε 2 χ 2 α (d). (3.1)
Furthermore, we have established that 2 n ≤ min 0≤i≤d f i is also a necessary condition for confidence in f . Therefore:

η 1 = ε 2 m min 0≤i≤d f i ≤ nε 2 m+1
and, as x → 1-x

x 2 decreases on (0, 1):

1 -nε 2 m+1 nε 2 m+1 2 ≤ 1 -η 1 η 2 1 Hence: n ≥ 1 -nε 2 m+1 nε 2 m+1 2 χ 2 α (d)
is also a necessary condition for confidence in f which is not dependent on the distribution. This last inequality may be reduced to the following polynomial inequality:

n 3 + 2 m+1 ε χ 2 α (d)n - 2 m+1 ε 2 χ 2 α (d) ≥ 0
which is equivalent to:

g(x) = x 3 + px + q ≥ 0 with: p = 2 m+1 ε χ 2 α (d) and q = - 2 m+1 ε 2 χ 2 α (d)
This is a depressed cubic and, as we clearly have 4p 3 +27q 2 ≥ 0, it is equivalent to x ≥ a α where a α is the only real root of g given by:

a α = 3 - q 2 + q 2 4 + p 3 27 + 3 - q 2 - q 2 4 + p 3 27 (3.2)
The root a α is a lower bound for n α,f which is easily computed for small values of m.

For higher values of m, we use the approximation of the χ 2 distribution by a normal distribution described above which gives χ 2 α (d) ≈ d + N α √ 2d. This also allows us to determine the asymptotic behavior of a α when m increases which is given by:

a α ∼ γ a 2 m and γ a = 2 1 /3 ε 2 /3   3 1 + 1 + 2ε 27 + 3 1 -1 + 2ε 27   (3.3)
This last expression can be approximated for small values of ε by:

a α ∼ γ a 2 m and γ a ≈ 2 ε 2 /3 (3.4)
Upper bounds for n α,f . We have determined a lower bound a α,f for n α,f which is given by the expression 3.1. We have also determined a larger lower bound a α for n α,f whose expression, determined by the expression 3.2, does not depend on the distribution f and whose asymptotic behavior is approximately described by the expression 3.4. We will now give similar higher bounds for n α,f . Consider a distribution p such that δ(p, f ) = ε. Then max

0≤i≤d |p i -f i | = ε 2 m
and:

1 n χ 2 = d i=0 (p i -f i ) 2 p i ≥ 1 max 0≤i≤d p i ε 2 m 2 Hence, if: n max 0≤i≤d p i ε 2 m 2 ≥ χ 2 α (d) then H p is rejected. As max 0≤i≤d p i ≤ ε 2 m + max 0≤i≤d f i ≤ 1.
This gives an upper bound b α,f for n α,f which depends specifically on f :

n α,f ≤ b α,f where b α,f = 2 m ε 2 ε 2 m + max 0≤i≤d f i χ 2 α (d)
and a more general upper bound b α for n α,f which does not depend on f :

n α,f ≤ b α where b α = 2 m ε 1 + 2 m ε χ 2 α (d)
As χ 2 α (d) ∼ 2 m , we have the following asymptotic behavior for b α :

b α ∼ γ b 8 m and γ b = 1 ε 2
Note that both the lower and upper bounds, a α and b α , have asymptotic behaviors for large values of m which are constant in α.

The case of the uniform distribution. In the previous paragraphs, we have determined upper and lower bounds for n α,f . In the general case, we have:

a α ≤ n α,f ≤ b α
with the following asymptotic behaviors for the two bounds:

a α ∼ γ a 2 m and b α ∼ γ b 8 m 123
This gives quite a large interval for n α,f which can be reduced with the bounds a α,f and b α,f for a specific empirical distribution f .

In the case of a uniform distribution defined by f i = 1 2 m for all i ∈ 0, d , the specific bounds give the following interval for n α,f :

1 -ε ε 2 χ 2 α (d) ≤ n α,f ≤ 2 m 1 + ε ε 2 χ 2 α (d)
However, in this specific case, the exact value for n α,f can be determined.

Indeed, consider p such that δ(p, f ) = ε. We have max

0≤i≤d |p i -f i | = ε 2 m so we can consider i 0 such that |p i 0 -f i 0 | = ε 2 m . Hence: 1 n χ 2 p,f = d i=0 f 2 i p i -1 = 1 2 2m d i=0 1 p i -1 = 1 2 2m    1 p i 0 + d i=0 i =i 0 1 p i    -1
This is minimized for:

p i = 1 -p i 0 d for all i = i 0 Therefore, if p minimizes χ 2 p,f : 1 n χ 2 p,f = 1 2 2m 1 p i 0 + d 2 1 -p i 0 -1 = (1 -2 m p i 0 ) 2 2 m p i 0 (2 m -2 m p i 0 ) Furthermore, we have |p i 0 -f i 0 | = ε 2 m so that: 2 m p i 0 = 1 -ε or 2 m p i 0 = 1 + ε
This gives:

1 n χ 2 p,f = ε 2 (1 -ε) (d + ε) or 1 n χ 2 p,f = ε 2 (1 + ε) (d -ε) As: 1 (1 -ε) (d + ε) - 1 (1 + ε) (d -ε) = 2ε(d -1) (1 -ε 2 ) (d 2 -ε 2 ) > 0
we have:

min δ(p,f )=ε χ 2 p,f = nε 2 (1 + ε) (d -ε) Hence: n α,f = (1 + ε) (d -ε) ε 2 χ 2 α (d)
Note that, in this case, n α,f is relatively close to the upper bound b α,f as:

b α,f = (1 + ε) (d + 1) ε 2 χ 2 α (d)
and they have the same asymptotic behavior:

n α,f ∼ γ4 m where γ = 1 + ε ε 2
The limits of pure empirical science. Considering a precision ε = 0.001 and a degree of confidence α = 0.99 as in Figure 3.5 and assuming the empirical distribution corresponds to the uniform case (which can also be seen as the theoretical distribution associated to m independent coin tosses), the size of a dataset which is necessary to be confident in the empirical distribution is Moreover, as we have explained previously, such confidence is necessary for defining data-driven models. This leads to the following dilemma: either only consider small number of attributes which seriously limits the scope of scientific knowledge or accept that we cannot base all scientific knowledge purely on induction. The choice here is easily made. If we consider 100 coins being tossed, we want to be able to say that the mathematical model of 100 independent Bernoulli trials with parameters (n, 1 /2, 1 /2) is a good model for the data without having to toss coins roughly 1.6 × 10 66 times before we can consider any possible model. Scientific knowledge cannot only be inferred inductively.

Empirical distribution precision. In this section, we have mostly focused on determining the minimum number of transactions n α,f which is needed to be confident in a distribution f to the precision ε and the degree α. In many cases, the number of transactions will be a fixed parameter. Hence, it might be more interesting in these cases to determine the precision associated to a distribution f , a number of transactions n and a degree α. This is simply the value ε α,f ,n defined as the infimum value for ε such that we are confident in the empirical distribution f to the precision ε and the degree α.

Formulating hypotheses

In this section, we will assume that we are considering a dataset for which we are confident in the empirical distribution to the precision ε and the threshold α. Our aim is now to determine which hypotheses can be formulated from this empirical distribution. We will distinguish between two types of hypotheses: global and local. A global hypothesis corresponds to a probability distribution p on the Boolean lattice B described previously (see section 3.6) while a local hypothesis corresponds to the projection of such a distribution on a subset L of B.

Global hypotheses

In order to qualify as truly objective, the formulation of hypotheses must rely on two scientific principles: Newton's "Hypotheses non fingo" and Occam's razor.

"Hypotheses non fingo", which translates to "I do not feign hypotheses", is associated to the notion that hypotheses must be entirely based on the data and no more12 . Applying this principle, we determine that the only objective hypotheses which can be formulated on the probability distribution p is that it equates with the empirical distribution f at least partly. In other words, there is a subset K of the Boolean lattice B such that

p |K = f |K .
However, this is generally not sufficient in order to define a probability distribution on B and we must therefore rely on Occam's razor to determine p. Indeed, this scientific principle supports that, if a choice must be made between various models which are equivalent in terms of results (in this context, possible probability distributions p such that p |K = f |K ), then the one based on the fewest assumptions should be considered. In the case of a constrained probability distribution as described above, Occam's razor translates to the principle of maximum entropy (see [Jay82, Jay03, CK11] as well as section 3.3.3 and chapter 4). This means that we consider the distribution corresponding to the MaxEnt model where entropy is maximized among all distributions p satisfying the constraint p |K = f |K .

Note that there are two aspects in the hypotheses which we have defined:

one approximate and one exact. Indeed, the first aspect of such a hypothesis is given by the equation p |K = f |K which is the approximation of the underlying distribution by the empirical distribution on K. The empirical distribution could correspond exactly to the underlying distribution but this is highly unlikely and, in some cases, even impossible (if the underlying distribution takes irrational values on K). The fact that we are confident in the empirical distribution ensures, however, that this approximation is close enough. The second aspect of the hypothesis consists in using the principle of maximum entropy to define p entirely and therefore implies that p can be naturally derived from its values on K. In other words, the interactions between the attributes observed can be entirely described by those existing within K which is why we have described this property as the mutual independence of the attributes constrained by K (see chapter 4). This aspect of the hypothesis is exact in the sense that such statements are discrete and finite and at least one such statement is true about the underlying distribution. Hence, even though no such hypothesis can describe exactly an underlying distribution for which, for example, p a = π 4 , p b = 1 2 and p a∧b = π 8 , we can consider a hypothesis which describes exactly the fact that a and b are independent.

Local hypotheses

Given that the Boolean lattice B is generated by 2 m elements where m is the number of items considered, it may be difficult or even practically impossible to define a distribution p entirely on B. Therefore, one might have to resort to local hypotheses rather than global hypotheses. Let L ⊂ B such that L is still naturally isomorphic to a measurable space through the Boolean structure.

Then we can define a local hypothesis on L for any subset K ⊂ L ⊂ B by considering a distribution p |L on L such that p |K = f |K and defined on the rest of L through the principle of maximum entropy.

Considering local hypotheses may lead to a number of various issues some of which we have already described. Indeed, when simultaneously considering multiple local hypotheses, one must be aware that they might be mutually inconsistent (see section 3.5 and particularly section 3.5.1.2). While this is not an issue if the hypotheses are considered to be concurrent, this is problematic if we want such hypotheses to jointly describe different local aspects of the single underlying probability distribution. Even when considering globally consistent local hypotheses, issues may arise in the evaluation step of the process as we will describe in section 3.7.4.

Selecting hypotheses

While we have identified what type of hypotheses can be formulated, we have not specified which of these hypotheses should be selected for evaluation. In the classical hypothetico-deductive model, individual hypotheses are formu-lated and evaluated through new empirical observations. However, as we are considering a static dataset, a slightly different approach must be adopted.

The formulation step in the hypothetico-deductive model is arguably the less transparent step in current scientific processes and is often associated to the notion of intuition. This simply shows that, although we believe hypotheses are formulated based on prior knowledge, we have little understanding of the exact processes which lead to their formulation. This explains why a hypothesis should normally not be tested based on the same empirical observations which led to its formulation: if we do not know exactly why a hypothesis was formulated we might end up validating a hypothesis based on the reasons which led to its definition (which comes down to the issue of overfitting as we have previously explained). As we are considering a single static dataset, we must adopt a different strategy in order to ensure that this does not occur.

One solution is to define the hypotheses which we want to evaluate through this dataset without consideration for the dataset.

At first, this may seem to be in contradiction with our suggestion to consider hypotheses based on the empirical distribution as described in sections 3.7.3.1 and 3.7.3.2. However, considering a dataset for which we are confident in the empirical distribution allows to avoid this issue. Indeed, this ensures that the equality p |K = f |K for a given hypothesis is the only reasonable assumption (given the precision ε and the threshold α) and therefore not relevant towards the definition of the hypothesis. In a sense, considering a dataset for which we are confident reduces each hypothesis to its exact aspect (the mutual constrained independence of the attributes) as described in section 3.7.3.1. As this aspect of the hypothesis depends only on the choice of K, we can select the hypotheses to evaluate based on K.

Following the recommendations in section 3.6, we can first choose to restrict the selection of hypotheses by considering only subsets within a sound and complete family of patterns (such as the set of itemsets). Given such a restriction, we must still define which hypotheses to evaluate. An initial idea is to evaluate them all and consider the hypothesis which evaluates best. This is the same principle as the one used in strategies within the mining as compression paradigm. However, there is no theoretical guarantee in this case that overfitting is avoided. Another option is to define a total order on subsets of the family considered. Such an order should be associated to a notion of complexity so that the simplest set of patterns come first (this is not necessarily a trivial issue as described in section 3.6.2.3). Hypotheses may then be evaluated, following the order previously defined, until a hypothesis is tested positively. This last hypothesis is the least complex hypothesis which provides a reasonable explanation for the dataset and represents the scientific knowledge extracted from the dataset. This again is a direct transcription of Occam's razor.

Evaluating hypotheses

We consider, in this section, that the hypotheses which we mean to evaluate correspond to MaxEnt models as described in section 3.7.3. We will focus first on the issue of evaluating hypotheses based on global MaxEnt models (as described in section 3.7.3.1) before addressing some issues with the evaluation of local MaxEnt models (as described in section 3.7.3.2).

Evaluating global hypotheses

Compression scores. In the literature, a few criteria have been suggested by the proponents of the mining as compression paradigm (see section 2.3.3.4) for the evaluation of MaxEnt models defined by the frequencies of a set of itemsets. These include the Bayesian Information Criterion (BIC) or a Minimum Description Length (MDL) score [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF][START_REF] Van Leeuwen | Mining and using sets of patterns through compression[END_REF]. As BIC may be seen as a simplified MDL score, we will concentrate on the latter in this paragraph.

Recall from section 2.3.3.4, that given the output of a program generated by a universal machine, there is a shortest possible program that generates the exact same output. The length of this program in bits is the Kolmogorov complexity of the output. Considering Kolmogorov complexity as a score for evaluating the best possible compression for a dataset would be solidly grounded in theory. However, Kolmogorov complexity has been shown to be uncomputable in general. Hence, computable substitutes, such as the ones mentioned above, have been considered instead. Note that, when considering such substitutes, a class of models must be defined a priori, together with a specific language to describe the models within this class. In the current context, the class of models considered is that of MaxEnt models defined by the frequencies of a set of itemsets.

In practice, two-part MDL scores are considered. They are computed as:

L(D, M ) = L(M ) + L(D|M )
where L(M ) is the length of the description of the model M in the specific language and L(D|M ) is the length of the description of the deviation of the data D with regards to the model. Such a score does not take into account the complexity of the task of generating the dataset from the model (i.e. the decompression task) and focuses instead on the complexity of the description of the model in the specific language. While [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF] justifies this simply by saying that the aim of the process is "to summarize the data with a succinct set of itemsets, not model it with a distribution", the authors of [START_REF] Van Leeuwen | Mining and using sets of patterns through compression[END_REF] go further by stating that the complexity of the decompression algorithm is constant 13 . We entirely disagree with this statement as the complexity of the task of computing a MaxEnt model varies, in fact, significantly depending on the set of itemsets whose frequencies define the model. In practice, the variation is such that the search is limited to a subclass of MaxEnt models which can be reasonably computed (see [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF] and chapter 4 on these issues). This is an important theoretical issue because it undermines the theoretical foundations of the approach. Indeed, if the complexity of the task of generating the model from its description is irrelevant, what justifies the use of a general theoretical model in which it is not? In this respect, an approach based on statistical testing makes more sense as it disregards the issue of computing the model entirely.

Statistical testing. Statistical testing, which is the standard method for evaluating a hypothesis based on a data model, appears to be the most relevant option for evaluating a hypothesis in the current context. As any given hypothesis which we consider here defines a global model for the data, we should consider a statistical test which tests the model globally, such as a χ 2 test of adequacy. However, as the hypothesis considered corresponds to a description of the model given by the empirical frequencies of a set of itemsets 13 The article states precisely while referring to the differences between two-part MDL and Kolmogorov complexity that "One important difference is that L(D, M ) happily ignores the length of the decompression algorithm-which would be needed to reconstruct the data given the compressed representation of the model and data. The reason is simple: its length is constant, and hence does not influence the selection of the best model ".

or, in other words, the mutual independence of the items given a set of constraints, the number of degrees of freedom of the χ 2 test depends on the set of constraints (see proposition 5.1.1). Now, considering that χ 2 tests are used to evaluate each individual hypothesis, we must still determine a means to discriminate between two concurrent hypotheses which both pass such a test. A seemingly simple solution would be to consider the χ 2 test as a score and look the hypothesis which scores best.

However, such an approach conflicts with the principle of statistical testing itself, which should only be used to reject hypotheses. A more meaningful approach is to consider the simplest model which passes the test. Such an approach implies that a total order relation between all potential hypotheses, corresponding to a notion of relative complexity, be defined. As discussed previously in section 3.7.3.3, in order for the process to remain objective, the choice of such an order must be defined a priori. We set aside the issue of determining the most appropriate order for such hypotheses for the moment. Note, however, that such an order should be determined by objective factors (such as the number of itemsets considered or the total number of items within the itemsets considered) but must also necessarily rely on arbitrary factors (such as a lexicographic order between itemsets of same size) if we want to define a total order between hypotheses.

Evaluating local hypotheses

In section 3.7.3.2, we describe the possibility of defining local hypotheses rather than global ones. While such an approach is theoretically suboptimal, it can be necessary in practice if defining global MaxEnt models is technically infeasible.

The first issue to consider is the issue of global consistency of local, nonconcurrent, hypotheses (see section 3.5.1.2). This is a highly complex topic in itself, which can be related to the study of junction trees (see, for example, [CL68, WP06a, KS10, SK12]), and a thorough analysis of this issue is well beyond the scope of this thesis. However, one simple method for dealing with this issue is to consider a partition of the set of items into blocks of items, so that any local model considered is defined on one of the local Boolean lattices L generated by the items within a given block. The issue of defining such a partition is quite tricky itself, at least if we wish to preserve meaningfulness throughout the process, but could eventually be justified by the knowledge (or simple assumption) that the itemsets in different blocks do not truly interact with each other (either because the itemsets within separate blocks are independent, or because they are incompatible, or a combination of both depending on the blocks considered). We discuss the possibility of such an approach in section 5.3.

The second issue to consider is one related to multiple testing. Indeed, when performing tests on a large number of local models, each corresponding to a different projection of a global model, false positives are known to appear [LTP06, Han11, KIA + 17]. Once again, the study of this issue is far beyond the scope of this thesis and we do not provide any solution which allows to tackle it. However, we do believe that considering partition models as described above would, without solving the issue altogether, allow to apprehend it in simpler terms.

Recommendation

Throughout this section, we have defended the idea that we can use the hypothetico-deductive model within a general mathematical modeling process in order to meaningfully model the objectivity in objective frequency-based interesting pattern mining. The model we have proposed allows to define and evaluate hypotheses based on this principle.

Because we are dealing with static and finite data, we have argued that such a model must include a means of determining whether or not the number of transactions n in the data is sufficient to meaningfully and objectively define data-driven hypotheses. This led us to the definition of the notion of confidence in the empirical dataset. More generally, we recommend that such an indicator be used in pattern mining processes, to evaluate the confidence which can be expected in the scienticity of the information extracted in such processes. Furthermore, we recommend that the hypotheses considered should be formulated as MaxEnt models associated to constraints defined by the empirical frequencies of a set of patterns in the dataset.

Finally, we support the idea that the patterns which are to be extracted in a meaningful objective frequency-based interesting pattern mining should correspond to the simplest possible hypothesis that passes a statistical test.

We fully acknowledge that a rigorous implementation of the model which we have described is practically quite difficult, either because we do not have sufficient data or because we do not have enough resources to entirely compute the hypotheses considered. However, it is important to understand the limits of our model in order to determine why and how compromises with such a rigorous approach can be made. In fact, these limits also provide information about some of the theoretical boundaries that surround empirical science. Hoping to be able to extract objective scientific information about the mutual interactions of a hundred items, based solely on the frequencies of their associations in a finite dataset is, as we have shown, completely unreasonable. This gives food for thought as a hundred items is most generally considered small data in the current data mining community. It also allows us to reflect on the manner one should go about building an artificial scientist because it means such an artificial intelligence should only consider datasets on a large number of items if it has developed some form of prior knowledge about the relationships between these items.

Conclusion

The meaningfulness of any process based on mathematical models, as well as the meaningfulness of the output of such a process, stems directly from the meaningfulness of the mathematical modeling process on which it relies. This is why it is important to make explicit such modeling processes and understand how choices in terms of modeling affect and determine the meaningfulness of the general process.

While focusing on the specific case of mathematical modeling for objective frequency-based interesting pattern mining processes, we have exhibited general issues related to mathematical modelings and, as such, much of the research presented in this chapter can be seen as a contribution in the field of the philosophy of science. This includes our definitions of the notions of model and modeling, the notions of phenotypic and genotypic modeling, the notion of pragmatic modeling, the notions of patchwork and holistic modeling, and our approach towards the modeling of the scientific method.

Concomitantly, we have determined a number of principles for meaningful mathematical modeling in the specific case of objective frequency-based interesting pattern mining processes which are summarized at the end of each section of this chapter. The definitions of our own pattern mining processes CHAPTER 4

Mutual constrained independence models

In the previous chapter, we have listed a number of recommendations for a meaningful modeling of the objective interestingness of itemsets. As we have presented, MaxEnt models are an essential tool for such a modeling because they provide for an objective answer to the issue of defining a probability measure given a set of constraints on that measure (see section 3.7.3). This statement can be supported by a number of various approaches towards the definition of MaxEnt models, the two most notable ones being Claude E. Shannon's original presentation of information entropy and E. T. Jayne's approach (see section 3.3.3 for more detail). In the course of this doctoral thesis, we have added a third approach towards the definition of MaxEnt models in the context of itemset mining: mutual constrained independence (MCI). We will start this chapter by defining this notion and proving the mathematical properties on which it is founded. We will then present some of the properties related to MCI models and exhibit their link to MaxEnt models. Finally, we will provide novel methods for computing these models.

Theoretical foundations of MCI

Preliminaries

Notations

Let A = {a 1 , ..., a m } be a set of m items. We will consider the following notations from section 3.6:

• B is the Boolean lattice associated to A;

• is the top element of B corresponding to the empty itemset;

• d = 2 m -1;

• Ω = (ω i ) 0≤i≤d ⊂ B is the set of minimal generators of B ordered by the natural lexicographic order as described in section 3.6.2.5;

• I = (I i ) 0≤i≤d ⊂ B
is the set of all itemsets ordered by the natural lexicographic order described in the section mentioned above;

• for any probability measure p defined on B, we write p i in place of p(ω i ), for all i ∈ 0, d , and p X in place of p(X), for all X ∈ B \ Ω.

Transfer matrix

Before we make explicit how we aim to objectively define a probability measure on B from given constraints, we introduce a mathematical object which is quite useful for defining such probability measures: the transfer matrix from Ω to I. Recall that any measure on B is defined naturally by its values on the minimal generators Ω of B (which correspond to the generalized itemsets of size m). Furthermore, a measure can also be entirely defined by its values on the itemsets I, as we have described in section 3.6. As such, these families of patterns can be seen as bases for representing probability measures.

Switching from one representation to the other is easily accomplished using a transfer matrix. Indeed, consider the binary matrix T of size 2 m × 2 m such that T k,l = 1 if and only if (ω l =⇒ I k ). It results from the properties of a measure that, for any measure g on B, we have the following equality:

T X g =     g I 0 . . . g I d     where X g =     g 0 . . . g d    
The values for the coordinates T k,l of the matrix T can be computed directly from the indices k and l. To do this, we note that k and l can both naturally be represented by binary vectors k = (k 1 , ..., k m ) and l = (l 1 , ..., l m ) to which we associate them. The coordinates of the matrix T are then given by the Generalized itemset

X ∈ Ω a 1 a 2 a 3 ω 0 = ¬a 1 ∧ ¬a 2 ∧ ¬a 3 a 1 a 2 a 3 ω 1 = ¬a 1 ∧ ¬a 2 ∧ a 3 a 1 a 2 a 3 ω 2 = ¬a 1 ∧ a 2 ∧ ¬a 3 a 1 a 2 a 3 ω 3 = ¬a 1 ∧ a 2 ∧ a 3 a 1 a 2 a 3 ω 4 = a 1 ∧ ¬a 2 ∧ ¬a 3 a 1 a 2 a 3 ω 5 = a 1 ∧ ¬a 2 ∧ a 3 a 1 a 2 a 3 ω 6 = a 1 ∧ a 2 ∧ ¬a 3 a 1 a 2 a 3 ω 7 = a 1 ∧ a 2 ∧ a 3 Itemset X ∈ I ∅ I 0 = a 3 I 1 = a 3 = ω 1 ∨ ω 3 ∨ ω 5 ∨ ω 7 a 2 I 2 = a 2 = ω 2 ∨ ω 3 ∨ ω 6 ∨ ω 7 a 2 a 3 I 3 = a 2 ∧ a 3 = ω 3 ∨ ω 7 a 1 I 4 = a 1 = ω 4 ∨ ω 5 ∨ ω 6 ∨ ω 7 a 1 a 3 I 5 = a 1 ∧ a 3 = ω 5 ∨ ω 7 a 1 a 2 I 6 = a 1 ∧ a 2 = ω 6 ∨ ω 7 a 1 a 2 a 3 I 7 = a 1 ∧ a 2 ∧ a 3 = ω 7
Table 4.1: Correspondence between elements in I and Ω for m = 3. following equation (where • is the dot product).

T k,l = 1 if (d -l) • k = 0 0 if (d -l) • k = 0 (4.1)
Furthermore, we can see that T is invertible and that the value for the coordinates T -1 k,l of its inverse are given by the following equation.

T -1 k,l = (-1) (l-k)•d if (d -l) • k = 0 0 if (d -l) • k = 0 (4.2)
Equation (4.1) is obtained quite directly from the definition of T . Indeed, we see that (ω l =⇒ I k ), if and only if, (∀i ∈ 1, m , k i = 1 =⇒ l i = 1), which is equivalent to the equation (d -l) • k = 0. Equation (4.2) can then be verified by multiplying both matrices. Indeed, let M be the matrix obtained by multiplying T with the matrix whose coordinates are defined by (4.2). The coordinates of M are given by:

M i,j = d k=0 (d-k)•i=0 (d-j)•k=0 (-1) (j-k)•d .
From (d -k) • i = 0, we get that if i has a coordinate equal to 1, then the coordinate with the same index in k is also equal to 1. From (d -j) • k = 0, we see that if j has a coordinate equal to 0, then the coordinate with the same index in k is also equal to 0. Hence, if i > j, M i,j = 0. Furthermore, if i = j, then necessarily k = i from which we get M i,j = 1. Finally, if i < j, then by grouping all the values of k for which k has the same number r = (j -k) • d, we get:

M i,j = (j-i)•d r=0 (j -i) • d r (-1) r = 0 .
Hence, M is equal to the identity matrix which proves the result. Notice also that T -1 has all its coordinates in {-1, 0, 1} which will be used in the proof of theorem 4.1.3.

T =            
1 1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 0 1 1 0 0 1 1 0 0 0 1 0 0 0 1 0 0 0 0 1 1 1 1 0 0 0 0 0 1 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 1

            , T -1 =             1 -1 -1 1 -1 1 1 -1 0 1 0 -1 0 -1 0 1 0 0 1 -1 0 0 -1 1 0 0 0 1 0 0 0 -1 0 0 0 0 1 -1 -1 1 0 0 0 0 0 1 0 -1 0 0 0 0 0 0 1 -1 0 0 0 0 0 0 0 1             Figure 4
.1: The transfer matrix T and its inverse T -1 for m = 3.

Problem statement

As we have described previously, our aim here is to define a probability measure on B given a number of constraints on itemsets. We formalize this aim through the following problem statement.

Let K ⊂ I be a set of itemsets and f |K be the restriction to K of a probability measure on B which corresponds to an empirical distribution in a dataset of transactions. In the following, we will refer to such a set of itemsets K as a constrained set, f |K as a constraint function and C = K, f |K as a constraint system on B. We say that a probability measure p on B satisfies the constraints given by the constraint system C, if its restriction to K is equal to f |K (i.e.

∀X ∈ K, p X = f X ). We consider the problem of objectively hypothesizing the values of a probability measure on B which satisfies a constraint system C.

In other words, we aim to define a probability measure p on B as a hypothesis for the value of f , as naturally and objectively as possible, based on the sole knowledge that is given about f by the constraint system C. Note that this problem statement is not a purely mathematical problem as the notion of objectivity is not a mathematical one per se. We must therefore model this notion in order to transform this into a purely mathematical problem.

Formulating objective hypotheses

Before we provide an answer to the problem statement described above, let us consider the wider issue of formulating hypotheses on mathematical objects based on partial knowledge of these objects. In section 3.7, we aimed at presenting a mathematical modeling for the discovery of scientific knowledge about the world from data. The modeling which is suggested in this previous section relies on the formulation of hypotheses based on knowledge acquired from the data in which we may be confident. However, in this specific step of hypothesis formulation, there is no intention to discover new knowledge from the data: the hypothesis must be formulated or, in other words, inferred based on knowledge already acquired from the data. More generally, for an intelligent system, we can differentiate between its ability to acquire knowledge from the world and its ability to reason based on its knowledge of the world. We will focus here on this second aspect.

Consider an intelligent system whose representation of the world is given by a mathematical model. The system has knowledge about the world stored in its memory from which it can directly infer further knowledge about the world using methods from mathematical reasoning. For example, if the system knows that a = 768, b = 453 and c = a × b as well as basic arithmetic, it will be able to answer that c = 347, 904 to the question "What is c equal to?". This answer is part of the scope of the knowledge of the system even though it is not necessarily part of the knowledge stored in its memory. In a sense, the fact that the scope of the knowledge of the system reaches beyond the knowledge stored in its memory is a defining characteristic of intelligence.

However, it would be quite limiting to consider that the scope of the knowledge of a system can only be reached through mathematical reasoning. Indeed, one might be interested in an exact numerical value as an answer to a question when pure mathematical reasoning may only provide an interval. For example, even if one does not know the exact age of the last person one has met, and cannot derive it exactly from one's knowledge, one can generally still provide an answer if asked to guess that person's age. In every day life, such an answer is called an educated guess and is based on the person's prior knowledge about people and ages and the world in general (even though the mechanisms that lead to its formulation are essentially a black box). Similarly, we can formalize the notion of an educated guess in the case of an intelligent system whose knowledge of the world is a mathematical model. As we do not include any form of black box in our formulation process, we will use the term objective hypothesis rather than educated guess.

In order to formulate such objective hypotheses, we rely on the principle of indifference (also referred to as the principle of insufficient reason). This principle states that, when confronted to a model in which different possibilities arise and no information allows to differentiate between any of them, then each possibility should be considered as equally likely. The system should therefore consider every possible interpretation of the world as equally likely thus defining a uniform distribution on the set of possible interpretations of the world (that is, if such a probability measure is definable on this set, which is always the case if the set is finite but not, a priori, the case if the set is infinite). In the case in which a value must be provided for a variable, such a uniform distribution induces a distribution on the set of possible values for this variable. We can then use this last distribution to define an objective hypothetical value for this variable.

In such a case, several approaches can be used to determine this hypothetical value. A first approach, based on information theory, considers the value which adds the least information to the system. A second approach considers the value with the highest likelihood (which is not necessarily possible if the distribution is not discrete). A third option, which is the one that we shall study in detail in this chapter, is to consider the expected value for this variable (which is possible only if the variable is numerical and the expected value is well defined). These three approaches correspond to the three approaches towards the definition of MaxEnt models described previously (Shannon's, Jayne's and our own respectively) and we shall show how they all relate to each other in the specific context studied here.

Note that we have not discussed the practical manner in which an intelligent system may compute such hypotheses. This is of course a consideration of the utmost importance, notably because the theoretical scope of the knowledge of an intelligent system, which corresponds to the notion we have described above, is not a priori equal to the practical scope of its knowledge, which comprises only the conclusions the system might reach within the limits of its resources. Therefore, a process resulting in the formulation of hypotheses must be defined and its complexity must be taken into consideration. In particular, a naive process which would consist in an exhaustive review of all the different interpretations of the world would be practically infeasible in general. Hence, more elaborate mathematical tools are necessary to compute hypotheses while bypassing the computation of the underlying uniform distribution.

Application to the problem statement

Let us now try to understand how the approaches described in section 4.1.1.4 can apply to the problem statement defined in section 4.1.1.3. The world is represented here as a dataset of transactions on items whose empirical distribution is described by a probability measure f on B. However, we only have partial knowledge about the world. The knowledge we have is represented by the restriction f |K of f to a set of itemsets K ⊂ I. In other words, our knowledge of the world is defined by the constrained system C = K, f |K . Our aim is to define a probability measure p on B which can be seen as an objective hypothesis about f based on the partial knowledge defined by C. Given our representation of the world, any interpretation of the world corresponds to a dataset whose empirical distribution h satisfies the constraints given by the constraint system C. As described in section 4.1.1.4, we would like to define p as the expected value for h given a uniform distribution on the set of all these datasets. However, this raises an issue as this set is infinite and there is no natural way to define a uniform distribution on it.

One first approach is to consider only datasets of a given size (i.e. the number of transactions n can therefore be seen as an additional constraint).

uniquely define 1 n µ as we will show in the third of the following three examples.

Particular constrained sets

Empty set. The first specific case which we consider is the case in which K = ∅ and, therefore, K = { }. This case is quite trivial and can be seen as the case in which there is only a constraint on the number of transactions.

By symmetry, we see that all µ i are equal. As there sum is equal to n, we get

µ i = n
2 m for all i ∈ 0, d . Hence, 1 n µ corresponds to the theoretical probability distribution for m random independent coin tosses.

Independence model. In this case, K = A = {a 1 , ..., a m }. This corresponds to the case in which the frequencies n a 1 , ..., n am corresponding to each item, as well as the total number of transactions n, are fixed constraints. Considering the natural representation of a dataset of n transactions on these m items as a binary matrix, we see that the constraints correspond to the column margins. As each constraint corresponds to an individual column, we see that the set of all n × m binary matrices satisfying the constraints has a natural one-to-one correspondence with the Cartesian product of the m sets of vector columns of size n corresponding to each individual constraint. Therefore, in this case, 1 n µ corresponds to the distribution given by the independence model.

All proper subitemsets. The last specific case we consider here is the case in which K contains all the proper subitemsets of a given itemset. This case was presented in [START_REF] Delacroix | Constrained independence for detecting interesting patterns[END_REF]. Without any loss for generality, we may limit our study to the case in which the itemset considered is

I d (recall that I d = m i=0 a i )
and hence K = I \ {I d }.

We suppose that we are considering measures h on B constrained so that, for all i ∈ 0, d -1 , h I i = n i , where the integers n i correspond to some empirical dataset (note that n 0 = n necessarily). Then, for all j ∈ 0, d , h j is determined entirely by the values n i together with one variable k such that

h I d = k.
More precisely, considering the transfer matrix T and its inverse as defined in section 4.1.1.2, we have:

    h 0 (k) . . . h d (k)     = T -1       n 0 . . . n d-1 k      
.

Furthermore, we know that the possible values for h I d correspond exactly to an interval l, u whose bounds are entirely defined by the constraints n i .

This result, which is presented by Calders and Goethals in their work on nonderivable itemsets [START_REF] Calders | Mining all non-derivable frequent itemsets[END_REF], can be rephrased using the transfer matrix. Indeed, recall that k ∈ l, u is equivalent to h i ≥ 0 for all i ∈ 0, d . Hence, if we write the previous equation as:

    h 0 (k) . . . h d (k)     = T -1       n 0 . . . n d-1 0       + T -1       0 . . . 0 k       = T -1       n 0 . . . n d-1 0       + k       (-1) (d-0)•d (-1) (d-1)•d . . . (-1) (d-d)•d      
, we can say that:

l = max i∈ 0,d (d-i)•d even (-c i ) and u = min i∈ 0,d (d-i)•d odd (c i ) where:     c 0 . . . c d     = T -1       n 0 . . . n d-1 0       .
We can therefore express µ(h I d ) through the following formula:

µ(h I d ) = u k=l k d i=0 h i (k)! u k=l 1 d i=0 h i (k)! (4.4)
which can be computed directly using T -1 . The value obtained allows in turn to determine h entirely.

For the case in which m = 3, equation (4.4) becomes:

µ(h I 7 ) = u k=l k (n 0 -n 1 -n 2 + n 3 -n 4 + n 5 + n 6 -k)!(n 1 -n 3 -n 5 + k)! (n 2 -n 3 -n 6 + k)!(n 3 -k)!(n 4 -n 5 -n 6 + k)!(n 5 -k)!(n 6 -k)!k! u k=l 1 (n 0 -n 1 -n 2 + n 3 -n 4 + n 5 + n 6 -k)!(n 1 -n 3 -n 5 + k)! (n 2 -n 3 -n 6 + k)!(n 3 -k)!(n 4 -n 5 -n 6 + k)!(n 5 -k)!(n 6 -k)!k!
where l = max(0, -n

1 + n 3 + n 5 , -n 2 + n 3 + n 6 , -n 4 + n 5 + n 6 ) and u = min(n 0 -n 1 -n 2 + n 3 -n 4 + n 5 + n 6 , n 3 , n 5 , n 6 ).
This last formula allows us to check that 1 n µ is not, in general, uniquely defined by This remark is important because it shows that the finite approach does not allow to define a hypothetical value for a probability distribution in general :

C n = K, 1 n g |K . Indeed,
the number of transactions must be defined. As such it does not provide for a generalization of the independence model (which can be defined regardless of the number of transactions) even though we do obtain the same model as the independence model when considering K = A.

Computing µ

Another one of the issues with the finite approach is the difficulty in computing the value of µ. Indeed, if we set aside some trivial cases such as the one corresponding to the independence model for which the formula simplifies easily, computing µ directly from equation 4.3 becomes practically infeasible as soon as n or m are too large. This is due to the combinatorial nature of this formula which contains many factorials. In fact, even in the particular case that we have described previously in which all proper subitemsets of an itemset are known (which can be considered an easy case because the number of liberties for h is equal to one), the formula cannot be reasonably computed if m ≥ 3 and n × m ≥ 10 3 . Therefore, other means for computing µ must be envisaged.

One alternative approach is to use randomization methods in order to determine an approximate value for µ. Such methods have been considered in itemset mining for a similar yet distinct problem (see [HOV + 09]) in which the randomization method simulates a uniform distribution on all datasets of a given size that share the same row and column margins as a given dataset as well as constraints on the values of some itemsets. Such methods can be slightly more scalable than a direct computation but the gain is still limited and, given the results on complexity in [HOV + 09], they cannot be reasonably computed if m ≥ 3 and n×m ≥ 10 6 . Furthermore, there is no reason to believe that removing the constraints on the row and column margins would help in this respect and more likely the opposite as the methods suggested are based on methods for randomly generating matrices based on their row and column margins. In any case, the size of the datasets that may be considered for such methods remain quite small compared to the Big Data considered in data mining processes and much too small with regards to the aim of discovering scientific knowledge in the data as presented in section 3.7.

Another means to approximate µ is through the MaxEnt model defined by the corresponding constraint system. Indeed, as we will show in section 4.1.3, 1 n µ converges towards the distribution given by this MaxEnt model and this limit may be used to approximate µ. As a matter of fact, it is the observation of this convergence on specific examples that led us first to the invention of the notion mutual constrained independence. Moreover, we will show that this value is arguably a more relevant theoretical choice than the measure µ which is tied to the number of transactions.

Asymptotic approach

MCI convergence theorem

The main principle behind the asymptotic approach is that, when considering finite constraints all corresponding to a same constraint system (or at least corresponding to a converging sequence of constraint systems), the sequence of probability distributions resulting from finite approaches converges towards a limit. This is formalized through the following mathematical result.

Theorem 4.1.1 (MCI convergence theorem). Given a constraint system C = K, f |K on B, there exists a unique probability measure p such that, for any sequence of functions g

(k) |K k∈N
, the three following conditions:

• ∀k ∈ N, g (k)
|K is the restriction to K of a measure on B with integer values;

• g (k) -→ k→+∞ +∞; • 1 g (k) g (k) |K -→ k→+∞ f |K ; imply that 1 g (k) µ (k) -→ k→+∞ p, where µ (k) is the measure defined by K, g (k)
|K as in section 4.1.2.

Model justification

Assuming the validity of theorem 4.1.1 (the proof of which is provided in section 4.1.3.3), we can consider p to represent the objective hypothesis regarding f given the knowledge provided by the constraint system C as described by the problem statement in section 4.1.1.3.

In comparison to the answer provided by the finite approach, this answer is more satisfying theoretically in several respects. Indeed, in many cases the transactions observed in a dataset are but a sample of a much larger, potentially infinite pool of transactions. This is notably the case if the aim is to use the observed dataset to extrapolate about other unobserved datasets and, in particular, if the data is seen as being generated by a random variable which we aim to describe. In such a case, a hypothesis on the distribution of this random variable is better defined through this asymptotic behavior. Note also that, as p is defined uniquely by the constraint system, this approach provides for a true generalization of the notion of independence as we will formalize with the definition of mutual constrained independence. On a practical note, as p is not determined by any given number of transactions, the complexity for computing this probability measure is not determined by the number of transactions in a dataset. This allows to consider truly big data, at least in terms of the number of transactions as the number m of items must still be taken into account.

As we will make explicit in section 4.1.5, the link between p and MaxEnt models further justifies the use of this asymptotic approach.

Proof of the convergence theorem

Our proof of theorem 4.1.1 is a constructive one which allows to characterize p. Hence, we will at the same time give the proof to a stronger version of this theorem. We start by setting up some notions which will be useful for the characterization of p.

Preliminary step 1: Reduced transfer matrix. Recall that the aim is to define the probability measure p from a constraint system C = K, f |K where K ⊂ I is a set of itemsets. In the following, we will use the matrix T to transfer this question around I towards Ω, where it is more easily answered.

We will then bring the problem back to I. For this purpose, we introduce the notion of reduced transfer matrix and constraint vector. Consider a constraint system C = K, f |K on B. We define the reduced transfer matrix T K to be the submatrix of T composed of the lines of T corresponding to the elements in K and the constraint vector K to be the column vector with coordinates equal to f I k for all I k ∈ K. Now, for any probability measure g, we see that g satisfies C if and only if T K X g = K.

Table 4.3 gives an example of a constraint system and its corresponding matrix equation for m = 3. The constraints are given here on three itemsets:

a 2 , a 3 and a 1 ∧ a 2 ∧ a 3 . X ∈ K f X 1 a 3 1 /2 a 2 1 /3 a 1 ∧ a 2 ∧ a 3 1 /5 ←→     1 1 1 1 1 1 1 1 0 1 0 1 0 1 0 1 0 0 1 1 0 0 1 1 0 0 0 0 0 0 0 1        g 0 . . . g 7    =     1 1 /2 1 /3 1 /5     Table 4
.3: A constraint system and its corresponding matrix equation.

As we will make explicit with theorem 4.1.3, the kernel of the reduced transfer matrix plays a significant role in obtaining the solution p to our problem.

We can notice here that we can obtain a basis B K of Ker(T K ) by considering the columns of T -1 which correspond to the lines removed from the matrix T .

Figure 4.2 gives the basis B K defined by the columns of T -1 for the constraint system given as an example in Table 4.3.

B K =                         1 -1 -1 1 0 0 0 0             ,             -1 0 0 0 1 0 0 0             ,             1 -1 0 0 -1 1 0 0             ,             1 0 -1 0 -1 0 1 0                         Figure 4.2: The basis B K of Ker(T K ) with T K as in Table 4.3.
Preliminary step 2: Largest derivable constraint system. In order to prevent issues related to boundary conditions, we distinguish between the information that can be obtained directly through mathematical properties from the rest, as described in section 4.1.1.4. This comes down to the same problem as distinguishing between derivable and non-derivable itemsets [START_REF] Calders | Mining all non-derivable frequent itemsets[END_REF].

For this purpose, we introduce the notions of derivable constraint system and largest derivable constraint system.

Definition 4.1.1. Let C = (K, f |K ) be a constraint system on B. A derivable constraint system of K is a constraint system C = (K , f |K ) such that the probability measures on B that satisfy C are exactly those that satisfy C .

Notice that, if C is a derivable constraint system of C, then we can define the union constraint system

C = (K , f |K ) by K = K ∪ K , f |K = f |K and f |K = f |K . Furthermore, C is a derivable constraint system of C. Therefore, we
can define a largest derivable constraint system (LDCS) C * = (K * , f * |K * ) of C by considering the union of C with all its derivable constraint systems.

We say that the LDCS is complete if C * = I and incomplete otherwise.

In terms of linear equations, a probability measure g satisfying C corresponds to a vector X g of [0, 1] 2 m such that T K X g = K. The set of all probability measures satisfying C is therefore the convex polytope of R 2 m defined as the intersection of the hypercube [0, 1] 2 m and the affine space of equations T K X = K. A constraint system C is a derivable constraint system of C if and only if the polytope defined as the intersection of the hypercube [0, 1] 2 m and the affine space of equations T K X = K is the same as the one for C. Hence, the largest derivable constraint system C * corresponds to the smallest affine space such that the intersection with the polytope of probability measures gives the same convex polytope as for C.

I ∈ K f I a 3 1 /2 a 2 1 /2 a 2 ∧ a 3 1 /6 a 1 1 /2 a 1 ∧ a 3 1 /6 a 1 ∧ a 2 1 /6 -→ I ∈ K * f * I 1 a 3 1 /2 a 2 1 /2 a 2 ∧ a 3 1 /6 a 1 1 /2 a 1 ∧ a 3 1 /6 a 1 ∧ a 2 1 /6 a 1 ∧ a 2 ∧ a 3 0 Table 4.4: A complete LDCS I ∈ K f I 1 a 3 1 /2 a 2 1 /2 a 1 1 /3 a 1 ∧ a 2 ∧ a 3 1 /3 -→ I ∈ K * f * I 1 a 3 1 /2 a 2 1 /2 a 1 1 /3 a 1 ∧ a 3 1 /3 a 1 ∧ a 2 1 /3 a 1 ∧ a 2 ∧ a 3 1 /3 Table 4.5: An incomplete LDCS
In Table 4.4 and Table 4.5, we give examples of constraint systems and their corresponding largest derivable constraint systems. In Table 4.4, the LDCS is complete. This means that there is only one probability measure on B which satisfies the constraints. In Table 4.5, the LDCS is incomplete. There is therefore an infinite number of probability measures on B which satisfy these constraints.

Preliminary step 3: Equations. As we will demonstrate in the proof to theorem 4.1.3, the limit in 4.1.1 is obtained as the solution to two easily defined equations which we present in this section. The variable in these equations is

a vector X =     x 0 . . . x d     in [0, 1] d+1
The solution corresponds to the vector

    p 0 . . . p d     ,
allowing to define the probability measure p. We will also consider the vector

ln(X) =     ln(x 0 ) . . . ln(x d )     , where ln : [0, +∞) → R[∞]; x → ln(x) if x = 0 and -∞ if x = 0.
Lemma 4.1.2. Consider C, C * , T K * and K * , with notations as above. Then, there exists at most one vector

X =     x 0 . . . x d     in [0, 1] d+1 such that: T K * X = K * and ln (X) ∈ Ker (T K * ) ⊥ Proof. Suppose X and Y are two such vectors. Then Y -X ∈ Ker(T K * ) and (Y -X) T ln(X) = (Y -X) T ln(Y ) = 0. Therefore, Y T ln(X) = X T ln(X) and X T ln(Y ) = Y T ln(Y ). As X T ln(X) ∈ R, we get Y T ln(X) ∈ R.
Therefore y i = 0 when x i = 0. By symmetry, we get x i = 0 ⇐⇒ y i = 0. We will therefore limit ourselves to the case where y i = 0 for all i as the other indices may be dropped for our current purposes.

Define the function ϕ Y : (0, 1] d+1 → R; Z → Z T ln(Z). We will consider the problem of minimizing ϕ under the constraint that T K * X = K * . Via the method of Lagrange multipliers we have the following necessary condition for a local optimum: ∇ϕ(Z) ∈ Im(T T K * ). Now, on the one hand,

∇ϕ(Z) =     1 . . . 1     + ln(Z)
and, on the other hand, as we are in finite dimension, Im(T |K k∈N be a sequence of functions satisfying the three following conditions:

T K * ) = Ker(T K * ) ⊥ . Furthermore,     1 . . . 1     ∈ Im(T T K * ),
• ∀k ∈ N, g (k)
|K is the restriction to K of a measure on B with integer values;

• g (k) -→ k→+∞ +∞; • 1 g (k) g (k) |K -→ k→+∞ f |K .
Consider:

• X k = 1 g (k)     µ (k) 0 . . . µ (k) d     where µ (k)
is the average finite measure defined by

K, g (k)
|K as in section 4.1.2;

• C * = (K * , f * |K * )
the largest derivable constraint system of C;

• and T K * the reduced transfer matrix as defined above.

Then (X k ) k∈N converges towards the unique vector X ∈ [0, 1] d+1 such that:

T K * X = K * and ln (X) ∈ Ker (T K * ) ⊥ Proof. As (X k ) k∈N is a sequence of vectors of [0, 1] d+1
, which is a compact space, it is sufficient to show that all convergent subsequences of (X k ) k∈N converge towards the same limit. Rather than considering a subsequence, we will consider, with no loss of generality, that (X k ) k∈N converges towards a limit and show that this limit is uniquely defined by C.

Let X be the limit of (X k ) k∈N . We know that, for all k ∈ N, T K * X k = K * k , and that K k -→ k→+∞ K. Hence, by continuity, T K * X = K * , which is the first of the two equations needed. Obtaining the second one is slightly more complex and is detailed in the following.

Let Y be a vector from the basis B K * of Ker (T K * ) as defined previously.

We know that the coordinates of Y are in {-1, 0, 1} and that d i=0 y i = 0, so we can set N Y = y i =1 y i =y i =-1 y i . Let n = g (k) and consider k so that n ≥ N Y .

We consider the space D k of all datasets of size n × m satisfying the constraints given by g (k)

|K . If we look at a dataset in D k , each line of the dataset corresponds uniquely to an element ω i of Ω. Consider the subsets D k,Y + (resp. D k,Y -) of D k of all matrices for which each of the N Y first lines correspond to one of the ω i such that y i = 1 (resp.

y i = -1). Then |D k,Y + | = |D k,Y -|.
Notice here that x i = 0 if y i = 0. Indeed, if x i = 0, this means that the convex polytope of the vectors Z which correspond to probability measures satisfying K * is contained in the affine space defined by the equation z i = 0. As the direction of this affine space is Ker(T K * ), then for any vector Y from a basis of Ker(T K * ), y i = 0. Furthermore, we will show that we have

|D k,Y + | /N Y !|D k | -→ k→+∞ y i =1 x i and |D k,Y -| /N Y !|D k | -→ k→+∞ y i =-1
x i . To prove this point, we will consider a probability with uniform distribution on the finite set of matrices D k . We note this probability Prob k . Let [L j = ω i ] denote the set of matrices of D k for which the j-th row corresponds to ω i and [|ω i | = l] the set of matrices of D k for which exactly l rows correspond to ω i . We can hereafter express our previous quantities as probabilities. For the first of the two fractions, this gives:

|D k,Y + | /N Y !|D k | = Prob k N Y j=1 L j = ω σ(j)
where σ : 1, N Y → {i ∈ N | y i = 1} is any bijection. Note that we only need to consider one of the two cases as the following demonstration is easily transposed to the other case. Furthermore, by the definition of X k , for all j ∈ 1, n and i ∈ 0, d , we have Prob

k (L j = ω i ) = x k,i (where x k,i is the i-th coordinate of X k ). In addition, as X k -→ k→+∞ X, we have Prob k (L j = ω i ) -→ k→+∞ x i . Hence, to prove our point, it is sufficient to show that Prob k N Y j=1 L j = ω σ(j) - N Y j=1
Prob k L j = ω σ(j) -→ k→+∞ 0 for any bijection σ defined as previously. As this is obvious for

N Y = 1, let us consider that N Y ≥ 2.
The convergence towards 0 corresponds to the following intuitive idea. If N Y is fixed while we consider larger and larger datasets (i.e. larger n), the events that any given one of the N Y first rows corresponds to any given ω i become gradually independent because the incidence that the value of one single row has on another single row becomes gradually negligible. We show this is true for two rows and the rest follows easily by iteration. Let i = j such that y i = y j = 1

and

H k = Prob k ([L 1 = ω i ] ∩ [L 2 = ω j ])-Prob k (L 1 = ω i ) Prob k (L 2 = ω j ). We see that H k = Prob k (L 1 = ω i ) (Prob k (L 2 = ω j | L 1 = ω i ) -Prob k (L 2 = ω j )). But we also have Prob k (L 2 = ω j | L 1 = ω i ) = Prob k (L 2 = ω j | |ω i | ≥ 1) = Prob k (L 2 = ω j ) Prob k (|ω i |≥1 | L 2 =ω j ) Prob k (|ω i |≥1) = Prob k (L 2 = ω j ) 1-Prob k (|ω i |=0 | L 2 =ω j ) 1-Prob k (|ω i |=0)
.

Hence,

H k = Prob k (L 1 = ω i ) Prob k (L 2 = ω j ) 1-Prob k (|ω i |=0 | L 2 =ω j ) 1-Prob k (|ω i |=0) -1 . And both Prob k (|ω i | = 0) -→ k→+∞ 0 and Prob k (|ω i | = 0 | L 2 = ω j ) -→ k→+∞ 0.
Therefore, H k -→ k→+∞ 0, quod erat demonstrandum. Note that the previous demonstration is only valid because, if y i = 1, both x i = 0 and the sequence (x k,i ) k≥1

is strictly positive for large enough k.

Now, the results of the two previous paragraphs can be combined and we get

y i =1 x i = y i =-1
x i . Hence,

y i =1
ln(x i ) -

y i =-1
ln(x i ) = 0, which can also be written Y T ln (X) = 0. As this is true for all Y from the basis B K * of Ker(T K * ), this gives ln (X) ∈ Ker (T K * ) ⊥ .

We conclude from lemma 4.1.2 that X is uniquely defined by K which ends the proof.

Note that this result is not limited to the case in which f |K is necessarily the restriction of a probability measure corresponding to an empirical distribution.

Indeed, the density of the rationals in the reals, together with the continuity of the functions defining the equations, ensure that it still holds if f |K is the restriction of any probability measure on B. More precisely, such a condition on f |K is only necessary for defining constraint systems in the finite approach and can be omitted when defining the asymptotic constraint system here.

Definition of MCI

In section 4.1.1, we have presented an approach for formulating an objective hypothesis on the values of a probability measure for the distribution of items given constraints on the values of this measure for certain itemsets. In section 4.1.3, we have shown that this approach leads to a solution which we can characterize mathematically as the unique solution to a system of equations.

Conversely, this characterization may be seen as a property of distributions of items which indicates how the items relate to each other: tied by a certain number of interrelations and entirely free otherwise. Because this characterization corresponds to the intuitive notion of independence under constraint and because it generalizes the mathematical notion of mutual independence, we have named this property mutual constrained independence. We give its formal definition below.

T K * X = K * and ln (X) ∈ Ker (T K * ) ⊥
where K * is the vector representation of f reduced to K * .

MCI and maximum entropy

As stated in the introduction, the notion we have defined is related to MaxEnt models. This is made explicit in the following theorem.

Theorem 4.1.4. Consider notations as in section 4.1.4. Then a 1 , ..., a p are mutually constrainedly independent in B with regards to the constraints defined by K if and only if

X = arg max T K * Z=K * Z∈[0,1] 2 p H(Z)
where H is the information entropy function and K * is the reduction of X to

K * .
Proof. The proof to this theorem is already contained in the proof to lemma 4.1.2. Indeed, we have shown the unicity of X (which corresponds to the solution to the mutual constrained independence problem) by showing that, if X exists, it is the minimum of a function which is none other than the opposite of the entropy function. As we have shown its existence in Theorem 4.1.3, it coincides therefore with this optimum.

This result implies that MCI models are MaxEnt models, where the maximization of the information entropy is constrained by the values of the empirical frequencies of the itemsets within the constrained set K. Such MaxEnt models have already been considered and employed successfully in the field of data mining (see section 2.3.3). As such, MCI models cannot be considered to be entirely novel models. However, the MCI approach towards the definition of these models is new and this novel approach brings forth a number of new perspectives. First, it brings further understanding to the maximum entropy principle as we have described in section 3.3.3 which helps to strengthen the use of MaxEnt models in mathematical modelings. Second, the MCI characterization of these models allows to envisage new properties of these models and, most notably, new methods for computing them, which we present in section 4.2.

MCI Models: properties and computation

In this section, we discuss some properties of MCI models and propose a novel approach for computing such models. We start by considering the specific case in which we consider all the proper subsets of A as the constrained set. We then propose a method based on algebraic geometry allowing to compute any MCI model. Finally, we use this method to provide exact algebraic formulas for all MCI models when m ≤ 4.

K

= I \ {I d }
The case of computing an MCI model in which K is the set of all proper subsets of I is relatively easy to consider because the number of degrees of liberty defined by the linear constraints is equal to one. Chronologically, it is the first case which was studied in the course of this thesis and is the focus of one of our published papers [START_REF] Delacroix | Constrained independence for detecting interesting patterns[END_REF]. The corresponding MaxEnt models have also been considered previously in the itemset literature but strictly from the perspective of an optimization problem [START_REF] Meo | Theory of dependence values[END_REF][START_REF] Tatti | Maximum entropy based significance of itemsets[END_REF].

Algebraic expression of the model

Consider a constraint system C = K, f |K such that K = I \ {I d }. Let p be the corresponding MCI model and let X =     p 0 . . . p d   
 be its vector representation in the basis Ω. Then:

T X =       f I 0 . . . f I d-1 p d      
As in section 4.1.2.1, we can express all the coordinates of X as of function of p d , like so:

X = C + p d V
where:

C = T -1       f 0 . . . f d-1 0       and V =       (-1) (d-0)•d (-1) (d-1)•d . . . (-1) (d-d)•d      
Furthermore, we know that p d ∈ [l, u] where:

l = max i∈ 0,d v i =1 (-c i ) and u = min i∈ 0,d v i =-1 (c i )
There are two possibilities:

• either l = u, which implies p d = l and the model is determined immediately (this corresponds to the case in which K * = I);

• or l < u, in which case we use the MCI characterization to determine p (in this case, K * = K).

In the second case, the MCI characterization translates directly to the following condition:

d i=0 v i =1 p i = d i=0 v i =-1 p i
Hence, p d is a real root of the polynomial Q defined as follows:

Q(x) = d i=0 v i =1 (c i + x) - d i=0 v i =-1 (c i -x)
Furthermore, this is necessarily the only real root of Q which lies in the interval [l, u] as this would otherwise imply that the MCI characterization does not uniquely define the MCI model. Hence, this root may be computed from the expression of Q via numerical methods.

Note that Q is a monic polynomial with degree exactly equal to d-1 2 . Indeed, if we factor each product and group by degree, we get:

Q(x) =    x d+1 2 +    d i=0 v i =1 c i    x d-1 2 + R + (x)    -(-1) d+1 2    x d+1 2 -    d i=0 v i =-1 c i    x d-1 2 + R -(x)   
where R + and R -have degree d-3 2 or less. As d = 2 m -1, d+1 2 = 2 m-1 is even for all m ≥ 2. Hence:

Q(x) = d i=0 c i x d-1 2 + R(x)
where R = R + -R -has degree d-3 2 or less. Finally, as:

d i=0 c i = d i=0 (p i -p d v i ) = d i=0 p i -p d d i=0 v i = 1 -p d × 0 = 1
we find that:

Q(x) = x d-1 2 + R(x)
For m = 3, we can see that p 7 is the only real root within the interval [l, u] of the polynomial Q 3 defined by:

Q 3 (x) = x 3 + αx + βx + γ
where:

α = f I 4 f I 2 + f I 4 f I 1 + f I 2 f I 1 -(f I 4 + 1)f I 3 -(f I 2 + 1)f I 5 -(f I 1 + 1)f I 6 β = f I 4 f I 3 (f I 3 -f I 2 -f I 1 ) + f I 2 f I 5 (f I 5 -f I 4 -f I 1 ) + f I 1 f I 6 (f I 6 -f I 4 -f I 2 ) +2f I 6 f I 5 f I 3 + f I 4 f I 2 f I 1 + f I 6 f I 5 + f I 6 f I 3 + f I 5 f I 3 γ = f I 6 f I 5 f I 3 (f I 4 + f I 2 + f I 1 -f I 6 -f I 5 -f I 3 -1) l = max(0, -f I 1 + f I 3 + f I 5 , -f I 2 + f I 3 + f I 6 , -f I 4 + f I 5 + f I 6 ) u = min(1 -f I 1 -f I 2 + f I 3 -f I 4 + f I 5 + f I 6 , f I 3 , f I 5 , f I 6 )

Distance to the MCI model

Consider a probability measure f that defines a constraint system C = K, f |K such that K = I \ {I d } and let p be the associated MCI model. An interesting question is to determine how these two probability measures compare and one of the simplest ways to compare them is to subtract one to the other. As f I = p I for all I = I d , fp is equal to zero for all items except I d . We will note δ A this difference:

δ(A) = f d -p d
Furthermore, for any itemset I in I, we can always consider the localization on I of the problem defined previously (by transposing the entire problem from the set of items A to the corresponding subset of items A I ), so that we can define a function ∆ for all I ∈ I by:

∆(I) = δ(A I )
The function ∆ may be seen as an objective interestingness measure allowing to determine for local redundancy in itemsets (see section 2.3.3.1). Indeed, it compares the value for the frequency of an itemset to the value given by the local MCI model where the constraints are set by all the proper subitemsets of that itemset. Although we have expressed some theoretical reservations with regards to the use of such measures in objective itemset mining in Chapter 3, such measures do present some practical advantages due to their algorithmic properties and they have been successfully put to use in data mining processes. We exhibit here one such property for ∆ which we originally presented in [START_REF] Delacroix | Constrained independence for detecting interesting patterns[END_REF].

Proposition 4.2.1. For all itemsets I ∈ I,

|∆(I)| ≤ 1 2 |I|
where |I| is the number of items in I. Furthermore, there is a dataset for which this upper bound is reached.

Proof. Without loss of generality, we will show that |δ(A)| ≤ 1 d+1 . This proposition is related to a nonlinear optimization problem. It can be solved by using the Karush-Kuhn-Tucker theorem (see, for example, [START_REF] Ruszczynski | Nonlinear optimization[END_REF], p.116, theorem 3.25).

Let V + = {i ∈ 0, d | v i = 1} and V -= {i ∈ 0, d | v i = 1}
where v i are the coordinates of the vector V as defined in section 4.2.1.1 and let V * + = V + \ {d} and V * -= V -\ {d -1}. Consider the functions f , h 0 , h 1 , ..., h d+1 defined for all z = (x 0 , ..., x d , y 0 , ..., y d ) ∈ R 2d+2 + by :

• f (z) = x d -y d ; • h k (z) = x d-1 + x k -(y d-1 + y k ), ∀k ∈ V * + ; • h k (z) = x d + x k -(y d + y k ), ∀k ∈ V -; • h d (z) = i∈V + x i - i∈V - x i ; • h d+1 (z) = d i=0 x i -1 ; • h d+2 (z) = d i=0 y i -1 .
We will consider the nonlinear optimization problem of minimizing f (z) on

R 2d+2 + subject to h k (z) = 0 for all k ∈ 0, d + 2 .
First, see that for z such that :

• xk = 1 d+1 for all k ∈ 0, d ;

• ỹk = 2 d+1 for all k ∈ V + ;

• and ỹk = 0 for all k ∈ V -;

the conditions are satisfied and f (z) = -1 d+1 . Let ẑ = (x 0 , ..., xd , ŷ0 , ..., ŷd ) be a minimum to the optimization problem (as all the coefficients of z are positive, the conditions given by h d+1 and h d+2 show that z belongs to the compact set [0, 1] 2d+2 and, hence, that such a minimum exists). Notice that the condition (h k (ẑ) = 0, ∀k ∈ 0, d -1 ) gives

f (ẑ) = xk -ŷk for all k ∈ V + and f (ẑ) = ŷk -xk for all k ∈ V -. Now, suppose xk = 0 for some k ∈ 0, d . The condition h d (ẑ) = 0 tells us that there is a couple (k + , k -) ∈ V + × V -such that x k + = x k -= 0. This gives y k + + y k -= 0 and, therefore, y k + = y k -= 0. Hence, f (ẑ) = 0, which is impossible as f (ẑ) ≤ -1 d+1
. Therefore, xk is strictly positive for all k. Now, suppose ŷk = 0 for some k ∈ V + . Then f (ẑ) = xk -ŷk = xk > 0, which is impossible as previously. Hence, ŷk is strictly positive for all k ∈ V + .

Let us now apply the Karush-Kuhn-Tucker theorem to the optimization problem. The theorem gives the existence of a z = (x 0 , ..., x d , y 0 , ..., y d ) ∈ R 2d+2 + and d + 2 Lagrange multipliers λk ∈ R for k ∈ 0, d + 2 , such that z • ẑ = 0 and z = ∇f (ẑ) + d+2 k=0 λk ∇h k (ẑ). As all coordinates are positive, the orthogonality condition implies that x k = 0 for all k and that y k = 0 for all k ∈ V + . This translates as follows:

x k = 0 ∀k ∈ V * + =⇒ λk + λd M xk + λd+1 = 0 ∀k ∈ V * + (1) x k = 0 ∀k ∈ V * - =⇒ λk -λd M xk + λd+1 = 0 ∀k ∈ V * -(2) x d = 0 =⇒ 1 + i∈V - λi + λd M xd + λd+1 = 0 (3) x d-1 = 0 =⇒ i∈V * + λi + λd-1 -λd M xd-1 + λd+1 = 0 (4)
y k = 0 ∀k ∈ V * + =⇒ -λk + λd+2 = 0 ∀k ∈ V * + (5) y d = 0 =⇒ -1 - i∈V - λi + λd+2 = 0 (6)
where

M = i∈V + xi = i∈V - xi .
By combinations, we see that:

(1) + (5) =⇒ λd

M xk + λd+1 + λd+2 = 0 ∀k ∈ V * + (3) + (6) =⇒ λd M xd + λd+1 + λd+2 = 0 which gives λd M 1 xk - 1 xk = 0 ∀ (k, k ) ∈ (V + ) 2
As M > 0, this implies that

λd = 0 or xk = xk ∀ (k, k ) ∈ (V + ) 2
If λd = 0, then (1) and ( 2) give λk = -λd+1 for all k ∈ V * + ∪ V * -. Hence, λk + λd+2 = 0 (7)

i∈V - λi = i∈V * - λi + λd-1 = i∈V * + λi + λd-1 . Therefore, (3)
If k = d -1, it implies - i∈V * + -λd-1 + λd+2 = 0 (8)
In both cases, by combining (2) + (7) or (4) + (8), we get

-λd M xk + λd+1 + λd+2 = 0
Hence, xk = -x d < 0. Again, this is a contradiction and we can conclude that y k > 0 for all k ∈ V -, which in turn implies that ŷk = 0 for all k ∈ V -.

We This result echoes with the shrinking property described in [START_REF] Calders | Non-derivable itemset mining[END_REF] which states that the width of the interval [l, u] (see section 4.2.1.1) shrinks exponentially with m. More precisely, the proposition in [START_REF] Calders | Non-derivable itemset mining[END_REF] allows to say that, for two probability measures f and g on B which share the same values for the frequencies of all proper subitemsets of I d ,

|f d -g d | ≤ 1 2 m-1
Similarly, our own proposition shows that, if p is the MCI model associated to f and defined by the proper subitemsets of I d , then

|f d -p d | ≤ 1 2 m
Such propositions may prove helpful if ∆ is used as an objective interestingness measure in a mining process so that only itemsets I satisfying ∆ ≥ η are considered interesting, for some threshold η. For example, one of the main issues with level wise algorithms is the possible explosion in the number of candidate itemsets from one layer to the next and it is often decided to stop the algorithm at a fixed layer. These proposition can tell us at which layer we can stop, without having to define any candidates on a further layer. Compared to the shrinking property in [START_REF] Calders | Non-derivable itemset mining[END_REF], our own proposition allows to stop one layer earlier, which given the explosion of candidates between two layers can represent an important gain. As we have also shown that the bound which we have obtained is the best possible bound, proposition 4.2.1 may also be used to determine the algorithmic complexity of such mining algorithms.

Particularity of the MCI approach.

As we have stated previously, we do acknowledge that mathematically equivalent models have been considered in the literature before. Rosa Meo has even presented an interestingness measure in [START_REF] Meo | Theory of dependence values[END_REF] which is mathematically equivalent to ∆. This article, in which she calls p d the "maximum independence value" and ∆ the "dependence value", aims at defining and computing ∆. However, it is quite interesting to see that, though the focus of her article is on the same mathematical objects as this current section, none of the results which are presented in this section are presented in [START_REF] Meo | Theory of dependence values[END_REF]. In particular, she did not reduce the equations of the optimization problem to its algebraic solution, even for m = 3. This is also true for other related articles which we have found within the literature such as [START_REF] Tatti | Maximum entropy based significance of itemsets[END_REF]. We believe this is linked to the idea that, as the notions in these articles are defined with respect to a problem of entropy maximization, the tools that are considered to compute them come from standard optimization theory.

In comparison, the approach which has led us to the results presented here is entirely different because, when we first proved them, we had not yet established the link between MCI models and MaxEnt models. As such, we can see the MCI approach as a means to envisage MaxEnt models from a different mathematical perspective. In the specific case discussed in this section, it is quite apparent that the model can be computed by considering a polynomial from the MCI point of view. Interestingly enough from an epistemological perspective, the fact that we had not yet made the connection between our models and MaxEnt models allowed us to follow through. Hence, when we considered more general MCI models, we focused on developing a generalized approach for computing them also based on polynomials. We present this generalization in the next section.

Algebraic geometry for computing MCI models

In the particular case for which K * = I \ {I d }, we have shown that we can algebraically reduce the equations defining the MCI model p and determine a univariate real polynomial Q together with d + 1 real affine functions A i (for each i ∈ 0, d ) such that, there exists a unique t ∈ R satisfying:

1. Q(t) = 0; 2. ∀i ∈ 0, d , A i (t) ≥ 0;
and solving this equation gives p as p i = A i (t) for all i ∈ 0, d . Hence, we can easily determine p by computing each root of Q successively until we find t such that A i (t) ≥ 0 for all i ∈ 0, d which gives us the solution.

Note that we can compute the roots of a real univariate polynomial up to any desired precision with a number of algorithms from optimization theory which guarantee successful termination. Hence, we have defined a general process which allows to determine the MCI model with guaranteed success (provided sufficient resources) when K * = I \ {I d }. Furthermore, the reduction phase only needs to be computed once for any m so that computing an MCI model may be performed very efficiently for different values of K * corresponding to a common value of K * (which, in this case, is entirely determined by m).

Our goal now is to show that this principle can be generalized to any K * .

This offers an interesting alternative to the more classical option of computing the model through constrained optimization algorithms based only on the equations in definition 4.1.3. Indeed, multivariate optimization algorithms do not allow to solve such problems in generic cases and thus must necessarily be performed over and over again for each specific case.

Note that this is not the first attempt to describe such models through algebraic geometry. In fact, Berd Sturmfels uses a similar description for a more general class of maximum likelihood models in [START_REF] Sturmfels | Solving systems of polynomial equations[END_REF]. However, the algorithm he suggests remains an analytical one.

Algebraic geometry for polynomial system solving

As we will show, the equations defining the MCI model can easily be transposed into a multivariate polynomial system. Solving a multivariate polynomial system is a difficult task in general which has been mostly addressed within the field of algebraic geometry and a number of algorithms for solving real polynomial systems are now known to exist [START_REF] Sturmfels | Solving systems of polynomial equations[END_REF][START_REF] Basu | Algorithms in Real Algebraic Geometry[END_REF][START_REF] Bochnak | Real algebraic geometry[END_REF].

We present here the main result on which our approach is based. However, we do not include a detailed presentation of the mathematical background in algebraic geometry which is necessary to fully grasp the concepts which we cover in this section. We refer the reader to the aforementioned literature for further insight on this topic. Furthermore, to avoid any ambiguity, we have conformed the terminology in algebraic geometry used in this thesis with the the main focus here is on accomplishing the first step. However, computing a univariate representation raises two important issues.

Firstly, the polynomial system P which we consider must be zero-dimensional and, as we will show, this is not entirely straightforward. Secondly, algebraic algorithms do not tolerate approximate values well. In particular, floating point representations may not be used in the algorithms which we consider here. Instead, the coefficients of the polynomials considered in the algorithms, as well as the operations performed on these coefficients, must be considered within a formal calculus structure. While this is not technically infeasible, it may require significant computational resources both in time and memory. In order to accomplish this, two main options can be considered. The first option is to represent P as a system of polynomials in Q[X] (which is technically the case if the constraints given by K are defined by an empirical dataset) and perform operations in a formal representation of Q[X]. This is the easier option of the two to code and is also generally faster to compute, but it only allows to determine a univariate representation corresponding to a particular constraint system defined by K * , f |K * . The other option is to consider that the poly-

nomials in P belong to Q(f 1 , ..., f d )[X]

A zero-dimensional polynomial system

Let K * , f |K * be a constrained system and X the vector associated to the MCI model as in definition 4.1.3. The vector X is characterized as the unique solution to a linear and loglinear problem (theorem 4.1.3). We will show how we can transpose the equations of this characterization into a roughly equivalent zero-dimensional polynomial system in R[X].

Linear part. Firstly, let us define, polynomials L i for all i ∈ 0, d such that:

L i = d j=0 t i,j X j -f i
in which t i,j are the coordinates of the matrix T . For example, when m = 3, this gives:

L 0 = X 0 + X 1 + X 2 + X 3 + X 4 + X 5 + X 6 + X 7 -1 L 1 = X 1 + X 3 + X 5 + X 7 -f 1 L 2 = X 2 + X 3 + X 6 + X 7 -f 2 L 3 = X 3 + X 7 -f 3 L 4 = X 4 + X 5 + X 6 + X 7 -f 4 L 5 = X 5 + X 7 -f 5 L 6 = X 6 + X 7 -f 6 L 7 = X 7 -f 7
The linear equation T K * X = K * is then equivalent to the polynomial system P L = (L j ) j∈J where J = {j ∈ 0, d | I j ∈ K * }. We note r = |J| the number of polynomials in P L and we can easily notice that the dimension of P L is equal to s = 2 m -r (because it is equal to the dimension of its set of zeros Z L as a vector space). The algorithm for computing P L is here entirely straightforward:

1 . P L ← ∅ ;

2 . f o r j i n J :

M Y = d i=0 y i >0 X y i i - d i=0 y i <0 X -y i i ∈ R[X]
and the equation ln (X) ∈ Ker (T K * ) ⊥ implies that M Y (X) = 0. Our aim now is to pick a family of vectors in Ker (T K * ) ∩ Z d+1 which defines a polynomial system P M that can be concatenated with P L to obtain a polynomial system P which allows to characterize the MCI model X.

The first idea which comes to mind is to consider the basis B K * of Ker (T K * ) the dimension of Ker (T K * ) ⊥ is equal to r = 2 m -s so that the set of all

X ∈ R * + d+1 satisfying ln (X) ∈ Ker (T K * ) ⊥ is a smooth r-manifold. Hence, Z M is locally of dimension r around all X ∈ Z M ∩ R * + d+1 . This property extends to all X ∈ Z M ∩ (R * ) d+1 because X ∈ Z M implies |X| ∈ Z M where |X| =     |x 0 | . . . |x d |     ∈ (R + ) d+1 . Indeed, for any M Y ∈ P M , then M Y (X) = 0 ⇐⇒ d i=0 y i >0 x y i i -d i=0 y i <0 x -y i i = 0 ⇐⇒ d i=0 y i >0
x

y i i = d i=0 y i <0 x -y i i =⇒ d i=0 y i >0
x

y i i = d i=0 y i <0 x -y i i ⇐⇒ d i=0 y i >0 |x i | y i = d i=0 y i <0 |x i | -y i ⇐⇒ d i=0 y i >0 |x i | y i -d i=0 y i <0 |x i | -y i = 0 ⇐⇒ M Y (|X|) = 0. Therefore, if the dimension of Z M is greater than r,
this is necessarily due to its behavior within R d+1 ∩ d i=0

H i where H i is the hyperplane defined by X i = 0.

In other words, if P M is determined by a generating family of vectors of Ker (T K * ) ∩ Z d+1 , its dimension should be equal to r, unless there is a subset S ⊂ 0, d with cardinality s = |S | < s defining a linear space Z = X ∈ R d+1 ∀i ∈ S , x i = 0 of dimension r = 2 m -s > r such that Z ⊂ Z M . Hence, in order to show that there is a family of generating vectors of Ker (T K * )∩Z d+1 such that the associated polynomial system P M has dimension r, we must show the following lemma.

Lemma 4.2.1. There is a family of generating vectors of Ker (T K * ) ∩ Z d+1 which defines a polynomial system P M such that:

{S ⊂ 0, d | (s < s) ∧ (Z ⊂ Z M )} = ∅
The proof of lemma 4.2.1 relies on the other following lemma from linear algebra.

Lemma 4.2.2. Let V be a vector space of R d+1 such that:

∃S ⊂ 0, d , ∀X ∈ V \ {0}, S + (X) ∩ S = ∅ and S -(X) ∩ S = ∅ where S + (X) = {i ∈ 0, d | x i > 0} and S -(X) = {i ∈ 0, d | x i < 0}. Then: dim (V) ≤ s
vectors Y obtained through such a process would not, a priori, have minimal cardinality. In the previous example, in which m = 3 and K * = { }, the cardinality of Y would be necessarily greater than 7, which is th cardinality of B K * , while the family W defined by:

W =                                 1 -1 0 0 0 0 0 0                 ,                 1 0 -1 0 0 0 0 0                 ,                 1 0 0 -1 0 0 0 0                 ,                 1 0 0 0 -1 0 0 0                 ,                 1 0 0 0 0 -1 0 0                 ,                 1 0 0 0 0 0 -1 0                 ,                 1 0 0 0 0 0 0 -1                                
satisfies the conditions of lemma 4.2.1. Hence, we resort to a number of heuristics in order to obtain concise forms of Y, leading to simpler polynomial systems to solve.

First, we can see that if Y is such that y i = 0 for all i ∈ 0, d \ {j, j }, y j = 1 and y j = -1, for some j, j ∈ 0, d with j = j , then M Y is a linear function. Hence, if there is such a Y ∈ Ker (T K * ) ∩ Z d+1 , then we can consider M Y within the linear part of the system, which can be solved first to reduce the general complexity of the problem. Therefore, we start by determining a subfamily of Y, corresponding to such linear functions, which we note Y L .

This can be accomplished through the following algorithm:

1 . i n i t i a l i z e J ← ∅ ; By combining all these algorithms, we obtain an algorithm for computing

P M : 1 . compute Y L v i

Speed-up for independence cases

The computational complexity of this algorithm is quite difficult to characterize because the computational complexity for determining a univariate representation of Z is itself quite difficult to characterize (unless a Gröbner basis for P is provided but this is not the case here). Obviously, the computational complexity increases at least exponentially with m as the number of variables considered is equal to d+1 = 2 m . But given m, the complexity varies also enormously with the structure of K * . Cases such as K * = { } or K * = I \ {I d } are extremely easy cases to compute while cases corresponding to standard (unconstrained) mutual independence between items or itemsets appear to be the most difficult ones. Hopefully, such cases may be identified and divided into cases corresponding to strictly smaller values of m which prove to be easier to compute.

Consider for example that m = 5 and K * = { , a 1 ∧ a 2 , a 3 ∧ a 4 , a 4 ∧ a 5 , a 3 ∧ a 5 }. None of the constraints on a 1 and a 2 are linked in any way to the constraints on a 3 , a 4 and a 5 . Hence, we can consider two MCI models: the probability distribution p 1 over the Boolean lattice B 1 associated to

A 1 = {a 1 , a 2 }, defined by K * 1 , f |K * 1
where K * 1 = { , a 1 ∧ a 2 }, on the one hand; and the probability distribution p 2 over the Boolean lattice B 2 associated to

A 2 = {a 3 , a 4 , a 5 }, defined by K * 2 , f |K * 2
where K * 2 = { , a 3 ∧a 4 , a 4 ∧a 5 , a 3 ∧a 5 }, on the other hand. The MCI model p is then obtained by the independence of these two models via:

p (a * 1 , a * 2 , a * 3 , a * 4 , a * 5 ) = p 1 (a * 1 , a * 2 ) p 2 (a * 3 , a * 4 , a * 5 )
where a * i ∈ {a i , a i } for all i ∈ 1, 5 . More generally, we can define the undirected graph G = (V, E) of the mutual constraints between items by:

• V = {a 1 , ..., a m }; • {a i , a j } ∈ E if and only if ∃I ∈ K * such that I =⇒ (a i ∧ a j ).
Let n c be the number of connected components of G and V 1 , ..., V nc the set of items associated to each component. Then, each set of items V i corresponds to an MCI model p i over the Boolean lattice associated to V i , defined by K * i , f |K * i where:

K * i =    I ∈ K * a j ∈V i a j =⇒ I  
 and the MCI model p is entirely defined by:

p m j=1 a * j = nc i=1 p i   a j ∈V i a * j  
If G has only one connected component, then there is no gain, but the cost of computing G and its connected components is highly negligible in comparison to the gain that occurs when G has at least two components. This is true when the MCI model is computed through algorithms in algebraic geometry, but it is also true if they are seen as MaxEnt models and computed through algorithms in optimization theory and a similar process is described in [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF].

Speed-ups for step 4

As stated previously, the bottleneck of algorithm 4.8 in terms of computational complexity resides in its step 4, in which a univariate representation of Z is computed. In order to speed this step up, we can use substitutions to reduce significantly the number of variables considered before solving the polynomial system. These speed-ups were essential to compute the algebraic forms of all MCI models for m = 3 and m = 4.

The first trick is to reduce the linear part of P separately and perform substitutions in the nonlinear part of P based on this reduction. The linear part of P comprises the polynomials in P L , as well as the polynomials in P M which correspond to the family of vectors Y L as determined by algorithm 4.2

(noted P M L in algorithm 4.6) and the polynomials added to P in step 3 of algorithm 4.8 (we will note these P D ). Each of these polynomials corresponds naturally to a vector with coordinates in (X 0 , ..., X d , 1) so that we can see the linear part of P as a matrix with d + 2 columns and as many row as polynomials in the sets mentioned above. We can then consider its reduced row echelon form and obtain a set of free variables from which the remaining pivot variables are entirely determined. The pivot variables are then substituted in the remaining polynomials of P (noted P M N L in algorithm 4.6) by their expressions as affine functions of the free variables. In this manner, a new zero-dimensional polynomial system is obtained whose variables are the free variables determined previously. The reduction in terms of number of variables is quite substantial. For m = 3, this brings down the number of variables down from 8 to 1, 2 or 3 depending on K * . For m = 4, this brings down the number of variables down from 16 to 7 or less. Note that the part of this reduction which is based on the elements of P M L is mostly equivalent to the reduction based on blocks described in [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF] for the computation of MaxEnt models. Now that we have obtained this reduced polynomial system, the second trick is to find any variable for which at least one polynomial in the system has degree exactly 1. Indeed, if a polynomial P has degree 1 in a variable, say X 0 , then P (X 0 , ..., X d ) = A(X 1 , ..., X d )X 0 + B(X 1 , ..., X d ) and, therefore:

P (X 0 , ..., X d ) = 0 ⇐⇒ A(X 1 , ..., X d )X 0 = -B(X 1 , ..., X d )
(Note that we write X 0 , ..., X d for simplicity even though we are now consid-ering a set of variables which is strictly contained in {X 0 , ..., X d }.)

Furthermore, as we have P (x 0 , ..., x d ) = 0 and x 0 = 0 when considering the MCI model (because the variables equal to zero have already been set aside in the reduction described above), then either A(x 1 , ..., x d ) = B(x 1 , ..., x d ) = 0 or A(x 1 , ..., x d )B(x 1 , ..., x d ) = 0. Each of these cases can be associated to a zero-dimensional polynomial system which is easier to solve than the current one. On one side, if A(x 1 , ..., x d ) = B(x 1 , ..., x d ) = 0, we can consider the polynomial system in which P has been replaced by A and B. And, on the other side, if A(x 1 , ..., x d )B(x 1 , ..., x d ) = 0, we can consider that X 0 = -B A (where A and B can be reduced so that they contain no common factors because x 0 does not correspond to a root of A or B) and thus substitute X 0 by -B A in all the polynomials of the system and multiply each of these by A as many times as necessary to obtain a polynomial (which corresponds to the degree of X 0 in the polynomial). In this case, the new polynomial system has one polynomial less (the polynomial P initially considered) and one variable less (X 0 in this example). Note that, in all the cases which we have computed for m = 3 and m = 4, when such a reduction was possible, the solution of the system associated to the MCI model always corresponded to the reduced polynomial system in which a variable was substituted by a rational expression

-B
A . Hence, though we have not proved this generally, for all the cases which we have computed, such a reduction corresponds to decreasing the number of variables in the polynomial system by one. This process may be repeated until the system may no longer be reduced in this manner. However, note that, if at one point in the process there is more than one variable which may be considered, the choice of the variable may influence how much the system may be reduced. In practice, the gain provided by reducing the number of variables is such that we explore all possible choices until we have found one which gives an optimal reduction in terms of number of variables. In the course of this doctoral research, we have computed such formal univariate representations for a sufficient number of cases of K * such that m ≤ 4, allowing for a fast computation of all MCI models in which m ≤ 4 or consisting of independent groups of items satisfying this condition. The number of different cases of K * for a given m is equal to 2 2 m -1 which is the number of subsets of I that contain . However, it is sufficient to consider only a fraction of these cases because if a set K * 1 may be obtained from a set K * 2 by a simple permutation of the items defining the itemsets, then a formal univariate representation associated to K * 1 may be obtained from the formal univariate representation computed for K * 2 . Hence, we need only consider a single representative for each equivalence class defined by the set of permutations on items which brings down the number of cases to compute significantly enough. This corresponds to sequence A000612 in [START_REF] James | The On-Line Encyclopedia of Integer Sequences[END_REF], which is described as the number of non-isomorphic sets of nonempty subsets of an n-set. The number of cases to compute can be brought down slightly further still by computing only the cases which do not correspond to independence cases using the principles described in section 4.2.2.4. The number of such cases corresponds to sequence A323819 in [START_REF] James | The On-Line Encyclopedia of Integer Sequences[END_REF], which is described as the number of non-isomorphic connected set-systems covering n vertices. The number of cases to compute is therefore reasonable enough for us to envisage computing all the cases for m ≤ 4 on a personal computer. Given more computational power, computing the cases for m = 5 may also be considered. However, though the gain in terms of number of cases to compute is asymptotically a factor m!, this is not sufficient to envisage an exhaustive computation of all cases for any value of m beyond m = 5.

Setting aside the question of computing a large number of cases, the issue with performing computations in the field Q(f 1 , ..., f d ) (or, more precisely, in the polynomial space Q(f 1 , ..., f d )[X 0 , ..., X d ]) resides in the augmented cost of basic operations and simplifications of expressions which must be performed both a great many times and with expressions that are potentially quite long.

However, in order to curtail the size of the expressions considered, the coefficients of the polynomials can always be reduced to an irreducible rational fraction (based on the continuity of the solution with regards to the variables f 1 , ..., f d ). This means that we can also consider operations on polynomials with coefficients in Q[f 1 , ..., f d ] that are setwise coprime, which is the option we have adopted in our implementation.

Last, once a formal univariate representation is computed it may possibly be reduced. Indeed, it may appear, in some cases, that one or several of the roots of the polynomial Q of a univariate representation (Q, B, A 0 , ..., A d ) can be ignored: either because they lead to solutions which can be formally identified as not satisfying the conditions of the MCI model (necessarily leading to negative or non real values for x 0 , ..., x d ); or because they lead to solutions which necessarily correspond to a situation of derivability (where one of the values for x 0 , ..., x d at least is equal to zero which can be ignored because of the continuity of the MCI model with regards to f 1 , ..., f d ).

The code with which we have obtained the formal univariate representations described in this section will be made freely available.

Solutions for m = 3. We list below the computed algebraic expressions corresponding to representatives for each of the 30 different equivalence classes described above. For each case, we give the subset of {f 1 , f 2 , f 3 , f 4 , f 5 , f 6 , f 7 } corresponding to the fixed frequencies. If solving the system includes computing the roots of a polynomial Q with coefficients in Z[f 1 , ..., f 7 ], we indicate this in the upper right corner and give the corresponding polynomial below.

We then list the algebraic expressions for each x i based on the values f 1 , ..., f 7 as well as the previously computed values of x i and a root t of Q. The MCI model is obtained by considering a root t of Q such that all x i are positive.

{f 7 } {f 6 , f 7 } {f 5 , f 6 } Q x 0 =
1-f 7 7

x 0 = 1-f 6 6

x 7 = t

x 1 = x 0 x 1 = x 0 x 0 = 1-f 5 -f 6 +x 7 5 x 2 = x 0 x 2 = x 0 x 1 = x 0 x 3 = x 0 x 3 = x 0 x 2 = x 0 x 4 = x 0 x 4 = x 0 x 3 = x 0 x 5 = x 0 x 5 = x 0 x 4 = x 0 x 6 = x 0 x 6 = f 6 -f 7 x 5 = f 5 -x 7 x 7 = f 7 x 7 = f 7 x 6 = f 6 -x 7 Q = 4T 2 -(1 + 4(f 5 + f 6 ))T + 5f 5 f 6 {f 5 , f 6 , f 7 } {f 4 , f 7 } {f 4 , f 6 , f 7 } x 0 = 1-f 5 -f 6 +f 7 5 x 0 = 1-f 4 4 x 0 = 1-f 4 4 x 1 = x 0 x 1 = x 0 x 1 = x 0 x 2 = x 0 x 2 = x 0 x 2 = x 0 x 3 = x 0 x 3 = x 0 x 3 = x 0 x 4 = x 0 x 4 = f 4 -f 7 3 x 4 = f 4 -f 6 2 x 5 = f 5 -f 7 x 5 = x 4 x 5 = x 4 x 6 = f 6 -f 7 x 6 = x 4 x 6 = f 6 -f 7 x 7 = f 7 x 7 = f 7 x 7 = f 7 {f 4 , f 5 , f 6 } {f 4 , f 5 , f 6 , f 7 } {f 3 , f 5 , f 6 } Q x 7 = f 5 f 6 f 4 x 0 = 1-f 4 4 x 7 = t x 0 = 1-f 4 4 x 1 = x 0 x 0 = 1-f 3 -f 5 -f 6 +2x 7 4 x 1 = x 0 x 2 = x 0 x 1 = x 0 x 2 = x 0 x 3 = x 0 x 2 = x 0 x 3 = x 0 x 4 = f 4 -f 5 -f 6 + f 7 x 3 = f 3 -x 7 x 4 = f 4 -f 5 -f 6 + x 7 x 5 = f 5 -f 7 x 4 = x 0 x 5 = f 5 -x 7 x 6 = f 6 -f 7 x 5 = f 5 -x 7 x 6 = f 6 -x 7 x 7 = f 7 x 6 = f 6 -x 7 Q = 20T 3 + 4(1 -5(f 3 + f 5 + f 6 ))T 2 + ((1 -(f 3 + f 5 + f 6 )) 2 + 16(f 3 f 5 + f 3 f 6 + f 5 f 6 ))T -16f 3 f 5 f 6 {f 3 , f 5 , f 6 , f 7 } {f 3 , f 4 , f 7 } {f 3 , f 4 , f 6 } Q x 0 = 1-f 3 -f 5 -f 6 +2f 7 4 x 0 = 1-f 3 -f 4 +f 7 3 x 7 = t x 1 = x 0 x 1 = x 0 x 0 = 1-f 3 -f 4 +x 7 3 x 2 = x 0 x 2 = x 0 x 1 = x 0 x 3 = f 3 -f 7 x 3 = f 3 -f 7 x 2 = x 0 x 4 = x 0 x 4 = f 4 -f 7 3 x 3 = f 3 -x 7 x 5 = f 5 -f 7 x 5 = x 4 x 4 = f 4 -f 6 2 x 6 = f 6 -f 7 x 6 = x 4 x 5 = x 4 x 7 = f 7 x 7 = f 7 x 6 = f 6 -x 7 Q = 2T 2 + (f 4 -2f 3 -3f 6 -1)T + 3f 3 f 6 {f 3 , f 4 , f 6 , f 7 } {f 3 , f 4 , f 5 , f 6 } Q {f 3 , f 4 , f 5 , f 6 , f 7 } x 0 = 1-f 3 -f 4 +f 7 3 x 7 = t x 0 = 1-f 3 -f 4 +f 7 3 x 1 = x 0 x 0 = 1-f 3 -f 4 +x 7 3 x 1 = x 0 x 2 = x 0 x 1 = x 0 x 2 = x 0 x 3 = f 3 -f 7 x 2 = x 0 x 3 = f 3 -f 7 x 4 = f 4 -f 6 2 x 3 = f 3 -x 7 x 4 = f 4 -f 5 -f 6 + f 7 x 5 = x 4 x 4 = f 4 -f 5 -f 6 + x 7 x 5 = f 5 -f 7 x 6 = f 6 -f 7 x 5 = f 5 -x 7 x 6 = f 6 -f 7 x 7 = f 7 x 6 = f 6 -x 7 x 7 = f 7 Q = 4T 3 + (1 -4(f 3 + f 5 + f 6 ))T 2 + (3(f 3 f 5 + f 3 f 6 + f 5 f 6 ) + (1 -f 3 -f 4)(f 4 -f 5 -f 6))T -3f 3 f 5 f 6 {f 2 , f 4 , f 7 } Q {f 2 , f 4 , f 6 , f 7 } {f 2 , f 4 , f 5 , f 7 } x 6 = t x 0 = 1-f 2 -f 4 +f 6 2 x 6 = (f 4 -f 5 )(f 2 -f 7 ) 1-f 5 x 0 = 1-f 2 -f 4 +f 7 +x 6 2 x 1 = x 0 x 0 = 1-f 2 -f 4 +f 7 +x 6 2 x 1 = x 0 x 2 = f 2 -f 6 2 x 1 = x 0 x 2 = f 2 -f 7 -x 6 2 x 3 = x 2 x 2 = f 2 -f 7 -x 6 2 x 3 = x 2 x 4 = f 4 -f 6 2 x 3 = x 2 x 4 = f 4 -f 7 -x 6 2 x 5 = x 4 x 4 = f 4 -f 5 -x 6 x 5 = x 4 x 6 = f 6 -f 7 x 5 = f 5 -f 7 x 7 = f 7 x 7 = f 7 x 7 = f 7 Q = T 2 + (2 -f 2 -f 4 )T -(f 4 -f 7)(f 2 -f 7) {f 2 , f 4 , f 5 , f 6 } {f 2 , f 4 , f 5 , f 6 , f 7 } {f 2 , f 3 , f 4 , f 5 } Q x 7 = f 5 f 6 f 4 x 0 = 1-f 2 -f 4 +f 6 2 x 6 = t x 0 = 1-f 2 -f 4 +f 6 2 x 1 = x 0 x 7 = x 6 -1 -f 2 + 2f 3 -f 4 + 2f 5 + 2(f 2 -f 3 )(f 4 -f 5 ) x 6 x 1 = x 0 x 2 = f 2 -f 6 2 x 0 = 1-f 2 -f 4 +x 6 +x 7 2 x 2 = f 2 -f 6 2 x 3 = x 2 x 1 = x 0 x 3 = x 3 x 4 = f 4 -f 5 -f 6 + f 7 x 2 = f 2 -f 3 -x 6 x 4 = f 4 -f 5 -f 6 + x 7 x 5 = f 5 -f 7 x 3 = f 3 -x 7 x 5 = f 5 -x 7 x 6 = f 6 -f 7 x 4 = f 4 -f 5 -x 6 x 6 = f 6 -x 7 x 7 = f 7 x 5 = f 5 -x 7 Q = T 3 +(1-(1+f 2 -f 3 )(1+f 4 -f 5 )-(1-f 3 )(1-f 5 ))T 2 +(f 2 -f 3 )(f 4 -f 5 )(f 2 -2f 3 +f 4 -2f 5 +3)T -2(f 2 -f 3 ) 2 (f 4 -f 5 ) 2 {f 2 , f 3 , f 4 , f 5 , f 7 } Q 1 {f 2 , f 3 , f 4 , f 5 , f 6 } Q 2 {f 2 , f 3 , f 4 , f 5 , f 6 , f 7 } x 6 = t x 7 = t x 0 = 1-f 2 -f 4 +f 6 2 x 0 = 1-f 2 -f 4 +f 7 +x 6 2 x 0 = 1-f 2 -f 4 +f 6 2 x 1 = x 0 x 1 = x 0 x 1 = x 0 x 2 = f 2 -f 3 -f 6 + f 7 x 2 = f 2 -f 3 -x 6 x 2 = f 2 -f 3 -f 6 + x 7 x 3 = f 3 -f 7 x 3 = f 3 -f 7 x 3 = f 3 -x 7 x 4 = f 4 -f 5 -f 6 + f 7 x 4 = f 4 -f 5 -x 6 x 4 = f 4 -f 5 -f 6 + x 7 x 5 = f 5 -f 7 x 5 = f 5 -f 7 x 5 = f 5 -x 7 x 6 = f 6 -f 7 x 7 = f 7 x 6 = f 6 -x 7 x 7 = f 7 Q 1 = T 2 -(1 + f 2 -2f 3 + f 4 -2f 5 + f 7 )T + 2(f 2 -f 3 )(f 4 -f 5 ) Q 2 = 2T 3 + (f 2 -2f 3 + f 4 -2f 5 -3f 6 )T 2 + (f 2 f 4 -f 2 f 5 -f 2 f 6 -f 3 f 4 + 2f 3 f 5 + 2f 3 f 6 -f 4 f 6 + 2f 5 f 6 + f 2 6 )T -f 3 f 5 f 6 {f 1 , f 2 , f 4 , f 7 } Q {f 1 , f 2 , f 4 , f 6 , f 7 } {f 1 , f 2 , f 4 , f 5 , f 6 } x 6 = t x 5 = (f 1 -f 7 )(f 4 -f 6 ) 1-f 6 x 7 = f 5 f 6 f 4 x 5 = (f 1 -f 7 )(f 4 -f 7 -x 6 ) 1-f 7 -x 6 x 3 = (f 2 -f 6 )(f 1 -f 7 -x 5 ) 1-f 4 x 3 = (f 1 -f 5 )(f 2 -f 6 ) 1-f 4
x 3 = which j = i (i.e. D = ∅). For each of these distributions, we considered the computation of 30 different MCI models, each of which corresponded to one of the pre-computed cases in section 4.2.2.6 above. Each of these computations were performed 100 times using the pre-computed algebraic expressions and 100 times using an implementation of the Iterative scaling procedure by Darroch and Ratcliff for computing MaxEnt models [START_REF] John | Generalized iterative scaling for log-linear models[END_REF] 2 . In order to make comparisons in terms of execution as meaningful as possible, the computations were performed on the same computer (Intel Core i7-8550U CPU 1.80GHz × 8, 7.7 GiB RAM) and both were based on a Python 3 implementation. The total execution time using the algebraic expressions was approximately equal to 2.14 seconds, while it took approximately 6 minutes and 21 seconds for the purely numerical method. Hence, the method based on the algebraic expressions was about 150 times faster here. Note that a more detailed observation of the execution times in the process described above allowed us to ensure that the gain in time was not concentrated on any distribution or constrained set in particular (though there was some variations between constraint sets).

Even though the gain in terms of execution time obtained here is quite impressive, it must be put into perspective. Such a gain can only be obtained if we consider specific cases corresponding to previously computed generic cases, the computation of which is itself quite time consuming. As mentioned in section 4.2.2.6, we have managed to compute all generic cases corresponding to m ≤ 4 but we also acknowledge that doing so is intractable for any value of m ≥ 6.

Nevertheless, the inability to compute the exhaustive list of all generic cases for larger values of m does not necessarily represent a serious limitation to the interest of the MCI approach, for both practical and theoretical applications.

Regarding practical applications, it must be noted that it is, in general and regardless of the method employed, practically infeasible to consider a full description of a probability measure on B, for even a limited number of itemsets, because such a description requires the definition of 2 m -1 individual values a priori. In itemset mining, global models (i.e. probability distributions where B is defined by all items) are not considered directly in practical applications.

Instead, they are replaced by numerous small local models (where B is defined by a small subset of itemsets). If we are considering a large number of local MCI models, each of which are defined around 3 or 4 items, the algebraic method becomes highly relevant. Furthermore, the explicit computation of reduced algebraic expressions for MCI models can be useful from a theoretical perspective, as it may bring insight on the structure of these models. Notably, we have hope that the explicit computation of reduced algebraic expressions for MCI models based on the frequencies of all itemsets of size 1 and 2 for low values of m can help us determine an explicit algebraic formula for such models and provide an interesting alternative to Chow-Liu tree models [START_REF] Chow | Approximating discrete probability distributions with dependence trees[END_REF].

Lastly, the previous remarks only apply to the approach in which we try to compute an MCI model using the algebraic method in a generic case before considering a specific case (that is we perform computations in Q(f 1 , ..., f d ) before substituting the f i by their values). If we compute the MCI model using the algebraic method in a specific case (that is we perform computations directly in Q), the computation time is individually much lower than computing the generic case. Though we speculate that, for the computation of a specific individual case, the numerical method is faster still than the algebraic method, we have yet to perform comparisons between these two approaches.

As the algebraic method on a specific case performs better when the values for the numerators and denominators of the f i are small (which can notably be the case if the number of transactions is not too large), it is possible that the algebraic approach (eventually combined with an approximation scheme) may outperform the numerical method in a number of cases.

Conclusion

In this chapter, we have presented our own approach towards MaxEnt models in the context of itemsets: mutual constrained independence models. We have demonstrated how this perspective sheds further light upon the rationale of such models and described a new approach towards their computation, based on tools for algebraic geometry. This approach has allowed us to determine exact algebraic expressions for all the MCI models when m ≤ 4, as well as for the MCI models defined by the frequencies of all proper subsets of an itemset for any m. These expressions allow for an increase in the computation speed of the corresponding MCI models by several orders of magnitude in comparison to standard methods for computing MaxEnt models. We are hopeful that further research based on this approach might help determine algebraic expressions for a wider range of models and we would like to investigate the issue of determining algebraic expressions for MCI alternatives to Chow-Liu tree models. Note also that, although we have defined the constraints of MCI models as constraints on the frequencies of itemsets, the results we have shown can easily be generalized to a much wider range of constraints. In particular, they still hold if we replace itemsets by any sound and complete family of patterns in B (see section 3.6.2.5) as this would only modify the expression of the transfer matrix T (see section 4.1.1.2).

MCI models correspond exactly to the type of models that define the objective data-driven hypotheses which we have described in chapter 3 (see section 3.7.3 in particular) and, as such, represent a central notion in our general framework for objective frequency-based interesting pattern mining. In the next chapter, we present the corresponding pattern mining algorithms.

CHAPTER 5

Extracting objectively interesting patterns from data

In the current and final chapter of this doctoral thesis, we present pattern mining algorithms based on the principles for a meaningful mathematical modeling of objective interestingness in patterns described in chapter 3, as well as the MCI models presented in chapter 4.

Recall from chapter 4 that we can include the scientific method within the mathematical modeling for the pattern mining process so that the pattern mining is seen as a process in which a scientifically valid hypothesis about the data is discovered. The hypotheses that we will consider in this chapter are mutual constrained independence hypotheses each of which is associated to a MCI model. Hence, each of the hypotheses considered corresponds to a set of itemsets with their associated frequencies. As such, the processes which we describe in this chapter fall within the field of itemset mining. However, as we have previously noted in section 4.3 regarding MCI models, such hypotheses could still be defined if we considered other types of patterns based on logical expressions. In particular, the contents of this chapter could easily be adapted if we replaced the set of itemsets I with any other sound and complete family of patterns (see section 3.6.2.5).

Proposition 5.1.1. The distribution of the χ 2 p,f statistic for the dataset given the MCI hypothesis asymptotically converges towards a χ 2 distribution with:

d + 1 -#D -#P L|D
degrees of freedom, where #D is the number of indices for which p i = 0 and #P L|D is the rank of the space generated by the polynomials in P L when setting

X i = 0 for all i ∈ D (see section 4.2.2.2).
Proof. The proposition is a straight forward application of the standard result by Pearson, while taking into account the number of degrees of freedom (see, for example, [START_REF] Buonocore | On the pearsonfisher chi-squared theorem[END_REF]). The only important aspect here is to be careful when counting the number of degrees of freedom, in order to take into account the possible border effects when D = ∅ so that d + 1 is reduced by #D + #P L|D rather than simply #P L .

Note that, because we cannot address every issue raised in this thesis, we do not provide a description of the rate at which the distribution of the χ 2 p,f statistic converges towards the χ 2 distribution. Moreover, the issue of the error in the approximation of the distribution of the χ 2 p,f statistic by a χ 2 distribution has been mostly set aside by researchers in the case of Pearson's test of independence as it is only significant when considering extremely small frequencies, [START_REF] Serra | Continuity correction of pearson's chi-square test in 2x2 contingency tables: A mini-review on recent development. Epidemiology[END_REF]. However, in this case, this issue could be more significant because of the possibly high dimension of the probability spaces considered and we believe it should be addressed eventually.

χ 2 test

We now define the χ 2 test of mutual constrained independence given K following standard methodology for χ 2 tests. Definition 5.1.2. Let α be a predefined probability threshold. We note H the MCI hypothesis for a dataset D defined by a set of itemsets K. We say that the hypothesis H is rejected by the χ 2 test of mutual constrained independence for the threshold α if:

χ 2 p,f > χ 2 α (d )
where d = d+1-#D-#P L|D is the number of degrees of freedom determined above and

χ 2 α (d ) is the value such that Prob (Z > χ 2 α (d )) = α if Z ∼ χ 2 (d ).
As for most statistical tests, it is important to note that a single MCI hypothesis corresponds in fact to three distinct statements, not all of which are tested equally by the statistical test. First, the hypothesis contains the statement that the we can model the data as n independent identically distributed samples of a random variable. Second, the hypothesis says that p |K = f |K .

And third, the hypothesis states that the items a 1 , ..., a m are mutually constrainedly independent in B with regards to the constraints defined by K for the measure p (see definition 4.1.2). The first statement is not tested per se by the statistical test because the test is only meaningful if such an assumption is made. The second and the third statements of the hypothesis, however, are both simultaneously tested by the test because they define the probability measure p which is compared to the empirical distribution f . This is not necessarily an issue, if ones aim is really to test the hypothesis as a whole. But it is an issue, if one is more interested in the third statement of the hypothesis (i.e. the mutual constrained independence of the items relative to K) than the second, which is quite often the case when researchers test for independence in current studies. Indeed, if the hypothesis H is rejected, this does not automatically imply that we would reject any alternative hypothesis based on a probability measure q such that the items a 1 , ..., a m are mutually constrainedly independent in B with regards to the constraints defined by K for this measure q but for which q K = f K . 2 One way to tackle this issue is to use the notion of confidence in the empirical distribution which we have defined in section 3.7.2.1. Indeed, if we are confident in the empirical distribution, then we consider the second statement to be true whether the hypothesis is rejected or not. Hence, in such a case, we can consider that rejecting H does indeed imply a rejection of the more general hypothesis that the items are mutually constrainedly independent relative to K.

In the following sections, we will use this statistical test as a tool for pattern mining. However, we also believe this tool could be put to use to assess single predefined hypotheses, as in standard statistical testing. In order to facilitate 2 To prove this, it is sufficient to show an example in which nχ 2 p,f > nχ 2 q,f . Such an example can be obtained easily by twitching the parameters of a MCI model. For example, consider two items a 1 and a 2 , an empirical distribution such that f = (0.4, 0.2, 0.3, 0.1) and the two following probability measures p and q both defined by the independence of a 1 and a 2 together with the constraints that p a1 = f a1 = 0.4 and p a2 = f a2 = 0.3, on the one hand, and q a1 = 0.401 and q a2 = 0.302, on the other hand. Then nχ 2 p,f ≈ 0.007937 while nχ 2 q,f ≈ 0.007922.

such a use, we are currently implementing a Python module for MCI testing in low dimension, based on the algorithms described in chapter 4.

Discovering a valid global MCI hypothesis

In this section, we will only consider global MCI hypotheses which are defined on the entire Boolean lattice B (see section 3.7.3.1 for more details on global hypotheses). Local MCI hypotheses will be discussed briefly in section 5.3.

We consider a binary dataset D as previously and we will assume that the number of transactions n in the dataset is sufficiently large to be confident in the empirical distribution. Furthermore, we will start by assuming that we have no prior knowledge about the data. Note that, together, these two assumptions imply that f i = 0 for all i ∈ 0, d (see section 3.7.2.2). We will address the issue of datasets for which some values of f i are null in section 5.2.4.

Valid MCI hypotheses

As we have expressed previously, our aim is to discover scientific information about the data in the form of a MCI hypothesis. If a hypothesis qualifies as scientific information about the data, we will say it is a valid hypothesis. Of course, a valid hypothesis should not be rejected if tested. However, passing a test is not sufficient to qualify a hypothesis as scientific information. In fact, there could be a large number of hypotheses that would not be rejected if they were tested and not all of these hypotheses should be considered valid hypotheses. For a hypothesis to be considered valid, we must be able to justify that we have a reason to test this hypothesis. This idea is related to the notion of the burden of proof which can be illustrated by Bertrand Russell's famous celestial teapot analogy [SK97]: even if the claim that there is a china teapot orbiting the sun between the Earth and Mars cannot be disproved, we should still ignore it because there are no grounds for such a claim.

In the classical hypothetico-deductive model for the scientific method (see section 3.7), hypotheses are formulated based on prior observations and reasoning. If a hypothesis is rejected, then a new hypothesis may be formulated, based on the accumulated knowledge resulting from the prior knowledge, as well as the observations which led to the rejection of the initial hypothesis.

In our case however, as we have discussed in section 3.7.1, the fact that we are considering finite static data implies that we cannot formulate a hypothesis based on prior empirical observations. However, we can rely on Occam's razor and define the simplest hypothesis possible. If this hypothesis is not rejected given the data, then the process ends there. Otherwise, we consider the simplest remaining hypothesis and we continue the process until we have determined the simplest possible hypothesis which is not rejected. In order to accomplish this, we must therefore define a total order relation between all possible hypotheses, based on a notion of simplicity and regardless of the dataset considered, as mentioned in section 3.7.4.1.

Ordering P(I)

Defining an order relation on all possible hypotheses regardless of the data considered comes down to defining an order relation on the powerset P(I). As we have mentioned above, the order defined should reflect a notion of simplicity (or, inversely, a notion of complexity) because we need to be able to compare two hypotheses in terms of simplicity to choose a hypothesis based on Occam's razor. This is not a trivial task and is linked to the issue of the complexity of Boolean expressions discussed in section 3.6.2.3. While we do not aim at providing a perfect solution to this problem, we suggest some criteria which may be taken into account and present an order relation defined accordingly.

Note that considering a set K and the set K ∪ { } are equivalent in terms of MCI hypotheses (because K * = K ∪{ } if f i = 0 for all i ∈ 0, d ). In fact, the order relation which we are considering need only be defined on {K * | K ⊂ I} which is equal to {K ∪ { } | K ⊂ I} here. Hence, all sets K considered in the following will be such that ∈ K.

A possible order relation

First, it seems legitimate to require that K ≤ K∪{I} for any K ⊂ I and I ∈ I.

In other words, adding more itemsets to the set of itemsets with constrained frequencies complexifies the hypothesis. Second, it also seems reasonable to require that K ∪ {I} ≤ K ∪ {J} for all K, K ⊂ I, I ∈ I \ K and J ∈ I \ K , such that K ≤ K and |I| ≤ |J|. Indeed, we can reasonably consider that an itemset I is less complex than an itemset J, if I has less items than J, and that this relationship should pass on to sets of itemsets. However, these two requirements are not enough to define a unique ordering on P(I) and we need to add other criteria.

One approach is to build on the idea that standard mutual independence is considered a rather simple case. If we require that the set corresponding to the independence model is the simplest possible one (given the two previous requirements) then the set { , a 1 , ..., a m }, which contains all the single items, is considered less complex than { , I} for any I ∈ I such that |I| ≥ 2. Hence, a single itemset of size 2 or more is considered more complex than any number of itemsets of size 1. We can then generalize this idea so that any single itemset of a given size is considered more complex than any number of itemsets with strictly smaller size.

Nevertheless, such a criteria is still not sufficient to define a total order relation on P(I) because it does not allow to compare two sets of same size, each of which contain itemsets of a same given size. However, if we consider that all itemsets of a given size are equally complex because items are interchangeable in terms of complexity, then we might consider that all sets of a given number of itemsets of a given size are equally complex3 . Hence, we can choose any total order relation between such sets and one of the simplest options is a double lexicographic order (itemsets of same size are ordered by lexicographic order and sets of itemsets of same size are ordered by lexicographic order based on the initial lexicographic order).

Formally, we can express the order described above as follows. For any K ⊂ I, we note:

K = i∈K 1 a i , ..., i∈Kr a i
and associate, to each K, sets C j defined for all j ∈ 0, m by:

C j = {K i | i ∈ 1, r ∧ |K i | = j}
Now, we can say that:

K (1) ≤ K (2)
if and only if:

∃i ∈ 0, m , C (1) i < C (2) i ∧ ∀j ∈ i + 1, m , C (1) j = C (2) j ∀i ∈ 0, m , C (1)
i = C (2) i ∧ ∃i ∈ 0, m , C (1)
i ≺ C (2) i ∧ ∀j ∈ i + 1, m , C (1) j
= C

(2) j where C

(1) i ≺ C

(2) i is given by the double lexicographic order described above.

Further discussions on the definition of an order relation

One of the issues with this order relation is that sets of itemsets corresponding to a same equivalence class do not all appear subsequently in this order (where the equivalence relation is defined by the set of permutations on items as in section 4.2.2.6). For example, when considering three items a 1 , a 2 and a 3 , the order defined above gives:

{ , a 1 , a 2 , a 1 ∧ a 2 } < { , a 1 , a 2 , a 1 ∧ a 3 } < { , a 1 , a 3 , a 1 ∧ a 3 }
where { , a 1 , a 2 , a 1 ∧ a 2 } and { , a 1 , a 3 , a 1 ∧ a 3 } belong to a same equivalence class and { , a 1 , a 2 , a 1 ∧ a 3 } belongs to a different one. This is because we have considered that the complexity of sets of a given number of itemsets of a given size is constant so that we can choose to order such sets in any given way. One could consider instead that complexity is only constant on the equivalence classes described above, so that sets in any given equivalence class should appear subsequently for the total order which we define.

Such an approach could bring the notion of complexity which we are trying to model closer to a notion of Kolmogorov complexity because the computational complexity for obtaining the MCI model from the algebraic reductions presented in section 4.2.2 is the same for all the sets K in any given equivalence class. Note that, if the order relation is defined based solely on the computational complexity described above, the condition that any single itemset of a given size be considered more complex than any number of itemsets of strictly smaller size would no longer hold. Indeed, the case K = I \ {I d }, which we have studied in section 4.2.1, corresponds to a relatively easy case in terms of computational complexity, while it necessarily holds a median position in the

Comprehensive search

A naive approach would be to test all potential hypotheses, in increasing order, until a valid hypothesis is determined. This is practically infeasible for all datasets for which m ≥ 7 (and still extremely difficult for m = 6) because the number of hypotheses to test is equal to 2 2 m -1 . However, without any firm mathematical result on the behavior of the χ 2 statistic with respect to K (which we will note χ 2 K in the following), it is impossible to suggest an alternative which would be both more efficient computationally and guaranteed to succeed (even if only asymptotically almost surely). This is why limiting the search to a smaller number of hypotheses, as described in the last example in section 5.2.2.2 in which the search is limited to 2 m hypotheses, could be considered an option. We can even reduce the size of the search space further by another logarithmic factor if we start by { } (which is the complete layer of itemsets of size 0) and incrementally complexify the hypothesis by adding the next complete layer of itemsets of a given size at each iteration. In such a case, the size of the search space is reduced to m + 1, which is a tremendous gain in terms of computational complexity, but we must not forget that this also corresponds to a tremendously simplified version of the problem.

Greedy algorithms

Alternatively, we could rely on various heuristics to obtain an eventually suboptimal solution. Greedy algorithms offer a simple and quite satisfying option.

In the following, we consider two different greedy approaches, which can be combined together: greedy-up and greedy-down. In the algorithms presented below, we will use the following indicator:

ξ K = χ 2 K χ 2 α (2 m -1 -|K|)
which is greater than one if and only if the hypothesis associated to K is rejected for the threshold α.

The greedy-up approach is detailed in the following algorithm:

1 . i n i t i a l i z e K ← ∅ ;

2 . w h i l e ξ K ≥ 1 and ∃I ∈ I \ K, ξ K∪{I} < ξ K : lution in 998 cases, leaving but 2 cases in which the optimal solution was not reached. These are extremely encouraging results, not only because the greedy-up, greedy-down approach manages to reach the optimal solution in 99.8% of the cases, but also because the general method allows to determine the exact nature of the generating model in 96.2% of the cases observed and a very close model in 100% of these cases.

Naturally, the tests which have been conducted concern a particular type of datasets but we hope to be able to confirm the success of this method when applied on artificial datasets generated through more elaborate models.

Comparison with compression approaches

It has also been suggested in the literature that the sets of itemsets for a given database can be ranked by interestingness based on the compression scores of their associated MCI models (see section 3.7.4.1). In order to compare this approach with our own, we also used compression scores (specifically MDL and BIC as defined in [START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF]) to rank the sets of itemsets for each of the 1000 artificial datasets mentioned in section 5.2.3.3 above.

In every single case, the random coin toss model associated to { } was ranked best. Even when removing this specific model, the best model after this one was never associated to the independence model, despite the fact that the data was generated based on this model. Hence, for this particular example at least, our approach seems to be much more adequate. Further tests will be conducted on artificial datasets generated through more elaborate models in order to assess the efficiency of our approach in comparison to approaches based on compression scores on a wider scale.

Dataset with locally null frequencies

In the previous sections, we have only considered datasets D for which f i = 0 for all i ∈ 0, d . However in most datasets, there is a non-empty set D such that f i = 0 for all i ∈ D.

Theoretical issues

One of the important issues with such datasets is that, we can never be confident in the corresponding empirical distribution, unless we have background knowledge that states that p i = 0 for all i ∈ D. However, if we consider all potential hypotheses associated to MCI models, some of these will correspond to probability measures p such that p i = 0 for some i ∈ D. Hence, if we consider all possible MCI models, we are ignoring the background knowledge which is actually necessary to consider the process of extracting scientific knowledge from the dataset.

The second important issue related to such datasets is actually determining the corresponding background knowledge. Up to now, we have considered this to be an independent issue which must be settled before we examine the data. Indeed, it seems difficult to justify that we use the empirical distribution to determine background knowledge which is necessary to assess the confidence which we have in the empirical distribution. However, there are also theoretical grounds that justify that we do use the dataset to define the

set D = {i ∈ 0, d | p i = 0}.
Indeed, one can argue that it is in fact a stronger assumption to presume the possibility of the existence of a given eventuality compared to the assumption that this eventuality is impossible. This is related to the philosophical notion of the burden of proof which we have already evoked in section 5.2.1. If an eventuality has never been observed, then we should have some kind of reason to believe that it could be observed if we want to consider it as a possibility. With this perspective, an absence of knowledge leading to the possible existence of any given eventuality should correspond to the belief in the impossibility of this eventuality. Hence, if we have no prior knowledge whatsoever about a dataset of transactions on items, then it is the fact that we observe the transactions in the dataset that allows us to say that these particular transactions can exist. By contrast, we have no basis to assume that the transactions which are not present in the data can exist. Hence, only hypotheses corresponding to probability measures such that p i = 0 whenever f i = 0 should be considered. If we follow this theoretical argument, then D should be in fact defined from the data as D = {i ∈ 0, d | f i = 0} and this knowledge should be not be ignored during the pattern mining process.

Nevertheless, note that this is a delicate stance to uphold. Indeed, it mostly never occurs that we can claim to have a complete absence of knowledge which would lead to the possible existence of any transaction before its observation in the data. In fact, if we choose to build a dataset of transactions between items, this already means we have some reason to believe that some transactions are possible (because, if this is not the case, the dataset itself would be an impossible object) but this does not necessarily imply that we have reason to believe that all transactions are possible. In the classical market basket example, we have reason to believe that some transactions are possible even if we have not observed a single transaction because we know how retail stores work. However, this does not imply that we have reason to believe that a single transaction containing all the items in the store is possible and, in fact, we also know that such a transaction is impossible because of our knowledge of how retail stores work. Such a nuanced position, in between the two extremes defined by D = ∅, on the one hand, and D = {i ∈ 0, d | f i = 0}, on the other hand, cannot be meaningfully justified without an additional modeling layer, which would include a description of the knowledge we had on items prior to considering a dataset of transactions on these items. As addressing this issue goes way beyond the scope of this thesis, we leave it as open problem for further research.

Practical implications

Rather than complexifying the issue of finding a valid MCI hypothesis, considering a dataset for which D = ∅ can only make the process more easily computable. In fact, as we will show, this may even allow us to consider cases with a larger number of items m than what is technically feasible if D = ∅.

In the following, we will note t the number of different transactions existing within the dataset, so that t = 2 m -|D|. Furthermore, note that t ≤ n. In practice, the number of transactions n in a dataset is rarely exponential in the number of items m. Hence, for most datasets considered in itemset mining, t 2 m . The practical implications of this statement can be viewed with regards to three aspects.

Firstly, if we consider that D is determined from the data, then we must add the constraints that p i = 0 for all in i ∈ D when computing the probability

Using local MCI hypotheses

In sections 3.5 and 3.7, we addressed a number of theoretical issues pertaining to the use of local models and advised mostly against their use if the meaningfulness of the modeling for the pattern mining process is a main concern.

Oppositely, it is practically infeasible to compute multiple global MCI models and perform the corresponding χ 2 tests of mutual constrained independence in order to discover a valid global MCI hypothesis for datasets with values of m as low as 20 (at least if D = ∅). For this reason, some form of compromise should be made if we want to consider such datasets and replacing global MCI hypotheses with local MCI hypotheses is one way to go. As this comes to the cost of meaningfulness, which is a central issue in this doctoral thesis, we will not delve much on such an approach. We present in this section some of the theoretical issues related to an approach based on local MCI hypotheses, and layout the bases for future work on this topic.

In the following, we will assume that we must limit the number of different items which can be considered to define a MCI model to a fixed value l max for technical reasons. Furthermore, we will consider a dataset D of n transactions on m items, with m > l max , so that we cannot define any global MCI models based on the empirical distribution defined by D. The local MCI models which we will consider instead will all be defined as probability measures p over a Boolean lattice B associated to a subset A of the set of items A = {a 1 , ..., a m }.

We will note l the number of items in A and we necessarily have l ≤ l max .

Theoretical issues

For each of the subsets A ⊂ A which we consider, we can use the methods described in section 5.2 to determine a valid local hypothesis4 H if such a hypothesis exists. Hence, we can obtain a set of valid local hypotheses, noted H, which is uniquely determined by the dataset D and the set of subsets A which we have considered, which we will note A. As our aim is to extract some piece of scientifically valid information about D generally, we must construct a valid global hypothesis H which is consistent with the hypotheses H in H.

This raises two issues: the consistency of H and the validity of H.

The issue of overlapping and global consistency

The issue of global consistency, already mentioned in sections 3.5.1.2 and 3.7.4.2, is a complex issue which we will not settle in general in this section.

We will limit the discussion here to how inconsistency might arise and how it can be simply avoided.

If A is not a partition of A, that is, if two elements of A , A ∈ A overlap, then it is possible that an MCI model p defined over B and an MCI model p defined over B are not globally consistent, in the sense that there is no probability measure p on B such that the restriction of p to B is equal to p and the restriction of p to B is equal to p (see section 3.5.1.2 for an example of this type of inconsistency). This issue does not arise when considering nonoverlapping elements of A (i.e. A and A such that A ∩ A = ∅), as we can at least consider a joint independent model as in section 4.2.2.4. As each valid local hypothesis H corresponds to a local MCI model, if we consider that A is a non-overlapping covering of A (in other words a partition of A), then we avoid the issue of inconsistency altogether.

Note, however, that it is not obvious that we could build an example of inconsistency between valid local MCI hypotheses as easily as we have con-structed an example of inconsistency between local MCI models. Indeed, we would expect that one at least of the two local MCI hypotheses corresponding to two local MCI models would be rejected if these two models were globally inconsistent, because this is asymptotically the case and we are considering datasets for which we are confident in the empirical distribution. Nevertheless, without further mathematical knowledge with regards to this specific aspect, we will restrict ourselves to cases in which A is a partition of A.

The issue of multiple testing and global validity

Global validity is another important issue which can be related to the effects of simultaneously testing a large number of hypotheses. The larger A is, the higher the likelihood that some of the elements in H correspond actually to false positives. This aspect refrains us from being able to assert that the hypotheses in H are globally valid. While this issue has been studied in other contexts, it has not been addressed with regards to MCI hypotheses (or equivalent hypotheses based on MaxEnt models) other than those associated with independence (see, for example,[LTP06, Han11, KIA + 17]). We set this topic aside here, leaving it to be investigated in further research.

Note that, even if we are certain that all the local hypotheses in H are considered valid, this does not imply that any global hypothesis H, consistent with the local MCI hypotheses H in H, can also be considered valid. In particular, even a global MCI hypothesis which is consistent with a great number of valid local MCI hypotheses can be rejected.5 Hence, we cannot define a valid global hypothesis H from H alone.

Thoughts on the partition model

As we have mentioned above, global inconsistency is no longer an issue if A is a partition of A, regardless of the partition A chosen. Hence, considering a pattern mining approach based on local MCI models associated to such a partition seems to be an interesting perspective. However, meaningfully determining an appropriate partition is not a trivial task.

Indeed, if we would randomly partition A in blocks A of size less than l max , the hypotheses H associated to each block would be globally consistent.

However, we would not know how we should join the local probability measures p together to make a global probability measure p. Furthermore, the information within each partition would not necessarily be the most interesting because items would not be grouped on the basis that they are strongly connected through a complex pattern. Hence, the most interesting patterns could be hidden within the definition of the junction model.

A simple junction model

Ideally, the partition of A should be determined from the data, concomitantly with a means to construct a junction model p from the local models p , and in such a way that the local models in each block of the partition contain the most complex aspects of the structure of p. Intuitively, if the complexity of the general model is concentrated in the local models, then the junction model should be quite simple.

Given two models p and p , defined for two disjoint blocks A and A , we can consider two simple junction models:

• the joint independence model;

• the joint incompatibility model.

On the one hand, the joint independence model corresponds to the model defined in section 4.2.2.4. On the other hand, the joint incompatibility model is defined by p a ∧a = 0 for all a ∈ A and a ∈ A .

More generally, we can define the joint mutual independence model for any number of models corresponding to mutually disjoint blocks by analogy with the definition of mutual independence. Similarly, we can also define a joint mutual incompatibility model. Lastly, we can define a combined junction model so that local models p can be joined in groups of joint mutually independent models, each of which can then be joined through a joint incompatibility model.

Determining the right partition

Partitioning the set of items A is effectively equivalent to variable clustering.

Such clustering methods have been considered in the context of association rule mining [PNS + 07] and similar methods could be envisaged in the present context. Note however that, for the sake of meaningfulness, the choice of a given clustering method should be consistent with a chosen junction model.

If a joint incompatibility model is considered, the clustering should reflect the notion that the frequency of any itemset spanning over multiple clusters is negligible. A very simple option would be to consider a graph such that:

(1) vertices are identified with items; (2) there is an edge between two vertices if and only if the frequency of the itemset defined by the two corresponding items is above a given support threshold (under which the frequency of an itemset is considered negligible). We can then identify each cluster of items to a connected component of the graph. The Apriori principle ensures that the frequency of any itemset spanning over multiple clusters is lower than the given support threshold and can therefore be considered negligible. Note that the value of the support threshold should be the lowest possible value that guarantees that the size of each cluster is less than l m in order to achieve computability while limiting the cost to meaningfulness.

Similarly, if a joint independence model is considered, the clustering method should ensure that the joint independence model between the local empirical distributions for each cluster may be considered a decent approximation of the global empirical distribution. This is a more complex issue than in the case of the joint incompatibility model and we leave it for further investigation.

Furthermore, in order to determine an adequate partition for a combined junction model, a clustering method adapted to the joint incompatibility model can easily be combined with a clustering method adapted to the joint independence model in a two phase process, with the initial clustering phase corresponding to the joint incompatibility model between the clusters and the second clustering phase corresponding to local joint independence models between subclusters within each initial cluster. In this manner, we can hope to reduce the complexity of determining an adequate clustering for a joint independence model by setting a cap on the size of the initial clusters.

Alternately, the choice of the partition could also rely on background knowledge about the items and their interactions, which brings us once again to the necessity of combining induction with other forms of inference.

Conclusion

In this chapter, we presented a novel approach for mining objectively interesting patterns in a dataset of transactions on items. The principle of this approach may be summarized as follows.

The patterns considered are sets of itemsets, together with their corresponding frequencies in the empirical distribution. Each pattern is associated to an objective hypothesis stating that the data is the result of the sampling of n independent identically distributed random variables whose distribution is given by the MCI model defined by the pattern. Hypotheses are then individually tested, following an increasing order of complexity, using a χ 2 test of mutual constrained independence. The process terminates as soon as it reaches a hypothesis which is not rejected by the test. Following Occam's razor, this particular hypothesis gives the objectively interesting pattern which we extract from the data.

Note that this approach relies on a total ordering of the hypotheses based on their complexity. Defining such an ordering is not necessarily trivial and we presented a possible solution to this problem.

Furthermore, the search space which contains all objective hypotheses has cardinality 2 2 m -1 . This is much too large to consider an exhaustive search beyond m = 5 and we must resort to heuristics instead to determine a (possibly suboptimal) solution. Considering artificial datasets with m = 5 (thus allowing for an exhaustive search), we experimented with greedy approaches.

We showed that a combined approach of a greedy-up algorithm, followed by a greedy-down algorithm, can produce very satisfying results, reaching the optimal solution in the vast majority of cases. These first experiments on artificial datasets are also encouraging with regards to the aim of the approach as the generating models for the artificial data were correctly identified in a large majority of cases. From this perspective, our approach surpasses by far the existing pattern mining approaches using MaxEnt models for summarization which systematically failed to determine these generating models.

Nevertheless, it is important to note that the approach described above is limited to small values of m (roughly less than 20), even when considering a greedy search algorithm. In order to consider possible applications for datasets with larger values of m, it must be combined with other approaches. We presented the contours of an approach based on the partitioning of the set of items in small independent or incompatible clusters which we intended to develop in the near future.

More generally, much of the work presented in this chapter calls for further investigation and we intend to pursue our research accordingly (see section 6.1 for more details).

methods from numerical analysis, including an analysis of the variations when considering different constrained sets and different distributions.

• An empirical study of the performances of the algebraic method for computing MCI models without pre-computed generic cases versus standard methods from numerical analysis.

Regarding the research presented in chapter 5, we will concentrate on the following five elements:

• An evaluation of the ability of the greedy-up, greedy-down algorithm to correctly identify generating models with higher levels of complexity and a comparison of the results obtained with a wider range of pattern mining methods.

• The experimentation of the method on real data (with low values of m) and a comparison with other pattern mining methods on this aspect.

• The experimentation of the method on real data with higher values of m after reducing this value using feature selection and a comparison with other pattern mining methods on this particular aspect.

• The development of a clustering method adapted to the combined junction model as described in section 5.3.2.2, the experimentation of the corresponding search algorithm on data with large values of m and a comparison with other pattern mining methods on this specific aspect.

Finally, a number of elements have been coded and obtained through computation in this thesis. We are working on making some of these elements both freely available and intelligible, in the form of a Python 3 module including implementations for:

• The algorithms for computing MCI models based on the algebraic approach while performing computations in Q.

• The algorithms for computing reduced algebraic expressions for generic MCI models while performing computations in Q[f 1 , ..., f d ].

• An algorithm for computing any MCI model when m ≤ 4 based on a database containing the corresponding pre-computed algebraic expressions.

• The iterative scaling algorithm for computing MCI models.

• The χ 2 test of mutual constrained independence.

• The greedy-up, greedy-down algorithm for discovering a valid MCI hypothesis.

The artificial scientist

During the twentieth century, the idea that the scientific method could not be logically formalized or, in other words, mathematically modeled became commonly accepted within the philosophy of science community [START_REF] Glymour | The evaluation of discovery: Models, simulation and search through "big data[END_REF]. This I said above that the work of the scientist consists in putting forward and testing theories. The initial stage, the act of conceiving or inventing a theory, seems to me neither to call for logical analysis nor to be susceptible of it. The question how it happens that a new idea occurs to a man -whether it is a musical theme, a dramatic conflict, or a scientific theory -may be of great interest to empirical psychology; but it is irrelevant to the logical analysis of scientific knowledge.

As we have moved well into the twentieth century, it is time to consider such positions outdated. The complexity of the task of elaborating a meaningful and objective approach towards automatic discovery in science should not lead us to discard the process as inherently subjective. In fact, throughout the entire course of this thesis, we have worked towards the goal of conceiving a meaningful and objective approach for producing scientific information.

In chapter 3, we focused on the modeling that underlies such an approach.

In chapter 4, we studied some of the mathematical tools on which it relies.

In chapter 5, we concentrated on its algorithmic aspects. Of course, the algorithms which we have presented in this last chapter represent but a very small step towards the definition of a general artificial scientist. For one part, the world as seen by our algorithms is extremely simple as it corresponds to no more than the empirical distribution of transactions on a limited number of items. And for another part, the scope of the potential the hypotheses that they can produce is also quite limited. Nevertheless, the results we have obtained through the implementation of our greedy-up, greedy-down algorithm (see sections 5.2.3.2 and 5.2.3.3) seem to show that, at least within this limited context, our approach performs very well.

In order to move forward towards the aim of developing an artificial scientist, the big challenge now is scaling up. Not so much in terms of the number of possible attributes considered, but in terms of the complexity of the representation of the observable world and the complexity of the hypotheses and theories that describe it. In [START_REF] Carl | Thoughts on the limitations of discovery by computer[END_REF], Carl Hempel argued that a computer program may only discover hypotheses within the limits of the vocabulary of its language, while "the formulation of powerful explanatory principles, and especially theories, normally involves the introduction of a novel conceptual and terminological apparatus". Similarly, our study of the practical limitations on the number of attributes which can be considered in our approach (see section 3.7.2.2) and our discussion on the issue of pattern complexity (see section 3.6.2.3) both lead us to say that an artificial scientist cannot consider more than a limited number of attributes unless it is capable of complexifying its native vocabulary. However, this does not lead us to conclude on the impossibility of the artificial scientist: it simply tells us in which direction we must pursue our research.

Contexte de recherche

3. 1 . 1

 11 Modeling and mathematical modeling . . . . . . . . . . . 3.1.1.1 Modeling as translation . . . . . . . . . . . . . 3.1.1.2 Mathematical modeling . . . . . . . . . . . . . 3.1.1.3 Complex modeling processes . . . . . . . . . . . 3.1.2 Chapter outline . . . . . . . . . . . . . . . . . . . . . . . 3.2 The data: subject or object of the modeling process . . . . . . . 3.2.1 The data modeling process in the case of objective interestingness measures for rules . . . . . . . . . . . . . . 3.2.2 The data modeling process when considering the fixed row and column margins constraint . . . . . . . . . . . . 3.2.3 Recommendation . . . . . . . . . . . . . . . . . . . . . . 3.3 Phenotypic modeling and genotypic modeling of interestingness 3.3.1 Phenotypic and genotypic modeling: a definition . . . . . 3.3.2 Phenotypic and genotypic approaches for measuring objective interestingness . . . . . . . . . . . . . . . . . . . . 3.3.2.1 Phenotypic approaches for defining objective interestingness measures . . . . . . . . . . . . . 3.3.2.2 Genotypic approaches for modeling interestingness . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 When phenotypic modeling meets genotypic modeling: modeling information . . . . . . . . . . . . . . . . . . . . 3.3.3.1 Phenotypic approaches for modeling entropy . . 3.3.3.2 Genotypic approaches for modeling information 3.3.4 Recommendation . . . . . . . . . . . . . . . . . . . . . . 3.4 Pragmatic modeling . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 Pragmatic modeling of interestingness . . . . . . . . . . 3.4.2 Meaningfulness first, computability second . . . . . . . . 3.4.3 Recommendation . . . . . . . . . . . . . . . . . . . . . . 3.5 Patchwork and holistic modeling processes . . . . . . . . . . . . 3.5.1 Patchwork modeling in interesting pattern mining . . . . 3.5.1.1 Type 1 patchwork modelings (PW1) . . . . . . 3.5.1.2 Type 2 patchwork modelings (PW2) . . . . . . 3.5.2 Recommendation . . . . . . . . . . . . . . . . . . . . . . 3.6 Mathematical modeling of patterns . . . . . . . . . . . . . . . . 3.6.1 Measure spaces and Boolean lattices . . . . . . . . . . . 5.3.2.1 A simple junction model . . . . . . . . . . . . . 5.3.2.2 Determining the right partition . . . . . . . . . 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Epilogue 6.1 Follow-up research . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 The artificial scientist . . . . . . . . . . . . . . . . . . . . . . . . 7 Extraction objective et signifiante de motifs intéressants sur la base de leur fréquence CHAPTER 1 Foreword 1.1 The two cultures of statistical modeling In 2001, Professor Leo Breiman described "Two Cultures" of statistical modeling: the data modeling culture, on the one side, and the algorithmic modeling culture, on the other side [B + 01]. According to Breiman, both cultures study

  O'N16, RS17]. More recently, on the 27th of March 2019 the Association for Computing Machinery awarded the 2018 ACM Alan Mathison Turing Award to Yoshua Bengio, Geoffrey Hinton and Yann LeCun the "Fathers of the Deep Learning Revolution" [ACM], a revolution which led to the proliferation of algorithmic black boxes as described by Leo Breiman [VBB + 18b].

Figure 2 . 2 :

 22 Figure 2.2: The association rule lattice (one lattice for each frequent itemset of size 2 or more).

Figure 2 . 3 :

 23 Figure 2.3: Contingency tables for absolute and relative frequencies.

  estingness measures: Subject (Who evaluates?); Object (What is evaluated?); Unit (By how many objects?); Criterion (Based on what criterion?); and Theory (Based on what theory?). The last theoretical factor was explicitly linked to the meaningfulness of the measures and a list of 39 measures were classified into five different categories depending on what they represented: N (Number of instances included in the antecedent and/or consequent of a rule); P (Probability of the antecedent and/or consequent of a rule); S (Statistical variable based on P); I (Information of the antecedent and/or consequent of a rule);

Figure 2 . 5 :c

 25 Figure 2.5: In this representation of tidsets using a Venn diagram all eight subitemsets of {a, b, c} are closed.

  Figure 2.7: The eight generalized itemsets of size three based on itemset {a, b, c} correspond to the eight disjoint areas in the Venn diagram.

  ) ≥ δ X (I) if |I \ X| even Hence, supp (I) ∈ [LB(I), UB(I)] where: LB(I) = max X⊆J⊂I |I\X| even (δ X (I)) and: UB(I) = min X⊆J⊂I |I\X| odd (δ X (I))

  and the idea by Herbert Stachowiak in[Sta73] that: all of cognition is cognition in models or by means of models, and in general, any human encounter with the world needs a "model" as the mediator [...]. 2

  Figure 3.1: Modeling as a translation

  Figure 3.2: Mathematical modeling

[ GMMT07 ,

 GMMT07 HOV + 09, LPP14] but also commonly used when considering Max-Ent models [KDB10, TM10, DB11, MVT12, MV13]. We show here how considering this model may hinder the ability to meaningfully generalize the obtained results.The swap-randomization methods in the literature mentioned above all share a common genealogy with a general problem from discrete tomography: describe the space of all n × m binary matrices with given row and column margins[START_REF] Gabor | Advances in discrete tomography and its applications[END_REF][START_REF] Gabor | Discrete tomography: Foundations, algorithms, and applications[END_REF]. It has been demonstrated that this space can be represented via a graph, in which the vertices represent the matrices and the edges represent a swapping operation between two matrices, composed of a single connected component[START_REF] Brualdi | Matrices of zeros and ones with fixed row and column sum vectors[END_REF]. This representation provides for a mean to utilize methods such as random walks on graphs to randomly generate a matrix satisfying the same conditions on the margins as a given initial matrix.Swap-randomization methods have been used to model data particularly in the domains of ecology [SMS98, CDHL05, SNB + 14] and psychometrics[START_REF] Ponocny | Nonparametric goodness-of-fit tests for the rasch model[END_REF][START_REF] Chen | Exact tests for the rasch model via sequential importance sampling[END_REF][START_REF] Verhelst | An efficient mcmc algorithm to sample binary matrices with fixed marginals[END_REF].

[ Pon01 ,

 Pon01 [START_REF] Chen | Exact tests for the rasch model via sequential importance sampling[END_REF][START_REF] Verhelst | An efficient mcmc algorithm to sample binary matrices with fixed marginals[END_REF] or the corresponding MaxEnt model: the Rasch model [Mas82, Kel84, KKB+ 17]. However, it must be understood that the associated modeling processes do not provide any basis for generalizing the results obtained to a larger pool of potential examinees.

Figure 3

 3 Figure 3.3: Patchwork modeling

  ematical formalism. Well defined holistic mathematical modeling are often behind the great paradigm shifts in science and technology. Considering the World Wide Web as a single oriented graph with web pages as nodes and hyperlinks as vertices is what allowed Sergei Brin and Larry Page to define the PageRank algorithm [BP98]. Thanks to this holistic modeling they understood that the general structure of the web pointed preferentially towards certain pages and their algorithm revolutionized the world of search engines.3.5.1 Patchwork modeling in interesting pattern mining 3.5.1.1 Type 1 patchwork modelings (PW1)

Figure 3

 3 Figure 3.4: Hypothetico-deductive model for the scientific method

  in order to be confident in f to the precision ε and to the degree α. As we have already established that we must have ε < 2 m min 0≤i≤d f i , we can consider η 1 = ε 2 m min 0≤i≤d f i and η 2 = η 1 . Note that this means that e = max 0≤i≤d f i min 0≤i≤d f i ε ≥ ε which gives us the condition δ(p, f ) ≥ ε. Hence, we have a first lower bound a α,f for n α,f which depends on the empirical distribution f :

n

  Figure 3.5: Upper and lower bounds for n α,f and exact values for the uniform distribution for α = 0.99 and ε = 0.001.

  so the condition becomes ln(Z) ∈ Ker(T K * ) ⊥ . By the strict concavity of ϕ, we conclude on the uniqueness of such an optimum thus obtaining the desired result. Strong version of Theorem 4.1.1 and proof. Lemma 4.1.2 is central in the proof we provide to Theorem 4.1.1. As stated previously, this proof is constructive, leading to the following stronger result. Theorem 4.1.3 (MCI convergence theorem, strong version). Let C = K, f |K be a constraint system on B and g (k)

  Definition 4.1.2 (Mutual constrained independence). Consider a probability measure p on B and a set of itemsets K ⊂ I. Let X = representation of p in the basis Ω. We say that the items a 1 , ..., a m are mutually constrainedly independent in B with regards to the constraints defined by K, if and only if ln (X) ∈ Ker (T K * ) ⊥ . (See notations preceding lemma 4.1.2 for the definition of ln.)Note that this definition is not restricted to the context of itemsets and to applications in data mining. It applies more generally to the field of probabilities, as any finite family of events A 1 , ..., A m of a probability space can naturally be associated to a set of items a 1 , ..., a m . It is a straight forward generalization of the notion of mutual independence. Indeed, the mutual independence of m items corresponds to the mutual constrained independence of these items with regards to K = {a 1 , ..., a m }. It is therefore quite natural to consider statistical tests for mutual constrained independence similarly as the well known tests of independence performed by statisticians. This implies that one might define a statistical MCI model from a dataset in the same fashion as one defines an independence model. Definition 4.1.3 (MCI model). Let f be a probability measure on B defined as the empirical distribution of a dataset of transactions on items and K ⊂ I be a set of itemsets. The MCI model for the data defined by K is the probability measure p defined by its vector representation X =

  -(4) gives 1 = 0 and, by contradiction, we now know that xk = xk for all (k, k ) ∈ (V + ) 2 or, equivalently, xk = xd for all k ∈ V + . Now, suppose that y k = 0 for some k ∈ V -. If k = d -1, this implies:

  have thus shown that both xk = xd for all k ∈ V + and ŷk = 0 for all k ∈ V -. Therefore, as f (ẑ) = xk -ŷk for all k ∈ V + and f (ẑ) = ŷk -xk for all k ∈ V -, we have ŷk = ŷd for all k ∈ V + and xk = ŷd -xd for all k ∈ V -. Moreover, from h d+2 (ẑ) = 0, we get d+1 2 ŷd = 1 and fromh d (ẑ) = 0, xd+1 2 d -(ŷ d -xd ) d+1 2 . This solves easily to (x d , ŷd ) = 1 d + 1 , 2 d + 1This implies that ẑ = z and, therefore, min f (z) = -1 d+1 . Moreover, the symmetries of the problem induce similarly that max f (z) = 1 d+1 . Hence,max |f (z)| = 1 d + 1This concludes the proof of the proposition as the set of vectors z that satisfy the constraints given by the optimization problem identifies to the set of tuples (p, f ) where f is a probability measure on B and p is the corresponding MCI model where the constraints are taken on all proper subitemsets of I d .

  and we require a formal representation of Q(f 1 , ..., f d ). While the latter option implies more elaborate programming, and calculations in Q(f 1 , ..., f d ) may, in this case, represent the computational bottleneck of the general process, it does allow us to determine a definite univariate representation which can be used for any MCI model corresponding to a given K * .

  3 . add L j t o P L ; Algorithm 4.1: Computing P L Loglinear part. Secondly, let Y = Ker (T K * ) ∩ Z d+1 . Then, we can define the following polynomial:

  t i a l i z e Y L ← () ; 3 . f o r j from 0 t o dj = 1 , y j = -1 ; 9 . i f Y ∈ Ker (T K * ) : 1 0 . add j t o J ; 1 1 . add Y t o Y L ; Algorithm 4.2: Computing Y L Then, we need to add a family Y N L to Y L , corresponding to the strictly non-linear part of P M , in order to define Y. To do this, we can complete Y L to form a basis of Ker (T K * ) ∩ Z d+1 , initialize Y to be equal to this basis and then incrementally add vectors to Y until S = ∅ as described previously. Notice that, in this process, it suffices to consider the subset T = {S ⊂ 0, d | (s = s -1) ∧ (Z ⊂ Z M )} of S rather than S because T = ∅ necessarily implies S = ∅. Furthermore, there is no need to iterate more than once over the elements of {S ⊂ 0, d | s = s -1} because T decreases whenwe add elements to Y. Hence, the outline of the algorithm becomes as follows:1 . i n i t i a l i z e Y ← Y L ; 2 . complete Y t o form a b a s i s o f Ker (T K * ) ∩ Z d+1 ; 3 . f o r S ∈ {S ⊂ 0, d | s = s -1} : 4 . i f Z ⊂ Z M :5 . c h o o s e Y a p p r o p r i a t e l y ; 6 . add Y t o Y ; Algorithm 4.3: Computing Y N L The issue of choosing Y in step 5 of the previous algorithm can be resolved as follows: 1 . c o n s i d e r th e matrix B such t h a t each row c o r r e s p o n d s t o an element from B K * ; 2 . r e o r d e r th e columns o f B so t h a t th e f i r s t s columns c o r r e s p o n d t o th e columns with i n d i c e s i n S ; 3 . r e d u c e B t o i t s reduced row e c h e l o n form ; 4 . s e t Y t o t he l a s t row o f B ; 5 . r e a r r a n g e th e columns o f Y back t o th e o r i g i n a l o r d e r o f i n d i c e s ; Algorithm 4.4: Computing Y Furthermore, the cardinality of Y may eventually be reduced as it can contain a subfamily which satisfies the condition from lemma 4.2.1. We reduce the size of Y using a greedy algorithm: 1 . w h i l e ∃Y ∈ Y N L such t h a t Y \ Y i s a g e n e r a t i n g f a m i l y o f v e c t o r s o f Ker (T K * ) and T = ∅ : 2 . remove Y from Y ; Algorithm 4.5: Reducing Y N L

  4.2.2.6 Algebraic solutions for all cases when m ≤ 4In section 4.2.2.1, we explained that the computations for determining a univariate representation may be performed in Q, based on specific rational values for f 1 , ..., f d , or in Q(f 1 , ..., f d ), based on formal values for f 1 , ..., f d . In the case in which formal values are employed, the univariate representation obtained for a given K * corresponds to a formal and simplified algebraic representation of the MCI model (for this given K * ). This representation can be stored allowing for a fast and precise computation of the corresponding MCI models given any specific values for f |K * .

1

  view was widely influenced by the positions defended by highly prominent philosophers such as Karl Popper and Hans Reichenbach. Indeed, both argued that, in the production of science, there is a clear distinction between the formulation of hypotheses and theories, on the one hand, and their evaluation, on the other hand, or, in Reichenbach's words, the context of discovery and the context of justification; and they both agreed that the initial creative aspect of science was an entirely subjective matter which could not be logically described[START_REF] Reichenbach | Experience and prediction: An analysis of the foundations and the structure of knowledge[END_REF] Pop59].1 In terms of artificial intelligence, this philosophical stance has important implications: if the process for discovering hypotheses cannot be mathematically modeled, then it cannot be implemented within the source code of an artificial intelligence, hence leading to the impossibility of conceiving an artificial scientist. In fact, the impossibility of developing a computer program that discovers hypotheses and theories is exactly the idea defended by another highly influential philosopher of science of the twentieth century, Carl Hempel, in Thoughts on the Limitations of Discovery by Computer [Hem85]. This position can be easily summarized by Popper's own words in The Logic of Scientific Discovery [Pop59]:

  Ce travail de thèse s'inscrit tout d'abord dans la continuité de travaux de recherche en fouille de données et en extraction de motifs et, plus particulièrement, de travaux sur l'extraction d'itemsets et de règles objectivement intéressants. En effet, malgré l'engouement général qui a suivi la publication des travaux de Rakesh Agrawal et son équipe sur l'extraction d'itemsets fréquents [AIS93, AS94a], il a été rapidement établi que les itemsets fréquents (et les règles d'associations qui en découlaient) ne présentaient, en tant que tels, qu'un intérêt limité. Les chercheur•e•s se sont donc penché•e•s sur la question de l'intérêt de ces motifs et, plus particulièrement, de leur intérêt objectif. Parmi les travaux de recherche, on discerne différentes approches générales: les mesures cherchant à quantifier l'intérêt objectif d'un motif [TKS04, GH06, LMVL08, LBLL12a]; les représentations condensées exactes d'une classe de motifs [AIS93, PBTL99, CG02]; les méthodes identifiant l'intérêt objectif à l'étonnement statistique [HOV + 09, LPP14] ou à l'informativité (au sens de la théorie de l'information) [MTV11, VVLS11, MVT12]. Toutefois, si la question de l'intérêt objectif des motifs extraits est centrale dans un nombre important de publications scientifiques dans ce domaine, la question de la signification du processus d'extraction a souvent été mise de côté comme si celle-ci était triviale voire secondaire. C'est pourtant une question fondamentale qu'il convient de poser car elle revient à expliquer en quoi on peut affirmer que les informations extraites des données par un tel processus présentent un intérêt objectif. Alors qu'un nombre croissant de voix s'élèvent aujourd'hui pour demander l'ouverture de la boîte noire et exiger une réelle transparence dans les prises de décisions assistées par des processus automatisés, il est essentiel que les chercheur•e•s qui développent ces méthodes soient pleinement capables d'en expliquer la signification [O'N16, RS17, VBB + 18b]. C'est pourquoi, nous avons d'abord cherché à prendre un peu de recul pour analyser les mécanismes de modélisation mathématique qui permettent d'expliquer clairement la signification d'un processus d'extraction de motifs dits objectivement intéressants. Notre travail s'intègre ainsi également dans une recherche académique en philosophie des sciences et ceci à deux titres: d'une part, sur la question de l'étude des modèles et des modélisations dans les processus scientifiques et technologiques [Min65, RWLN89, Sup07, Bok11, Pot17]; et d'autre part, sur la question de la création de connaissances objectives et, en particulier, sur la question de la formalisation et de l'automatisation de ce processus [Pop59, Hem85, GRZ19].Modélisations mathématiques et signification dans les processus d'extraction de motifsNous soutenons l'idée que la signification d'un processus reposant sur des modèles mathématiques, ainsi que la signification de ce qui en aboutit en sortie, découle directement de la signification du processus de modélisation mathématique sur lequel il s'appuie. C'est pourquoi il est important de rendre explicite ce processus de modélisation et de comprendre la manière dont les choix en termes de modélisation impactent et déterminent la signification du processus d'extraction de motifs dans son ensemble. En cherchant à caractériser les processus de modélisation mathématique dans le cadre spécifique de l'extraction de motifs objectivement intéressants, nous avons identifié des problématiques générales en modélisation mathématique qui dépassent le cadre de l'extraction de motifs. À ce titre, une partie de la recherche que nous présentons dans cette thèse peut être considérée comme une contribution en philosophie des sciences et nous définissons les contours d'un nouveau cadre pour l'étude et l'analyse qualitative des modélisations dans un processus scientifique. Nous présentons de nouvelles formulations pour définir et représenter les notions de modèle et de modélisation. En nous appuyons sur ces représentations, nous introduisons un certain nombre de concepts visant à caractériser les processus de modélisation, tels que: les notions de modélisation phénotypique et de modélisation génotypique; la notion de modélisation pragmatique; ou encore les notions de modélisation en patchwork et de modélisation holistique. Nous utilisons ce cadre sur différents exemples directement issus du domaine de l'extraction de motifs afin d'expliciter comment les choix de modélisation (implicites ou explicites) induisent des différences fondamentales (et même parfois inconciliables) en termes de signification entre les différentes approches étudiées en extraction de motifs. Notre démarche est volontairement sceptique et nous cherchons à questionner tous les choix qui peuvent être faits en termes de modélisation mathématique dans un contexte d'extraction de motifs objectivement intéressants. C'est ainsi que nous interrogeons le choix de l'itemset en tant que motif fondamental dans un tel contexte et nous apportons en réponse un certain nombre d'éléments objectifs qui permettent de justifier la pertinence toute particulière de ce choix. Par ailleurs, partant de l'idée que la recherche de connaissances objectives est une entreprise scientifique, nous examinons la possibilité d'intégrer une modélisation mathématique de la méthode scientifique, et plus particulièrement sa description via le modèle hypothético-déductif, dans la modélisation associée à un processus d'extraction de motifs. Nous établissons ainsi le lien entre notre problématique initiale et la question bien plus générale de la formalisation mathématique de la recherche scientifique en vue de son automatisation. Au premier abord, le contour particulier de notre problématique, partant d'un jeu de données statique et fini, semble peu adapté pour considérer le processus dynamique et sans fin qui est décrit par le modèle hypothético-déductif. Toutefois, nous montrons que l'introduction de nouveaux outils de modélisation permet de contourner cette difficulté. Nous introduisons ainsi la notion de confiance en la distribution empirique, sur laquelle nous nous appuyons pour décrire l'extraction de motifs objectivement intéressants comme un processus dans lequel des hypothèses sont successivement formulées puis évaluées. Notons que l'une des particularités de notre approche est qu'elle va permettre à un processus automatisé d'extraction de motifs de répondre qu'il y a insuffisamment de données pour conclure, ce qui est totalement attendu dans un processus de recherche scientifique, mais généralement absent dans les processus d'extraction de motifs. Enfin, notre analyse sur la complexité de notre démarche nous amène à conclure que, en dehors de cadres très restreints, un système intelligent doit nécessairement être capable de complexifier le langage qu'il utilise pour décrire le monde si l'on veut qu'il en fasse une description scientifique et objective. Nous proposons enfin, pour chacun des points d'analyse que nous considérons, un certain nombre de principes et de recommandations pour l'élaboration de modélisations mathématiques signifiantes dans le contexte spécifique de l'extraction de motifs objectivement intéressants sur la base de leur fréquence. Nous notons que les approches qui identifient l'extraction de motifs à de la compression de données [MTV11, MVT12, VVLS11] sont en concordance avec une majorité des recommandations que nous établissons. Une différence subsiste cependant sur la signification de ce qui est recherché. Là où ces méthodes cherchent à déterminer une compression optimale des données, nous cherchons plutôt à exhiber le mécanisme sous-jacent qui permet de générer les données. Si ces deux objectifs peuvent se rejoindre dans certains contextes, ce n'est pas nécessairement le cas a priori. Indépendance contrainte mutuelle En s'appuyant sur les principes que nous avons élaborés pour la modélisation mathématique des processus d'extraction de motifs objectivement intéressants, nous avons construit une notion d'indépendance contrainte mutuelle (abrégé MCI pour Mutual Constrained Independence). Cette notion permet de définir les différentes hypothèses objectives qui constituent les motifs objectivement intéressants (ou du moins potentiellement intéressants) dans notre approche : les modèles MCI. Comme nous le démontrons les modèles MCI que nous définissons sont mathématiquement équivalents à des modèles de maximum d'entropie (Max-Ent models) [Jay03]. De tels modèles ont d'ailleurs déjà été considérés en tant que tels dans la littérature en extraction de motifs objectivement intéressants, particulièrement chez celles et ceux qui identifient l'extraction de motifs objectivement intéressants dans les données à la compression optimale de ces données [MTV11, MVT12]. Toutefois, l'approche MCI diffère significativement de l'approche entropique par sa construction. En effet, les modèles MCI sont construits comme la solution asymptotique d'un problème de recherche du modèle moyen d'un ensemble de modèles satisfaisant une contrainte particulière, ce qui permet d'entrevoir le principe de maximum d'entropie sous un angle nouveau. Cette approche particulière de tels modèles de maximum d'entropie nous a permis d'exhiber de nouvelles propriétés de ces modèles ainsi que de nouvelles méthodes pour les calculer. Nous présentons, dans un premier temps, le cas simple dans lequel le modèle MCI est défini par des contraintes sur l'ensemble des sous-ensembles propres d'un itemset. Nous déterminons une expression algébrique exacte pour les modèles MCI de cette classe et analysons les propriétés de la distance entre des données empiriques et le modèle correspondant. En nous appuyant sur des algorithmes de géométrie algébrique réelle, nous généralisons cette approche au calcul de tout type de modèle MCI. Les différentes étapes de notre algorithme sont présentées en détails et nous montrons que celui-ci permet d'une part le calcul direct d'un modèle MCI défini par des données empiriques et d'autre part le calcul d'expressions algébriques réduites pour des classes de modèles MCI. L'utilisation de telles expressions algébriques calculées préalablement permet une réduction significative (d'un facteur 150 pour notre cas d'étude) du temps de calcul d'un modèle MCI, en comparaison avec la méthode de Darroch et Ratcliff [DR72] qui est une méthode de calcul numérique standard utilisée pour le calcul des modèles de maximum d'entropie équivalents en extraction de motifs [PMS03, TM10, MVT12]. Cet avantage doit être tempéré par l'impossibilité pratique de calculer les expressions algébriques réduites pour l'ensemble des classes possibles de modèles MCI. Toutefois, nous montrons que cette méthode peut être utilisée pour calculer des expressions algébriques réduites exactes pour tout type de modèle MCI défini sur un faible nombre d'items, ce qui lui permet tout de même de jouer un rôle important dans l'accélération de processus d'extraction de motifs objectivement intéressants. Algorithmes d'extraction de motifs En nous appuyant sur la recherche que nous avons menée sur les modèles MCI, nous montrons qu'il est possible de définir un test d'indépendance contrainte mutuelle. Nous utilisons alors ce test afin d'évaluer successivement des hypothèses d'indépendance contrainte mutuelle dans un algorithme d'extraction d'information scientifique sur les données. L'algorithme évalue d'abord si la taille de l'échantillon est suffisante afin d'avoir confiance en la distribution empirique puis, le cas échéant, évalue successivement les différentes hypothèses d'indépendance mutuelle contrainte par ordre de complexité croissante jusqu'à obtenir une hypothèse non réfutée qui constitue, selon le principe du ra-soir d'Ockham, le meilleur modèle explicatif pour les données dans l'état des connaissances disponibles. Cet algorithme peut être assimilé à un processus d'extraction de motifs objectivement intéressants car l'hypothèse d'indépendance mutuelle contrainte retenue est définie par un ensemble d'itemsets. Il faut noter, que la définition d'un ordre sur la complexité d'une hypothèse d'indépendance mutuelle n'est pas nécessairement évidente et nous engageons donc une discussion sur différentes façons de définir un tel ordre. Comme indiqué ci-dessus, l'algorithme a pour but la résolution d'un problème de recherche d'optimum: on cherche l'hypothèse la moins complexe qui n'est pas réfutée par un test. Or la taille de l'espace de recherche pour cette solution est doublement exponentiel en le nombre d'items et donc rapidement bien trop grand pour qu'une recherche exhaustive soit envisagée. Nous avons donc étudié différentes approches reposant sur une heuristique gloutonne pour remplacer la recherche exhaustive. En particulier, nous montrons la très grande efficacité de l'algorithme consistant à ajouter des contraintes sur des itemsets de manière gloutonne jusqu'à obtenir une hypothèse non rejetée, puis à en retrancher de manière gloutonne tant que l'hypothèse associée n'est pas rejetée. De par son élaboration, l'algorithme que nous avons présenté constitue un exemple fonctionnel de l'automatisation du modèle hypothético-déductif de la méthode scientifique. Certes, le contexte dans lequel il peut s'appliquer reste extrêmement restreint, mais il faut noter que l'idée même que l'on puisse envisager d'automatiser ce processus, et en particulier la phase d'élaboration des hypothèses à tester, est elle-même extrêmement récente [GRZ19]. En ce sens, l'approche que nous avons développée peut être vue comme un point de départ pour des recherches futures sur la problématique de l'automatisation de la création de savoir scientifique. Par ailleurs, les limites de l'étendue des applications pratiques de notre algorithme sont liées aux exigences extrêmement fortes que nous avons fixées sur la signification et l'objectivité du processus d'extraction. Nous discutons de ces limites et nous proposons un certain nombre d'adaptations qui permettent d'envisager l'utilisation des outils que nous avons développés dans des contextes moins restreints, au prix d'un compromis raisonnable sur ces exigences.

  Table 2.1: A toy database with four items and ten transactions.

	TID Transaction
	0	{a 1 , a 3 }
	1	{a 1 , a 2 }
	2	{a 1 , a 3 , a 4 }
	3	{a 3 , a 4 }
	4	{a 2 }
	5	{a 2 , a 4 }
	6	{a 3 }
	7	{a 1 , a 3 }
	8	{a 2 , a 3 , a 4 }
	9	{a 1 , a 3 }

  Properties of objective interestingness measures. In order to suggest the most adequate interestingness measures, researchers performed de-

	tailed analyses of the properties and performances of objective interestingness
	measures. Many different types of properties were considered, corresponding
	mostly to different views of what an objective interestingness measure should
	be.
	Algorithmic properties. The study of algorithmic properties of objec-
	tive interestingness measures corresponds to a very pragmatic view towards
	the definition of interestingness measures: if interestingness measures have

nice algorithmic properties, it is much easier to define algorithms for discovering interesting patterns. This is actually an important factor in the definition of support and confidence as interestingness measures in the association rule model. Indeed, their monotonicity properties, described previously in section 2.2.2.1, are essential to association rule mining algorithms. Such algorithmic properties are related to the way interestingness measures behave with regards to the underlying mathematical structures of the search spaces (generally, one of the lattice structures described in section 2.2.2.1). Some research, such as [WHC01, LBLL09, LBLML09, LBLL10, LB11, LBLL12b] focused primarily on the study and generalization of such algorithmic properties in the context of itemset and association rule mining. An entire branch of frequent pattern mining which developed particularly well, constraint-based (or query-based) frequent pattern mining, also focuses nearly exclusively on such properties [NLHP98, LNHP99, BJ98, PHL01, BGMP03, ZYHP07, NZ14]. In fact, in constraint-based frequent pattern mining the issue of defining interestingness

Table 2

 2

	.4: Objective interestingness measures that naively compare a statistic
	to its predicted value for the standard independence model.
	In addition to the standard independence model described above, other

  X∧Y log p X∧Y -p X∧Y log p X∧Y -p X∧Y log p X∧Y -p X∧Y log p X∧Y

	Kullback-Leibler divergence between distributions.
	For a distribution p in the contingency table, the information entropy H(p)
	is defined as:
	H(p) = -p High entropy indicates high randomness while low entropy indicates high struc-
	ture. Given partial knowledge on the values of this distribution, then there is
	a unique distribution which maximizes the entropy while matching this knowl-
	edge. This maximum entropy distribution corresponds to the idea that there
	is no more structure to the data than what is already described through the
	given partial knowledge. Typically, if p X and p Y are given by f X and f Y , the
	maximum entropy model corresponds exactly to the previous independence
	model.
	The Kullback-Leibler divergence D KL (f	p) between the empirical dis-
	tribution f in the contingency table and the distribution given by the local
	independence model p is given by:

  TCCY12, LMT14, LGFV + 16] or mining in data streams [JA07, LJA14, AH17]. Furthermore, as our work focuses on theoretical aspects of rule and itemset mining, we have not mentioned the very wide range of applications. This include notably data clustering [FAT + 14, ZAV14, ZMM15, DBFVL18], data classification [GBMTCO10, NVHT13, ZN14, EGB + 17] and a large panel of application domains ranging from web mining [IV06, SSLL08, NB12, K + 12, SO15] to text mining [BEX02, ZYTW10, AZ12], biology [PCT + 03, MO04, CTTX05, MMB + 18, ZAZ19] to chemistry [BB02, DKWK05, HXH + 20], medicine [Kha11, CTH + 13, TPMD + 13] to sociology [AAR09, NCC + 12, FC13, MJM + 17] and many more.

, GLFV

+ 19]

, uncertain data

[START_REF] Charu | Frequent pattern mining with uncertain data[END_REF]

  .3.

	General patchwork modeling
		Mathematical
		modelings
	Projection modeling	Aspect 1	Mathematical model 1
			Mathematical
	World	Aspect 2	problems 1
		. . .	Mathematical solutions 1
			Mathematical
		Questions	model 2
	Questions about	about aspect 1 Questions about aspect 2	Mathematical problems 2
	the world	. . .	Mathematical
			solutions 2
		Answers 1
	Answers	Answers 2	. . .
		. . .

table 3

 3

	.1 below. Note that these are valid constraints for an empirical distri-
	Itemset Frequency
	a 1	.330
	a 2	.680
	a 3	.558
	a 4	.613
	a 1 ∧ a 2	.222
	a 1 ∧ a 3	.133
	a 1 ∧ a 4	.157
	a 2 ∧ a 3	.277
	a 2 ∧ a 4	.360
	a 3 ∧ a 4	.269

Table 3

 3

	Minimal generators 7 Absolute frequency
	ω 0	5
	ω 1	3
	ω 2	5
	ω 3	199
	ω 4	9
	ω 5	228
	ω 6	195
	ω 7	26
	ω 8	15
	ω 9	16
	ω 10	42
	ω 11	35
	ω 12	69
	ω 13	97
	ω 14	47
	ω 15	9
	Table 3.2: A corresponding empirical distribution

.1: A set of conditions on itemset frequencies bution as the dataset for which the empirical distribution is exactly defined by table 3.2 satisfy these conditions. Using the four local MaxEnt models for the 7 See section 3.6.2.5 for an explanation of the notation used in Table

3

.2.

  F = (P i ) 0≤i≤l can be represented by a unique binary (d+1)×l matrix. The family is complete if and only if the matrix is surjective and sound if and only if it is injective. Hence, F is a complete and sound family of patterns if and only if its corresponding matrix representation in base Ω is invertible.

	Proposition 3.6.1. Ω, I and R are sound and complete families of patterns
	in B.

  the two set of values for n i presented in table 4.2 correspond to a same constraint system yet do not yield the same

	value for	µ(h I 7 ) n .
			Case 1 Case 2
		n 0	12	24
		n 1	7	14
		n 2	8	16
		n 3	4	8
		n 4	9	18
		n 5	5	10
		n 6	6	12
		µ(h I 7 ) n	0.241	0.237
	Table 4.2: Finite constraints corresponding to a same constraint system.

  As any coordinate of the vector X defining the MCI model is equal to zero if and only if this can be derived directly from the constraints (in the sense of derivable itemsets, see sections 2.3.2.3 and 4.1.3.3), the MCI model corresponds to the only X ∈ Z such that x i = 0, ∀i ∈ D andx i > 0, ∀i ∈ 0, d \ D,where D is the set of indices for which we can derive x i = 0 directly. Hence, the general structure of the algorithm may be summarized as follows:Note that this algorithm corresponds to the case in which the values in f |K * are specified (otherwise D cannot be computed). By contrast, if the values in f |K * are seen as formal variables, we can only perform steps 2 and 4 and, eventually, step 5 if it may be solved formally (or at least reduced) under the assumption that D = ∅ (as all cases in which D = ∅ can be obtained by continuity from cases in which D = ∅).

	1 . compute D ; 1
	2 . d e t e r m i n e P from	K * , f |K * v i a a l g o r i t h m 4.7 ;
	3 . add X i t o P f o r a l l i ∈ D ;
	a a l g o r i t h m 4.2 ; 4 . s o l v e P ( i . e . d e t e r m i n e a u n i v a r i a t e r e p r e s e n t a t i o n
	2 . compute Y N L v i a a l g o r i t h m 4.3 and a l g o r i t h m 4.4 ; o f Z u s i n g an a l g o r i t h m as mentioned i n s e c t i o n 4.2.2.1 ) ;
	3 . r e d u c e Y N L v i a a l g o r i t h m 4.5 ; 5 . f i n d X ∈ Z such t h a t x i > 0, ∀i ∈ 0, d \ D ;
	4 .	i n i t i a l i z e P M L ← ∅ ; Algorithm 4.8: Computing the MCI model
	5 . f o r Y ∈ Y L :
	6 .	add M Y t o P M L ;
	7 .	i n i t i a l i z e P M N L ← ∅ ;
	8 . f o r Y ∈ Y N L :
	9 .	add M Y t o P M N L ;
	1 0 . P M ← P M L P M N L ;
		Algorithm 4.6: Computing P M
		Finally, we can determine P through algorithm 4.1 and algorithm 4.6:
	1 . compute P L v i a a l g o r i t h m 4.1 ;
	2 . compute P M v i a a l g o r i t h m 4.6 ;
	3 . P ← P L P M ;
		Algorithm 4.7: Computing P
	An implementation of this algorithm in Python 3 will be made freely available.
	4.2.2.3 General structure of the algorithm
	We have shown in the previous section that we can transpose the equations
	characterizing an MCI model into a zero-dimensional polynomial system. This
	system can be solved using algorithms from algebraic geometry as mentioned in
	section 4.2.2.1 and we can check each solution of the system (of which there is a
	finite number) until we find the one which corresponds to the characterization
	of the MCI model.

  Table 4.6: Sequences for the number of cases to compute.

	m	2 2 m -1	A000612	A323819
	2	8	6	3
	3	128	40	30
	4	32,768	1,992	1,912
	5 2,147,483,648	18,666,624	18,662,590
	6	9.223 × 10 18	1.281 × 10 16 1.281 × 10 16
	7	1.701 × 10 38	3.376 × 10 34 3.376 × 10 34

  measure p for an MCI model, regardless of the set of itemsets K defining this particular MCI model. If D = ∅, this can only reduce the complexity of computing p because the number of variables p i to compute is equal to t rather than 2 m . Secondly, the size of the search space is also reduced as D increases. Indeed, the search space corresponds to {K * | K ⊂ I}, which has cardinality 2 2 m -1 if D = ∅, but decreases when D increases, because increasing D increases the number of frequencies of itemsets which can be directly derived from the frequencies of other itemsets. The manner in which the search space decreases is, however, not a trivial issue and we leave this to further research.Thirdly, the number of transactions n which is necessary to be confident in the empirical distribution is at least equal to O(2 m ) and at most equal to O(8 m ) with a reasonable approximation at O(4 m ) (see section 3.7.2.2) if we consider that D = ∅. However, the demonstrations presented in section 3.7.2.2 still hold if we consider that the number of degrees of freedom is equal to t -1 rather than simply 2 m -1 and the number of necessary transactions to be confident in the empirical distribution becomes at most equal to O(t 3 ). This can seriously bring down the total number of transactions which is necessary to be confident in the empirical distribution if the number t of existing transactions in the data is reasonable.

Note that this last process is not described in the most commonly cited version of the paper which is known for introducing the Apriori algorithm[START_REF] Agrawal | Fast algorithms for mining association rules[END_REF] but in its expanded version[START_REF] Agrawal | Fast algorithms for mining association rules[END_REF], which was published concomitantly.

We use the itemset notation X ∪ Y here for continuity with the previous sections rather than the notation for conjunction of attributes X ∧ Y which we prefer for the rest of the thesis. Both are equivalent in this context and both are used in the literature though the even more ambiguous XY notation is the most common.

We use the logical conjunction notation here, which corresponds to the original notation in[START_REF] Piatetsky-Shapiro | Discovery, analysis, and presentation of strong rules[END_REF], rather than the equivalent standard notation for itemsets, which is n X∪Y .

In a sense, the Poisson model and the fixed column margins model diverge from the standard independence model in opposing directions. In all models, the empirical dataset is seen as one observation out of all the potential datasets which can be generated by the random process described by the model. The set of all potential datasets in the standard model has cardinality 4 n , it is strictly contained in the set of potential datasets in the Poisson model which has infinite cardinality, but it strictly contains the set of potential datasets in the fixed column margins model.

We use the following notations for Table2.5:

In addition to the notations previsouly defined, we use the following notations for Table2.6:• I p (E) for the information content of the observation of E given a distribution p;• D KL (p q, E) for the Kullback-Leibler divergence between p and q restricted to E.

Note that this description also applies to the scientific process, as we will discuss in the following chapter.

As we will show in the next chapter, it is important in such a process to dissociate between background knowledge based on the type of patterns that are being mined and background knowledge based on other aspects in the data.

As Gregory Chaitin, one of its founders, described, Algorithmic information theory is "the result of putting Shannon's information theory and Turing's computability theory into a cocktail shaker and shaking vigorously."[START_REF] Cristian | Information and randomness: an algorithmic perspective[END_REF].

Translation from German to English from[START_REF] Podnieks | Philosophy of modeling: Neglected pages of history[END_REF].

Martin Thomson-Jones states that "one outstanding issue" with his own taxonomy of models is that it does not encompass such models[START_REF] Thomson-Jones | Models and the semantic view (with 2012 note to the reader)[END_REF].

See section 2.3.1.2 for a detailed review.

We use the term supposedly characteristic as they are meant to characterize interestingness but fail to do so entirely as we have previously mentioned.

Note that the formulation presented here is a slight generalization of what is presented in[START_REF] Jaynes | On the rationale of maximum-entropy methods[END_REF] which only considers a linear constraint (i.e. f is a linear function) which is needed for considering the number of degrees of freedom in the concentration theorem but not necessary at this point.

We prefer the term frequency to the term support, which are synonyms, as the term frequency is more commonly used in the broader statistical literature.

Note that these recommendations specifically aim at defining meaningful mathematical modeling in objective frequency-based interesting pattern mining and that we do not disqualify rule mining in pattern mining in general.

There are, of course, much better approximations for χ 2 α (d) (see, for example,[START_REF] Beh | Exploring how to simply approximate the p-value of a chi-squared statistic[END_REF] on such approximations) but this simple approximation is sufficient for the purpose of our asymptotic analysis here.

This quote from Newton's Principia[START_REF] Newton | Philosophiae naturalis principia mathematica. Editio secunda auctior et emendatior[END_REF] has been often mistranslated as "I frame no hypotheses" thus wrongly implying that Newton did not believe in formulating hypotheses at all[START_REF] Cohen | The first English version of Newton's hypotheses non fingo[END_REF].

This is a simple problem in linear programming which can be solved through the use of a simplex algorithm for example.

(f 1 -f 7 -x 5 )(f 2 -f 7 -x 6 ) 1-f 4 x 0 = 1-f 1 -f 2 -f 4 +f 6 +f 7 +x 3 +x 5 x 0 = 1 -f 1 -f 2 -f 4 + f 5 + f 6 + x 3 x 0 = 1-f 1 -f 2 -f 4 +2f 7 +x 3 +x 5 +x 6 x 1 = f 1 -f 7 -x 3 -x 5 x 1 = f 1 -f 5 -x 3 x 1 = f 1 -f 7 -x 3 -x 5 x 2 = f 2 -f 6 -x 3 x 2 = f 2 -f 6 -x 3 x 2 = f 2 -f 7 -x 3 -x 6 x 4 = f 4 -f 6 -x 5 x 4 = f 4 -f 5 -f 6 + x 7 x 4 = f 4 -f 7 -x 5 -x 6 x 6 = f 6 -f 7 x 5 = f 5 -x 7 x 7 = f 7 x 7 = f 7 x 6 = f 6 -x 7 Q = (1 -f 1 )T 2 -(1 -2f 7 -f 1 f 2 -f 1 f 4 + 2f 1 f 7 + f 2 f 4 )T + (f 2 -f 7 )(f 4 -f 7 )(1 -f 1 ) {f 1 , f 2 , f 4 , f 5 , f 6 , f 7 } {f 1 , f 2 , f 3 , f 4 , f 5 , f 6 } Q {f 1 , f 2 , f 3 , f 4 , f 5 , f 6 , f 7 } x 3 = (f 1 -f 5 )(f 2 -f 6 ) 1-f 4 x 7 = t x 0 = 1-f 1 -f 2 +f 3 -f 4 +f 5 +f 6 -f 7 x 0 = 1 -f 1 -f 2 -f 4 + f 5 + f 6 + x 3 x 0 = 1-f 1 -f 2 +f 3 -f 4 +f 5 +f 6 -x 7 x 1 = f 1 -f 3 -f 5 + f 7 x 1 = f 1 -f 5 -x 3 x 1 = f 1 -f 3 -f 5 + x 7 x 2 = f 2 -f 3 -f 6 + f 7 x 2 = f 2 -f 6 -x 3 x 2 = f 2 -f 3 -f 6 + x 7 x 3 = f 3 -f 7 x 4 = f 4 -f 5 -f 6 + f 7 x 3 = f 3 -x 7 x 4 = f 4 -f 5 -f 6 + f 7 x 5 = f 5 -f 7 x 4 = f 4 -f 5 -f 6 + x 7 x 5 = f 5 -f 7 x 6 = f 6 -f 7 x 5 = f 5 -x 7 x 6 = f 6 -f 7 x 7 = f 7 x 6 = f 6 -x 7 x 7 = f 7 Q = T 3 -(f 3 + f 5 + f 6 -f 1 f 2 -f 1 f 4 -f 2 f 4 + f 1 f 6 + f 2 f 5 + f 3 f 4 )T 2 + (f 3 f 5 + f 3 f 6 + f 5 f 6 + f 1 f 2 f 4 -f 1 f 2 f 5 -f 1 f 2 f 6f 1 f 3 f 4 -f 1 f 4 f 6 + f 1 f 2 6 -f 2 f 3 f 4 -f 2 f 4 f 5 + f 2 f 2 5 + f 2 3 f 4 + 2f 3 f 5 f 6 )T + f 3 f 5 f 6 (f 1 + f 2 -f 3 + f 4 -f 5 -f 6 -1)4.2.2.7 Pros and cons of the algebraic methodAs stated previously, one of the important advantages of the algebraic method is that it allows us to determine reduced algebraic expressions for MCI models in generic cases, from which we can then compute specific MCI models very efficiently. When the corresponding generic cases have been computed, the increase in computation speed is quite astounding in comparison to standard methods for computing MaxEnt models.In order to check this, we chose 20 different samples of m = 3 items among the 70 items of the plants database[START_REF]The plants database[END_REF], each corresponding to an empirical distribution f such that no single f i could be derived from the other f j for

This specific algorithm was chosen based on the fact that it has been commonly used for computing such MaxEnt models in the context of itemset mining[START_REF] Pavlov | Beyond independence: Probabilistic models for query approximation on binary transaction data[END_REF][START_REF] Tatti | Using background knowledge to rank itemsets[END_REF][START_REF] Mampaey | Summarizing data succinctly with the most informative itemsets[END_REF]

Recall that, in the MCI model, we have p i =⇒ f i . Hence the sum only excludes indices for which both p i and f i are equal to 0.

As we will show further on, this is also debatable, but we will still use this as an assumption here to define a total order on P(I).

We use the term valid local hypothesis to refer to a local hypothesis which would be valid if we were to consider the corresponding local context as a separate global context.

The fact that we can construct a probability measure such that all items are pairwise independent which, at the same time, does not correspond to the mutual independence model is sufficient to prove this point.

(see chapter 5), as well as the mutual constrained independence models they rely on (see chapter 4), are deeply rooted in these principles.

We call this approach the finite approach and discuss this in section 4.1.2. As we show, this approach poses both theoretical and practical issues. Another approach is to consider the limit, when n goes towards infinity, of the solutions obtained when considering datasets of size n. As we show, this limit is well defined and, in contrast with the finite approach, it does not suffer from the same theoretical issues and is more easily computed. This asymptotic approach, presented in section 4.1.3, is central to the notion of mutual constrained independence described in this article.

Finite approach

Consider a set of itemsets K ⊂ I and define K = K ∪ { }. Let g |K be the restriction to K of a measure on B with integer values so that g |K can be seen as corresponding to a dataset with n transactions where n = g . Then, the set M K,g |K defined below as the set of all measures on B with integer values which are equal to g |K for all itemsets in K is finite:

Furthermore, for each measure h ∈ M K,g |K , there are exactly n! h! distinct datasets which can be associated to h where h! = d i=0 h i !. Hence, we can define the expected measure µ when considering a uniform distribution on all possible datasets corresponding to a measure in M K,g |K by:

By linearity, µ is of course a measure on B such that, ∀X ∈ K, µ X = g X . In particular, µ = n. This measure is entirely defined by K, g |K . Note that K, g |K is not a constraint system per se because g is not a probability measure (excluding the trivial case for which n = 1). We can naturally bring this problem down to probability measures and constraint systems by considering the constraint system C n = K, 1 n g |K and noticing that 1 n µ is a probability measure satisfying C n . However, this constraint system does not, in general, terminology defined in [START_REF] Basu | Algorithms in Real Algebraic Geometry[END_REF].

The following notations will be used within this section. For any field F, let F[X] = F[X 0 , ..., X d ] be the ring of polynomials in d + 1 variables X 0 , ..., X d with coefficients in F. The fields we will consider here all satisfy Q ⊂ F ⊂ C.

To maintain consistency with previous notations, X will be used to refer to an element of F d+1 with coordinates equal to x 0 , ..., x d . The term polynomial system will refer to a finite subset of F[X] and we will generally note such a system P. Solving a system P in C means determining the set of zeros of P in C d+1 , which is the set:

and we will generally note Z for Z P unless there is some cause for ambiguity.

The dimension of a polynomial system will refer to the dimension of its set of zeros in C. Hence, a polynomial system is zero-dimensional if its set of zeros in C is finite.

Our approach is based on the fact that, given a zero-dimensional polynomial system P ⊂ F[X], there are algebraic algorithms (see, for example, algorithm 12.12 p.468 in [START_REF] Basu | Algorithms in Real Algebraic Geometry[END_REF]) which allow us to determine (given sufficient computational resources) d + 3 univariate polynomials Q, B, A 0 , ..., A d with coefficients in F such that Q and B are coprime and

In this case, (Q, B, A 0 , ..., A d ) is called a univariate representation of P. This implies that, if we manage to express the equations defining an MCI model as a zero-dimensional polynomial system P, we could break down the problem of determining the MCI model into two steps:

• determining a univariate representation of Z;

• determining the MCI model from this univariate representation.

If the first step is performed, then the second step follows quite easily. In fact, we will show that the first step of the process may be performed only once for any K * (as was the case for each K * such that K * = I \{I d }) which then allows for a very fast computation of MCI models in common cases of K * . Hence, (see section 4.1.3.3). However, this does not always result in a zero-dimensional polynomial system. Indeed, let M j be the polynomial defined by the j-th column of T -1 , for each j ∈ 1, d . For example, when m = 3, this gives:

Now, suppose that we define P M from these polynomials. Then P M = (M j ) j∈J where J = {j ∈ 0, d | I j / ∈ K * } and P = (L j ) j∈J (M j ) j∈J . Considering the case in which m = 3 and K * = { }, we get:

We can see that P is at least 3-dimensional. Indeed, consider Z as below:

Then, we get the following intersection between the set Z of zeros of P and

which is a 3-dimensional linear space. Hence, in this case, P is at least 3dimensional.

The issue in the example given here is that the dimension of P M is at least equal to 4 (as Z ⊂ Z M ) while we could expect it to be equal to 1. Indeed, where s = |S|.

Proof of lemma 4.2.2. Consider S ⊂ 0, d such that,

and

Therefore, X = X and the dimension of V is at most s.

Proof of lemma 4.2.1. Let Y be a family of generating vectors of Ker (T K * ) ∩ Z d+1 and P M the corresponding polynomial system. Note S the set defined by:

and suppose S = ∅. Let S ∈ S . Then, based on the converse of lemma

Hence, if Y is equal to the family Y augmented by Y and P M is the cor-

If S = ∅, we are done. Otherwise, we can repeat the process and define a strictly increasing sequence Y ⊂ Y ⊂ ... ⊂ Y (k) associated to a strictly

M together with a strictly decreasing sequence 0, d ⊃ S ⊃ S ⊃ ... ⊃ S (k-1) , until S (k-1) = ∅, which is bound to happen eventually as 0, d is finite.

Hence, Y (k) is a generating family of vectors of Ker (T K * ) ∩ Z d+1 satisfying the desired property.

Through lemma 4.2.1, we see that we can consider a polynomial system P M based on a generating family of vectors of Ker (T K * ) ∩ Z d+1 which has dimension r and which defines a zero-dimensional P when concatenated with P L .

Computing P. The proof to lemma 4.2.1 is a constructive one, which provides a baseline for an algorithm to determine P M as desired: initialize Y to B K * and incrementally add vectors to Y until S = ∅. However, a family of

Testing the MCI hypothesis

Definition of the MCI hypothesis

Unless specified explicitly, we consider the same notations here as in chapter 4.

Let D be a binary dataset corresponding to n observations of the presence or absence of m items in a statistical population (i.e. a dataset of n transactions on these items) and K ⊂ I be a set of itemsets. We note f the probability measure on B defined by the empirical distribution in the dataset D.

Definition 5.1.1. The mutual constrained independence hypothesis for the dataset D defined by K is the hypothesis that the dataset corresponds to n independent identically distributed samples of a random variable whose distribution p is given by the MCI model for the data defined by K (see definition

4.1.3).

For a detailed explanation of the rationale behind the definition of such a hypothesis, we refer to the two previous chapters.

Statistical testing of the MCI hypothesis

χ 2 statistic

Following the classical approach used for statistical tests of independence, we suggest the use of a χ 2 test for testing mutual constrained independence. The χ 2 statistic for the dataset is defined as:

where D is the set of indices for which p i = 0 as described in section 4.2.2.3 1 .

The distribution of the χ 2 p,f statistic converges towards a χ 2 distribution as described in the following proposition. order relation if the aforementioned condition is true. Though we acknowledge this general issue, we do not address it any further in this thesis, leaving the task of quantifying the complexity of each equivalence class for further research.

Another approach, which allows to avoid the issue of forcibly and arbitrarily defining an order relation between sets of itemsets within a same class, is to only consider hypotheses which are defined by sets that are invariant by permutation of items. For example, when considering three items, we would only consider the following sets, ordered as such:

This approach also has the advantage of only considering 2 m potential hypotheses rather than 2 2 m -1 . However, it has the disadvantage of disregarding potentially interesting hypotheses.

Search algorithms

In the following, assume we are using the order relation described in section 5.2.2.1. Our aim now is to find the smallest possible K for this order relation, such that the associated hypothesis is not rejected by a χ 2 test of mutual constrained independence. The algorithm consists in building a strictly increasing sequence of sets of itemsets (K), corresponding to a strictly decreasing sequence (ξ K ). It terminates as soon as we have reached a K for which the associated hypothesis is no longer rejected or if we have reached a local minimum for ξ K for which the associated hypothesis is still rejected. In the first case, the associated hypothesis is considered a valid hypothesis and, in the second case, we consider that no valid hypothesis can be formulated about the data.

In the greedy-down approach, we start from a constrained set of itemsets K 0 , associated to a hypothesis H 0 which is not rejected and gradually simplify it. Note that we can only apply such a process if we already have such a K 0 at our disposal.

1 . i n i t i a l i z e K ← K 0 ;

2 . w h i l e ∃I ∈ K, ξ K\{I} < 1 :

.2: Greedy-down

A greedy-up algorithm can be combined with a greedy-down one: first, K is complexified until we find a hypothesis which is not rejected, and then this hypothesis is simplified as much as possible. In this case, we can consider the resulting hypothesis as valid.

In the worst case scenario, ξ K is computed for 2 m (2 m -1) 2 different values of K for the greedy-up algorithm and k 0 (k 0 -1) 2 values for the greedy-down algorithm (where k 0 = |K 0 |). Hence, the computational complexity of the combined greedy-up, greedy-down algorithm is in O(2 2m ) (under the simplified assumption that the complexity for computing the MCI model is constant which is not necessarily true as we have previously mentioned).

While this is an improvement in comparison to testing all potential hypotheses, of which there are 2 2 m -1 , it can still be considered too much in many cases. In order to reduce this further to a more reasonable quantity, one may consider limiting the size of the itemsets considered to a certain value k. In this case, the number of hypotheses to test is polynomial in m of degree 2k.

Efficiency of the greedy algorithms

In order to assess the efficiency of the greedy approach, we performed some tests to check how often the algorithms described above reached the same solution as the naive approach. As the naive approach can only be performed for a small number of items, we fixed the number of items to m = 5. Furthermore, in order to be confident that the hypotheses extracted corresponded indeed to meaningful knowledge about the data, we only considered artificially generated data.

The datasets considered were generated as follows. For each dataset, five values p 1 , ..., p 5 were randomly and independently picked between 0.1 and 0.9.

We then generated a million random binary vectors in {0, 1} 5 each of which corresponded to five independent Bernoulli trials with parameters p 1 , ..., p 5 . This procedure was repeated 1000 times, thus generating 1000 different datasets.

The probability threshold for rejecting hypotheses was set to α = 99.9%.

Out of the 1000 datasets generated, the optimal constrained set for the hypothesis corresponded to:

• K ind = { , a 1 , a 2 , a 3 , a 4 , a 5 } (i.e. the independence model) in 962 cases;

• a subset of K ind within the permutation class of K ind \ {a 5 } in 32 cases (corresponding to cases in which the values of the randomly generated probabilities for the missing item were very close to 1 2 );

• a superset of K ind within the permutation class of K ind ∪ {a 1 ∧ a 2 } in 3 cases;

• a superset of K ind within the permutation class of K ind ∪ {a 1 ∧ a 2 ∧ a 3 } in the 3 remaining cases.

The greedy-up algorithm led to the optimal solution in 432 cases. The greedyup algorithm followed by the greedy-down algorithm led to the optimal so-CHAPTER 6

Epilogue

We conclude this doctoral thesis by presenting some specific aspects which we will be working on in the close future, followed by some more general considerations. For a concise summary of the specific contributions of this thesis to current research, we refer the reader to section 1.5.

Follow-up research

The research we have conducted during this doctoral thesis has opened a wide scope of potential developments and we cannot list them all here. Instead, we concentrate solely on the specific elements of research which we have already engaged in, or for which we have at least sketched up some ideas.

Regarding the research presented in chapter 3, our main development concerns:

• An expanded presentation of the concepts developed around models and modeling, including a much wider range of examples from various fields.

Regarding the research presented in chapter 4, we will concentrate on the following three elements:

• The identification of a specific algebraic characterization of MCI models defined by all itemsets of size 1 and 2 for any given m (i.e. an MCI alternative to the Chow-Liu tree model).

• An empirical study of the performances of the algebraic method for computing MCI models with pre-computed generic cases versus standard

Abstract :

In this thesis, we study objective interesting pattern mining processes on datasets such as used in itemset mining. We focus on the notions of objectivity and meaningfulness in mining processes. We establish a link between the meaningfulness of a mining process and that of its corresponding mathematical modeling. We formulate a number of recommendations in terms of modeling choices for increasing both meaningfulness and objectivity. We also establish a link between the study of objective interesting pattern mining and the issue of the automation of scientific discovery.

Our theoretical analysis exhibits the adequacy of considering maximum entropy models in such mining processes. We then proceed with presenting a novel mathematical construction for such models, based on a notion of constrained independence, which is specifically adapted to the itemset context. Based on this construction and on tools from algebraic geometry, we present an exact method for computing maximum entropy models. Last, based on our recommendations for the mathematical modeling of pattern mining processes and our notion of constrained independence, we present a new pattern mining algorithm.