The deconstruction of a text: the permanence of the generalized Zipf law-the inter-textual relationship between entropy and effort amount Thierry Lafouge ; Abdellatif Agouzal ; Genevieve Lallich Boidin

Context and Introduction

To our knowledge, research on Zipf's law was mentioned for the first time [START_REF] Petruszewycz | L'histoire de la loi d' Estoup-Zipf[END_REF] by J.-B Estoup [START_REF] Estoup | Random texts do not exhibit the real Zipf's law-like rank distribution[END_REF] and is still valid. In fact, when we searched for the "Zipf's law" character chain in the titles of articles and when we conducted a WOS query in February 2015, 70 relevant articles showed up, published since 2008. These articles came from various disciplines: physics, mathematics, economics, biology, geography. The law's original formulation is as follows: if any object in a corpus, such as a word in a text, can be characterised by a positive integer, obtained by counting, and called 'frequency', it is always possible to assign a rank to each object. George K. Zipf [START_REF] Zipf | Human behavior and the principle of least effort[END_REF], an American linguist, then showed that rank and frequency were not independent and approximately verified the relationship:

!"#$×!"#$%#&'( ≈ !"#$%&#% This simple relationship, the original formulation of Zipf's law for a text, does not depend on any parameter. We could say that it is a paradigm [START_REF] Benguigui | The end of a paradigm : is Zipf's law universal[END_REF], or, perhaps, an axiom. Egghe classified this type of informetric law in the category of "Universal laws, not dependent on any distribution or parameter" [START_REF] Egghe | The functional relation between the impact factor and the uncitedness factor revisited[END_REF]. The general form of this law is slightly more complex and involves a parameter, namely an exponent β positive close to 1 in its formulation (see equation ( 1)).

! ! ≈ ! ! ! ! > 0 ! = 1, . . ! (1)
where ! f (r) denotes the frequency of the word of rank r, S the number of distinct words in the corpus (also known as the size of the text's lexicon or number of sources in informetrics) and ! is a positive constant. In this article, equation ( 1) serves as a model to adjust the word distribution frequency, where ! is a number between 1 and 2. We call equation (1) the generalized Zipf law. We won't confuse this law with Zipf's shifted law, as proposed by Mandelbrot 1 , which is also sometimes known as the generalized Zipf law.

The properties of these "rank-frequency" distributions have been observed and studied in many fields. In Informetrics, the place of Zipf's law has been studied in distributions, which are often called Lotkaian in reference to the work of Lotka in 1926 [START_REF] Lotka | The frequency distribution of scientific productivity[END_REF]. These distributions model the regularities observed in the process of production or in the usage of information. A complete study of these laws can be found in [START_REF] Egghe | Power Laws in the Information Production Process: Lotkaian Informetrics[END_REF], as well as a study of Zipf's law's place among other informetric laws (see chapter 2, paragraph 4: "The place of the law of Zipf in Lotkaian informetrics").

1 ! ! ≈ ! !!! ! ! ≈ 1 ! > 0
In this article, we only consider texts in which the objects are words or chains of characters2 and the frequency is their number of occurrences in the text. We place ourselves in a particular position, if we compare it to other examples. If we reason with the formalism [START_REF] Egghe | On the duality of informetric systems with application to the empirical law[END_REF] of the IPP (Information Production Process) where sources (words, researchers, cities, species, etc. ) produce items (word frequency, number of articles published, number of inhabitants, number of specimens, etc. ), the self-similarity of the pair (word-frequency of the word in the text)3 can be troubling. It justifies the Mandelbrot fractal approach.

The first question relates to the explanation of this law and, of course, its relationship with language and writing. How can we explain it when it seems to be a universal constant (the parameter β is always close to 1 and varies little from one text to another)? The nature of this regularity is called into question with the controversial example of monkeys who randomly typed letters and pushed on spacebars, thus producing a random sequence of words [START_REF] Mitzenmacher | A brief history of generative models for power law and lognormal distribution[END_REF] and still verifying Zipf's law. However, if we randomly generate a text, a recent study (Ferrer i Cancho and Elveag, 2010) seems to demonstrate that there is a difference between a random text and a text written in a specific language. The authors conclude that "Zipf's law might in fact be a fundamental law in natural language". We do not provide a definite conclusion to this debate. In fact, the notion of random text is not as simple as it seems. We must first remember the various theories explaining this phenomenon, and the many different types of models [START_REF] Piantadosi | Zipf's word frequency law in natural language : a critical rewiew and future directions[END_REF] that have also been offered to explain it. We shall cite the three most common approaches in informetrics. 1) Herbert A. Simon [START_REF] Simon | On a class of skew distributions functions[END_REF] described a stochastic process which produces these stationary laws. He used the example of the writing of a book to illustrate his demonstration. This describes a process to generate a text. Many demonstrations are possible to explain different phenomena with this type of process. The "cumulative advantages" law [START_REF] Price | A general theory of bibliometric and other cumulative and other advantage processes[END_REF] falls into this category.

2) Mandelbrot [START_REF] Mandelbrot | An informational Theory of the statistical Structure of languages, chapter Communication Theory[END_REF] argued his view differently. He considered the generation of language as the transmission of a signal and he used information theory. He showed that, in seeking to minimise the cost of producing words (the number of characters, also known as the size of the alphabet, plays a key role), the results of a generalized Zipf law could be obtained (see section 4.2.2). Of course, we must agree on the choice of a cost function, which plays an indirect role in the length of the words. This cost function, which we use, leads to words that are rare, thus requiring a much greater effort to be produced.

3) Mandelbrot later presented a second argument [START_REF] Mandelbrot | The Fractal Geometry of nature[END_REF] of a geometric kind, which we recall in section 4.2.4 to explain this law.

We can conclude this broad state-of-the-art by saying that this law is universal and applies to all texts. Texts can be selected according to various criteria (size, language, date, subject, etc.). Such a distribution can be seen in languages that have disappeared today and that we do not yet know how to translate [START_REF] Bouchet-Franklin Institute | Investigation of the Zipf-plot of the extinct Meroitic language[END_REF].

A text is never finished, it is not limited and the beginning or end are irrelevant to Zipf's law. Zipf's law seems to apply as soon as the ratio between the number of sources (size of lexicon) and the total number of words are balanced4 .

Hypotheses and Objectives

In this article, we discuss results taken from models 2) and 3) as described above. The first Mandelbrot model is taken from a larger class of models and depends on communicative optimization principles (see section 4.4 Commutative accounts in [START_REF] Piantadosi | Zipf's word frequency law in natural language : a critical rewiew and future directions[END_REF]). The second model consists in saying that Zipf's law is essentially a statistical artefact (see section 4.1 Random typing accounts in [START_REF] Piantadosi | Zipf's word frequency law in natural language : a critical rewiew and future directions[END_REF]). In both of these models, the size of words is very important, but in different ways. In the first model, coding is optimized when the message is sent depending on the length of the words. It pertains to the field of language. The second model pertains more to the field of writing, in which patterns of certain lengths are repeated.

We therefore ask the following question: when the text is modified, does the regularity of word frequencies stay the same, thus making the length of words invariant? In order to verify our hypothesis, our idea consisted in taking a text, which we modified and degraded; hence the term 'deconstruction' in the article's title. To be more precise, our method consisted in applying an algorithm that replaced characters one by one with a "joker' in the whole text through successive iteration. We then segmented the modified text into words. We therefore didn't change the distribution of word lengths in the text. We then checked whether a generalized Zipf law (see ( 1)) was still valid. We found that it was constant with a different coefficient that increased with each step. However, a degraded text is no longer a text. The segmented sources are no longer words belonging to a text or to a specific language! In the final step, all of the characters were deleted (the text became a string of jokers and separators). The distribution of frequencies and the distribution of word lengths merged. However, the distribution of lengths was not zipfian. What happened during this deconstruction? Also, the fact that we can always adjust the distribution frequency of sources during the first steps raises new questions. This result is not incompatible with Mandelbrot's second demonstration as long as the size of the alphabet is not too small. It is also possible to process these 'degraded texts' with Shannon's signal theory. In that case, we must calculate the degraded text's entropy and its associated amount of effort. If Mandelbrot's hypothesis concerning language transmission is true, there must be a trace of it during the deconstruction phase. Entropy and the amount of effort cannot be independent. This allows us to see that entropy and the amount of effort are not independent and that inter-text regularities can be highlighted. This is why we subtitled our article as "the inter-textual relationship between entropy and effort amount". We do not aim to build a new explanatory model, since many of them already exist and are very relevant. Instead, we want to shed new light on inter-textual statistical regularities and build on Mandelbrot's work.

Our article is organised in four parts:

--The method of depleting a text (see section 3) is first explained.

-We then recall the mathematical tools and concepts used: entropy, amount of effort. These results show that, for degraded texts, a relationship exists between entropy and the amount of effort at each step (see section 4). This section develops Mandelbrot's theoretical results and applies them to a degraded text. --Our method is applied on a text of 114,730 words segmented into 12,740 sources (see section 5). --Thanks to the latter experiment, we are able to show that there is a linear relationship between the amount of effort and entropy (see section 6). The theoretical results (section 4) that were previously found are insufficient to explain the experimental results in a ! " satisfactory way. A possible explanation is offered, using recent works [START_REF] Lafouge | The Source-effort Coverage of an Exponential Informetric Process[END_REF] that combine power laws and effort functions. Finally, we end with a conclusion.

The Depleting Method

Lexical Definitions

A text is one of the tangible forms that stems from intellectual activity, namely writing. A text is a set of signs, ordered and grouped into words. It is therefore composed of a sequence of ordered words. Before defining what we call a "word", we must clarify a few definitions. We define signs within a text, namely all characters and separators, as graphemes belonging to the whole and consisting of two disjointed subsets.

--By characters, we mean elements of the language's alphabet: letters (uppercase and lowercase) and numbers. --We distinguish three types of separators:

-Physical separators: blanks and line breaks -Punctuation separators: full stops, commas, exclamation marks, and quotation marks -Special signs: open brackets, closed brackets The 9 separators that we retained for the experiment are: Blanks, line breaks, open brackets, closed brackets, exclamation marks, question marks, full stops, commas and quotation marks. This list of separators is arbitrary and other choices could have been made. We define a "word" as being a sequence of characters placed between two separators. The definition of the separator in subscript allows the computer to identify words automatically. Physical separators generally suffice for experiments using Zipf's law. This choice has very little effect on the results. However, we can regret the absence of clarification by authors when verifying the validity of Zipf's law. In the following experiment, we pre-processed uppercases. All uppercase letters were replaced by lowercases, except for proper noun initials. This can be done automatically if proper nouns are marked.

An Algorithm for the Deconstruction of a Text

Characters are substituted one by one in the text, and replaced by a sign that does not exist in the language, known as the joker and written as @. Our algorithm is illustrated with the sentence: " the monkey types on the keyboard with strength.

Initiation :

The monkey types on the keyboard with strength.

2. Elimination of "e" : Th@ monk@y typ@s on th@ k@yboard with str@ngth 3. Elimination of "s, a, t, n, r " :@h@ mo@k@y @yp@@ o@ @h@ k@ybo@@d wi@h @@@@@@@h 4. Elimination of "m, h, d, k ": @@@ @o@@@y @yp@@ o@ @@@ @@ybo@@@ wi@@ @@@@@@@@ 5. Elimination of "w, i, b, o, y" : :@@@ @@@@@@ @@@@@ @@ @@@ @@@@@@@@ @@@@ @@@@@@@@ The total number of signs is invariant, the size of the alphabet decreases, going from 16 (this is the number of iterations needed to completely deconstruct the text) to 1. The total number of words is invariant and remains equal to 8. The number of sources, equal to 7, decreases and is then equal to 5 (we will prove later on that, when deconstructing a text, the number of sources either decreases or remains constant at each step). Replacing deleted characters with a joker maintains the same distribution of length for each word throughout the process. The text structure is preserved since the proportion of separators does not change over the course of the algorithm. The joker is a simple artefact and is not a critical part of the process. In the last iteration, if we segment the text, we obtain the distribution of word lengths.

To implement this algorithm, we remove the characters by decreasing frequency. Other character substitution methods are tested in this paper, with comparable results.

Segmenting and Formatting Distributions

At each step, noted i, we segmented the depleted text -later written as ! ! -into words. We identified the number of sources, noted ! ! , and we ranked the frequency of sources -written as ! ! (!) -as well as their frequency in the text. Finally, to verify the generalized Zipf law, we constructed a frequency-rank distribution:

!, ! ! ! ! = 1. . ! ! (2)
where sources are ranked by decreasing frequency.

We note that, for each iteration i, when a character is substituted, then the number of sources ! ! decreases 5 . Let us prove this result. If we call ! ! the number of distinct characters in a text at step i and L the maximum length of a chain of characters in the text, then the text's number of possible sources !"#! ! in step i is equal to the finite sum:

!"#! ! = (! ! ) ! ! !!!
We can easily show that !"#! ! decreases at each step i because ! ! decreases by 1 at each iteration i. !"#$ ! is equal to L when the last character has been substituted. This allows us to say that the number of sources ! ! therefore also decreases at each step i (it may remain constant when we substitute a rare character). So ! ! is a decreasing sequence.

The Adjustment of Distributions

At each deconstruction step (the text is not modified in the first iteration, since only one letter has changed), we carry out an adjustment of the distribution (2). We postulate6 that distribution (2) is a generalized Zipf law at each step. This allows us to write:

! ! ! ≈ ! ! ! ! ! ! ! > 0 ! ! ≥ 1 ! = 1, . . ! ! (3)
We limit ourselves to ! ! ≥ 1, which is almost always verified in practice. We know that the word frequency distribution is more complex than initially thought (see section 3, "The word frequency distribution is complex" in [START_REF] Piantadosi | Zipf's word frequency law in natural language : a critical rewiew and future directions[END_REF]). For example, the log-normal law [START_REF] Petruszewycz | Loi de Pareto ou loi log-normale : un choix difficile[END_REF] is very similar to the inverse power-distribution. In order to calculate the adjustment parameters ! ! and ! ! , we carry out a simple linear regression after having transformed the coordinates on an !" -!" scale. By calculating the coefficient ! ! , we measure the quality of the adjustment. When it is greater than 0.97, we speak of an adjustment 7 with the generalized Zipf law. Many other methods exist to adjust this type of distribution. We can, for example, refer to [START_REF] Clauset | Power-law distribution empirical data[END_REF]. The visualisation of the curves and classic test are sufficient for this study. In fact, we know that a very significant test can sometimes hide a distribution that is not uniform; hence the importance of viewing the graph to avoid errors. The adjustment itself isn't the determining factor in this study, and this is why we chose not to test other models. The most interesting statistical regularities here (see section 6) don't depend on a model and depend even less on an adjustment method.

Results and Mathematical Treatments

Informetric Results

Let us first establish an informetric result which is a property of the generalized Zipf law (see ( 1)), and is proved in Annex 1. This mathematical result could have been omitted from this article as it is not fundamental. However, it can help to answer some questions that could arise during the experiment.

In what follows, we use the continuous mode. Each stage is characterised by the number of sources ! ! . The number of words (written M) is invariant throughout the deconstruction process. If adjustment (3) is perfect, we come up with the following equality at each step of the deconstruction:

! ! ! ! ! ! ! ! !" = !
This amounts to writing the equation:

!" ! ! ! ! ! ! = ! ! ; ! ! = ! ! ! (4)
The theoretical question we ask is as follows: if (! ! , ! ! ) is a sequence, does a unique ! ! exist so that equation ( 4) is verified? In Annex 1, we prove the necessary and sufficient condition needed to solve this mathematical problem and we present an additional result that explains why ! ! is an increasing sequence.

2 The Theoretical Relationship between Entropy and Effort Amount in the Degradation of a Text

We built on Mandelbrot's results and applied them to the deconstruction of a text. We proved the "linearity" relationship (12) between entropy and the inter-textual amount of effort.

Entropy and the Amount of Effort at Each Step of Deconstruction

7 For each adjustment, we display the graph with the points that are aligned to see if there is no bias.

R 2 ! R 2
We place ourselves within Shannon's signal theory. The entropy 8 of the degraded text ! ! as represented by distribution (2) is

! ! = - !"( ! ! !!! ! ! (!)). ! ! (!) (5)
where

! ! ! = ! ! (!) ! !
where ! ! ! is the probability of a word of rank r in the degraded text ! ! .

We assume that there exists an effort function written as ! ! (!) that is strictly positive. This function allows us to define the total amount of effort ! ! of the degraded text T i with:

! ! = ! ! ! ! ! !!! . ! ! (!) (6)
The problem is then to choose which effort function to use. We use the effort function ( [START_REF] Mandelbrot | An informational Theory of the statistical Structure of languages, chapter Communication Theory[END_REF]) defined by Mandelbrot .

! ! ! = !" (!) !" (! ! ) (7)
This effort function has the following characteristics:

-As the number of distinct characters is reduced, the number of characters in words increases, and, according to (7), the amount of effort needed to produce them increases.

-The higher the rank is, the more words tend to have a large amount of characters and the rarer they are. According to (7), more effort amount is needed to produce them as the rank increases.

Minimizing the Cost of Transmission: Mandelbrot's Optimization Model

Knowing the entropy ! and the amount of effort ! (see ( 5), ( 6), ( 7))), Mandelbrot then calculated the probability !(!) of a word by minimizing the average cost of information, ! = ! ! Following this calculation, he obtained the following equality (see for example the demonstration section Power Laws via Optimization in [START_REF] Mitzenmacher | A brief history of generative models for power law and lognormal distribution[END_REF])):

! ! = ! ! ! ! = ! !.!" (!) (8)
We therefore obtain a proportional relationship between entropy and the amount of effort:

! = !. Ln ! . ! (8b)
In the case of a text's deconstruction, this hypothesis seems harder to apply to each deconstructed text: we choose not to use this result here, but we retain formula (7) to calculate the amount of effort.

3 The Link between Entropy and the Amount of Effort

8 Here, entropy is measured by a natural logarithm; it should be divided by Ln(2) to express it in bits.

If we calculate entropy (5) assuming that the distributions are generalized Zipf law, using the equation ( 3),( 4),( 6),( 7) we obtain a relationship between entropy and amount of effort:

! ! = ! ! + ! ! . ! ! ! ! > 0, ! ! = ! ! . Ln ! ! > 0 (9)
The ! ! text's entropy depends on adjustment parameters, on the amount of effort and on the number of distinct characters. The proof of this result can be found in Annex 3. Later on, we modified this result using Mandelbrot's geometric study of Zipf's law.

The Theoretical Extension of Mandelbrot' Work: the Inter-textual Relationship between Entropy and Effort Amount

We need a strong hypothesis to demonstrate Mandelbrot's 1953 result (8). This hypothesis is applied to the construction of the text and consists in minimizing the cost of information when transmitting the signal.

This hypothesis is criticised by Simon. A controversy recounted in [START_REF] Mitzenmacher | A brief history of generative models for power law and lognormal distribution[END_REF] ensued between the two researchers. Later, Mandelbrot supplied quite a different proof for Zipf's law. Each word is a sequence of characters framed by two separators, and is characterised by its length. The coefficient ! can be interpreted as a fractal dimension. This approach is more appropriate for processes that deal with writing [START_REF] Lafouge | Statistiques de l'intellect : lois puissances inverses en sciences humaines et sociales[END_REF]. Mandelbrot shows that ! depends on the size of the vocabulary ! ; more precisely, he obtains [START_REF] Mandelbrot | The Fractal Geometry of nature[END_REF]):

! = !!" (!) !" (!) (10)
where ρ, a character's probability of occurrence, is a difficult parameter to measure and is not independent of V. We necessarily have ! ≤ 1 ! . This formula is difficult to verify. A detailed demonstration of (10) can be found on pages 43-44 [START_REF] Egghe | Power Laws in the Information Production Process: Lotkaian Informetrics[END_REF]. This demonstration can be applied to deconstructed texts, as long as ! ! is big enough. In this case, we speak of a fractal hypothesis, because ! ! can be interpreted as a fractal dimension.

If we choose ! = ! ! , we therefore find, according to (10) :

! = !" (!!!) !" (!) . ( 10b
)
The coefficient's formula (10b) is identical to the result of [START_REF] Li | Random texts exhibits Zipf's law like word frequency distribution[END_REF]. In his article, Li shows that any random text has a word frequency distribution that verifies the generalized Zipf law offered by Mandelbrot (see footnote 1). If we assume that ! = 26, we have, according to (10b): ! ≈ 1,01.

The more the size of the alphabet increases, the closer the coefficient is to 1. Conversely, a text coded with two characters (text with 0 and 1 and with one separator) has a coefficient such that ! ≈ 1,584.

The value of the ! coefficient for random texts varies between 1 and 1.6.

Equation (10) shows that, if ! is considered constant during the deconstruction (which is plausible since the proportion of separators remains constant) then ! ! increases when ! ! decreases.

If ! is considered to be constant during the deconstruction, ! ! . ln (! ! ) doesn't depend on i and is therefore constant if we have:

! ! . Ln ! ! = ! (11)
Equation ( 9) is thus written as:

! ! = -Ln ℎ ! + !. ! ! ! = 1. . ! ( 12
)
where I is the number of iterations for which a generalized Zipf law is always acceptable. The next experiment aims to study the relationship between entropy and effort amount during the deconstruction of a text.

Can the « quasi-linearity » relationship (12) between ! ! and ! ! be verified experimentally?

To summarize, proving formula (12) supposes that the following hypotheses are verified:

- The following experiment will allow us to verify hypotheses (a) and to delve further into the Inter-textual relationship (12) between entropy and effort amount.

Application: the Deconstruction of a Text

The Text's Characteristics

The deconstructed text is Principes de gégraphie humaine (Principles of Human Geography) written in 1921 by Paul Vidal de la Blanche, a French geographer .

9 The lexicometric characteristics, calculated after the segmentation of the text of 212 pages, are:

--Total number of signs: 680,564 --Percentage of separators: 18.9% --Percentage of characters: 81,1 % --Total number of segmented words: 114,730 --Number of identified sources: 12,747 --Number of distinct characters: 82 This text is available online using the URL address displayed in the footnotes. The reader who wishes to test this algorithm can do so on any kind of text and will obtain similar results.

The Characteristics of 27 Deconstructed Texts

In the annex 2, 10 lines of text are deconstructed at the 5th, 10th and 15th iteration. We took away the 27 most frequent characters in the text. The results of the deconstruction can be seen in table 1. Each line (except for the first one) corresponds to a step in the text's deconstruction (namely, the suppression of the characters in column 1). The first line corresponds to the text's characteristics before the deconstruction. It verifies Zipf's law with a coefficient close to 1, as in ! ≈ 1.02.

From left to right, we can read: Column 1, characters deleted in the whole text Column 2, the percentage of deleted characters in the text, Column 3, ! ! , the number of segmented sources, Column 4, ! ! , the amount of effort (see equation ( 6)), Column 5, ! ! , entropy (see equation ( 5)), Column 6, the coefficient ! ! (see equation ( 3)), Column 7, ! ! , the normalisation coefficient (see equation ( 3), Colum 8, ! ! , the squared linear coefficient

The second line (replacing « e » with @) is identical to the first line, since the structure of the text has not actually changed. At the 27 th step, 79.5% of signs are deleted with approximately 1.5% of characters left. The only visible elements are the separators and the jokers @ (see the 15 th deconstruction in the annex). The text is composed of 57 distinct characters. Capital letters have not yet been deleted. We must also remember that the percentage of separators remains constant throughout the entire deconstruction. The amount of effort and entropy decrease steadily at each step. We must check that the number of segmented sources ! ! (see section 3.3) is a decreasing sequence and that ! ! (the adjustment coefficient) is an increasing sequence 10 , according to (10b) and ( 11). The normalisation coefficient increases steadily up to the 15 th iteration. Variations then become irregular. We notice that the necessary and sufficient condition to solve equation ( 4) is verified. As an example, for the first deconstruction, we have: This calculation is verified at each step, with, at the 27 th step:

! ! = ! ! ! = 114
! !" = ! ! !" = !!",!"# !!,!"# = 5.05 ≤ = Ln ! !" = Ln 936 = 6.84
We observe an acceptable adjustment for each source-frequency distribution up to the 27 th iteration (936 sources). The coefficient of determination R 2 is equal to 0.97, which is the value of R 2 in the first adjustment before degrading the text. While the adjustment is not very good for high frequencies, it remains classic in form when we represent the distribution (frequency rank) of a text in general with a Ln-Ln scale. (see figure 1, figure 2) 10 We can show that this sequence is decreasing thanks to the proposition proven in Annex 1 . Indeed, the ! ! sequence decreases through construction and the ! ! -! ! sequence, which equals 12,00 0,00 1,00 2,00 3,00 4,00 5,00 6,00 7,00 8,00 9,00 10,00

!"#,!" ! ! -! ! is also decreasing. Car % ! ! ! ! ! ! ! ! ! ! ! ! 12747

Ln(Frequency)

Ln(Rank)

Other Deconstruction Experiments

We deconstructed our text again (this experiment is useful when studying the variation of entropy according to effort11 ). But this time, instead of deleting characters by decreasing frequency, we deleted them randomly. The initial point and the final point of deconstruction were the same. But the order of characters changed. Two experiments were conducted.

The results can be seen in We have the same properties as in previous cases: ! ! decreases, we have an adjustment with the generalized Zipf law, ! ! decreases. In the 2 nd deconstruction, ! ! 's variation is more chaotic. Entropy and the amount of effort decrease steadily. In the next paragraph, we will see that the variation of these two quantities are linked.

Car ! ! ! ! ! ! ! ! Car ! ! ! ! ! ! ! ! x

Analysis and Discussion

Adjustment with the Generalized Zipf Law

We just saw that the adjustment is sustainable in all three experiments. Even after having been highly degraded, an adjustment with the generalized Zipf's law is still acceptable. We can't, however, consider the generalized Zipf law to be fully satisfactory. Figure 2 shows that the high frequencies stray away from the regression line. Such a deviation is common when one aims to verify Zipf's law. But here, the deformation is accentuated as the deconstruction occurs.

In the three experiments, 27 characters were deleted. Approximately 2% of the characters in the text remained. The text, degraded in this way, is no longer a text. The only things remaining throughout the deconstruction are the text's structure and the length of the words (see the text at the tenth step in Annex 2). ! varies from 1.01 to 1.51; the adjustment coefficient would have had exactly the same variation if the texts had been randomly generated (see the use of formula (10b), section 4.2.4).. If we were to delete all the characters, the text ! !" would only be composed of words containing the @ joker. The final distribution (frequency rank) would correspond to the distribution of the length of the words. We know that this is not a generalized Zipf law.

If we had only found the adjustments as results, such a basic deconstruction approach wouldn't greatly contribute to real language and would simply be a novelty. In the following section, we therefore aim to quantify and connect the different states of the text.

The Linearity between Entropy and Amount of Effort.

In this section, we analyse the relationship between the entropy ! ! and the amount of effort ! ! obtained when deconstructing a text, in light of the theoretical results previously mentioned (see equations (11) (12). We notice a progressive decrease in the amount of effort E i and in the entropy

H i .
After having done a linear regression on the pair of points (! ! , ! ! ) ! = 1. .27 (see table 1) we obtain, for the first experiment, the following equation:

! ! ≈ 5.61! ! + 1.1 ! = 1, . .27 ! ! ≈ 0.996 (see figure 3) (13)
FIGURE 3 -Variations in Entropy according to the Amount of Effort This result is surprising since there is an almost perfect linear relationship between entropy and the amount of effort. This result therefore differs from formula ( 12)

! ! = -Ln ℎ ! + !. ! ! ! = 1. . !
Let us verify hypothesis (see p 9) ! ! . !"(! ! ) is constant and is equal to ! If we calculate the average of the 27 values ! ! . !"(! ! ) (with ! ! varying from 83 to 57) using table 2, we obtain 5.12 a standard deviation of 0.70. This value is different from the 5.61 regression coefficient (see ( 13)) which was previously calculated.

In the two others deconstruction experiments, we obtain a near-perfect linear variation with the exception of the two extreme points, when the deconstruction is nearly complete:

! ! ≈ 5.45! ! + 1.16 ! = 1, . .27 ! ! ≈ 0.998 (see figure 4) (14) ! ! ≈ 5.56! ! + 1.1 ! = 1, . .27 ! ! ≈ 0.998 (see figure 4) (15)
By calculating the average of the values ! ! . Ln ! ! using the table 2, we obtain: -5.10 with a standard deviation of 0.74 -5.19 with a standard deviation of 1.18 These values also differ from the regression coefficients, which are of 5.45 and 5.56. Analysing and interpreting these results is delicate, mainly because the calculation of entropy only depends on segmentation and is the result of the experiment that is linked to deconstruction.

Calculating the amount of effort depends on segmentation and the chosen effort function, as seen here:

! ! ! = Ln (!)
Ln (! ! ) The observed linearity doesn't seem to depend on the adjustment parameters' results. We could have verified linearity (see ( 13),(14) (15)) directly by doing a regression, after having segmented the text and ranked the words by decreasing frequency. The product ! ! . ln ! ! is not rigorously constant. It depends on the adjustment coefficient's calculation. The obtained value is different from the regression coefficient's calculation. Such a difference could be due to a lack of precision when calculating the ! coefficient, or, more largely, to the chosen model, which may not be accurate -or complex enough -throughout the deconstruction process.

Generalizing the Result

The results of the 3 experiments can be analyzed together. If we conduct a linear regression of the 77 pairs: (! ! , ! ! ) ! = 1. .77, where (! ! , ! ! ) are the entropy and the effort amount of the degraded text ! ! , we obtain a quasi-perfect linear regression (after removing duplicates):

! ! ≈ 5.49! ! + 1.16 ! = 1, . .77 ! ! ≈ 0.997 (see figure 5) (16)
We notice that, when the text is highly degraded (low entropy and low amount of efforts), the dots aren't completely aligned. However, we feel that this linearity isn't as questionable as the graph with the Zipf curve. Our hypotheses and the formula ( 12) -which we proved in section 4 with the help of Mandebrot's joint results combining entropy and amount of effort -are insufficient to explain the linearity observed in (13,14,15,16).

Let us first enounce the proven result in a more general way: Let us call a text T and its set of distinct characters C (between 50 and 80 characters for a text written in French), thus ! ! k = 1,..n is a sequence n characters (between 20 and 30 following the deconstruction experiments), representing approximately 80% of the characters of the text. We then have the following result: Let ! ! be the text obtained after deleting from T the sequence of k characters ! = 1, . . ! ! ≤ !. We can then confirm that, on the basis of this study, if we call H k the entropy and E k the quantity of effort, there is a linear relationship of the type:

! ! = ! + !. ! ! ! = 1, … ! ! > 0 ! > 0 ( 17
)
Where P is the number 12 of deconstructed texts. In the example used, P is equal to 77.

In order to confirm the universality of such an inter-textual linearity, we present the degradation of another text in Annex 4. The deconstructed text is "The Value of Science" by Henri Poincaré, a philosophical essay of approximately 180 pages, published in 1901. A bibliometric analysis of this text has already been conducted in [START_REF] Lafouge | Statistiques de l'intellect : lois puissances inverses en sciences humaines et sociales[END_REF], chapter 4, section 4.2.1.

We offer a possible explanation for this linearity with the following model: More generally, if we assume that, at each step i, the source-frequency distribution can be written as: 12 The number of texts that can be deconstructed is very high ≈ 10 ! !

! ! ! = !. exp -!. ! ! ! ! > ! ! = 1. . ! (18)
Where ! ! is a sequence of functions (called effort function), I is the number of deconstructions. When we name, as we did before, ! ! as entropy and ! ! as the amount of effort, we can show (see Annex 3) that the entropy and amount of effort are linked by the relationship:

! ! = -Ln ! + !. ! ! (19)
In this case, the A B constants are independent of i. Throughout the text's deconstruction, each step i is characterised by an effort function ! ! .This formalism uses the effort function concept [START_REF] Lafouge | An interpretation of the effort function through the mathematical formalism of exponential informetric process[END_REF]) [START_REF] Lafouge | The Source-effort Coverage of an Exponential Informetric Process[END_REF].

If we choose ! ! ! = !"(!) !"(! ! )
, we find the previous result, which is ! ! = ! !" (! ! ) meaning that A is constant. This model remains incomplete and isn't fully satisfactory, but provides us with new research perspectives.

Conclusion

This article's goal isn't to provide a new explanatory model for Zipf's law. However, the results that we obtained, and which could not be predicted, shed new light on the term "paradigm" used to qualify Zipf's law in the introduction. By degrading a text, we wanted to jointly explore both interpretations of Mandelbrot. What interested us was to confront both hypotheses. This naturally led us to focus on the entropy and the amount of effort of a degraded text. The reader may feel baffled by the frustrating method that consists in degrading a text mechanically with no regard for lexicographic knowledge (differentiating vowels from consonants…) or grammar (respecting the type of words: nouns, adjectives, conjunctions…). But this deconstruction can be applied to any text and, in this way, is a universal method. The only information used during the degradation process had to do with the status of the signs in the text as either being characters or separators. The number of steps for the algorithm corresponded roughly to the number of characters (between 20 and 30) that were essential to write the text. At the end of the deconstruction, only a few rare characters (in the statistical sense) were still visible. Wentiam Li's results [START_REF] Li | Random texts exhibits Zipf's law like word frequency distribution[END_REF] are challenged by the fact that the generalized Zipf law applies to any deconstructed text. Li concludes his study by saying that "In conclusion, Zipf's law is not a deep law in natural language as one might first have thought". We could have thought that the adjustment wouldn't be valid after such a significant reduction of the number of sources (divided by 10 in our example).

The deconstruction has highlighted an unexpected result. If, like Mandelbrot (see 4.2.2), we define C as being the ratio between the amount of effort and entropy ! = ! ! , this article shows that, for the average cost of information -known as ! !" -the difference between two states of the same text, known as ! and !, such that

! ! !! ! ! ! !! !
, is constant. As in the original formulation of Zipf's lawnamely that the rank by frequency is constant -the inter-textual linearity between entropy and effort seems to be a paradigm. This result puts the work of Mandelbrot in perspective concerning language generation and, more generally, concerning the models on commutative optimization principles. But we would need to confirm this by studying the variation range of the C coefficient on a larger amount of texts. We can however suppose that, since the adjustment coefficient ! varies little and stays close to 1, the variation range of ! (see equation ( 17)) would not be very large. Can we say that it is another expression of the Zipf 's law, or a consequence of it? Although the explanation given for equation ( 18) -an another formulation of the generalized Zipf law -is an approximation and isn't entirely satisfactory, it does, however, open new research perspectives. What happens at the end of the deconstruction, when the rare characters left also disappear? We will end this brief conclusion with a warning by Stumpf (Stumpf and Porter, 2012) who analyzed the relevance and the validity of various studies on power laws in general and in various fields: "Power laws do have an interesting and possibly even important role to play, but one needs to be cautious when interpreting them". where. ! ∈ 1. . ∞ . This function is strictly decreasing and continuous. Then we have: ! 1 = !" !. For x > 1, we have:

! ! = 1 ! -1 . (1 - 1 ! !!! )
If x tends towards ∞ then !(!) tends towards 0. The theorem of intermediate values then allows us to say that there is a unique value x such that ! ! = !.

Proposition

We have the following results:

If S i a decreasing sequence and M i -S i is a decreasing sequence, then β i is an increasing sequence.

Proof

We show the identity:

we have also :

! ! is
! ! = -Ln ℎ ! + ! ! ℎ ! ! ! ! ! ! !!! Ln ! ! ! ≥ 1, 0 < ℎ ! according to ! ! ! = !" (!)
!" (! ! )

we have: !). ! ! (!) we have:

! ! = -Ln ℎ ! + ! ! . Ln ! ! . ℎ ! ! ! ! ! ! !!! . ! ! ! according to ! ! = - ! ! ! ! !!! (
! ! = -Ln ℎ ! + ! ! . Ln ! ! . ! !
we obtain a relationship between entropy and amount of effort:

! ! = ! ! + ! ! . ! ! ! ! > 0, ! ! = ! ! . Ln ! ! > 0 (9)
Demonstration of the relationship (16) between the amount of effort and entropy in paragraph 6.3

! ! ! = !. exp -!. ! ! ! ! > ! ! = 1.
. ! where ! ! is a strictly increasing unbounded effort function we suppose :

! ! ! !!! ! = 1 then ! ! = -ln ! + !. ! ! Proof ! ! = !"( ! !!! ! ! ! ). ! ! ! ! ! = - !"(!. exp -!. ! ! ! ! !!! . !. exp -!. ! ! ! ! ! = - !" ! . !. exp -!. ! ! ! - -!. ! !!! ! !!! ! ! ! . !. exp (-!. ! ! ! ) ! ! = - !" ! . ! ! ! !!! ! + !. ! !!! ! ! ! . ! ! (!) ! ! = -ln ! . ! ! ! !!! ! + !. ! ! ! !!! ! . ! ! (!) ! ! = ! ! ! !!! ! . ! ! (!) ! ! = -ln ! + !. ! ! If ! ! ! = !"(!) !"(! ! )
is the effort function defined by Mandelbrot : 5,308 sources ! ! = 1,17 la r@ch@rch@ d@ la véri@é doi@ ê@r@ l@ bu@ d@ @o@r@ ac@ivi@é ; c' @@@ la @@ul@ fi@ qui @oi@ dig@@ d' @ll@. @a@@ dou@@ @ou@ d@vo@@ d' abord @ou@ @fforc@r d@ @oulag@r l@@ @ouffra@c@@ humai@@@, mai@ pourquoi ?

! ! ! = !. exp -!. !" ! ! ! = !. 1 ! ! ! ! ! = ! ln (! ! )

  -(a) a generalized Zipf law at each step of the deconstruction, --(b) an effort function equal to ! ! ! = !" (!) !" (! ! ) ! = 1, . . ! ! (function used by Mandelbrot) at each step of the deconstruction -(c) a constant ! ! . !"(! ! ) product (the consequence of Mandelbrot's fractal hypothesis).

  28 ≤ = Ln ! ! = Ln 12,747 = 9.45

FIGURE 1 -

 1 FIGURE 1 -The Source-frequency Distribution throughout the Deconstruction

FIGURE 4 -

 4 FIGURE 4 -Variation in Entropy according to the Quantity of Effort

FIGURE 5 -

 5 FIGURE 5 -Variation in Entropy according to the Quantity of Effort

  Let a pair (! ! , ! ! ) ∈ 1, . . ∞ ×ℝ ! , then there exists a unique real !, ! ≥ 1 such that:

  decreasing and also ! ! -! ! hence, we can write:Then we have Y < 0 hence, we can write: ). ! ! (!) assuming that the distributions are generalized Zipf law, using the equation! ! ! ≈ ! ! ! ! ! ! ! > 0 ! ! ≥ 1 ! = 1, . . ! ! with ℎ ! =

Table 2

 2

	Column 1, 7: deleted characters
	Column 2, 8 : number of segmented sources
	Column 3,9 : effort amount
	Column 4,10 : entropy
	Column 5,11 : adjustment coefficient !.

Table 2 : Lexical Results and Deconstruction Adjustments

 2

		12747	1.02	6.77	1.05	'	12747	1.02	6.77	1.05
	m 12745	1.02	6.77	1.05	a 12743	1.01	6.73	1.04
	é 12741	1.02	6.77	1.05	b 12743	1.01	6.73	1.04
	à 12739	1.02	6.77	1.05	c 12723	1.01	6.73	1.04
	è 12735	1.03	6.77	1.05	d 12685 1.00	6.68	1.04
	r 12704	1.03	6.77	1.05	e 12404	0.97	6.54	1.04
	p 12663	1.03	6.77	1.05	f	12362	0.98	6.54	1.04
	d 12594	1.03	6.76	1.05	g 12306	0.98	6.53	1.04
	c 12488	1.03	6.71	1.05	h 12270	0.98	6.53	1.04
	o 12378	1.03	6.69	1.05	i	12049	0.97	6.49	1.05
	u 12160	1.02	6.67	1.06	j	12015	0.98	6.48	1.05
	l	11844	0.99	6.52	1.06	l	11742	0.93	6.28	1.05
	e 10734	0.95	6.35	1.09	m 11452	0.93	6.25	1.06
	j	10674	0.95	6.33	1.09	n 10613 0.90	6.09	1.08
	y 10610	0.95	6.32	1.09	o	9795	0.87	5.95	1.11
	s	9310	0.90	6.04	1.13	p	9186	0.86	5.85	1.12
	n	8209	0.83	5.74	1.16	q	9134	0.86	5.84	1.12
	t	6481	0.75	5.30	1.22	r	7084	0.80	5.57	1.2
	i	4497	0.64	4.77	1.3	s	5161	0.70	5.04	1.29
	a	2671	0.51	4.01	1.43	t	3248	0.57	4.38	1.41
	'	2662	0.48	3.85	1.42	u	2204	0.44	3.65	1.48
	b	2278	0.45	3.70	1.46	v	1805	0.41	3.42	1.51
	h	1903	0.43	3.56	1.51	x	1652	0.39	3.31	1.53
	f	1626	0.41	3.38	1.54	y	1467	0.38	3.26	1.57
	q	1479	0.38	3.21	1.54	à	1459	0.38	3.23	1.56
	g	1129	0.35	2.98	1.59	é	1021	0.33	2.8	1.53

  This text is available on line using the URL http://fr.wikisource. org/wiki/ La_Valeur_de_la_Science, website consulted in February 2015.Variations in Entropy according to the Amount of EffortResult of deconstruction at the 4 th and 12 th sources )iteration la recherche de la vérité doit être le but de notre activité ; c' est la seule fin qui soit digne d' elle. sans doute nous devons d' abord nous efforcer de soulager les souffrances humaines, mais pourquoi ?

	The text's Characteristic
	Total number of signs : 344,292
	Percentage of separators : 20,2%
	Percentage of character : 79,8 %
	Total number of degmented words : 61,086
	Number of identified sources : 5,853
	Number of distinct character : 75
	Adjustment coefficient !:1,12

Amount of effort Entropy

	7
									Car	E	H
										0.96	6.28
									e	0.96	6.28
	6								s	0.96	6.27
									n	0.95	6.22
									t	0.94	6.16
									i	0.92	6.1
	5								a u	0.91 0.87	6.02 5.86
									r	0.85	5.71
									o	0.8	5.5
	4								l c	0.72 0.68	5.15 4.93
				y = 5,5375x + 1,0353					d	0.62	4.62
				R 2 = 0,9941					p	0.57	4.36
	3								m é	0.53 0.49	4.11 3.88
									'	0.46	3.72
									q	0.43	3.54
	2								v f	0.4 0.38	3.33 3.18
									b	0.36	3.05
									g	0.34	2.92
	1								h à	0.33 0.33	2.81 2.78
									x	0.32	2.69
									j	0.31	2.6
	0								è y	0.3	2.53 2.47
	0,3	0,4	0,5	0,6	0,7	0,8	0,9	1	1,1

1,811 sources ! !" = 1,31

Chains, unlike words, are devoid of meaning ; semantics are absent.

In fact, in this case, a word in a text is characterised by its frequency alone: variables and probabilites seem to merge.

We find in[START_REF] Simon | On a class of skew distributions functions[END_REF] paragraph 4 "The empirical distributions Word frequency" many ideas on the increased size of the lexicon: I quote p. 434 : "An author writes not only by processes of association -i.e. sampling earlier segments of word sequence -but also by processes of imitations -i.e. sampling segments of word sequences from other works he has written, from works of onther authors... "

This property is true since we delete the same character throughout the text.

This hypothesis is verified at all steps of the deconstruction.

This text can be found at the following address: https://archive.org/details/principesdegogr00blacgoog website consulted in Février 2015.

We could indeed assume that the observed results arose from the fact that the characters were deleted by decreasing frequencies.