Salvador Mascarenhas

Benjamin Spector

Emmanuel Chemla

Philippe Schlenker

Philip Johnson-Laird

Maria Aloni

A note on the cardinalities of sets of scalar alternatives

Introduction

Scalar implicatures as in (1) are cancellable inferences that appear to arise from considering certain alternative sentences.

(1) I saw John or Mary.

Alternative: I saw John and Mary. Scalar Implicature: The speaker did not see both John and Mary

When first described by Paul Grice (1967Grice ( , published in 1989)), scalar implicatures were taken to be the result of processes of pragmatic inference. In a nutshell: starting from what the speaker of (1) literally said, the hearer will consider what stronger statements the speaker chose not to utter, and conclude that the speaker believes all such statements to be false, as long as their falsehood is compatible with what she literally said. More recent approaches reject the pragmatic dimension of the phenomenon, proposing unpronounced operators that derive scalar implicatures as a matter of literal meaning rather than the result of pragmatic inferences [START_REF] Chierchia | The grammatical view of scalar implicatures and the relationship between semantics and pragmatics[END_REF].

In this note, I show that there is one technical component, universally employed in formal theories of scalar implicature, that has very puzzling consequences under the assumption that these theories ought to be psychologically tenable. The issue can be summarized as follows. Every modern theory of scalar implicature makes crucial use of a set of scalar alternatives, sentences that are in a precise sense related to the sentence uttered by a speaker. These are the alternatives that a hearer will take into consideration when thinking of what the speaker could have said but chose not to. They are the alternatives that unpronounced operators in non-pragmatic approaches must access in order to deliver truth conditions for sentences containing them. I show that the cardinalities of these sets increase at very fast rates, and moreover that, even for sentences with a relatively small number of coordinated clauses, the cardinalities of alternative sets are very large numbers. If the theories of alternatives considered here are making claims about psychological processes, then these claims are very difficult to square with what we know about alternatives in grammar and in reasoning. On the other hand, if these theories are to be taken as mathematical idealizations or theories of pragmatic competence, then it becomes necessary to investigate what psychologically tenable heuristics might implement this competence. It does not follow from the facts I report here that modern formal approaches to scalar implicature are doomed. Instead, I point out a collection of puzzling and previously unnoticed facts that require answers.

2 Alternative sets and their cardinalities I take the work of [START_REF] Chierchia | Scalar implicatures, polarity phenomena, and the syntax/semantics interface[END_REF], [START_REF] Sauerland | Scalar implicatures in complex sentences[END_REF], [START_REF] Spector | Scalar implicatures: Exhaustivity and Gricean reasoning[END_REF][START_REF] Fox | Free choice disjunction and the theory of scalar implicature[END_REF] to be representative examples of formal approaches to scalar implicature. All of these theories assume or propose some mechanism that generates formal alternatives as a function of a source and possibly a context. I restrict my attention to explicit proposals about the set of formal alternatives that have not been falsified at the time of writing.1 There are three approaches in the literature that fit this criterion.

Positive propositions

While making no commitments about the inner workings of the alternative-generating procedure, [START_REF] Spector | Scalar implicatures: Exhaustivity and Gricean reasoning[END_REF] theory makes use of the set of positive propositions that can be constructed from a source S. The notion is simple: given a source S with a set A of atomic propositions, the set P + (A) of positive propositions based on A is the closure of A under conjunction and disjunction. Thus, for a simple source S = a ∨ b, we get the alternative set

P + ({a, b}) = {a, b, a ∧ b, a ∨ b}.
The set of positive propositions based on a set of atoms A corresponds to the set of nonconstant monotonic (increasing) Boolean functions of |A|-many variables. A Boolean function is monotonic if switching any of its inputs from false to true can never bring about a switch from true to false in the output. Thus, conjunction and disjunction are monotonic functions in this sense, but not negation or implication. Monotonic Boolean functions on n variables are precisely those that can be defined with the Boolean meet and join. The "non-constant" proviso simply excludes the verum and falsum functions from the list.2

The number of monotonic Boolean functions on n variables M(n) was defined by Dedekind in 1897 (see for example [START_REF] Kleitman | On Dedekind's problem: The number of monotone boolean functions[END_REF]. Accordingly, the set of all M(n)

for n ∈ N is known as the set of Dedekind numbers. [START_REF] Sauerland | Scalar implicatures in complex sentences[END_REF] proposes a syntactic substitution mechanism where only terminal nodes corresponding to binary connectives are the targets of substitutions. For σ a parse tree, the set of Sauerland alternatives consists of all possible trees created by substituting for each binary connective in σ a connective from the set {∧, ∨, L, R}. The connectives L and R stand for left and right respectively, and take their names from their semantics: ϕ L ψ = ϕ , and conversely for R.

To allow for straightforward comparison with [START_REF] Spector | Scalar implicatures: Exhaustivity and Gricean reasoning[END_REF] positive propositions, I consider only trees σ that contain no repetitions of atomic propositions. This way we can express the cardinality of a set of alternatives as a direct function of the number of distinct atoms in both approaches. For a σ with n atoms, we find n -1 positions for binary connectives. Since we have four possible connectives to plug into each of these positions ({∧, ∨, L, R}), we find 4 n-1 Sauerland alternatives for a tree with n atoms. 3Sauerland alternatives therefore grow at an exponential rate.

Katzir and Fox complexity

Katzir (2007) and [START_REF] Fox | On the characterization of alternatives[END_REF] propose a different mechanism for generating formal alternatives based on a notion of structural complexity. As explained by [START_REF] Katzir | Structurally-defined alternatives[END_REF], there are contexts where, in order to derive the observed scalar implicatures, a non-monotonic expression must be compared with a formal alternative that is structurally simpler than it without being its scale-mate. For example:

(2) I doubt that exactly three semanticists will sit in the audience.

(3) I doubt that three semanticists will sit in the audience.

An utterance of (2) suggests that the speaker does not find it unlikely that at least three semanticists will be in attendance. This means that (3) must be a formal alternative to (2). But the quantifier in ( 3) is not a scale-mate of the one in (2). The relevant relation between the two sentences seems to be one of complexity: (3) is less complex than (2), and therefore it is an alternative relevant for pragmatic computations. [START_REF] Katzir | Structurally-defined alternatives[END_REF] theory is given in a fully explicit way. First we need to define a substitution source. This will contain all elements that can legally substitute elements of the speaker's utterance when generating alternatives. The rationale is that nothing in the substitution source can be more complex than the original utterance.

(4) Substitution source: Let σ be a parse tree. The substitution source for σ is the union of the lexicon of the language with the set of all subtrees of σ .

Next, we define a relation of structural complexity on syntactic structures. Finally, we define the set of structural alternatives in terms of structural complexity.

(5) Structural complexity: Let σ , σ be parse trees. If we can transform σ into σ by a finite series of deletions and replacements of constituents in σ with constituents of the same category taken from the substitution source of σ , we say that σ is at most as complex as σ , in symbols σ σ .

(6) Structural alternatives: Let σ be a parse tree. The set of structural alternatives for σ is defined as

A(σ ) := {σ : σ σ }.
The alternative sets produced by this procedure are surprisingly large even for very small numbers of atoms in the source. In order to make discussing concrete examples easy, let us apply this theory to a propositional language, where in particular each binary connective is immediately dominated by a three-way branching node with further trees in the left and right branches, and a binary connective in the middle. Then, even if we exclude the lexicon (other propositional atoms) from the substitution source (4), a source

σ = a ∧ b, with structure [[a] ∧ [b]],
will have 20 syntactically distinct alternatives. As a reference point, there are 16 possible propositions that can be formed out of two atoms, and only four positive propositions. It is instructive to go over a simple example of the procedure at work, so I will presently prove that there are indeed 20 alternatives for this source.

Ignoring the connective for now, there are three nodes in a ∧ b that can be the target of a substitution: the root of the tree, the left node, and the right node. There are three subconstituents in the source: a, b, and a ∧ b. This means:

1. Substitutions at the root return three possible trees: a, b, and a ∧ b, in other words the set of subconstituents in the substitution source themselves.

2. Substitutions at the left and right nodes (still ignoring the connective) amount to all ordered pairs that can be formed from elements of the substitution source. The substitution source has three elements, so we get 3 2 = 9 possible substitutions. For example, and using ⊗ as a placeholder for the binary connective we are still ignoring at this stage:

[a ⊗ a], [b ⊗ a], [(a ∧ b) ⊗ b].
3. These nine possible substitutions each have one slot for a connective, represented as ⊗ in the examples just given. This is the only possible target for substitutions of connectives. Assuming two possible connectives (∧, ∨) for each of these nine trees, we get 2 • 9 = 18 distinct trees.

4. One of the trees generated in step 3. is a∧b, from [a⊗b] in step 2. with substitution of ∧ for ⊗. This formula was already counted in step 1. So we remove this tree from the count of step 1., and get a total number of 2 • 9 + 2 = 20 distinct trees.

In the example I just gave, I did not use the deletion transformation made available by the definition in (5). This was legitimate because any deletions of nodes from a source a ∧ b will be identical to substitutions of one of the terminal nodes for the root of the tree. Deletions in this case were subsumed under a special case of substitutions. This is not so in the general case, and indeed tallying up all of the alternatives that are generated by the exact procedure in ( 5) is harder than it might seem. Since the precise number is not of intrinsic interest, I will instead establish a clear and intuitive lower bound for these syntactic alternatives. I make the following simplifying assumptions.

1. Atomic propositions occur only once in a parse tree. This assumption serves the same purpose as above, in the case of Sauerland alternatives. It allows for a straightforward comparison with the semantic approach of Spector (2007).

2. I adapt the definition in (5) to outlaw (A) deletions and (B) substitutions of elements for points in the tree other than terminal nodes. That is: for σ the source, we say that σ is a less-complex transformation of σ just in case σ is the result of substituting subconstituents of σ for terminal nodes of σ . Note that this algorithm is a restriction of the original. It will always generate a subset of the alternatives generated by the original procedure. Indeed, in all but the trivial case, it will generate a proper subset of what the original procedure generates.

With these assumptions, we can show that we get, for a source with n atoms,

(2n -1) n • 2 n-1
distinct syntactic alternatives. 4 Here is why:

1. There are n terminal nodes and 2n -1 subconstituents to be substituted with repetitions for any of those terminal nodes. 5 Whence (2n -1) n .

2. There are n -1 positions in which to place one of two binary connectives, so there are 2 n-1 possible strings of connectives to be plugged in, for each of the connectiveless tree-skeletons given by the term in 1. above. Whence the multiplication by 2 n-1 .

4 This expression provides a lower bound for the theories of alternative generation by [START_REF] Katzir | Structurally-defined alternatives[END_REF] and [START_REF] Fox | On the characterization of alternatives[END_REF]. It is an open question exactly how individual variants of these theories will fare. For example, [START_REF] Trinh | Constraining the derivation of alternatives[END_REF] propose a constraint on these theories that potentially will prune elements from the substitution source, making it smaller, without ever adding elements to it. Consequently, at least in some cases, [START_REF] Trinh | Constraining the derivation of alternatives[END_REF] will generate fewer alternatives than the original theories. Yet, since I have shown that the syntactic alternatives are an exponential function of the number of elements in the substitution source, the theory by [START_REF] Trinh | Constraining the derivation of alternatives[END_REF] will still generate sets of alternatives that grow exponentially with the size of the substitution source.

5 This is perhaps easiest to see for a simple propositional language exclusively with binary connectives. We can define a function s recursively that returns the number of subconstituents of a formula in this language: s(p) = 1 for p an atom, s(ϕ ⊗ ψ) = s(ϕ) + s(ψ) + 1 for ⊗ a binary connective. We can also define a function a that returns the number of atoms (i.e. terminal nodes) in a formula: a(p) = 1, a(ϕ ⊗ ψ) = a(ϕ) + a(ψ). It's easy to prove by induction on the complexity of formulas that s(ϕ) = 2a(ϕ) -1.

Discussion

Table 1 summarizes the findings in this article. It includes the total number of propositions made out of n atoms merely to provide an anchor that will be familiar to the reader.

(2n -1) n • 2 n-1
For all three algorithms, the size of the output set of alternatives increases fast as a function of the number of atoms in the input. This is particularly striking even for small numbers in Table 1 in the case of [START_REF] Katzir | Structurally-defined alternatives[END_REF] theory. Indeed, in this theory the number of predicted alternatives grows at an exponential rate. While less visible within the range of small numbers in Table 1, the Sauerland algorithm is still clearly an exponential function of the length of the input. The positive-propositions approach fares no better, and grows at an exponential rate as well [START_REF] Kleitman | On Dedekind's problem: The number of monotone boolean functions[END_REF]).6

Thus, the three theories of alternative generation considered here have outputs that grow at least as an exponential function of the size of the input. These three theories are quite distinct, and they constitute, as far as I can tell, the most recent explicit algorithms for generating alternatives. Since these theories are representative of the state of the art, it is significant that all three algorithms are intractable. 7

Algorithmic-level accounts and the values for small numbers

The values seen in Table 1 for small values of n are already rather large. This is again less obvious for the Sauerland and positive-propositions approach, which seem manageable even for n = 4 when compared to the syntactic approach. I submit that these values for small numbers are interesting in their own right, irrespective of considerations of relative growth.

Gilbert found that the nth Dedekind number is greater than or equal to 2 ( n n/2 ) , where a b notates a binomial coefficient. Later work improves on this lower bound [START_REF] Kleitman | On Dedekind's problem: The number of monotone boolean functions[END_REF][START_REF] Kleitman | On Dedekind's problem: The number of isotone Boolean functions II[END_REF], but the earlier lower bound is already clearly exponential.

7 Ideally, one would want to show that the problem of generating alternatives is in itself intractable, not just that our current best algorithms are inefficient. Unfortunately, such a result is not forthcoming. There is simply nowhere near enough of a consensus in the field about precisely what the empirical scope of theories of scalar implicature ought to be. Consequently, we cannot articulate a general computational-level theory of alternative generation of which the three theories considered here would be algorithmic-level instantiations. Instead, these three theories constitute competing computational-level theories themselves.

The output of an algorithm may grow as an exponential function of the input without ipso facto being cognitively untenable. It suffices that only small enough numbers ever occur in practice. However, there is good reason to think that we need for our theories to handle a range of numbers that already produce alternative sets that are far too big.

For concreteness, Table 1 tells us that the most economical alternative-generating algorithm will produce 64 alternatives to a sentence as in (7) below.

(7) Ann and Bill, or Carl and Dan will come to the party tonight.

While it is clear that ( 7) is not a very easy sentence to parse, it is by now well established that speakers of English can process and reason systematically on the basis of sentences with this kind of structure. [START_REF] Walsh | Coreference and reasoning[END_REF] studied deductive reasoning problems with premises with four atomic propositions along the lines of ( 7). Moreover, the mistakes that humans make when reasoning with and about sentences like (7) constitute precisely the kind of inference-making behavior that we would expect if they were interpreting (7) with a strong scalar implicature to the effect that "Only Ann and Bill, or else only Carl and Dan will come to the party tonight" [START_REF] Mascarenhas | Formal semantics and the psychology of reasoning: Building new bridges and investigating interactions[END_REF]. This means that it is entirely fair to expect our theories of scalar implicature to apply to sentences with four atoms as in (7). It also suggests that we can ask the same questions about sentences with five or more atoms, where the numbers of predicted alternatives become even greater.

The literature on the psychology of reasoning offers theories of reasoning with and about alternative propositions. In particular, mental model theory (Johnson-Laird 1983) is an account of the human faculty of reasoning whose hallmark is precisely a theory of how humans entertain and manipulate alternative propositions, called alternative mental models. It is reasonable to ask whether the formal alternatives in theories of scalar implicature fit in the mental models picture of reasoning with and about alternative mental models.

The short answer is no. Mental model theory has established correlations between the number of alternative mental models under consideration and humans' performance in reasoning tasks. Every study within this paradigm indicates that humans can efficiently reason with/about rather small numbers of alternative mental models at any given point in time, with five to seven concurrent mental models being the limit. 8 These results cannot be squared with the predictions of even the most economical of current theories applied to a sentence as in ( 7). This observation is immediately problematic for pragmatic accounts, which propose that an online inferential process is responsible for scalar implicatures. If this process is just general-purpose reasoning, then it flies in the face of what the only extant account of reasoning with alternatives from psychology has found. If this process is in fact not general-purpose reasoning, but part of a module specially dedicated to pragmatic reasoning, then it seems to have properties dramatically different from those of generalpurpose reasoning, a puzzling fact in its own. The situation is different for grammatical approaches. Results from mental-model theory are of no consequence here, since grammatical approaches propose that the faculty of language is responsible for the relevant computations, and not a reasoning faculty of a general or special kind. Indeed, when framed within the faculty language instead of reasoning, the puzzle I raise here becomes a very general question, since alternatives just like those in scalar implicature play a role in the semantics of "only" and other related operators. What specific mechanisms allow the faculty of language to handle exponentially growing numbers of alternatives so seamlessly?

Alternative generation at the computational level

To the best of my understanding, the theories reviewed here were not conceived as algorithmic-level theories. Instead, they were designed as part of computational-level theories of pragmatics. At this level of analysis, exponential blowup is not necessarily a damning feature, provided that these computational-level theories can be coupled with algorithmic-level implementations that are tractable. Indeed, we know that these computational-level theories can in principle be dramatically reduced in (algorithmic) practice. The syntactic approach for one will generally produce great amounts of syntactically distinct but equivalent alternatives. For example, a source like a ∨ b ∨ c will generate the alternatives a, a ∧ a, and a ∧ a ∧ a.

The positive propositions approach, due to its semantic nature, does not include equivalent-but-distinct alternatives, yet it still produces alternatives that are altogether idle in the computation of scalar implicatures.

It is useful to consider a simpler variant of (7) above to illustrate how both theories generate unnecessary alternatives. Sentences as in (8), first discussed in the formal pragmatics literature by [START_REF] Spector | Scalar implicatures: Exhaustivity and Gricean reasoning[END_REF], have the implicature in (9).

(8) Either John and Mary or Bill will come to the party.

(a ∧ b) ∨ c (9) Neither John nor Mary will come, or else Bill won't come.

(¬a ∧ ¬b) ∨ ¬c

For n = 3 as in the example in (8), we get 17 equivalence classes of syntactic alternatives in [START_REF] Katzir | Structurally-defined alternatives[END_REF] theory, versus 18 for [START_REF] Spector | Scalar implicatures: Exhaustivity and Gricean reasoning[END_REF] positive propositions approach (see Mascarenhas 2014 for a detailed discussion). The alternative present in the positive propositions approach but not in the syntactic theory is

(a ∧ b) ∨ (a ∧ c) ∨ (b ∧ c).
In words: "at least two of a, b, and c." First, I point out that this alternative is not required to derive the observed implicature in (9). In fact, we actually need only one alternative to derive the implicature, namely (a ∨ b) ∧ c, which is present in both theories, and whose negation is the observed implicature in (9). Second, the missing alternative does not give rise to an observed implicature. The implicature would be that only one of a, b, and c is true, which contradicts the literal meaning of (8). Consequently, while the syntactic theory generates far more alternatives than the positive-propositions theory for sentences as in ( 8), an algorithmic-level implementation of this theory that only considered equivalence classes of substitutions would generate fewer alternatives than its competitor.

But finding such an implementation is far from trivial, for note that it is not enough to simply prune the set of alternatives after the fact, or to ask during the computation, for each new potential alternative, whether it is equivalent to an alternative already included in the output. Either of these strategies still requires that the algorithm generate astronomical numbers of alternatives, even if it then decides not to include them in the output. 9

Zooming out, the theories reviewed here may be right as computational-level accounts.

But plausible algorithmic-level implementations do not exist at the time of writing of this article, and designing them is far from a straightforward task.

Conclusion

Modern theories of scalar implicature make crucial use of sets of alternatives. In this article I showed how the three most precisely defined alternative-generating procedures in the literature (a) generate very large sets even for the case of small inputs and (b) generate sets whose size increases (at least) exponentially as a function of the input. These facts highlight a puzzling state of affairs that deserves investigation. If these theories of alternative-generation should have algorithmic-level psychological import, then one is at a loss trying to integrate them within the broader existing research on cognitive mechanisms that deal with reasoning and with alternatives. If on the other hand they are to be taken only as theories of pragmatic competence, then we must ask what mechanisms implement this description of competence; what heuristics does the mind use that allow it to consider only a manageable subset of these large collections of alternatives, while deriving the observed implicatures.

9 A strategy that is in principle available is for the algorithm to start from a manageable set of contextually available alternatives and ask for each of those potential alternatives whether it is generated by the general theory of alternatives. The first step in finding out whether this is a solution to the challenge I raise in this article is to determine the complexity of the problem for a non-deterministic machine. This is because plausibly the problem would now best be framed in terms of checking whether a particular alternative is in the set of alternatives, rather than generating the full set in itself. I do not have an answer to this question, which would require a complete complexity result for each of the algorithms discussed here in the non-deterministic case, rather than deterministic lower bounds as I establish.

Be that as it may, a reviewer points out that the idea of starting with an independent set of candidate alternatives runs against arguments by [START_REF] Fox | On the characterization of alternatives[END_REF] that context does not freely restrict the alternatives to scalar implicature. In sum, it is in principle possible that alternatives aren't generated fully and then winnowed, but rather one starts from a small set of contextual alternatives and then tests each of those to see if it can be generated. But it is an open question whether even this process is tractable, and it is unclear whether a complete theory of scalar implicature can be built from such a theory of alternatives.

  The sequence grows very rapidly.

	As of early 2021, no closed-form expression for M(n) is known, and exact values
	for M(n) are only known for n ≤ 8. The set of positive propositions with n atoms
	inherits the mathematical unwieldiness of Dedekind numbers, since, for A a set of atomic
	propositions, |P + (A)| = M(|A|) -2.
	2.2 Syntactic substitution approaches
	2.2.1 Sauerland alternatives

Table 1 :

 1 Number of alternatives by procedure for a source with 2, 3, 4, and n atoms.

		2	3	4	n
	Propositions	16 256 65, 536	2 2 n
	Positive propositions	4	18	166 Dedekind numbers: M(n) -2
	Sauerland (2004)	4	16	64	4 n-1
	Katzir (2007) lower bound 18 500 19, 208

For example, the classical theory by[START_REF] Horn | On the semantic properties of the logical operators in English[END_REF]) is an explicit theory of alternatives in terms of substitutions of connectives from lexically determined scales, but it fails to generate alternative sets rich enough to derive implicatures as in (8), an example discussed at the end of this article.

See also for example the article "Monotone Boolean function" in the online Encyclopedia of Mathematics (Springer) for a connection between monotone Boolean functions and disjunctive normal forms without negations, i.e. positive propositions.

These are syntactic alternatives, they may well include alternatives that are equivalent while being syntactically distinct. Indeed,[START_REF] Spector | Comparing exhaustivity operators[END_REF] has shown that there are 3 n -1 2 equivalence classes of Sauerland alternatives. Consequently, for all cases except the trivial n = 1, the equivalence classes of Sauerland alternatives will be smaller than the syntactic alternatives.

As mentioned before, no closed-formed expression is known for calculating Dedekind numbers.[START_REF] Kleitman | On Dedekind's problem: The number of monotone boolean functions[END_REF] attributes the earliest proof of a lower-bound for Dedekind numbers to[START_REF] Gilbert | Lattice-theoretic properties of frontal switching functions[END_REF].

For a general discussion of some of these results, see[START_REF] Johnson-Laird | Mental models and deductive reasoning[END_REF]. For correlations between the number of mental models under consideration and reaction times for very small numbers (one vs. two models), see for example[START_REF] Walsh | Coreference and reasoning[END_REF].

and two anonymous reviewers for detailed and extremely helpful feedback. This work was funded in part by Agence Nationale de la Recherche grants ANR-18-CE28-0008 (LANG-REASON, PI: the author) and ANR-17-EURE-0017 (FrontCog, Department of Cognitive Studies, Ecole Normale Supérieure).


Amal Alsayed
email: amal.alsayed90@gmail.com

Rébecca Deneckère
email: rebecca.deneckere@univ-paris1.fr

A sustainability matrix for smartphone application

Keywords: Sustainability, Smartphone, Application, Criteria, Matrix. 1

Nowadays, smartphones are an indispensable part of most people's lives, they are a means of enabling communication and exchange that makes everyone comes together. However, it comes with a cost as the production and use of these systems is affecting the global earth ecosystem. Considerations about sustainability are more and more in people minds and the possibility to offer sustainable smartphone applications can be an asset to any company. We propose in this work a sustainability matrix for Smartphone in order to identify their sustainability degree and offer some new thinking to their developers. We created this matrix with a state of the art study, expert interviews and a public questionnaire. We evaluate our matrix on an application and discuss the results with its development project manager.

Introduction

Technologies are progressing considerably to meet the ever-increasing needs of humans. However, we now that this quest for progress has harmful effects on the environment and therefore on humanity. One of the most threatening impacts is ecological. A large majority of scientists agree on the severity of climate change [START_REF]Integrating sustainable development and climate change in the IPCC fourth assessment report[END_REF]. Increase in the average temperature at the globe surface, decrease in soil fertility, ice melting at the poles are effects which must be limited as far as possible and as soon as possible. There is clear evidence that information technologies contribute to this environmental impact and climate change [START_REF] Orgerie | L'informatique émet plus de gaz à effet de serre que l'aviation[END_REF]. Living a smart life comes now with a smart way to handle the life cycle of technological products in the best possible way.

The strategy followed by companies to reduce these effects is called Green IT. However, even if green IT presents huge potential on the corporate level [START_REF] Brauer | The State of the Art in Smart City Research -A Literature Analysis on Green IS Solutions to Foster Environmental Sustainability[END_REF], there is little research regarding its application and the potential outside the organizations [START_REF] Vom Brocke | [END_REF]. In this field, it has been shown that the production of products traditionally represents a factor of 100 in relation to their use. We will address the issue of sustainable use of hardware through sound software management. If the environmental issue is based on making a material sustainable, then we can wonder if the software and its life cycle cause the premature renewal of terminals. [START_REF] Maurey | Rapport d'information fait au nom de la commission de l'aménagement du territoire et du développement durable par la mission d'information sur l'empreinte environnementale du numérique[END_REF] states that "Most of the figures available today show that digital power is responsible for 3.7% of the world's total greenhouse gas (GHG) emissions in 2018 and 4.2% of global primary energy consumption. Worldwide, 44% of this footprint is due to the manufacture of terminals, computer centers and networks and 56% to their use." The equipment life cycle consists of several steps that begin with prefabrication and end with the end of life of the equipment [START_REF] Byung-Chul | Life Cycle Assessment of a Personal Computer and its Effective Recycling Rate[END_REF]. Each step has its own impact on the environment. The prefabrication phase has the greatest impact on the environment, since it requires the extraction of non-renewable materials, followed by the use phase [START_REF] Byung-Chul | Life Cycle Assessment of a Personal Computer and its Effective Recycling Rate[END_REF]. The latter has an impact on the environment because of the induced energy consumption.

We believe that software can cause the premature renewal of terminals. This comes in many forms, like the disproportionate demand for machine resources (due to poor design and/or software development), the impact of software and the demands of associated hardware devices (misuse) or a contribution to programmed obsolescence (i.e. mercantile in this case).

Considering the average population of a developed country and its collective unconsciousness, people believe that software, due to its lack of physical existence, would have no direct impact on the environment. To demonstrate the opposite, it is enough to recall that the hardware stores information (hard drive, USB drive), and runs the software that operates calculations to render services (processors, memory, etc.). These elements have an impact on the environment at every stage of their life cycle. Therefore, to accept this hypothesis or reject it, we must measure the influence of software to determine its impact on equipment and therefore environmental impacts.

Software sustainability can be read at first as an oxymoron. Indeed, the turnover of technologies, hardware, etc. is such today that it may seem vain to want to make a software sustainable. On the one hand, the definition of sustainability focuses more on the environmental impact than on the objective of making software last for a long time. On the other hand, some works shows that it is possible to design a software incorrectly, to develop it incorrectly, to use it incorrectly and to leave a strong residual imprint [START_REF] Mobbs | A practical guide to sustainable IT[END_REF] [START_REF] Ardito | Green IT -Available data and guidelines for reducing energy consumption in IT systems Sustainable Computing[END_REF].

The purpose of this paper is therefore to propose a software sustainability matrix that outlines the criteria for software sustainability. Ideally, this matrix will allow us to determine whether a software is sustainable or not. There are several families of software but we restricted our work mainly on the SmartPhones software family.

Section 2 gives some information about the background. Our research methodology is explained in section 3 with an identification of the sources of our defined criteria. Our proposed sustainability matrix is then completely defined in Section 4 and evaluated in Section 5. We conclude in Section 6.

Background

One of the most common interrogation nowadays is the problem of how to preserve the environment. How can we reduce greenhouse gas pollution and increase the percentage of carbon dioxide in the air? In this context, the concept of Sustainability has emerged among many efforts. It has been applied in several areas such as real estate [START_REF] Warren-Myers | The value of sustainability in real estate: a review from a valuation perpective[END_REF], home appliances [START_REF] Kelly | Sustainability at home: Policy measures for energy-efficient applicances[END_REF], marketing [START_REF] Jones | Marketing and Sustainability[END_REF], transportation [START_REF] Dobranskyte-Niskota | Indicators to assess sustainability of transport activities[END_REF] and so on. Of course, one of those areas is information technology, which in turn tries to reduce its negative impact on the environment [START_REF] Dao | From green to sustainability: information technology and an integrated sustainability framework[END_REF].

Sustainable Management is the long-term process of optimizing environmental, economic and social performance simultaneously while taking into account natural resource restrictions, This allows business to continue without compromising the needs of future generations [START_REF] Loeser | Green IT and Green IS: Definition of Constructs and Overview of Current Practices[END_REF].

It is necessary to make a clear distinction between Information Technology and Information Systems (IS). Information Technology refers to all that is computer hardware, software or peripheral information equipment. Green Information Technology refers to measures and initiatives that aim to reduce the negative environmental impact resulting from the manufacture, operation and disposal of IT equipment and infrastructure [START_REF] Loeser | Green IT and Green IS: Definition of Constructs and Overview of Current Practices[END_REF]. Information System, on the other hand, is a broader concept that covers both technological components and human activities related to the process of managing and using technology within the organization. Information technology affects our environment in many different ways. Every stage of a computer's life, from its production to its disposal and use, poses environmental problems. Each CP used generates approximately one ton of carbon dioxide each year [START_REF] Murugesan | Harnessing Green IT: Principles and Practices[END_REF]. Green Information Systems refers to practices that aim to reduce the negative environmental impacts of IS, business operations and IS-based products and services. All this by determining how we use to invest, deploy, use and manage IS information systems [START_REF] Loeser | Green IT and Green IS: Definition of Constructs and Overview of Current Practices[END_REF].

We have to better understand the environmental impacts of information technology and how to make our IT infrastructures, products, services, operations, applications and practices environmentally friendly. [START_REF] Murugesan | Harnessing Green IT: Principles and Practices[END_REF] proposes to apply Green IT on four several ways : Green use (to reduce the energy consumption of computers and other IS), Green disposal (to refurbish, reuse or recycle old computers), Green design (to design energy efficient and environmentally sound components, computers, servers, and cooling equipment), and Green manufacturing (to manufacture electronic components with minimal or no impact on the environment).

Talking about software durability makes no sense if one does not consider the durability of the hardware on which they are instantiated. The ecological footprint depends on the durability of the equipment. There are many types of computer hardware, such as computers, smartphones, printers, video game consoles, televisions connected to a box, connected objects, etc. Each type has an ecological footprint.

There are several studies that clarify the ecological footprint concept. One of these studies is the GreenIT.fr team's Global Digital Environmental Footprint [START_REF] Bordage | Empreinte environnementale du numérique mondial[END_REF], which focuses on quantifying the global digital environmental footprint and its evolution between 2010 and 2025. This study applies to all electronic equipment that manipulates binary data. The methodology followed is based on Life Cycle Analysis (LCA), and has quantified environmental impacts based on three categories (users, networks, computer centres) that are aggregated by a meta-model. The four indicators considered in this study are: [START_REF] Bordage | Empreinte environnementale du numérique mondial[END_REF] Abiotic Resource Depletion (ADP) (impact of technology on the depletion of mineral stocks), Global Warming (GHG) (climate change), Energy balance (EP) (The so-called "primary" energy is the energy required to produce the final energy. So in digital as in any other industry, in order to manufacture the equipment and depending on the stage of its life cycle, there will be different types of primary energy to produce the final energy needed to achieve that stage), Tension on Fresh Water (Water) (After breathing air, fresh water is classified as the second most important basic physiological resource also for humans than for other life forms. It should be noted that only blue water is accessible to humans, unlike green water, which is only accessible to plants. Therefore, if digital technology increases the consumption of blue water, it will be less available for other uses at some point).

As Anne-Cécile Orgerie explains [START_REF] Orgerie | L'informatique émet plus de gaz à effet de serre que l'aviation[END_REF], ICTs and all computer equipment account for between 2 and 10% of carbon dioxide emissions according to the studies, with an agreement around 4-105%. However, aviation accounts for about 2% of carbon dioxide emissions. Studies show that both manufacturing and recycling phases are responsible for the highest proportion of carbon dioxide emissions. Hence the importance of coupling between hardware and software to develop durable software to make hardware more durable with the longest possible period of use.

Software durability alone is not enough to make hardware durable. At the same time, there are also prerequisites for the durability of the equipment. It appeared that the software, by its poor design, makes the computer hardware obsolete faster than it is expected [START_REF] Mobbs | A practical guide to sustainable IT[END_REF]. From a general perspective, the life cycle of computer hardware can be segmented into five major phases: [START_REF] Byung-Chul | Life Cycle Assessment of a Personal Computer and its Effective Recycling Rate[END_REF] Prefabrication, Manufacture, Distribution and Transport, Use, End of life. All the phases are important when talking about sustainability [17].

Some works exists on environmental sustainable mobile applications, as stated in a literature review presented in [START_REF] Brauer | Green by app : the contribution of mobile applications to environmental sustainability[END_REF]. Some applications provide feedbacks about sustainability information on an application but are restricted to specific domains, whereas others facilitates sustainable behaviors. [START_REF] Brauer | Green by app : the contribution of mobile applications to environmental sustainability[END_REF] categorized the identified applications by their goals and functionalities to provide an overview of existing usercentric Green IS solutions from IS research and practice. They assessed the potentials of mobile applications to contribute to environmental sustainability and provide a holistic perspective by performing an extensive classification of existing apps.

Applications like Ecometer1 , GTmetric2 or EcoIndex3 for instance, proposes a free analyze of a website sustainability. Quentic4 proposes a customizable sustainability software to track and manage environmental compliance online. All these services use some criteria that can be rapidly characterized in order to provide a quick feedback.

Research methodology

Every phase of terminal life cycle, from manufacturing to construction or recycling, has an impact on the environment. This is why Green IT is responsible for studying each phase separately and finding more sustainable solutions. We will restrict our work on this research on the two first phases of software lifecycles. Once the research question was identified, in order to identify the more important criteria for smartphone software sustainability, we conducted a state of art literature review, interviews with experts and launched a questionnaire. We then used the found criteria to create a sustainability matrix that we evaluated on a specific application (figure 1).

Identify research question

Our hypothesis is that the lifecycle of a software can cause the renewal of the hardware.

We stated that we have to measure the influence of the said software to determine its impact on equipment and therefore environmental impacts. Our research question is then the following: Is it possible to evaluate a software, based on predefined criteria, to identify its sustainability?. We will specifically address in this work the software family of smartphones and the two first phases of a software lifecycle.

Literature study: State of art

A lot of works proposes some criteria of sustainability for information systems, softwares or websites. Pang et al. conducted a survey presented in the article "What Do Programmers Know about Software Energy Consumption? [START_REF] Pang | What Do Programmers Know about Software Energy Consumption?[END_REF]', which revealed that programmers had limited knowledge of energy efficiency, were not aware of best practices for reducing software energy consumption and were not sure how software consumes energy. This issue of a non-functional requirement becomes very important with the growing popularity of mobile computing and the emergence of large-scale cloud deployments, and knowing that while energy consumption at the individual level was negligible, it would not be the case on a global scale because the energy consumed by all mobile devices and data centers is multiplying.

The energy behavior of a smartphone has been studied in detail by an empirical study with laboratory measurements by C. al. [START_REF] Heiser | An Analysis of Power Consumption in a Smartphone[END_REF] According to the results of this study, the biggest energy consumers are the GSM module and the display, including the LCD panel, the touch screen, the accelerator/ graphics driver and the backlight. The results of the study also indicate that audio consumes a large portion of static consumption, in the range of 28-34mW. Overall, according to the study, RAM, audio, and SD card have little effect on the electrical consumption of the device and therefore offer little potential for energy optimization.

Identify Research question

Study Literrature

Conduct Interviews

Lauch Questionnaire Create Sustainability matrix Evaluate Sustanability matrix [START_REF] Mobbs | A practical guide to sustainable IT[END_REF] proposes a practical guide to think about sustainable computing holistically, starting with the choices you make when buying technology, through to the software and peripherals you use, how you store and work with information, manage your security, save power, and maintain and dispose of your old hardware. It looks at the use of IT from the viewpoint of the information which the system manipulates.

[21] propose a label called EcoSoft that takes into account the involvement of stakeholders (project manager, software architects, developers) in the process of integrating sustainability into the project. The three stages of the software's life cycle, namely the development, usage and end of life, are analyzed to determine the environmental impacts they generate. The analysis focus on the energy consumption of software components, which is an important aspect for the overall quality of the software, especially for the user experience on the mobile device, but also because digital energy consumption has a high environmental footprint. [START_REF] Ardito | Green IT -Available data and guidelines for reducing energy consumption in IT systems Sustainable Computing[END_REF] proposes a review about Energy Consumption & IT systems, starting from the view-point of Green IT. They introduce a taxonomy of concepts, present recent data on energy consumption trends and some guidelines to write energy efficient software.

We identified several criteria in the literature, as shown in the table 1.

Qualitative study: Interviews

Seven experts working at RTE in the R&D and DSIT departments were selected to be interviewed. Questions asked were fairly open to cover as much as possible aspects related to the issue raised. The interviews, which were rich in information, made it possible to build up and strengthen the research axis. They also focused on the main points to be raised when dealing with the subject of Green-IT. During these interviews, several definitions of the Green-IT concepts were clarified, and several substantive criteria to optimize the performance of the software were determined in order to visualize their impacts on the sustainability of the latter (cf. table 1).

One criteria that popped up in the interviews as a very high concern about sustainability is the programmed obsolescence (the fact that some applications are designed to be "out of date" when the devices are getting older), also widely discussed in [START_REF] Mobbs | A practical guide to sustainable IT[END_REF]. However, this criteria is not so easy to identify and we decided not to include it in our matrix. Another criteria concerning the other phases of the lifecycle appears in the answers, like an excessive use of the battery or the performance on devices.

Quantitative Study: Questionnaire

A questionnaire was conducted to collect the criteria that make the software more sustainable or not according to the developers. Two versions were conducted, one in French and the other in English. There were two main sections. The first provides information on respondents' years of experience and programming languages. It also lets you know whether respondents are smartphone app developers or not. Because this questionnaire was addressed first to smartphone developers, and then to all developers. The second allows to know if the developers take into account the energy consumption, and to collect the criteria that allow to build the sustainability matrix. In addition, this section includes two open-ended questions for developers to suggest ways to assess the sustainability of software and technical design methods. 33 persons answered the questionnaire. The results showed that developers had different levels of experience with a higher representation of more than 10 years and 3 to 5 years of experience. In addition, the most commonly used programming languages are: Java, C++ and Python, with a lower representation of: Objective C, Swift and KotLin.

The difference in the level of experience of the developers allows to observe that the participants do not have the same knowledge of how the software consumes the resources of the machine and, consequently, the awareness of energy consumption. This is also confirmed by the results of the fourth question: Do you take into account energy consumption when developing software? If so, how do you account for energy consumption? Most of the answers to the fourth question showed a definite interest in this problem, but a difficult implementation. Some criteria were identified with the questionnaire results. For instance the lazy loading and the attachment of security updates to functional updates are of utmost importance for our experts to make a software more sustainable. Intensive use of CPU or of memory is also of critical importance for the renewal of terminals. The criteria found in the questionnaire are indicated in table 1 with the percentage of respondents who identified these criteria in their answers.

The last two questions were open-ended questions for developers to see if they were aware of ways to assess sustainability, a technical design that promotes software sustainability. These answers allowed to see that developers are not familiar with the right design and development techniques, nor the ways software damages hardware. In addition, some companies do not consider design and sustainability in their strategies.

Sustainability matrix

We defined a maturity matrix where the software score identify the sustainability degree of a software (From 1 for a bad sustainability to 5, for an excellent one). Each criteria is a characteristic of either the design phase or the development phase. They are associated to a weight identified by their appearance in the literature, the interviews and the questionnaire.

Design sustainability criteria

Design sustainability is composed of 11 criteria.

Separation between security updates and functional updates (Weight: 30)

It is known that partial updates require significantly less energy for all technologies [START_REF] Ruckebusch | Modelling the energy consumption for over-the-air software updates in LPWAN networks: SigFox, LoRa and IEEE 802.15.4g[END_REF]. It is then useful to identify which kind of updates are strictly necessary and the others. This criterion means that the software publisher must be transparent in separating security updates from functional updates by giving the customer the choice of accepting security updates and accepting or refuting functional updates. Because security updates are essential for the proper disposal of the software, unlike functional updates, which are facultative.

Lazy loading (weight: 15)

In a Web application, data retrieval of other services is sometimes slow. Thus, users may feel that the website is slow if the web page is deployed only after the data is fully processed. Nevertheless, using Lazy Loading allows the page to render and defer only a small portion until the request is ready, pushes the complete content at a later date [START_REF] Del | Analyzing best practices on Web development frameworks: The lift approach[END_REF]. This happens in order to provide a better user experience but it also decrease the energy consumption at run-time. Lazy loading is a design model that initialize an object only when it becomes necessary. In this way, Lazy Loading actually contributes to the efficiency of the operation of the software if it is used in a correct and adequate way. This makes Lazy Loading an ideal property for use where network content is accessible and the initialization time has to be reduced to minimum, as in the case of web pages. The inverse of Lazy Loading is Eager Loading [START_REF] Fain | Angular Development with TypeScript[END_REF].

Object Oriented vs Functional Programming (weight: 10) According to our experts, object-oriented programming is lighter and faster than functional programming since it uses only the objects needed to perform the expected functions. This limits the use of material resources to a minimum. As a matter of fact, in object oriented programming, the modeling step is of great importance, since the transcription of real elements in virtual form takes place during this step. In functional programming, all elements are defined as functions and the code runs through successive calls of functions. Therefore, object-oriented programming is more environment-friendly.

Cloud Synchronization (weight: 9) Terminals we use generate very significant environmental and social impacts, not only in terms of manufacturing, but also in terms of use and end of life. Companies use the cloud to store their data and enable their employers to work remotely. This therefore requires a sharp increase in hardware requirements to back up data, usually accompanied by high expectations in terms of security, which translates into an oversized physical infrastructure. Moreover, data security deployments, the illusion of infinite capability, encourage the unbridled obese computing. Studies begin to appear on the sustainability of cloud use, as [START_REF] Gupta | Examining the impact of Cloud ERP on sustainable performance: A dynamic capability view[END_REF] that shows that cloud ERP services have a positive impact on the environmental performance of an organization.

Database VS file (weight: 9)

The software structure of files is simple. However, the software structure of database is more complicated and robust. Thus, and in order to use less energy in order to preserve the environment, the software structure of the database must be simple and organized in such a way as to facilitate queries.

Local VS Network Storage (weight: 9)

This criterion identifies what type of storage is used. Network storage requires a network to store data in data centers. The operation of data centers requires constant energy and cooling, which increases the negative environmental impact. Local storage stores data directly in memory, so it uses only part of the energy needed to operate the computer device. The latter can be turned off, which reduces the negative environmental impact.

Network Call (weight: 5)

This criterion indicates the number of requests that the software has submitted. Higher the number of requests, greater the impact of the software on the environment across the hardware and network.

Compiled VS Interpreted (weight: 5)

This criterion indicates the type of programming language used: Compiled or Interpreted. Compiled language is faster and translates the code directly into machine language, while interpreted language needs an interpreter, which complicates the procedure and consumes more energy. Thus, compiled language is more favorable in support for environmental requirements.

Background work (weight: 4)

This criterion indicates whether the software contains components that work in the background, that is, whether the software works when it is not used by the user. Indeed, such software consumes energy, even if it is in an inactive case. This announces that this criterion makes the software less Green.

Automatically launch on default startup (weight: 2)

This criterion means that software dependencies work directly by default at boot time. Therefore, this software consumes energy and hardware components, although they are not used at the user's request.

Night/Day Mode (weight: 2) According to experts, night mode has proven to be less energy-intensive. Thus, its use prolongs the life of the battery, unlike the use of the day mode.

Development sustainability criteria

This phase is composed of 9 criteria.

Optimize the use of the CPU (weight: 40)

This criterion indicates the number of accesses to the CPU. The higher the number of accesses to the CPU, the greater the power consumption, and the shorter the life of the CPU. [START_REF] Sundriyal | Modeling of the CPU frequency to minimize energy consumption in parallel applications[END_REF] proposes a model for determining a frequency level that minimizes energy consumption during parallel application execution (with an energy saving of 7% for NAS benchmarks).

Optimizing Algorithms (Human Action) (weight: 14)

This criterion means that developers have improved already existing algorithms, to improve their performance by making them more sober. The aim is to reduce the use of hardware resources and energy consumption.

Optimize code instructions ( Compiler action) (weight: 10)

This criterion determines whether the compiler, by improving the algorithm without the need for developer intervention, makes the software execution procedure greener.

I/O RAM VS Hard Drive Rate (weight: 10)

This criterion indicates the number of accesses to the RAM and hard disk. After consulting the experts, it appeared that the hard disk consumes more energy than the RAM, which makes the optimization of the use of the hard disk favorable to the protection of the environment.

Optimizing the use of the Memory (weight: 5)

This criterion stresses the importance of the sobriety of the algorithms and the way the software was programmed, in order to preserve the environment. The simpler and more efficient the algorithms and the programming way, the less the software needs access to memory, and the less energy it consumes.

Binary that takes up space (weight: 5)

The design should focus on the needs to avoid making the software "obese". The latter will take up a lot of space if it embeds unnecessary code (typical example of the developer copying source code from Stack Overflow). Thus, the software will use more hardware resources with no real benefit to the user. This surplus will apparently take more binary space than the one imagined in the design phase, and it will consume more of the smartphone's hardware resources without any real benefit for the user. We can say that this way of planning a software with a surplus of features contributes to the programmed obsolescence, and causes the premature renewal of the smartphone.

Percentage of use of Open Source (weight: 4)

This criterion defines if the software is developed in Open-source or not. Its advantage lies in the ability of users to use the software without being dependent on the editor and its updates. These updates often render old computer devices obsolete, which requires the purchase of new ones. The ability of users to use, improve, and modify open-source software allows them to extend the life of their computer devices, thereby protecting the environment.

Bugs (weight: 4)

This criterion means that the software has a lot of bugs, and if it requires regular maintenance. In this case, the software must be updated regularly. This makes the software more obese, therefore more obsolete.

The poor legibility of the code to better understand it (Evolution correction) (weight: 4)

This criterion means that the software is well developed, and that the code is well written and clear. This facilitates development by developers and execution by the compiler.

Evaluation

In order to validate the matrix, a practical case concerning the "Éco2mix" mobile software, developed by the company RTE, was studied. This application is accessible to everyone, not just RTE customers or its agents. It is integrated into another RTE site, called RTE France. Éco2mix is operable on Android and IOS. This application is dedicated to exposing RTE data on energy uses and production (nuclear, solar, hydraulic, photovoltaic, etc.) both throughout France and at the level of administrative regions and in certain metropolitan areas. The application also gives the average energy consumption of a house in France and the possibility to compare it with those of individuals. Its objective is to better manage the energy balance. Trade in all electrical parameters at the level of the French regions and between France and its neighboring countries has also been included in this application.

Matrix construction

We interviewed the project manager of this application development and filled the sustainability matrix with him (cf. figure 2). Next step is to calculate the score of the application based on the matrix results. Calculus of category score is made on the following way. (a) each level is multiply by its weight. For instance, first criteria obtain the result of (1*30)=30). (b) Then the s um of all results is done for the category and Here, for the design category, the sum of all weighted criteria is 279, then we calculate its rate over 500 (the maximum score) which give 55,8%, an average score. And the development category gives a rate of 76,875%, which is quite good.

Results evaluation

We discussed these results with the project manager who recognized the results as true for this application. From a sustainability point of view, Éco2mix is a good software both in the design aspect and in the development aspect. Moreover, this interview allowed the project manager to identify some improvement for the future to improve the software sustainability, as follows.

• It would be good for Éco2mix to make the separation between security updates and functional updates and to let the client free to choose the type of update to install. • Developers have every interest in reducing the number of requests with the network, in order to optimize the environmental performance of Éco2mix. • The energy performance of Éco2mix is affected by the lack of night/day mode, so it seemed useful to treat this aspect in order to improve the performance. • It appeared that open-source development is more suitable for the sustainability of software, by facilitating the maintenance, modification and improvement of the code at the level of security as well as functional. • Developers will be encouraged to optimize the Eco2mix code in order to have fewer bugs, which will increase its environmental performance. • The code is written in a rather complicated way, which reinforces the interest of making it easier to read, modify and improve. The project manager found the test useful, as it allowed him to identify the elements that make up sustainability of software, elements that seem relevant to him and that he will use in his next projects.

Conclusion

We developped a matrix for project managers to test the sustainability of their software. This matrix was based on the results of interviews with experts, as well as a questionnaire published on the web on professional networks. This questionnaire contained two types of criteria: positive and negative making the software more or less durable. We evaluated the sustainability matrix on a smartphone application and discussed the results with the application project manager. The criteria of the matrix were assembled theoretically on the basis of the results of the interviews and questionnaire, as well as the literature. However, it is preferable for future work to benefit from measurement equipment in order to truly assess the relevance of these criteria and their impact on software sustainability. Each criterion should be tested separately. As mentioned earlier, at this stage there are only sustainability testing tools dedicated to websites. It will therefore be necessary to build sustainability testing tools for software implemented on all computer devices, such as smartphones and computers.

The matrix forms a good basis for evaluating the durability of software. It has been divided into four categories: Design, Development, Use and Integration. The work presented here covered the first two categories, while the next two categories will be the subject of further interesting work.

Fig. 1 .

 1 Fig. 1. Research medthodology

Fig. 2 .

 2 Fig. 2. Eco2mix sustainability matrix

Table 1 .

 1 Identified criteria

	Criteria	SoA	Interviews Quest.
	Separation between security updates and functional			52,2%
	updates
	Lazy loading		X	52,2%
	Object Oriented vs Functional Programming		X
	Cloud Synchronization	[8] [22] [23]	X
	Database VS file	[7]	X
	Local VS Network Storage	[7]	X
	Network Call	[7]	X
	Compiled VS Interpreted	[19]	X
	Background work	[19]		47,8%
	Automatically launch on default startup			39,1%
	Night/Day Mode		X	43,5%
	Optimize the use of the CPU	[19]		39,1%
	Optimizing Algorithms (Human Action)	[21]	X
	Optimize code instructions (Compiler action)	[21]	X
	I/O RAM VS Hard Drive Rate	[8] [24]	X
	Optimizing the use of the Memory	[8]		21,7%
	Binary that takes up space		X	4,3%
	Percentage of use of Open Source	[25] [7]		13%
	Bugs			20%
	The poor legibility of the code to better understand it		X
	(Evolution correction)

http://www.ecometer.org/

https://gtmetrix.com/

http://www.ecoindex.fr/

https://www.quentic.com/sustainability-software/