Shokhista Ergasheva
email: s.ergasheva@innopolis.ru

Dragos Strugar

Artem Kruglov

Giancarlo Succi

Energy efficient software development process evaluation for MacOS devices

Keywords: Energy efficient software, MacOS software development, Software development process metrics

InnoMetrics is the system that aims at collecting software development process metrics in a non-invasive way to access and optimize the development process and its efficiency. This paper demonstrates the development and analysis of energy consumption of MacOS systems based on the software process measurement data. It represents the experience of the development of MacOS system collector and Transfer, in addition to the user interface and early analysis of energy consumption metrics calculations.

Introduction

The importance of energy efficiency of any system and devices becomes sufficient property of the present Information Technology life. Thus, measuring and what is more further analysis of energy efficiency and its prediction can be managed by monitoring the software development process. Before going deep into the software development process metrics let's define what is the measurement itself. The history of measurement can be divided into two generations. The first generation is the Personal Software Process(PSP) -self improvement process that helps developers to control, manage and improve the way they work. This method can be called as invasive as it requires the direct involvement of participants in the data collection process. Users of the-invasive method should create and print forms in which they log their effort, size and defect information. One obvious downside of this invasive approach is the high overhead cost it entails. The developers should often switch between development tasks and metrics collection tasks, which imposes a high cognitive burden to the developers while the second generation of measurements, non-invasive measurements, do not require manual intervention of the participants during metrics collection [START_REF] Bykov | A new architecture and implementation strategy for non-invasive software measurement systems[END_REF]. The main aim of the successful use of a non-invasive measurement system is to support the developers, the development process optimization and maintain the data privacy. Innometrics is one of the promising software development process data collection systems with automatic data collection and transfers to the server for further data analysis. It allows the developers and managers to be aware of the process strength and weaknesses based on the data from the developers performed. The crucial insights are visualized concerning the metrics and analysis. The benefits of such system can be counted as real-time process analysis on a daily basis, the system can be accustomed to different levels of company sizes and audits of the software development process and development itself can be monitored at the same time.

Metrics

The first thing in the way to data collection framework was to perform a Systematic Literature Review (SLR) of existing research papers, energy-related metrics and best practices to collect data during the software development process. The Systematic Literature Review surveyed more than 500 studies in the field of software metrics collection and analysis. As a result, at about 170 metrics were derived, and divided into 3 categories as code, product and process metrics. Based on the findings from SLR, we concluded that the mostly unexplored branch of software development metrics is process metrics [START_REF] Ergasheva | Metrics of energy consumption in software systems: a systematic literature review[END_REF]. Besides, all the tools observed during the research project consider static analysis of the code, nevertheless, the process of developing the software still has no much analysis. The main reason for this insufficient analysis of process measurements is the absence of the tool. Furthermore, the process cost increases, since usually, it requires the developers' participation.

The system of InnoMetrics is developed based on the monitoring of the software development process energy consumption and its efficiency, the developer's productivity. All direct and model-based measurements that involve usage of third-party hardware tools to get energy metrics from various components were considered out of scope of non-invasive software development process analysis approach. The energy consumption of software applications was thoroughly researched and concluded based on the research findings. Battery draining applications result in lower user experience and dissatisfied users [START_REF] Rahmati | Understanding human-battery interaction on mobile phones[END_REF]. Optimal battery usage (energy usage) is an important aspect that every client must consider.

Application energy consumption is dependent on a wide variety of system resources and conditions. Energy consumption depends on, but is not limited to, the processor, the device uses, memory architecture, the storage technologies used, the display technology used, the size of the display, the network interface that you are connected to, active sensors, and various conditions like the signal strength used for data transfer, user settings like screen brightness levels, and many more user and system settings [START_REF] Rahmati | Understanding human-battery interaction on mobile phones[END_REF].

For precise energy consumption measurements, one needs specialized hardware [START_REF] Saborido | On the impact of sampling frequency on software energy measurements[END_REF]. While they provide the best method to accurately measure energy consumption on a particular device, such a methodology is not scalable in practice, especially if such measurements have to be made on multiple devices. Even then, the measurements by themselves will not provide much insight into how the application contributes to the battery drainage, making it hard to focus on any application optimization efforts.

The InnoMetrics system aims at enabling users to estimate their application's energy consumption without the need for specialized hardware. Such estimation is made possible using a software power model that has been trained on a reference device representative of the low powered devices applications might run on [START_REF] Musílek | On the Sensitivity of CO-COMO II Software Cost Estimation Model[END_REF][START_REF] Scotto | A Relational Approach to Software Metrics[END_REF][START_REF] Ronchetti | Early estimation of software size in object-oriented environments a case study in a cmm level 3 software firm[END_REF][START_REF] Rossi | Modelling Failures Occurrences of Open Source Software with Reliability Growth[END_REF][START_REF] Pedrycz | Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity[END_REF]. Based on the findings of the research, metrics like following were investigated [START_REF] Jagroep | Energy efficiency on the product roadmap: An empirical study across releases of a software product[END_REF]:

-Software Energy Consumption (SEC) -the total energy consumed by the software;

-Unit Energy Consumption (UEC) -the energy consumed by a specific unit of the software;

Considering our profiling method and the tools available for us, the ability to attribute the energy consumption was possible only at the process level in coarse granularity. However, the hardware resource usage can fill the gap when it comes to accurately relating Energy Consumption (EC) to individual software elements hence enabling the computation of the UEC.

Profiling the performance requires a basic understanding of hardware components that has to be monitored through "performance counters", which is possible in Windows System. While interpreting performance data for further analysis, the context information has to be taken into account (e.g. hardwarespecific details).

To evaluate the Unit Energy Consumption (UEC) the following hardware resources should be monitored:

-Hard disk: disk bytes/sec, disk read bytes/sec, disk write bytes/sec; -Processor: percentage of processor usage; -Memory: private bytes, working set, private working set; -Network: bytes total/sec, bytes sent/sec, bytes received/sec; -IO: IO data (bytes/sec), IO read (bytes/sec), IO write (bytes/sec).

Attributing some weights to elements of the UEC or by some reliable assumption such as considering the power model to be linear in the nature for each individual component, the SEC Metric is computed.

Besides, the energy usage can also be appraised using Performance Counter, Performance Counter Category and related classes that are available with .NET Framework. To be specific, by analyzing the MSDN documentation to attain the goal of collecting energy related metrics, it was concluded that the information about CPU time, Total Processor Time per process, CPU usage, Memory usage, network usage that Performance Counter provides can be reliable.

The bottleneck in this situation is that it is difficult to match up constantly changing application process IDs and names. The energy consumption of the system depends on a variety of factors that are not limited to those which can be collected using the above mentioned performance classes.

MacOS energy metrics

In order to obtain energy-related data in MacOS devices, we explored some of the APIs that MacOS provides. The first and most obvious tool was Activity Monitor, an application that comes built-in every MacOS system. One of its aspects is the Energy consumption, shown in Fig. 1.

Fig. 1. MacOS energy metrics

The second column, Energy Impact, attracted our attention immediately. We then wanted to figure out what these values (4.0, 0.6, 1.6, etc) represent. It has been then brought to our attention that the definition of Energy Impact is not precisely defined by Apple. According to Activity Monitor's documentation, the definition of Energy Impact says "Energy Impact: A relative measure of the current energy consumption of the app. Lower numbers are better" [START_REF]How to use activity monitor on your mac[END_REF].

As other documentation says "The Energy tab of Activity Monitor displays the Energy Impact of each open app based on a number of factors including CPU usage, network traffic, disk activity and more. The higher the number, the more impact an app has on battery power" [START_REF]About mac notebook batteries[END_REF]. Both of these are vague, and we needed a concrete way of obtaining this metric's values.

One article on the Mozilla blog attempted to figure out a formula for this. They indicated that the result of MacOS's top command line tool which performs periodic measurements of all kinds of metrics; including ones relevant to energy consumption: CPU usage, wakeups and the power measure. The article we mentioned above suggests running the following command to get the abovementioned information:

Yielding the results (trimmed): top -stats pid,command,cpu,idlew,power -o power -d They go on to suggest that POWER measure is calculated using a simple formula, and a specific configuration file (tuned for every machine's architecture). Using the findings from that article, we decided to use some of the most impacting metrics:

-Battery percentage -Battery status (is charging or not) -RAM measurement (how much RAM does the active process use) -vRAM measurement (how much vRAM does the active process occupy) -% CPU utilized (per process) All of these metrics were obtained using the macOS command line interface. E.g. to get the current battery percentage we used pmset -g batt, and for other measurements we used the ps -axm -o command with varying parameters (depending on the use case). It was also possible to use the top command, but as we are performing periodic checks anyway, top was not necessary. Incorporating these metrics and collection process to the InnometricsCollector was successful and results are available on GitHub: https://github.com/InnopolisUniversity/innometrics-agent-os-mac/.

Collector development

InnoMetrics system contains of the following components [START_REF] Maurer | Software Process Support over the Internet[END_REF][START_REF] Vernazza | Defining Metrics for Software Components[END_REF][START_REF] Sillitti | Measures for mobile users: an architecture[END_REF]:

-Data Collectors -the separate for widespread Operating Systems (Windows, MacOS, Linux) data collecting frameworks; -Server -which includes analytic module of obtained data referring quantitative and qualitative analysis; -Dashboard -the component that responsible for visual representation of the analyzed development process data.

The scope of the paper do not consider describing the data collectors for all operating systems, nevertheless MacOS collector will be described thoroughly. MacOS collector includes two separate applications using a service oriented approach [START_REF] Sillitti | Service Oriented Programming: A New Paradigm of Software Reuse[END_REF]:

-InnoMetrics Collector -InnoMetrics Transfer
InnoMetrics Collector is a daemon process, it runs in the background. The only interaction the user has with the application is that the collector sits in the background and collects data. It displays the user information (Figure 2):

-User Name; -System version and specifications; -IP address; -Mac address of the device; as well as the currently open application. The Idle Time tab (Figure 3) shows the total idle time and the top 3 application that have been idle for the longest. In addition, the user can stop the process of collection, as well as they can quit the app.

Metrics being Collected

The data being collected on a per-process basis consists the following types:

-Metric type (app focus/idle); -Application Name; -Bundle Domain of the app (e.g.com.apple.finder); -Bundle Path (where the app is installed, e.g. Applications/Yandex) -Timestamp of the opening of the app/process; -Timestamp of the quitting of the app/process; -Duration of the process open (timestamp end-timestamp start); After collecting the metrics, the system saves the current session with detailed information (user name, login name, IP address, mac address, etc) only if it has not been changed. Then it saves the data to a local database, allowing the other app (MacOS InnoMetrics Transfer App) to send the data to the server.

Regarding the Innometrics Transfer app, shown in the Figure 4, it is a separate application. However, it is not a daemon, it is a GUI application which allows users to see all the metrics that have been collected in addition to the functionality of sending such data to the back end. It consists of two main areas: the left bar and the process metric list on the right. We have the feature to filter the processes based on keywords, applications, by date. Also, the user can select multiple metrics and hit the Backspace key to delete the metrics before sending them to the server (privacy feature). The right panel presents the data obtained from the collector (connects to a local database where collector was sending data). Finally, user can send the metrics to the server. After selecting the metrics user wants to send to the server, and hitting the "Send Metrics" button, user gets presented with a pop up (Figure 5). Merging InnoMetrics Collector and InnoMetrics Transfer As mentioned above, there is a dependency between these two applications. One first needs to collect data using the collector and then transfer the data using transfer. The UX would be better if everything was done in the same application, not requiring users to use both of the apps. That is why it was decided to merge the two applications. As a result of this decision, no longer do the users have to click on the submit button to send the data to the server. The data is automatically and periodically being sent as they are using the computer, without any distractions.

Auto Update functionality Desktop applications in general (deployed outside of the Apple App Store) suffer from the inability to update frequently [START_REF]Mozilla blog: What does the osx energy impact actually measure?[END_REF]. That is why it was decided to implement the auto update functionality. For this we used the open source framework for auto update of MacOS applications called Sparkle (https://github.com/sparkle-project/Sparkle) [START_REF] Mathur | A human-centered approach to improving the user experience of software updates[END_REF]. It's a widely known and used framework that allowed to easily update our apps. As the two applications have been merged, there was no need to add the auto update functionality to the Innometrics Transfer. After implementing this functionality, the process of quickly iterating on versions became much quicker and it allows to become more agile when responding to user feedback [START_REF] Pedrycz | A model of job satisfaction for collaborative development processes[END_REF][START_REF] Sillitti | Understanding the Impact of Pair Programming on Developers Attention: A Case Study on a Large Industrial Experimentation[END_REF][START_REF] Coman | Cooperation, collaboration and pair-programming: Field studies on backup behavior[END_REF][START_REF] Janes | Lean Software Development in Action[END_REF][START_REF] Corral | Software Assurance Practices for Mobile Applications[END_REF]. Let us stress that the use of an open source application appears particularly suited as it enhances the reliability and the adoption of the system, as evidenced in several existing works [START_REF] Kovács | Open Source Software for the Public Administration[END_REF][START_REF] Fitzgerald | Adopting open source software: A practical guide[END_REF][START_REF] Di Bella | A multivariate classification of open source developers[END_REF]

Conclusion

To sum up, the system of InnoMetrics was developed for MacOS have been described thoroughly throughout the paper. The energy-related metrics are easily derived from the device with the help of Activity Monitor service in the MacOS. All kinds of metrics were derived and analysed based on the research studies. MacOS framework consists of two basic components: MacOS Collector and Transfer. The details of collecting data and its types, properties and their transfer is integrated in the system. Furthermore, the User Interface developed as a separate system -InnoMetrics Dashboard-for visual representation of the collected data and analysis, was presented. In this report, we have presented: the overall information how to get energy metrics in terms of MacOS systems, a deeper view of how the data collector and transfer operate with the back-end, displayed some results of development phase.

Further improvements of the system will be concentrated on the testing of the system with the functioning industrial companies, adding the external agents for deeper analysis of the software development process like Trello, GitLab, Github, and others. Besides, future work will include exploration of data analysis patterns and best practices for data analysis referring to the data records collected from the Data Collector.

Fig. 2 .

 2 Fig. 2. MacOS Data Collector: Current Process

Fig. 3 .

 3 Fig. 3. MacOS Data Collector: Idle Process

Fig. 4 .

 4 Fig. 4. InnoMetrics Transfer

Fig. 5 .

 5 Fig. 5. Popup for sending data to the server

Table 1

 1

		. top results
	PID COMMAND	% CPU IDLEW POWER
	50300 firefox	12.9	278	26.6
	76256 Plugin-container 3.4	159	11.3
	151 Coreaudiod	0.9	68	4.3
	76505 Top	1.5	1	1.6
	76354 Activity Monitor 1.0	0	1.0

This research project is carried out under the support of the Russian Science Foundation Grant №19-19-00623.


Nursultan Askarbekuly

Andrey Sadovykh

Manuel Mazzara

Combining Two Modelling Approaches: GQM and KAOS in an Open Source Project

Keywords: goal question metric, GQM, KAOS, goal model, software metrics, requirements engineering, open source software

GQM is a software metrics method that produces meaningful and appropriate measurement metrics based on the specific goals of an organisation. KAOS goal modelling is a software engineering approach that allows to identify high level goals of an organisation, refine and expand them into lower-level more concrete goals and assign them to specific system agents. The two approaches can be combined, such that the goal modelling is used to derive specific goals rooted in the organisational context, policies and strategies of an organisation. The GQM, in turn, utilises the derived specific goals to produce appropriate and meaningful metrics that can be used to obtain valuable insights and track the state and progress of the formulated goals. Moreover, both models can be presented within a single diagram, thus providing an expressive and concise overview of the systems goals, actors and measurements. The suggested combination is then applied in the context of an open source project, followed by the discussion on potential benefits and drawbacks of using the two approaches..

Introduction

Goal, Question, Metric (GQM) is a measurement modelling approach applicable in a wide variety of contexts ranging from assessing software reliability and validation of engineering techniques to evaluating the impact of business strategies [START_REF] Ivanov | Comparing the reliability of software systems: a case study on mobile operating systems[END_REF], [START_REF] Ivanov | Design and Validation of Precooked Developer Dashboards[END_REF], [START_REF] Basili | Determining the impact of business strategies using principles from goaloriented measurement[END_REF].

It allows engineers to choose appropriate software measurement metrics. The GQM method takes a set of specific goals of an organisation as a basis, and connects them with metrics in order to measure their achievement and progress. Thus having a set of specific goals of an organisation is a prerequisite to applying the GQM model and extracting the metrics.

To produce the specific goals, understanding the higher-level strategic goals and the organisation environment is necessary and requires a substantial amount of effort, such as interviewing the stakeholders and working in close collaboration with the organisation's team. However, the GQM in its essence does not reflect the context and high-level goals from which it was derived, and therefore requires other more general modelling approaches to reflect the bigger picture.

KAOS is a goal modelling approach used in requirements engineering [START_REF] Van Lamsweerde | Requirements engineering[END_REF], [START_REF] Basili | The Goal Question Metric Approach[END_REF]. It identifies high level system goals and proceeds top down to refine them, thus producing more concrete lower-level goals. The lower-level goals are then assigned to agents, such as software modules, external systems or individuals, responsible for the satisfaction of these goals. The diagrammatic notation used in the approach allows one to express and analyse various complex relations between the system goals and entities, and allows one to capture a system's functional and nonfunctional aspects and constraints.

The correspondence and complementary nature of the two approaches makes combination and interchange of their techniques and tools possible. In particular, the KAOS goal model's comprehensive diagrammatic notation can be used to reflect and facilitate the derivation process of the specific set of goals from the organisational context and higher level strategic goals. The GQM, in turn, can use the derived set of the specific goals to produce the set of appropriate metrics to measure and analyse the state and achievement of those goals. As a result, the two approaches produce a mapping of high-level strategic goals to more concrete lower-level goals, which are then assigned to agents, responsible for their fulfilment, and to appropriate metrics, that allows tracking, analysis and further interpretation.

In the following two sections a brief overview of the two modelling approaches is presented. Then in the fourth section an example of combining the two approaches is examined, and its main advantages are outlined. The fifth section is a case study on applying the combination of two approaches in the context of an open source software project. The authors demonstrate how the combination can be used for an actual project, and then list possible benefits and drawbacks to using the approach for the purposes of documenting requirments for an open source software (OSS) project.

Goal, Question, Metric (GQM) method

One way for an organisation to have some valuable insights regarding itself, its assets and processes is to conduct some appropriate measurements and interpret them correctly. The question of what comprises appropriate measurements is critical. The GQM method helps one to arrive at those measurements, provided there is a set of specific goals for that organisation [START_REF] Basili | The Goal Question Metric Approach[END_REF]. To get from each specific goal to a measurement the method defines three levels (Fig. 1), each representing a step on the way from the goal to a metric:

-Goal : Conceptual level, where each of the specific goals is represented.

-Question : Operational level, at which questions are posed against the goal in order to assess its achievement and identify some of its aspects through which the progress can be observed and measured. -M etric : Quantitative level, at which appropriate metrics is selected to answer the questions posed at the operational level in a quantitative manner.

Fig. 1: Levels of the GQM model

Following is an example of the GQM process using a simple example of a web application of a company selling airplane tickets. The process starts with a specific goal. It has to be formulated in a particular way and be associated with an object, which in the context of software development could be an artefact, process or resource. Within the goal a purpose is defined, together with a particular quality issue of the object and a viewpoint from which it is looked at. The example goal is to improve usability of the ticket booking feature from the point of view of a person booking a ticket (end user). The object of the goal is the ticket booking feature. The purpose is to improve, the quality issue is the usability and the viewpoint is that of a person booking a ticket.

Once the specification of the goal is ready, questions can be posed against the goal and metrics can be proposed to answer the questions. Looking at the example goal above, its purpose and quality issue one can deduce following questions:

-What are the completion rates of the feature?, i.e. what portion of users actually accomplishes booking a ticket. -What is the average user task time?, i.e. how much time does it normally take a user to accomplish booking a ticket. -How satisfied are the end users with the feature's usability? Each of the questions can have one or several metrics answering it. The whole GQM can be presented in a tabular format [START_REF] Basili | The Goal Question Metric Approach[END_REF], as in the Table 1. Often, there is a set of specific goals and a GQM is produced for each one of them. Evidently, In fact, the team constructing GQMs needs to accomplish a thorough work of understanding the context of the organisation itself, its mission, strategy, policy and high level goals. The team also needs to understand the local context of the object being measured, which can be a product or some process within a project. Finally, viewpoints of various stakeholders need to be taken into consideration to serve as another important element formulating the goals. The whole process involves interviewing stakeholders, analysing the environment and collaboration between the corporate management, project and GQM team. The derived goals need to be analysed with regards to their relevancy and relation to the high level goals. [START_REF] Basili | The Goal Question Metric Approach[END_REF] Evidently, deriving the specific goals is a lengthy and nontrivial process. Therefore presenting only the specific goals and their corresponding GQMs is insufficient to understand the greater context and the relationship between specific goals, the environment and strategic high level goals of the organisation. So the question arises, what techniques and approaches can be used to simplify the process of deriving the set of specific goals? Furthermore, can the context and its relationships with the specific goals be represented in a concise and comprehensive manner?

Yet one more important aspect related to the metrics is related to the actors involved. Does the measurement process involve personnel, devices or software? Who or what is responsible for accomplishing the goal and gathering the measurement data?

The GQM's tabular format above is in fact only a concise output of the work preceding it, while the context and the actors involved with the system can be conveyed through text or by using other modelling approaches [START_REF] Basili | The Goal Question Metric Approach[END_REF]. The KAOS goal modelling approach provides tools and techniques necessary to reflect the strategic context and actors involved. In the following section, KAOS and its tools are introduced, and the benefits of combining it with GQM is elaborated upon based on an example.

Goal modelling: Keep All Objectives Satisfied (KAOS)

Goal modelling is an approach widely used in requirements engineering, and it allows to capture requirement by analysing a software system's high-level goals and contextual environment. The specific approach being introduced here is KAOS. The abbreviation stands for Knowledge Acquisition in automated specification or Keep All Objectives Satisfied, but within this paper it will be interchangeably referred to it as KAOS and the goal model. The goal model uses expressive and concise diagrammatic notation to convey the flow and refinement of a system's high level strategic goals into lower-level technical goals. Moreover, it can conve various complex relations between the goals at different levels [START_REF] Van Lamsweerde | Requirements engineering[END_REF]. Figure 2 demonstrates a simple example of applying goal model in the context of building a social network for professionals. This fairly simple diagram contains several aspects that worth noting:

1. The top level element are the high level strategic goals of the system being built. There are two high-level goals: to generate revenue, and to connect companies with professionals. The strategic goals are then broken down into finer-grained goals in a top down fashion, such that the goals on the third level in the diagram are actually describing the technical design objectives of the system. 2. Relationships between goals at different levels can be identified and denoted.

The first type is the relationship between a parent goal and its direct children. Does the system need to achieve all of the child goals in order to satisfy a parent goal, or the child goals are actually alternatives, and fulfilling only one of them is sufficient? The KAOS notation uses something called or-and and-refinement to expressing that, and the refinements are denoted by the blank and solid circles serving as connection between the goals (Fig. 3).

The solid black circle denotes that all child goals have to be fulfilled, and is call complete and-refinement, as in AttractP rof essionals goal and its children [START_REF] Van Lamsweerde | Requirements engineering[END_REF]. The case in-between, not present in the example diagram, is referred to as incomplete and-refinement, which has the meaning that only some and not all the child goals need to be achieved in order to satisfy the parent goal. It is denoted by a single blank circle pointing an arrow at the parent goal, with several lines coming out of it and connecting the circle with the child goals. 3. Having system goals in one place, allows one to analyse the goals and to identify and denote the conflict between the goals on different levels. A lightning bolt shape connection is used for that, as can be observed in the case of two apparently conflicting goals from the example: ChargeJobSeekers and M aintain[F reeF orJobSeekers]. Together with identifying conflict, the goal model approach suggest various mechanisms of resolving the conflicts [START_REF] Van Lamsweerde | Requirements engineering[END_REF]. 4. As can observed, the goals on the third level have the prefix Maintain before them. This comes from the notion of behavioural and soft goals used in KAOS. Behavioural goals are preceded by prefixes such as Achieve, Maintain and Avoid, and denote some logical constraints on the behaviour of the system. Achieve means that the goal has to be achieved at some point, and often comes with a conditional close dictating when it should be achieved. Maintain defines that some desirable state needs to be up-kept in the system at all times, while avoid means the opposite, i.e. that some state must never occur. Soft goals, in turn, denote other less stringent conditions on the system, and use prefixes such as maximise, minimise. This technique of defining various types of goals provides an expressive and fairly flexible tool of conveying constraints on the system and its environment through the goal statements.

For some goals, a different refinement strategy can be used. Instead of defining subgoals, an obstacle preventing the goal from being achieved can be defined. Figure 4 represents an example of such a case.

Fig. 4: Goal Model obstacles and agents

There are several important aspects in this example too:

1. A goal common to most software systems dealing with personal user data is keeping the user data confidential. An obstacle obstructing that goal, i.e. preventing its satisfaction, would be exposure of some sensitive user data.

The obstacle is further refined into two alternative sub-obstacles, and then countermeasures for both of them are found. According to the notation the crossed arrows denote obstruction and resolution relationships. An attentive eye will also notice that obstacles are represented inside a horizontally flipped parallelogram, which signals that obstacles are opposite of goals. 2. There is a table on the right of the SensitiveDataExposed obstacle, providing additional details such as the definition, category, likelihood, criticality and even a formal specification expressed using predicate logic. Such tables can be provided for any goals or obstacles, that need an extra level of detail or clarification. 3. Yet, the aspect of biggest interest is the fact that the lowest level goals are attached to hexagons, which represent agents. Agents can be software modules, individuals or devices. Individuals are normally denoted with a human figure, while non-human agents are denoted with titles only. This is one of the key aspects of the goal model, since it allows to attach each goal on the lowest level to an agent, thus connecting the strategic goals with the system or environmental elements responsible for satisfying them. Normally, each low-level goal will have one agent responsible for fulfilling it.

In cases when a goal has several agents, the possibility of further refining the goal should be considered. On the other hand, if an agent is responsible for too many goals, the refinement and better definition of the agent should be considered too.

It is worth noting that the examples in Fig. 2 and Fig. 4 had both business and human related goals (e.g. GenerateRevenue and EducateU sersOnSecurity), which were included on purpose. The purpose of including such goals is to demonstrate that the goal model approach can be used in various context, not limited to technical requirements specification. In fact goal modelling can be used to reconcile and align technical project decision with the business goals [START_REF] Ellis-Braithwaite | Towards an Approach for Analysing the Strategic Alignment of Software Requirements using Quantified Goal Graphs[END_REF]. The goal model diagram in its essence is a mind map, which allows one to visualise the goals and the context, analyse how the elements relate and synchronise between each other, expand the solution, spot conflicts and obstacles.

To summarise, goal modelling allows one to define strategic goals, expand and refine them into lower-level finer-grained goals, and express their breakdown structure and flow. The concise and expressive notation tools such as or-/andrefinements, conflict and obstacles identification allow one to express various relations between goals and subgoals, and identify possible barriers on the way of achieving them. The notion of agents allows to explore and define the actors involved with the system, and to allocate and assign their responsibilities. Altogether, it comprises a powerful tool allowing one to convey the strategic and environmental context, technical and business constraints and actors involved.

Combining GQM and KAOS

In this section, an example of combining the two approaches is presented based on the case of the ticket booking company's web application from the second section.

Before building a GQM, the company's high level strategic goals need to be analysed and understood. Increasing revenue is a high level goal shared by many companies. The techniques and tools introduced in the previous section will be used to derive lower level specific goals from the strategic goal, and reflect the thought and derivation process (Fig. 5).

As was mentioned in the previous section, the goal model diagram is normally constructed in a top down manner. Despite the fact that the diagram in the example (Figure 5) is of relatively small size, there is a number of techniques used to describe various relationship between the goals: The diagram in the Fig. 5 describes the process of arriving at the specific goal of maximising the booking feature's usability, the GQM of which has been constructed in the second section of this paper, and has another specific goal next to it, for which a GQM can be constructed in a similar manner. The most important thing to note, is that the goal model diagram provides a concise and comprehensive way of visually conveying the context and derivation process of the low-level specific goals used for constructing GQMs.

Interestingly, GQMs can also be constructed for higher level goals, in particular for M aximise[CustomerRetention], IncreaseN umberOf Bookings and Achieve[CustomerAcquisition]. For example, the customer retention goal calls for a question of "Do the users come back to make repeat bookings through the service?", for which a simple metric can be provided as an answer, namely "Percentage of returning users making repeat bookings". The fact that the GQM can be used to measure the achievement and state of goals at various levels, implies that the the GQM metrics can enhance the goal model approach, especially when it comes to soft goals. The very formulation of soft goals using verbs such Fig. 6: Integrating GQM into the Goal as Maximise, Minimise and Optimise calls for metrics to be used in order to track the state of the system, and declare concrete satisfaction criteria for these goals.

Another important point is that GQM can be integrated into the diagrammatic notation of the Goal model. Fig. 6 exemplifies the case of the GQM constructed in the section 2.

The notation introduced is a rounded-rectangle for questions and a trapezoid for metrics. This way the GQM can be integrated into the diagrammatic format of the goal model. The advantage of it is two-fold. Firstly, the exact placement of the GQM within the greater context can be observed. Secondly, and more importantly, the agents responsible for gathering particular metrics can be identified and denoted, as in the case of ascribing the user satisfaction metrics to the Marketing Office, and the rest of the metrics to the Web Analytics module.

Applying the technique to an Open Source project

The combination of the two modelling approaches has been tried and tested by the authors on an open source software project called KinApp [START_REF] Kinapp | KinApp Mobile Open Source Application[END_REF]. KinApp is a mobile application that helps users to keep in touch with their family and friends while being away. During user research stage preceding requirements elicitation, the authors were able to establish that there is a segment of potential users who would like to keep in touch with their relatives, but for various reasons are not able to do so. The most common reason named was forgetfulness due to being busy and having other priorities to take care of. Based on these findings, the authors developed and implemented a concept for the KinApp mobile application. It is a simple reminder app where users can add people they want to keep in touch with, and configure the desired frequency of contacting for each of them. The application will send reminders, and keep track of streaks (unbroken sequence of successfully contacting the family member or friend on a regular basis).

KAOS approach is used to produce specific goals, corresponding to functional requirements within the app. GQM in turn provided a way of measuring the effectiveness of the tool in achieving the high and intermediate level goals by providing measurement metrics. Fig. 7 demonstrates the process of applying KAOS and GQM to the high level goal of M aximize[KeepingInT ouch], right in the middle of the graph. As has been established during user research, the most common preventer is forgetfulness due to having other responsibilities. This can be represented as the F orgetf ulnessBeingBusy obstacle within the goal model. The two countermeasures are 1) having a prioritized list of people to contact and 2) to remind users to keep in touch. We introduce them as the two intermediate goals. The prioritized list represents the main functionality of the app. It will require several essential features:

-Adding people to list -Configuring regularity of contact, i.e. how often does the user want to contact each person -Displaying the list of added people This translates into two specific goals with the highest implementation priority: AddContactW ithRegularity and DisplayContacts. Another intermediate goal RemindT oKeepInT ouch translates into a specific functional requirement of sending push notifications to the user as reminders that sufficient time has passed since the last contact with a particular person. GQM is then applied to make the above goals measurable. Table 2 demonstrates the process of arriving to metrics for M aximize[KeepingInT ouch].

Other metrics are similar to the ones demonstrated in the previous sections, and for the sake of keeping the example concise will not be delved into. Noteworthy, the introduced streak metrics necessitate a new functional requirement of having to track streaks for users. The corresponding specific goal is placed under the intermediate ListAndP rioritize goal, due to being closely related to the functionality within the goal. This points to the fact, that GQM and KAOS can complement each other and help to reveal implicit requirements during an early stage of a project's timeline.

Importantly, the resulting diagram also demonstrates the reasoning behind each of the specific goals, and serves as a concise and readable way to present the requirements for the project to both end-users and developers willing to join the effort. This can be especially valuable in the context of an open source project, as readability of requirements is one of the important characteristics for the success and adoption of a project by the open source community [START_REF] Scacchi | Understanding the requirements for developing open source software systems[END_REF]. Moreover, the presence of high-level goals can allow the community to suggest alternative ways of achieving the goal, and could facilitate ideation process.

On the other hand, a possible drawback is that maintaining the diagrams relevant can become difficult as the project evolves, which is a common reason the open source projects tend to keep a minimally possible amount of informal documentation [START_REF] Scacchi | Understanding the requirements for developing open source software systems[END_REF].

Conclusion

In summary, the GQM and KAOS goal model can be used together to derive and represent a system's high-and low-level goals, actors, relationship between them and appropriate metrics to measure various important aspect of the system.

The goal model can facilitate and reflect the extensive process of mapping the strategic goals and concrete specific goals of the system, which are a prerequisite to implementing GQM.

The GQM, in turn, provides a quantitative perspective on this mapping in the form of appropriate metrics rooted in the system's strategic context, and can provide quantification for goals at different abstraction levels. The complementary union of the two models enables a more comprehensive and encompassing approach to defining, analysing and refining software systems. The combination of two approaches has been tried in the context of an open source mobile application, and allowed the authors to arrive at a set of specific functional requirements and metrics to measure the effectiveness of these functionality in achieving the high level business goal.

Moreover, the inter-complementary effect of both approaches was demonstrated, such that the KAOS serves specific goals to GQM, which simplify producing metrics. Whereas GQM facilitates discovery of implicit low-level goals through the questioning process of making high-level goals measurable.

Potential benefits and drawbacks of applying the combination in the context of an open source project were suggested. The benefits include increased readability and transparency of requirements to the open source developers community and the ability to see the origin and intent of each particular functionality. The latter can allow to suggest alternative or complementary ways of achieving high goals. A possible drawback is the difficulty of maintaining the diagrams and prose upto-date as the project progresses and evolves. Future studies of applying the two models and those similar to them could allow to further analyze and identify the advantages and disadvantages of using the suggested combination.

Fig. 2 :

 2 Fig. 2: Goal model for a professional social network

Fig. 3 :

 3 Fig. 3: Refinement types

Fig. 5 :

 5 Fig. 5: From strategic to specific goals

Fig. 7 :

 7 Fig. 7: KinApp's combined GQM and KAOS models

Table 1 :

 1 Exemplary GQM

	Goal	Improve the usability of the ticket booking feature from the point of
		view of a person booking a ticket (end user).
	Purpose	Improve
	Quality issue usability
	Object	the ticket booking feature
	Viewpoint	end user
	Question	Q1: How long does it take a user to complete the booking?
	Metrics	M1: Average task time
		M2: Standard deviation from the average
		M3: Percentage of cases exceeding the average maximum limit
	Question	Q2: What are the completion rates of the feature?
	Metrics	M4: Percentage of successfully completed bookings
	Question	Q3: How satisfied are the users with the feature?
	Metrics	M5: Average subjective rating by users grading the feature
	having these set of specific goal is what makes deriving metrics possible. The
	question arises then of how the set of specific goals for an organisation is actually
	produced.

Table 2 :

 2 GQM for M aximize[KeepingInT ouch]

	Goal	Maximize keeping in touch with family and friends from the point of
		view of a person affected by forgetfulness and being busy (end user).
	Question Q1: Is the person able to follow up with the configured regularity for
		each added contact?
		Q2: What is the longest streak (an unbroken sequence of successfully
		contacting an added person in accordance with the regularity)?
		Q3: What is the average streak for all added contacts?
	Metrics M1: Longest streak count
		M2: Average streak count


Shokhista Ergasheva
email: s.ergasheva@innopolis.university

Vladimir Ivanov

Ilya Khomyakov

Artem Kruglov

Dragos Strugar

Giancarlo Succi
email: g.succi@innopolis.ru

InnoMetrics Dashboard: The Design, and Implementation of the Adaptable Dashboard for Energy-Efficient Applications Using Open Source Tools

Keywords: Dashboards, Energy Efficiency, Adaptable Systems

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Due to modern-day rapid-changing business requirements, especially in the mobile space, and the advent of agile methodologies for tracking project status came need to continuously monitor the overall picture of energy consumption. At the beginning, it was assumed that only hardware metrics influence energy efficiency levels [START_REF] Pinto | Mining questions about software energy consumption[END_REF]. Lately, more focus has been given to software (code) and process related metrics [START_REF] Pinto | Energy efficiency: a new concern for application software developers[END_REF][START_REF] Liu | Data-oriented characterization of application-level energy optimization[END_REF]. In order to achieve the products' continuous delivery it is necessary in each step of software development process to help the development team to assess energy efficiency levels and suggest ways of potentially improving it. Therefore, our research tries to pinpoint the software, process and projectrelated metrics that have the influence on energy efficiency, and then optimizing these metrics. This paper highlights the steps we took to design, architect and develop a dashboard specifically made for the use case of monitoring energy efficiency and optimizing the set of parameters needed to be displayed to different users.

Outline of the paper

We start by mentioning the importance of the problem we are trying to solve as well as the research hypothesis by giving a literature review in subsection 1.2. The primary sections containing the majority of paper's novel contributions are sections 2, 3 and 4 where we decomposed our action plan and outlined all the metrics we used, and major design and architecture decisions. Here we talk at length about the aspects like technology choice, design style guide, layout, widgets, charts, security, complex systems and more. As the solution we have developed is still in its infancy, section 5 pinpoints the key ways our research is going to progress with the dashboard being one of the central tools. Finally, section 6 brings out some reflections.

Related Work

The field of effective dashboard design has gained traction at the beginning of the new millennium thanks to Stephen Few, among others. His "Information dashboard desig" [START_REF] Few | Information dashboard design[END_REF] is still being referenced by many. He starts by arguing that dashboards in the 1990s had major flaws, and that their root problem was in the inability to effectively communicate. Few also said that dashboards have to hold a balance between clarity and complexity [START_REF]Stephen Few and Perceptual Edge. Dashboard confusion revisited[END_REF]. With recent advances in the field of User Experience (UX) and User Interface (UI) design, dashboards started getting significantly more traction [START_REF] Malik | Enterprise dashboards: design and best practices for IT[END_REF][START_REF] Danovaro | PKM: Knowledge Management Tool for Environments Centered on the Concept of the Experience Factory[END_REF][START_REF] Janes | Effective dashboard design[END_REF][START_REF] Ivanov | Precooked Developer Dashboards: What to Show and How to Use -Poster[END_REF][START_REF] Ivanov | Design and validation of precooked developer dashboards[END_REF]. This resulted in dashboards being way more able to focus on the communication aspect, thus increasing use cases, leading to their wide-spread adoption [START_REF] Few | Information dashboard design[END_REF]. Dashboard are becoming also of stragegic importance in lean and agile environments [START_REF] Irina D Coman | Cooperation, collaboration and pair-programming: Field studies on backup behavior[END_REF][START_REF] Janes | Lean Software Development in Action[END_REF], to spread knowledge across workers [START_REF] Pedrycz | A model of job satisfaction for collaborative development processes[END_REF][START_REF] Pedrycz | Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity[END_REF], and in distributed development environments [START_REF] Maurer | Software Process Support over the Internet[END_REF][START_REF] Sillitti | Service Oriented Programming: A New Paradigm of Software Reuse[END_REF][START_REF] Corral | Software Assurance Practices for Mobile Applications[END_REF].

Meanwhile, with the increasing popularity of mobile devices, energy efficiency has become an ever-growing concern. Recent research by Pinto et al. [START_REF] Pinto | Energy efficiency: a new concern for application software developers[END_REF] pinpointed that energy efficiency has to be addressed at all levels of the software stack. More specifically, they argue that it is due to the developers having the lack of knowledge and lack of tools. These bottlenecks prevent software developers from identifying, fixing, refactoring, and removing energy consumption hotspots [START_REF] Choon | Energy efficient utilization of resources in cloud computing systems[END_REF].

Finally, it would be advisable to take an open source approach to this topic, since it would ensure a larger diffusion of the dashboard [START_REF] György L Kovács | Open Source Software for the Public Administration[END_REF][START_REF] Fitzgerald | Adopting open source software: A practical guide[END_REF][START_REF] Di | A multivariate classification of open source developers[END_REF].

Energy-Efficiency related metrics

In this section we describe the importance of metrics included in the dashboard we are developing. We start by providing an overview of the project as well as additional information to place the system in context. Firstly, the dashboard we are building is mainly concerned with monitoring the energy efficiency of software artifacts. The goal is to make it easy and convenient for managers as well as other users to track the energy consumption of the software. Overall, energy consumption metrics referring to the energy consumption can be divided into:

-Process metrics -Project metrics -Product metrics We provide several Process and Product metrics throughout this paper that encapsulate the functionality of the dashboard.

Dashboard's primary purpose is to share important information in terms of different metrics to the end user. Determining what what it is that is important to the user is the most critical part of designing the dashboard. By considering the metrics provided one would gain crucial insights, allowing them to make more educated decisions faster.

By working closely with the industry representatives, as well as consulting the literature in this field (data and findings revealed in [START_REF] Li | An investigation into energy-saving programming practices for android smartphone app development[END_REF][START_REF] Chatzigeorgiou | [END_REF][START_REF] Vlad Vasile | Mobile phones and energy consumption[END_REF][START_REF] Ching | Toward using software metrics as indicator to measure power consumption of mobile application: A case study[END_REF][START_REF] Cruz | Performance-based guidelines for energy efficient mobile applications[END_REF]), we have come up with the list of metrics that the potential users would find most valuable [START_REF] Vernazza | Defining Metrics for Software Components[END_REF][START_REF] Sillitti | Measures for mobile users: an architecture[END_REF][START_REF] Scotto | A Relational Approach to Software Metrics[END_REF] (Table 1).

These metrics are of paramount importance in qualifying the results of the work so that developers can judge objectively the development status. The main contribution can be directed into the energy efficient development of software in teams and individually. The proposed metrics were computed by applying rules common to the existing hierarchical measures of other internal software attributes. For example, by knowing the Number of classes in the source code we can easily calculate other metrics like: Number of Immediate sub-classes of a Class, Response for a Class, Weighted methods per Class as well as the Number of Children metric. It can be followed by better development estimates and more qualified product that can maintain the customers expectations. As it was stated in the paper [START_REF] Ching | Toward using software metrics as indicator to measure power consumption of mobile application: A case study[END_REF], developers need a set of valid indicators in assisting the developers for better applications. Thus, the study results showed that Cyclomatic complexity, Number of classes, LOC, number of parameters, nested block depth, number of methods, weighted methods per class and method lines have direct relationship with power consumption of mobile applications. Authors of the paper [START_REF] Li | An investigation into energy-saving programming practices for android smartphone app development[END_REF] demonstrated specific coding practices for reading array length information, accessing class fields and performing invocations help to lower the overall energy consumption and improve the usability of the software.

Expanding the list of energy consumption metrics is already on our research agenda. However, not only obtaining additional metrics is of high significance. Combining the already existing ones, as well as easily modifying them is very important. Thus, section 4 describes how we utilized these metrics in greater detail. [START_REF] Few | Dashboard design: Taking a metaphor too far[END_REF][START_REF] Few | Information dashboard design[END_REF][START_REF]Stephen Few and Perceptual Edge. Dashboard confusion revisited[END_REF][START_REF] Few | Data visualization: past, present, and future[END_REF] pointed out several key characteristics that all dashboard should have. This section makes a bridge between the section 2 and the design guidelines. As he (Few) puts it, "A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance."

We strongly agree with the majority of his guidelines. First, all elements of a dashboard have to have a purpose and only the most important information that truly contributes to a specified goal should be shown to the end user. A dashboard is a mean of communication, and in order to effectively convey the message to the user, the message has to be clear, concise and goal-oriented.

Message has to be clear The information presented on a dashboard is most commonly a combination of graphics and text, with the emphasis on visual representations. The reason is that visuals can communicate with richer meaning than text on its own. Having clear, easy to understand graphics is one of the crucial requirements from the design perspective.

Message has to be concise It is highly advisable to fit the main metrics of the dashboard to one screen and one screen only, without horizontal and vertical scrolling. The users should be able to easily scan through all the visuals effortlessly and obtain the insights they are looking for. If the system consists of dozens and even hundreds of smaller metrics and they all need to be shown, the ones that are less important should be moved elsewhere. Only the metrics with high level of importance should appear on the main page.

Message has to be goal-oriented Whether it is to gain an important business insight, or to successfully finish a project on time, dashboard's goal is to get users as close to their objectives as possible. What we have found is that a feedback loop, where users would report which metrics are more relevant to the goal than others, is one of the most valuable insights we could get to improve the set of metrics that get shown. This approach will be thoroughly examined later.

Visualization

As mentioned in the previous subsection, the visual representation and layout play a crucial role in designing a goal-oriented dashboard. Hence, visuals like widgets [START_REF] John O Louch | Synchronization of widgets and dashboards[END_REF], graphs, charts all require special attention. Thus, we needed to develop a style guide, a set of standards and styles that all of our visual elements would conform to. It would enforce style to improve communication via consistency and clarity. The fully developed style guide can be viewed at: https://innometrics-12856.firebaseapp.com//dashboardhttps://innometrics-12856.firebaseapp.com/.

Here is a comprehensive list of visual elements and containers chosen for the style guide:

-Colors -Typography -Container Elements (cards, carousels, modals, tables, etc.) -Functional Elements (forms, menus, etc.) -Charts

We have divided colors into three parts: theme colors, grays, and additional colors. This way the theme could be changed, and the users could tune the settings to their liking. Next, typography for headings one through six, as well as bolded, italicized and underlined text has also been defined. In addition, cross-functional and organizational elements such as forms (with buttons and input fields), tables (with cells), and different types of containers also needed a well-defined set of styling rules.

However, the main page of the dashboard consists almost entirely of widgets and charts. As we noted earlier, it is crucial to keep the main page free of clutter and not have to scroll or click elsewhere to get to the crucial information. That is why, due to their effectiveness and expressive power, we chose to fine tune their visual appearance. Taking multiple existing charting libraries into consideration and comparing their advantages and disadvantages, we decided to go with https://www.chartjs.org/ChartJS . It has all the visuals we wanted, such as bar, line, and area charts and allows for editing when needed. Figure 2 depicts some of them that we decided to include in our prototype.

Technology

Considering the architectural decisions from section 2 and design decisions from sections 3.1 and 3.2, choosing the platform was relatively straightforward, it had to be a web application. That means that the main programming language for development should be JavaScript (JS).

However, choosing the appropriate JS framework/library was a challenging task. There are numerous options, and all of them offer similar functionality that we need. The one that stood out and ultimately we ended up choosing is Reac-tJS. The primary reason being that its Virtual DOM (Document Object Model) is optimized for real-time applications and that it facilitates the overall process of writing components. Our system could potentially consist of hundreds of metrics, and reusing existing components while at the same time having impressive performance is critical.

In addition, file structure in ReactJS, although not opinionated, could be organized in very practical and clear ways. Consider, for example, our code structure, shown in the figure 3. It captures almost all the functionality while To manage application's state we use https://redux.js.org/Redux , a flexible, and predictable state container. It allows us to easily handle user input, authentication and authorization, as well as storing the results from REST API calls. One of the main benefits of using a state container is the centralization, where application's state and logic enable capabilities like undo/redo, state persistence and much more. definition

Dashboards as Complex Systems

As mentioned in the subsection 3.1, all dashboards must be optimized for a certain objective. As energy efficiency became a greater problem in recent years [START_REF] Murray G Patterson | What is energy efficiency?: Concepts, indicators and methodological issues[END_REF], our research agenda encapsulated that. It was immediately noticeable that the majority of state-of-the-art energy efficient software solutions had a very limited and strictly defined set of metrics to optimize for. We took a different approach to addressing energy efficiency concerns. Instead of having a fully deterministic system, one in which each and every metric is manually chosen to be displayed, we opted for a system where an algorithm would decide whether or not to display a certain metric. The system would still Fig. 3. File Structure of the Implementation take into consideration the manually entered metrics, albeit not relying on them heavily. All of this would be supported by a feedback loop, where users would rate the importance of the metrics they have been shown.

Self-adaptable Systems and Energy-Efficiency Dashboards

Self-adaptable systems (systems with behavior just mentioned) have recently gained traction [START_REF] Anthony | Dynamics of complex systems[END_REF], and they could be used as an abstraction to represent the software artifact, such as a dashboard. The rationale behind choosing Complex Systems (CS) to perform the self-adaptation is the following; by definition, a CS is a system with components particularly difficult to model. As it has not yet been fully investigated which factors (metrics) influence the efficiency the most, one approach would be to try and make all combinations of metrics and record the results. However, one may notice that as the number of metrics significantly increases, generating all combinations becomes unfeasible. In addition, metrics do not necessarily need to interact with others in linear fashion [START_REF] Michael | Nonlinear dynamics and chaos[END_REF]. In fact, most of the interactions in CS are nonlinear, and often stochastic.

Therefore, we argue that dashboards for energy efficiency applications should be represented as a Complex System. Metrics (presented in section 2), key components of dashboards, become agents within the system. That allows us to closely monitor the relationships each metric has with the other. It became apparent that almost no metrics are isolated, i.e. energy saved depends on a vast majority of software metrics that interact with each other. Thus, carefully monitoring and reacting to change is critical.

By embracing feedback loop and adaptation, these metrics are able to synchronize their internal states with the other metrics in the system. Additionally, the system should be able to recognize these changes and self-adjust with the emergence of globally coherent patterns of adjustment developing. This way the system would notice a change, and adapt to it.

So far we have only discussed the interactions between the metrics contributing to the global state. However, a major part of any CS is the ability of the system to feed back the globally coherent patterns to micro-level agents. These patterns are discovered either by the system or the dashboard user. An example of a system-discovered pattern in the energy efficiency context would be battery drainage per minute. The system, if noticed irregularities, would notify the agents, and they would adjust to that situation. The next section showcases the implementation details of such an approach.

Application of Evolutionary Algorithms

To be able to achieve the self-adaptation needed from our dashboard, a mechanism for handling the fittest metrics is needed [START_REF] Sn Sivanandam | Genetic algorithms[END_REF][START_REF] Forrest | Genetic algorithms: principles of natural selection applied to computation[END_REF] 1 . The goal can be derived from the fitness function, which we define as follows: definition Definition 1. An agent has a higher contribution/fitness to the overall system if and only if a slight change in a specific metric would yield a significant change in the amount of energy saved, the difference between the expected energy efficiency level and the current level diverge significantly, or a metric answers custom energy-related questions that users may have.

With this in mind, by applying the fitness function to each of the metrics inside the system we would get a numeric score for each of them. This tells the system how much does each individual metric influence the overall energy efficiency level of system as a whole. Furthermore, the user would be shown only the metrics particularly relevant to the current state of the system. Thus, by choosing this exact fitness function, these would follow:

metrics that deviate from the mean are more likely to be shown to the user; metrics whose improvement may result in other metrics' fitness are more likely to be shown to the user; metrics of all sizes whose improvement would not yield a substantial increase/decrease in energy efficiency levels, have less change of being shown to the user.

The final algorithm would then run as follows:

1. assign a fitness score to each agent in the system (according to the fitness function defined above)

2. select members to act upon using some variation operators (crossover and mutations) [38]2 3. replace certain members of the population with these children from variation operators 4. keep some members from the previous population in the new population The next step would be then to perform the natural selection between the metrics. More specifically, we chose the tournament-like contest in which the winner would continue to breed, and the loser would be most likely eliminated [START_REF] Blickle | A mathematical analysis of tournament selection[END_REF][START_REF] Gw Greewood | Emphasizing extinction in evolutionary programming[END_REF].

Dynamic Equilibrium.

As energy efficiency monitoring solutions became larger, stakeholders operating such systems would have significant determining which metric to display and optimize for, and which to ignore. Such a system is said to be deterministic, as all the change has to be manually done and the system has no means of improvement.

Advances in the field of big data and machine learning indicate that the above-mentioned system architecture could be transformed to be stochastic, with no absolute governance. In addition, the data being collected is growing exponentially [START_REF] Anagnostopoulos | Handling big data: research challenges and future directions[END_REF], and thus more metrics are going to become available for processing. A non-deterministic system operating in such a way would neither be in the state of maximum entropy, nor would it be completely stable. Such a state is referred to as Dynamic Equilibrium, where actors are loosely bound to each other while still having plenty of room to improve.

Future Research Agenda

To date, research on software-oriented energy efficiency has been focusing almost solely on code-related metrics, such as those presented in section 2. However, no evidence proves the importance of process-related metrics (productivity, turnaround time, throughput, iteration burn-down, error rate, etc) to the energy efficiency. Thus, our research agenda includes primary research to investigate the correlation of above-described metrics to the energy consumption and saving levels.

In addition, we aim to extend the list of energy consumption code metrics considered by the dashboard. For example by knowing the Number of Classes (NoC) in the source code we can easily calculate other metrics like: Number of Immediate sub-classes of a Class, Response for a Class, Weighted methods per Class as well as the Number of Children. This may lead to performance benefits and ultimately savings in computing power. Performance is one of the key aspects that is also a part of our research agenda.

Finally, we would like to supplement metrics with a reliable prediction system [START_REF] Musílek | On the Sensitivity of COCOMO II Software Cost Estimation Model[END_REF][START_REF] Ronchetti | Early estimation of software size in object-oriented environments a case study in a cmm level 3 software firm[END_REF][START_REF] Rossi | Modelling Failures Occurrences of Open Source Software with Reliability Growth[END_REF][START_REF] Sillitti | Understanding the Impact of Pair Programming on Developers Attention: A Case Study on a Large Industrial Experimentation[END_REF].

Conclusions

Energy efficiency, the minimization of the amount of energy required to provide products and services, has become a growing issue. Mobile phones, smart watches and other portable devices operate in a very power-constrained environment. Research in the area of energy efficiency would facilitate the efforts to cut energy costs, and ultimately reduce greenhouse gas emissions [START_REF] Brookes | The greenhouse effect: the fallacies in the energy efficiency solution[END_REF].

Advent of software solutions, on the other hand, have profoundly impacted almost all spheres of human existence. To date, more attention has been given to the study of energy efficiency in hardware, rather than in software [START_REF] Capra | Measuring application software energy efficiency[END_REF]. This paper considered code, as well as process and project-related metrics as indicators of potential energy savings. Additionally, we presented a dashboard that tracks such metrics and only displays the ones that are most relevant. It manages to do so by combining the notions from Complexity Theory (Non-linear dynamics, Dynamic Equilibrium) with Evolutionary Algorithms (natural selection) to synthesize an adaptable stochastic tool.

Fig. 1 .

 1 Fig. 1. Typography -Headings 1 through 6

Fig. 2 .

 2 Fig. 2. Different types of Charts from ChartJS

Table 1 .

 1 Code metrics for the energy efficiency measurement Collections types in Java Number of frameworks that provide an architecture to store and manipulate the group of objects in Java.

	6	Classes	The number of classes used in the source code.
	7	LOC (XML)	Lines of code in XML.
	8	LOC (Java)	Lines of code in Java.
	9 10	Invocation	Number of invocations of a program or a function.
	3 Design, Visualization & Development
	3.1 Design
	S. Few in his

# Name of the metric Description 1 Performance tips Strategies that can be used to conserve and extend the battery life. 2 Code size The length of the code and row column weight in the form. 3 Loops Number of For and While loops in the source code. 4 Code Smells Internal Setter, Leaking Thread, Member ignoring method and Slow Loop consume up to 87 times more. 5 Cyclomatic complexity It counts the number of flows through a piece of code.

Notice the use of the word fittest, indicating the usage of natural selection as a mean of choosing the metrics that tell the most about the specified goal.

Crossover is combination of parents' genetic information, and mutation is a change in agent's genetic information

Acknowledgments

The work presented in this paper was supported by the grant of Russian Science Foundation # 19-19-00623.


Alberto Sillitti
email: a.sillitti@innopolis.ru

Gerta Kapllani
email: g.kapllani@innopolis.university

Ilya Khomyakov

Ruzilya Mirgalimova

An Empirical Analysis of the Maintainability Evolution of Open Source Systems

Keywords: Maintainability, empirical software engineering, software evolution

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Maintainability is one of the most important features of software systems [START_REF] Kan | Metrics and Models in Software Quality Engineering[END_REF]. Though, defining maintainability has been a topic of interest for many researchers over decades. A software application might fulfils all the traditional requirements and yet be of little use if the cost of maintaining it is too high. For such reason, maintainability has been considered one of the requirements that must be imposed to software products [START_REF] Cheaito | Defining and Measuring Maintainability[END_REF]. Several authors have investigated the models and metrics used for the best estimation of maintainability also known as prediction or software maintainability prediction models. Traditional models based on their accuracy of prediction have been represented by Shafiabady et al. [START_REF] Shafiabady | Investigation of software maintainability prediction models[END_REF]. Moreover, an analysis of the evolution of software maintenance models over the four past decades has been introduced by Lenarduzzi et al. [START_REF] Lenarduzzi | Analyzing Forty Years of Software Maintenance Models[END_REF]. Defining the right tools on code analysis to perform research studies on software maintenance prediction is also a crucial aspect and has received much attention in the last few years [START_REF] Moser | A model to identify refactoring effort during maintenance by mining source code repositories[END_REF]. Recent papers [START_REF] Bordeleau | Fifteen Years of Open Source Software Evolution[END_REF] [5] [START_REF] Khomyakov | A Novel Approach for Collecting and Sharing Software Metrics Data[END_REF] [15] [START_REF] Petrinja | Comparing OpenBRR, QSOS, and OMM Assessment Models?[END_REF] have identified tools that are available and can be used to apply maintenance models published in the studies in this area.

The paper is organized as follows: Section 2 briefly analyzes the state of the art; Section 3 introduces our investigation; Sections 4 and 5 present and discuss the results; finally, Section 6 draws the conclusions and introduces future work.

Related Research

During the years, several models have been proposed for measuring the maintainability of software. A considerable amount of studies have employed different models and techniques for predicting software maintenance using basic metrics and models: McCall's model in 1976 [START_REF] Rawashdeh | A new software quality model for evaluating COTS components[END_REF], Barry Boehm's quality model presented in 1978 [START_REF] Boehm | Characteristics of Software Quality[END_REF], Sneed-Mercy Model in 1985 [START_REF] Sneed | Automated software quality assurance[END_REF], Li-Henry Model in 1993 [START_REF] Li | Object-oriented metrics that predict maintainability[END_REF], Marcela Genero Model in 2004 [START_REF] Genero | Building UML class diagram maintainability prediction models based on early metrics[END_REF]. Later, slightly different techniques were used from simple statistical models such as regression [START_REF] Zhou | Predicting object-oriented software maintainability using multivariate adaptive regression splines[END_REF] to machine learning [START_REF] Van Koten | An application of Bayesian network for predicting object-oriented software maintainability[END_REF] [7], and deep learning [START_REF] Kumar | Hybrid functional link artificial neural network approach for predicting maintainability of object-oriented software[END_REF] [START_REF] Jha | Deep Learning Approach for Software Maintainability Metrics Prediction[END_REF]. To predict maintainability of software, different metrics have been proposed in literature. Among the large variety of metrics, the most used ones are the OO metrics depth of the inheritance tree (DIT), response for a class (RFC), number of children (NOC), coupling between objects (CBO), lack of cohesion of methods (LCOM), and weighted method per class (WMC) [START_REF] Chidamber | Towards a metrics suite for object oriented design[END_REF]. Other metrics are popular, such as: number of methods (NOM), lines of code (LOC), number of semicolons in a class (SIZE1), number of properties (SIZE2), and CHANGE metrics as dependent variable to predict software maintainability by calculating the changed number of lines in the class during maintenance process [START_REF] Li | Object-oriented metrics that predict maintainability[END_REF]. Investigating different primary studies and secondary studies from the literature, we have found out that even if there are a number of different methods to predict maintainability, almost all the studies built their models from scratch and do not extend existing ones.

Our investigation

We decided to analyse software maintainability change following the approach proposed in [START_REF] Molnar | Discovering maintainability changes in large software systems[END_REF] for the jEdit open source project using the JHawk tool to get useful insights. jEdit is an open source project developed in Java available in SourceForge 1 with all existing versions with the related source code. We analyzed the application starting from version 3.2 up to version 5.5, a total of 12 versions. There are in total 41 versions but they have almost no differences compared to the selected ones. A significant difference compared to [START_REF] Molnar | Discovering maintainability changes in large software systems[END_REF] is that we rely on general information outside the source code to examine and find potential relation between version numbers and changes to maintainability during the evolution of the project.

JHawk is Java-based open source framework which can analyse source code while performing static analysis generating graphical form results. It takes Java code as input and generates code metrics. We are interested in gathering features such as average cyclomatic complexity (CC), number of lines of code (LOC), and maintainability index (MI).

Collected Metrics

In this study we have focused on the following metrics:

1 https://sourceforge.net/projects/jedit/files/jedit/ -Cyclomatic Complexity (CC): it is used to measure the complexity of a piece of code from the point of view of the possible execution paths. In particular, it measures the independent path through a source code as a proxy of its complexity. The higher the value, the more complex is the system and this fact results in difficulty to maintain the code. We have measured the average of CC which is stated by McCabe to be cyclomatic complexity per function in a file and is calculated as the sum of the CC of all function definitions, divided by the number of function definitions in the file. -Lines of code (LOC): it is used to measure the size of a software by counting the number of lines included in a program. There are multiple ways to count the lines of code but in this work we focus on the simplest definition that consider the physical lines of code of the source files. -Maintainability index (MI): it is based on Halstead Volume (HV) metric, the cyclomatic complexity (CC) metric, and the average number of lines per code per module (LOC) metric. The higher this value, the more maintainable the software results [START_REF] Scotto | Dealing with Software Metrics Collection and Analysis: a Relational Approach[END_REF]. If MI of software increases it means that the software can be maintained easily, otherwise if it decreases the degree of difficulty to maintain the software is high [START_REF] Khondhu | Is it all lost? A study of inactive open source projects[END_REF]. Values of MI index as follows:

• MI < 65: difficult to maintain • MI 65-85: maintainable • MI > 85: good maintainability

We have to highlight the fact that in JHawk MI is reported in two forms, one that takes into account comments of the program and ones which does not. In our database we have reported both of them, but the final result of the graphs below do contain only the ones with comments. It is stated that the reason that MI formula with comments does exist is because the formula to find comments is not standardized and comments are considered to be subjective from developer to developer and do vary from project to project. In such cases it is recommended to use the MI without comments as reported from official source development of the JHawk tool. However, we think that comments are an important part of the source code, therefore we considered them in our study.

Results

In Fig. 1 we present the results achieved from data collection of MI for each version of jEdit, as we have mentioned in the above sections.

We can notice that from version 3.2 to 4.2 there is a constant value of MI of about 130. Upon release of version 4.0, there is a tremendous decrease of MI to about 58. Then, with next releases up to version 5.3.0, there seem to be a slight increase about 68. The last versions have a value of MI up to 144.

We also have to consider the other two features collected (Fig. 2, 3, 4). It is of high relevance to mention that the value of the average CC has been higher for versions of low MI index in other words for early releases. A slightly different trend follows the LOC metric. For higher values of LOC there is also a higher value for MI and vice versa. We can notice the trend of size metric for each version. For an increase of size from versions 4.3 to 4.4.1, there is a decrease in MI in these versions. For a decrease of size in versions 4.5 to 5.3, there is a slight increase of MI. Surprisingly, for an increase of size in 5.4.0 and 5.5.0, there is also an increase in MI.

From Fig. 3 and4, we can state that for an increase in CC from version 4.2 to 4.3, there is a decrease in MI. Moreover, for a decrease in CC from versions 4.3.3 to 4.4.1, there is an increase in MI. The same result follows 5.3.0 to 5.4.0 with a decrease in CC and an increase in MI. These outputs clearly highlight the fact that a high value of CC means more complexity added to the system which increases the degree of difficulty for the system to be maintained.

Discussion

In this section we are going to investigate further the results obtained from previous section. We are going to test hypothesis of possible correlation of LOC, CC with MI and LOC with CC. Since the MI is defined as a function that include LOC and CC as independent variables, we expect that MI is highly correlated to them but we want to investigate how strong is the effect of such variables on the overall values of the MI.

We use the same data provided in the initial steps of our work collected with the JHawk tool. To test correlation, we are postulating hypothesis in Table 1. In overall, correlation claims the strength of a relationship. We use the Cohen definition of correlation [START_REF] Cohen | Statistical Power Analysis for the Behavioral Sciences[END_REF] where he defined a value less than 0.3 as a weak correlation, 0.3 to 0.5 as medium correlation, and greater than 0.5 as strong correlation. We are interested to understand the strength of such correlation. To measure the strength of relationship in our data variables we use Spearman rank correlation for the reason that it does not assume any linear relation between variables and it does not assume any specific distribution of data. We analyze the data in Python, using a Jupyter Notebook with the stats.py statistical package. For this project, Spearman correlation coefficients are:

1. 0.129 for LOC and MI with p-value 0.522 which aims result is not significant, so we can not reject hypothesis null. 2. -0.098 with a p-value 0.628, again not significant so we can not reject hypothesis null. 3. 0.513 with p-value 0.006 is significant therefore we can reject hypothesis null and conclude that a relationship exists.

We have performed the same kind of analysis over other 4 open source projects:

-SoundHelix (0.6-0.9 versions): a Java framework for algorithmic random music composition. It can generate random songs, play them, and it is highly customizable using XML configuration as described in SourceForge.net. -jWebUnit (1.0-3.2): a Java framework for testing web applications.

-jXLS (0.7-2.7.1): a Java library for writing Excel files using XLS templates and reading data from Excel into Java objects using XML configuration. -jTDS (0.1-1.

3.1): an open source driver for Microsoft SQL Server

We went again through same process described for the analysis of jEdit to collect LOC, MI, and CC metrics, and perform the correlation analysis as reported in Table 2.

As we can notice in Table 2, the values in red identify significant values. All projects include some significant values except SoundHelix. There is a strong correlation of the LOC metric and MI in JXLS and JTDS and a strong correlation of CC and LOC in jEdit project. Also to be highlighted is the fact that exist strong negative correlations. The not significant results obtained from the SoundHelix project might derive from the fact that it is the project with the least data collected, this could have affected the overall analysis since only 4 versions were analyzed.

Conclusions and Future Work

The paper has introduced a preliminary analysis of the maintainability of 5 popular open source projects and how strongly it is correlated with some basic code metrics (lines of code and cyclomatic complexity) that are used in the definition of the Mainteinability Index.

A systematic analysis of the mainteinability of popular open source projects to understand how the code is managed and how the development team address the evolution of the code is important to increase the level of awareness in the usage of such products and to understand the possible effects in the long term.

Fig. 1 .

 1 Fig. 1. MI evolution.

Fig. 2 .

 2 Fig. 2. LOC and MI evolutions.

Fig. 3 .

 3 Fig. 3. CC evolution.

Fig. 4 .

 4 Fig. 4. CC and MI evolutions.

Table 1 .

 1 Hypotheses.

	No. Hypotheses
	1	Null: LOC and MI are unrelated.
		Ha: LOC is positively correlated with MI.
	2	Null: CC and MI are unrelated.
		Ha: CC is positively correlated with MI.
	3	Null: LOC and CC are unrelated.
		Ha: LOC is positively correlated with CC.

Table 2 .

 2 Correlations in all the considered projects.

	Project	LOC-MI (pval) CC-MI (pval) LOC-CC (pval)
	jEdit	0.129 (0.522)	-0.098 (0.628) 0.513 (0.006)
	SoundHelix -0.816 (0.184)	0.333 (0.667)	-0.816 (0.184)
	JTDS	0.733 (0.000)	-0.578 (0.000) -0.249 (0.161)
	JXLS	0.695 (0.000)	-0.812 (0.000) -0.567 (0.000)
	JWebUnit -0.320 (0.226)	0.200 (0.457)	-0.789 (0.00)