Anna Hannemann

Kristjan Liiva

Ralf Klamma

Navigation Support in Evolving Open-Source Communities by a Web-based Dashboard

The co-evolution of communities and systems in open-source software (OSS) projects is an established research topic. There are plenty of different studies of OSS community and system evolution available. However, most of the existing OSS project visualization tools provide source code oriented metrics with little support for communities. At the same time, self-reflection helps OSS community members to understand what is happening within their community. Considering missing community-centered OSS visualizations, we investigated the following research question: Are the OSS communities interested in a visualization platform, which reflects community evolution? If so, what aspects should it reflect?

To answer this research question, we first conducted an online survey within different successful OSS communities. The results of our evaluation showed that there is a great interest in community-centered statistics. Therefore, we developed an OSS navigator: a Web-based dashboard for community-oriented reflection of OSS projects. The navigator was filled with data from communication and development repositories of three large bioinformatics OSS projects. The members of these OSS communities tested the prototype. The bioinformatics OSS developers acknowledged the uniqueness of statistics that the NOSE dashboard offers. Especially, graph visualization of the project social network received the highest attention. This network view combined with other community-oriented metrics can significantly enhance the existing visualizations or even be provided as a standalone tool.

Introduction

Success of an OSS project is tightly interwoven with the success of its community [START_REF] Raymond | The Cathedral and the Bazaar[END_REF], [START_REF] Eric | Open source software and the "private-collective" innovation model: Issues for organization science[END_REF]. OSS systems co-evolve strongly with their communities [START_REF] Ye | The co-evolution of systems and communities in free and open source software development[END_REF]. Thus, the more successful a project is, the higher is the degree of its complexity in terms of project structure and community size. The complexity affects the awareness of community members of what is happening in their community. In interviews with OSS developers Gutwin et al. in [START_REF] Gutwin | Group awareness in distributed software development[END_REF] find out that the awareness of other developers within OSS projects is essential for an intact project life. Within the study, project mailing lists (MLs) and text chats are determined as the main resources for maintaining group awareness. However, in large OSS projects, it gets very difficult for community members, especially for the less experienced ones, to establish a complete and correct perceptional awareness model. In such cases, Gutwin et al. suggest to develop new representation methods for communication and its history.

Considering OSS mining research, there are already studies concentrating on OSS communication analysis: to investigate social network structure [BGD + 06], to analyze content [START_REF] Bohn | Contentbased social network analysis of mailing lists[END_REF], to estimate the sentiment within OSS communities [START_REF] Jensen | Joining free/open source software communities: An analysis of newbies' first interactions on project mailing lists[END_REF]. OSS communication repositories reflect complete communities of the corresponding projects. In contrast, OSS source code repositories are restricted to the developers only. If we take a look at the OSS visualization platforms (e.g. GitHub, Ohloh, etc.), then they are focused either on source code or individual contributors. Platforms which provide OSS metrics based on project communication are still missing. To investigate this research niche, we address the following research question: Are the OSS communities interested in a platform reflecting community evolution and if so, what evolution aspects should it reflect?

The rest of the paper is organized as follows. Section 2 provides an overview on related systems for OSS project evolution visualization. To address our research questions, we executed an iterative study (Section 3). The achieved results are presented in Section 4. Section 5 concludes the paper and gives an overview of some ideas for future work.

Related Research

There are already plenty of related applications available for OSS development visualization. To give an overview of existing concepts and principles, the more notable ones are presented.

GitHub 1 offers a web-based hosting for software projects. Additionally, it provides visualizations focused mainly on project source code (commit activity, code amount) and some statistics on project contributors (contributor activity, followers and following people, projects, organizations, etc). Another popular web-platform for software projects' hosting is SourceForge 2 . It offers just some statistics on project traffic (hits on the project, number of downloads) and SVN activity. What statistics are visible to users depends on the project settings. In contrast, a web-service Ohloh 3 does not host the actual source code, but simply crawls and analyzes the OSS data. Ohloh offers many charts regarding the source code and contributors (their ranking and activity). Pure statistics are transformed into textual statements. Ohloh also provides data on project estimation effort based on the COCOMO model for software cost development . The supported analysis is divided into three different types according to data resource used: data from source code repositories, data from mailing lists archives and data from tracker system repositories. However, not all projects have data regarding all three resources. Melquiades offers important metrics, like activity over time, growth and member inflow rate. The next two visualizations Open Source Report Card (OSRC)6 and Sargas [SBCS09] provide contributororiented metrics. Based on the data from GitHub OSRC establishes developers profiles based on their daily and weekly activities, project participation, etc. Whereas, Sargas estimates the social profile of contributors based on their behavior within four social networks: open discussion forum, developers discussion list, discussions about the bugs and social network extracted from the source code.

To summarize, the existing applications are ranging from source code hosting services with visualization tools to pure analysis and visualization platforms. The last clearly proves the need and the interest of the OSS communities in self-monitoring tools. However, the existing systems focus mainly either on the system source code or on individual contributors. For monitoring of community evolution the information need to be presented from different perspectives.

Study Settings

Figure 1 represents the workflow of our study. To find out if the OSS members are interested in the community-related reflection of their projects, we first conducted an online survey within OpenStack, PostgreSQL, GIMP, Mozilla, Oracle VM VirtualBox, GNOME, TomCat OSS communities. The survey addressed questions related to the developer interest in a community-oriented metrics and what metrics are missing in the existing OSS navigators. Most of the questions had an optional comment field. We contacted OSS developers via the Internet Relay Chat (IRC) channels. The survey was anonymous, therefore, it was not possible to trace from which project the participants originated. Nevertheless, based on the survey results and by observing each chat for the next four hours, no malicious users were detected. The result of the survey was a positive answer to the first part of our research question. The OSS members do have strong interest in platforms reflecting OSS community evolution. Additionally, the OSS members suggested several ideas for metrics/aspects, which were assumed to be important to be aware of. To evaluate the feasibility of the collected ideas, we next applied prototype testing.

We selected a dashboard as a technological approach. "Dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance" [START_REF] Few | Information Dashboard Design: The Effective Visual Communication of Data[END_REF]. The goal of the developed Navigator for OSS Evolution (NOSE) is to provide community-oriented navigation support for OSS project members. We filled the NOSE dashboard with the data from three long-term bioinformatics OSS (BioJava, Biopython and BioPerl), which have been already analyzed in our previous studies (e.g. [START_REF] Hannemann | Community dynamics in open source software projects: Aging and social reshaping[END_REF]). Therefore, we were able to proceed with the prototype testing immediately after the development. An iterative development process of the NOSE dashboard was executed. Before starting the survey with bioinformatics communities, the dashboard was evaluated with 10 computer scientists. This evaluation was used to identify the design shortcomings of the developed dashboard. The relevance of the presented metrics and the data quality reflected in NOSE were directly investigated within bioinformatics OSS communities. We contacted the bioinformatics OSS community members via private emails. Such textual inquires encourage the participants to explain their answers and, thus, provides a more detailed feedback. Moreover, informal email exchange could trigger a fruitful discussion. The sent out email consisted of a short description of the NOSE goal and three questions:

-Do you find that the visualization features offered by GitHub are sufficiently informative? -Would you additionally like to have features to represent the community and its structures? -If so, would network graph be a viable option?

Navigator for OSS Community: Evaluation Results

In following the results of both conducted surveys and prototype testing are presented.

Survey of OSS Developers

From OSS developers we received 32 responses with many (49) comments. Some developers provided initial feedback directly in the chat. Everyone who started the survey also finished it, what indicates the true interest of the participants in the survey. Figure 2 displays the survey questions with the collected feedback. Every question was optional, therefore some questions had less than 32 answers.

Web-platforms like Ohloh or GitHub were used by 75% of the participants. There were 16 comments in total, with GitHub being mentioned 13 times, SourceForge 5 times and Ohloh 4 times. 63.3% of the OSS developers were interested in the statistics related to community evolution. However, in four of the seven comments the participants mentioned, that it was unclear what kind of information was meant or "How would/could I benefit from those information?" Social Analysis The OSS developers were mostly interested in getting statistics from the MLs regarding the whole community. MLs were recognized as a useful source of information for getting an approximate user base size. However, one participant also mentioned a negative aspect, that "[...] too much statistical evaluation could put the community of as they feel 'observed' ". The

!"#$%&'(

)#$ *' +,%&'(-. !"#$"%#%&'#()*+,,-/01234456 .#(./* 012'#$"%#'2'3#4"56378%6'9#6"#15$#()*+,,#:3";'46#('<=#8%=#3':"36*-/312789756 >#(?<>/* +@A"@B#C760%8#159#&7D7A13#:A16E"3D&#E"4%&#"5#F"GH4"DD76#&6167&674&B# I@'3'1&#J#I156#6"#E"4%&#"5#4"DD%576$(K:'":A'*#'2"A%67"5<#!"#$"%#E759#6@'# 75E"3D167"5#6"#8'#756'3'&675=-37128/9/56 >>#(?L<M/* J#I156#6"#:3"279'#&6167&674&#"5#6@'#I@"A'#4"DD%576$B#8%6#1A&"#'2'3$# 4"56378%6"3#E3"D#6@'#D17A75=#A7&6<#N"%A9#$"%#E759#76#%&'E%A-0:12;<56 O#(PQ/* R3'#$"%#756'3'&6'9#75#1#5'6I"3S#27'I#(&"471A#=31:@*#"E#$"%3#()*+,,# 4"DD%576$-0/12;39756 T#(PO<>/* R3'#$"%#756'3'&6'9#75#1#6'U6#D7575=#151A$&7&#"E#$"%3#()*+,,#:3";'46# 4"DD%574167"5#('<=<#9'6'3D75'#D175#6":74&#D17A75=#A7&6* 04128:9<56 >>#(?Q<Q/* R3'#$"%#756'3'&6'9#75#&'567D'56#151A$&7&B#'&67D1675=#6@'#VD""9V#"E#'14@# D'&&1='#159#:3"27975=#1==3'=16'9#&6167&674&#E"3#6@'#I@"A'#:3";'46->.#(?><?/* 00128=9=56 !"#$"%#%&'#I'8W&'3274'&#A7S'#+@A"@B#C760%8#"3#"6@'3#&7D7A13#:A16E"3D&-0:12;<56 Q#(>Q<L/* X"6#4%33'56A$B# 8%6#J#@12'# 9"5'#&"#75#6@'# :1&6Y#?#(T<Z/* N"%A9#1#:'3&"51A7['9#91&@8"139#('<=<#4@""&75=#9161B#27&%1A7[167"5#E"3D#'64<*# 8'#D"3'#%&'E%A#6@15#6@'#:3";'46W"37'56'9#27'I#"EE'3'9#8$#+@A"@#159# C760%8-O#(PQ/* P#(L<?/* X''9#6"#63$#76# E73&6Y#22 (68.8%) J&#6@'3'#15$#&6167&674&#3'=13975=#$"%3#()*+,,#:3";'46#6@16#$"%#I"%A9#A7S'#6"# @12'-#)"3#'U1D:A'B#7E#$"%#@12'#%&'9#C760%8#"3#+@A"@#159#76#I1&#D7&&75=# &"D'6@75=#7D:"36156<

)3''#E"3D#15&I'3&

Fig. 2. OSS Developers Survey Results

most opinions of network graph representation of an OSS community were again positive with only two answerers mentioned that they would find it more interesting than useful.

Text Mining (TM) Communication presents not only a source for social network analysis. It also provides a great unplugged pool for TM. TM methods allow to determine end-user requirements, discover conflicts, etc. Special area of TM is sentiment analysis. The mood of a user can be implicitly estimated based on opinionated documents generated by the user (e.g. postings in MLs). However, the OSS developers were mostly uninterested in sentiment analysis. The negative reaction could be the result of little awareness of the sentiment analysis meaning. For example, one of the participants said that "Only slightly interested. I doubt that much useful information could be drawn from such an analysis, but I would need to know more about the methodology and findings to be sure, and it sounds interesting at least". Another participant expressed the concern that such analysis "Would be interesting/fun, but i m not sure whether its useful for me [...]". However, there were also participants that were clearly in favor of sentiment analysis: "Definitely. I would have stopped before entering some projects if I would have known about the mood swings of their contributors beforehand...". In contrast, other responses were clearly against it, for example "I don't think public statistics of the form "messages from developer X are mostly aggressive" would do anything good". The concern of feeling observed was already mentioned in the context of social analysis. Consequently, the developers are rather interested in the aggregated statistics. For example: "I think this [sentiment analysis] is only useful if combined with a certain segmentation of the user groups".

Majority of the participants were interested in TM analysis of the ML communication for other purposes:

-"[...] determine the needs of the users in addition to voting and tagging in bugtrackers" -"[...] creating FAQs for new contributors" -"[...] finding out in which direction the community wants to evolve" Missing Functionalities Finally, the following statistics were missed in the existing OSS visualizations:

-"[...] which wiki pages are consulted often, which problem appear in lists/forums frequently and so on." -"Some projects [...] allow Users to be credited in commits for there Testing or Bug reports (e.g. Reported-By: Tested-By:) including those contributions in statistics would by nice [...]" -"Last time on ohloh I wanted to see a simple list of contributions, but I only found timelines and such, which I find hard to browse" -"More informations about project activity. Most FOSS project have stalled development, are abandoned. This is for me the #1 FOSS problem" Dashboard A personalized dashboard was considered useful by 25% of the participants, while the majority of the participants (68.8%) replied that they would need to try it first. One participant mentioned that "The projects I contribute to have their own dashboards, I don't think an external dashboard would match my expectations". Indeed, many of the OSS hosting platforms offer their own built-in analysis and visualization.

Summarized, the OSS developers showed a strong interest in both social and text-based community communication analysis. To find out which statistical charts and designs were truly useful, there had to be an application that the OSS developers could try out.

Bioinformatics OSS Developers

Figure 3 displays a screenshot of the NOSE dashboard evaluated by bioinformatics OSS developers. It consists of five widgets: inflow vs. outflow of members in project MLs, number of commits, sentiment within community vs. commit activity, size of community core, social network graph of the project community. The last widget additionally provides several options: to search for a person, to select a yea or a release, and to highlight the core. The survey was sent to members of all three bioinformatics communities. We selected the participants who were active in the MLs in the last two years. In total, we sent an email to 46 project participants. From the 46 persons, nine replied to our email. The participants were also open for the discussion, thus, many issues got an extensive clarification. One of the BioPython developers even posted an article about the NOSE dashboard to his blog7 , which again supports the community interest in the self-monitoring and -reflection.

The proposed community metrics mainly received a positive feedback from the bioinformatics OSS developers. However, three bioinformatics developers reported that, they were not using the OSS visualizations. Nevertheless, one of them mentioned that the NOSE dashboard looked fairly attractive, but he was unsure what he would use it for. Another developer gave a longer explanation saying that "I don't find this type of community information particularly useful. Over time I've developed a habit of doing my own, informal, reputation scores in my head, based on people's list participation and tone, their code, their constructive criticisms, etc.... Who's talking to whom, etc... has never been particularly useful". Similar skepticism among OSS developers was previously reported in [START_REF] Gutwin | Group awareness in distributed software development[END_REF]. Despite some critical opinions, other evaluation participants were more in favor of the community statistics. One BioJava developer stated, that although BioJava project currently uses Ohloh for visualization, the NOSE dashboard could be complementary to existing visualization platforms. The developer added that these kinds of statistics would be useful for recruiting and funding. "For instance, we use these types of stats when applying for Google Summer of Code sponsorship".

The network graph received by far the most praise and interest. One developer commented that adding "[...] the social graph to the existing GitHub facilities would be valuable". Another reply was: "The social aspects of OSS projects are no less intriguing than the technological ones!". The developer additionally named two gains from such statistics. Firstly, that visualization platforms like the NOSE dashboard are great for getting an overview of the project's history. Secondly, that "[...] there is a lot to learn from this on how OSS projects get off the ground, what makes a successful project, etc".

Discovered Weaknesses

More Data Sources Many of the developers mentioned that additionally it would be nice to have data from GitHub. One developer expressed interest in comparing the social networks created based on GitHub and ML. GitHub is "a great place to discuss code specifics, so is often easier to go back and forth on than writing e-mails. It would be cool to see how the interactions there overlay on this". Another developer expressed interest in getting such communication statistics from the project LinkedIn8 group.

Network Graph Broadcasts were excluded from the network graph visualization. If the broadcasts were included, it would create an enormous amount of edges. That would make the rendering of a comprehensible network almost impossible. Additionally, it would create many hubs, thus lowering the presence of actual core developers. Further, one of the evaluation participants identified one core developer, who was split into two aliases. This splitting decreased his/her social role in the network graph. Nevertheless, bioinformatics developers believed the network graph captures the community quite accurately. There were many suggestions and requests. Most of the feature requests were directed at getting statistics from additional sources and not only from the ML. Some metrics requests were related to the social network graph:

-"[...] add a graph to the dashboard that shows, for each year who are the top linked nodes" -"[. The evaluation within three long-term bioinformatics OSS showed a strong interest of OSS developers in visualization of community statistics. Especially, the network graph visualization of the communities was recognized as the most interesting metric. The developers are more interested in aggregated statistics in order to avoid the feeling of being observed among the project participants. On contrary, sentiment analysis did not get much attention, which might be a result of a poor description or little awareness of the analysis method. However, some evaluation participants saw the NOSE platform more as fun, than as a useful evolution barometer. Further, the dashboard was suggested as a possible extension for the existing platforms and not as a standalone application. Our next steps are to realize the identified requirements. In terms of analysis metrics, the OSS members wish topic-based text mining [START_REF] Günnemann | An interactive system for visual analytics of dynamic topic models[END_REF] measures, with the goal to see where users struggle. Considering the data, there are many requests to extend the data sources, for example by the data from GitHub. Further studies with domains outside bioinformatics are needed to achieve truly generalizable results. Currently, we apply the concept of the NOSE dashboard to support and manage an OSS community around a EU project Learning Layers9 .

  Fig. 1. Study Workflow

Fig. 3 .

 3 Fig. 3. Screenshot of the NOSE Dashboard Prototype

1

  GitHub, https://github.com, last checked 2013/09/10 2 SourceForge, https://sourceforge.net, last checked 2013/09/10 3 Ohloh, www.ohloh.net, last checked 2013/09/10 [Boe81]. The next web front-end Melquiades 4 provides visualization for the data collected within FLOSSmetrics 5 research project [HIR + 09]

  ..] graph comparative metrics, such as consensus linkage, slope of the edgenumber histogram [...]". The developer suggested that it could be used for comparing the three bioinformatics projects. -"[...] use the graph's connecting edges to indicate the strength/weight of the the connection (i.e. line thickness to number of email conversations)" 5 Conclusions and Future Work In this paper, we addressed the research question: Are the OSS communities interested in a visualization platform, which reflects community evolution? If so, what aspects should it reflect? To answer this research question, we surveyed members from different OSS communities. Based on the survey results, we developed a dashboard prototype for community-oriented navigation in OSS projects.

Melquiades, melquiades.flossmetrics.org, last checked 2013/09/10

FLOSSmetrics, flossmetrics.org, last checked 2013/09/10

Open Source Report Card, osrc.dfm.io, last checked 2013/09/10

'The Bio* projects: a history in graphs", http://bytesizebio.net/2013/09/07/ bio-projects-a-history-in-graphs, last checked 2013/11/27

http://www.linkedin.com/groups/BioJava-58404?home=\&gid=58404\&trk= anet\_ug\_hm, last checked 12.09.2013

Learning Layers, http://learning-layers.eu, last checked 2013/11/10

Acknowledgement

This work is supported by the LAYERS FP7 ICT Integrated Project (grant agreement no. 318209) of the European Commission.


Amir Azarbakht
email: azarbaam@eecs.oregonstate.edu

Carlos Jensen
email: cjensen@eecs.oregonstate.edu

Drawing the Big Picture: Temporal Visualization of Dynamic Collaboration Graphs of OSS Software Forks

How can we understand FOSS collaboration better? Can social issues that emerge be identified and addressed as they happen? Can the community heal itself, become more transparent and inclusive, and promote diversity? We propose a technique to address these issues by quantitative analysis and temporal visualization of social dynamics in FOSS communities. We used social network analysis metrics to identify growth patterns and unhealthy dynamics; This gives the community a heads-up when they can still take action to ensure the sustainability of the project.

Introduction

Social networks are a ubiquitous part of our social lives, and the creation of online social communities has been a natural extension of this phenomena. Free/Open Source Software (FOSS) development efforts are prime examples of how community can be leveraged in software development, groups are formed around communities of interest, and depend on continued interest and involvement in order to stay alive [START_REF] Nyman | Understanding code forking in open source software[END_REF].

Though the bulk of collaboration and communication in FOSS communities occurs online and is publicly accessible, there are many open questions about the social dynamics in FOSS communities. Projects might go through a metamorphosis when faced with an influx of new developers or the involvement of an outside organization. Conflicts between developers' divergent visions about the future of the project might lead to forking of the project and dilution of the community. Forking, either as a violent split when there is a conflict or as a friendly divide when new features are experimentally added both affect the community [START_REF] Bezrukova | Violent Splits Or Healthy Divides? Coping With Injustice Through Faultlines[END_REF].

Most recent studies of FOSS communities have tended to suffer from an important limitation. They treat community as a static structure rather than a dynamic process. In this paper, we propose to use temporal social network analysis to study the evolution and social dynamics of FOSS communities. With these techniques we aim to identify measures associated with unhealthy group dynamics, e.g. a simmering conflict, as well as early indicators of major events in the lifespan of a community. One set of dynamics we are especially interested in, are those that lead FOSS projects to fork. We used the results of a study of forked FOSS projects by Robles and Gonzalez-Barahona [START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF] as the starting point for out study, and tried to gain a better understanding of the evolution of these communities.

This paper is organized as follows: We present related literature on online social communities. We then present the gap in the literature, and discuss why the issue needs to be addressed. After that, in methodology, we describe how gathering data, doing the analysis, and the visualization of the findings was carried out. At the end, we present results, discussion and threats to validity.

Related Work

The social structures of FOSS communities have been studied extensively. Researchers have studied the social structure and dynamics of team communications [START_REF] Bird | Latent social structure in open source projects[END_REF][10] [START_REF] Howison | FLOSSmole: A collaborative repository for FLOSS research data and analyses[END_REF], identifying knowledge brokers and associated activities [START_REF] Sowe | Identifying knowledge brokers that yield software engineering knowledge in OSS projects[END_REF], project sustainability [START_REF] Howison | Social dynamics of free and open source team communications[END_REF], forking [START_REF] Nyman | Forking: the invisible hand of sustainability in open source software[END_REF] [19], their topology [START_REF] Bird | Latent social structure in open source projects[END_REF], their demographic diversity [START_REF] Kunegis | Diversity dynamics in online networks[END_REF], gender differences in the process of joining them [START_REF] Kuechler | Gender Differences in Early Free and Open Source Software Joining Process[END_REF] and the role of the core team in their communities [START_REF] Torres | Analysis of the Core Team Role in Open Source Communities[END_REF], etc. All of these studies have tended to look at community as a static structure rather than a dynamic process. This makes it hard to determine cause and effect, or the exact impact of social changes.

The study of communities has grown in popularity in part thanks to advances in social network analysis. From the earliest works by Zachary [START_REF] Zachary | An information flow model for conflict and fission in small groups[END_REF] to the more recent works of Leskovec et al. [START_REF] Leskovec | Graphs over time: densification laws, shrinking diameters and possible explanations[END_REF] [START_REF] Leskovec | Statistical properties of community structure in large social and information networks[END_REF], there is a growing body of quantitative research on online communities. The earliest works on communities was done with a focus on information diffusion in a community [START_REF] Zachary | An information flow model for conflict and fission in small groups[END_REF]. Zachary investigated the fission of a community, the process of communities splitting into two or more parts. He found that fission could be predicted by applying the Ford-Fulkerson min-cut algorithm [START_REF] Ford | A simple algorithm for finding maximal network flows and an application to the Hitchcock problem[END_REF] on the group's communication graph; "the unequal flow of sentiments across the ties" and discriminatory sharing of information lead to "subcommunities with more internal stability than the community as a whole."

Community splits in FOSS are referred to as forks, and are relatively common. Forking is defined as "when a part of a development community (or a third party not related to the project) starts a completely independent line of development based on the source code basis of the project." Robles and Gonzalez-Barahona [START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF] identified 220 significant FOSS projects that have forked over the past 30 years, and compiled a comprehensive list of the dates and reasons for forking. They classified these into six main categories. (Table 3.) which we build on extensively. They identified a gap in the literature in case of "how the community moves when a fork occurs".

The dynamic behavior of a network and identifying key events was the aim of a study by Asur et al [START_REF] Asur | An event-based framework for characterizing the evolutionary behavior of interaction graphs[END_REF]. They studied three DBLP co-authorship networks and defined the evolution of these networks as following one of these paths: a) Continue, b) k-Merge, c) k-Split, d) Form, or e) Dissolve. They also defined four possible transformation events for individual members: 1) Appear, 2) Disappear, 3) Join, and 4) Leave. They compared groups extracted from consecutive snapshots, based on the size and overlap of every pair of groups. Then, they labeled groups with events, and used these identified events. The communication patterns of FOSS developers in a bug repository were examined by Howison et al. [START_REF] Howison | Social dynamics of free and open source team communications[END_REF]. They calculated out-degree centrality as their metric. Out-degree centrality measures the proportion of the number of times a node contacted other nodes (outgoing) over how many times it was contacted by other nodes (incoming). They calculated this centrality over time "in 90-day windows, moving the window forward 30 days at a time." They found that "while change at the center of FOSS projects is relatively uncommon," participation across the community is highly skewed, following a power-law distribution, where many participants appear for a short period of time, and a very small number of participants are at the center for long periods. Our approach is similar to theirs in how we form collaboration graphs and perform our temporal analysis. Our approach is different in terms of our project selection criteria, the metrics we examine, and our research questions.

The tension between diversity and homogeneity in a community was studied by Kunegis et al. [START_REF] Kunegis | Diversity dynamics in online networks[END_REF]. They defined five network statistics used to examine the evolution of large-scale networks over time. They found that except for the diameter, all other measures of diversity shrunk as the networks matured over their lifespan. Kunegis et al. [START_REF] Kunegis | Diversity dynamics in online networks[END_REF] argued that one possible reason could be that the community structure consolidates as projects mature.

Community dynamics was the focus of a recent study by Hannemann and Klamma [START_REF] Hannemann | Community Dynamics in Open Source Software Projects: Aging and Social Reshaping[END_REF] on three open source bioinformatics communities. They measured "age" of users, as starting from their first activity and found survival rates and two indicators for significant changes in the core of the community. They identified a survival rate pattern of 20-40-90%, meaning that only 20% of the newcomers survived after their first year, 40% of the survivors survived through the second year, and 90% of the remaining ones, survived over the next years. As for the change in the core, they suggested that a falling maximal betweenness in combination with an increasing network diameter as an indicator for a significant change in the core, e.g. retirement of a central person in the community. Our approach builds on top of their findings, and the evolution of betweenness centralities and network diameters for the projects in our study are depicted in the following sections.

To date, most studies on FOSS have only been carried out on a small number of projects, and using snapshots in time. To our knowledge, no study has been done of project forking that has taken into account the temporal dimension.

Methodology

We argue that the social interactions data reflects the changes the community goes through, and will be able to describe the context surrounding a forking event. Robles and Gonzalez-Barahona [START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF] classify forking into six classes, listed in Table 1, based on the motives for forking. The first three of the six motives listed are social, and so should arguably be reflected in the social interaction data. For example, if a fork occurs because of a desire for "more community-driven development", we would perhaps expect to see patterns in the data showing a strongly-connected core that is hard to penetrate for the rest of the community prior to the fork. In other words, the power stays in the hands of the same people over a long period of time while new people come and go. Our goal was to visualize and quantify how the community is structured, how it evolves, and the degree to which involvement changes over time. To this end, we picked projects from the aforementioned three categories of forked projects. This involved obtaining communication archives, creating the collaboration graphs, applying social network analysis (SNA) techniques to measure key metrics, and visualizing the evolving graphs. We did this in four phases as described in the following:

Phase 1: Data Collection

The study of forks by Robles and Gonzalez-Barahona [START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF] included information on 220 forks and their reasons. We applied three selection criteria to those projects. A project was short-listed if it was recent, i.e. the fork had happened after the year 2000; data was available; and their communities were of approximately the same size. This three stage filtering process resulted in the projects listed in Table 2.

Data collection involved analyzing mailing list archives. We collected data for the year in which the fork happened, as well as for three month before and three months after that year in order to capture the social context context at the time of the fork.

Phase 2: Creating Communication Graphs

Many social structures can be represented as graphs. The nodes represent actors/players and the edges represent the interaction between them. Such graphs can be a snapshot of a network -a static graph -or a changing network, also called a dynamic graph. In this phase, we processed the data to form a communication graph of the community. We were looking for how people interacted with each other. We decided to treat the general mailing list as a person, because the bulk of the communication was targeted at it, and most newcomers start by sending their questions to the general mailing list. Each communication effort was captured with a time-stamp. This allowed us to form a dynamic graph, in which the nodes would exist if and only if they had an interaction with another node during the period we were interested in.

Phase 3: Temporal Visualization and Temporal Evolution Analysis

In this phase, we wanted to analyze the changes that happen to the community over a given period of time, i.e. three months before and three months after the year in which the forking event happened. We measured betweenness centrality [START_REF] Brandes | A Faster Algorithm for Betweenness Centrality[END_REF] of the most significant nodes in the graph, and the graph diameter over time.

Figures 234show the betweenness centralities over the 1.5 year period for the Camailio, Amarok and Asterisk projects respectively. To do temporal analysis, we had two options; 1) look at snapshots of the network state over time, (e.g. to look at the network snapshots in every week, the same way that a video is composed of many consecutive frames), and 2) look at a period through a time window. We preferred the second approach, and looked through a time window of three months wide with 1.5 month overlaps. To create the visualizations, we used a 3 months time frame that progressed six days a frame. In this way, we had a relatively smooth transition. We visualized the dynamic network changes using Gephi [START_REF] Bastian | Gephi: an open source software for exploring and manipulating networks[END_REF]. The videos show how the community graph is structured, using a continuous force-directed linear-linear model, in which the nodes are positioned near or far from each other proportional to the graph distance between them. This results in a graph shape between between Früchterman & Rheingold's [START_REF] Fruchterman | Graph drawing by force-directed placement[END_REF] layout and Noack's LinLog [START_REF] Noack | Energy models for graph clustering[END_REF].

Results and Discussion

Kamailio Project

Figure 1 shows four key frames from the Kamailio project's social graph around the time of their fork (the events described here are easier to fully grasp by watching the video. A node's size in a proportional to the number of interactions the node (contributor) has had within the study period and the position and edges of the nodes change if they had interactions within the time window shown, with six day steps per frame. The 1 minute and 37 seconds video shows the life of the Kamailio project between October 2007, and March 2009. Nodes are colored based on the modularity of the network.

The community starts with the GeneralList as the the biggest node, and four larger core contributors and three lesser size core contributors. The big red-colored node's transitions are hard to miss, as this major contributor departs from the core to the periphery of the network (Video minute 1:02) and then leaves the community (Video minute 1:24) capturing either a conflict or retirement. This corresponds to the personal difference category of forking reasons. Figure 2 shows the betweenness centrality of the major contributors of Kamailio project over the same time period. The horizontal axis marks the dates, (each mark represents a 3-month time window with 1.5 months overlap). The vertical axis shows the percentage of the top betweenness centralities for each node. The saliency of the GeneralList -colored as light blue -is apparent due to to its continuous and dominant presence in the stacked area chart. The chart legend lists the contributors based on the color and in the same order of appearance on the chart starting from the bottom. One can easily see that around the "Aug. 15, 200815, -Nov. 15, 2008" tick mark on the horizontal axis, several contributors' betweenness centralities shrink to almost zero and disappear. This helps identify the date of fork with a month accuracy. The network diameter of the Kamailio project over the same time period is also shown in Figure 3. The increase in the network diameter during this period confirms the findings of Hannemann and Klamma [START_REF] Hannemann | Community Dynamics in Open Source Software Projects: Aging and Social Reshaping[END_REF].

This technique can be used to identify the people involved in conflict and the date the fork happened with a months accuracy, even if the rival project does not emerge immediately.

Amarok Project

The video for the Amarok project fork is available online1 , and the results from our quantitative analysis of the betweenness centralities and the network diameters are shown in Figure 3. The results show that the network diameter has not increased over the period of the fork, which shows a resilient network. The video shows the dynamic changes in the network structure, again typical of a healthy network, rather than of simmering conflict. These indicators show that Amarok fork in 2010 arguably belongs to the "addition of technical functionality" rationale for forking, as there are no visible social conflict.

Asterisk Project

The video for the Asterisk project is also available online, and the results from our quantitative analysis of the betweenness centralities and the network di-Fig. 3: Amarok project's top contributors' betweenness centralities and network diameter over time between Oct. 2009 to Mar. 2011 in 3-months time windows with 1.5 months overlaps ameters are shown in Figure 4. The results show that the network diameter remained steady at 6 throughout the period. The Asterisk community was by far the most crowded project, with 932 nodes and 4282 edges. The stacked area chart shows the distribution of centralities, where we see an 80%-20% distribution (, i.e. 80% or more of the activity is attributed to six major players, with the rest of the community accounting for only 20%). This is evident in the video representation as well, as the top-level structure of the network holds throughout the time period. The results from the visual and quantitative analysis links the Asterisk fork to the more community-driven category of forking reasons.

Conclusion

We studied the collaboration networks of three FOSS projects using a combination of temporal visualization and quantitative analysis. We based our study on two papers by Robles and Gonzalez-Barahona [START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF] and Hannemann and Klamma [START_REF] Hannemann | Community Dynamics in Open Source Software Projects: Aging and Social Reshaping[END_REF], and identified three projects that had forked in the recent past. We mined the collaboration data, formed dynamic collaboration graphs, and measured social network analysis metrics over an 18-month period time window.

We also visualized the dynamic graph (available online) and as stacked area charts over time. The visualizations and the quantitative results showed the differences among the projects in the three forking reasons of personal differences Fig. 4: Asterisk project's top contributors' betweenness centralities and network diameter over time between Oct. 2009 to Mar. 2011 in 3-months time windows with 1.5 months overlaps among the developer teams, technical differences (addition of new functionality) and more community-driven development. The personal differences representative project was identifiable, and so was the date it forked, with a month accuracy. The novelty of the approach was in applying the temporal analysis rather than static analysis, and in the temporal visualization of community structure. We showed that this approach shed light on the structure of these projects and reveal information that cannot be seen otherwise.

Fig. 1 :

 1 Fig. 1: Snapshots from video visualization of Kamailio's graph (Oct. 2007 -Mar. 2009) in which a core contributor (colored red) moves to the periphery and eventually departs the community.

Fig. 2 :

 2 Fig. 2: Kamailio top contributors' betweenness centralities and network diameter over time (Oct. 2007 to Mar. 2009) in 3-month time windows with 1.5-month overlaps

Table 1 :

 1 The main reasons for forking as classified by Robles and Gonzalez-Barahona[START_REF] Robles | A comprehensive study of software forks: Dates, reasons and outcomes[END_REF]

	Reason for forking	Example forks
	Technical (Addition of functionality)	Amarok & Clementine Player
	More community-driven development	Asterisk & Callweaver
	Differences among developer team	Kamailio & OpenSIPS
	Discontinuation of the original project	Apache web server
	Commercial strategy forks	LibreOffice & OpenOffice.org
	Legal issues	X.Org & XFree

Table 2 :

 2 Forked projects for which collaboration data was collected

	Projects	Reason for forking	Year
	Amarok & Clementine Player Technical (Addition of functionality)	2010
	Asterisk & Callweaver	More community-driven development	2007
	Kamailio & OpenSIPS	Differences among developer team	2008

Video visualizations available at http://eecs.oregonstate.edu/˜azarbaam/OSS2014/


Yuki Manabe

Daniel M German

Katsuro Inoue

Analyzing the relationship between the license of packages and their files in Free and Open Source Software

Introduction

One way of reducing software development cost is to reuse components. Today Free and Open Source Software (FOSS) has become a common and viable source of components that are ready to be reused. It is possible to find many components from open source project hosted by open source project hosting site such as SourceForge.net1 , Google Code2 and GitHub3 . In addition, users can use software component retrieval system such as SPARS-J [START_REF] Inoue | Ranking significance of software components based on use relations[END_REF] and Oholo code 4 .

The relationship between licenses is complex. It is not trivial to understand when a file with one license can be reused inside a package of another license. Some authors of licenses provide guidelines that try to clarify this; for example, the Free Software Foundation tries to clarify the relationship between the General Public License and other licenses [START_REF] Foundation | Various licenses and comments about them[END_REF]. However, there is a lack of empirical evidence that shows the relationship between the license of the package and the license of the files it contains.

In this paper we describe an empirical study that investigates how different software licenses are reused as white-box components in the software packages found in Fedora, a popular Linux distribution. We show the relationships between the licenses of packages, and the licenses of its files. Our goal is to assist developers, license compliance officers and lawyers in understanding how licenses are actually used.

The contributions of this paper are:

-An empirical study of how the relationship between the license of a package and the license of the files it contains (white-box reuse). -The definition of License Inclusion Graphs that show the licenses that are used inside a packages of a given license. -The license inclusion graphs from Fedora version 19 for the most popular licenses.

Background

The monetary cost of reusing a FOSS component can be zero, but it requires the user to read and accept its FOSS license. In general a software license is a set of permissions that the intellectual property owner grants to the user of the software after a set of conditions have been satisfied (these conditions could be, for example, the payment of a fee). A FOSS license is a software license where the permissions granted include the right to make derivative works of the software and further redistribute those works in exchange to the acceptance of certain conditions (see [START_REF] Rosen | Open Source Licensing: Software Freedom and Intellectual Property Law[END_REF] for a detail description of FOSS licenses, and [START_REF] German | License integration patterns: Addressing license mismatches in component-based development[END_REF] for a formalization of grants and conditions). For example, the original X11/MIT license grants permissions to "deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so"; the only condition it places to grant those rights is "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software." When developers built software by reusing FOSS, either by linking to them or by copying their source code, they have to satisfy the conditions of each of their licenses [START_REF] German | License integration patterns: Addressing license mismatches in component-based development[END_REF]. Fig. 1 illustrates this problem. In this scenario, the main source code of the project is developed under license A. It reuses two libraries by linking to them (reused as components-i.e. black-box reuse) each under license B and C. To complicate things further source code files from another component (under License D) has been copied into the project. The problem becomes, what license can be used to release the new software system that is compatible with the license of the source code (A) the copied source code (D) and the linked components (B and C). This is not always an easy to answer question, especially when the newly created project reuses many components. For example, Scacchi et. al. [START_REF] Scacchi | Understanding the role of licenses and evolution in open architecture software ecosystems[END_REF] examine Google Chrome and showed that it uses 27 components and libraries under 14 different licenses. Another problem is the proliferation of licenses. The Open Source Initiative5 , the body responsible for the definition of Open Source, has approved 69 licenses as open source [6] and BlackDuck claims that the Black Duck Knowledge Base includes data related to over 2200 licenses [START_REF] Black | Black duck knowledge base[END_REF] Previous research has studied license incompatibility. Alspaugh et. al. proposed a model of licensing terms represented by a tuple actor, modality, action, object, license and an approach to calculate conflicts of rights and obligations that they implemented on ArchStudio4, a software traceability tool. German et. al. shows integration pattern to solve conflicts of license [START_REF] German | License integration patterns: Addressing license mismatches in component-based development[END_REF]. However, solving license conflicts is not an easy task since it requires legal background. Also, most of the literature regarding license compatibility has focused on the Free Software Foundation licenses (the different versions of the General Public License -GPL-and the Lesser General Public License -LGPL).

In general, files are expected to contain licensing information in them, embedded in the comments of the file. This licensing information describes one or more licenses under which the file is "open-sourced". For this paper we analyzed as a corpus of FOSS the source packages of the Fedora Core 19 distribution. A FOSS system is usually represented in Fedora as a "source" package. A source package is a collection of files, including source code and documentation from where binaries are created. Fedora documents the license of every source package. We call this the declared license of the package. This information is intended to inform the user of the package of the legal obligations acquired when using it. This information is usually created manually by inspecting the documentation of the software [START_REF] Callaway | Fedora: Licensing guidelines[END_REF]. Fedora restricts what licenses packages can have in order to be included in the distribution, but this list is fairly comprehensive, currently including 253 different FOSS licenses [START_REF] Callaway | Fedora: Software licenses[END_REF].

Table 1 shows the most common license names and the abbreviation used in this paper. This paper uses a suffix v<number> as means to specify a version of that license; for example, GPLv2 means GPL version 2. If the license name is followed by "+" then the file can be used under such license or newer versions of it; for example GPLv2+ means the file can be used under the terms of the GPLv2 or the GPLv3 (including future versions of the license).

License inclusion

Any FOSS package is licensed under at least one FOSS license, and each of its source code files is expected to be licensed under a FOSS license too. Ideally every file should explicitly indicate its license, although it is not uncommon to find files without a license.

If a software package is of a given license, it does not mean that all the files that compose it are also of the same license. For example, it is widely accepted that files under the MIT/X11 license is included in software that is licensed under any other license, as long as the minimal requirements of MIT/X11 license are satisfied (see above). We call this relationship the inclusion of one license into another license. In this case, it means that the MIT/X11 license is included in packages under the GPLv2. Using the inclusion relationship, we can compute the licensing inclusion graph of a given collection of software packages as follows:

-Create a node for each of the licenses of packages.

-Create a node for each license found in files.

-For every file f in a package p, we create a directed edge license(f ) → license(p).

We then define the license inclusion graph of a package license as the subgraph that ends in a given package license. All the edges in the graph which share a single destination node and come from different nodes with the same license are merged into a single edge. We put the number of the merged edges as the weight of this new edge.

For example, assume that a package in GPLv3 includes 10 source files in BSD2. This relation is represented as a graph including two nodes "BSD2" and "GPLv3" and a single edge from "BSD2" to "GPLv3" with weight 10.

Empirical Study

We conducted an case study of FOSS software packages. Its goal is to answer the question What are the inclusion relationships between licenses of packages and licenses of source code?

Subject

For this study we used the 2484 source packages in Fedora Core 196 . Each package includes an archive of source files and one or more spec files. The spec files provide metadata of the source package including its license.

Methodology

To answer the research question we created the license inclusion graphs of package licenses found in Fedora, using the following procedure: First we extracted for each package its declared license, and the license of its files as follows:

1. We extracted its declared license from its spec file. 2. For each source code file in the package, we identified its license(s). For this step we used Ninka7 , a license identification tool with a reported accuracy of 93% [START_REF] German | A sentence-matching method for automatic license identification of source code files[END_REF]. Only 2,013 packages have at least one source code file. The median number of files per package is 60 files, average 748.5, and maximum 125,400.

Ninka is not able to identify the license of all files. Frequently this is because Ninka does not know the way a specific project licenses its files. In that case, Ninka reports "Unknown". In 62.2% of the packages, at least one file was "Unknown" (i.e. in 37.8% of the packages, Ninka identified the licenses of all the files). The distribution of the proportion of "Unknown" files had median of 2.7%, with a 3rd quartile of 25%. This meant that we had incomplete licensing information of packages with high ratio of "Unknown". For this reason we decided to remove from the study packages that had a proportion of 50% or more "Unknown" files (328 packages-16% of the total). Some packages contain more than one spec file. If the licenses of the spec files of a package were different, that meant that some files in the package were distributed under one license, and some under another. For that reason we removed packages with spec files with different licenses. This resulted in another 210 packages being removed from our study.

In total, we kept for our analysis 1,475 packages with at least one source code file, for total of different 511,308 files.

The abbreviation names of licenses in Fedora are different than the names used in Ninka. To make both lists comparable we created an equivalence table. An excerpt of this table is shown in Table 2. The complete table can be found in the replication package (available at http://http://www.dbms.cs. kumamoto-u.ac.jp/ ~y-manabe/replication_package/index.html/). Table 3 shows the most frequently found declared licenses in packages. Table 4 shows the number most frequently licenses in source files.

With this information we created the license inclusion graphs of the distribution, and from it the license inclusion graph of the most common licenses.

Results

Due to space limitations we only show the subgraphs of the most common licenses. The rest are available in the replication package. Figure 2 shows the file-to-package licensing subgraphs for the different versions of the General Public License. The number of package in each source file license node means how many packages have files under the license. The sum of the number of packages in source file is often larger than that in package license node because many packages include license not under same license but under various license. As it can be observed, there is no apparent inconsistency: all licenses of the files (the left node) can be relicensed as the license on the packages (the right node). What is apparent is that the GPLv3+ reuses fewer licenses. Figure 3 shows the subgraph for the most commonly found permissive licenses. The LesserGPLv2+ shows two main inconsistencies: the use of the GPLv2+ and the GPLv3+. We inspected the files that created this inconsistency, and in most cases, they were in directories that contained the name "test" or "demo", suggesting they were testing and sample programs (most of them very small). It is interesting that these packages would be under the LesserGPL, but the test and demo files under the GPL.

Fedora does not distinguish between the BSD2 and the BSD3, labelling both BSD, as shown in the diagram. In both the BSD and the X11/MIT, the proportion of files without a license is higher than other licenses (> 30%). The "GPL+ or Artistic" is a license that is used mainly for Perl and its modules. It is interesting that most of the files in this graph have no license, or the license "License Same as Perl". This is likely because the template for the creation of a module in Perl uses this license. Notice that in this license, as well as the two Apache, no other license is used. These communities (Perl and Apache) do not seem to reuse code under other licenses. Packages under Apache have the simplest graphs: most files (> 90%) contain the given license, and in the case of Apachev2, only 1.5% of files are under another license. For Apachev1.1 all its files are under the same license.

Limitations and threats to validity

With respect to the threat of internal validity, in this paper we didn't consider how source files were used. We focused on only relations between a source package and source files included in it. However, not all source files in a package in general, identifying the license of a package from its source code is not a trivial problem. Our work is an extension to theirs, as we use similar data. The difference is that we focused on the relationships between licenses and not packages.

Stewart et.al [START_REF] Stewart | Impacts of license choice and organizational sponsorship on user interest and development activity in open source software projects[END_REF] addressed the impact of licenses on software projects. Alspaugh et. al [START_REF] Alspaugh | Intellectual property rights requirements for heterogeneously-licensed systems[END_REF] have analyzed the requirements that licensing imposes on software. Scacchi et. al [START_REF] Scacchi | Understanding the role of licenses and evolution in open architecture software ecosystems[END_REF] have looked into the impact of licenses in the evolution of FOSS software.

Conclusions

In this paper we extracted the relationship between the licenses of packages and the licenses of the files are composed of in the Fedora Core 19 distribution. We visualize this information using license inclusion graphs. These graphs show that the different variants of the General Public License are more likely to include other licenses, while licenses such as the Apache tend to contain files only under the same license and they are better at stating the license of their files.

As future work, we would like to analyze the build systems of packages to determine which files are actually part of the binaries. We would like to also explore the licensing relationships between packages, and repeat this study in other collections of FOSS.

Fig. 1 .

 1 Fig.1. Example of the challenges of reuse of various licenses. In this example, the code has been developed under license C, but some of the source code was copied from a component with license D. Later this code was compiled and linked with components under license A and B. Can the resulting product be licensed under license A, B, C, D or other license?.

Fig. 3 .

 3 Fig. 3. License inclusion subgraphs of permissive licenses.

Table 1 .

 1 Names of common open source licenses and their abbreviations as used in this article.

	Abbrev.	Name
	Apache	Apache Public License
	BSD4	Original BSD, also known as BSD with 4 clauses
	BSD3	BSD4 minus advertisement clause
	BSD2	BSD3 minus endorsement clause
	CPL	Common Public License
	CDDLic	Common Development and Distribution License
	EPL	Eclipse Public License
	GPL	General Public License
	LibraryGPL	Library General Public License (also known as LGPL)
	LesserGPL	Lesser General Public License (successor of the Library
		GPL, also known as LGPL)
	X11/MIT	Original license of X11 released by the MIT

Table 2 .

 2 Excerpts from conversion table of license names between Fedora and NinkaDeclared license of package (Fedora Name) License in file (Ninka Name)

	ApacheSoftwareLicense	Apachev2
	ASLv2	Apachev2
	BSDwithadvertising	BSD4
	EPL	EPLv1
	LGPLv2	LibraryGPLv2
	LGPL2.1	LesserGPLv2.1
	PHP	phpLicV3.01

Table 3 .

 3 Number of packages under each license more than 10 packages)

	License name	Packages
	GPLv2+	338
	LGPLv2+	205
	X11mit	154
	GPL+ or Artistic	109
	BSD	91
	GPLv3+	63
	ASL 2.0	50
	GPLv2+ and GFDL	48
	GPLv2	44
	GPLv2+ and LGPLv2+	30
	LGPLv2	28
	LGPLv2+ and GPLv2+	19
	LGPLv3+	16
	EPL	16

Table 4 .

 4 Most common licenses used by source files.

	License Name	Src Files
	NONE	111311
	EPLv1	70004
	GPLv2+	48063
	Unknown	32874
	LibraryGPLv2+	32745
	GPLv3+	29041
	BSD3	25570
	Apachev2	22099
	LesserGPLv2.1+	21733
	BSD2	19844
	X11mit	15365
	GPLv2-classPathExcep	14509
	GPLv2	13958
	CPLv1	12060

sourceforge.net

code.google.com

github.com

ohloh.net

opensource.org

ftp:///ftp.iij.ad.jp/pub/linux/fedora/releases/19/Fedora/source/SRPMS/

ninka.turingmachine.org

Acknowledgements

This work is supported by Japan Society for the Promotion of Science, Grantin-Aid for Young Scientists (B) (No.24700029), Grant-in-Aid for Scientic Research (S) Collecting, Analyzing, and Evaluating Software Assets for Effective Reuse (No.25220003), and by an Osaka University International Collaboration Research Grant.

are used in building software. Therefore, we may extract the relations between packages and unused source files. We believe this effect is small.

Regarding construction validity, we used Ninka to identify the license of each source file. The quality of the data extracted depends heavily on Ninka's license detection quality. In our previous work [START_REF] German | A sentence-matching method for automatic license identification of source code files[END_REF] we reported that Ninka is accurate in 93% of files. We believe this is enough to represent the most common relationships between licenses. To identify the license of software packages we used the spec files created by the Fedora Project. Our study depends on the accuracy of this information. German et. al. [START_REF] German | Understanding and auditing the licensing of open source software distributions[END_REF] observed that this data is manually generated and an mostly correct. There were very few cases, however, where a package had upgraded to a new license, but the license in the spec file was not updated.

Regarding external validity, we only used source packages in Fedora19. Our results are affected by the selection bias of the Fedora Core Project. To make sure that licenses are comprehensively represented it is necessary to analyze other repositories of FOSS. We plan to do this for future work.

Related Work

German et.al [START_REF] German | Understanding and auditing the licensing of open source software distributions[END_REF]. studied license compatibility in software packages by identifying licenses of source packages and source files in them. They found that


Susana Sánchez Ortiz

Alfredo Pérez Benitez

FOSS service management and incidences

Keywords:

The Free Open Source Software (FOSS) solutions have been reaching a high demand, usage and global recognition, not only in the development of applications for companies and institutions also in the management of services and incidents. With the upswing of Information Technology (IT), the development of tools that enable the reporting of problems and incidents on any organization or company is necessary. Every day you need more applications, software generally, that make easier the user's actions. This paper describes the need to use these tools and recount the development of a web application that allows the management of reports and incidents from users of Nova, the GNU/Linux Cuban distribution.

Introduction

The services of Information Technology (IT) are more and more complex, which means that their management is most needed to remain efficient. In recent years IT have begun to rely on technological tools and Service Level Agreements(SLAs), which has allowed the development of a more efficient work, streamline their processes and transactions and have information for decision-making in real time.

In order to improve support and management in the area of information have been developed in the world the Systems and Service Management Incidents; technologies that manage incidents and routine requests for new services. These systems provide a set of interrelated components, which owe their existence to the needs of IT services and the human factor that has been increasingly demanding quality management. In Cuba, this subject has not been materialized on a large scale in all enterprises and institutions. The University of Informatics Sciences (UCI) has not been exempt of this situation, this affects their important change process, starring Free Open Source Software (FOSS) solutions. Because the importance of the migration process and the need to maintain with maximum efficiency the functioning of all the girded areas in the process, it has become necessary to have tools to support it.

In Cuba has been developed the GNU/Linux distribution Nova oriented to novice users who are facing a migration process from Microsoft Windows to GNU/Linux. Once the operating system provides its services in the country, it is impossible that their developers give support to all the questions raised by the users, considering that in many cases these concerns can be clarified or solved using new technologies.

In correspondence to the problem presented, the work will be focused on the presentation of the NovaDesk web application, its features and functionalities.

Development

Technologies of incidences and services management

In order to develop a system to manage the incidents that could present a user in its work with Nova was developed an investigation about the management of services and incidents and associated technologies. The Office of Government Commerce in the United Kingdom [START_REF]OGC ITIL v3-Estrategia del servicio. 1ª publicación[END_REF] defines the management of services as a set of specialized organizational capabilities which provide value to customers in the form of services. The services provide value to customers and facilitate achieve their objectives at less cost and less risk, because the responsibility is assumed by the contracted company [START_REF]OGC ITIL v3-Estrategia del servicio. 1ª publicación[END_REF]. Some of the technologies of services management and incidences are the Service Desk, Help Desk and Call Center. The Call Center can be defined as a place where telephone communications are concentrated in a company [START_REF] Team | Definición de Call Center[END_REF]. A Help Desk is a part of the technical support group established by an organization to keep their computers running efficiently [3].

The Service Desk have been perceived traditionally as a group of specialists who collect everything and who, hopefully, have the appropriate technical skills to answer practically any questions or complaints. As is represented in Information Technology Infrastructure Library (ITIL) [START_REF]IT Management Fundamentals -ITIL -What is ITIL?[END_REF], this discipline of Service Desk has evolved to the point that can be executed with a high degree of efficacy [START_REF]Service Desk[END_REF]. Differs from a Call Center or Help Desk in order that it have a larger scope and is more focused on the customer, because it is responsible to facilitate the integration of business processes on IT infrastructure [START_REF]Service Desk[END_REF]. One of the most important processes developed by the Service Desk is the management of incidents. For this reason the Service Desk should be supported by Help Desk technology that performs this type of management by excellence.

Development of NovaDesk

After completed the study of the technologies discussed above and considering that ITIL proposes the use of the Service Desk was decided that this would be the technology to be used. It was then performed an investigation of the open source tools that allow the services management and incidences. It was presented the problem of the low existence of open source applications based on this technology. So the system was initially developed in the Help Desk OneOrZero (v1.8). But the base technology was changed due to several setbacks among which were that the last version of this system (v2.5.1) did not allow it to be used freely.

Due to the amount of skills and to be containers of features that make possible the implementation of the processes and be able to satisfy greatly shortcomings identified in OOZ three options were selected: IRM Information Resource Manager, GLPI and ExoPHPDesk. It was made a comparison and concludes that the most suited to the needs expressed was GLPI.

GLPI is the Information Resource-Manager with an additional Administration-Interface. It has enhanced functions to make the daily life for the administrators easier, like a job-tracking-system with mail-notification and methods to build a database with basic information about your network-topology. GLPI is under the GPL license. The principal functionalities of the application are the precise inventory of all the technical resources and management and the history of the maintenance actions and the bound procedures [START_REF]GLPI -Gestionnaire libre de parc informatique[END_REF].

GLPI has a group of characteristics [START_REF]GLPI -Gestionnaire libre de parc informatique[END_REF] among which are: multi-entities and multiusers management; multiple authentication system (local, LDAP, AD) and multiple servers; permissions and profiles system; complex search module; export system in PDF, CSV, SLK, PNG and SVG; management of problems; tracking requests opened using web interface or email; SLA with escalation (customizable by entity); Assignment of interventions demands to the technicians; statistics reports by month, year, total in PNG, SVG or CSV; management of a basic system of knowledge hierarchical. Furthermore, this system has a large number of advantages among which include: reducing costs, optimizing resources, rigorous management of licenses, high quality, satisfying usability and security.

Functionalities of NovaDesk

Besides the features of GLPI, NovaDesk has a group of features among which are: chat room for technicians, which allows exchanging problems and solutions between them; email notification of monitoring incidents; improved attention through user's chat; edition of chat conversations that provide solutions for uploading to the knowledge base; statistical monitoring of time spent in chat; management reporting assignment; account registration, introducing obligatory and optional fields for the user; password recovery for registered users; user validation; display data from the hardware which users are working through a query of the OCS inventory database and management of the fields of incidence, allowing to adapt to the infrastructure of the organization.

NovaDesk also has an expert system that allows the management of its knowledge base. This system has several functions within which are: manage rules and questions of expert system; manage and identify the category of the problems; fill the knowledge base; identify problems; show answers and solutions of problems; send the solutions of problems by email and in the case in which the expert system does not identify the problem proceed to register as a new incident.

After implemented the expert system has been developed an analysis of their behavior that consisted of verification, validation and evaluation. This process allow to check that the behavior of the expert system is similar to the behavior of a human expert and because of this, the expert system can diagnose the problems presented by users in their work with Nova at the same way that would be performed by a human expert.

Importance of services management and incidences with NovaDesk

There are a large number of issue tracking system and bug tracking system like Bugzilla, Jira, Redmine, Mantis, WebIssues. When comparing NovaDesk with these tools, it demonstrates that all are at the same level. The development of NovaDesk has allowed the management of incidents presented by the users to interact with Nova. It has enabled this process more pleasant and accessible to the user, because it can clarify their doubts in different ways, either using the chat, the expert system or recording a new incident. It is also a tool that has been customized for this function and adapts to all companies and Cuban institutions that wish to face a process of migration to free and open source software. NovaDesk also uses the Apport [START_REF]Apport -Ubuntu Wiki[END_REF] service of the operating system and enables automatic report generation with the information provided, classifying this information on the category of the report and automatically assigning this incident to the group of developers of Nova.

Conclusions

NovaDesk provides great benefits to users of Nova, because it enables them to make reports related to deficiencies and disagreements. The purpose of this tool has been to optimize the system functionalities based on the registered incidents and give immediate response to the users concerns. The added features and the change of OOZ by GLPI are the basis for achieving the maximum expression of the processes proposed in ITIL, the implementation of a future Service Desk. The expert system constitutes a way for users to be able to solve the most common problems that can presented without the dependency of a human expert.


Andrea Janes
email: andrea.janes@unibz.it

Tadas Remencius
email: tadas.remencius@unibz.it

Alberto Sillitti
email: alberto.sillitti@unibz.it

Giancarlo Succi
email: giancarlo.succi@unibz.it

Towards Understanding of Structural Attributes of Web APIs Using Metrics Based on API Call Responses

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

More and more businesses are taking advantage of the so-called API Economy every day. Powered by the (open) web APIs, this new way of doing business [START_REF] Gat | API Economy: Playing the Devil's Advocate[END_REF] offers a number of exciting opportunities [START_REF] Gat | A Survey of the API Economy[END_REF], such as getting additional value of your company's business assets and fostering innovation from third-parties [START_REF] Clark | Selecting components in large cots repositories[END_REF] -all at a very low cost 1 .

The number of available web APIs has been increasing in a rapid manner. For example, the registry of open web APIs at programmableweb.com [4] shows close to exponential growth rate (Figure 1).

In principle, a web API can be implemented both as a proprietary and as an open-source software application, yet, conceptually, it represents a middleground between these two models. The code of the API or at least the source of the data/services it exposes is typically hidden from the API consumers, giving protection to the API provider and maintaining its ownership of the valuable business assets. The exposed API interface, on the other hand, becomes the open Fig. 1. Open web API growth rate, as extracted from programmableweb.com [4] pseudo-code that is generally available [START_REF] Gat | API Economy: Playing the Devil's Advocate[END_REF] to third-parties to use and extend as they please. For example, consumers can combine APIs from different providers to offer completely new products with very little effort required. It's a model that follows a win-win approach, enabling both providers and consumers to get benefits from the exposed API while maintaining a certain degree of separation and independence from each other. Success of one side means greater chances of success for the other. This makes the risk of one of the parties abusing the work of the other much lower.

As one could expect, however, the API Economy comes with its own set of challenges and difficulties [START_REF] Gat | A Survey of the API Economy[END_REF], [START_REF] Remencius | Tailoring ITIL for the Management of APIs[END_REF]. One of those is how to create a successful API. This includes being able to attract API consumers and providing the right level of help and support so that they remain satisfied. Bad interface design can make the API very difficult to learn and to use, thereby hindering its adoption. Failing to maintain backwards compatibility or introducing breaking changes can not only make existing consumers abandon the API but can also prevent arrival of new ones, because of the damage to the reputation of the provider.

From the technical (development) side, implementation of web APIs is not much different from implementation of other type of software solutions. In fact, because the code of the API is a black-box to consumers, the way how it is actually implemented is not that important -what matters is that it works according to current expectations.

The API interface, on the other hand, becomes the visible meta code that really matters. Its structure has direct impact on API complexity, maintainability (i.e., providing backwards-compatibility) and easiness to learn and use. Unfortunately, there are yet no established metrics that would allow to measure and understand the APIs based on their exposed interface. Our work is aimed to facilitate progress in this direction.

In the following sections, we describe a set of relatively simple metrics that could be used as a starting point in API analysis. While it is still too early to say how beneficial these metrics could be in practice, one can already get a sense of their potential applicability to different situations. The examples provided in the paper come from our analysis of some of the popular open web APIs 2 .

Web APIs

The term web API can be somewhat confusing to the old-school programmers who are used to the original notion of the API (i.e., as a library of functions, methods, and/or classes written for a specific programming language). In the context of the API Economy this term becomes more of a meta concept. The part that stays the same is that the API is still a collection of functionalities. However, it is no longer tied to a specific programming language and its purpose is to expose business assets as opposed to facilitation of reuse of the existing code. Here are the definitions that we use: Definition 1 (API Economy). An economy in which companies expose their (internal) business assets or services in the form of (web) APIs to third parties with the goal of unlocking additional business value.

Definition 2 (Web API). A software interface that exposes certain business assets to other parties (systems and/or people) over the web.

Definition 3 (API Call

). An HTTP request to the web API and a corresponding response.

Web APIs can generally be executed using a web browser and their output typically is in a human-readable form. In particular, REST [START_REF] Fielding | Architectural styles and the design of network-based software architectures[END_REF] and REST-like 3 APIs that use XML or JSON format are dominating the market (Table 1).

JSON and XML Based Responses

JSON is a data format that has been gaining a lot of popularity in recent years. One of the main reasons for its quick spread is that it is very easy to transform data from an XML structure to a JSON one and visa-versa. As a result, a number of web APIs support both output formats (Table 1).

JSON and XML enable representation of hierarchical semi-structured data. As such, every response to a request could be represented as a tree. An API as whole could then be seen as a tree composed of all call trees. This is the basis of our approach to the analysis of the API structure.

Our Approach

Our current strategy is to focus solely on the response part of API calls. The structure of call requests is generally much simpler and shorter and, therefore, has less impact on the resulting API. That said, however, we do have plans for incorporating request analysis in the future.

We also consider only the structure of the response and not the actual values contained within. Analyzing actual data could have its uses, in particular when it comes to testing, but again, we felt that it was important to focus on the structural aspects of the APIs first. This has a side benefit, as it means that data does not need to be present during the analysis, eliminating information security and privacy concerns.

The results of our ongoing research can be followed and freely accessed at our website [7]. We provide an open web API implementing analysis and metric computation [START_REF] Scotto | A relational approach to software metrics[END_REF][START_REF] Scotto | A noninvasive approach to product metrics collection[END_REF], as well as a web GUI interface that is built on top of that API. We encourage any interested readers to visit our site and contribute with feedback, ideas, and suggestions.

Measurement Framework

Our analysis of API structure is based on three directions: (1) computation of tree-based metrics, (2) computation of node-name-based (semantic) metrics, and (3) visualization of APIs and API calls.

Objectives

It is difficult to predict all possible uses for the metrics in advance. Nevertheless, we have several general objectives that serve as our guideline:

-Evaluate how difficult it is to learn and/or use the given API.

-Understand what the problems in the current design of the API are.

-Evaluate API consistency and backwards-compatibility.

-Understand how one API differs from another and what the impact of those differences is.

Metrics

As described in the previous section, at the moment we use only the structure of call responses as our analysis target. As such, when we analyze an API we include only those calls that have output in the form of XML or JSON. We also consider only the typical responses to calls, disregarding special cases, like errors and exceptions. The latter typically have a very different structure from normal responses and their inclusion into analysis is left for future research.

We do consider different call responses based on variations in request parameters, but only when there is a structural difference present (e.g., when different request parameters result in different type of data returned).

It is common for responses to contain sets of elements of the same type. The same request might result in a different number of items returned depending on the context (e.g., the user who makes the request, current amount of data, etc.). From the structural point of view we do not care how many items are returned as long as they all are of the same type. For this reason, every request is first sanitized by merging sibling nodes that have exactly the same structure (see Figures 2 and3). This means that no matter the amount of actual data returned, the resulting response representation stays the same. Fig. 2. JSON response and its visualization as a tree (black rectangles represent meta nodes). API: PayPal, API call: GET https://api.sandbox.paypal.com/v1/ payments/capture/8F148933LY9388354

Basic Tree Metrics

Every tree can be characterized by some simple metrics that reflect on its dimensions and/or shape.

One simple way to look at the size of a tree is to count the number-of-nodes (structural elements) it contains.

A related metric is the number-of-unique-nodes. Unique in this context means having a different node name from those that were counted before. This measure shows the richness of structure in terms of node variety. Used in conjunction with normal node count (e.g., as a ratio) it can be employed to identify Another way of looking at the size of a tree is to consider its node topology. We use three metrics for that purpose: tree-depth, tree-breadth, and number-ofchildren-per-node.

Tree-depth is the length of the path from the root of the tree to its furthest leaf.

Tree-breadth is the maximum number of nodes on one level of the tree. The level is the distance of a node from the root (i.e., the root node is at level 0, its child nodes at level 1, and so on).

When depicted visually, depth represents the height of the tree shape and breadth -the width.

The last metric we use is the number-of-children-per-node. It is computed as an average, though minimum and maximum values per tree might also offer valuable insights. This metric shows how much nodes tend to branch.

API Metrics

We employ two ways of computing API-wide metrics. The first one is based on the aggregation of metrics computed for each API call. This usually means computing average, minimum, and maximum values.

The second approach is to combine all call trees into one big API tree (see Figure 4 for an example of an API tree visualization) and compute the metrics defined in the previous subsection. This is performed in two steps:

1. We introduce a virtual root node and add all call trees as child nodes. 2. We merge sibling nodes that have the same structure. This way two or more calls might get combined into one subtree.

The merging step has two parameters that can be adjusted based on the desired effect. The first one determines whether to do the merge when a single element is compared to a list of elements of the same type as that element. Our default strategy is to execute the merge. This is because a single node can be seen as a list containing just one element (that node). In the situations when the multiplicity of nodes (i.e., one vs. many relationship) is considered to be important, such nodes can be left separate.

The second parameter is the merge factor. By default, we do the merge only when the structure of siblings matches exactly. However, this criteria can be relaxed using the similarity measure (described later in the paper). The motivation for merging similar but not exactly the same subtrees comes from the fact that some APIs have optional fields that are only returned if the appropriate values are present or if a specific parameter was added to the request.

Node-Name-Based (Semantic) Analysis

A different approach to analyzing API structure is to focus on the actual names of the nodes. These names are normally not given at random and represent semantics of the structure.

The simplest set of metrics of this type is the count-of-each-named-node. It can be computed for individual calls and for the whole API. The most frequent and the least frequent names are usually of most interest. Outlier values, in terms of name frequency, might indicate bad design or names that require special attention in the help manual or in the API reference.

A more advanced form of analysis is to consider which node names go together and how often. For example, one can count the occurrences of different node pairings when they appear as siblings (nodes that have the same parent node).

Similarity-Based Metrics

A known problem in the analysis of tree-based data is how to compare two trees. A common way of computing the similarity between trees is to use the so-called edit-based distance [START_REF] Bille | A survey on tree edit distance and related problems[END_REF]. It can be expressed as the minimum number of edit operations required in order to convert one tree to the other.

Such similarity measure can also be used for comparing API responses, but it has one major disadvantage, besides being complex and computationally intensive. It considers the number of required edits but does not consider their location (does not account for the significance of the hierarchy). In the case of the API response structure, we feel that the differences on the higher level of the tree (closer to the root) should have more impact. At the same time, we do not need a measure with the perfect precision. It is enough that we can tell exactly when two trees are the same, and to express their similarity in a somewhat intuitive manner in other cases. For this purpose we have devised the following similarity metric.

We consider that the similarity of two trees can be described by similarities among their subtrees, where each subtree has the same impact, independently from its size. So, if there are 5 subtrees across both trees, the impact of each would be 20%. However, we also want to account for the names of the root nodes, otherwise two trees that have the same root names but totally different subtrees would have a similarity of 0. There are different possibilities here in terms of how much weight we could give to the matching of such roots. It all depends on the point of view. It could be a fixed ratio, like 33% or 50%, but that would set the fixed minimum value for trees with matching roots. Therefore we feel that a weight related to the impact of one subtree is better fitting. At this point we simply use the weight of 1.0 -the same impact as that of a single subtree. When the root names do not match the trees are considered not similar and get the score of 0. The exception to this rule are the unnamed nodes from JSON -elements and lists ({} and []) -they are counted as matching but are not considered to impact similarity by themselves (impact weight of 0).

The detailed steps to compute the similarity are shown below:

1. Check if the names of the roots match or if both are meta nodes. If not, the similarity is 0. Otherwise proceed to the analysis of subtrees (step 2). 2. Take the tree that has fewer subtrees (T S ; L TS -the number of subtrees of T S ). For each subtree i (i = 1, ..., L TS ) of T S compute the similarity with subtrees of the other tree (T L ; L TL -the number of subtrees of T L ). Memorize the highest similarity S i (take the first one if multiple similarities match; stop the checks if a similarity of 1.0 -an exact match -is found) and remove the corresponding subtree of the tree T L from further computations, unless no similar subtrees are found (when maximum similarity S i was 0). In the latter case just memorize the value (0) and proceed to the next subtree. Once all subtrees from T S are processed, combine the memorized impacts (step 3). 3. If the names of the roots matched, compute the final similarity score thusly:

S =   1 + 2 • LT S i=0 S i   ÷ (1 + L TS + L TL ) . (1)
Otherwise (in the case of meta nodes), the roots are considered to have no impact:

S =   2 • LT S i=0 S i   ÷ (L TS + L TL ) . (2)
The resulting value of the similarity ranges from 0.0 (not similar) to 1.0 (exactly the same).

Besides using this metric directly to compare two API calls and as a criteria for merging sibling nodes (as described in the previous sections), we also use it to build the API similarity matrix (see Figure 5 for an example). API similarity matrix is the matrix containing similarities between every two calls of the API. Such matrix can be used to determine which calls could be grouped together (e.g., in the API manual, for better comprehension) or to verify if existing groupings are consistent. We call the average of all similarities from the matrix, computed excluding self-similarities (i.e., a call compared to itself), the similarity score of the API. It represents the general consistency of the response structure within that API (see Table 2 for examples). The similarity metric can also be used to analyze version evolution and the amount of structural changes (impact on backwards-compatibility) for an API call. It can also be applied for comparison of two different APIs (Figure 3).

Conclusion

Although the metrics presented here have been tested on a set of open web APIs and appear to highlight certain differences in the call response structure, their practical value has yet to be properly evaluated.

Nevertheless, we hope that this paper will give ideas and motivation to other researchers and practitioners, and help in establishing a deeper body of knowledge regarding web APIs, their design, and their evolution throughout the API lifecycle.

Fig. 3 .

 3 Fig. 3. Visualization of the call tree after merging matching subtrees (left), displayed with some of the call metrics (right). API: PayPal, API call: GET https://api. sandbox.paypal.com/v1/payments/capture/8F148933LY9388354

Fig. 4 .

 4 Fig. 4. Visualization of the API trees: left -Twitter API, right -Bing Maps API

Fig. 5 .

 5 Fig. 5. Color-coded similarity matrices for the Twillio (left) and Wikipedia (right) APIs. Colors have the following mapping to the similarity scores: red -0 (not at all similar), yellow -(0.0-0.5] (somewhat similar), green -(0.5, 1.0] (very similar)

Table 1 .

 1 Most used protocols and response formats 4

	Format	APIs	%	Protocol	APIs	%
	XML or JSON	8882	81.08%	REST-like	7771	70.94%
	XML	6047	55.20%	SOAP	2135	19.49%
	JSON	5176	47.25%	not specified	1086	9.91%
	XML and JSON	2341	21.37%	JavaScript	594	5.42%
	not specified	1843	16.82%
	Total APIs: 10955

2

Based on the list of popular APIs from programmableweb.com [4], http://www. programmableweb.com/apis/directory/1?sort=mashups 3 Following loose adherence to REST.

4

Based on data extracted from programmableweb.com

[4]

Table 2 .

 2 Examples of API similarity scores computed on different APIs

	API	Number of Calls	Similarity Score
	PayPal	19	0.2966
	Twillio	94	0.2263
	Twitter	78	0.1104
	NASA	9	0.3723
	Wikipedia	154	0.1726
	BingMaps	24	0.9041
	Salesforce	26	0.0644

Table 3 .

 3 Examples of similarity scores computed between pairs of APIs

		PayPal	Twitter	NASA	Twillio	Wikipedia	Salesforce
	PayPal	0.2966	0.0145	0	0.0003	0	0.0223
	Twitter	0.0145	0.1104	0.0035	0.0033	0.004	0.0102

Compared to normal effort and investment needed to enact new business initiatives, such as development of new products/features or starting marketing campaigns for reaching new markets.