Igor Steinmacher
email: igorfs@utfpr.edu.br

Marco A Graciotto

Marco A Gerosa
email: gerosa@ime.usp.br

Barriers faced by newcomers to open source projects: a systematic review

To remain sustainable, some open source projects need a constant influx of new volunteers, or newcomers. However, the newcomers face different kinds of problems when onboarding to a project. In this paper we present the results of a systematic literature review aiming at identifying the barriers that a newcomer can face when contributing to an Open Source Software project. We identified and analyzed 21 studies that evidence this kind of problem. As a result we provide a hierarchical model that relies on five categories of barriers: finding a way to start, social interactions, code issues, documentation problems and newcomers' knowledge. The most evidenced barriers are newcomers' previous technical skills, receiving response from community, centrality of social contacts, and finding the appropriate way to start contributing. This classification provides a baseline for further researches related to newcomers onboarding.

Introduction

Some open source software (OSS) communities composed of volunteers from different parts of the globe contributing and collaborating. According to Qureshi and Fang [START_REF] Qureshi | Socialization in open source software projects: A growth mixture modeling approach[END_REF], motivate, engage, and retain new developers is the way to promote a sustainable amount of developers in a project. However, newcomers often face difficulties and obstacles when onboarding to a project [START_REF] Dagenais | Moving into a new software project landscape[END_REF]. This obstacles can lead newcomers to give up their collaboration. Therefore, a major challenge for OSS projects is to provide ways to support the joining of newcomers.

To reduce these problems, newcomers generally post questions and request help to choose their tasks in forums and mailing list or send emails to developers who have central roles in the project (e.g. owners, project leaders) [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF][START_REF] Von Krogh | Community, joining, and specialization in open source software innovation: A case study[END_REF]. However, receiving replies that do not offer guidance or unpolished answers can result in newcomers to give up contributing [START_REF] Steinmacher | Why do newcomers abandon open source software projects?[END_REF]. Given this scenario, it is important to understand the OSS newcomers needs. This understanding may enable the creation of mechanisms and tools to offer a better support for newcomers.

The objective of this research is to identify the barriers faced by newcomers when onboarding to OSS projects. Onboarding is the stage in which an outsider decides to contribute to a project. Onboarding is highly impacted by a steep learning curve as well as reception and expectation breakdowns [START_REF] Steinmacher | Attracting, onboarding, and retaining newcomer developers in open source software projects[END_REF].

In this paper, the methodology chosen to collect these issues is the Systematic Literature Review. From the best of our knowledge, there is no study that directly focused on problems or barriers encountered by newcomers of Open Source Software projects. On the other hand, several articles report these barriers as a side product of the studies. Thus, knowledge is spread across the literature. This study main contribution is aggregating the barriers evidenced by different studies and creating a model with them.

Research Method

To perform our systematic review, we defined the following question: What are the barriers that influence newcomers' onboarding to OSS projects? By answering this question, we aim to capture the barriers that a newcomer can face when contributing to an OSS project. We are not interested in newcomers' motivation to join a project, but in the issues they can face after deciding to contribute.

After using different synonyms and combinations to refine our search, the query presented below was used to retrieve the studies from the following digital libraries: ACM, IEEE, Scopus and Springer Link. These libraries were selected because they index the most relevant venues of computer sciences, mostly written on English, they support searching using boolean expression and provide access to the complete text of the paper. We also consulted specialists for conferences, workshops, journals, and websites that could provide relevant studies for our research. However, no new source was added after their advices.

((OSS OR "Open Source" OR "Free Software" OR FLOSS OR FOSS) AND (newcomer OR "joining process" OR newbie OR "new developer" OR "new contributor" OR "new member" OR "new committer" OR novice OR beginner OR "potential participant" OR retention OR joiner OR onboarding))

For each selected paper obtained from digital libraries, we conducted snowball sampling checking if the authors of the selected studies published other relevant studies not retrieved from the Digital Libraries. We checked their profiles in ACM, IEEE, DBLP, and personal homepages (when available).

We considered for selection the papers that were available for download, written in English, that dealt with newcomers onboarding in open source software projects, that presented experimental results, and that were published in journals or workshop/conference proceedings.

Subsequent to the definition of the primary studies list, the researchers read the full documents. To classify the barriers we followed an "inductive coding" approach [START_REF] Thomas | A general inductive approach for analyzing qualitative evaluation data[END_REF], which is widely applied in qualitative studies of different knowledge areas. In this kind of approach, the evaluator identifies text segments that contain meaningful units and creates a label for a new category to which the text segment is assigned. Afterwards, connections between the codes are identified and they are grouped according to their properties to represent categories.

The results of the selection and screening are as follows. After running the query on the digital libraries systems, we got 291 candidate papers. For each paper, title, abstract and keywords were analyzed by two independent researchers. In a consensus meeting, we came to 33 candidate papers. We checked other papers published by the authors of these 33 candidate studies, finding 20 other candidate papers. After analyzing the abstract of these papers we selected 9 relevant papers, coming to a total of 42 candidate papers. After further analysis, a total of 21 papers were considered relevant to this review and were considered to extract relevant data.

Barriers faced by newcomers

The main purpose of this systematic review was to find what are the barriers faced by newcomers to open source projects reported by the literature. For each selected study, we analyzed any barrier reported that was empirically identified or evaluated. We extracted the barriers from the selected studies, and organized them as a hierarchy of barriers, as shown in Figure 1. The figure presents five categories: Social Interactions, Finding a Way to Start, Documentation Problems, Code Issues, and Newcomers' Knowledge. In the Figure 1 it is also possible to observe the number of studies that offer evidences for each barrier. The studies were conducted with different projects and different number of projects analyzed. In the following sections we will discuss the barriers found and the evidences that support the problem.

Social interactions

This category represents the barriers related to the way newcomers interact with the community, including who are the members they exchange messages, the size of their contact network, how they communicate and how the community communicate with them.

Socializing with project members

The study conducted by [START_REF] Ducheneaut | Socialization in an open source software community: A sociotechnical analysis[END_REF] highlights the influence of social and political organization for newcomers willing to become a core developer. The author analyzes mailing list discussions and conduct an in-depth analysis of the socialization of a successful developer. He emphasizes the need to build an identity in the project: "what the newcomer has to learn is how to participate and how to build an identity that will help get his ideas accepted and integrated." Other authors also report the importance of socializing with central members. For example, Bird [START_REF] Bird | Sociotechnical coordination and collaboration in open source software[END_REF] quantitatively analyzed mailing lists and report that "the social network measure, indegree, . . . had a significant effect on immigration."

All studies that analyzed the centrality/importance of the contacts found that the closer the newcomer is to the center of the community, the more successful the newcomer is. However, the newcomer usually does not choose who will answer her questions. So, when the most appropriate community members receive the newcomers, the chance of retention is higher.

Receiving (timely and proper) response

The answers received from the community play an important role during newcomers onboarding. There are evidences of this barrier in seven studies found in this review. Some of them [START_REF] Jensen | Joining free/open source software communities: An analysis of newbies' first interactions on project mailing lists[END_REF][START_REF] Zhou | What make long term contributors: Willingness and opportunity in oss community[END_REF][START_REF] Von Krogh | Community, joining, and specialization in open source software innovation: A case study[END_REF][START_REF] Steinmacher | Recommending mentors to software project newcomers[END_REF] report only the impact of receiving a (timely) response from the community as a barrier. Other researchers [START_REF] Singh | Newcomer integration and learning in technical support communities for open source software[END_REF][START_REF] Steinmacher | Why do newcomers abandon open source software projects?[END_REF][START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] also report the impact of the content of responses (properness).

One of the studies that analyze the impact of the answer contents found that "almost all non-returning newcomers can be attributed to not receiving a response or receiving a condescending response" [START_REF] Singh | Newcomer integration and learning in technical support communities for open source software[END_REF]. Regarding the studies that analyze the timely response, [START_REF] Jensen | Joining free/open source software communities: An analysis of newbies' first interactions on project mailing lists[END_REF] analyzed mailing list archives and found that "nearly 80% of newbie posts received replies, and that receiving timely responses, especially within 48 hours, was positively correlated with future participation."

We can see that community social skills can influence newcomers' decision to contribute to the project. Generally, newcomers demand attention and friendly hands to start contributing. We understand that core members need to stop their main tasks to receive newcomers with no guarantee that they will contribute. However, a good reception can be crucial to retain more newcomers.

Sending a correct/meaningful message [START_REF] Singh | Newcomer integration and learning in technical support communities for open source software[END_REF] reported a problem related to newcomers' communication behavior. By analyzing the history of a support forum they found that "the newcomers who used informative subject lines for their first message improved chances of getting responses as well as getting their problems solved by the community . . . if the newcomer does not post comprehensible messages or uses a language that the forum responders do not understand. . . ." Therefore, newcomers who want to be welcomed by the community should focus on the quality of their community-oriented initial interactions.

Finding mentorship/expertise Easiness to find an expert or a mentor is also evidenced in some studies [START_REF] Canfora | Who is going to mentor newcomers in open source projects[END_REF][START_REF] Cubranic | Hipikat: a project memory for software development[END_REF]. Cubranic et al. [START_REF] Cubranic | Hipikat: a project memory for software development[END_REF] report that "It can be difficult for newcomers to join such groups [OSS projects] because it is hard to obtain effective mentoring." To alleviate it, Canfora et al. [START_REF] Canfora | Who is going to mentor newcomers in open source projects[END_REF] proposed a tool that recommends mentors to newcomers. They evaluated the tool by surveying some project members and found that mentoring is important to newcomers.

Mentorship is presented as be a good way to help newcomers. However, its actual applicability need to be studied in deep. It is not clear if this kind of policy can be applied in OSS communities, as it depends on experienced volunteers to do this specific task.

Finding a Way to Start

This category represents the barriers related to difficulties that newcomers face when trying to find the right place to start contributing.

Finding appropriate issue/task Finding the appropriate task to work on was classified as a barrier. Park and Jensen [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF] reported that ". . . subjects expressed a need for information specific to newcomers, for instance, how to get involved and become active (e.g. communication channels, available sources of information for starters, etc.), what to contribute to (e.g. open issues, required features, sample tasks to start with), and working practices."

Von Krogh et al. [START_REF] Von Krogh | Community, joining, and specialization in open source software innovation: A case study[END_REF] also report on this issue. They found that 56.7% of the cases members of the community encouraged the new participants to find some part of the software architecture to work on that would match with their specialized knowledge. In only 16.7% of the cases new participants were both encouraged to join and given specific technical tasks to work on." This occurs because, according to their interviews, the community expects new participants to find their own task to work on instead of receiving a specific piece of work.

Communities point of view is that newcomers should be able to find the most appropriate task themselves, as reported by [START_REF] Von Krogh | Community, joining, and specialization in open source software innovation: A case study[END_REF]. However, other researches show that the projects should give special attention to this issue [START_REF] Ben | Mining developer contribution in open source software using visualization techniques[END_REF][START_REF] Capiluppi | From the cathedral to the bazaar: An empirical study of the lifecycle of volunteer community projects[END_REF][START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF].

Finding the correct artifacts to fix an issue When the newcomer find a task to work on, another issue can impact his contribution: how to find the correct artifacts. Cubranic et al. [START_REF] Cubranic | Hipikat: a project memory for software development[END_REF] proposed Hipikat, a tool that recommends artifacts that are relevant to a task that a newcomer is trying to perform. When conducting an experiment with Eclipse project, they found that "newcomers can use the information presented by Hipikat to achieve results comparable in quality and correctness to those of more experienced members of the team."

Newcomers really need support on finding code artifacts related to the chosen task, as projects' structure/architecture are not always trivial and straightforward. So, projects would benefit from tools like Hipikat to support newcomers first steps, as evidenced by the study conducted by Cubranic et al. [START_REF] Cubranic | Hipikat: a project memory for software development[END_REF].

Code Issues

This category comprises the barriers that are related to the source code of the products. To contribute a newcomer usually needs to change existing source code. Therefore, it is necessary for the newcomers to have enough knowledge about the code to start their contributions.

Dealing with Code Complexity/Instability Some studies focus on how code complexity can affect the newcomers to OSS projects. Studying Source-Forge projects Midha et al. [START_REF] Midha | Improving open source software maintenance[END_REF] show that "cognitive [code] complexity has a strong negative influence on the number of contributions from new developers." Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] highlight some complaints of newcomers about project structure/architecture of Open Source projects.

Understanding the Project structure/architecture. Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] highlight some complaints of newcomers about project structure of OSS projects. One subject reported that "the hierarchy of the source code directory was counter intuitive for someone with little architecting experience." Cubranic and Murphy [START_REF] Cubranic | Hipikat: recommending pertinent software development artifacts[END_REF] also present an issue faced during their experiment: "We also had reports of a pair missing a relevant suggestion because they lacked knowledge about the overall structure of the system..."

Park and Jensen [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF] analyzed "the potential benefits of information visualization in supporting newcomers through a controlled experiment." They reported that "providing visual information such as the class diagrams or dependency views . . . would help new developers understand the structure of existing code and find problems to work on."

The main complain regarding code is that its structure is hard to understand, and learning it would take too much time. The use of visualization [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF], or even artifact recommendation tools [START_REF] Cubranic | Hipikat: a project memory for software development[END_REF] can alleviate this problem.

Setting up Local Workspace

The feedback obtained by Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] evidenced that newcomers have difficulties when setting up their environment. They reported some obstacles, for example: "a challenge was that some [subjects] did not have any experience or knowledge on checking out source code from the version control system." To welcome newcomers, the communities should provide easy access to tutorials and step-by-step cookbooks on how to obtain the code, setup up and build a local workspace.

Documentation Problems

Project documentation was also explored by some studies. Newcomers need to learn technical and social aspects of the project to contribute. Thus, problems related to documentation were recurrently reported. [START_REF] Steinmacher | Recommending mentors to software project newcomers[END_REF] report some issues faced by newcomers regarding outdated information: "We can see many demotivating facts that occurred in this case: ... outdated information in the issue tracker made the developers waste time on an already existent feature and on checking each issue they pick to address..." Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] also report issues regarding outdated documentation. They report that the subjects "... were uncertain whether the available diagrams were still up to date and relevant for the current version of the software... Another reported challenge was the uncertainty whether the available documentation was up to date for the current version of the software." Finding outdated documentation can make the newcomer gave up onboarding. So, documentation provided by projects must be up-to-date enough to support new developers.

Outdated Documentation Steinmacher et al

Information overload [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF][START_REF] Cubranic | Hipikat: recommending pertinent software development artifacts[END_REF] conduct experiments to assess the benefits that tools that support dealing with information overload can bring to newcomers. Cubranic et al. [START_REF] Cubranic | Hipikat: recommending pertinent software development artifacts[END_REF][START_REF] Cubranic | Hipikat: a project memory for software development[END_REF] presents a tool called Hipikat that aims at recommending source code artifacts that should be related to the issue a newcomer is working on. Park and Jensen [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for open source newcomers[END_REF] evaluate the use visualization tools to alleviate problems with overload and report that "[the tools] provided more efficient ways to handle large amounts of data and understand dependencies in source code, reducing the learning curve and information overload experienced..." A rich documentation is essential for newcomers trying to understand the projects. However, just providing a bunch of documentation leads to information overload. So, the project should provide easy ways to find this documentation.

Code Comments not Clear In addition to outdated information and information overload, Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] report a problem related code comments. "It was reported that the code was not very well documented, which made it more difficult to understand the source code."

Newcomers' Previous Experience

This category comprises the barriers related to the experience of the newcomers regarding the project and the way they show this experience when joining the projects.

Lacking of Process and Practices

We found just one study presenting evidence of learning project practices as a barriers that can hinder the newcomers onboarding. The study conducted by Schilling et al. [START_REF] Schilling | Who will remain? an evaluation of actual person-job and person-team fit to predict developer retention in floss projects[END_REF] found that previous knowledge regarding the project practices influence newcomer first steps. They report that "familiarity with the coordination practices of the project team has a strong association with the time they spend on their projects after GSoC."

Lacking of Domain expertise Von Krogh et al. [START_REF] Von Krogh | Community, joining, and specialization in open source software innovation: A case study[END_REF] claim that " feature gifts by newcomers emerge from the newcomers prior domain knowledge and user experience." In the study conducted by Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF], the subjects "reported their unfamiliarity with the domain to be a hindrance." So, newcomers who present previous domain knowledge have more chances to have a successful onboarding and to be well received by the community.

Lacking of Technical expertise Schilling et al. [START_REF] Schilling | Who will remain? an evaluation of actual person-job and person-team fit to predict developer retention in floss projects[END_REF] reported that ". . . level of practical development experience is strongly associated with their continued permanence." Some studies report sending messages or patches to mailing list or issue tracker presenting previous technical skills can benefit the newcomer when joining. Stol et al. [START_REF] Stol | Identifying architectural patterns used in open source software: approaches and challenges[END_REF] evinced that "when newcomers mentioned that they had already tried some options to fix their problem and have put efforts to look for a solution in the forums . . . then the responders were quick to respond and were very helpful."

Ducheneaut [START_REF] Ducheneaut | Socialization in an open source software community: A sociotechnical analysis[END_REF] reports that "expertise is not enough to become a core member in Python: one also has to create material artifacts. . . " Bird et al. [START_REF] Bird | Sociotechnical coordination and collaboration in open source software[END_REF] also investigate the impact of sending patches when start the contribution and found that "demonstrated skill level via patch submission plays an important role in Python and Postgres." All studies evidence that the newcomer who wish to contribute must check if the technical skills required for a given task or project match with their skills. Newcomers can be proactive and search for the required background, but the project must also provide ways for a newcomer to search which tasks fit to his technical profile.

Summary

Considering the model defined in the Figure 1, based upon barriers identified by using inductive coding through out the selected studies, we can summarize the evidences for each barrier as shown in Table 1. The category more throughly studied is social interaction, accounting for 13 studies while the others range from 8 to 9 related studies each.

Due to the nature of the approach to establish the model, there is at least one paper associated with a barrier. Considering the most studied one, we found that the most evidenced barriers are newcomers' previous technical experience, from Previous Experience category, and aspects regarding social network characteristics and response reception, from Social Interaction category.

Threats to Validity

This review may have missed some papers that address barriers encountered by newcomers to OSS projects, since we did not search into every possible source and some relevant papers may not contain the chosen terms. To reduce bias, we contacted some specialists in OSS domain. We adjusted the criteria to cover all relevant papers that were of our knowledge and conducted pilot studies. Most part of the studies analyzed do not present as main focus analyzing the newcomers needs or the problems they face during their first steps. The papers that aim to analyze newcomers obstacles and problems focus on very specific problems. We know that it would be hard -or even impossible -to identify every problem that can affect newcomers. However, keeping the analysis to just some specific problems restricts the value of the outcomes and their applicability.

The findings of this review may have also been affected as the classification is a human process and it is based on some subjective criteria. In particular, the terms of the area do not have a common definition among all studies. The problems were classified based on inductive coding approach, which also relies on manual classification. To reduce interpretation bias related to human process, this review involved two researchers cross checking each paper for inclusion, and a third researcher responsible for reviewing and discussing the information generated after each step.

Conclusion

In this paper we identified 21 studies that evidence barriers that can hinder newcomers' onboarding in OSS projects. We aggregated the barriers evidenced across the literature in a single place. By using an inductive coding approach to organize the barriers, we proposed a model that relies on five categories: finding a way to start, social interactions, code issues, documentation problems and newcomers' knowledge. This model is the main contribution of this systematic review.

As a result of this classification we found that the most evidenced barriers are newcomers' lack of previous technical experience, receiving improper response from community, socializing with project members and finding the appropriate task/issue. This classification provides a baseline for further researches related to newcomers onboarding.

As future work we aim to conduct qualitative studies to confirm the barriers evidenced by the literature. We are conducting some interviews with OSS experienced developers and newcomers to verify what are the main barriers faced by newcomers from their perspective. We plan to refine the classification model based on the results of the interviews. Additionally, based on this model it is possible to propose strategies to offer a better support for newcomers, and study how these mechanisms can benefit newcomers.

Fig. 1 .

 1 Fig. 1. Hierarchical map of barriers found in the literature.

Table 1 .

 1 Studies that evidence each barrier

	Category	Barrier	Studies
	Finding a Way	Finding appropriate task/issue	[1, 5, 13, 22]
	to Start	Finding the correct artifacts to fix an issue	[6]
	Newcomers'	Lacking of Domain expertise	[22, 20]
	Previous	Lacking of Previous Technical Experience	[3, 2, 16, 24, 9, 22, 15]
	Knowledge	Lacking of Knowledge on processes and practices	[15]
		Dealing with code complexity/instability	[3, 12]
	Code Issues	Understanding architecture/code structure	[6, 13, 20]
		Setting up Local Workspace	[20]
	Documentation Problems	Outdated documentation Code comments not clear Information overload	[19, 20] [20] [6, 13, 20]
	Social Interactions	Socializing with project members Receiving (timely and proper) response Sending a correct/meaningful message	[3, 2, 10, 23, 24, 9, 14] [11, 16, 24, 22, 18, 20, 19] [16, 24]
		Finding Help -Mentor/Expert	[4, 6, 19]

Acknowledgements

The authors thank Fundacao Araucaria and CNPq (process 477831/2013-3) for the financial support. Marco Gerosa receives a grant from the CNPq and FAPESP. Igor Steinmacher receives grant from CAPES (BEX 2038-13-7).


Xavier Franch
email: franch|dameller@essi.upc.edu

Ron Kenett

Fabio Mancinelli
email: fabio.mancinelli@xwiki.com

Angelo Susi

David Ameller

Ron Ben-Jacob

Alberto Siena

A Layered Approach to Managing Risks in OSS Projects

Keywords: OSS, Open Source, Risk Management, Layered Model

In this paper, we propose a layered approach to managing risks in OSS projects. We define three layers: the first one for defining risk drivers by collecting and summarising available data from different data sources, including human-provided contextual information; the second layer, for converting these risk drivers into risk indicators; the third layer for assessing how these indicators impact the business of the adopting organisation. The contributions are: 1) the complexity of gathering data is isolated in one layer using appropriate techniques, 2) the context needed to interpret this data is provided by expert involvement evaluating risk scenarios and answering questionnaires in a second layer, 3) a pattern-based approach and risk reasoning techniques to link risks to business goals is proposed in the third layer.

Introduction

Translating dynamics of a complex system into focused management insights has been a challenge in various application domains [START_REF] Harel | Modeling Web Usability Diagnostics on the basis of Usage Statistics[END_REF]. In this paper we focus on organisations adopting, integrating and maintaining open source software (OSS) components in order to reduce time to market, introduce innovation and overcome development bottlenecks. Several companies have been observed to understand which are the main risks and risks indicators related to this OSS-related activities that are perceived by technical and business managers [START_REF] Li | A State-of-the-Practice Survey of Risk Management in Development with Off-the-Shelf Software Components[END_REF].

We propose a three layered approach: 1) the first layer focuses on collecting and summarising available data from different data sources, including human-provided contextual information; 2) the second layer, converts these data risk drivers into risk indicators [START_REF] Kenett | Process Improvement and CMMI for Systems and Software: Planning, Implementation, and Management[END_REF] and 3) the third layer assesses how these indicators impact into the business of the adopting organisation. Key methodological as well as theoretical questions need to be answered to derive risk related insights from measurable data as described in [START_REF] Ligaarden | ValidKI: A Method for Designing Key Indicators to Monitor the Fulfillment of Business Objectives[END_REF][5] such as, the indicators to define for measuring risk events, how to operationalise an indicator into one or more specific metrics for measurement and the predictive ability of measurements related to risks events needs to be validated.

A

Here we we are pr

La

In  Statistical analysis of data from OSS communities allows determining the trends and distributions of data.  Bayesian networks are used to link the community data gathered from the community data sources and the community risk metrics to the risk indicators and the community risk indicators using data generated by experts' assessment based on their experience in OSS adoption and community context.  The community measures can be also analysed via Social Network Analysis techniques in order to understand the structure and evolution of the OSS community.

All the risk indicators will contribute to the definition of a risk model. This model allows the representation of the possible causes of risks, basically the risk indicators, and of their connection to the possible risk events for the adopter organisation. Moreover, the model also allows representing the impact that the possible risk events have on the strategic and business goals of the organisation.

Layer 3: Business goals

Business goals describe which are the aims of the organization that adopts OSS. They are impacted by several kind of risks we summarise into four categories: 1) Strategic risks, mainly related to the company's strategy and plan, such as Pricing Pressure, failures in comply Regulation, Industry or sector downturn, or Partner issues; 2) Operational risks such as poor capacity management or cost overrun; 3) Financial risks such as assets lost, debts or accounting problems; 4) Hazard risks related to, for example, macroeconomic conditions or to political issues. Also in this case Bayesian networks may be used in order to link concepts from the two layers.

Modelling the layers

Business goals are included in models that represent the ecosystem that blends together communities, OSS adopting organizations and other key actors. The key relationships between these actors are represented through dependencies in goaloriented models expressed in the i* language [START_REF] Yu | Modelling strategic relationships for process reengineering[END_REF], which allow representing, and reasoning about, business goals and business processes. Reasoning is based upon different techniques, and in our layered context, we are particularly interested in bottom-up evaluation, since the leaves are directly linked to the risk model. A typical model will include the two fundamental actors of the OSS ecosystems, the Community and the Adopter, and how they depend on each other; some of their internal goals and activities, and their further AND/OR decompositions. We have then the risk model with the risk event (e.g., Risk of difficulty in code refinement) that "impacts" one of the activities of the Adopter (e.g., Bug Report) and that is propagated up to the higher level activities. The risk event is identified via the measurement and statistical analysis of the behaviour of the community and on the expert intervention that can rate the evidence of a risk indicator via the Bayesian Networks.

Conclusions and Future Work

The RISCOSS framework is designed to face with the problem of risk management in OSS related projects in a holistic way, allowing to pass smoothly from the dimension of the measures to those related to the decision-making in contexts where several technical and business constraints are present.

We believe that the approach can give an effective way of overcoming problems related the adoption phase. In particular, the huge volume and potential heterogeneity of the data is isolated into a layer collecting the available and potential new techniques suited for this problem; the correct interpretation in the context of the adopting organization is made also with the help of experts that can evaluate specific scenarios of risks; a pattern-based approach and risk reasoning techniques is proposed in the third layer that can help in linking risks to business goals.

Several points have to be addressed in the following years of the project. We plan to refine the approach clearly defining the boundaries of the layers and adding to each one of the layers the suitable techniques for data reasoning. An important point here is that of developing the approach in such a way to be adapted to the needs of the particular organisation that should be able to also feed in a contextual way the necessary data to effectively exploit the approach. Also we plan to integrate better our results to those coming from projects with related aims, as FLOSSMetrics (http://flossmetrics.org/), QualiPSO project (http://qualipso.org/), QualOSS (http://www.qualoss.eu/) and OSSMETER (http://www.ossmeter.eu/).

Acknowledgments

This work is a result of the RISCOSS project, funded by the EC 7th Framework Programme FP7/2007-2013 under the agreement number 318249.


Michel Evaristo

Febles Parker
email: mfparker@uci.cu

Yusleydi Fernández

Del Monte
email: ydelmonte@uci.cu

The agile management of development projects of software combining Scrum, Kanban and Expert consultation

Keywords: Scrum, Kanban, Agile management of projects

At the

Introduction

The Methods of agile software project management are guides for planning and control thereof. Currently several software development methodologies are focused on this style. The free software applications by the need to respond quickly to the constant changes in its requirements, technology and its short development period, are who most use them. In the Department of Operating System of the center CESOL, are developed open source software solutions using free tools and agile methodologies, organized in various development projects. Besides, the UCI is working with a view to improving the quality of his process of development up to Level 2 of CMMI. To check the correct execution of the model, audits and reviews are performed to the projects by the company specialists CALISOFT, the institution responsible for validating the quality of the process. ACF is a project that belong to the center CESOL, where a system to auditing the source code of software systems is made using an agile management and free software tools to develop. The present investigation shows the experiences gained during the combined application of the methods of agile project management, Kankan and Scrum, along with expert judgment method during the construction phase of the life cycle of the ACF project which was audited quality. Therefore the objective of this investigation is to show the experiences gained during the combined application of the agile methods project management, Scrum and Kankan, along with expert judgment, to achieve a pleasant management of a computer project.

Discussion

The agile management of projects is a management able to adapt and respond to new requirements and changes dictated by the environment [1] .Inside of this model are found the agile methodologies of development of software, that seek the early delivery of incremental software [2], among which are Extreme Programming (XP), Adaptive Software Development (ASD), Open Unified Process (OpenUp), Kanban and Scrum. Scrum is a process that apply regularly a set of best practices for working collaboratively. [3] Divide the team into small specialized groups managed by themselves, dividing the work into a list of small tasks or requirements for deliverables. Sort the list by priority and estimated the relative effort of each element [4]. Kanban is used to monitor the progress of work in the context of a production line and is currently used for agile project management, often with Scrum (known as Scrumban) [START_REF]  Palacios | Agile project management: basics concepts: conceptos básicos[END_REF].

CMMI is a model that is not focused on the principles of agile development. It is an adaptable guide to raise the quality level of the software development process of an entity. Particularly in the UCI there is a project called Programa de mejora, currently at version 3.4, in order to adapt CMMI to the different development centers that exist in it; lightening the documentation as possible, in order to fulfill model without making conflict with the environments of agile development. Particularly the case study is an example of a stage of the life cycle of the ACF project where was used the estimation method in view of solving the macro task "Fix no conformities identified during the audit quality", conformed by small subtasks.

How to combine both methods?

It starts with a set of tasks or requirements (the term is to taste) to perform. Then it proceeds to prioritize tasks using various criteria defined by the person responsible for managing the process. These may be important for the customer, the level of complexity, the amount of resources required for the implementation and the dependency among the requirements. The criteria should not be less than three. Each task is evaluated using these criteria according to a metric that can also be defined by the person who manage the application of the method, preferably [1-5], [1-10] and [START_REF]  Palacios | Agile project management: basics concepts: conceptos básicos[END_REF][6][7][8][9][10]. After evaluating each task, the values obtained for each criterion are added together and this is the value to use as a criterion for prioritizing tasks, sorting them in descending leaving those with highest numerical value as the first to be executed. In the event that the comparison test match, you can optionally choose the order that those tasks will have between them. Later proceeds to define the time duration of the tasks. To make this process intervals are defined from the values obtained as a result of the comparison test; intervals can also be defined optionally. These intervals are associated durations for tasks. In the interval where the criterion of value is within, the time corresponding to the interval is associated to the task. The times are defined using analogies of old tasks, expert consultation, experience and personal judgment.

Experiences in the combined use of both methods:

In June 2013 the ACF project was audited by specialists CALISOFT where a set of nonconformances that must be resolved in the shortest time as possible for the project were consistent with the quality model and resume its planning in the shortest possible time too. The initial group of tasks to be performed was as follows:

1-Perform document "Technical project", 2-Perform document "Project plan", 3-Perform document "Glossary of terms", 4-Perform document "Validation of the requirements", 5-Perform document "Plan of iteration", 6-Perform document "Work item list", 7-Perform document "Use case specification", 8-Perform document "Use case model, 9-Perform document "Specification of the requirements", 10-Perform document "Vision", 11-Perform document of architecture, 12-Perform document "Art state of the product to develop", 13-Perform document "List of risks" and 14-Perform document "Requirements of support".

The criteria defined for determining priority were complexity (task difficulty), size (effort needed to accomplish the task ), importance (importance to the project) and the interest (interest of the project team of to execute the task), being the metric used 1-5. The result of the prioritization was the following list of tasks: Tasks 1, 2, 3, 4, 5 and 6 with priority 20; task 7 with priority 18; tasks 8, 9,10,11 and 12 with priority 17; task 13 priority 13 and task 14 with priority 12. The estimation of time intervals defined initially were: the tasks with priority [20] will last 3 days, with priority will last 2 days and with priority [1-17] will last one day, which estimate a total of 29 days. Splitting the time between the number of workers on the project who is three, is obtained as a result approximately 9.7 days. To restrict the number of tasks was taken into account that two person on the team had the ability to perform two tasks simultaneously, for that reason was defined a working limit of 5 tasks for the columns Assigned, Developing and Reviewing. The second day, during a review of the remaining tasks was determined reassess the priority of task 9, leaving with a score of 20 therefore ascends to be the first task to execute. At the end of the third day of work, the tasks 2 and 10 were assigned and the tasks 9 and 8 upgraded to review. An active risk1 in the ACF project is the lack material resources and eventually in the stage in question was materialized, affecting the implementation of task 11. Therefore, as corrective action was determined to eliminate the task 11 of the board because at that time do not had the necessary resources to execute it. At the end of the sixth day of work the task 2 entered into development and tasks 1, 5, 6 and 2 into review when the seventh day finished, on both days the percentage of implementation was 71.4%. On the eighth day all tasks were completed for a 100% of execution.

Lessons Learned

It is possible reassess the task duration by increasing or decreasing their priority based on the new that arises in the development of software. The feedback determine that for this case, the future tasks that are similar to the first 6 and number 9, the duration should be 8 days, which represents over five days than estimated. For the tasks of 2 days, must add them one day and the tasks of 1 day behaved as estimated, allowing it to update the initial estimates.

It can be estimated and reassign tasks at the moment, allowing time to mitigate the risks that may exist. During process execution task 9 was prioritized again, flexibly changing the allocation of the task by a need of the development team. Following the realization of a risk associated with the project, task 11 was removed as a corrective measure.

In the prioritizing the tasks influence the characteristics of the team in terms of their skills and behaviors. The meetings of checking allow analysis of the performance of tasks and receive feedback from the experiences of the entire team to make adjustments to improve the planning, estimation and execution.

Allows to analyze in short term, the trends of the development team and take steps to improve.

Allows to know the speed of the work team and for inexperienced project leaders to estimate and to update that estimate at the same time that the project it is running.

It can work objectively by prioritizing tasks. The prioritization of the requirements allows to obtain a list of work focused on the key elements to achieve the project objectives; because each time a task is completed the final product evolves.

Conclusions

The investigation arrived at four conclusions. The first is that was showed the experiences gained during the joint implementation of Kanban and Scrum are detailed, along with the expert consultation to achieve a pleasant management in the project ACF. As a second conclusion, is that the combination of these methods allows a more precise estimate of the work, mainly for inexperienced project leaders and ordinary people. The third conclusion is that with his application the project can quickly reach his objectives that the product evolves in each review, that the planning be flexible and analyzes the existence of risks, his mitigation and that the entire work team participate in the management of the project. The fourth conclusion is that Scrum and Kanban complement themselves, by the characteristics of an agile environment there will always be changes in the requirements or the tasks during the development process, being necessary to insert them in that process. The board of Kanban by him selves, describes the workflow very well, join it with the prioritized list of SCRUM, the possibility of Kanban to modify the tasks without having to wait for the next iteration and a workflow guided by goals, demonstrate why is best to use them together.

Table 1 .

 1 View of the Kanban board at the end of the first day of work, it can see that the task 3 was completed on the first day.

	Task list	Assigned	In development	In review	Finished	% of execution
	8,9,10,11,12,13,14	2,5,6,7	1,4		3	7%

Table 2 .

 2 View of the Kanban board at the end of the second day of work.

	Task list	Assigned	In development	In review	Finished	% of execution
	10,11,12,13,14	2,9,8	1,5,6,7		3,4	14%

Table 3 .

 3 View of the Kanban board at the end of the fourth day of work.

	Task list	Assigned	In development	In review	Finished	% of execution
	11	2,12,13,14	1,5,6,7		3,4,9,8,10	35,7%

Table 4 .

 4 View of the Kanban board at the end of the fifth day of work

	Task list	Assigned	In development	In review	Finished	% of execution
		2	1,5,6		3,4,9,8,10,12,13,14,11,7	71,4%


Iftekhar Ahmed
email: ahmed@eecs.oregonstate.edu

Soroush Ghorashi
email: ghorashs@onid.oregonstate.edu

Carlos Jensen
email: cjensen@eecs.oregonstate.edu

An Exploration of Code Quality in FOSS Projects

Keywords: Code Quality, Success Metrics, FOSS, Open Source Software

It is a widely held belief that Free/Open Source Software (FOSS) development leads to the creation of software with the same, if not higher quality compared to that created using proprietary software development models. However there is little research on evaluating the quality of FOSS code, and the impact of project characteristics such as age, number of core developers, code-base size, etc. In this exploratory study, we examined 110 FOSS projects, measuring the quality of the code and architectural design using code smells. We found that, contrary to our expectations, the overall quality of the code is not affected by the size of the code base, but that it was negatively impacted by the growth of the number of code contributors. Our results also show that projects with more core developers don't necessarily have better code quality.

Introduction

Free/Open Source Software (FOSS) is associated with collaborative development model bringing developers together from different geographical locations to create cost effective and efficient software. The adoption of FOSS is growing; Walli et al. found that, out of the 512 U.S. companies surveyed, 87% of them used FOSS [START_REF] Walli | The Growth of Open Source Software in Organization[END_REF]. Another survey in 2007 also found that not only does FOSS have a significant market share, but that this development model in many cases produces the more reliable and highest performing software option [START_REF] Wheeler | Why Open Source Software/Free Software (OSS/FS,FOSS, or FLOSS)? Look at the Numbers![END_REF].

The quality of FOSS software has been subject to debate. Though some studies have argued for that FOSS projects can produce high-quality code [START_REF] Robbins | Adopting open source software engineering (OSSE) practices by adopting OSSE tools[END_REF][START_REF] Schmidt | Leveraging open-source communities to improve the quality & performance of open-source software[END_REF][START_REF] Del Bianco | An Investigation of the users' perception of OSS quality[END_REF], critics often point to the lack of formal project management practices and requirements as roadblocks to reliably achieving high software quality [START_REF] Scacchi | Understanding free/open source software development processes[END_REF]. That said, the FOSS community has developed its own methods of quality assurance and quality control [START_REF] Robbins | Adopting open source software engineering (OSSE) practices by adopting OSSE tools[END_REF], and researchers have shown that FOSS projects produce more secure code due to peer-review and the openness of the code [START_REF] Hoepman | Increased security through open source[END_REF]. Despite this controversy, there is a lack of large-scale empirical studies of objective code quality .

Evaluating the quality of software can be a difficult task because there are a large number of properties that could be evaluated [START_REF] Sommerville | Software Engineering[END_REF][START_REF] Wennergren | Clarifying Guidance Regarding Open Source Software (OSS)[END_REF], such as functionality, adherence to specifications, security, usability, etc. [START_REF] Jung | Measuring software product quality: A survey of ISO/IEC 9126[END_REF][START_REF] Del Bianco | An Investigation of the users' perception of OSS quality[END_REF]. It is not clear that these factors, even when combined, would give an adequate definition of quality. As a result many researchers have used different project characteristics as a substitute measure of the quality of a project, including longevity [START_REF] Golden | Making Open Source Ready for the Enterprise, The Open Source Maturity Model[END_REF], operational software characteristics [START_REF] Rating | Business readiness rating for open source[END_REF], number of open bugs [START_REF] De Groot | Call for quality: Open source software quality observation[END_REF], etc. None of these properties measure code quality directly (e.g. open bug reports is at best an indirect measure, as it is confounded by the size of the code base and the quality and extent of testing). One could argue that in the absence of requirements and objective definitions of quality, the more objective measure to focus on would be the quality of the code itself. We decided to examine the quality of FOSS source code using two types of code smells: implementation level code smells and design level code smells, two objective measures of code quality. A code smell is not an error or problem in itself, but rather a set of heuristics for identifying poor coding practices or code structure, often associated with poor maintainability, bugs, or inefficiencies [START_REF] Landis | The measurement of observer agreement for categorical data[END_REF]. This definition refocuses code quality to the maintainability and efficiency of the code.

We used existing code-smell detection tools to analyze implementation level code smells (focusing on violations of standard coding practices), and design level smells (focusing on higher-level issues associated with architectural decisions), and see if there were interactions between these and other project characteristics. In this paper we sought to answer the following questions:

• What impact do project characteristics such as longevity, size of code base and number of developers have on the low-level quality of the code? • What impact do project characteristics such as longevity, size of code base and number of developers have on the quality of design-level decisions? The outline of this paper is as follows. We review related work in FOSS development model. Next we describe our research methodology. Then we present our findings followed by the discussion, and finally outline future work and conclusion.

Background

Two of most important characteristics of the FOSS development model, for the purposes of this paper, are the distributed development process and the reliance on unpaid developers. These two factors can have an important effect on software quality because they lead to decentralization and difficulty compelling developers to take specific actions. That said, these characteristics are not indicative of a lack of quality; there are plenty of examples of outstanding FOSS projects, such as Linux, Apache, Gnome, Mozilla, etc.

Most FOSS research is focused on these successful and popular projects [START_REF] Mockus | Two case studies of open source software development: Apache and Mozilla[END_REF][START_REF] Koch | Effort, cooperation and coordination in an open source software project: GNOME[END_REF], while ignoring the many smaller FOSS projects that fail to reach this level of quality, popularity, or success. While a core reason for failure can be a lack of interest or volunteers [START_REF] Subramaniam | Determinants of open source software project success: A longitudinal study[END_REF], some suggest that lack of management is a major cause [START_REF] Senyard | How to have a successful free software project[END_REF].

One key to the success of FOSS is that source code availability allows faster evolution and higher quality because of parallel development and the theory that having more eyes on the code identifies more bugs quickly [START_REF] Raymond | The cathedral and the bazaar[END_REF]. Because of this, as Stamelos et al. [START_REF] Stamelos | Code quality analysis in open source software development[END_REF] found, mature FOSS code is generally of good quality.

Many metrics have been used to define the success of an information system (IS) such as a FOSS project. Delone and McLean's IS success model is perhaps the most cited [START_REF] Delone | The DeLone and McLean Model of Information Systems Success: A Ten-Year Update[END_REF][START_REF] Delone | Information Systems Success: The Quest for the Dependent Variable[END_REF][START_REF] Delone | Information Systems Success Revisited[END_REF]. In this model they identify six measures of success, with system quality being the most important measure for user satisfaction and adoption [START_REF] Lee | Measuring open source software success[END_REF].

Looking beyond FOSS, Boehm et al. [START_REF] Boehm | Quantitative evaluation of software quality[END_REF] and Gorton and Liu [START_REF] Gorton | Software Component Quality Assessment in Practice: Successes and Practical Impediments[END_REF] among many software engineering researchers, have explored different measures for software quality; including completeness, usability, testability, maintainability, reliability, efficiency and etc. that are all relevant to our analysis.

Code smells are symptoms of problems in the source code, and an indicator of where refactoring is needed [START_REF] Fowler | Refactoring: improving the design of existing code[END_REF]. This in turn has been associated with errors [START_REF] Li | An empirical study of the bad smells and class error probability in the post-release object-oriented system evolution[END_REF] and code maintainability problems [START_REF] Fowler | Refactoring: improving the design of existing code[END_REF]. Researchers have come up with different types of code smells depending on their type of impact [START_REF] Marticorena | Extending a taxonomy of bad code smells with metrics[END_REF]. The identification of code smells is typically done during development, testing, and maintenance.

Many approaches have been proposed for code smell detection, such as metric based [START_REF] Lanza | Object-oriented metrics in practice: using software metrics to characterize, evaluate, and improve the design of objectoriented systems[END_REF] and meta-model based [START_REF] Moha | Using FCA to suggest refactorings to correct design defects[END_REF]. Metric based measures show that code smells impact software quality [START_REF] Marinescu | Detecting design flaws via metrics in object-oriented systems[END_REF]. Most of the code smell detection tools are based on metrics analysis [START_REF] Lanza | Object-oriented metrics in practice: using software metrics to characterize, evaluate, and improve the design of objectoriented systems[END_REF]. This static analysis based approach has its drawbacks. Fowler and Beck claimed that "No set of metrics rivals informed human intuition" [START_REF] Fowler | Refactoring: improving the design of existing code[END_REF] when it comes to deciding whether an instance of a code smell should be refactored.

Methodology

Our goal was to analyze the impact of different project characteristics on the overall quality of code, measured using coding practice violations and design level code smells. We wanted to see if project characteristics such as longevity, number of core developers, the size of the developer community, the frequency of code changes, and size of the code base were correlated with the quality of the resulting code. This could shed valuable light on prevalent assumptions about the evolution of FOSS projects, and prescribed best practices.

We sought to perform an analysis of representative FOSS projects, but we also wanted to control for as many external factors as possible. One such factor was the programming language used. We decided to restrict our study to Java for two reasons: First, Java is one of the most popular languages (according to Github [12] and the Tiobe index [41]). Second, the number and robustness of Source Code Analysis (SCA) and code smell detection tools for Java compared to other languages.

For code smell detection we built on the work of Fontana et al [START_REF] Fontana | An experience report on using code smells detection tools[END_REF] and selected the InFusion [START_REF]InFusion[END_REF] toolset. To identify the standard coding practice violations, we used "Codepro Analytix" [START_REF]CodePro Analytix[END_REF] which uses a set of 643 rules collected from textbooks of java such as The Elements of Java Style [START_REF] Vermeulen | The Elements of Java (TM) Style[END_REF] and Effective java [START_REF] Bloch | Effective java[END_REF]. We used the 110 most popular Java projects hosted on Github [12]. Popularity was measured using the number of stars given to projects, a product of the number of users who liked the project and chose to get updates about it. Core developers in this paper are defined as the group of contributors that contribute major portions of the commits and also act as reviewers for patches submitted by others. Table 1 gives an overview of the key characteristics of the projects in our dataset. In the second phase we analyzed the projects to check if project characteristics contributed towards specific design issues. We categorized code smells into broader categories, as suggested by the code smell literature [START_REF] Marticorena | Extending a taxonomy of bad code smells with metrics[END_REF]. Our categories were: Bloater, Object oriented abusers, Coupler, Dispensables, Encapsulators and Others.

• Bloaters are smells that leads the code to balloon so that it cannot be effectively handled. This includes data clumps, large class, long method, long parameter list and primitive obsession [START_REF] Marticorena | Extending a taxonomy of bad code smells with metrics[END_REF]. • Object oriented abusers are smells that do not fully exploit the advantages of object-oriented design. Some of the smells include Switch statements, parallel inheritance hierarchies, and alternative classes with different interfaces [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. • The Coupler category contains smells related to high coupling between objects, in defiance of good object oriented design principles. Smells in this category include feature envy and inappropriate intimacy [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. • The Dispensable category contains smells such as the lazy class, data class and duplicate codes [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. • The Encapsulators category contains smells that deal with data communication or encapsulation, and includes message chain and middleman smells [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. • Others is an aptly named catch-all category. Two researchers independently performed this categorization. Inter-rater agreement was calculated using Cohen's Kappa. As the categorization was straightforward and there were only 6 categories, we achieved a Cohen-Kappa score of 0.90 after the first round of coding. According to Landis and Koch [START_REF] Landis | The measurement of observer agreement for categorical data[END_REF], Cohen-Kappa score greater than .85 indicates almost perfect agreement between coders.

Results

We calculated correlation coefficient between the number of low-level smells and the project characteristics mentioned earlier. None of these showed a Pearson Correlation Coefficient greater than 0.50, meaning that there was no strong linear correlation between any of the single properties and overall low-level code quality.

Next we used linear regression analysis with the intercept 𝛽 ! set to zero (absence of files or other variables results in zero code violations). Our model discarded Popularity and Number of commits as significant factors (see Table 2). To answer our second research question we checked for correlations between the number of design level smells and the project characteristics. Surprisingly we found that there was a strong liner relationship between the number of design level code smells and the number of core developers (Pearson Correlation Coefficient = 0. 73).

Next we used linear regression analysis with the intercept 𝛽 ! set to zero (absence of files or other variables results in zero code violations). Our model discarded LOC, Number of contributors and Popularity as significant factors (see Table 3). Figure 1 shows the mapping of design-level code smells into the six high-level categories discussed in our methodology section. We found that Couplers and Bloaters are the most common design-oriented code smells (solid bars), with 39.52% and 35.47% of all occurrences respectively. However, when we look at the percentage of projects that have at least one of these smells we see a much more even distribution (striped bars).

Discussion

The linear regression models that we found have R-squared values of 0.4735 and 0.5349 for low-level smells and design-oriented smells respectively. This means that a handful of project factors can account for roughly 50% of the variance in the sample. We also found that for the design smells, there was a strong correlation between the number of core developers and smells.

Looking at the data we see few interesting things standing out. For low level smells we were not surprised to see a between the size of the code base or the number of contributors on one hand, and the number of smells on the other. This was expected; the more complex the code and the more coders there are, the harder it is to curate the code effectively, thus allowing more code smells to manifest. We also expected file count and number of core developers to have a positive effect on code quality, as the first leads to better organization, and the second better curating and supervision.

What was somewhat surprising was to see that the age of the code-base had a negative effect on code quality. One would expect the code to improve over time, but it appears as if this is not necessarily the case. It is likely that the urge to add new features, or the turnover of programmers outweigh any refactoring and fine-tuning performed by the community.

For high-level, or design oriented smells, we found that file count, number of commits and number of core developers actually led to an increase in the number of smells. Most of these correlations make sense. While the number of files help manage the low-level complexities of the code, adding files makes it harder to maintain the high-level conceptual design. Likewise, while adding core developers help projects stay on top of low-level code reviews, but hinders the high-level design vision. Too many cooks do seem to spoil he broth.

Unexpectedly we found that age did decrease the number of design smells, which means that though age is associated with more low-level smells, it is also associated with fewer design smells. It may be that over time, refactoring is more often aimed at removing design flaws rather than low-level coding convention violations.

Turning to the last part of our analysis, we find support for our interpretation of the results; the code smell categorization gives us a deeper understanding of the underlying causes for code smells. Coupler was the most common code smell, and represents the smells that causes high coupling between objects [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. This might be an indication of a lack of adherence to object oriented design principles. We can hypothesize that, this type of ad hoc change happens due to the lack of knowledge about the design decisions of the project and due to low adoption rate of modeling and design tools in FOSS community [START_REF] Robbins | Adopting open source software engineering (OSSE) practices by adopting OSSE tools[END_REF].

Bloaters were the second most common code smell, and represent smells that lead the code to grow out of control, and is often caused by small changes and additions to the code [START_REF] Mäntylä | Bad smells in software-a taxonomy and an empirical study[END_REF]. Bloating is associated with centralized control structures using object-oriented languages, and Arisholm et al. identified that novice developers perform better with centralized control styles [START_REF] Arisholm | Evaluating the effect of a delegated versus centralized control style on the maintainability of object-oriented software[END_REF]. So it's most likely that novice developers turned contributors are pushing these changes. This potentially raises a question about quality assurance in FOSS.

In our analysis we have used the total number of coding standard violations and code smells as the indicator of FOSS projects quality. Identified regression models indicate that the number of core developers is correlated with quality of the FOSS project. This is in line with the findings of Sen et al., who found that number of developers reflect the "healthiness" of a FOSS project [START_REF] Sen | Open source software licenses: Strong-copyleft, non-copyleft, or somewhere in between?[END_REF]. Contrary to one of the findings of Sen et al. we didn't find any strong relationship between the popularity of a project and the quality of the project. This can be explained by the "lurking" phenomenon that is prevalent in online communities [START_REF] Nonnecke | Lurker Demographics: Counting the Silent[END_REF] up to 90% [START_REF] Mason | Issues in virtual ethnography[END_REF]. The "lurkers" are members of the community, who do not participate in any activity except watching, so it make sense that they have little impact on the actual quality of the code. Further empirical analysis is required to identify the actual contribution of lurking towards this observation to make any concluding remarks on this topic.

Limitations

It's always difficult to generalize a diverse movement such as FOSS. While we tried to ensure diversity amongst the projects we selected, there were limitations to our methodology and selection criteria. We list the most important here:

Our analysis was of 110 FOSS projects, which though a reasonable sample, is small compared to a population of 375,486 projects. Though we selected the projects based on popularity we did not consider the application domain of the selected projects. During the data collection, we also did not exclude folders that were not directly related to the functionality of the system itself (test folders, contribution folders, demo folders, etc.).

Conclusion

We were able to show that there is a correlation between a number of important factors such as the longevity, the number of developers, and code-base size on one side, and the number of low-level code smells on the other. This implies that as projects grows and ages, the quality of the code decreases, unless counteracted by a larger group of core contributors to curate submissions. However, we also found that increasing the number of core developers negatively affects the higher-level design of the code. It is therefore important to carefully balance the size of the core group.

Fig. 1 .

 1 Fig. 1. Percentage occurrence of code smell categories

Table 1 . Summary of the project characteristics Title

 1

		Min	Max	Median	Average	Stddev
	File count	480	462,846	22,811	51,270.0	74,354.5
	LOC	701	968,287	45,564	10,702.1	175,146.2
	# of commits	20	120,947	555	2,880.7	11,891.5
	Age (days)	5	5,485	910	1,086.0	905.8
	# of core developers	1	44	3	6.1	7.6
	# of contributors	1	225	25	43.7	47.2
	Popularity (stars)	695	7,252	1,011	1,376.5	987.8
	Total design code smells	0	4,981	28	231.7	645.3
	Total coding violations	18	4,894	921	1,074.1	908.9

Table 2 . Quality indicator of the linear model for

 2 measuring low-level code smells

	Title	Estimate	Std. Error	t value
	File count	--0.4401	0.1996	--2.205
	LOC	0.3233	0.1549	2.087
	# of core developers	--0.6321	0.2715	--2.328
	# of contributors	0.8408	0.2020	4.162
	Age (Days)	0.4563	0.1506	3.030
	*R--Squared	0.4735

Table 3 . Quality indicator of the linear model for measuring design level code smells

 3

	Title	Estimate	Std. Error	t value
	File count	0.14303	0.10169	1.406
	# of core developers	0.49993	0.08675	5.763
	# of commits	0.16490	0.7962	2.071
	Age (Days)	--0.13961	0.08079	--1.728
	# of lines deleted	0.13450	0.07667	1.754
	*R--Squared	0.5349

Acknowledgements

We would like to thank the Oregon State University HCI group for their input and feedback on the research.


Gladys Marsi

Peñalver Romero

Lisandra Isabel Leyva Samada

Abel Meneses Abad

Abel Meneses Abad

Flow Research SXP Agile Methodology for FOSS Projects

Keywords: methodology SXP, open-source, production, research, software. 1

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The University of Informatics Sciences (UCI) has seven faculties and several development centers which in turn are composed of productive projects. Each center specializes in a different line of development; one of them is the Free Software Center (CESOL) which is dedicated to the development of the GNU/Linux Cuban distribution and lead the migration process to open source applications in the country. The migration process of companies to open source application is one of the most important services offered, in addition to counseling, consulting, training and support, and to expedite the work of specialists have developed applications that automate many of the process governing the service provided, raising the quality and quick response customer service.

For these products is created the SXP agile methodology [START_REF] Romero | Agile Methodology for FOSS projects[END_REF], with the aim to develop projects with the greatest rapidity and quality expected by the end user. The great need for each project was implemented in less than one year, with frequent deliveries, with teams of no fewer than 10 members, where self-management and the ability of each of the members of the development team were some of the reasons why this methodology is used during the life cycle of these projects.

Although its application has fulfilled its main objective, did not include artifacts, and activities that define the research process that occurs when developing software [START_REF] Calderín | Method for controlling research tasks in the production of software in the UCI[END_REF], thus resulting in the need to integrate him a workflow process to collect research in the various projects, defining artifacts, activities and roles in order to obtain higher quality products, competitive, and could control and disseminate knowledge that occurs during software development.

Development

When starting a project already has how to carry out the development process, although at the beginning an entire previous research, which identifies the object of study, the current domestic and international situation is realized, which is framed in the first phase of any project; and it is the same theory that is followed in SXP.

Development methodology SXP

SXP is a Cuban hybrid agile, which is premised on avoiding duplication of efforts and customer integration into the development team which ensures no need for extensive documentation, and thus is well recorded which will be used in a future re-use. Behold the good practices of the XP and SCRUM agile methodologies, besides the quality guidelines defined by Calisoft, which is the entity responsible for monitoring the status of each of the projects that are developed in the development centers of the UCI and CMMi quality model. It is divided into four phases which form the basis of the structure of your project file [START_REF] Fernández | Proposal for a record, for FOSS productive projects[END_REF], these are: Planning, Definition, Development, Delivery and Maintenance. Each of these phases is made up of a number of activities which generate artifacts that documenting the process and guide the development of the products.

¿ Where is reflected the investigative work in SXP?

In the Planning-Definition phase is where the vision is established, the expectations are set and securing project financing is done. However not considered a researchoriented approach, because it is not spoken any time of writing the research project or research tasks that are thought to develop as a result of the production process, although this element should lead to torque software development process.

When starting the development cycle, where the implementation of the product is made, also carried hand research, which can be very intense according to the different technical aspects to be analyzed in order to define the most sensible when developing. A final product is obtained but nothing the study is documented, being within the knowledge of the researcher. Sometimes it happens that other projects need this information when developing, and not have them accessible, they should start their research from scratch, which impacts the product development time. So we can conclude that given the number of projects being developed, despite being guided by the SXP methodology, lose the opportunity to document the wealth of research that systematically develop them. This is knowledge that is generated and that in turn runs the risk of being lost with time, showing affected the scientific activity by this situation. By this reason it became necessary to develop a research stream with the inquiry process that takes place during the software production, allowing the control and disseminate the knowledge produced.

Flow Research with artifacts and role for SXP methodology

The methodology has a workflow that contains a number of artifacts that enables the control and management of research. This research stream will not be located in a specific phase, because its location will be chosen by the working group for each of the projects, although it is recommended that some of the artifacts begin the construction in early stage (Planning -Definition). Below are by each of the evidence gathering activity that takes place in the workflow: The Research Development Plan (IDP): artifact is a document that reflected the initial planning of the investigative activities to develop also should be done by the Research Manager, taking into account the characteristics defined for this role. The State of the Art artifact: is a deliverable that will be developed after a preliminary investigation, so it is proposed that has its beginnings in the Planning -Definition phase, which does not mean that can't come changes in the remaining phases. This artifact may involve different roles of the development team in its preparation should not only be developed by the Research Manager. The Research Report: artifact suggests that develops in the Development phase because that's where is getting the results of developed investigations. This artifact may involve different roles of the development team in its preparation should not only be developed by the Research Manager [START_REF] Samada | Research flow SXP[END_REF].

Role: Research Manager

Person responsible to manage all the research tasks developed. It is responsible for planning the development of the researches, verify compliance and quality of them. It is responsible for the preparation of PDI. Not necessarily must have computer knowledge, but some domain of Research Methodology.

Assessment of the proposal

This method is novel because there isn't a pervasive culture about the documentation on the research conducted during the software development, although some productive projects manage the research without defining roles, artifacts or take responsibility for the control and dissemination of the same. It must be emphasized that this research stream can be implemented not only by the SXP methodology, but may be included in any other software methodologies analyzed in this research considering the stage of development that is more convenient when incorporate it. The artifacts can be applied to any research task running on a productive project because it meets the adjustable parameters to their characteristics. With the implementation of this proposal fails to meet four key fundamentals: First, the problem of the organization of research in productive projects is solved, and second, the basis for publication are encouraged, in addition to the socialization of knowledge produced in software development, third, a favorable economic impact is obtained in projects, reusing the basis of research to accelerate the development of future products and finally, the training of human resources for software production is favored. Since 2008 the methodology has been applied in products like SistClon: System Image Cloning , applications operating system support for the Cuban Nova, NovaDesk and Service Desk for Nova, Xeiba: layout Manager for GNU / Linux , Summon Systems : Fitters distribution packages for Nova, Guano: lightweight desktop environment for Nova, Nova system Tools, Tools to configure the operating system and Serere Nova:

  Nova Installer distribution where developments have obtained very good quality developed in less than 8 months which have been tested with satisfactory results. Them much of the evidence that helped shape new projects that leave us publications and awards to date in the National Computer Competition with the presentation of the Migration Tool for Telematic Services Administration and the XV Convention and Fair took Havana International Computer Nova 2013 and in national events such as the Technological Peña Interior Ministry , the Twelfth Week FORDES Technological and international as the 1st Iberoamerican Congress on Project Engineering and Project Management IV International Congress where SXP methodology is presented .In general we can draw the following conclusions: The insertion of the research flow enables documenting the research tasks in the production of FOSS projects, it includes new artifacts and roles to software development methodology analyzed (SXP). The knowledge gained in studies for reuse in new developments is guaranteed, and that serve as a basis for specialists who are interested in such projects. Control the research tasks in productive projects, raising the engagement of specialists to exchange knowledge and scientific-technical work. It encourages collaborative work between developers and members of development teams, because centralizing research in a repository which can be accessed. With the inclusion of PDI, State of the Art and Research Report artifacts, is guaranteed the quality control and documentation of the investigative work.

	3	Conclusions


Igor Steinmacher
email: igorfs@utfpr.edu.br

Marco Aurélio Gerosa
email: gerosa@ime.usp.br

How to Support Newcomers Onboarding to Open Source Software Projects

While onboarding an open source software (OSS) project, contributors face many different barriers that hinder their contribution, leading in many cases to dropout. Many projects leverage the contribution of outsiders and the sustainability of the project relies on retaining some of these newcomers. In this research, we aim at understanding the barriers that hinder onboarding of newcomers to OSS projects, by means of different empirical approaches, and proposing a set of strategies that can be used to support the first step of newcomers.

Introduction

Open Source Software (OSS) communities are generally self-organized and dynamic, counting with contributions of volunteers spread all over the globe. These communities demand a high influx of newcomers to keep alive [START_REF] Park | Beyond pretty pictures: Examining the benefits of code visualization for Open Source newcomers[END_REF]. According to Qureshi and Fang [START_REF] Qureshi | Socialization in Open Source Software Projects: A Growth Mixture Modeling Approach[END_REF], motivate, engage and retain developers is a way to foster a sustainable amount of developers in a project. However, newcomers face difficulties and obstacles when they start interacting to the project, resulting in a high dropout rate [START_REF] Steinmacher | Why do newcomers abandon open source software projects?[END_REF]. Newcomers need to learn both social and technical aspects of the project, exploring mailing lists, wikis, issue trackers and source code repositories [START_REF] Scacchi | Understanding the requirements for developing open source software systems[END_REF].

In a previous study [START_REF] Steinmacher | Recommending mentors to software project newcomers[END_REF], some developers who tried to onboard in two well-known OSS projects reported the obstacles they faced. Developers indicated that the lack of awareness and guidance during the course of their first steps (setup and choosing the right mean to start) discouraged further contributions. Therefore, a major challenge for OSS projects is to provide ways to support the onboarding of newcomers.

The goal of this research is to explore the barriers faced by newcomers to onboard to OSS projects and analyzing which strategies can be used to support newcomers to overcome these barriers. The overall question to be answer in this thesis is: "How to support the onboarding of newcomers in OSS projects?" To guide answering this question, some specific research questions were defined for this thesis:

 What are the barriers faced by newcomers when onboarding OSS projects?  What are the forces that influence the joining and retention of developers in OSS projects?  What strategies can be used to help newcomers?

The research of this thesis uses different empirical methods to answer the research questions defined. This research is a result of the combination of systematic literature reviews, interviews, case study and experiments. The research design is composed of three phases and some studies, as presented in Figure 1, described below:  Warm up. This phase consists of one case study (S1) conducted in order to motivate the problem addressed by this thesis and to investigate whether absence of response, politeness, usefulness or the author of answers influence the onboarding of newcomers in an OSS project.  Phase I -Find Barriers. This phase is composed of three studies that objective to raise what are the barriers that hinder newcomers onboarding to OSS projects. We will empirically gather the barriers from the literature (S2), by means of Systematic Literature Review, and from the OSS community (S3), using interviews with project members and feedbacks from newcomers. The results obtained in the systematic review and in the qualitative analysis will be confirmed by means of a survey to be administered with OSS community members (S4).  Phase II -Proposing support to newcomer onboarding. In this phase, the goal is to consolidate the barriers that hinder newcomers onboarding to OSS and map them to proposed strategies that can support newcomers overcoming them. We plan to propose solutions based on: (i) suggestions and studies that emerged during Phase I; (ii) systematic review on awareness mechanisms (S5); and (iii) state-ofthe-practice, by analyzing the strategies that some projects currently adopt to support newcomers.

Based on the results of the survey conducted during Phase I, we will select a subset with the most relevant barriers to further explore in this phase. We plan to design and conduct a controlled experiment to assess the effectiveness of using the proposed solutions to the selected barriers (S6). In order to situate our problem and delimit the scope of this thesis, we analyzed the existent literature and proposed a model [START_REF] Steinmacher | Attracting, Onboarding, and Retaining Newcomer Developers in Open Source Software Projects[END_REF] that represents the stages (onboarding and contributing) that are common to and the forces that are influential to newcomers being drawn or pushed away from a project.

We conducted a case study [START_REF] Steinmacher | Why do newcomers abandon open source software projects?[END_REF] to analyze newcomers dropout reasons. We collected five years of communication of the list of emails from developers and discussions from the task manager (Jira). There was no indication that the number of responses influences the withdrawal. We found evidence that receiving inadequate answers affects the decision of newcomers to abandon the project

We also collected feedback from some developers that tried to contribute to two OSS projects [START_REF] Steinmacher | Recommending mentors to software project newcomers[END_REF]. The developers reported some demotivating facts: unanswered emails, outdated documentation, outdated issues that resulted in waste of time and flaming threads.

We conducted a systematic literature review aiming at identifying the barriers faced by newcomers to OSS projects. As a result, we provided a hierarchical model that relies on five categories: social interactions, newcomers' previous knowledge, finding a way to start, documentation problems, and code issues. Some barriers that hinder newcomers to OSS were also identified using a qualitative analysis on data obtained from newcomers and members of OSS projects. The results enabled us to create a model composed of 38 problems, grouped into seven different categories.

Figure 1 .

 1 Figure 1. Research Design

Acknowledgements

The authors thank CAPES (BEX 2038-13-7) and CNPq (process 477831/2013-3) for the financial support. Marco Gerosa receives a grant from the CNPq and FAPESP.