Hanna Mäenpää
email: hanna.maenpaa@helsinki.fi

Terhi Kilamo
email: terhi.kilamo@tut.fi

Tomi Männistö
email: tomi.mannisto@helsinki.fi

In-between Open and Closed -Drawing the Fine Line in Hybrid Communities

Keywords: crowdsourcing, customer community, open source software

Today, the community driven development model extends into a variety of new, often web based collaborations. Among these are hybrid open source development set ups in which various online tools are used to facilitate cooperation between virtual teams of commercial and voluntary stakeholders. As yet, how these relationships form and evolve is not understood extensively. This article presents a longitudinal case study of a smartphone startup that founded its early product development strategy on reliance on feedback from its customers through a web based question and answer forum. With this, the company managed to extend values typical for open source communities to support development of its proprietary software. Our main findings include that the challenge in similar settings lies in striking the right balance between the open and the proprietary -while overt openness may risk the competitive advantage of a company, leaving too much behind closed boundaries can create unnecessary friction in the relationship.

Introduction

The open innovation approach allows profit-oriented companies to outsource tasks that are essential for their production process to the general public, yielding results for significantly less expense than companies would otherwise be willing to invest [START_REF] Kleemann | Un(der)paid innovators: The commercial utilization of consumer work through crowdsourcing[END_REF][START_REF] Pénin | Crowdsourcing of inventive activities: definition and limits[END_REF][START_REF] Estellés-Arolas | Towards an integrated crowdsourcing definition[END_REF]. A desired audience for these collaborations are lead users who pioneer in identifying needs for technical solutions and are willing to bear some of the costs and risks of fulfilling them [START_REF] Hippel | Lead users: A source of novel product concepts[END_REF]. Therefore, collaborating with Open Source Software (OSS) communities and adopting their ways of working have been identified as promising growth strategies for software companies [START_REF] Henttonen | Managerial perspective on open source collaboration and networked innovation[END_REF]. These opportunities are especially highlighted at both early stages of product development [START_REF] Kim | Focusing the fuzzy frontend in new product development[END_REF] and when introducing new innovations to the marketplace [START_REF] Duarte | Separating the wheat from the chaff: a taxonomy of open innovation[END_REF][START_REF] Kleemann | Un(der)paid innovators: The commercial utilization of consumer work through crowdsourcing[END_REF][START_REF] Schenk | Towards a characterization of crowdsourcing practices[END_REF].

Understanding how these relationships in between companies and their 'unpaid employees' [START_REF] Kleemann | Un(der)paid innovators: The commercial utilization of consumer work through crowdsourcing[END_REF] can form and be managed has not yet received much attention from the scientific community. Future research is called for to understand reasons for entering these relationships [START_REF] Henttonen | Managerial perspective on open source collaboration and networked innovation[END_REF], as well as on principles, processes [START_REF] O' Mahony | The governance of open source initiatives: what does it mean to be community managed?[END_REF] and forms of control [START_REF] Tullio | The governance and control of open source software projects[END_REF] that take place in their governance.

The main contribution of our paper is to report how proprietary software development can be supported by an open innovation community for customers, describing how knowledge can act as both a reward and a detriment for motivation in this setting. In the next section, a theoretical background is laid for understanding the context of our study. Section 3 introduces the research approach. A description of the case company is provided in section 4, followed by results of the study in section 5. The work is concluded by discussing implications in sections 6 and 7.

Background

Open Source Software communities consist of individuals, many of whom participate voluntarily and out of their own inherent interests in the development activities. Here, decision power is distributed among people who belong to the same community and individuals are empowered to make decisions based on their personal merit [START_REF] Sharma | A framework for creating hybridopen source software communities[END_REF][START_REF] O' Mahony | The governance of open source initiatives: what does it mean to be community managed?[END_REF][START_REF] Antikainen | Motivating and supporting collaboration in open innovation[END_REF]. When commercial stakeholders enter this environment, their interests become blended with those of the voluntary contributors, creating a hybrid environment where a balance needs to be struck between the community based decision making and centralized governance that is typical for companies [START_REF] Benkler | Coase's penguin, or linux and the nature of the firm[END_REF][START_REF] Fitzgerald | The transformation of open source software[END_REF][START_REF] Sharma | A framework for creating hybridopen source software communities[END_REF].

In these collaborations, the commercial stakeholder must often be prepared to align its own strategy to fit that of the existing community -or to establish a new one [START_REF] Schenk | Towards a characterization of crowdsourcing practices[END_REF]. In the latter case, the amount of power vested in external collaborators can manifest in initial decisions about who gets to participate in the activities, how responsibilities are shared [START_REF] Midha | Governance practices and software maintenance: A study of open source projects[END_REF] and how rights to contribute to decisions are gained [START_REF] O' Mahony | The governance of open source initiatives: what does it mean to be community managed?[END_REF]. An apt division of knowledge, labor and decision power is needed to build trust and reciprocity between the stakeholders [START_REF] Tullio | The governance and control of open source software projects[END_REF][START_REF] Dahlander | How do firms make use of open source communities?[END_REF][START_REF] Sharma | A framework for creating hybridopen source software communities[END_REF]. This balance can be achieved through conscious decisions on how the community is built and governed and what socio-technical infrastructure it is to be supported by [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF].

When voluntary, non-affiliated individuals affect product development remarkably, an unclear legal relationship between a company and its contributors may emerge [START_REF] Wolfson | Look before you leap: Legal pitfalls of crowdsourcing[END_REF]. This can be managed by keeping the intellectual property rights of selected software assets in the company, creating a "gate" for the voluntary contributors [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF]. This limits their possibilities to not only modify and re-use the software, but also to know about and have influence on its development decisions [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF]. As the values of community driven development ideology become compromised, the motivation of contributors decrease as they feel their opinions are not being heard [START_REF] Shah | Understanding the nature of participation and coordination in open and gated source software development communities[END_REF][START_REF] O' Mahony | The governance of open source initiatives: what does it mean to be community managed?[END_REF]. In this context, encouraging a global voice of the customers on open platforms may prove harmful as users willingly share their bad experiences in public. Therefore it is important to have a strategy for curating the community created content and for evaluating users by their trustworthiness. Equally important to identifying malicious users is rewarding users that are influential and produce interactions that are of a high quality [START_REF] Doan | Crowdsourcing systems on the world-wide web[END_REF]. A visible commitment from top management is often required [START_REF] Dahlander | How do firms make use of open source communities?[END_REF][START_REF] Henttonen | Managerial perspective on open source collaboration and networked innovation[END_REF].

Research is called for to understand the reasons why companies choose to use OSS collaborations as the core of their value creation processes [START_REF] Henttonen | Managerial perspective on open source collaboration and networked innovation[END_REF], what principles and processes constitute community management in this environment (O'Mahony, 2007) and how these managerial practices can evolve on a longitudinal perspective (Di Tullio et al. 2013).

Research approach and questions

We employed a mixed method case study [START_REF] Runeson | Guidelines for conducting and reporting case study research in software engineering[END_REF][START_REF] Yin | Case study research: Design and methods[END_REF] on a longitudinal perspective from December 2013 to November 2015, focusing on how a company created and managed an online customer community1 . Our aim was to answer the following research questions: As the phenomenon in focus is inseparable from its context, a combination of different viewpoints, data sources and research methods were used to create a realistic representation [START_REF] Yin | Case study research: Design and methods[END_REF] . These are overviewed in the following subsections.

Viewpoint of the company

Our research started by characterizing the company and forming a general understanding of its crowdsourcing strategy. For this, freely available documents such as press releases, blog posts and social media interactions of managers were observed manually. This was continued untill the end of the research period.

In January 2014, we selected two of the company's employees based on their active role in the customer community and performed an in person, semi structured interview to understand their conscious aims in building the community. Questions included: "What is your role in your company's internal software development process? How do you use the knowledge from the online community in your work?". Follow-up interviews were performed in June 2014 later in April 2015 to review how the community created knowledge was currently influencing software development decisions, how the managerial practices had evolved and what concrete new forms the collaboration had taken throughout the months.

Viewpoint of customers

From early on, we recognized the need for including the customers' viewpoint in the study. To prepare for this, a semi structured telephone interview was performed with a voluntary community moderator (CM) in June 2014, seeking answers to questions such as: "What have been the success factors and pitfalls of building the community? Why do people choose to work for the benefit of the company without rewards? How do you see the role of the online platform in this collaboration?" Results highlighted the need for understanding motivation of individual community members and prepared us for choosing a theoretical alignment for addressing it [START_REF] Antikainen | Motivating and supporting collaboration in open innovation[END_REF]. We composed a survey with 12 Likert scale questions of motivational factors and two open ended questions about other reasons for participating. The survey was instrumented to the online community website on June 25th 2014, yielding 192 responses through the three week survey period. This study was repeated in November 2015 to review whether the motivation of the members had changed along time. The latter survey received 101 responses.

Actions of the stakeholders

In addition to self reporting by individuals, we wanted an objective representation of actions the community members had performed during the research period. For this aim, the community created content was first sampled in February 2014 with the help of a web crawler2 . A quantitative overview of meta keywords attached to the messages by community members was formed to distinguish the most common discussion topics. 25 examples of discussions were chosen and qualitative analysis of their textual content was performed to model their interaction sequences, two of which are illustrated in Figure 2. In addition to the community created content, we harvested all freely available user profile records to review the growth of the registered user base.

An identical data set was yielded later in November 2015 using a Software as a-Service tool tool 3 and a retrospective of the community's development and concrete actions its members had engaged in could be formed. The number of items in each data sample are presented in Table 3.3. Due to the space limitation, we are able to present only a summary of this work. . The company's collaboration strategies with the external OSS communities were various: work of the Linux community set the foundation of their product. They sought benefit from and contributed to the work of the Mer and Qt communities. For application development, Jolla chose to host an application store platform -trusting that an ecosystem of independent application developers would emerge among their first customers. Jolla's largest contribution to Sailfish OS were libraries that defined fundamentals of user experience, including hand gestures, user interface themes and basic applications for calling, messaging and managing contacts. For them, a proprietary licensing strategy was chosen, securing the company's competitive advantage of building an user experience "unlike" those of its competitors. Figure 1 presents the Sailfish OS software architecture, related licensing models and the collaboration strategies.

Sailfish OS was introduced as a minimally viable prototype to the customers and the smartphone was to be used as a toolkit for lead user innovation in early product development. For harvesting feedback, an online forum, together.jolla.com, was opened. Later in the text, we call this forum the online (customer) community or Together for short. Users of the forum we refer to as online community members or customers. The following chapters present our findings on how this relationship was initiated and what practices were used in guiding the customer community to support the company's internal proprietary software development process. Implications for similar settings are discussed later in chapter 6.

Results

While sections 5.1 and 5.2 address RQ2. Sections 5.3 and 5.4 seek to provide answers for RQ3. Based on these, RQ1 is later discussed in chapter 6.

Initiating the customer community

Jolla used minimal marketing efforts in initiating the customer community: new smartphone owners had received an invitation in their delivery box and management had sent Twitter messages [START_REF]Welcome message from the company to join the together.jolla.com online customer community[END_REF] welcoming people to join. The online community platform7 allowed asking questions, answering and commenting on them, attaching keywords and voting publicly on any piece of the content created. "The first evening we had hundreds of people logged in and contributing", explained the quality manager in January 2014. Jolla's value proposition was "We want to create an innovation platform for ideas, opportunities and openness." To highlight this, customers were engaged with aspects that were not commonly discussed in public such as packaging and manuals8 , brand values 9 , delivery strategies10 and future visions of the hardware 11 . Ideas were vividly shared, refined and ranked by voting. Section (A) of Figure 2 displays a typical idea harvesting interaction pattern. As the Askbot platform supported gathering personal "karma" community members were allowed to self-organize: moderators were elected via a process of self-representation and voting and best practices for content organization were agreed upon. In addition to the moderator status, employees of the company were marked with a logo alongside their member profile. Jolla's software quality manager (QM) established a visible and interactive role among the community members. "He did great work in focusing the discussion and solving conflicts constructively. [sic] He wanted to have both customers and employees have an equal voice in both asking and answering questions", a community moderator explained in April 2014.

Creating a focus

The online community acted as the first touchpoint for customers and defects quickly emerged as the most discussed topic. A majority of questions concerned elementary interactions: importing contacts, typing text, using email and browsing the web. The company had prepared for this -questions such as "Did you have trouble with Jolla not discovering GPRS/3G data settings?" had been submitted to wait for the customers' attention. This particular question briefly resulted in 29 professionally crafted test reports from 17 geographical locations around the world.

Most defects were submitted by the customers themselves and were followed by comments with rich detail of the testing process. As a defect had been described satisfyingly, community members created hypotheses about its origins and presented suggestions for solving it. Voting complemented these interactions by providing indicators on the urgency of the problem, feasibility of the solutions and quality of related comments. Figure 2 (B) presents a common defect reporting interaction. After six months, feedback from the customer community had become the main source of knowledge for the company's software product development decisions.

Sustaining motivation of contributors

As already in February 2014 the company's integration and release mechanism allowed a stable software update to be delivered monthly, discussing a bug and then seeing it fixed was gratifying for all involved. Members had become aware of the value of their contributions, demanded more say on prioritization 12 and information about the progress of the development 13 . "Customers are demanding in this regard. Sometimes we give a time window for releases but it is not enough. When the window is starting to get smaller, users ask: Hey Where is the update, What's happening? I want it now!" QM explained. Here, the software program manager (PM) came to be in a key role to control visibility to the progress. Status codes began to be marked by modifying headlines of the original defect report messages (Figure 3).

As pointed out by the forum moderator, the Askbot platform "had not served its purpose well". Rapid pace of interactions encumbered organization of the content and voting based community evaluation emphasized opinions of the most active and loudest members. "Even if crucial, a feature that is important to a niche user segment does not reach enough attention to be noticed by the company" he explained. At the same time, the extensive input from the customer community seemed to be overwhelming for the company. "In some cases it might not be feasible to implement even those (features) that the great majority advocates for" the quality manager explained. However cumbersome managing the in-and outflow of knowledge was, the company still ruled out giving visibility to their internal bug reporting and workflow coordination tools. For the proprietary licensing of the user experience components, community members perceived the subject of their contributions as a black box. While no external rewards or guarantees on uptake of their innovations were made by the company, the motivation of the participants became of interest. Our first survey was performed in June 2014 for 193 customer community members. Results revealed that a total of 89% were owners of the Jolla smartphone and 32% indicated an affiliation with OSS development. A prime motivator was to make a better product for themselves (92% agreed) and to receive (90%) and to give (77%) help in using the smartphone. While creating new ideas inspired the customers (77%), a majority of the respondents wanted to help the company directly to succeed (90%).

The second survey of 101 persons in November 2015 confirmed that the motivational factors had not changed in almost two years. The most active online community members had remained smartphone owners (83%) and the proportion of open source developers in the population was roughly one third. Contributors were still inspired by the product, the company's way of working and altruism. A noteworthy change was that neither recognition among peers (8% positive, 26% negative) nor showcasing expertise to the company (7% positive, 26% negative) were dominant among the community members. Willingness to help the company to succeed had decreased only somewhat (Strongly agree -19%, agree somewhat -12%). Still, as the company faced the need for steamlining its operations in November 2015, the loyalty of the customers manifested as initiatives for crowdfunding campaigns 14 to help the company.

When asked about the members' "other" and "ideological" reasons for participating, a clear majority reported being motivated by characteristics of the OSS ideology: freedom, openness and empowerment of individuals to control their own information and assets. As one respondent summarized: "I love the idea to exit from iOS, Windows and especially Android world: with Sailfish we're talking about real open source, freedom and the close contact between us users and the Company giving us the possibility to do something new, unlike and interesting.".

Evolution of practices

As the number of Sailfish OS devices grew, pre-release testing had gained emphasis and the company strengthened its internal process with a small, closed group of community members who were given release candidates before they were delivered to the customers. "This helps us to confirm the quality of our releases before they are released to the rest of our customers." explained the software program manager in March 2015. Later, the company had started an "early access" test community by providing any Sailfish OS user with an unreleased version of the OS after the C-Beta group had sufficiently tested it. "If no critical issues are found from in field, we release the update to the rest our end users." After the improvements, feedback from the web based question and answer forum started to have less impact on the company's short term planning compared to its early stages. "Decisions mostly rely on technical know-how and debts in software to stir the development in the right direction" explained the software program manager in March 2015, continuing: "Feedback and reports from Together still guide us to identify the most painful issues for our end users." Figure 4 displays the relationship of the described sub communities to the whole customer population.

Discussion

First, limitations of the study are described. Sections 6.2 and 6.3 provide answers to research questions RQ2 and RQ3, laying the foundation of understanding how the open community model can aid the development of proprietary software (RQ1) in similar settings. This question is discussed in Section 6.4 with implications.

Limitations

Several factors of both the phenomenon and our study design limit generalisation of the results. Our surveys, in spite of the high number of participants, represent at best only a fraction of the Together community's active members who on their part represent a limited subset of Jolla's customers (see Figure 4). Our sampling strategy is thus inclined to present characteristics of not a random set but a purposefully selected sample [START_REF] Yin | Case study research: Design and methods[END_REF] of those customers who are the most actively involved in the community. We claim that these people have been some of the most influential in the relationship and argue that the set represents that of early adopters (i.e. lead users) of the Sailfish OS technology. The history of the company as a continuum of Nokia's work may partly explain the community's loyalty and proneness to support values of OSS development. Therefore, the results can not be generalized to any product development context as is. However, we claim the results to be directional to situations where both the product in question relies on OSS software and at the same time a large personal investment on a hardware product has already been made.

Initiating the collaboration

Offering the smartphone as a pre-order campaign helped the company to test the viability of the product concept and size of its market. Customers made an initial financial commitment to the company and were set to expect an unpolished product. This, combined with an invitation to help build the product together with the company and low barrier of communication with employees created a promising start for the collaboration.

The OSS related history of the product resonated strongly with a small, yet very active group of lead users. However, using an independent, web based community platform that anybody could easily register to invited non-technical users to participate in the collaboration. While the platform supported merit based self-representation and autonomous decision making, users were given tools to organize and manage their own contributions. Even though the customer community was open for anybody to join, a between-the-lines prerequisite was the customership: discussions dealt almost solely with either using or not being able to use the product. This created an aim for contributors: to solve problems and help others in doing so. Personal achievement and helping others were altruistically celebrated. As the community managed its own contributions the company was relieved from curating the content. Voicing constructive criticism was allowed, increasing the quality of contributions while multiple viewpoints could be represented. Benefits for the crowdsourcing practise were indisputable. As at that time the company operated in only two cities in Finland, the extent of the testing activities could not have been achieved without the help of its customers.

Managing the collaboration

The company had chosen a gated source approach for the UX components of Sailfish OS, yet its strategy was to acquire feedback for developing them from customers from the customer community. This allowed hiding specifics of the software process from external contributors. Surprisingly, for the first years, no visible incentives were required to sustain the collaboration.

A large reward for contributors was the frequent and timely delivery of OS upgrades and thus, knowing when and what was to be shipped acted as a powerful incentive. While the company promised its customers openness and transparency, the "gate" created pressures for disclosing information. Maneuvering the situation required careful consideration on what, where, and to whom knowledge should be let out. Neither decision power, full transparency of the development process were ever offered as incentives -the company chose to offer more information about its internal development process through personal interactions with only those customers who were willing to contribute more to their development process.

Applying the community driven model in similar settings

Our main research question (RQ1) calls for understanding how an open community of contributors can support development of proprietary software. We described a specialized setting where a company founded its product on open source collaborations, searched a crowd for funding its development and harnessed the first customers for contributions that helped the company take its first steps for the first two years of its existence.

From this, we learned that in order to gain benefit for the proprietary development, the values of the company have to match those of the community. A sincere relationship between the stakeholders needs to be created and more interactive this relationship is, the richer results are to be expected. However, all interactions require time and effort from the company and to balance this, a sufficient degree of autonomy for the community in managing its own actions and contributions is required. From the online tools this requires support for merit-based self-representation, consensus decision making and general enough functionality that allows diverse types of interactions to emerge.

As a community matures, value of its work becomes established. Therefore, the need and means for rewarding voluntary contributors should be reconsidered from time to time. While several motivations are at play, reciprocity in the community membership experience is important, as well as sustaining the focus of the community's work on its original purpose. In our case, weaving all this together required the hosting company to have a strict policy of what knowledge to disclose, what issues to take a stand on and what not.

Conclusions

The interplay of community driven values and commercial interests flourish in hybrid OSS communities. To explore this phenomenon, we used interviews and surveys supported with publicly available data to follow the inception and management of an open, online customer community. Our main findings include that the challenge in similar settings lies in striking the right balance between the open and the proprietary -while overt openness may risk the competitive advantage of a company, leaving too much behind closed boundaries can create unnecessary friction in the relationship. Where the fine lines are drawn depends on how much knowledge, decision power and autonomy are vested in the community.

RQ1:RQ2:

  How can an open community aid the development of proprietary software? How can a company initiate an open community for collaboration in software development? RQ3: How can a company manage the collaboration with its open community?

Fig. 1 .Fig. 2 .

 12 Fig. 1. Licensing architecture of Sailfish OS and the company's strategies for collaborating with external communities. Adapted from https://sailfishos.org/about/

Fig. 3 .

 3 Fig. 3. Notification of progress in a message headline on together.jolla.com

Fig. 4 .

 4 Fig. 4. Roles of customers in supporting the company's internal software process

Table 1 :

 1 Data items in the customer community content data samples

	Sample	Febuary 2014	November 2015
	Number of messages	2 473	13 332
	Keywords in the content	1 151	10 577
	User profile records	2 716	8 449

http://together.jolla.com

Scrapy Python script

https://www.kernel.org/

http://merproject.org

http://qt.io

http://www.askbot.com

"What's missing from the user guide?"

"What does Jolla mean to you?"

"What hardware / devices would you like to see Sailfish OS on?"

"Poll: The Other Half of Your Dreams"

"Jolla should target business users"

http://www.n4s.fi/

http://www.tekes.fi/en/tekes/

Acknowledgments

The authors wish to thank Digile's Need4Speed program 15 funded by the Finnish Funding Agency for Innovation (Tekes 16 ) for its support for this research. We are also grateful to Sezin Yaman, Max Pagels, Sini Ruohomaa, Soumya Billal and Eric Le Roux.


Gregorio Robles

Laura Arjona
email: laura.arjona@upm.es

Jesús M González-Barahona

Santiago Dueñas Domínguez

Women in Free/Libre/Open Source Software: The situation in the 2010s

Keywords: Gender, FLOSS, free software, open source, survey

Women are underrepresented in the IT sector. But the situation in FLOSS (free, libre, open source software) development is really extreme in this respect: past publications and studies show a female participation of around 2% to 5% and have shed some light into this problem. In this paper, we give an update the state of knowledge to the current situation of gender in FLOSS, by analyzing the results of surveying more than 2,000 contributors to FLOSS projects in 2013, of which more than 200 were women. Our findings confirm that women enter the FLOSS community later than men, do primarily other tasks than coding, participate less if they have children, and have slightly different reasons to enter (and to stay in) the development communities they join. However, we also find evidence that women are joining FLOSS projects in higher numbers in recent years, and that the share of women devoting few hours per week to FLOSS and full-time dedication is higher than for men. All in all, comparing our results with the ones from the 2000s, the context of participation of women in FLOSS has not changed much.

Introduction, motivation and goals

The percentage of female participation in FLOSS projects is by all means very low (from 2% to 5% according to several surveys, such as FLOSS 2002 [START_REF] Rishab A Ghosh | Free/libre and open source software: Survey and study[END_REF]). Although the IT industry already shows a disparity in gender, the amount of women is estimated to be between 25% and 30% [START_REF] Hill | Why So Few? Women in Science[END_REF], in FLOSS projects this disparity is even larger. Some projects, such as GNOME, have been actively promoting participation of female collaborators by means of scholarships exclusively for women. In many projects, as for instance Debian, there are specific mailing lists to welcome new female contributors and discuss how to increase participation of women. However, even if this fact has been known for over a decade now, the situation changes at a slow rate.

The goal of this paper is to provide some insight on the current participation of women in FLOSS development communities, testing and extending findings from previous studies on female participation in FLOSS. For that, we have used data obtained by means of an open web survey, answered by FLOSS contributors. We have analyzed answers by men and women to different questions, mainly regarding their involvement with FLOSS, their educational level and background, and their personal status.

The structure of this paper is as follows: next, related research, in particular studies on the involvement of women in FLOSS, is presented. Section 3 contains the research questions. Then, we will introduce the methodology. Section 5 offers the results obtained for the research questions. Finally, some conclusions are drawn.

Related research

An ample research literature exists on gender issues in IT. In the context pertinent to this paper, many efforts have been devoted to better understand why the number of women in science, technology, engineering and mathematics (STEM) is so low [START_REF] Hill | Why So Few? Women in Science[END_REF]. It is interesting to note that this gender gap has not only been reported at the professional level, but as well on related activities such as accessing the Internet [START_REF] Bimber | Measuring the gender gap on the internet[END_REF]. In this regard, scholars have studied whether the problem of such a low share of women lies in a "leaky pipeline" or in a "gender filter" [START_REF] Clark | Women and science careers: leaky pipeline or gender filter?[END_REF], even to the point of asking students directly about this issue [START_REF] Miliszewska | The issue of gender equity in computer science-what students say[END_REF], and have theorized about it [START_REF] Trauth | Understanding the under representation of women in it: toward a theory of individual differences[END_REF].

Among STEM, computer science and programming are areas where male predominance is among the largest [START_REF] Margolis | Unlocking the clubhouse: Women in computing[END_REF], something which is not easy to explain. Some recent studies postulate that a wider perspective of the problem is needed, keeping into account "metaphors of programming, inclusion and exclusion, the notion of beautiful code, understandings of masculinity and programming" to "obtain a more complex analysis than a dualistic focus on differences between men and women" [START_REF] Boivie | Women, men and programming: Knowledge, metaphors and masculinity. Gender Issues in Learning and Working with Information Technology: Social Constructs and Cultural Contexts[END_REF].

Some studies focus on the area of FLOSS, where the amount of women is dramatically low [START_REF] Paul | Community-based production of open-source software: What do we know about the developers who participate?[END_REF]. For example, Adam has written about gender and the hacker phenomenon [START_REF] Adam | Hacking into hacking: Gender and the hacker phenomenon[END_REF], or Vasilescu et al. have studied StackOverflow, the largest Q&A site for programming and technical questions nowadays, to measure quantitatively online participation and representation of gender [START_REF] Vasilescu | Gender, representation and online participation: A quantitative study of stackoverflow[END_REF].

Other studies have studied female joining and participation patterns in FLOSS (and if they differ from the ones of males). So, Qui et al. have studied how women join and socialize in the KDE-women group [START_REF] Qiu | Joining and socialization in open source women's groups: An exploratory study of KDE-Women[END_REF], while Kuechler et al. found a disproportionate participation rate in several FLOSS projects under study [START_REF] Kuechler | Gender differences in early free and open source software joining process[END_REF].

Probably, the most significant effort on the gender issue on FLOSS is the integrated report of findings authored by Nafus, Leach and Krieger in the framework of the European funded FLOSSPOLS research project [START_REF] Nafus | Gender: Integrated report of findings[END_REF]. As one of the key findings, FLOSSPOLS states that "[w]omen are actively (if unconsciously) excluded rather than passively disinterested" and offers some interesting insight on how this is related to the hacker ethic. The research presents some evidence that women start later using computers and having computers on their own, and explains some of its findings by stating that women engage mainly in activities different than coding or that women have a burden as they normally still assume a disproportionate amount of domestic responsibilities. To some extent, this research wants to update the findings from the FLOSSPOLS project, with data from almost a decade later -the FLOSSPols data is from 2005, while the one used in this research is from late 2013.

Research questions

This paper addresses specifically the following questions:

1. Do men and women incorporate themselves to the FLOSS movement at the same age? Is the number of women entering FLOSS growing in the last years? 2. Do men and women perform the same type of contributions in FLOSS projects? Do they lead (coordinate) the same number of projects? 3. Is the proportion of men and women with children and collaborating in FLOSS projects similar? 4. Are reasons to start and to stay in FLOSS development similar for men and women? 5. Do men and women devote the same amount of time to FLOSS projects? With these questions, we aim to address not only the reasons, as perceived by the studied individuals, but also the actual situations that could help to learn about the context for those reasons.

Finally, we would like to compare our results with the ones known from the FLOSSPOLS project, to see if any changes can be seen in the "gender issue" in FLOSS after one decade.

Methodology

The data used for this research has been obtained by means of a survey. This survey, called FLOSS 2013, tried to follow the same philosophy as the one of the original, and well-known FLOSS 2002 survey [START_REF] Rishab A Ghosh | Free/libre and open source software: Survey and study[END_REF]. Thus, it is an open webbased survey, where participation is self-selected. The strategy to attract survey respondents has been to announce the survey in those channels where FLOSS developers communicate, specifically community news sites (such as Slashdot) and mailing lists. In addition to communication flows already used in FLOSS 2002, we spread the news of the survey through Twitter and other (free) social networks.

We made an important effort to preserve the privacy of the survey respondents. Thus, the survey could be answered anonymously, as IP addresses of respondents were not tracked and cookies were not used. However, participants were asked to provide their e-mail address -or some part of it-to validate that they really were FLOSS contributors. Respondents were also informed that their answers would be made publicly available in an anonymized way. Hence, if the e-mail address had been introduced, this would be handled as private information that would not be made public or shared with other research groups.

We intended to be as close as possible to the original FLOSS 2002 questionnaire in order to allow comparisons. However, some questions were added, others removed and some modified, after over a dozen meetings involving the authors where the questions were individually addressed for their meaning and goals. Mainly the changes were because of the following reasons: (1) the experience from the original FLOSS 2002 survey had shown that the question was not clear enough; [START_REF] Bimber | Measuring the gender gap on the internet[END_REF] we modified the question adopting it to the current situation;

(3) we wanted to obtain information about a new phenomenon/situation that was not relevant ten years ago; and (4) we removed the question because we thought it was of no interest for the survey or that this information can be gathered by other means, such as by mining software repositories (versioning systems, issue tracking systems, mailing lists, Q&A-sites, etc.).

An example of a question added to address the current situation (type 2 and 3) is the one related to the type of contribution to FLOSS. The original FLOSS 2002 survey focused exclusively on software developers, but the FLOSS phenomenon, even if the final product is a software, includes a number of other participants that do not code, such as translators, artists, community managers, promoters, etc. As a consequence, we target with FLOSS 2013 not only FLOSS developers, but all type of contributors. To implement this, a first question was added asking about the type of contribution to FLOSS projects by the respondent. This was a branching question, as subsequent developer-related questions were only shown to developers. All questions in the survey were optional except this first one.

The survey consisted of a total number of 58 questions1 . The survey opened November 12 th 2013 and closed December 6 th 2013. The survey responses are publicly available on the survey website in several formats and is described in [START_REF] Robles | Floss 2013: a survey dataset about free software contributors: challenges for curating, sharing, and combining[END_REF].

A limitation of the methodology is that our results are not representative of the percentage of women in FLOSS projects in general. This is because of the self-selected participation in the survey. However, the results of our survey allow to draw a picture of the context in which participation in FLOSS occurs, trying to identify differences. Actually, the survey has been especially promoted in female-specific mailing lists and communication channels to have sufficient respondents to be representative of this context.

Results

Table 1 provides the number of respondents to the FLOSS 2013 survey by gender. As respondents could leave any question unanswered, from the total number of 2,183 respondents, we only have information on the gender of 2,002 of them. 226 of them are women. The question was addressing gender self-definitions ("Which of the following describes how you think of yourself ?") so a third option "In another way" with a textbox was included. Since the total number of respondents who selected this option is small (less than 50), we have chosen only men and women as groups of study. As there are almost 10 times more men than women who answered the survey, our results are given in relative terms.

Age and date of entry

Figure 1 shows how old respondents were when they entered the FLOSS movement -the figure on the left is the one for men, women are depicted on the right. We can infer visually that men start participating at an earlier age than women. So, while men and women peak both at 21 years, the tail for men is much more abrupt than for women; the values for men in the early 30s are a third of the ones of 10 years younger, while for women the values in the early 30s are over 70% the ones in the 20s. Table 2 provides a basic statistical analysis for this question 2 . As it can be seen, the age of incorporation for women is always higher than for men, in Fig. 1. Age of first contribution to FLOSS (men left, women right). The horizontal axis is in years, while the vertical axis gives the number of respondents of that age.

general from three to five years. So, while half of the male contributors enter during their university years or before, for female this happens when many are already professionally active.

Table 3 shows that women entered the FLOSS movement more recently: 38.54% of women have started in the last four years (compared to only 18.75% of men) prior to the survey. 20%. However, the distribution of contributions for women is different. In this case, other type of contributions is the main task performed (with almost 45%), while coding comes next (31%).

Fig. 2. Type of contributions to FLOSS projects (men left, women right). The vertical axis is given in number of respondents for the three choices: "code programming", "other type of contributions" and "both".

The survey asked in how many projects respondents were involved in as project leader, coordinator or administrator. Table 5 shows that 51.49% of women were involved in these kinds of tasks (compared to 65.89% of men). However, probably due to women entering FLOSS more recently, only (approx) 5% of women coordinate more than 3 projects (compared to 18.47% of men).

Children

Figure 3 provides information on how many FLOSS contributors have children, sorted by gender. Results are presented in a bar plot, in such a way that the height of the "No" answer is the same, thus allowing to compare the proportions. The vertical axis is given in number of respondents; it should be noted that the scale used in both graphs is not the same as it has been adjusted so that the number of "No"s are represented at the same height.

The graph shows that a minority of contributors to FLOSS have children, but that the number of men with children (34%) is proportionally twice as high as the number of women with children (19%).

Reasons to start (and to continue) contributing to FLOSS

Table 6 shows the top seven reasons (with 20% or more respondents selecting them) to start in FLOSS for men, while Table 7 shows the reasons for women 3 .

The top reason for starting to contribute to FLOSS for both groups is to learn and develop new skills. Women give more importance to participate in new forms of cooperation, and men to improve FLOSS products of other developers. Improving job opportunities looks more important for women, but this could be because women are entering the FLOSS scene more recently, when the open source market has become more mature.

To find out if the reasons to stay contributing were the same or not, we have selected women and men that started later than 2011 in FLOSS, and checked the respondents of the the multiple-optional question "And today? For what reason(s) do you go on with contributing to FLOSS?". Learning and sharing knowledge and skills are still at the top both for men (58.98% and 57.28%) and women (64.78% and 62.26%), and participating in new forms of cooperation is still more important for women (31.45%) than for men (25.10%), while men still focus on improving FLOSS products of others (39.18%, against 28.30% of women). However, improving the job opportunities is a similar concern for both genders (25.85% of men, 25.79% of women selected that option).

Involvement in FLOSS

While FLOSS was merely a volunteer activity in the mid-90s, in the last decade it has seen a high professionalization [START_REF] Fitzgerald | The transformation of open source software[END_REF] and more contributors work full or parttime on FLOSS projects. Figure 4 presents the amount of time that respondents commit to FLOSS by gender. Although the distribution seems very similar at first, it is noteworthy that the share of women devoting less than 5 hours/week (53.69%) is higher than for men (49.71%), and that the amount of women working 40 or more hours per week (14.77% of women, 12.01% of men). So, when contributing to FLOSS projects women are over-represented among the less active participants or among the professional full-time FLOSS contributors, in the latter case probably working for an industrial software company.

Conclusions

In this paper we targeted a number of research questions presented in section 3.

Our results show that:

1. Women start to participate in FLOSS projects at a later age than men, and have begun to enter FLOSS projects in growing numbers in recent years. 2. Women perform majoritarily other types of contributions than coding, while men mostly contribute with code. More than half of the women are involved in leading, administering or coordinating FLOSS projects. The share of women leading projects is lower than for men. 3. The proportion of men with children contributing to FLOSS projects is almost twice the proportion of women. 4. Reasons to start and to stay in FLOSS development are similar for men and women, but men tend to focus on the product, and women in the forms of cooperation. 5. The share of women who devote less than 5 hours and more than 40 hours per week is higher than for men.

Our results confirm (and extend) the main ones from the FLOSSPOLS study, even if almost 10 years have passed between both. So, although it is possible that the percentage of women in FLOSS may have slightly increased during these years 4 -probably due to the involvement of the software industry-, many contextual patterns have remained the same. We can therefore talk about a lost decade in the general inclusion of women in FLOSS.

As future work, we plan to extend this study with the other questions included in the survey that might give more insight into the gender issue.

Fig. 3 .

 3 Fig. 3. Answers to the question "Do you have children?" (men left, women right).The vertical axis is given in number of respondents; it should be noted that the scale used in both graphs is not the same as it has been adjusted so that the number of "No"s are represented at the same height.

Fig. 4 .

 4 Fig. 4. Number of hours per week devoted to contributing to FLOSS projects (men left, women right). The vertical axis is given in number of respondents. Graphs have different scales.

Table 1 .

 1 Gender self-definition: number of respondents.

	Gender Respondents Percentage
	Male	1,776	81.36%
	Female	226	10.35%
	Other	33	1.51%
	NA's	148	6.78%
	Total	2,183 100.00%

Table 2 .

 2 Distribution of age when starting to participate in FLOSS projects.

		Min. 1st Qu. Median Mean 3rd Qu. Max.
	Men	10.00 19.00 23.00 24.49 28.00 55.00
	Women 13.00 22.00 26.00 28.01 33.00 55.00

Table 3 .

 3 Year of entry in FLOSSThe FLOSS 2013 survey, in contrast to the original FLOSS survey from 2002, was open not only for FLOSS developers, but also to any other person who performed other type of contributions to FLOSS projects. Therefore a question was included, where respondents could specify if they were mainly coders, performed other activities (such as documentation, translations, tests, artwork...) or both.Table4and Fig2show that men are mostly devoted to coding and that the amount of male contributors who perform other tasks lies slightly above

		Men	% Women	%
	Before 1990 58 3.69	0 0,00
	1990-1999	315 20.03	21 10.24
	2000-2009	905 57.53	105 51.22
	2010-2013	295 18.75	79 38.54
	Total	1573 100.00	205 100.00
	5.2 Type of contributions and number of projects involved

Table 4 .

 4 Type of contributions to FLOSS projects.

Table 5 .

 5 Number of projects as leader, coordinator, or administrator.

		Men Women
	None 34.11% 48.51%
	1	24.20% 21.78%
	2	14.41% 18.32%
	3	8.82% 5.94%
	4 or more 18.47% 5.45%

Table 6 .

 6 Top 7 reasons to start in FLOSS (men)

	Reasons to start

Table 7 .

 7 Top 7 reasons to start in FLOSS (women)

The complete questionnaire, including answers, can be obtained from http:// floss2013.libresoft.es/.

10 and 55 is given as the minimum/maximum age, because the survey allowed only responses such as "10 years or less" or "55 years or more".

The multiple-optional question was: "Remembering the time you started contributing to FLOSS, what was the reason for this?"

A study of GitHub developers from 2015 found that only around 6% were women, see http://www.toptal.com/open-source/is-open-source-open-to-women


Alexandru Coman

Alexandru Cîtea

Sabin C Buraga

Alexandru Ioan

Towards Open Source/Data in the Context of Higher Education. Pragmatic Case Studies Deployed in Romania

The open source ideology is unfortunately not so popular in Romania. This subject represents, to this day, an untackled problem especially in various local educational areas. The paper describes an interesting initiative taken this year by the Faculty of Computer Science, University of Iaşi, Romania to change the collective opinion by progressively pushing the new generations of students through a binding process with the ideas involved in the open source philosophy. Three ongoing initiatives addressing this problem are detailed, including the results we have obtained so far through them, and also the steps that are planned to be taken soon on the matter.

Introduction

Based on multiple discussions with students in various events around the country, we observed that a high percentage of those with whom we talked have a few bad preconceptions when it comes to the open source ideology, from the fact that someone will use/steal your work to the idea that open source software is of poor quality. We set out to change that perspective and build a better community around the concept of open source software, ideally promoting collaboration and teamwork among fellow and future engineers [START_REF]Open Data as Open Educational Resources: Case Studies of Emerging Practice[END_REF].

The paper presents several details about three ongoing initiatives -sharing via GitHub of the most interesting student projects (Awesome FII Projects), a two-day national conference on various open source technologies (Open Source Camp), and an academia-industry joint training event (CloudbaseLABS ) -in order to help current and new generations of students get initiated and accustomed to the open source philosophy, methodologies, techniques, and tools in the context of actual Romanian higher education system -in particular, the Faculty of Computer Science from Iaşi, Romania. Several insights about these ongoing activities are provided, followed by conclusions and further work.

Advantages of a Fully Open Faculty

Each year, the students tackle a series of challenges meant to prepare them for real-life scenarios in the industry and sharpen their skills. Some of these challenges imply working in teams and using a wide range of industry technologies (e.g., various programming paradigms, languages, methodologies, software tools). Most of their work usually gets lost and is not saved and archived to be available online, and for that reason multiple generations of students receive similar tasks, those tasks not keeping up with developments in the industry. Also, each generation of students starts from scratch and usually has little ways of interacting with more experienced, older students. According to the Open Education Consortium1 , "education is sharing knowledge, insights and information with others, upon which new knowledge, skills, ideas and understanding can be built."

From our point of view, there are multiple advantages to a open-sourceoriented faculty. Besides knowledge not getting lost, multiple generations can contribute to the same project over time. Additionally, there will be a high concentration of educational materials and resources that students can access to be able to get up to speed faster. Furthermore, due to the work older generations put in place, the bar is raised each year as a side effect. Another key feature of such a faculty is the fact that it offers its students a higher grade of visibility, resulting in more opportunities to get integrated in the job market.

In addition, we consider that our proposal approaches could be a suitable solution for learning beyond the classroom [START_REF] Scott | Learning "Beyond the Classroom" within an Enterprise Social Network System[END_REF], by adopting both formal and informal instruction virtual spaces -in this case, focusing on open source code/data public repositories stored by GitHub.

Our Current Situation

At the moment, several disciplines taught at the Faculty of Computer Science Iasi (FII) already encourage students to build software projects under various open source licences, for various disciplines: Operating Systems, Computer Networks, Web Technologies, Software Engineering, etc. As an example, at the Web Application Development course, students receive specifications for projects (such as Ark, FaW, and SAPA)2 that are based on reusing open data, including governmental data, to be able to build public REST APIs that offer valuable machine-processable information.

Several timid initiatives to publicly offer content from the faculty have appeared due to enthusiastic professors, but unfortunately only reserved to specific disciplines and not driven by a unitary educational policy at the faculty, institutional or governmental level. These initiatives tackle student projects, that could eventually be of interest to the industry, or content to be able to educate students in the means of open source possibilities.

Multiple types of materials have emerged from these initiatives, from presentations, tutorials, workshops and seminars (available in Romanian or English) to a wide variety of events organised at FII such as RoPython, Open Source Iaşi 2014, Design Jam (2011,2012,2014, in collaboration with Mozilla Foundation), Web Technologies local workshops (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2010)(2011)(2012)(2013)(2014). Other national important initiatives are the series of "Informatica la Castel" Summer School on Linux and Open Learning Environments (since 2004), ROSEdu (Romanian Open Source Education) community projects, InfoEducation (a national contest on software, Web, and robotics -established in 1993). Various details regarding the history and experiences of using open virtual environments in Romania are presented in [START_REF] Jalobeanu | From Web-based Teaching to Virtual Learning Environment and Towards Social Media Used in Education[END_REF] and [START_REF] Jalobeanu | A 43 Years History, Passing from the Gutenberg Project Initiative to the Open Educational Resources Movement[END_REF].

First Steps. Initiatives in Progress

With the help provided by the FII academic staff, we started three projects to help new generations of students get initiated and accustomed to the open source philosophy, methodologies, and techniques. Their purpose is to not only build up interest in the Open Source subject, but also initiate and support the transition to a fully Open Source Faculty by offering the students a spectrum of opportunities to interact with the concept.

Each of the proposals has a specific objective, some of them ongoing and some already offering us a set of results to analyse.

Awesome FII Projects

Our first task was to solve the problem of lost knowledge (and wheel reinvention) over multiple generations. In that sense, we started a project entitled Awesome FII Projects3 . This project represents our initiative to build a virtually-stored open knowledge library for future students to browse through and keep as a reference to their work. With its purpose in mind, we built an organisation on the GitHub online platform [START_REF] Chacon | Pro Git. Apress[END_REF] associated with the FII in order to keep track and store the top ranking projects, homeworks, assignment papers, technical/scientific articles, or useful posts.

Currently, three pilot courses students take in their first year of pursuing a Bachelor's Degree and the first year of pursuing a Master's Degree. Those courses are Introduction to Programming, Object-Oriented Programming, and Web Application Development. Also, students are encouraged to use a version control system. We are planning to extend this initiative to other disciplines of interest: Web Technologies and Human-Computer Interaction.

To be able to automate the process of adding the projects to the repository, we developed a REST API built upon the one exposed by GitHub which will help us manage the students and the projects they are building each year. When a new team of students is formed, a new repository is automatically created.

After the evaluation period has ended, based on the obtained grade and a set of default thresholds (e.g., grade bigger than 7, "readme" and "changelog" files present, code available under the terms of a specific open source license), matched projects are moved into the Hall Of Fame.

In the case of already published projects, a crawler was built -based on the GitHub API and GitHub Search Engine4 . This tool is able to search for projects having specific keywords directly related to FII disciplines (usually, these terms are predefined by the professors). As an example, if we search for "web fii", we can find 9 repositories and approximately 40 000 occurrences in the source-code. Every found repository is then going through a validation process in which the authors of the project are verified against a list of the enrolled students. If the authors of the repository match students from our faculty and those students form a known team, we can proceed in processing that repository.

Discussing results, due to the current student evaluation schedule, for Awesome FII Projects only one course (Introduction to Programming) has finalised out of the three pilot courses. This introductory discipline covered the basics of programming (arrays, functions, recursive functions, basic algorithms) over the span of 14 weeks of teaching. Each first year student (from the 14 groups, each containing on average 28 persons) had to develop -as a last laboratory evaluation step -a simple application chosen from a list of proposed projects by using the procedural paradigm (i.e. C language). At the end of the evaluation period, we selected the top ranking projects in terms of complexity and quality and added them to the Awesome FII Projects list. From 392 built and successfully assessed projects, we chose about 2 percent and added them to the faculty's GitHub public repository 5 . We observed that each repository has an average of 6 commits and an average of 2391 lines of code.

In the near future, about 35 projects of the Web Application Development discipline will be assessed -using a combination of criteria adapted from [START_REF] Allemang | Semantic Web for the Working Ontologist (2nd Edition)[END_REF], [START_REF] Casteleyn | Engineering Web Applications[END_REF], and [START_REF] Heath | Linked Data: Evolving the Web into a Global Data Space[END_REF] -and the best of them will be chosen to be included into the specific Awesome FII Projects collection. The project (developed by a team of 2-4 students) consists of an application built by using the existing social and semantic Web technologies. There are no restrictions regarding the implementation solution, but the open source approaches are highly encouraged. All data managed by the developed Web application must be modelled by using the RDF (Resource Description Framework) specifications [START_REF] Allemang | Semantic Web for the Working Ontologist (2nd Edition)[END_REF][START_REF] Heath | Linked Data: Evolving the Web into a Global Data Space[END_REF]. The project should be developed and deployed according to the actual practices in software/Web engineering by using a cloud-based approach for the code management.

Having established such a software project collection, we will able to assess the usability, usage spread, code quality and evolution of the most popular sub-repositories, following the ideas and proposals enumerated by [9] and [START_REF] Rajanen | Examining Usability Work and Culture in OSS. Open Source Systems: Adoption and Impact[END_REF].

Open Source Camp

The second project is called Open Source Camp6 and is meant to be a threein-one event consisting of a conference, a hackathon, plus several workshops, during two days (7-8 May 2016). The purpose is to gather various professionals from the industry and, with their help, build an event for all skill levels and perspectives (open source or not) and gather as much interest in the community.

Open Source Camp is based upon the idea launched in 2014 and called "Open Source Iaşi", a 10-hour local event gathering about 100 participants. This year, we are taking our event to a national level and hopefully gathering more than 300 students and developers, currently managed to invite a large part of the local and national communities dedicated to the open source movement: RoPython, ROSEdu, Rogentos, Open Knowledge Romania, and others to be investigated and contacted. We are further preparing the event and gathering support to promote the open source values and best practices.

CloudbaseLABS

The third already started project is CloudbaseLABS7 , initiated in collaboration with Cloudbase Solutions, a private IT company having an office in our city. With its help, we envisioned a two week training event for groups of eight students that will cover multiple technologies and pragmatic knowledge used in the industry -e.g., system and network administration, virtualisation, OpenStack8 architecture, Python programming.

We created a campaign in order to evaluate the number of students interested in open source. Out of 1873 contacted students, only 146 students actually opened our message -from those, 105 accessed our online provided resources (about 5%). After organising a social event and another campaign, we managed to climb to 7% of the students. Thus, there is much work to be done concerning the promotion of what the FLOSS movement is and its benefits for education. Finally, 24 students were selected to be part of the training program. Due to the large number of applications, this event was hosted in two rounds at FII.

During the training program, the main methodologies of development in open source projects were promoted. The participants had at their disposal a code repository 9 with a series of challenges which they were asked to solve. For each proposed solution, the participants received feedback from the organisers and the other students and were encouraged to improve their code.

As expected, their work progressed very well after a few days of constant feedback and supervising. The participants offered daily anonymous feedback to signal any problems and also to suggest what other things may be of interest to them. Accordingly, the training program was adapted to suit their needs and unanswered questions. Analysing the data from the provided source-code, a positive tendency is observed, from commits with large amount of issues to more refined code with a lower number of problems -see Figure 1. The three described initiatives join together in the same idea: to offer students the opportunity to learn about and interact with the open source ideology. Even if several disciplines encourage students to use open source software and to provide their code and data under the terms of open licenses, there is a large amount of potential. We have managed to awaken the curiosity and enthusiasm of a small group of people, which we hope will help us further to promote the FLOSS values and build an educated generation of students further, including the academic motivation and self-regulation while learning online by inspecting, understanding, and contributing to the existing knowledge [START_REF] Artino | Academic Motivation and Self-Regulation: A Comparative Analysis of Undergraduate and Graduate Students Learning Online[END_REF].

We plan to integrate a semantic Web-based system that outputs statistics and metadata from all of the FII projects in the RDF format, following the Linked Open Data initiative [START_REF] Heath | Linked Data: Evolving the Web into a Global Data Space[END_REF]. Using this system, we will be able to provide multiple types of information regarding each repository -e.g., the evolution of our students at a technical level, security problems in several types of applications, the most common mistakes found in code writing, and many others.

Fig. 1 .

 1 Fig. 1. Visualising data regarding source-code quality of student projects

Open Education Consortium: http://www.oeconsortium.org/

List of projects developed in 2015/2016 by the postgraduate students enrolled at Web Application Development discipline: http://profs.info.uaic.ro/ ~busaco/ teach/courses/wade/projects/

Public repository of the most interesting projects developed by the FII students: https://github.com/info-uaic/awesome-fii-projects

GitHub Developer: https://developer.github.com/

Introduction to Programming selected projects: https://github.com/info-uaic/

Open Souce Camp: https://oscamp.eu/

CloudbaseLABS: http://labs.cloudbase.it/

OpenStack official Website: https://www.openstack.org/

CloudbaseLABS challenges: https://github.com/alexandrucoman/labs

Appendix

Reference presentations publicly available for students as open educational resources under the terms of "Creative Commons (CC) Attribution-NonCommercial-ShareAlike License" 10 :


Gema Rodríguez-Pérez

Jesús M Gonzalez-Barahona

Gregorio Robles

Dorealda Dalipaj

Nelson Sekitoleko

BugTracking: A tool to assist in the identification of bug reports

Keywords: Issue Tracking System, Code Review System, Bug Triage, Tool

Issue tracking systems are used, in most software projects, but in particular in almost all free open source software, to record many different kinds of issues: bug reports, feature requests, maintenance tickets and even design discussions. Identifying which of those issues are bug reports is not a trivial task. When researchers want to conduct studies on the bug reports, managed by a software development project, first of all they need to perform this identification.

The job for researchers here is very different from the bug triaging that researchers do. In the latter case, people with a considerate experience in the project make a decision based on the information available at that time (maybe just a short comment by some user), asking, if needed, for more details. In the former case, researchers usually have not that experience in the project, but they have at their use all the information produced, until the moment the issue was closed. This may include not only all comments and actions on the issue tracking system, but for example, discussions about a fix in the code review system, or the final fixing patch in the source code management system. Having all that information conveyed to the researchers, in an easy, flexible and quick way, accelerates and makes their decision process much more reliable. It simplifies large scale manual analysis of issues (in hundreds or thousands), helping researchers to ensure that they are really working with what they intend to work: bug reports. This paper presents a tool designed to solve exactly the problem of providing the researchers with all the relevant information needed to decide whether an issue corresponds to a bug report or not. The tool uses information extracted automatically from the projects repositories. It offers a web-based interface which allows collaboration, traceability and transparency of the identification of bug reports. All this makes the process easier, faster, and more reliable.

Introduction

While a software system is being developed, software engineers use version control repositories to produce and manage their code. Researchers and testers report issues, which are then stored in other repositories, known as issue-tracking systems, where many kinds of issues can be found.

Issue-tracking systems facilitate the process of solving these bugs, but their shortcoming is the difficulty in distinguishing which of the reports are bug reports or not. These systems provide an interface to manage reports of maintenance activities where researchers can report issues describing bug reports, features or code optimizations. During the bug triage process it is difficult to distinguish bug reports from other issues; a study describes that two of five issues are misclassified [START_REF] Herzig | It's not a bug, it's a feature: how misclassification impacts bug prediction[END_REF]. This misclassification causes bias predicting bugs where non-bug reports are taken into account.

To distinguish the bug reports we could have used automatic classification systems, as described in [START_REF] Antoniol | Is it a bug or an enhancement?: a text-based approach to classify change requests[END_REF], but the vocabulary used in the description of the issues could change from project to project, as well as the policy depending on the project. Consequently, data validation is recommended as mentioned in [START_REF] Herzig | It's not a bug, it's a feature: how misclassification impacts bug prediction[END_REF].

Linking a bug report in a issue-tracking system and the corresponding fixcommit may not be a trivial task. Traditionally, the methods used in link recovery [START_REF] Zimmermann | Predicting defects for eclipse[END_REF][START_REF] Zimmermann | Preprocessing CVS data for finegrained analysis[END_REF] are based on text patterns or the mining of key phrases. Unfortunately, these methods include many false negatives causing bias in data [START_REF] Bird | Fair and balanced?: bias in bug-fix datasets[END_REF][START_REF] Nguyen | A case study of bias in bug-fix datasets[END_REF]. Therefore other methods, such as the Mlink approach, have been developed to link bug report with fixes using features in the changed source files corresponding to commit logs in addition to the traditional textual features [START_REF] Nguyen | Multi-layered approach for recovering links between bug reports and fixes[END_REF]. But in all of these methods, it is supposed that the issues are bug reports.

In this paper, we present a tool that displays, to the benefit of the researchers, a collection of all the necessary information needed to decide if an issue is a bug report or not. The tool, through the collection of exhaustive data on bug reports and the corresponding fix-commit, along with researchers extensive knowledge of the system, will help the last in their decision making, leading them into choosing only bug reports. This way they will not recur in any bias induced by non bug reports. To the best of our knowledge, this is the first tool that provides support to the identification of bugs and classification of bug reports. The need of the contribution of this tool arises from the increasing interest that both the academy and industry world is showing in the bug classification as a primary factor in modern software development.

The tool

The tool is a web application, therefore it runs in a browser. It displays the main data for distinguishing bug reports from others issues. The researchers will be responsible for classifying the issues from Launchpad as bug reports or not, and can thereby explain their decision for each issue. The issues are what we will refer as tickets during the paper.

Architecture

The tool integrates information from Launchpad as issue-tracking system and Gerrit as code review system. The figure 1 presents the architecture of the tool. The tool was developed using JavaScript, Node, JQuery and HTML5 technologies. The queries to the API of Gerrit and Launchpad are executed on server side. The responses are displayed on the client side. The end user can view the information displayed and interact with the server through events. Both sides exchange information using JSON files along with using their own REST API. Furthermore, we use a third-party application between GitHub and the browser in order to integrate some functionalities from GitHub.

Main Features

Figure 2 illustrates a screen short of the main tab of the tool. In particular, the tool displays the ID of the tickets which are extracted randomly from each issue-tracking repository of OpenStack. Other related information is displayed. Based on all these data, the researcher can decide whether the issue is a bug report or not. We focused on displaying the main parameters that help in the classification of reports, such as title and description of the report, as well as the description of the fix commit. For example for ticket ID 1531734 the tool displays the information related with the ticket in Launchpad and its corresponding review in Gerrit.

There is other additional information that the tool does not displays. If the researchers find it necessary, they can access the Launchapd and Gerrit web pages, respectively of the ticket and review, through the links provided by the tool. Thereby they can access extra information such as the comments written by code review researchers that correspond to that particular ticket. This provides a mean for tracking the history of the ticket from the moment it was opened until it was closed.

The tool further facilitates researchers to record and express their opinion about the ticket after reading all the information that is automatically displayed. They have to classify the ticket as Bug report or Not Bug report. Due to unsophisticated description used in the ticket, the researchers could doubt the classification. For this reason we add an extra option in the classification, Undecided. Furthermore, the researchers have a text area to write keywords found in the title, in the description of the ticket and commit message, that support their classification. Finally, they can leave their comment on why they classified a report as Bug, Not Bug or Undecided. Such information, in the future, will help us building an automatic bug classification system.

Another feature of the tool is that it allows to carry out a blind analysis of the tickets. Since all the data analysis inserted about a ticket is saved in a file on ones GitHub account, such analysis can be done by two or more researchers in parallel. By saving the data in GitHub, we could also measure the time that each researcher need for an analysis, which tickets were more difficult to analyze and other metrics that can help us understanding the current problem of issue misclassification.

Fig. 2. Screenshot of Analyze Tab

The web page provides different functionalities depending on the tab the researcher is browsing. We explain these functionalities in the following: At the current state we present the initial version of the tool which is available at;1 , as well as a demonstration video2 . It is licensed under GPL 0 (General Public License) and you can find the code at a GitHub page 3 . Anyone can use the tool, regardless of having GitHub account or not. However, it should be noted, for the researcher to save, modify data and see statistics of analysed tickets automatically, he/she should create a GitHub repository with the same name as the OpenStack project to be analysed, for example if the OpenStack project name is Nova, than the GitHub repository name should be Nova.

Results

We have analyzed 459 different tickets under the support of the initial version of the tool. 125 tickets where from Cinder, 125 from Nova, 125 from Horizon and 84 from Neutron. All the tickets have been analyzed by two out of the three researchers. The Table 1 shows the percentage of tickets classified as bug reports for the different researchers. These results don't report for some combinations of researchers because of in some projects, only a researcher analyzed all the tickets and the two remaining analyzed the half of these tickets each one.

The percentages between R1 and R3 are really similar, whereas the R2 has identified more Bug Reports in his analysis. But, the three results support the misclassification present in bug tracking systems. Furthermore, according to [START_REF] Herzig | It's not a bug, it's a feature: how misclassification impacts bug prediction[END_REF]'s work, approximately two of five issues are misclassified in the analysis of R1 and R3.

Focusing in the concordance between researchers analyzing the same ticket, 417 tickets present a double bind review process, obtaining that each ticket was analyzed by two researchers. Table 2 shows the percentage of concordance between researchers in each repository after the analysis of the tickets. Table 2 shows that the concordance of the researchers is high, but, also demonstrate the difficulty to classify tickets as bug report or as bug report, because each researcher can have different opinions about a specific ticket. The concordance could be higher if they were expert in the project.

All data from the analysis are available in the GitHub repositories of the researchers4 , the repositories having the same name of the projects analyzed in OpenStack.

Future Work

Since we are conducting empirical studies based on OpenStack projects, the current tool is limited to OpenStack as a pilot project. In the future, our aim is to extend the tool at extracting tickets from others bug tracking systems, such as Bugzilla or GitHub where the server will operate against them to analyze the most OSS project. Additionally, we aim to study the misclassification in this OSS projects. We would like to add more features to the tool. One of them would be to display information such as the lines of code changed in the files affected from the fix-commit, along with the code in the bug seeding moment. Furthermore, we would like to implement an automatic classifier for the tickets, based on the semantic of the description of the ticket and the fix-commit. The result will indicate a percentage of confidence about whether a ticket is a bug report or not. However, the researcher will always make the final decision. The automatic classification will enable researchers to focus only on problematic issues, which can be easily misclassified.

We also aim to investigate what will be the results if the data sources used by the tool to automatically extract tickets are used in isolation to to manually classify bugs or other possible bug classifying tools. This will help us validate our results and the tool to further improve.

Fig. 1 .

 1 Fig. 1. Architecture of the tool.

  1. Tab Repository: In this tab you can choose which repository you want to analyze. Currently the tool supports the four principal repositories of Open-Stack: Cinder, Nova, Neutron and Horizon. 2. Tab Analyze: It is the Tab illustrated in Figure 2. It is where all the data from a specific ticket are displayed. The user can either select a random identifier or insert one of his choice. According to the data retrieved from Launchpad and Gerrit, the researcher can classify the ticket. 3. Tab Statistics: This tab extracts the data already analyzed by a researcher involved in the analysis from their user account in GitHub. It analyzes these data and displays a distribution of the classifications in a table; 4. Tab Modify: In case the researcher thinks to have inserted a mistake during the analysis, in this tab he/she can edit any of the data saved in his/her GitHub repository.

Table 1 .

 1 Classification statistics of each researcher

		Bug Report	Not Bug Report	Undecided	Total
	Researcher 1	(184) 55%	(115) 34%	(35) 11%	334
	Researcher 2	(188) 76%	(54) 22 %	(7) 3%	249
	Researcher 3	(188) 56%	(116) 35%	(30) 9%	334

Table 2 .

 2 Concordance between each researcher in each repository

		Nova	Cinder	Horizon	Neutron	Total
	R1 and R2	(44/63) 70%	(40/52) 77%	(37/62) 60%	-	68%
	R1 and R3	-	(46/63) 73%	(48/63) 76%	(26/42) 62%	71 %
	R2 and R3	(41/62) 66%	(10/10) 100%	-	-	71%

bugtracking.libresoft.es

https://www.youtube.com/watch?v=q0-TIvL4mqc&feature=youtu.be

https://github.com/Gemarodri/BugTracking

https://github.com/{Gemarodri,ddalipaj,nellysek}