Stephan Fellhofer
email: stephan.fellhofer@gmail.com

Annemarie Harzl
email: annemarie.harzl@ist.tugraz.at

Wolfgang Slany
email: wolfgang.slany@tugraz.at

Scaling & Internationalizing an Agile FOSS Project: Lessons Learned

Keywords: agile development, Kanban, distributed software development, documentation management, communication, scaling, internationalization

This paper describes problems that arose with the scaling and internationalization of the open source project Catrobat. The problems we faced were the lack of a centralized user management, insufficient scaling of our communication channels, and the necessity to adapt agile development techniques to remote collaboration. To solve the problems we decided to use a mix of open source tools (Git, IRC, LDAP) and commercial solutions (Jira, Confluence, GitHub) because we believe that this mix best fits our needs. Other projects can benefit from the lessons we learned during the reorganization of our knowledge base and communication tools, as infrastructure changes can be very labor-intensive and time-consuming.

Introduction

Scaling and internationalizing a Free and Open Source Software (FOSS) project is not an easy task, at least not in our experience. Our project grew from five contributors in 2010 to over 130 in 2014, which lead to various organizational problems. In this paper we describe our problems, approaches we devised to solve them, and lessons learned along the way. We think that other FOSS projects can profit from our experiences.

In our project Catrobat we develop a visual programming language which has been inspired by Scratch [START_REF] Resnick | Scratch: programming for all[END_REF] from the Lifelong-Kindergarten-Group at the MIT Media Lab. Our aim is to empower children and teenagers to easily create their own programs and to express themselves creatively using their smartphones. To reach this goal we develop Integrated Development Environment (IDE) and interpreter apps natively for several mobile platforms. Our app, Pocket Code, is available for Android on Google Play 1 . Versions for iOS, Windows Phone, and HTML5 capable browsers are currently in development. Catrobat was initiated by Wolfgang Slany and a team of students from Graz University of Technology, seeking development challenges beyond those seen in class and to practice what they have learned. This eagerness to apply software development principles taught in courses heavily influenced the basic structure of the project, with all its advantages and disadvantages.

On the positive side the usage of agile development methods such as Kanban [START_REF] Anderson | Kanban: Successful Evolutionary Change for Your Technology Business[END_REF][START_REF] Hiranabe | Kanban applied to software development: from agile to lean[END_REF] enables us to stay flexible and to easily adjust the scope of our project as we go. eXtreme Programming (XP) (especially pair programming [START_REF] Andres | Extreme Programming Explained: Embrace Change, 2nd Edition[END_REF]) facilitates knowledge transfer between developers. Test-Driven Development (TDD) [START_REF] Beck | Test-Driven Development: By Example[END_REF] ensures that the code remains functional and testable while new developers join and former developers leave the project. On the negative side our contributors additionally have to learn about agile development methods, which steepens the learning curve.

A dedicated Usability and User Experience (UX) team applying the personas method [START_REF] Hussain | Practical usability in xp software development processes[END_REF] and other usability techniques helps us focus on the users. This is particularly important because, in contrast to most FOSS projects, Catrobat's developers (mostly university students) are not a subgroup of Catrobat's targeted users (mostly children and teenagers). The UX team helps to create an understanding of user needs in all developing teams.

Communication within our teams was initially mainly face-to-face, which is good for localized agile teams, but leaves many decisions undocumented, which is disadvantageous for a larger, distributed FOSS project. Discussions tend to get started over and over again, when nobody remembers why and based on what information a decision was made in the first place. Later we will elaborate on the communication problems, because they are at the core of our difficulties with scaling and internationalizing.

The increasing number of contributors from five in 2010 to over 130 in 2014, and the participation of international contributors in our project made organizational problems apparent. Documentation was not equally available for everyone, the entire current project status was not visible online and we lacked important communication channels. For example most of our contributors did not use Internet Relay Chat (IRC), which is often an integral infrastructure of FOSS projects and compulsory for the participation in Google Summer of Code (GSoC)2 , in which our project participates since 2011. To address these problems and to be able to integrate more and international contributors, we had to change our project infrastructure, the ways we communicate and some of our tools.

Section 2 gives a short overview of work related to the approaches we formulated. In Section 3 we will identify the problems in greater detail and describe our approaches to solve them. Section 4 contains the lessons we learned on our way from a small, localized to a larger, more international project. Subsequently we discuss possibilities for future work and provide some concluding remarks.

Our main goal was to enable and facilitate contributions from project members around the world. Various studies ( [START_REF] Korkala | Communication in distributed agile development: A case study[END_REF][START_REF] Layman | Essential communication practices for extreme programming in a global software development team[END_REF][START_REF] Poole | Distributed product development using extreme programming[END_REF][START_REF] Scacchi | Collaboration practices and affordances in free/open source software development[END_REF][START_REF] Schmmer | Support for distributed teams in extreme programming[END_REF]) highlight the importance of communication in FOSS projects or in projects which use agile development techniques.

We focused on optimizing the communication for FOSS and agile software development projects. Specifically, we tried to address some of the issues and approaches highlighted in the literature: Layman et. al [START_REF] Layman | Essential communication practices for extreme programming in a global software development team[END_REF] recommended among other things that "when face-to-face, synchronous communication is infeasible, use an email listserv" and "use globally-available project management tools". Korkala et. al [START_REF] Korkala | Communication in distributed agile development: A case study[END_REF] suggested to "enable and support direct communication between the developers". Some technologies and common practices used in FOSS development such as instant messaging, IRC, news postings, how-to guides, Frequently Asked Questions (FAQ), or wikis are listed in [START_REF] Scacchi | Understanding the requirements for developing open source software systems[END_REF][START_REF] Scacchi | Collaboration practices and affordances in free/open source software development[END_REF]. In [START_REF] Yamauchi | Collaboration with lean media: how open-source software succeeds[END_REF] technologies such as versioning systems and TODO lists are mentioned. Difficulties for newcomers like "selection of a suitable task", "lack of up-to-date development documents", or "no response from core developers for their doubts" are mentioned in [START_REF] Shibuya | Understanding the process of participating in open source communities[END_REF].

Optimizing Services for Distributed Participation

Our project grew faster than the supporting infrastructure, which led to organizational problems. For each part of our infrastructure we will first describe the initial situation, then problems which occurred over time and finally how we resolved them.

User Management

Initial Situation. In the beginning there was no consistent user management. Every piece of infrastructure (for example: instant messaging, source code repository) had its own built-in user management, and accounts were created manually on demand. Since some services were used through shared user accounts, there was no accountability.

Resulting Problems. The effort necessary for user management and maintenance increased tremendously with the growing number of contributors, because every user had to be created manually and every change had to be populated manually to all platforms. This resulted in missing and outdated account information. Rights management was not even a topic. Sometimes this lack of restrictions caused inexperienced contributors to inadvertently change or delete infrastructure. Shared accounts made it impossible to trace who made changes to project services. On the side of the contributors the account management was elaborate too, as for every service, contributors had to use different credentials.

Method of Resolution. Our goal was to simplify the management of user databases and to support the contributors by providing them with only one account for (almost) all our infrastructure. Most parts of our infrastructure have a builtin support for Lightweight Directory Access Protocol (LDAP), so LDAP was the most suitable solution to simplify our user management. We decided to use LDAP groups as well for various reasons:

different experience levels need different rights different contributor groups need different resources and services no shared accounts, so there are clear responsibilities infrastructure administration should be left to experienced contributors

The goal was to design a user management which serves experts and beginners alike. Experts should have all the rights they need, while beginners are not overwhelmed by too many services and rights too soon.

Unfortunately, not all services support LDAP. For example, externally hosted services such as GitHub do not support foreign user directories and still need additional maintenance. Other services like Crowdin3 support OAuth4 as authentication method, but to take the user groups and corresponding rights needed for our project into account, the service's Application Programing Interface (API) or an additional configuration interface must be used.

Communication

Our project was and still is allowed to use a room at Graz University of Technology, where our local contributors can meet, discuss, and code. In the beginning all contributors fit in the room, and communication was mainly face-to-face. The reliance on face-to-face was very beneficial in the beginning but caused some communication and documentation problems later on as mentioned in Section 1.

Instant Messaging

Initial Situation. Other means of communication were and still are e-mail, mailing lists5 as recommended in [START_REF] Layman | Essential communication practices for extreme programming in a global software development team[END_REF] and Instant Messaging (IM). In the beginning we used a Skype group chat for project discussions with all contributors.

Resulting Problems. Many contributors joined the Skype group chat but did not participate actively, because they preferred face-to-face communication, e-mail, or were overwhelmed by the number of messages that did not concern them. This massive number of messages was a direct result of the growing number of contributors.

Another problem with Skype was that group chats are invite-only, which made it difficult for aspiring contributors to join the discussion. They first had to find a project member to invite them. Yet another problem with our Skype group was the language used. Almost all messages were in German, because all of the initial team members spoke German.

In our attempt to open up our project and allow for internationalization, we created an IRC channel, which was open to everyone and where it was obligatory to use English.

This attempt failed, because project issues were, as a matter of habit, still discussed in Skype and hardly anyone used IRC. So theoretically we maintained an IRC channel, but interested contributors still got "no response from core developers for their doubts and support request" [START_REF] Shibuya | Understanding the process of participating in open source communities[END_REF].

Method of Resolution.

We decided to switch our whole synchronous communication to IRC and deleted the Skype group chat. The reasons for this decision were:

one IM platform for all purposes anybody can join the channel he or she is interested in -'irrelevant' messages are reduced, because messages are posted only to the channels where they belong faster responses to questions from aspiring contributors due to increased online time of project members topic specific channels can be created and deleted easily, when needed To make the communication with IRC more attractive for our contributors we provide them with an IRC bouncer which records all messages when the user is offline and replays them when the user goes online. In our attempt to minimize the amount of messages every contributor has to read, we decided to split the conversation into different channels. There exist separate channels for subteams, technical support teams (for example for our continuous integration server) and one channel for general information and announcements. Contributors may still miss important information, if they do not join all channels, but the risk of information overload should be diminished.

One disadvantage of IRC as communication platform is that the technology seems old-fashioned to our contributors and most of our contributors have never used IRC before. Another disadvantage is that it is more time consuming to configure IRC with the bouncer than it is to configure Skype. Contributors have to authenticate to freenode 6 and to the project bouncer with different credentials.

Today IRC is widely accepted by our community and the communication improved compared to Skype because contributors only have to join and read channels they are interested in and people are by now used to IRC.

Agile Development Management

Initial Situation. As already mentioned we were and still are allowed to use a room at the university for our project. There, we have whiteboards serving as Kanban boards, as well as story cards on paper at our disposal. Initially every subteam had its own Kanban board in the room, though this became impossible as the number of subprojects grew. Our project used and still uses GitHub7 as source repository. Previously, we used the integrated issue tracker not just to track bugs and issues reported by users but also as a digital version of our local Kanban boards. Labels of issues were used to indicate which kind of issue it was (bug, story, enhancement, . . . ), the current working status (to do, in development, done, . . . ), the priority (low, medium, high, or critical ) and which part it affected (development environment or interpreter ). To enable children and teenagers to report bugs and issue requests without bothering with Github, we created a Google Group8 .

Resulting Problems. Our user stories and bug reports had to be synchronized between three different platforms, namely the local Kanban board, Github, and the Google Group. It is not hard to imagine that this had negative consequences: the local Kanban board was often outdated and the issue tracker did not cover stories, which were created locally on the Kanban board. The Github issue tracker did not support our Kanban board and lacked the functionality of hiding security relevant issues.

Method of Resolution.

We decided to switch to an online distributed agile development environment. There exist various tools and we did a comparison of some, namely GitHub, Bugzilla9 , Redmine10 , and Jira11 . We compared them based on the following criteria:

they should support LDAP, virtual whiteboards, and different layers of visibility for security relevant issues the workflow should be customizable to our needs the tool should be expandable through add-ons means for reporting and analysis should be integrated a wiki system should be integrated because the old one needed to be replaced (for further details see Section 3.4)

We compared the basic versions of the different tools, without considering all the available add-ons, because add-ons can be more easily abandoned by their developers than complete tools, and we can not afford commercial addons. Table 1 shows the results of our comparison. Based on this comparison we decided to try Jira, because together with Confluence12 , it would satisfy all our criteria and both tools are free for FOSS projects.

As a pilot test we used Jira during GSoC 2013. Every mentor/student pair received their own project with a simple Kanban board. Over this trial period we realized we had to customize the original workflow of issues to fit our developing principles (specifically review of newly created issues and code acceptance by experienced developers) and to integrate usability reviews into the workflow.

After the successful pilot test we decided to introduce Jira as a globallyavailable project management tool (recommended by [START_REF] Layman | Essential communication practices for extreme programming in a global software development team[END_REF]) and as replacement for the local whiteboards. This proved to be another important step towards becoming an international FOSS project.

Documentation Management

Initial Situation. Initially Google Docs, Dropbox, and a wiki system were used for distributing documents and information. The wiki system grew more or less naturally and the structure was seldomly revised. For a group of five people, which communicates mainly face-to-face, a less formal documentation management was useful, but once the number of contributors grew, we experienced severe problems.

Resulting Problems. Document owners left the project and the owner rights were not transferred. Documents were not updated, became outdated, and there was no general overview of documents and their content. File sharing tools like Google Drive and Dropbox were neither integrated in our wiki nor were they supporting our new user management with LDAP. These facts made it harder to share and find current information, documents, and their owners.

The wiki was not well maintained on all of its pages, and the organically grown structure could be an additional hurdle if one did not take the time to learn how to use the included tools. As discussed in [START_REF] Shibuya | Understanding the process of participating in open source communities[END_REF], outdated development documents lead to difficulties for newcomers. Even some of our experienced project members tended to avoid using the wiki because of its partly outdated content and complex structure.

Method of Resolution.

As explained in Section 3.3 we decided to use Jira as globally-available project management tool and Confluence as new knowledge management platform. The switch of systems provides us with an incentive to revise the structure and to eliminate or correct outdated information. This will make it easier to find up-to-date and useful information. The introduction of Confluence is still ongoing, because the information representation needs major rework.

Scacchi [START_REF] Scacchi | Collaboration practices and affordances in free/open source software development[END_REF] introduced Free and Open Source Software Development (FOSSD) informalisms, which are easy to use and publicly accessible resources like: threaded discussion forums, group blogs, news postings, how-to guides, to-do lists, and FAQ. Most of these technologies are supported either by Jira or Confluence and we expect that they will support our contributors in their search for information.

Lessons Learned

Human Related

Introducing and switching to IRC as the new IM service was time consuming and resulted in resistance. We believe that the confusions described in Section 3.2, the out-dated and uncomfortable user interface of IRC clients, and our underestimation of the need for change management were mainly responsible for the long (and still ongoing) resistance. To reduce this opposition to change we used techniques described in [START_REF] Aladwani | Change management strategies for successful erp implementation[END_REF][START_REF] Kotter | Choosing strategies for change[END_REF] during the introduction of Jira. We involved more developers during the configuration period of Jira and the workflow adaptation. We communicated the benefits of the new system more clearly. We believe that this communication of benefits, training, involvement of GSoC mentors and optimizing the workflow led to a smooth change of our issue tracker and the introduction of Jira as a project management tool.

Technology Related

Centralized management of team member accounts and information about them should ideally be introduced from the very beginning as it tremendously simplifies the administration of contributors and saves a lot of time later. User rights management should be well structured and at the same time adjustable to future changes. Integrated services are preferable from a maintenance perspective, because they save time and effort related to organizational tasks. This time can then be spent on project goals. Not all external services, for example GitHub support LDAP, but if an API is available for that service, and user maintenance for this service consumes a lot of time, at least some parts should be automated by scripts.

Future Work

The transition to the newly deployed services is not yet finished for every team. Jira is already widely applied throughout the teams, but some teams still have to make the transition. As stated in Section 3.4 Confluence is not yet deployed since the restructuring of the old system takes time and the focus is on easily accessible and maintainable data. Confluence will contain how-to guides, FAQ, blog-like news entries, and meeting notes to be more transparent and to provide new contributors with the most relevant and up-to-date information [START_REF] Scacchi | Collaboration practices and affordances in free/open source software development[END_REF]. To support the transition [START_REF] Aladwani | Change management strategies for successful erp implementation[END_REF][START_REF] Kotter | Choosing strategies for change[END_REF] from the wiki system to Confluence we did a survey to detect main concerns and problems with the current wiki and are involving senior contributors in the content adaptation process.

Conclusion

As explained in Section 2 communication and documentation are essential parts of agile software development and even more important in distributed development [START_REF] Shibuya | Understanding the process of participating in open source communities[END_REF]. Face-to-face communication is suitable at the beginning, but with the growth and internationalization of a project, good communication channels and project management tools have to be introduced. Every change of workflow or established tools is time consuming and needs proper change management to succeed. So before changing major aspects it should be well considered, if the changes are worth the effort. Appropriate user and rights management simplifies the administration of infrastructure and contributors. It saves time and supports contributors by giving them access to the project's services ideally with one account.

Table 1 .

 1 Comparison of bug tracker/project management software

		GitHub	Bugzilla Redmine Jira/Confluence
	Free for FOSS	Yes	Yes	Yes	Yes
	LDAP support	No	Yes	Yes	Yes
	Restrict viewing rights	No	Yes	Yes	Yes
	Workflow (customizable) Manually with labels Yes	Yes	Yes
	Agile development support 3rd-party websites Add-ons Add-ons Yes
	Reporting	Basic graphs	Yes	No	Yes
	Integrated wiki system	Yes	No	Yes	Yes
	Expandable	Via API	Yes	Yes	Yes

A global program from Google to support FOSS projects by sponsoring students and pairing them with mentors to develop given tasks over summer (https: //developers.google.com/open-source/soc/)

Tool to help non-developers to translate texts (https://crowdin.com/)

http://oauth.net/

https://groups.google.com/forum/?hl=en#!forum/catrobat

IRC network where our project runs all its channels (https://freenode.net/)

https://github.com/Catrobat

https://groups.google.com/forum/?fromgroups=#!forum/pocketcode

https://www.bugzilla.org/

http://www.redmine.org/

Planning and tracking tool for agile project management by Atlassian (https:// www.atlassian.com/software/jira)

Wiki system by Atlassian which is free for FOSS projectshttps://www.atlassian. com/software/confluence

http://developer.catrobat.org/credits

Acknowledgments

Thanks to all contributors of Catrobat 13 and also other supporting parties 14 . This work has been partially funded by the EC H2020Innovation Action No One Left Behind, http://www.no1leftbehind.eu/, Grant Agreement No. 645215.


Ann Barcomb
email: ann.barcomb@fau.de

Michael Grottke
email: michael.grottke@fau.de

Jan-Philipp Stauffert
email: jan-philipp.stauffert@fau.de

Dirk Riehle

Sabrina Jahn
email: sabrina.jahn@fau.dewww

How Developers Acquire FLOSS Skills

Keywords: competencies, informal learning, non-formal learning, open source, skills, software developer

With the increasing prominence of open collaboration as found in free/libre/open source software projects and other joint production communities, potential participants need to acquire skills. How these skills are learned has received little research attention. This article presents a large-scale survey (5,309 valid responses) in which users and developers of the beta release of a popular file download application were asked which learning styles were used to acquire technical and social skills. We find that the extent to which a person acquired the relevant skills through informal methods tends to be higher if the person is a free/libre/open source code contributor, while being a professional software developer does not have this effect. Additionally, younger participants proved more likely to make use of formal methods of learning. These insights will help individuals, commercial companies, educational institutions, governments and open collaborative projects decide how they promote learning.

Introduction

Free/libre and open source software (FLOSS) is important to the economy, with many companies now relying on FLOSS components not only internally but also as the basis for their commercial offerings. Fostering FLOSS talent is critical for companies, which now support an estimated 50% of FLOSS development [START_REF] Riehle | Paid vs. volunteer work in open source[END_REF]. Governments have also become increasingly concerned not only with FLOSS adoption but with building capacity for FLOSS development as a means of promoting innovation. It is therefore in the interests of companies, governments and FLOSS communities to know which skills or competencies 1 are necessary for FLOSS development and how they can be trained.

Contributing code to a FLOSS project requires both technical skills and social skills such as the ability to coordinate with others, to clearly articulate an argument [START_REF] Ghosh | Free/libre and open source software: Survey and study[END_REF], to give constructive feedback and to comply with social rules [START_REF] Kimmelmann | Career in open source? Relevant competencies for successful open source developers/Karriere in Open Source? Relevante Kompetenzen für erfolgreiche Open Source Entwickler[END_REF]. Basic information and communications technology (ICT) skills are described as including "the use of computers to retrieve, assess, store, produce, present and exchange information, and to communicate and participate in collaborative networks via the Internet" [START_REF] Ala-Mutka | Review of learning in ICT-enabled networks and communities[END_REF].

People can acquire skills through formal, non-formal and informal learning, often making use of multiple methods in acquiring a single skill [START_REF] Galanis | A social framework for supporting, evaluating and validating informal learning[END_REF]. Formal learning follows a course structure and results in certification, non-formal learning is structured but is not formally certified, while informal learning is neither structured nor formally recognized.

Although the skills necessary for FLOSS development have been identified, it is unclear which learning methods are being used to acquire them, especially by people who are not involved in FLOSS development. Our research shows that being a FLOSS code contributor and being a professional software developer have different effects on the learning methods used, and that age is an important factor in how FLOSS skills are acquired. We make use of an internet survey of users and developers of the beta release of a popular file download application to statistically validate the learning methods employed in the acquisition of FLOSS skills by group.

This paper makes three contributions. We confirm that the fact that someone is a FLOSS code contributor tends to make it more likely for the person to have used informal learning to acquire FLOSS skills, as discovered by Ghosh et al. [START_REF] Ghosh | Free/libre and open source software: Survey and study[END_REF]. We determine that being a FLOSS code contributor and being a professional software developer have different effects on the learning methods employed. Finally, we demonstrate that age affects the learning methods used.

Related Work

We identified three topics related to our research: the learning style preferences of individuals and groups, FLOSS and ICT skills acquisition, and comparisons between FLOSS code contributors and professional software developers.

Studies have examined the relationships between learning preferences and many other factors, such as hemisphericity ('right'-or 'left'-brain), social preferences, chronobiology and culture [START_REF] Dunn | Survey of research on learning styles[END_REF]. Age as an explanatory factor has recently attracted attention due to popular press claims that immersion in technology has created a generation of 'digital natives' with a uniquely self-directed and interactive learning style. Little support has been found for the premise in the literature [START_REF] Lai | Technology use and learning characteristics of students in higher education: Do generational differences exist[END_REF]. A preference for informal learning was found in young students [START_REF] Hong | Technology use and digital learning characteristics among Malaysian undergraduates[END_REF], but a preference for the informal techniques was observed irrespective of age in students aged from under 20 to over 30 [START_REF] Lai | Technology use and learning characteristics of students in higher education: Do generational differences exist[END_REF]. In a study of Canadian adults, older people were found to make greater use of independent learning [START_REF] Livingstone | Exploring the icebergs of adult learning: findings of the first Canadian survey of informal learning practices[END_REF].

FLOSS code contributors strongly prefer informal methods such as reading source code and weakly prefer non-formal methods such as participating in workshops over formal study [START_REF] Ghosh | Free/libre and open source software: Survey and study[END_REF]. Contributors already possess many FLOSS skills before joining the community [START_REF] Ghosh | Flosspols: skills survey interim report: 32[END_REF], but younger cohorts do improve skills signif-icantly within the community [START_REF] Glott | FLOSSCom phase 1 report: Analysis of the informal learning environment of FLOSS communities[END_REF]. In contrast with these studies, our research looks at FLOSS code contributors in relation to others.

The learning styles being used to acquire technical skills, regardless of preference, are largely informal and non-formal. One of the largest surveys of skills acquisition indicates that the majority of Europeans acquired their ICT skills informally, through learning by doing, informal assistance from others, self-study materials, or through non-formal courses [START_REF] Ala-Mutka | Review of learning in ICT-enabled networks and communities[END_REF]. In a survey of Canadian adults, computer skills related to employment were among the skills most frequently acquired informally [START_REF] Livingstone | Exploring the icebergs of adult learning: findings of the first Canadian survey of informal learning practices[END_REF]. These studies focus on how skills are acquired, but they do not examine differences between groups. Our research shows that the same skills may be acquired differently by different groups of people.

FLOSS code contributors are often professional software developers or otherwise employed in the ICT sector [START_REF] Arjona-Reina | The FLOSS2013 free/libre/open source survey[END_REF]. They often already possess technical skills before joining a FLOSS project [START_REF] Fang | Understanding sustained participation in open source software projects[END_REF]. However, there are some demographic differences between FLOSS code contributors and professional software developers: FLOSS code contributors are more likely to be male [START_REF] Arjona-Reina | The FLOSS2013 free/libre/open source survey[END_REF] and come from North America or north-western Europe [START_REF] Takhteyev | Investigating the geography of open source software through GitHub[END_REF]. In terms of motivations, most FLOSS volunteers do not differ significantly from paid FLOSS code contributors [START_REF] Lakhani | Why hackers do what they do: Understanding motivation and effort in free/open source software projects[END_REF], with most being motivated by need rather than altruism [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF]. In FLOSS projects, a small percentage of people contribute most of the code [START_REF] Kagdi | Who can help me with this source code change?[END_REF], just as in other joint production communities. Top contributors in such projects participate in fundamentally different ways than others [START_REF] Panciera | Wikipedians are born, not made: a study of power editors on Wikipedia[END_REF], and have different motivations [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF]. Our research compares the effects of being a FLOSS code contributor and being a professional software developer and establishes that there are differences in learning methods used in the acquisition of FLOSS skills.

Theory Development and Hypotheses

We consider three methods of learning-formal, non-formal and informal-which differ on two key attributes: whether it is structured and if it includes certification. Formal learning "refers to the education received from a recognized education center that leads to a certification" [START_REF] Galanis | A social framework for supporting, evaluating and validating informal learning[END_REF]. "Non-formal learning is provided by any organised, structured and sustained educational activity. . . but typically does not lead to certification" [START_REF] Ala-Mutka | Review of learning in ICT-enabled networks and communities[END_REF]. "Informal learning is undertaken on one's own, either individually or collectively, without either externally imposed criteria or the presence of an institutionally authorized instructor" [START_REF] Livingstone | Exploring the icebergs of adult learning: findings of the first Canadian survey of informal learning practices[END_REF].

FLOSS code contributors expressed a strong preference for informal learning and a weaker preference for non-formal learning compared to the formal method of learning [START_REF] Ghosh | Free/libre and open source software: Survey and study[END_REF]. In this, they differ from respondents of another large survey (of government employees) [START_REF] Grundmann | Making the Right Connections: Targeting the Best Competencies for Training[END_REF]. Because the effectiveness of a learning method depends on how well it matches a person's learning style [START_REF] Dunn | Survey of research on learning styles[END_REF], we expect that FLOSS code contributors will make less use of formal learning than other respondents.

Hypothesis 1 a : Being a FLOSS code contributor makes it more likely that FLOSS skills have been acquired via informal learning methods.

Hypothesis 1 b : Being a FLOSS code contributor makes it more likely that FLOSS skills have been acquired via non-formal learning methods.

It has been demonstrated that there are differences between prolific contributors and ordinary contributors in open collaborative projects [START_REF] Shah | Motivation, governance, and the viability of hybrid forms in open source software development[END_REF][START_REF] Panciera | Wikipedians are born, not made: a study of power editors on Wikipedia[END_REF] but there is no indication that the average FLOSS code contributor differs from professional software developers. Indeed, many FLOSS code contributors work in the ICT sector [START_REF] Arjona-Reina | The FLOSS2013 free/libre/open source survey[END_REF]. Therefore we anticipate that being a professional software developer exhibits the same effects on the learning methods used for skills acquisition as being a FLOSS code contributor.

Hypothesis 2 a : Being a professional software developer makes it more likely that FLOSS skills have been acquired via informal learning methods.

Hypothesis 2 b : Being a professional software developer makes it more likely that FLOSS skills have been acquired via non-formal learning methods.

The majority of studies which examined the relationship between age and a preference for learning methods found no indication that age affects preferences [START_REF] Lai | Technology use and learning characteristics of students in higher education: Do generational differences exist[END_REF][START_REF] Byrne | The effect of student attributes on success in programming[END_REF]. While a preference for a particular method does not require that a person acquire a skill using that method, informal methods of acquiring technical skills are readily accessible and we expect that people will make use of their preferred methods of learning when the opportunity exists. We do not expect that age will affect the learning methods used in the acquisition of FLOSS skills.

Hypothesis 3 a : Age does not influence the extent to which FLOSS skills have been acquired via informal learning methods.

Hypothesis 3 b : Age does not influence the extent to which FLOSS skills have been acquired via non-formal learning methods.

Data Sources and Research Method

Data Sources

The primary data source for this paper is an online questionnaire conducted between December 2013 and January 2014 [START_REF] Jahn | Teaching open source competency[END_REF]. The survey was distributed to users of JDownloader 2 Beta via a link in the client interface. JDownloader2 is an open source download management tool used by about 20 million people. A subset of users run the beta version.

The questionnaire was developed as a split survey with seven parts. In all parts, participants were asked common demographic questions and about their FLOSS participation. The distinct portion in each part related to the acquisition of FLOSS skills identified in prior work [START_REF] Ghosh | Free/libre and open source software: Survey and study[END_REF][START_REF] Kimmelmann | Career in open source? Relevant competencies for successful open source developers/Karriere in Open Source? Relevante Kompetenzen für erfolgreiche Open Source Entwickler[END_REF][START_REF] Kimmelmann | [END_REF]. The questions connected with each of the skills examined (see Table 1 in Section 5) were ordered randomly and each appeared in two survey parts. Participants were randomly directed to one of the seven parts, resulting in a different number of responses for each question.

A second data source is the FLOSS 2013 survey [START_REF] Arjona-Reina | The FLOSS2013 free/libre/open source survey[END_REF] on demographics of FLOSS participants, which was conducted in late 2013.

Survey Reliability

An estimated 200,000 people use JDownloader 2 Beta, of which a total of 26,853 people started to answer one of the survey parts and 5,878 continued to the end.

We compared the completed responses to incomplete responses, and found that people who completed the survey were more likely to have engaged in software development and to have participated in FLOSS. Although the questionnaire stressed that the survey was intended for a broad audience, the focus on ICT may have discouraged some respondents.

We further eliminated responses where the participant failed to answer followup questions or where we suspected age misreporting because the response was outside the expected age range (born 1930-2000) of our population, leaving a total of 5,309 responses for the combined survey.

Internal reliability of the survey was demonstrated by computing Cronbach's alpha for the original versions of the skills questions against the control versions. All results were in the range of 0.6 to 0.9, which is considered acceptable.

Survey Representation

In our survey, we provided several ways for people to describe their FLOSS participation. We compared our respondents who selected from the five options which had close representations in FLOSS 2013 by gender, age and income. This comparison involved all FLOSS participants, not just FLOSS code contributors.

Using Pearson's Chi-squared test for gender we determined that there were differences, with our FLOSS participants being less likely to be female (1.4% compared to 11.1%).

To determine age in FLOSS 2013, needed for a t-test, we used the year of initial FLOSS participation and age at the time. Interval values with a range were adapted with random numbers from a uniform distribution within the range. Unbounded intervals were adapted as follows: "before 1960" was set to 1960, "10 or younger" was given a distribution from the set {8, 9, 10} and "55 or older" followed a distribution from {55, . . . , 65}. A Welch two sample t-test showed with 95% confidence that our sample differed. Our sample was younger, as expected from the JDownloader population.

Income was expressed in intervals in both surveys. For a t-test, we converted the observations to values drawn from a uniform distribution within the interval limits and adjusted values to cover the same length of time. The results were consistent with a younger sample: our group had a lower average income.

Survey Design and Modeling Approach

Learning Style. Participants were asked to gauge their mastery of each skill shown in Table 1, by moving a slider between the extremes of "I am not skilled at all" and "I am very skilled." The maximum value corresponded with 10,000 but the numeric value was hidden from the participant. Participants who indicated some measure of skill were subsequently asked to evaluate to what extent various methods were used to acquire the skill. Five options were presented: 'learning in school, university or apprenticeship' (formal); 'reading a book or online tutorial' (informal); 'observing other people perform the activity or the result of their work' (informal); 'participating in workshops or advanced training courses' (nonformal); and 'learning by doing' (informal). Learning styles were also displayed as unnumbered sliders with an effective range of 0-100 and a visible range of "nothing at all" to "all." Informal learning was favored by all participants for all skills, accounting for 62-80% of learning, while non-formal learning had a range of 9-16% and formal learning from 11-26%.

FLOSS Code Contributors.

People who answered positively to the question "Have you ever participated in a FLOSS project?" and subsequently selected one or both of the participation options 'code contributions' and 'project founder' were categorized as FLOSS code contributors. The binary variable FCC was used to indicate if a respondent is a FLOSS code contributor. We observed that FLOSS code contributors differed from the rest of our sample by being younger (by 1.5 years) and less likely to be female (1.5% versus 3.4%).

Professional Developers. Professional software developers were classified by their selection of the answer "I work or worked in software development as part of my job" to the question "Have you worked in software development?" Based on this classification we created an indicator variable, Prof. It should be noted that all possible combination of values for the variables FCC and Prof occurred, giving us four different groups. The smallest group size was 196.

Age. Age was operationalized based on the year of birth, variable YoB, reported in the questionnaire by the respondents.

Technical Knowledge. We created a control variable, TechK, to describe an individual's technical knowledge. It contained the sum of the self-estimates of the mastery of technical skills (2, 5, 7, 11, 12 and 13) in Table 1.

Modeling Approach. For each skill, the vector y = (y a , y b , y c ) observed for a participant was assumed to follow a Dirichlet distribution with expectation π = (π a , π b , π c ) and precision φ, using the alternative parameterization proposed by Maier [START_REF] Maier | DirichletReg: Dirichlet regression for compositional data in R[END_REF]. Here, y a , y b and y c represent the observed relative learning acquired through informal, non-formal and formal learning styles, respectively, while π a , π b and π c denote the corresponding expected values. Choosing formal learning as the reference category, the parameters were modeled to depend on the explanatory variables and the control variable as follows:

ln π j π c = β 0j + β 1j • FCC + β 2j • Prof + β 3j • YoB + β 4j • TechK, j ∈ {a, b}, ln φ = γ 0 + γ 1 • FCC + γ 2 • Prof + γ 3 • YoB + γ 4 • TechK.
After estimating this model based on all observations available for a certain skill, we tested the null hypotheses H 0 (i j ) : β ij ≤ 0 for i ∈ {1, 2} and j ∈ {a, b} using t-test statistics. If the related p value of such a test statistic was smaller than the chosen significance level α, then the null hypothesis could be rejected, indicating support for the respective alternative hypothesis, which was one of the Hypotheses 1 a -2 b formulated in Section 3. In contrast to this, the simple Hypotheses 3 a and 3 b formed the null hypotheses H 0 (3 j ) : β 3j = 0 with j ∈ {a, b}.

Results

When fitting the above Dirichlet regression model for each one of the 17 skills listed in Table 1, only those respondents could be taken into account who met the following conditions: they all had some mastery of the respective skill, they allocated a non-zero value to at least one learning method for acquiring this skill, they gave their year of birth, and they answered the questions necessary for determining their technical knowledge as well their membership in the FLOSS code contributor and professional software developer groups. Table 2 lists the sample sizes N available for the 17 models, as well as the p values p ij obtained when testing the six null hypotheses H 0 (i j ), i ∈ {1, 2, 3}, j ∈ {a, b}. Results significant at a level α of 5% are shown in bold type. Hypothesis 1 a was supported for skills 1-3, 5, 10-13, and 15. Descriptions of the skills can be found in Table 1. As expected, the fact that a person is a FLOSS code contributor tends to increase his/her use of informal learning methods in the acquisition of some FLOSS skills. Hypothesis 1 b was not supported, except for skill 12. Being a FLOSS code contributor does not make it more likely that non-formal learning methods have been used to acquire FLOSS skills.

Hypothesis 2 a and Hypothesis 2 b were not supported. The fact that a person is a professional software developer does not tend to increase his/her use of informal or non-formal learning methods in the acquisition of FLOSS skills.

Hypothesis 3 a was rejected except for skills 13 and 14. Hypothesis 3 b was rejected except for skills 14 and 17. For most FLOSS skills, age influences the use of informal and non-formal methods of learning. More specifically, our results indicated that for all skills where age has an effect, being older is associated with a higher likelihood of having acquired FLOSS skills informally and non-formally.

Discussion and Limitations

The learning methods used by FLOSS code contributors were expected, but the fact that not all skills showed this increased tendency toward informal learning suggests that future work is needed to determine why this variation exists.

There are important implications of the finding that in the acquisition of skills necessary for FLOSS development, being a professional software developer has a different effect from being a FLOSS code contributor. It has generally been assumed that the pool of potential FLOSS code contributors consists of all software developers, but our results suggest that there may be fundamental differences between professional software developers who contribute to FLOSS projects and those who do not. Future research should examine the extent of these differences, in order to determine if it is possible to encourage FLOSS participation among software developers or if only a certain type of personone who makes greater use of independent learning and exploration-is likely to become a FLOSS code contributor.

The effects of age on skills acquisition may reflect the availability of learning, rather than preferences. Previously, there were fewer formal options for acquiring FLOSS skills. This suggests it may not be futile to try to teach FLOSS skills, since they can be acquired through more formal methods. Future research could examine not only the methods by which skills were acquired, but the extent to which the skills were mastered. It should be noted that in our sample the age effect tended to offset the higher preference for informal learning among the FLOSS code contributors, because the FLOSS code contributors were younger.

Our sample was one of convenience optimized for response rate and is not representative of the general population, FLOSS code contributors, or software developers (see Section 4.3). Although this may limit the general applicability of our findings, the results are relevant for young adults, a group which is one of the most important targets for increasing FLOSS skills. Furthermore, as all our respondents came from the same population, our observations about the relative use of informal learning methods by different groups likely remain true.

Conclusions

In this paper, we presented a statistical analysis of a survey on the learning methods employed in the acquisition of FLOSS skills. We found that-unlike being a professional software developer-the fact that someone is a FLOSS code contributor tends to make it more likely for the person to have used informal learning methods to master a number of skills. Moreover, age strongly predicted differences in learning methods, with younger people proving more likely to make use of formal learning.

Our results provide some indication of how companies, FLOSS projects and governments can promote the acquisition of FLOSS skills, but also demonstrate the need for further research on how and to what extent FLOSS skills are learned.

Table 1 :

 1 Skills (skills shown in gray had control questions)

	Skill #	Description
	1	to evaluate the work of others
	2	to work on own software module alone
	3	to communicate with many different target groups
	4	to understand English, especially technical discussion
	5	to document code
	6	to clearly articulate an argument
	7	to understand different software architectures
	8	to show respect for the work of others
	9	to follow discussions on mailing lists
	10	to communicate without offending others
	11	to write code in a way that can be reused
	12	basic/introductory programming skills
	13	to acquaint yourself with code from others
	14	to maintain contact with a community
	15	to coordinate own work with the work of others
	16	to change criticized behavior
	17	to understand and work with people from different cultures

Table 2 :

 2 Sample sizes and test results

Some authors distinguish between competencies and skills while others do not. In this paper the terms are used interchangeably.

http://jdownloader.org/

Acknowledgments

The authors would like to thank Thomas Rechenmacher and JDownloader for promoting the survey and Nicole Kimmelmann for her contribution to the survey design.


Megan Squire
email: msquire@elon.edu

Amber K Smith
email: asmith90@elon.edu

The Diffusion of Pastebin Tools to Enhance Communication in FLOSS Mailing Lists

Keywords: open source, pastebin, email, diffusion of innovations, software development

This paper describes how software developers who use mailing lists to communicate reacted and adjusted to a new supplementary collaboration tool, called a pastebin service. Using publicly-available archives of 8800 mailing lists, we examine the adoption of the pastebin tool by software developers and compare it to the model presented in Diffusion of Innovation (DoI) theory. We then compare the rate at which software developers decided whether to accept or reject the new pastebin tools. We find that the overall rate of pastebin adoption follows the S-curve predicted by classic DoI theory. We then compare the individual pastebin services and their rates of adoption, as well as the reaction of different communities to the new tools and the various rationales for accepting or rejecting them.

Introduction

Software developers working in distributed teams, such as on free, libre, and open source software (FLOSS) projects, have historically used digital tools to communicate with each other about bugs, features, and decisions related to the project. Developers on these types of projects are also often geographically and temporally distributed, making the use of digital media a requirement for communication. Traditionally, the most common digital tool for software development communication has been the email mailing list. Many of these email mailing lists are publicly-viewable and archived for the long term, since FLOSS development relies on a certain level of openness in participation, transparency in decision making, and institutional memory. Apache Software Foundation projects, for example, are required to conduct all official project business on the mailing list (e.g. [1] [2]). Software developers can use email mailing lists to send each other long text artifacts for review, such as bug reports, code snippets, error logs, and the like. However, when compared to newer social media web sites, email mailing lists can seem simplistic. As [START_REF] Storey | The (R)Evolution of social media in software engineering[END_REF] explains, over time, more software development will be conducted by "social programmers" using web sites and apps designed for sharing artifacts, and mailing lists risk obsolescence. For example, while mailing lists can be expressed as a type of primitive social network [START_REF] Bird | Mining email social networks[END_REF], they are not nearly as expressive of social relationships as are microblogging services like Twitter or the pull request system of Github. For reputation management, email lacks the badging and voting features of Q-and-A web sites like Stack Overflow. For collaborative editing, mailing lists are not as easy-to-use as a wiki or a shared code editor.

And yet, email persists as a pillar of "social programming". In the mid-2000s a class of web site was created to facilitate sharing source code via email. A site like this is called generically a "pastebin", after Pastebin.com, one of the first such sites. A pastebin is a web site that allows a user to paste in text and receive back a permanent short URL to the text that was pasted. Some pastebins even include syntax highlighting for common programming languages. This can be an appealing advantage when constructing an email with source code in it, or multiple attached log files, or long error logs, or complicated bug reports. Developers using a pastebin advocate that it enhances the utility of the email mailing list as a social communication tool: it makes sharing text easier and reading more efficient.

For this paper, our research questions center around diffusion of these pastebin tools on software development mailing lists, as follows:

• RQ1: What is the rate of diffusion for pastebins among FLOSS developers using mailing lists? • RQ2: Does the rate of diffusion change between different variants of the innovation? • RQ3: What are the stated reasons for and against adopting this innovation? By studying the rates of diffusion of this pastebin tool, we can better understand how contemporary distributed software development is done, and whether certain tools designed to facilitate social programming on older communication tools will be adopted or not.

To answer these questions, Section 2 gives some background on the pastebin innovation itself and we review the tenets of classical Diffusion of Innovation (DoI) theory. In Section 3 we present our methods for measuring the diffusion of this innovation in the FLOSS developer community, specifically in its email mailing lists. In Section 4, we review the results of this analysis and present our findings for how and why the innovation diffused, including differences across tools. In Section 5 we discuss the implication of our method and analysis on our research questions. In Section 6 we explain limitations of our method and in Section 7 we make recommendations for future study.

Background

Pastebins

As the frequently asked questions (FAQ) document of one popular pastebin tool explains, the website "is mainly used by programmers to store pieces of sources [sic] code or configuration information, but anyone is more than welcome to paste any type of text. The idea behind the site is to make it more convenient for people to share large amounts of text online." [START_REF]Frequently Asked Questions[END_REF] Examples of pastebins include: Pastebin.com, Github Gists, and Paste.org. The innovation of a plain pastebin represents the unification of several previous ideas: a pastebin at this level is simply a very quick way to publish a text document on the web and give it a shortened URL suitable for sharing. Pastebin.com requires no authentication (although that option is available), and the URLs can be set to expire in increments ranging from minutes to "never". The goal is to simplify the publication process from a complicated one involving file transfer and permissions-setting, to one involving only pasting into a web form.

More recently, there is also a class of web sites called an online IDE (integrated development environments) which not only allow the developer to paste in code and provides a link back, but also allows this code to be modified and run in the browser. Examples of online IDEs include CodePen, jsFiddle, and JS Bin (for testing JavaScript code), SQL Fiddle (for testing SQL), PhpFiddle (for testing PHP code), and CodePad (supports a variety of languages).

We should note that because of the anonymity and convenience provided by pastebin sites (and even online IDEs), they have also been used for non-software related purposes, including some illegal or of questionable legality (for example sharing of password lists, posting stolen credit card numbers, high volumes of spam link farming, and the like). See [6] for a description and timeline of some of the more notable illegal activities on Pastebin.

Another important aspect of all pastebin sites is the longevity of the provided URL. Each pasted document is initially given a unique URL, but if the document is allowed to expire, this URL will become a dead link ("link rot"). Pastebin sites have also gone defunct (perhaps due to their site administrators not expecting the high volume of "alternative" uses of their sites, see [START_REF] Kelion | Pastebin to hire staff to tackle hackers' 'sensitive' posts[END_REF] for details about the expense involved in eradicating spam on Pastebin.com) and this means any links are also defunct.

Diffusion of Innovations

We are particularly interested in how these pastebin tools diffused into common usage among software developers. How innovations diffuse (diffusion of innovations, or DoI) has a rich literature, stemming from the work of Everett Rogers [START_REF] Rogers | Diffusion of Innovations[END_REF] who laid the foundation for how to systematically study the process that a new idea goes through as it becomes accepted or rejected by a community. Among Rogers' contributions was to document the "S-curve" that innovations typically go through on their way to becoming accepted or diffused. If we plot time on an X-axis and usage/acceptance of an innovation on the Y-axis, the typical path of an innovation over time will look something like the letter "S". See Figure 1 for an example. The steepness or shallowness of the S will be interesting to the diffusion researcher (as will the lack of an "S" shape, if the innovation was not successful in diffusing at all). Rogers, 1964) DoI research also tends to be concerned with the processes that led to that particular rate of diffusion, including characteristics of the community (or individuals) that would affect the diffusion, such as the compatibility of the innovation with the community's values or the effectiveness of a change agent to champion the innovation. Diffusion can be quite complex, especially with technologies that are themselves complex or networked systems [START_REF] Lyytinen | What's wrong with diffusion of innovation theory? The case of a complex and networked theory[END_REF]. Entities (governments, companies or individuals) with a stake in getting their particular innovation adopted will attempt to understand the myriad variables that can affect the rate or likelihood of diffusion, in order to make that diffusion process more effective.

Once diffusion is determined to have occurred, interesting questions may center around the following: describing the initial discovery of the tool by early adopters, the steps taken by early adopters to encourage others to use the tool, the rejection and refusal by some community members to use the tool, and the expectation-setting and rule-setting by the community governing use of the new tool.

For this project, we are primarily interested in doing the foundational work of calculating the rate of diffusion of pastebin tools within the community of social programmers who use mailing lists to communicate. We also take the first steps toward understanding how early adopters encouraged later adopters to use the pastebin innovation.

Methods

Data Collection

To answer our research questions, we needed first to retrieve the count of times each pastebin tool was mentioned on software development mailing lists over time. The result will be plotted as a rate-of-diffusion curve. To get this data we first identified a source for searching software development mailing lists by keyword. All of our mailing list data came from a publicly-available email aggregation web site called MarkMail.org. MarkMail provides a search interface for approximately 70 million emails from more than 8800 software development mailing lists, in the time period 1992-2014. MarkMail allows broad keyword searches, and also allows searching within typical email headers (e.g. from, subject, and list address).

With MarkMail as a reliable source of email data, we then had to figure out which words were being used to describe pastebin services. The web site Pastebin.com was begun in 2002, and the first mention of either the Pastebin.com web site or the generic noun ("pastebin") on a mailing list was on May 11, 2003 [START_REF] Chapman | PEAR DB_DataObject. On Pear-general list[END_REF]. Though Pastebin.com was probably the first in widespread, public usage [START_REF] Kelion | Pastebin to hire staff to tackle hackers' 'sensitive' posts[END_REF], there are many more pastebin-style websites in existence now.

The generic noun "pastebin" is still used to refer to both the site type ("Use a pastebin to post your code!") and the actual text having been pasted ("I can't find that pastebin you sent"), but other, newer pastebin tool names are often used generically as well (for example, "I'm going to create a gist" or "Did you get my dpaste?"). We therefore needed to search for full or partial web site URLs (e.g. "pastebin.com", "gist.github.com"), and the corresponding generic nouns ("send me a pastebin" or "as you can see from my gist"). To answer our first research question about the rate of diffusion we needed to count all mentions of pastebins, whether by URL or by generic noun.

To identify relevant pastebin tools to use as search terms, we constructed a master list of 38 pastebin web sites and online IDEs that are used in software development. We then searched for each tool in MarkMail to see which were most used by software development teams. The results are shown in Table 1. Most of the tools have the same common name as their URL domain (e.g. jsFiddle). However, two of the pastebin tools, Github Gists and Paste.org, were harder to search for as generic nouns since their relevant stem is already a word in common English usage. Table 2 shows the results for paste and gist as generic nouns, versus their URL-specific versions. To fix these problems, we decided to search for Github Gists using the partial URL gist.github only, and we limited results for the "Paste.org" website to only those messages that included the partial URL paste.org. We recognize that, by doing this, we may have missed some instances of gist or paste used generically to refer to the pastebin tool. We also implemented some basic data cleaning procedures. One mailing list (com.googlegroups. jquery-br) used the words JS Bin and jsFiddle in nearly every signature line on over 50,000 messages sent. We removed these.

For each of the four most frequently used pastebin sites shown in Table 1 (more than 10,000 mentions), we collected the data for usage over time (message counts by month and year). We wrote scripts to download the counts for each term. The scripts are available on Github for anyone to use [11]. The next section describes our analysis of this data.

Data Analysis

We have created several charts showing the count of messages with each of the top four pastebin sites mentioned. Figure 2 (next page) shows the overall percentage of messages mentioning any pastebin site from May 2003 until February 2014. The percentage of mail mentioning any pastebin tool ranges from less than 0.1%, but currently hovers around 0.5% of all mail. Figure 3 shows each particular diffusion line for the different pastebin tools. The X-axis (time) for each sub-graph in Figure 3 differs by when that pastebin site was created. For example, the first mention of Pastebin.com itself was in May 2003 [START_REF] Chapman | PEAR DB_DataObject. On Pear-general list[END_REF], whereas the first mention of Github Gist was in July 2008 [START_REF] Rasmussen | Re: RFC: associations with :accessible => true should allow updating[END_REF]. Each sub-graph in Figure 3 has been smoothed, and each Y-axis shows the keyword as a percent of all mailing list messages in MarkMail (to take into account natural variations in email usage over time by teams in general). These range from <0.1% to nearly 0.3%.

Discussion

Figure 3 shows some of the same steep increases and "S-curve" that is expected in DoI theory for the largest pastebin sites (pastebin.com, Github Gists), and to a smaller extent with jsFiddle. Pastie is comparatively flat throughout its existence. In this section we describe each of the diffusion curves individually, compare their shapes, and give reasons for the differences. [START_REF] Chapman | PEAR DB_DataObject. On Pear-general list[END_REF]. By November of 2003, or one year after its creation, some mailing list users were treating the use of Pastebin.com as a social norm (an expectation for doing business on the mailing list), and some were actually shaming "newbies" for NOT using it [START_REF] Khalid | Re: BTML 2.0 released!!! On PHP-general list[END_REF]: > You need to get a website and post a link to the site instead of posting a giant email > with code that's not even indented and probably wrapped in some places.

Agree with CP here -please don't post large code get some webspace, or use www.pastebin.com; please don't send attachments; please don't send your messages priority, request return receipts; please turn of [sic] vacation auto-responders, please use plain text, and above all else, please don't top post. (where's that weekly newbie email?)

In Rogers' description of the diffusion of innovations, early adopters often become the champions or advocates of the innovation. However, one thing that is interesting about the message in [START_REF] Khalid | Re: BTML 2.0 released!!! On PHP-general list[END_REF] is that not only does this admonishment happen relatively quickly in the life of the pastebin tool, but this particular list (php-general) is a mailing list specifically designed for all types of users, whereas the original place that the tool was diffused was a developer-specific list. New users ("newbies") are apparently expected to know about the existence of this tool very quickly.

Indeed, the "weekly newbie email" to which the email author refers was indeed sent out just the day before the exchange above [START_REF] Brian | Pastebin: How a popular code-sharing site became the ultimate hacker hangout[END_REF], but it did not mention Pastebin.com nor did it even encourage people to point to code on the web by using a URL. In fact, this idea was not officially recommended to newbies on that mailing list until April of 2004, five months after this user was admonished. [START_REF] Kumar | Newbie Guide] For the benefit of new members. On PHP-general mailing list[END_REF] Another interesting pattern we see in the diffusion of Pastebin.com is how rapidly "pastebin" becomes a generic noun, and how quickly copycat web sites spring up. The first mention of a copycat web site is about 13 months after Pastebin.com was first created, and about 7 months after the first use of 'Pastebin.com' on mailing lists [START_REF] Wells | Overload() problem. On PHP-general list[END_REF]. The first mention of pastebin as a generic noun is in a February 2004 message [START_REF] Stepniewski | Twisted Weekly News #11. On Twisted-Python mailing list[END_REF] describing a new tool called "Trash: a pastebin application written using the Twisted framwork" [sic].

Eventually (certainly by 2009) there is some pushback in at least some portions of the developer community to using a pastebin on mailing lists. The common reasons given are, first, that emails with pastebin links are annoying to read because you have to click a separate link. As [START_REF] Brown | Re: 2 drives failed. On Linux-Raid-Vger list[END_REF] begs, ...please (please please) don't use pastebin. Just include the output inline in the mail message. It is much easer[sic] to get at then.

Second, developers noticed that the pastebin URLs eventually time out or disappear ("link rot"), making them unreliable for long-term issue tracking. For example the developer in [START_REF] Rice | On Freeswitch-users list[END_REF] explains, ...Pastebin is a useful tool for sharing information between people, but please do not reference pastebin URLs in Jira Tickets. Pastebin entries are not guaranteed to survive for any period of time, and they can and will disappear at some point. This means of [sic] you open a ticket and you put a backtrace in the pastebin, if the pastebin is cleared so is your backtrace.

Github Gists Discussion

Github introduced its Gists service on July 21, 2008, and the first mention of 'gist.github' on a mailing list was two days later [START_REF] Rasmussen | Re: RFC: associations with :accessible => true should allow updating[END_REF]. Whereas Pastebin.com took six months to diffuse onto mailing lists, the fact that it took Gists two days to diffuse confirms the fact that pastebins were well-understood by developers at this point. Thus the Gists adoption curve tracks upward more quickly than Pastebin (Figure 3). However, we still find evidence of some pro-innovation diffusion behaviors inside the developer community using Gists: developers encouraging others to use Gists when it is first introduced [START_REF] Brown | Re: Where to upload a ruby script to share it? On Ruby-lang Ruby-talk list[END_REF], new users commenting on the fact that they are trying Gists for the first time [START_REF] Stejerean | Re: [Chicago] DVCS Workflows? On Python Chicago list[END_REF] developers or teams setting rules or community procedures around Gists [START_REF] Kaiser | BP #4342] Announcement: BP-Testcase Page (html[END_REF] [START_REF] Leeming | How to use the django-users mailing list properly and doing your homework[END_REF], and new technical reasons for using Gists (e.g. the mailing list is set to reject attachments [START_REF] Ryaboy | Re: java.lang.OutOfMemoryError when using TOP udf[END_REF], or the mailing list software itself rejects messages with code in them [START_REF] Rowe | Re: Implement price range filter: DataImportHandler started. Not Initialized. No commands can be run[END_REF]). Like with Pastebin.com, there is even occasional shaming when others do not use Gists [START_REF] Demeshchuk | Re: riak on one node[END_REF].

Some developers also make the case for Github Gists as an improvement to the mailing list experience in terms of increased "social programming" power [START_REF] Stejerean | Re: [Chicago] DVCS Workflows? On Python Chicago list[END_REF] as follows:

> I've never used pastebins before. I heard about them the other day when I got roundly booed for pasting some code into an irc window. Can someone tell me how they work?

This lets someone see that file with syntax highlighting in a browser, instead of in an IRC window, etc.(…) I think it's kinda cool to be able to collaborate that way with someone who might not know much about git or source control tools in general, better than posting revisions (…) or sending patches around on mailing lists.

jsFiddle Discussion

jsFiddle represents an interesting case of a language-specific (JavaScript) online IDE which also provides static, sharable URLs. As such, jsFiddle does not have as broad a potential developer user base as Pastebin.com or Gists, both of which are language-neutral. Its adoption curve is therefore smaller than the other two (Figure 3). Also, jsFiddle is designed to help a developer code and test in the browser, so it will not be effective for error logs or other non-code messages. Finally, most of the mailing lists indexed by MarkMail are for FLOSS projects that are written in high-level, multi-purpose, non-browser languages (for example Java, Python, Ruby, C++), so they have little use for a language-specific interface-driven tool like jsFiddle. (We also ran into some issues with the way MarkMail collected data that affected jsFiddle, and these are discussed in the Limitations section.) Another study did note the popularity of jsFiddle on Stack Overflow [START_REF] Gomez | A study of innovation diffusion through link sharing on Stack Overflow[END_REF], finding that it was the sixth most common domain linked in postings.

The first mention of jsFiddle on the mailing lists in MarkMail was in January of 2010, but it took a year for jsFiddle mentions to accelerate to more than 100 mentions per month (0.03% of messages). As we mentioned in Section 3.1, we had to clean the data to remove one particular mailing list (jquery-br) because a list-wide email signature line was artificially inflating the numbers for both JS Bin and jsFiddle. However it is worth pointing out that the reason that the list members put that request in their signature lines ("Use JSBIN.COM / JSFIDDLE.NET for code") in the first place was as a way of directing the community towards a preferred behavior. This was the same pattern that we noticed in Pastebin.com, and Github Gists as well: the developers themselves would decide to adopt the innovation and then attempt to encourage others to use it by modeling positive behaviors, social norming and rule-setting, or shaming. Here is an exchange between a new user and a more experienced developer regarding jsFiddle [START_REF] Rpn | Re: Noob Question: Trouble binding computed value to table header[END_REF]:

> Maybe you can get your issue into jsFiddle? Thanks for the suggestion to use jsfiddle. It helped to quickly allow me to test out my solution. You can see it in action here...

Results

Our first research question asks what the rate of diffusion is for pastebins among software developers using mailing lists to communicate. The graph in Figure 2 shows the number of times popular pastebin tools were mentioned in messages to the 8800 mailing lists on MarkMail. The graph shows the classic DoI adoption Scurve: a slow ramp up, followed by a steeper period of increasing usage, which eventually levels out.

Our second question asks whether the diffusion curve differs between different variants of the innovation. Figure 3 shows different shapes to each of the top four pastebin tools. The "first mover" tools either took a long time to ramp up (Pastebin.com) or never quite took off (Pastie). The later-to-market tools were adopted quite quickly and enthusiastically by either a general audience (as with Github Gists) or even in a comparatively narrow niche market (as with jsFiddle).

Our third question is about the stated reasons for and against using this tool to enhance communication on mailing lists. In reading the emails in which developers try to convince their colleagues to adopt the pastebin innovation, they rely on a number of persuasive techniques. First, some developers issue simple entreaties to make everyone's reading experience better. These arguments center on the anti-social aspects of asking for help while simultaneously pasting in thousands of lines of code into a message. We also found pro-pastebin arguments that address the technical downsides to mailing lists, namely that they are overrun with spam and therefore may have been configured to have length limits or rules disallowing attachments.

But we find the most interesting group of arguments are those which claim that a pastebin or online IDE will actually make the programming process more social and therefore more effective. This is especially true in the case of Github Gists (which can be forked, and pull requests issued) and in the case of jsFiddle (which can be edited and the new URL re-sent). In this case, a pastebin is more than a simple link-to-text scheme; it actually adds a layer of social enhancement that did not exist with simple mailing lists.

We also found some anti-pastebin sentiment. A number of developers point out the negative side effects of linking to code rather than archiving it inside the message. Some point out the inconvenience of having to click a link, and many more mention the risk that the link may be broken over time and is therefore unreliable as a record of what has been done in the code and why. However, based on the diffusion curves, this argument does not seem to have much "won" on developer mailing lists as a whole.

Limitations

Definition of Diffusion. Our method of calculating the diffusion rate may differ from the traditional interpretation of "diffusion" because of our use of message counts and not sender counts in our calculation of the adoption rate. In the traditional DoI literature, diffusion is usually defined as "number of people who adopt an invention". In the realm of email, adoption can be described as the number of senders who mention the tool, or it could be the number of times (emails) a sender mentions the tool. Table 3 outlines the differences in calculating diffusion using these two different methods. In this paper we choose to measure pm/m (method 1) rather than ps/s (method 2). We determined that it is actually more accurate to count messages rather than senders because for example, a single sender (e.g. "bugzilla@redhat.com") may actually represent an unknown number of other developers on the project. For example, suppose five different developers submit their bugs to a Bugzilla bug-tracking system, and they all used a pastebin for the error log or code sample that they are attaching to the bug tracking email. These five bug reports will all be forwarded to the mailing list as a single sender: bugzilla@company.com. Our method recognizes these messages as representing five instances of pastebin use, rather than just one. We also acknowledge that it is considerably easier to measure messages than to disambiguate senders (who may have multiple email addresses or different aliases).

Pre-Pastebin Code-via-URL Sharing. One other limitation of our study is that we do not have a good way for finding out how widespread the practice was of sharing code-via-URL prior to Pastebin.com. The [START_REF] Khalid | Re: BTML 2.0 released!!! On PHP-general list[END_REF] message shows that in November of 2003 there is at least some acknowledgement that it is possible (and may also be desirable) to create a web space for code snippet or log message rather than posting it in the email message. However, in this paper we do not examine how prevalent that practice was. It would be quite challenging to gather this count, since any URL would be a potential code location.

MarkMail Limitations. As a source for keyword searching through very high volumes of emails, MarkMail is a good (but not perfect) source. We recognize that MarkMail does not collect every FLOSS mailing list to begin with, and we acknowledge that some of its lists are not about FLOSS. More important for this project, we noticed that MarkMail may occasionally stop collecting from a mailing list abruptly, even though the list is still being used. We had some issues with MarkMail stopping the collection of one small mailing list in June of 2013. This abrupt stoppage affected our graph for jsFiddle since that one, small mailing list did include a very high number of jsFiddle users. (We turned in a support ticket to MarkMail about why they dropped the "jsknockout" mailing list from its collection, but it has so far gone unanswered.) Still, for searching a large number of lists over a long period of time, MarkMail is useful.

Future Work

We are excited about some avenues of future work, particularly in studying pastebin tools and the innovation curve on two additional sources of FLOSS social programmer communication: IRC and Stack Overflow. These two tools are heavily used by active social programmers, especially in developing FLOSS and in distributed programming teams of all kinds. They are each very different ways of communicating, and have their own traditions, expectations, and culture. We suspect that the diffusion curves of the pastebin tool will look very different on each of these. Early evidence shows that pastebin tools caught on very quickly on IRC, in some cases producing an even sharper diffusion curve than on mailing lists. In contrast, we have encountered evidence of much more resistance to usage of pastebin tools in certain Stack Overflow postings but not in others. We suspect that this resistance is grounded in fear of link rot, as dead links are considered substantially more serious on a long-term reference site (which Stack Overflow aims to be) than on IRC and mailing lists. We also notice that some sub-communities within Stack Overflow are more accepting of pastebins (and online IDEs in particular) than other sub-communities. We are in the process of analyzing the IRC and Stack Overflow data and comparing it to mailing lists in order to provide evidence for how pastebins diffused on these other communication channels, and why.

Conclusion

Studying pastebin adoption on developer communication channels, mailing lists in this case, can further our understanding of how innovations diffuse in general, and in particular we can learn about how social media are impacting the traditional communication tools used by software developers. Will developers adopt a "social programming" innovation in order to enhance a tool that has already been in use for nearly forty years? In this paper we were able to confirm that the overall pastebin adoption does follow the S-curve of traditional DoI theory. Within particular tools, we find some differences in rates of diffusion depending on whether the tool being adopted was one of the earlier ones or one of the later ones. Most interestingly, we find that the reasons given for adopting the innovation include both very practical enhancements to the email communication medium, as well as attempts to improve the social aspects of collaborative coding.

Fig. 1 .

 1 Fig. 1. Typical Diffusion of Innovations curve (modeled afterRogers, 1964)

Fig. 2 .Fig. 3 .

 23 Fig. 2. Percent of mailing list messages making reference to any pastebin, 2003 -2014

  The curve for diffusion of Pastebin.com is shown in the top bar of Figure 3. The web site for Pastebin.com claims that it was started in 2002, and indeed the first Pastebin.com snapshot [13] on the Wayback Machine [14] was taken on November 23, 2002. But the first mention of Pastebin.com on any of the email mailing lists was six months later on May 11, 2003

Table 1 .

 1 Top 12 Most Used Pastebin Websites onMarkMail (2003MarkMail ( -2014) )

	Rank	Pastebin Website	Message Count Rank	Pastebin Website	Message Count
	1	Pastebin.com	64009	7	Codepad	847
	2	Github Gists	39557	8	Paste.org	467
	3	Pastie	13397	9	Codepen	224
	4	jsFiddle	10536	10	IDEone	196
	5	Dpaste	6740	11	SQLFiddle	54
	6	JS Bin	2416	12	all others	< 50 ea.

Table 2 .

 2 Comparing the message count for 'gist' and 'paste' as words and as URLs

	Original Keyword	Message Count	New Keyword	Message Count
	gist	68706	gist.github	39557
	paste	674760	paste.org	467

Table 3 .

 3 Two different methods of counting diffusion of pastebins on mailing lists

	Method 1	Method 2
	pm = count of messages mentioning a tool	ps = count of senders mentioning a tool
	m = count of all messages	s = count of all senders
	diffusion=pm/m	diffusion=ps/s