Dirk Homscheid
email: dhomscheid@uni-koblenz.de

Jérôme Kunegis
email: kunegis@uni-koblenz.de

Mario Schaarschmidt
email: mario.schaarschmidt@uni-koblenz.de

Private-Collective Innovation and Open Source Software: Longitudinal Insights from Linux Kernel Development

Keywords: Open Source, Open Source Community, Private-Collective Innovation, Linux Kernel

While in early years, software technology companies such as IBM and Novell invested time and resources in open source software (OSS) development, today even user firms (e.g., Samsung) invest in OSS development. Thus, today's professional OSS projects receive contributions from hobbyists, universities, research centers, as well as software vendors and user firms. Theorists have referred to this kind of combined public and private investments in innovation creation as private-collective innovation. In particular, the private-collective innovation model seeks to explain why firms privately invest resources to create artifacts that share the characteristics of non-rivalry and non-excludability. The aim of this research is to investigate how different contributor groups associated with public and increasing private interests interact in an OSS development project. The results of the study show that the balance between private and collective contributors in the Linux kernel development seems to be changing to an open source project that is mostly developed jointly by private companies.

Introduction

Open source software (OSS) has changed how researchers and practitioners look at software development and related business models [START_REF] Bonaccorsi | Entry Strategies Under Competing Standards: Hybrid Business Models in the Open Source Software Industry[END_REF][START_REF] Capra | Firms' Involvement in Open Source Projects: A Trade-Off between Software Structural Quality and Popularity[END_REF][START_REF] Riehle | The Single-Vendor Commercial Open Source Business Model[END_REF]. OSS differs from traditional in-house software development, as its outcome, the software, is freely accessible by anyone. As a result, major companies such as Facebook, Google and Twitter use open source technologies. In 2012 one million open source projects were catalogued and this figure was projected to double within two years [START_REF] Guignard | Ten Facts and Figures About Open Source Software[END_REF]. In addition, according to IDC research, the OSS market should be worth approx. $ 8 billion in 2013 [START_REF]IDC: Worldwide Open Source Software 2009-2013 Forecast[END_REF]. Due to this success, firms have also started to actively engage in OSS development [START_REF] Grand | Resource Allocation Beyond Firm Boundaries: A Multi-Level Model for Open Source Innovation[END_REF][START_REF] Schaarschmidt | Divide et Impera! The Role of Firms in Large Open Source Software Consortia[END_REF]. While in early years, software technology companies such as IBM and Novell invested time and resources in OSS development, today even user firms (e.g., Samsung) invest in OSS development [START_REF] Corbet | Linux Kernel Development. How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It[END_REF]. Thus, today's successful OSS projects receive contributions from hobbyists, universities, research centers, as well as from software vendors and user firms [START_REF] Teigland | Balancing on a Tightrope: Managing the Boundaries of a Firm-Sponsored OSS Community and its Impact on Innovation and Absorptive Capacity[END_REF][START_REF] Schaarschmidt | Firms' Resource Deployment and Project Leadership in Open Source Software Development[END_REF]. Theorists have referred to this kind of combined public and private investments in innovation creation as private-collective innovation [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF]. This concept asserts that a private investment model -where firms create and commercialize ideas themselves -and a collective invention model, where multiple economic actors create public goods innovations, may coexist under certain circumstances [START_REF] Stürmer | Extending Private-Collective Innovation: A Case Study[END_REF]. In particular, the private-collective innovation model seeks to explain why firms privately invest resources to create artifacts that share the characteristics of non-rivalry and nonexcludability [START_REF] Alexy | Private-Collective Innovation, Competition, and Firms' Counterintuitive Appropriation Strategies[END_REF].

The private-collective model also implicitly assumes that private and public investments in innovations are approximately equal. However, successful OSS projects receive more than 75% of their code from contributors 1 who are paid by a company [START_REF] Corbet | Linux Kernel Development. How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It[END_REF] and the majority of code is written between 9 am and 5 pm -again indicating that contributions are predominantly provided by firms [START_REF] Riehle | Paid vs. Volunteer Work in Open Source[END_REF]. These figures contrast with the picture of private-collective innovation as an invention mode where public and private interests manifest equally. The aim of this research therefore is to investigate how different contributor groups associated with public and increasing private interests interact in an OSS development project.

In order to study the interplay of both interest groups we not only need to consider demographic characteristics of the community but also the structural patterns of interactions in it. To achieve this goal, we analyze developers active in the Linux kernel (LK) development community from a social network point of view, as the interaction between the members of a software development community reflects the structure of their collaboration. In particular, we investigate degree distributions and the Gini coefficient in the contributor network with respect to the private and collective contributor groups. Network centrality measures are important indicators of influence in OSS development and are known to deviate according to having firm sponsorship or not [START_REF] Dahlander | A Man on the Inside: Unlocking Communities as Complementary Assets[END_REF].

We start with detailing what volunteer and firm-sponsored (i.e., employed) developers motivate to participate in OSS development. Then, we discuss the private-collective innovation model in more detail. Based on a dataset of mailing list communication of LK developers from 1996 to 2014 we calculate network measures for each type of developer (e.g., firm-sponsored, hobbyist, university-affiliated, etc.) and compare them for each year. We discuss implications for research and provide further avenues for research concerning private-collective innovation.

2

Theoretical Background

Open Source Software Contributors

OSS Communities

An OSS project relies on contributors who make up the core element of an OSS community. OSS is commonly understood as a type of software that can be used, changed, and shared by any person. The software itself is in most cases developed by a heterogeneous group of people and distributed under specific licenses, which guarantee the above-mentioned characteristics of OSS [START_REF]Open Source Initiative: The Open Source Definition[END_REF].

In general, a community arises when different people come together and share a common interest [START_REF] Preece | Online Communities. Designing Usability and Supporting Sociability[END_REF]. Thus, von Hippel and von Krogh [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF] conceptualize OSS development communities as "Internet-based communities of software developers who voluntarily collaborate in order to develop software that they or their organizations need" [11, p. 209]. Besides the fact that OSS communities consist of hobbyists, who voluntarily provide their resources to the community, the definition also involves another important contributor group -organizations. Organizations differ from hobbyists in terms of their motivation to engage and are represented in the community by their employed developers. In turn, employed developers might be considered as proxies for firm interests in the community.

Motivation of Voluntary OSS Developers

The pertinent literature specifies intrinsic and extrinsic motivation as major drivers for hobbyists to engage in OSS projects (e.g., [START_REF] Hars | Working for Free? Motivations for Participating in Open-Source Projects[END_REF][START_REF] Lakhani | Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects[END_REF][START_REF] Roberts | Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects[END_REF][START_REF] Wu | An Empirical Analysis of Open Source Software Developers? Motivations and Continuance Intentions[END_REF]). Intrinsic motivation is the execution of an activity due to the accompanying enthusiasm and not for the achievement of specific results [START_REF] Ryan | Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions[END_REF]. A behavior is extrinsically motivated when an activity is performed for reward, recognition or because of an instruction from someone or an obligation [START_REF] Ryan | Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions[END_REF]. Although researchers agree on different forms of intrinsic and extrinsic motivation, there is often disagreement about their relevance. The most relevant forms of both motivation types in the context of OSS developers are described briefly in the following.

In connection with OSS developers, researchers investigated a plethora of intrinsic motivators. Among these, joy-based intrinsic motivation is the strongest and most prevalent driver of OSS contributors [START_REF] Lakhani | Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects[END_REF]. Joy-based motivation is closely linked to the creativity of a person. Frequently, contributors to OSS projects have a strong interest in software development and related challenges [START_REF] Hars | Working for Free? Motivations for Participating in Open-Source Projects[END_REF].

Another fundamental aspect of intrinsic motivation is altruism, which is the desire to help others and to improve their welfare. In OSS communities, developers code programs, report bugs, etc., at their own expense, which includes the invested time and opportunity costs. They participate in the OSS community, without taking advantages of its outcome [START_REF] Hars | Working for Free? Motivations for Participating in Open-Source Projects[END_REF][START_REF] Wu | An Empirical Analysis of Open Source Software Developers? Motivations and Continuance Intentions[END_REF][START_REF] Baytiyeh | Open Source Software: A Community of Altruists[END_REF].

In addition, the OSS ideology plays a crucial role for many contributors and involves

• joint collaborative values, such as helping, sharing and collaboration,

• individual values, such as learning, technical knowledge and reputation,

• OSS process beliefs, such as code quality and bug fixing and • beliefs regarding the importance of freedom in OSS, such as an open source code and its free availability and use for everyone [START_REF] Stewart | The Impact of Ideology on Effectiveness in Open Source Software Development Teams[END_REF].

Besides these distinguishing aspects of participants' intrinsic motivation to engage in OSS projects, researchers have found that extrinsic stimuli can also have an impact on the activities of actors in communities (e.g., [START_REF] Hars | Working for Free? Motivations for Participating in Open-Source Projects[END_REF][START_REF] Lakhani | Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects[END_REF][START_REF] Roberts | Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects[END_REF][START_REF] Wu | An Empirical Analysis of Open Source Software Developers? Motivations and Continuance Intentions[END_REF]). An extrinsic stimulus is given through a personal need of a developer. Many OSS projects are launched because the initiators needed software with specific functions that are not available to date, and they have the willingness and knowledge to develop these [START_REF] Wu | An Empirical Analysis of Open Source Software Developers? Motivations and Continuance Intentions[END_REF].

OSS communities offer the possibility for developers to improve their programming skills and their knowledge through participation in a project. Programmers are free to choose in which tasks they participate according to their interests and abilities. As a result, the self-learning participants are experiencing a continuous learning curve and build a repertoire of experiences, ways and means to solve specific software development tasks [START_REF] Wu | An Empirical Analysis of Open Source Software Developers? Motivations and Continuance Intentions[END_REF].

In addition, "signaling incentives" as described by Lerner and Tirole [START_REF] Lerner | Some Simple Economics of Open Source[END_REF] can also be a reason for people to participate in OSS communities. The incentives cover, inter alia, the recognition by other members of the community and the improvement of the professional status.

Motivation of Firms Involved in OSS Development

In addition to hobbyists, companies are also active in OSS communities. While voluntary OSS contributors are driven by intrinsic and extrinsic values, economic and technological aspects motivate firms to participate in OSS projects. In recent times, companies open outwards to organize their innovation activities more effectively and efficiently. A means to complement their own resource base are innovation communities. In the case of software companies, OSS communities form a resource pool these firms can benefit from -depending on the strategy they pursue [START_REF] Grand | Resource Allocation Beyond Firm Boundaries: A Multi-Level Model for Open Source Innovation[END_REF][START_REF] Dahlander | How do Firms Make Use of Open Source Communities?[END_REF]. Literature investigating motivational aspects of companies active in OSS projects reveals that economic theory is not sufficient to explain the relation between firms and their OSS engagement. Andersen-Gott et al. [START_REF] Andersen-Gott | Why do Commercial Companies contribute to Open Source Software[END_REF] have reviewed this issue and identified the following three categories of motivational factors that are relevant for companies active in OSS communities.

Innovative capabilities

If the involvement of a company in an OSS project is aligned with the business model it maintains, the interaction with the community can lead to better or new products which imply a competitive advantage. The inclusion of external contributors increases the firm's innovative capacity.

Complementary services

The dominant way for firms to appropriate from OSS is by providing complementary services to customers (e.g., training, technical support, consultancy and certifications [START_REF] Fitzgerald | The Transformation of Open Source Software[END_REF]) aligned with their business strategy [START_REF] Dahlander | Appropriation and Appropriability in Open Source Software[END_REF]. Firms pursuing this concept deploy own employees that also contribute to the open source project and community work. Thus, the company (1) acquires external knowledge through their own employees active in OSS development and (2) has access to complementary resources in the community, which are difficult to replicate internally [START_REF] Riehle | The Economic Motivation of Open Source Software: Stakeholder Perspectives[END_REF].

Cost reduction

Companies can publish the source code of their proprietary software under an OS license, try to attract external developers and build a community around the software. In this case, the company will get, for example, ideas for new features, bug reports, documentation, and extensions of the software from external contributors without having to pay for it [START_REF] Lerner | Some Simple Economics of Open Source[END_REF][START_REF] Henkel | Open Source Software form Commercial Firms: Tools, Complements, and Collective Invention[END_REF]. Further, in the long run, the code is maintained by the community, such that the firm has lower costs than its competitors with proprietary software [START_REF] Hawkins | The Economics of Open Source Software for a Competitive Firm[END_REF]. However, it should be noted that establishing an ecosystem and an active community around released source code is no easy task as rivals could pursue similar strategies [START_REF] Dahlander | How do Firms Make Use of Open Source Communities?[END_REF][START_REF] Ågerfalk | Outsourcing to an Unknown Workforce: Exploring Opensourcing as a Global Sourcing Strategy[END_REF].

Private-Collective Innovation

In organization science, two different modes of innovation are dominant, namely the private investment and the collective action model.

The private investment model is associated with a rather closed innovation behavior. Innovators tend to protect their internally developed proprietary knowledge as this is the source of their profits and competitive advantage [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF]. Here innovation is clearly seen as a closed process driven by private investments in order to lead to private returns for the innovator [START_REF] Liebeskind | Knowledge, Strategy, and the Theory of the Firm[END_REF].

The collective action model of innovation is connected to the provision of a public good. Innovators collaborate in order to develop a public good under conditions of market failure. The produced good is characterized by non-excludability and non-rivalry [START_REF] Olsen | The Logic of Collective Action[END_REF]. This model requires that innovators supply their collected knowledge about a project to a common knowledge base and thus make it a public good. This innovation method can unfortunately be exploited by free riders, who wait until other contributors have done the work and use the outcome for free [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF][START_REF] Olsen | The Logic of Collective Action[END_REF]. OSS communities are an example for a mixture of both mentioned innovation models. OSS contributors freely reveal their privately developed source code as a public good. The developers do not make commercial use of their property rights, although the source code is created as a result of private investments. This innovation behavior is termed private-collective innovation [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF]. To get a deeper understanding of how OSS communities combine the best of both models, OSS innovation is first considered from the private investment and second from the collective action point of view.

From the private investment model perspective, OSS deviates in two major aspects from the conventional private investment model. First, software contributors are the actual innovators in OSS rather than commercial software developers, because they create software that is needed either by themselves or by the community. Second, OSS developers freely reveal the source code, which they have developed by private means; this manner stands in contrast to the classical innovation behavior. Due to the lack of a commercial market for the sale or licensing of OSS, it is made openly available as a public good [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF]. Rewards for the developers are provided in forms other than money or commercialization of property rights. Contributors gain private profits such as reputation, experience or reciprocity [START_REF] Lerner | Some Simple Economics of Open Source[END_REF][START_REF] Krogh | The Communal Resource and Information Systems[END_REF].

From the collective action model view, the community produces a public good with its attributes of non-excludability and non-rivalry [START_REF] Olsen | The Logic of Collective Action[END_REF]. Taking the above given description of the collective action model into account, the non-excludability would bring a dilemma with it because free riders benefit from the software but do not contribute to the good compared to the developers. This circumstance is not a problem, as in line with the OS ideology people voluntarily participate in OSS development and share the results without costs [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF]. Moreover, contributors obtain benefits, for example problem solving expertise, learning and enjoyment, from the participation on developing a public good, which the free rider cannot get [START_REF] Lerner | Some Simple Economics of Open Source[END_REF][START_REF] Raymond | The Cathedral & the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary[END_REF]. The benefits in form of selective incentives are connected to the development process of the good and thus only accessible for the participants. Therefore, OSS contributions cannot be seen as pure public goods as these have significant private elements that evolved out of the ideology, which support the community [START_REF] Hippel | Open Source Software and the "Private-Collective" Innovation Model: Issues for Organization Science[END_REF].

In sum, the private-collective model of innovation combines the advantages of both private investment and collective action model. Table 1 compares the most important aspects of the three innovation models from an economic perspective and in relation to OSS development.

Method

Research Context

To find a relevant OSS project for our research, we have taken different aspects into account (e.g., size of the project, activity and continuity, company involvement, availability of a large set of data). Finally, we chose the LK project as our research context, which has served as an example for OSS in many previous studies [e.g., 40].

The LK project was initialized by Linus Torvalds in 1991 and has been one of the most active OSS projects since its beginning. There are software releases every three month on average, which are possible because of the fast-moving development process and the broad foundation of contributors, ranging from hobbyists to companies. Thus, it involves more people than any other OSS project. The kernel itself makes up the core component of any Linux system and is used in operating systems for mobile devices right up to operating systems for supercomputers. Typically, a new release of the kernel comprises over 10,000 patches contributed by over 1,100 developers representing over 225 companies and is published under the GNU General Public Licence v2 [START_REF] Corbet | Linux Kernel Development. How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It[END_REF].

Besides the fact that the LK is one of the largest cooperative software projects ever started, it has also an economic relevance, as many companies have business models that rely on the LK or on software working on top of the LK, respectively. Many of these companies do actively participate in the improvement of the kernel and thereby take effect on the orientation of the development. Very active companies in the kernel development, among others, are RedHat, Intel, IBM, Samsung, Google and Oracle [START_REF] Corbet | Linux Kernel Development. How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It[END_REF].

Data Collection and Coding of Contributor Categories

To obtain the data needed for our research, we crawled the LK mailing list web archive 2 . We use mailing list data as it is suitable to calculate network positions that represent developers' influence [START_REF] Dahlander | A Man on the Inside: Unlocking Communities as Complementary Assets[END_REF]. The "linux-kernel"3 mailing list has the purpose of discussing LK development topics as well as of reporting bugs. The observation period of the LK community ranges from 1996 (beginning of the web archive) to 2014.

We identified actors that occur multiple times on the list, for example with different email addresses, but identical sender names. We have mapped these to one person object related to the email address s/he has used when sending a message to the list for the first time.

The identified people interacting in the LK mailing list act partly on behalf of companies. To get a deeper understanding if the actors in the mailing list are affiliated with a firm we used the domain name of the email addresses to assign people to a contributor category. Developers sending messages from a domain indicating that the person is employed by the corresponding company are classified as employed contributors, whereas people using email addresses from public email providers such as yahoo.com were classified as hobbyists. Likewise, we identified developers with email addresses indicating universities and research institutions. Assigning LK actors to a contributor category was done in a semi-manual and semi-automated process in order to obtain a high accuracy of the attributions. Detailed information about the different contributor categories is provided in Table 2. The cleaned dataset comprises 1,941,119 communication replies for the total time period with overall 86,509 contributors involved. The overall distribution of the contributor groups is made of 37.96% of company developers, hobbyists represent 51.22%, universities account for 9.65% and research institutions make up 1.17%. Descriptive information about the dataset is given in Figures 1 and2. Figure 1 shows the quantity of identified contributors per contributor group and year from 1996 to 2014. Figure 2 states the amount of messages sent per contributor group and year for the investigated period.

Fig. 1. Contributors per group and year

Fig. 2. Amount of messages sent per group and year

Social Network Analysis

A social network represents persons connected by edges. Social networks can represent friendship relationships, communication, interaction contacts or other types of social relationships. Social network datasets are widely used, not only in the area of social network analysis, but also in the areas of data mining, sociology, politics, economics and other fields [START_REF] Wasserman | Social Network Analysis. Methods and Applications[END_REF].

In order to study the interactions of developers in the LK community, we perform an analysis of the LK mailing list's communication network. Communication within the Linux developer community can be modelled as a directed network, in which nodes are developers and directed edges (i.e., arcs) are a reply of one developer to another. In our dataset, we ignore all messages that are not replies to other developers. A relationship between the sender of a starter message and others does only emerge when one or more people reply to the starter message. We perform a structural analysis of this network to study the interplay of developers interacting.

The directed network of replies we consider is annotated with two additional metadata:

• For replies, the posting timestamp is known. This allows us to make a longitudinal analysis of the considered network statistics. • For developers, we know their company, university or other affiliation, if any, allowing us to identify four categories of developers, as described in Section 3.2.

We perform social network analysis with Matlab and the KONECT Toolbox [START_REF] Kunegis | KONECT -The Koblenz Network Collection[END_REF].

The contribution of one user in a directed social network can be used to measure both the activity and the importance of that user in the community. We achieve this by considering the following network-based measures, each of which is defined for individual nodes:

• The in-degree of a node equals the total number of replies received by a developer. The in-degree can thus be interpreted as a measure of importance of a developer.

• The out-degree of a node equals the total number of replies written by a developer, and can thus be interpreted as a measure of the activity of a developer. • As a network-wide measure, we additionally define the Gini coefficient of the indegree distribution [START_REF] Kunegis | Fairness on the Web: Alternatives to the Power Law[END_REF], which denotes the inequality of the in-degrees. It is zero when all developers have equal in-degrees and one when a single developer received all replies. It can thus be interpreted as a measure of diversity of the community [START_REF] Kunegis | Diversity Dynamics in Online Networks[END_REF].

Results

Comparison of In-degree and Out-degree

In a first analysis, we compare the in-degree and the out-degree of all developers, i.e., the number of replies given vs. the number of replies received. Figure 3 shows the results of this analysis. We can observe that both measures are highly correlated -developers who receive many replies also write many replies. Thus, for the LK community the activity and the importance of developers correlate highly.

Fig. 3.

Comparison of in-degree and out-degree

Comparison of Degree per Group

In this analysis, we want to find out whether the developer-based measures of activity and importance vary from one group to another. We compute the distributions of outdegree and in-degree, for each group for the whole dataset aggregated over all years.

The results are shown in Figures 4 and5. The plots show that:

• The highest activity as measured by the out-degree is achieved by company developers, then hobbyists, and the lowest activity is given by developers from research institutions and universities. • The measure of importance, the in-degree, correlates and shows the same pattern as for the activity: company developers have the most importance, then hobbyists, and finally developers from research institutions and universities. These results are consistent with the observation that the measures of activity and importance correlate.

To verify the statistical significance of our results, we perform pairwise Mann-Whitney U tests, testing whether values of each statistic for one type of developer are statistically different from the values for another group. The group differences are statistically significant (p < 0.05; company developers vs. hobbyists: p < 0.10 for the outdegree), except for developers from companies vs. developers from research institutions and hobbyists vs. developers from research institutions for the in-degree and outdegree; developers from universities vs. developers from research institutions for the out-degree.

Longitudinal Analysis

In order to study the change of the community over time, we compute three group-wide measures of activity and importance for each individual year in the range 1996 to 2014.

• The average value of the out-degree and the in-degree of all developers in each group, restricted to all replies given and received, respectively during a given year. • The Gini coefficient of the in-degree distribution of all developers of a given group, restricted to all replies received during a given year. The results of the analysis are shown in Figures 6,[START_REF] Schaarschmidt | Divide et Impera! The Role of Firms in Large Open Source Software Consortia[END_REF] The average out-degree and in-degree (Figures 6 and7) show a consistent result with the degree distribution shown in Figures 4 and5. The average out-degree standing for activity of the developers of the different groups increases for the developers from companies as well as hobbyists over time and does not chance significantly for the developers of the other contributor categories. The measure of importance, the in-degree, shows a similar behavior. The values for the developers from companies and hobbyists increase and do not change significantly for the other types of developers. The network wide measure Gini coefficient (Figure 8) decreases slightly for developers from companies in the last years. The high fluctuation of the Gini coefficient for universities and research institutions is related to the small group sizes.

Across all developer groups and times, the Gini coefficient is very near to one (> 98%, up to fluctuations). This value is higher than in the large majority of social networks [START_REF] Kunegis | Fairness on the Web: Alternatives to the Power Law[END_REF], indicating that the importance in the community is concentrated in a very small number of actors when compared to other typical social networks.

Discussion and Conclusion

Discussion

This study provides results of an activity analysis of different contributor groups in the LK development from 1996 to 2014. The aim of this study was to investigate how different contributor groups associated with public and increasing private interests interact in an OSS development project. To achieve this goal, we analyze developers active in the LK development community from a social network perspective, as the interaction between members of a software development community reflects the structure of their collaboration.

The first result of our analysis shows that the out-degree, as a measure for activity, and the in-degree, as a measure of importance, correlate highly both in general and for each contributor group individually. Thus, developers who write many replies to the mailing list also receive many replies. This phenomenon in the LK mailing list differs from forum communication, where a variety of user roles with different communication patterns can be identified [START_REF] Chan | Decomposing Discussion Forums using Common User Roles[END_REF]. Both the highest activity and importance, is achieved by company developers followed by hobbyists. The lowest activity and importance can be attributed to developers from research institutions and universities. Connecting these results to the amount of contributors per contributor category (Figure 1) it can be seen that although the amount of developers from companies is less than the amount of hobbyists the impact made by employed developers on the LK community is larger. The mentioned impact is expressed by the amount of messages sent per contributor group and year (Figure 2) as well as stated by the measures of activity (Figure 6) and importance (Figure 7), clearly seen especially from 2010 on.

The second result of our analysis shows that the Gini coefficient, as a network wide measure, decreases in recent years for developers from companies and remains constant for the other groups. Although the decrease is small, this can be seen as a tendency that the importance in the community for the group of developers from companies is distributed to more actors. When considering the early years of the LK development (from 1996 until 2000) it can be seen that university members were the most active and important force in the project (Figure 6,7). From 2001 the activity and importance of hobbyists and companies increases alike. Although the amount of company contributors remains relatively stable for the period starting from 2010 until 2014 (Figure 1) the activity (Figures 2,6) and importance (Figure 7) in that time increases sharply.

Conclusion

The aforementioned observations help to answer the research question of how different contributor groups associated with public and increasing private interests interact in an OSS development project, here the LK development. In the beginning, the LK project was driven by intrinsic motivated enthusiasts who are hobbyists and university members (e.g., students) and are associated with pure collective interests. As software and services that were built on top of the LK got more and more influence, the participation of firms, for example, motivated by offering complementary services (see Section 2.1), increased. As a consequence the private interests in this project increased, too, especially from 2010 on with the diffusion of mobile devices powered by Android [START_REF]Google: Android @ I/O: the playground is open. Official Google Blog[END_REF], which uses the LK as foundation of the operating system. Summarizing the results of this study, it can be concluded that the balance between private and collective contributors in the LK development seems to be changing to an open source project that is mostly developed jointly by private companies. These firms need the kernel for their products and services. Thus, the LK project is no longer just an open source alternative for hobbyist to develop open source software for the reason not be locked in by proprietary software. The advantages for the participating companies outweigh the drawbacks in terms of collective copyright ownership and less control in the project (see also Section 2.2, Table 1) as the LK community can be utilized to complement their own resource base for innovations.

Implications for Research

Our findings can be classified into the context of private-collective innovation. The LK development project is an outstanding OSS project. The engagement of companies has increased in the last years, as more and more firms have business models that rely on the kernel. Although companies cannot dictate what the community should do, they can in a way influence the trajectory of the project by assigning employed developers to the project [START_REF] Schaarschmidt | Firms' Resource Deployment and Project Leadership in Open Source Software Development[END_REF][START_REF] Schaarschmidt | How Do Firms Influence Open Source Software Communities? A Framework and Empirical Analysis of Different Governance Modes[END_REF]. As the results of our study for the LK project show, employed developers can take key positions in a community due to the intensity of the commitment, expressed by activity and importance, for the community. With this in mind, future research should more thoroughly discuss the nature and structure of firm presence in OSS development. The majority of early research on OSS somehow neglected firm presence and centered on developer motivation while later research discussed emerging OSS business models [e.g., 1]. However, our study calls for more longitudinal studies on firm presence in OSS as (1) firm engagement varies over time and (2) former hobbyists might transform into employed developers (the latter was no focus of this study).

Limitations and Suggestions for Future Research

Our research has some limitations that have to be considered when utilizing our study's outcomes. The categorization of LK mailing list actors by the hostname of their email addresses into four contributor groups is an approximate but sufficient classification. It is known that there are developers that do not use their company email address while contributing and actors may do personal work out of the office [START_REF] Corbet | Linux Kernel Development. How Fast It is Going, Who is Doing It, What They are Doing, and Who is Sponsoring It[END_REF]. We mapped developers that occur more than once on the LK mailing list to one person object related to the email address s/he has used when sending a message to the list for the first time.

Multiple occurrences happen if the message sender uses different email addresses over time, but the same sender name (e.g., because of company changes). We have not considered these dynamics in our analysis. Furthermore, it has to be considered that the LK project is a unique OSS project, so that our conclusions cannot directly be transferred to other OSS projects where firm-sponsored developers are involved. Further research can consider to investigate the different types and content of the interaction as well as the aforementioned dynamics of developers or compare the multi-vendor project LK to single-vendor OSS projects.

Fig. 4 .Fig. 5 .

 45 Fig. 4. Out-degree distribution

  and 8.

Fig. 6 .Fig. 7 .Fig. 8 .

 678 Fig. 6. Average out-degree

Table 1 .

 1 Comparison of different aspects for the private, collective and private-collective innovation model (Source: adapted from Demil and Lecocq[START_REF] Demil | Neither Market nor Hierarchy nor Network: The Emergence of Bazaar Governance[END_REF] and Schaarschmidt et al.[START_REF] Schaarschmidt | A Problem-Solving Perspective on Governance and Product Design in Open Source Software Projects: Conceptual Issues and Exploratory Evidence[END_REF])

		Private	Collective	Private-Collective
	License	Proprietary	Open	Open
	Copyright ownership	Company	Collective	Collective
	Number of participating companies	One	None	One or more
	Revenue stream	Direct	None	Indirect
	Control intensity	High	Low	Low
	Knowledge sharing intensity	Low	High	High

Table 2 .

 2 Contributor categories

	Contributor	Description	Examples
	Category
	Companies	People with email addresses from companies	intel.com, redhat.com
	Hobbyists	People with private email addresses	gmail.com, yahoo.com
	Universities	People from universities	columbia.edu, duke.edu
	Research	People from research institutions and public au-	ieee.org, nasa.gov,
	Institutions	thorities (e.g., IEEE, government, military)	army.mil

In this study the terms contributor, developer, actor, participant and programmer are used synonymously to denote people who are active in OSS projects and in Linux kernel development.

http://marc.info/?l=linux-kernel

http://vger.kernel.org/vger-lists.html#linux-kernel


Antti Koskenvoima
email: antti.koskenvoima@utu.fi

Matti Mäntymäki
email: matti.mantymaki@utu.fi

Why do small and medium-size freemium game developers use game analytics?

Keywords: analytics, freemium, games

The increased use of the freemium business model and the introduction of new tools have made analytics pervasive in the video game industry. The research on game analytics is scant and descriptive. Thus, reasons for employing game analytics are not well understood. In this study, we analyze data collected with a set of in-depth interviews from small and medium-sized freemium game developers. The results show that game analytics is used to 1) assist design, 2) to reduce the risks associated with launching new games, and 3) to communicate with investors and publishers. The study advances the research on the business value of game analytics.

Introduction

Use of big data and analytics has become pervasive in the video game industry [START_REF] Bauckhage | How Players Lose Interest in Playing a Game: An Empirical Study Based on Distributions of Total Playing Times[END_REF]. The adoption of analytics has been driven by the fast development of cost-effective solutions that enable basic analytics even for start-up sized game developers. The second driver of game analytics is the diffusion of the freemium business model that has created a need to accurately measure, predict, and intervene player behavior 1 . For example, Supercell, the company behind the top-grossing freemium games Clash of Clans and Boom Beach, generated revenue of 1.8 billion USD from in-game purchasing in 2014 2 . The term freemium is a combination of words "free" and "premium" [START_REF] Wilson | The Freemium Business Model[END_REF] that describes a business model in which a service or a product is offered a for free but a premium is charged for advanced features, functionality or related products and services. Game analytics refers to applying analytics and big data in the gaming context [START_REF] El-Nasr | Game analytics: Maximizing the value of player data[END_REF]. Game analytics can be used to improve the players' gaming experience, or to maximize in-game purchases by tweaking various aspects of the game [START_REF] Drachen | Game data mining[END_REF][START_REF] Mcallister | Improving Gameplay with Game Metrics and Player Metrics[END_REF]. For example, the time taken for a specific task, the cost of a specific item or the power of a specific weapon [START_REF] Zoeller | Game development telemetry in production[END_REF][START_REF] Mellon | Applying Metrics Driven Development to MMO Costs and Risks[END_REF] can be optimized with game analytics.

Game analytics is a new field with limited research coverage [START_REF] Drachen | Towards Gameplay Analysis Via Gameplay Metrics[END_REF]. Prior research has focused on describing methods of data gathering and analyzing [START_REF] El-Nasr | Game analytics: Maximizing the value of player data[END_REF] as well as the role of analytics in game development [START_REF] Canossa | Benefits of game analytics: Stakeholders, contexts and domains[END_REF][START_REF] Gómez-Maureira | Improving Level Design through Game User Research: A Comparison of Methodologies[END_REF]. The limited amount of prior literature can be partly explained with the fact that game developers consider analytics confidential and are thus often reluctant to share information on their analytics processes [START_REF] Wallner | Game User Telemetry in Practice: A Case Study[END_REF].

As a result, the business aspects of game analytics are not well understood. Consequently, this study seeks to investigate why and how do small and medium-sized freemium game developers use game analytics? To answer the research question we have conducted a set of in-depth interviews among freemium game developers. Our results show that game analytics is used to 1) assist design, 2) to reduce the risks associated with launching new games, and 3) to communicate with investors and publishers. The study advances the research on the business value of game analytics.

2

Literature review

The freemium model

The freemium business model is extensively employed with digital products or services such as software, games, and web services. As a result, companies employing the freemium model strive to monetize their user base by striving to convert the users of the free version into paying customers [START_REF] Mäntymäki | Why do Teens Spend Real Money in Virtual Worlds? A Consumption Values and Developmental Psychology Perspective on Virtual Consumption[END_REF][START_REF] Mäntymäki | Purchasing Behavior in Social Virtual Worlds: An Examination of Habbo Hotel[END_REF][START_REF] Wagner | The Advertising Effect of Free--do Free Basic Versions Promote Premium Versions within the Freemium Business Model of Music Services[END_REF].

Freemium games typically employ micro-transactions and advertising as the main monetization strategies [START_REF] Fields | Mobile & social game design: Monetization methods and mechanics[END_REF]. Micro-transactions refer to buying virtual items or services that can be used and have value only inside a specific gaming environment [cf. 13]. Second, the players can be provided with the opportunity to buy "time", either to bypass waiting enforced by the game mechanics or to skip repetitive "grinding" phases [START_REF] Fields | Mobile & social game design: Monetization methods and mechanics[END_REF], or to unlock additional levels or areas of the game. Third, virtual items and benefits compared to the non-paying users can be bundled into a premium user account or a subscription [START_REF] Mäntymäki | Why do Teens Spend Real Money in Virtual Worlds? A Consumption Values and Developmental Psychology Perspective on Virtual Consumption[END_REF]. Fourth, in-game advertising and potentially offering an ad-free upgraded version against a fee can be used to monetize the players. Fifth and finally, game operators can sell information about the players to third parties, typically marketers [START_REF] Clemons | Business Models for Monetizing Internet Applications and Web Sites: Experience, Theory, and Predictions[END_REF].

Freemium games are typically designed to engage the player immediately, because there is no initial cost causing a lock-in effect [START_REF] Zauberman | The Intertemporal Dynamics of Consumer Lock-in[END_REF]. In addition, the game mechanics used in freemium games motivate the players to make in-game purchases [START_REF] Hamari | Game Design as Marketing: How Game Mechanics Create Demand for Virtual Goods[END_REF]. As a result, sustained play and customer lifetime value are critical for the economic success of freemium games.

Game analytics

Game analytics is a subset of analytics applied to the game development [START_REF] Drachen | Game analytics-the basics[END_REF]. Analytics in turn refers to using business intelligence in the process of discovering and communication patterns from the data and using the recognized patterns in solving business problems [START_REF] Davenport | Competing on analytics: The new science of winning[END_REF].

The need for game analytics has increased as games have become more sophisticated and complex [START_REF] Hullett | Data Analytics for Game Development: NIER Track[END_REF], and the rise of mobile gaming and the freemium business model has also had its effect. Gross, Hakken and True [START_REF] Gross | Getting Real about Games: Using Ethnography to Give Direction to Big Data[END_REF] maintain that studying online games is important from a managerial viewpoint "to understand ways that interactions while gaming can be improved, in order to make better games."

Today's video games, particularly online games and social media games, can collect data about almost all players' in-game activities [START_REF] Drachen | Game data mining[END_REF]. Game developers can use this data to obtain information about e.g. potential sources of player frustration [START_REF] Pruett | Hot Failure: Tuning Gameplay with Simple Player Metrics[END_REF]. For example, Replica Island, a game for Android devices, employed a player tracking system to identify instances where players were facing difficulties (e.g. player deaths). The whole metrics system was implemented by a one-man team at virtually no monetary cost, and has proven to be extremely valuable in identifying problems with the game design [START_REF] Pruett | Hot Failure: Tuning Gameplay with Simple Player Metrics[END_REF]. However, according to Drachen et al. [START_REF] Drachen | Gameplay Metrics in Game User Research: Examples from the Trenches[END_REF], game analytics is far from the traditional value chain of the video game development and hence not very highly prioritized.

Prior game analytics research has employed the purchase funnel concept to illustrate the challenges the freemium model poses [START_REF] El-Nasr | Game analytics: Maximizing the value of player data[END_REF][START_REF] Fields | Mobile & social game design: Monetization methods and mechanics[END_REF][START_REF] Moreira | Understanding Mobile Game Success: A Study of Features Related to Acquisition, Retention and Monetization[END_REF]. The AIDA-model (awareness, interest, desire, and action) from consumer behavior literature describes the process of new product adoption [START_REF] Webster | New Product Adoption in Industrial Markets: A Framework for Analysis[END_REF]. In application marketplaces where people typically download freemium games the customer can go through the process from awareness to action in seconds. However, out of the people who have downloaded a freemium game, only a fraction, e.g. 5 per cent will pay anything [START_REF] Fields | Mobile & social game design: Monetization methods and mechanics[END_REF]. As a result, successful employment of the freemium model requires sustained player engagement and efficiently managing a large pool of non-paying players towards conversion during the course of the play. A freemium game can be a hit without ever being profitable, and thus monetization is essential [START_REF] Davidovici-Nora | Paid and Free Digital Business Models Innovations in the Video Game Industry[END_REF].

Empirical research

Data collection

The empirical data was collected with five semi-structured interviews from experienced professionals from companies developing freemium games. The companies they worked in were based Finland. Obtaining the empirical data was challenging since finding the number of seasoned professionals with in-depth knowledge on freemium games as business in general and game analytics in particular is small. At the time of the interview, each of the five interviewees was in the process of developing a freemium game and using software to collect and analyze gameplay data. In addition, topics related to game analytics were frequently considered company confidential and hence many prospective informants declined the interview. The theme of the interview was presented to the interviewees in advance. A semistructured approach was selected due to the relative novelty of the research area. According to May [START_REF] May | Social research: Issues, methods and process[END_REF] the semi-structured approach allows the interviews to be flexible and interactive.

The interviews were done face-to-face in sessions from fifty minutes to one hour twenty minutes. All interviews were recorded and transcribed by the first author. In addition, notes were taken during the interviews. Appendix 1 summarizes the data collection.

Analysis

The analysis process started already during the interviews as notes about potentially insightful themes were taken. In the first round of analysis, the interviews were transcribed and coded based on the research question [START_REF] May | Social research: Issues, methods and process[END_REF]. Through iterations of the data, a constant-comparative method was applied to identify, elaborate, and clarify categories [START_REF] Corbin | Strategies for qualitative data analysis: Basics of qualitative research. techniques and procedures for developing grounded theory[END_REF]. Emergent categories were examined within and across interviews to determine salience and recurrence. Interconnections and discrepancies between the interviews and previous literature and between different interviews were also coded. The recurrence of certain key themes and limited number of new ones emerging over coding process indicated that the data exhibited saturation.

Results

We identified two main themes that describe the different roles of game analytics for small and medium-sized freemium game developers. These are analytics as a communication tool and analytics as a decision support tool. Based on these two themes we derived an emergent theme, analytics as necessity.

Analytics as communication support

"[Analytics] are a kind of tool for studios to justify their decisions, for example why a certain game is not ready for launch yet, because we need to improve this metric. The investors are more willing to give extra time, when they can see that in the long run the game will make more money if improved." (P4) As the quotation above illustrates, investors and publishers are very keen on analytics. The interviewees clearly stated that the key performance indicators have to be reported frequently to investors and publishers, and lack of improvement in these metrics will lead to questions. On the other hand, analytics help game designers make their case when they feel the launch schedule needs to be postponed as the game is promising but behind schedule. "If I were a game publisher, I would ask teams to soft launch and provide retention and ARPDAU [average revenue per daily average users] numbers before I would invest anything." (P3) Freemium games are typically continuously improved through scheduled updates and developers want to see the impact of changes as soon as possible. Sizeable and potentially risky adjustments are tested out with a small subgroup of players so that regular service would not be disturbed. Similarly, before the game is launched worldwide, it can be released in smaller market (e.g. Canada or Finland). The purpose of these soft launches is to collect data to ensure that the game will be profitable.

"We used to have a thing where once a week every studio [under that publisher] would report to the headquarters in California in an hour long conference call and give a pre-formatted presentation in which the key metrics were analyzed, future plans to improve them laid out etc." (P4)

The direct business benefits of game analytics include informed financial decisions, such as rationalizing marketing spending and budgeting for launch. The interviewees were very conscious that the video game industry has had several high-profile costly flops, i.e. projects that went overtime and over budget and made considerable losses. The interviews also indicated some of which could potentially have been avoided with the help of analytics. In addition, the interviewees recognized pressure from many stakeholders, competitors, players, and publishers on the gaming companies to adopt game analytics.

"Retention is the most important metric in the game industry" (P2)

The interviewees considered retention rate as the most important individual metric of business success in freemium game development. Consequently, all interviewees maintained that they are keen on tracking and monitoring retention figures. Retention rate refers to how players keep playing a game for subsequent sessions. A simple example of measuring retention is tracking the number of game overs per player. Typically retention was measured over a time period (e.g. 7 days, a month). The interviews also revealed that retention is a key metric for monetization as well as the basis of funnel analysis. In addition, retention is used to improve first impressions and the tutorials.

Interestingly, yet there was a consensus regarding importance and benefits from game analytics, the interviews also indicated that increasing volumes of data make it more challenging to extract relevant information from it. The interviewees wanted to keep the analytics process as simple as possible and focus on the key performance indicators (KPIs) and their trends over time and different versions.

"The most common wisdom in this free-to-play model is that if you don't have retention you are never going to make money. Retention stems exactly from that the game itself has some interesting aspects and is in some way fun."(P4)

In addition to retention, metrics that measure monetization and additional retention metrics, e.g. average revenue per user (ARPU), conversion rates, tutorial funnels and day 1, day 7, day 14 and day 30 retention rates, were mentioned as examples of metrics that most game developers monitor, or at least should monitor.

Additionally, customer lifetime value is used to measure to what extent the costs related to customer acquisition are covered. Customer lifetime value is an aggregate of other metrics, namely cost-per-install and ARPU, and is used to guide marketing spending. The following quotation depicts the role of customer lifetime value: "It is mostly based on things like is it profitable to invest in marketing the game. LTV (Lifetime Value) will tell you that." (P1)

Analytics as a decision support tool

"You cannot make a good game with just analytics, it's very challenging -[everything new] comes from the creative side" (P2)

The informants, particularly the ones who had been actively involved in actual game design, stressed quite strongly that analytics does not, and it also should not, drive game design. The interviewees maintained that analytics can support decision-making by e.g. occasionally disqualifying the intuition of the designers, but that analytics does not offer ready solutions. Further underlining that the design philosophy is not data-driven but data-supported, the interviewees stressed that game developers can and will outsource data collection, crunching and storage, but never game design. The analytics and metrics do not make the games, but they can help in making them better. The following quotation describes the role of analytics in relation to design: "[Regarding Game Analytics] maybe the larger benefit, at least in our case, is that instead of driving development, they are used to spot errors in the code and clear design mistakes. --It's more about monitoringfirst you design, then you code and then you monitor how well did it go." (P5)

We observed an interesting controversy regarding the interviewees' views on the role of game analytics. While all believed in the value of analytics and some emphasized tracking as much as possible to ensure having sufficient amount of data whenever needed, others preferred a more strict selection of metrics to avoid information overload and collecting what they referred as "vanity metrics". These interviewees also more deliberately emphasized the importance of evaluating the benefits against the respective costs.

A recurring theme in the interviews was that the informants considered players unpredictable and that they often behave differently than the designers expected, often seemingly irrationally. Analytics can help designers to understand players' perspective of the gaming experience. For example, one interviewee described a mobile game that had a design mechanic punishing players for erroneous behavior by reducing his/her points. An alternative solution was to end the game immediately after an error was made and force the players to start over. Somewhat counter-intuitively the latter version, which the designers felt was more "hardcore-oriented", was more appealing to players and led to higher retention levels. As a result, game analytics proved that, against game conventions and designers' expectations, the harder and more punishing version was more popular.

The analysis tools and methods in use were largely uniform among informants. Maybe for this reason they did not see analytics as a real source of competitive advantage. As all our informants worked for small or medium-sized companies with very limited human and financial resources to be spend on analytics, they had simply adopted 'off-the-shelf' analytics tools. Since most small and medium-sized game companies use a similar set of tools, the informants considered the expertise and proficiency in using tools and the ability to draw the right creative conclusions the best way to derive value from game analytics. The informants also stated that the use of analytics differs considerably between smaller and larger companies in the freemium games market.

There was a strong consensus among the informants that data allows real and accurate insights about player behavior. All of the interviewees had experience from pri-marily quantitative analysis and metrics. The informants' view was that game companies seldom utilize qualitative data, as it was considered more taxing, less effective, and harder to implement with third-party solutions. The informants also stated that interviewing players about their gameplay habits can lead to misleading results since people can seldom tell what they really want and would use.

"We don't do analytics because they are cheaper, but because they are better. --I never trust people who say "If you would develop this, I would use it all the time". Only when I can measure that they really use it, I will believe it." (P3) All informants stated that the analytics data is actively shared and communicated within the development team, but the interviews also indicated that game companies seldom share their sales numbers or metric data in public. Companies can however compare the KPIs of new launches with their prior games. This way game analytics also help guide the portfolio management of game publishers.

The data-supported design process is iterative. First, a visible problem is noticed in high-level metrics. Then the designers seek for possible causes by drilling down into the data. Specific changes are made to the game, and then the effects are measured. There is a constant loop for validating design decisions.

"[We tend to find a] --high-level issue and then try to find one specific user experience issue that you think you change, and then you iterate the process. (P4)

A typical way to tackle the issues that are found during the development process is to create two or more different versions of the game. These versions are then randomly distributed to players and their respective performance is measured. This is referred to as A/B testing. It cannot be used extensively for every decision since developing each alternative consumes resources and the inferior versions are a wasted effort. Acquiring relatively reliable results from the A/B tests also takes considerable time.

"When something new is added to the game, it is done in two different ways and half of the players get version A and the other half gets version B and then the metrics are compared. There may be a hypothesis behind the test, but it is more about trying to find what works and then developing the better version further, leave it as is or abandon both." (P1) Funnel analysis was another common analysis method used by the informants. Tutorials are a typical example of a funnel. Throughout the tutorial phase gradually fewer and fewer players reach each subsequent step. They have not necessarily paid anything for the game yet, so it is essential that they do not churn out this early. Measuring the return rate or retention of players in subsequent steps (or levels) in the game was the most common way of analyzing the quality of said funnels. When the designers note a notch in the retention, they know that there is a problem in that part of the funnel.

Analytics as a necessity

The findings strongly suggest that game analytics plays an important role in communicating with investors and publishers as well as in supporting game development and manager's decision-making. Hence, analytics was seen as a necessity in freemium game development. Every informant pointed out several benefits of game analytics. In fact, operating without utilizing analytics was considered "flying blind" and analytics was seen as a means to reduce the risk of failures. Interestingly, the interviewees also repeatedly stated that there had been no need to state an explicit business case to justify investments in game analyticsthe benefits were that clear.

"It is clear thatespecially in this free-to-play modelyou cannot operate under the mentality that you just launch a game and hope for the best." (P1) However, utilization of game analytics is characterized by the lack of resources. This explains why game analytics was seen as something that is important but not as a differentiating factor or a potential source of competitive advantage. Table 1 below summarizes the main findings.

Table 1. Summary of findings

Analytics as a communication tool

Investors and publishers follow certain key metrics The metrics provide a common ground for discussion Retention is the most important metric to track Average revenue per user (ARPU) and Lifetime Value (LTV) are also actively followed Analytics as a decision support tool Help to reduce the risk of total failure Role is to support design decisions, not drive them Analytics affect the whole development process, there is a constant loop of changes, assessment and improvement Developers use game analytics to e.g. identify problems in the game design and/or bugs in the code that lead to player churn Analytics as a necessity Analytics viewed necessary but not as a source for competitive advantage Reliance on third-party game analytics tools and software. Lack of resources (time, skills, and money) restricts the use of more advanced analytics The emphasis is on quantitative data 4

Discussion

Theoretical implications

The findings indicate that game analytics play a pivotal role in freemium game development. The interviewees emphasized a combination of designer creativity and analytics is required to obtain the best results and repeatable results. Our results imply that retention is the most important metric to track. Game analytics was not viewed as source of competitive advantage, at least for smaller companies relying on standard third-party analytics tools, but considered more as a risk reduction tool. The possible competitive advantage of utilizing game analytics is derived from the experience and creativity of the analysts and designers.

Prior research has presented the vast possibilities and introduced sophisticated solutions for game analytics [START_REF] Bauckhage | How Players Lose Interest in Playing a Game: An Empirical Study Based on Distributions of Total Playing Times[END_REF][START_REF] El-Nasr | Game analytics: Maximizing the value of player data[END_REF]. However, our results imply that the analytics processes used by small and medium-sized freemium developers are rather simple. There are two opposing philosophies in data gathering. One stresses that data cannot be tracked retroactively, and tracking as much as possible ensures you have the data when you need it. The other warns about collecting "vanity metrics" that do not contribute to game design and only confuse the analysis process [see e.g. 30].

Very large volumes of data call for increasingly sophisticated and powerful analysis tools, which in turn increase the costs. Prior research has promoted utilizing a combination of data sources (e.g. analytics, interviews, biometric measuring) to improve results [START_REF] Gómez-Maureira | Improving Level Design through Game User Research: A Comparison of Methodologies[END_REF]. However, our results show that small and medium-sized freemium developers primarily focus on measuring retention and a few other key metrics. Furthermore, the data analysis methods were simple and many of the professionals cited that they do not have advanced database management skills.

Altogether, these observations align with prior research arguing that alongside being actionable, the results from analytics should be human-readable and easily interpretable [as in e.g. the model presented in 31].

Retention rates were further utilized in evaluating which alternative was better in A/B tests and improving funnels, especially tutorials. The methods used resembled the split testing used by Andersen et al. [START_REF] Andersen | On the Harmfulness of Secondary Game Objectives[END_REF] to recognize the most engaging design choice from three alternatives. The small and medium-sized freemium developers also showed interest in adapting more complex analytical procedures, but did not have either the skills or resources to implement them. The interviewees did not share much about the gathering of qualitative data, which further emphasizes that the focus is on directly actionable, quantitative metrics.

Prior literature on game analytics has recognized the managerial importance of actionable results [START_REF] Canossa | Between Intention and Improvisation: Limits of Gameplay Metrics Analysis and Phenomenological Debugging[END_REF]. Our results add on this body of research by emphasizing the value of the predictability and decision support in analytics. In addition, our results demonstrate that for freemium game developers, game analytics provided important tools for investor communications. Third, game analytics can help in optimizing marketing spending by customer lifetime value calculations or campaigning for more time from the publisher citing promising retention trends.

Managerial implications

Compared to game industry's prior reluctance to adopt analytics [START_REF] Mellon | Applying Metrics Driven Development to MMO Costs and Risks[END_REF], our results indicate a change in mindset. Our set of interviews among game industry professionals indicated that the benefits from game analytics are so self-evident that they are not even always explicitly stated. Since basic analytics are today available for even startup companies, the initial costs of implementing game analytics are relatively low. For example, applying basic telemetry data analysis is one the most cost-efficient way to do user research, and even one-man teams can afford it [START_REF] Canossa | Interview with Nicklas "Nifflas" Nygren[END_REF] Our results imply that game analytics is viewed as a risk reduction tool. Using analytics during the development stage can decrease the risk of total failure in the launch stage. In addition, even the basic level of analytics with the standard key performance indicators can assist companies in e.g. terminating projects that are unlikely to generate sufficient revenue to become profitable.

Certain metrics have become close to industry standards. Key performance indicators that measure monetization and retention, such as ARPU, conversion rates, tutorial funnels and day 1, day 7, day 14 and day 30 retention and customer lifetime value were mentioned as examples of the key metrics. These metrics are also used to evaluate and predict the financial success of the game.

Many of the interviewees actually stated that investors require key ratios such as retention rate and ARPU when they discuss potential investments with the teams. Similarly, publishers expect results and want hard numbers to confirm the feasibility of the game concept. For example, certain leading publishers also already demand weekly reports that cite key metrics from their developers.

Limitations and future research

Like any other empirical research, the present study is subject to a number of limitations. First, the empirical data was collected solely from small and medium sized companies developing freemium games that were based in Finland. For example, large freemium developers such as Supercell are well known for their advanced use of game analytics. Thus, we suggest future research with a broader empirical coverage.

Second, the freemium model as well as analytics is employed in other fields such as online music as well as more traditional fields such as insurance. Future research could examine how analytics is utilized in other sectors to find best practices that could be applicable across industries.

http://www.theguardian.com/technology/2014/dec/09/clash-of-clans-billion-dollar-mobilegames

2 http://www.theguardian.com/technology/appsblog/2013/jul/23/clash-of-clans-supercell-freemium