Jean-Claude Berthelemy
email: jean-claude.berthelemy@univ-paris1.fr

Arnaud Millien
email: arnaud.millien@univ-paris1.fr

Impact of Decentralized Electrification Projects on Sustainable Development: A Meta-Analysis

Keywords: L93, O13, O18, O22 Decentralized electrification, sustainable development, impact assessment, meta-analysis

This paper is the first product of a project which aims at building a Collaborative Smart Mapping of Mini-grid Action (CoSMMA), whose principal objective is to identify best practices of decentralized electrification projects.

Using evaluations of 421 projects, from published research papers, we built a pilot CoSMMA which proves its feasibility. Its relevance is demonstrated by a meta-analysis, which reveals the principal characteristics of decentralized electrification projects which have positive impacts on sustainable development.

Four main characteristics were considered: technology (source or energy), system size (power), decision level (from local to country level), geographic location. When searching for best practices, technology and system size must be considered together, because the chosen technology may constrain the power, which is provided by the system. We find that the most popular projects, which are based on Solar Home Systems (SHS) are not the most effective. The problem with SHS is not the use of solar energy, but the small system size often chosen for SHS. Mini-grids, of larger size, especially those which use hybrid renewable sources of energy, have more positive impacts, because these systems combine the benefits of sustainability and flexibility. In terms of decision level, we find that both top-down and bottom-up approaches have advantages, with the observation of a U-shaped curve for the influence of the decision level on the probability of obtaining positive impacts. Geographical location matters, as it is very often the key to system feasibility. We find that DEPs are more effective in Latin America than in Asia, and more effective in Asia than in Africa.

We also attempted to study the type of effects resulting from DEPs. Descriptive data suggest that for some types of effects, positive impacts are more likely than for others. Decentralized electrification projects have a more positive impact on Lifestyle & NICT or Household agenda than on Economic transformation or Community life. However, this pilot CoSMMA does not contain

enough information to study precisely the types of effects, because some types of effects have not been studied frequently in the existing literature. This is the case, for instance, for environmental effects, which have been rarely measured scientifically.

Finally, we attempted to broaden our information set by including expert data, which was entered into the CoSMMA meta-analysis. We define expert data as data that are not supported by statistical tests with measures of significance, whereas the evaluations based on scientific data were supported by statistical tests of significance. The expert data may be valid, but our attempt to include it in the analysis failed at this stage. The determinants of unproven effects appear to be quite different from the determinants of proven effects in our meta-analysis, and using expert data would imply merging proven and unproven effects, which would totally blur the conclusions. Documents de travail du Centre d'Economie de la Sorbonne -2018.39 List of Tables Table 1 -Previous studies adressing a mapping of the socio-economic effects of electricity ........................... This paper is part of a project to build a collaborative database on decentralized electrification, named CoSMMA (Collaborative Smart Mapping of Mini-grid Action), with the objective of identifying best practice from the point of view of sustainable development (i.e. to identify the project characteristics that maximize positive impacts on sustainable development).

List of figures

To this end, DEPs are described in the CoSMMA in several dimensions:

 Basic technical characteristics of the project, such as the energy source and system size (electrical power)

 Project goal and expected impact  Ex-post evaluated effects Additional types of information include conditions of evaluation and document sources.

At the current stage of our project, we have built a pilot CoSMMA, with the objective of testing its feasibility and proving its relevance. We used information on 421 DEPs available in published research papers, which we analyzed and coded into variables describing the projects and their effects. This information was gathered by a structured search from 4 principal academic sources -Academic Search Premier, Business Source Complete, EconLit, and GreenFILE. The information was then processed through a meta-analysis regression, whose results shape the core of this paper.

Our principal tool is a multi-probit meta-regression, which shows which factors led to which effects of DEPs on sustainable development. We also attempt to break down the analysis by type of effect, but at this stage the available information limits our analysis. Only a few types of effect of DEPs have been sufficiently explored in the papers registered in the CoSMMA to allow uncovering their specific determinants. Finally, we attempted to enlarge the data base used in the meta-regression by including so-called expert data (i.e. evaluations provided by experts but not supported by statistical tests of validity). However, we were unsuccessful in this attempt, because statistically proven effects and unproven effects appear to have quite different determinants.

In section 2, we develop in more detail our research question and relate it to the existing literature.

In section 3 we document our sources of data and methods used to build the CoSMMA and we report descriptive statistics on project characteristics and project effects registered in the CoSMMA. In section 4, we describe the econometric methodology used to perform our meta-analysis. In section 5, we discuss our empirical results and their possible extensions. Section 6 is devoted to a discussion of the possible bias that could affect our results, section 7 concludes and proposes some possible areas for further research.

Research question and literature review

Definition of DEPs

Defining decentralized electrification is not simple, because many field practitioners and scientists refer to decentralized project as an obvious notion, although to the best of our knowledge, no clear criteria has been established so far.

CoSMMA is limited to off-grid or individual solutions, with no connection to the national grid. Our definition also includes a size limitation: any project above 100 MW is not considered as decentralized, because it could be involved in clearing price exchanges [START_REF] Dillig | The impact of renewables on electricity prices in Germany -An estimation based on historic spot prices in the years 2011-2013[END_REF] .

The need to identify best practice of DEPs

A variety of DEP projects have been implemented and evaluated so far, with a focus on solar nano solutions, the so-called SHS (Solar Home Systems). This focus comes from a convergence of interest between funders and developers, as they offer a low-commitment solution for the funder, and a low-cost market test for the developer. This focus does not imply that SHS represent the best practice in terms of positive impact on sustainable development. Institutions working in the sector frequently face the reality of economic or technical failures [START_REF] Ikejemba | The Documents de travail du Centre d'Economie de la Sorbonne -2018.39 empirical reality & sustainable management failures of renewable energy projects in Sub-Saharan Africa (part 1 of 2)[END_REF]. Failures are also repeatedly reported by NGOs promoting DEPs, with estimates of failure rates of project commonly being above one third. Clarifying the question of DEP performance and identifying best practice is thus important.

The development of DEPs faces three major challenges:

 Because projects are not connected to the grid, they show a large heterogeneity of economic and technical design  So far, no unified framework of knowledge and data on DEPs can offer a complete vision on the variety of field experiments, and qualify their ability to yield sustainable favorable impacts  There is no clear consensus on the types of effects that matter and the primary types of impacts that a DEP should address first

In this study, "best practice" is defined as the project characteristics that produce significant favorable effects on sustainable development. Significant favorable effects are also called "positive impacts" in this study. research. Using these observations is complex given their heterogeneity, however it can provide an approach to delivering an early assessment of DEP strategic choices.

A meta-analysis adds to the understanding of a phenomena by combining results obtained by researchers using a variety of data and methods [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF]. In conducting a meta-analysis with published results for DEP effects, we expect, like [START_REF] Carré | La coordination entre politique monétaire et politique macroprudentielle. Que disent les modèles dsge ?[END_REF], to have more robust conclusions than a mere review of separate regressions. Using a systematic selection from research databases, a meta-analysis avoids the classic pitfalls of a literature review, which could be unbalanced due to selection bias, or reflect the beliefs of authors who might tend to reject papers that run against their convictions [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF].

To the best of our knowledge, this study is the first meta-analysis which attempts to relate DEP characteristics to their impact on sustainable development, and hence which addresses clearly the question of best practice in decentralized electrification.

In order to base our contribution on previous literature, we review below two branches of research:

first we consider what has been proposed so far in terms of mapping DEP effects. Second, we analyze the methodological references for meta-analysis.

Previous mapping of DEP effects

To the best of our knowledge, there is no previous study which proposes a complete mapping of DEP effects in developing countries. Several studies have been done with more specific research questions, as shown in No field data.

Calibration made with data from individual assessment by other researchers.

Previous meta-analyses

Frequently used in medical studies, meta-analyses were popularized in social science [START_REF] Carré | La coordination entre politique monétaire et politique macroprudentielle. Que disent les modèles dsge ?[END_REF], and were widely used as a quantitative method of research synthesis to calibrate structural models, examine patterns of publication bias, and explain differences in the results of individual studies (Frequently used in medical studies, meta-analyses were popularized in social science [START_REF] Carré | La coordination entre politique monétaire et politique macroprudentielle. Que disent les modèles dsge ?[END_REF], and were widely used as a quantitative method of research synthesis to calibrate structural models, examine patterns of publication bias, and explain differences in the results of individual studies (http://meta-analysis.cz/).

In a seminal work, [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF] provides clear and comprehensive advice on the steps to follow and pitfalls to avoid, when conducing a meta-analysis that "employs conventional statistical methods and criteria to summarize and evaluate empirical economics". We follow this methodology, especially in the important step of defining the objective of the meta-analysis. To clarify to what extent our study fills a gap, we investigated 4 sources specialized in conducting international meta-analysis, a website http://meta-analysis.cz/ and reviews of Journal of Economic literature, Journal of economic perspective, Journal of economic surveys.

As shown in Table 2 there is no meta-analysis about access to electricity ("electrification"). Our research shows that our paper is the first meta-analysis on electrification effects.

We found 12 meta-analyses about "electricity", which proves the growing importance of the electricity economics field, as each meta-analysis is based on a populated set of underlying studies.

Those studies address topics so different from CoSMMA, that we can hardly use them as reference, but we can highlight 2 findings:

 Meta-analyses about electricity economics are feasible  CoSMMA fills a gap in off-grid electrification assessment.

It is worth noting that 6 of the existing meta-analyses are about USA electricity economics, 2 about developing countries, and 1 about renewable electricity.

Several literature reviews about energy economics were also investigated. Table 3 shows the number of articles reviewed. These numbers are small compared to the number of papers used in the CoSMMA (123).

Documents de travail du Centre d'Economie de la Sorbonne -2018.39 The author aims to evaluate the pricing of (positive) externalities from renewable electricity generation.

Off-topic: Discussion in the literature. [START_REF] Wolfram | How Will Energy Demand Develop in the Developing World?[END_REF] How Will Energy Demand Develop in the Developing World? Off-topic: Impact of growing energy demand on the grid [START_REF] Borenstein | The Trouble With Electricity Markets: Understanding California's Restructuring Disaster[END_REF] The The authors study models used in the literature to evaluate the demand for electricity (and its impact on reliability) Documents de travail du Centre d'Economie de la Sorbonne -2018.39 The studies on the impact of electrification show a wide scope of methodologies, data, and projects.

Following [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF], "differences in quality, data or methods do not provide a valid justification for omitting studies. Rather, such difference provide the underlying rationale for doing a metaregression analysis in the first place."

A systematic collection of research papers was made, with no ex-ante exclusion, but the topic relevance [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF] : "after reducing the sample of studies to those that contain some relevant empirical estimate, test or finding"). Off-topic studies (e.g. electrification of railways), macroeconomic studies, studies focused only on potential and barriers, ex-ante cost/benefit analyses, or technical feasibility studies) were not used for the CoSMMA. Papers with a developed country in the title were excluded. Only papers with a publication date later than 1980 were selected. This time span was set to avoid missing any important precursor publications about decentralized electrification. However, because the growing interest in decentralized electrification is recent, papers before 1990 are scarce (see Figure 1).

Publication conditions were also checked. Documents had to use a common language (English) and be peer-reviewed, or designed for such a process (e.g. working papers of research institutions). A few economic reports (from financing institutions or companies) were included because they had been through a quality control process before public dissemination. They represent 7% of the current primary sources of the CoSMMA. A key sentence containing words usually used to analyze decentralized electrification projects was defined and parsed through EBSCO for the 4 databases. Keywords were automatically reweighted by a smart text mining function in EBSCO. Some variants were also used. Finally 6 main queries were defined which gave 6 sets of documents, called "packs". For the most complex queries, a common set of additional keywords was used in order to limit the study more closely to decentralized electrification projects.

Reweighted queries were saved to keep track of the search, allowing for possible external replication.

Being keyword-based and systematic, this methodical sampling aims to define a neutral collection of papers, which is not influenced by the researcher's knowledge or a specific direction of research.

The keyword-based sampling approach warrants provides a random selection of papers related to the DEP effectiveness field of research. However, the ability of an algorithm to fit accurately to a field of research cannot be guaranteed, and so ex-post human checks were performed on the EBSCO selection results. Possible duplicates were eliminated, and a final check of the application of ex-ante selection criteria was made (e.g. residual macroeconomic studies or other off-topic papers were eliminated).

Within each of the 6 packs, keywords defined specific branches. Inside each branch, some articles with large bibliography were used to define sub-branches, in which some of the papers quoted in the bibliography of the head article were collected as well. However, the bibliographies of initial articles were used with parsimony, because too many papers from sub-branches could have introduced a bias toward the past into the meta-analysis, and also a direction bias: at a given point in time, a researcher can only cite previously published papers, and papers strongly related to his or her own research direction.

For reasons related to the research project's origin, 32 articles were used in addition, following a classic approach based on research about the econometric evaluation of decentralized electrification. These articles did not duplicate the EBSCO extraction. They constitute an additional pack in the meta-base. Additional papers (from sub-branches or historical pack) are 18% of all collected papers.

After this selection process, the dataset ready for statistical analysis contains 421 projects, evaluated in 123 papers, with a total of 2605 effects.

The inclusion criteria applied to project characteristics are presented in Table 4 below. Note that the number of exclusions results from the simultaneous application of criteria, and the number of exclusions is thus not the total number resulting from each individual criterion.

Figure 2 shows the geographical distribution of DEPs registered in the CoSMMA, showing that

CoSMMA is based on a wide variety of experiences. Annex 1 lists the various effects by quality of estimation. About two-thirds of the reported effects were submitted to statistical tests by their authors. We call scientific data the data associated with tested effects. We call the remaining data expert data. For reasons explained later, our conclusions from the meta-analysis are based only on scientific data. For this reason, we restrict at this stage our description of CoSMMA data to the sub-sample of scientific data.

Annex 2 describes the distribution of this sub-sample by technology, system size, decision level, and continent. Solar systems account for the vast majority (over 70%) of the DEPs in this subsample;

other sources are mainly renewables, and there is a small number of DEPs which use fuel.

The system size of DEPs can vary from less than 1 kW to 100 MW. Our classification by size is based on the following definitions: The most frequent decision levels are at country level or at local (municipality) level. This corresponds to two vastly different approaches: top down or bottom up.

 Nano: < 1 kW
As for geographical distribution, approx. 50% of projects are in Africa, approx. 40% are in Asia, and 10% are in Latin America.

Annex 3 describes the distribution of effects by direction. Some effects are qualified as favorable to sustainable development (e.g. they increase energy availability, develop income generating activities, save time for households, improve health or education, reduce environmental damage).

Some are qualified as unfavorable. About 2/3 are favorable and 1/3 are unfavorable (with a small proportion of inconclusive studies). This ratio of 2 to 1 corresponds to anecdotal evidence reported by NGOs on their success rates with DEPs. However, the most striking observation is that whatever the direction of the effects, a majority of effects are unproven: the usual statistical tests show that the probability that reported evidence is false is above 5%. Only about 20% of reported effects are proven, this proportion is a little higher for favorable effects than for unfavorable effects. As a consequence, out of the sample of 1,486 effects measured with scientific data, only 225 are proven favorable effects, which we call henceforth "positive impacts", and 81 are proven unfavorable effects, which we call "negative impacts".

Annex 4 shows the distribution of characteristics associated with positive impacts. Comparing this distribution with the distribution of observed effects in Annex 2 gives a primary assessment of where the best practices are. We observe that there are, relatively speaking, more positive impacts in projects using hybrid technology with renewables and less positive impacts in projects using fuel or hybrid technology with fuel. There are also a few more solar-based projects with positive impact than in the full sample. We observe also that there are more positive impacts in projects based on nano size systems. The proportion of positive impacts is the highest at provincial decision level.

Finally, there are more positive impacts in Africa than in Asia. These descriptive conclusions may however be misleading, because the different characteristics considered one by one are actually correlated. For instance, most solar systems are SHS, of nano size. We will show in section 5 that the predominance of hybrid and renewable technologies is confirmed by a multivariate analysis, but that nano size systems are not best practice, and that the provincial decision level is not as effective as the country decision level. As for geographical considerations, the multivariate analysis will not confirm the better performance of Africa than Asia suggested by simple descriptive statistics.

4. Methodology for a meta-analysis of DEP impacts 4.1. Objects of the meta-analysis

Because our meta-analysis does not use directly data from the field, it is important to define what are the objects of the analysis. As pointed out by [START_REF] Glass | 9: integrating findings: the meta-analysis of research[END_REF] : "the design of a study is a complex Documents de travail du Centre d'Economie de la Sorbonne -2018.39

judgmental process that produces as many different studies as there are researchers and settings in which they work".

The objects of this meta-analysis are the effects of DEPs observed from previous published evaluation studies, which used experimental or observational data. An evaluation study of electrification project is a document that:

 Describes the characteristics of the project  Describes the general purpose of the project  Documents or measures the effects of the project

Source of heterogeneity across control variables

A meta-study aims to exploit the variance along a common dimension across a set of various studies; but because each research is unique, it seems paradoxical to pretend to identify a common dimension from all the features that make every study unique. As noted by [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF], "because [...] most studies entail a unique combination of techniques, independent variables, data, time periods and other research choices, not every study characteristic can be coded and analyzed. Nor should a researcher wish to do so. Variation due to minor modeling choices may be treated as part of the random study-to-study background."

To achieve this separation between genuine sources of heterogeneity and heterogeneity that arises from modeling choices, we attempted to establish a clear distinction between a measured phenomenon (effects of a DEP), and the measurements performed to capture this phenomenon (estimation methods), or measurement conditions (Number of observations).

Some meta-analyses capture the number of observations, others capture the T-statistic (Doucouliagos and Paldam, 2006b) or standard error [START_REF] Havranek | Cross-country heterogeneity in intertemporal substitution[END_REF], some even include the date of collection, which gives a panel of studies [START_REF] Havranek | Does Daylight Saving Save Electricity? A Meta-Analysis[END_REF].

First, in this study, we controlled for the number of observations (N) when it was available. A clear distinction is made between using scientific data (N>1) and studies using expert data (N ≤ 1), as illustrated in Table 5. Second, we controlled for the methodology of evaluation. A large variety of research methodologies has been used by researchers to gather evidence of DEP effects (as reported in Annex 1), from the least sophisticated ones (citing others' results) to the most advanced ones (robust econometric evaluations which permit statistical inference).

The use of expert data merits a specific discussion. As shown by the dataset used in [START_REF] Havranek | Does Daylight Saving Save Electricity? A Meta-Analysis[END_REF], the absence of statistical tests in some studies, though infrequent, is not necessarily an obstacle to conducting a rigorous meta-regression, although it might be expected that introducing too high a proportion of studies without significance testing could weaken the ability to arrive at conclusions. [START_REF] Carré | La coordination entre politique monétaire et politique macroprudentielle. Que disent les modèles dsge ?[END_REF] also conducted a meta-regression using data without variance of the estimates, which confirmed that the methodology is feasible with expert data. They introduced a dummy for the quality level of observations.

In this study, we initially conducted a baseline meta-regression using scientific data only, reproducing the classic framework of meta-analysis. Then we introduced expert data in an attempt to enlarge the estimation sample, using the pool of expert knowledge on DEPs. However, as discussed later, this attempt to include expert data was inconclusive, because we observed large differences in best practice revealed by regressions to separate proven and unproven effects, and by regressions mixing them.

Third, we controlled for the time lag between the year of implementation of the system and the year of publication of its evaluation. This time lag may reflect the short term vs. long term nature of effects, but may also reflect other factors such as the difficulty to collect data which relies on the memory of survey respondents.

Specification and estimation strategy

As noted by [START_REF] Stanley | Wheat from Chaff: Meta-Analysis as Quantitative Literature Review[END_REF], "the independent variables -often called "moderator variables"-are those study characteristics that are thought to be consequential". In this meta-analysis, project specifications are expected to be the essential channel of DEP impact. First, this is in line with our objective of exploring best practice in DEPs. Second, DEPs show highly different characteristics, because in decentralized electricity there is no grid playing a structuring role in terms of balancing, interconnection standards, or demand response. The heterogeneity of DEP characteristics is higher across projects chosen for research evaluation, because evaluated projects are often the most innovative ones, either in terms of technological features or in terms of socio-economic environment and organizational features.

The outcome variable is the direction of the effect of a DEP, which is a categorical variable; we aim to explore the determinants of the probability of observing a given direction of effect. Basically, we could consider a dichotomous outcome, i.e. whether evaluated DEPs had favorable effects on sustainable development or not. However, given the large number of studies reporting unproven directions of the effects (Annex 3), our set of information would be too fuzzy in the absence of a Documents de travail du Centre d'Economie de la Sorbonne -2018.39 distinction between proven and unproven conclusions. In an attempt to avoid this shortcoming, we considered 5 distinct categories of effect directions, as reported in Annex 3, in which proven and unproven effects are separated.

The baseline estimation links a set of project characteristics and of methodological control of evaluations to the probability of achieving a given outcome. The possible outcomes are defined by the combination of the direction of the effects (favorable or unfavorable) and their nature (proven or unproven). The parameters are estimated with a multi-probit estimator, which yields simultaneously all equations, one for each direction and nature of outcome.

P(outcome ip = k) = constant + c.EvalCond ip + s.ProjectSpec p + error-term ip
Where:

 p is a project  i is an observed or reported effect  outcome = k is one of 5 possible outcomes  EvalCond ip is a vector of control variables of an evaluation's conditions, as developed in the next section ;

 ProjectSpec p is a vector of a project's specifications

This equation provides an assessment of best practice by evaluating s, a vector of parameters which describes the influence of project specifications (ProjectSpec) on the probability of obtaining a positive impact, after controlling for conditions of evaluation in underlying studies (EvalCond).

Although our interest is focused on the positive impact outcome, estimating the full set of parameters associated with all 5 outcomes in a multi-probit regression provides a way to enrich our estimation, because estimating the determinant of other outcomes conveys indirectly information about the DEP characteristics that limit their ability to have positive impacts.

Most of the variables in the vector ProjectSpec are categorical. We consider 4 different types of characteristics:

 Technology: the sources of energy and technics used to produce electricity  System size: electric power available for users connected to the system  Organization: decision level -from local to country or multi-country, at which the DEP was conceived  Geographical location.

The variance estimator uses clusters by project objectives, whose classification is described in Annex 5.

Excluding outliers

Documents de travail du Centre d'Economie de la Sorbonne -2018.39

In the same way as controls were introduced ex-ante in the estimation strategy, some essential robustness checks were performed before the estimation, to exclude the worst cases that could have spoiled the estimation. These checks correspond to the bottom 2 criteria in Table 4.

The most atypical observations were excluded from the estimation sample, because they could have a too high influence on the meta-estimation, and hide other more frequent relationships. Atypical data appear from uncheckable errors during the collection process, or due to abnormal observations at the extreme end of the variables' distributions.

The outliers were filtered based on a Dfbeta estimation and a quantile-based filter. Outliers above a limit for Dfbeta were excluded: from a simple LPM using all candidate covariates, Dfbeta were computed for all observations. Observations with a Dfbeta more than 5 times the usual threshold ( 5x 2/ N) were dropped. Observations above the 99% quantile of their distribution were also dropped.

Which characteristics of electricity projects yield positive impacts on sustainable development?

Table 6 shows the role of the characteristics in project effectiveness, after controlling for conditions of evaluation. This table presents average marginal effects (AME) of the probability of generating the different outcomes. As our variables of interest are categorical, estimated AMEs represent the difference between the probability that a given category generates the outcome and the probability associated with a reference category, which is denoted as "ref.

=".

Below, we discuss the role of source of energy, system size (power), decision level, and location

Role of source of energy

Many different sources of energy can be utilized in DEPs. The different sources of energy have different unit costs, intermittence, reliability, or maintenance requirements. At this stage we can only measure the average performance of the different sources, and we cannot compare them because in practice performance is conditioned by many other factors such as geography.

We chose as reference hydroelectric power projects, which were historically among the first DEPs based on renewable energy deployed in developing countries.

Solar power based on photovoltaic panels is by far the most popular technology for DEPs. Solar projects have +12.3 percentage points (pp) higher chance of generating positive impacts than hydroelectric power projects (Table 6 col 1). It is also interesting to note that the risk that solarbased DEPs create negative impacts is significantly lower (-10.7 pp) than for hydroelectric power projects.

Documents de travail du Centre d'Economie de la Sorbonne -2018.39 However solar energy is not necessarily best practice. All technologies have a higher chance to bring positive impacts than hydroelectric power projects, between +41.8 pp (hybrid renewables) and +12.3 pp (solar) (Table 6, col 1). Hydroelectric power projects are highly constrained by resource location;

other technologies used for DEPs could be more flexible and mobile, with a better ability to adapt accurately the energy supply to population needs, combining supplied power with local network of distribution. Hydroelectric power projects are constrained by the topography of hydrological basins, which can increase the cost of local power lines due to mountainous terrain and distance to local populations, and make a substantial difference between the theoretical potential of Small Hydro Power (SHP) and a feasible answer to electricity needs UNIDO (2010).

The most promising solutions appear to be hybrid renewables, suggesting positive combined roles played by sustainability and flexibility. However, hybrid renewable projects are very innovative, and for this reason could result in more attention being paid to the project's inclusion in its socioeconomic environment. Technology could thus hide other unobservable factors of project organization and governance, with which they might be correlated.

The fact that fossil fuel technologies also have a significantly high positive impact (higher than solar projects) suggests that flexibility is a major determinant of positive impact. Although environmental negative impacts could be expected, diesel generators are highly mobile: they have the capacity to fill a missing link in energy supply networks, and provide short term solutions in emergency situations. They prove here to bring positive impacts to populations, basically by bringing missing energy: 62% of effects observed with fuel technology are related to the Energy & energy market type of effect.

Renewable technologies cover a much broader scope of socio-economic and environmental effects, with 76% of effects appearing in all types of effects except Energy & energy market, with also a relatively small number of effects on environment (4.1%). This suggests the importance of taking into account the type of effects in further analysis (see section 5.6).

Biomass technology has mixed results for development: projects using biomass technology have 21.2 pp more chance of having positive impacts than hydroelectric power (Table 6, col. 1). But projects based on biomass also show a higher risk of having negative impacts (+3.5 pp). As a matter of fact, theThe dependency of biomass solutions on the availability of biomass resource limits their ability to respond to energy demand.

Role of system size

As reference we use the nano size, which is the system size most commonly associated with SHS. All projects bigger than nano size have a high chance of generating positive impacts. Micro-grids have the highest chances to generate positive impacts (+36.5 pp, Table 6, col. 1)

The difficulty for nano size solutions to bringhave positive impacts may come from the observation that many projects based on nano systems fail, because they do not generate enough new income Documents de travail du Centre d'Economie de la Sorbonne -2018.39

to cover their cost [START_REF] Roche | Design of a solar energy centre for providing lighting and income-generating activities for off-grid rural communities in Kenya[END_REF]. However, this observation may be wrong due to a lack of accurate evaluation of such projects. As shown by Table 6, col 2, favorable effects of nano projects have a significantly higher probability of remaining unproven than those of other project sizes.

Role of project decision level

DEPs can be decided at many different levels, from the local to the country level (or even the multicountry level, which we aggregated with the country level, due to lack of sufficient number of observations of multi-country projects).

The level of project decision could have different types of consequences. On the one hand, a locally decided project might take population needs better into account; it might also be based on a governance structure attentive to promoting cooperation in resource management, thereby preventing the emergence of free-riding issues. On the other hand, projects decided at country level, or at multi-country level, could benefit from a higher degree of expertise, experience, and scalability. Economies of scale in knowledge accumulation and a higher level of expertise can help to find, at least from a technical point of view, the most efficient solutions; public management and supervision systems provide country authorities with accurate feedback from the field, which can be used to identify good and bad practices in the project cycle.

The combination of these two sets or arguments suggests that both bottom-up approaches and top-down approaches can trigger positive impacts, which may lead to a U-shaped relationship between the level of decision and the probability of obtaining positive impacts. This is what we observe in Table 6, col 1. Taking the local level as reference, we observe that the farther the decision level is from the local level, the lower the probability of a positive impact when we compare decision levels from local level to county level. Above the county level, the probability of obtaining positive impacts increases again, and reaches its maximum at the country (or multi-country) level (+8.6 pp as compared to the reference local level). However, this last result could be biased, because country or multi-country programs may have more resources to implement proper ex-post evaluations. The probability of obtaining unproven favorable effect is significantly lower at country level than at local level (Table 6, col 2). The higher probability of positive impacts at country level could thus result from a higher probability of conclusive evaluation.

Role of location

Location is considered at the scale of continents and introduced as a broad control of project context.

An important issue of DEP deployment is to know whether the local context can alter the effectiveness of technical features. Addressing this question, for small geographic areas would be affected by many unobservable variables, which have been identified in the literature as important Documents de travail du Centre d'Economie de la Sorbonne -2018.39

factors. Factors like distance to raw material, cost of resource transportation, light intensity, solar incidence, wind speed, cost of local power line extension according to the morphology of terrain, and population density may influence the total costs of production and/or system performance. We do not mean that these geographical factors are unimportant, they are important, but we cannot disentangle them from other unobservable factors that affect the outcome of DEPs.

Continental location gives some information about the area of economic influence. For instance, the literature shows a genuine know-how of rural electricity development in China, where 400,000

villages have been electrified using small hydroelectric power (SHP) systems NRGExpert (2013) and UNIDO ( 2010)). This experience is spreading into neighboring countries. Also, experts and engineers are different on different continents, and this difference might lead to different practices in electrification projects.

Asia was chosen as the reference location, due to its long experience in developing rural electricity based on DEPs.

Our model shows that projects in Latin America have a significantly higher probability to of having positive impacts (+5.9 pp, Table 6 col 1) than those in Asia, Projects in Africa have a significantly lower probability of having positive impacts (-13.9 pp, Table 6, col 1).

Significant pairs of size and technology

Table 7 shows the most contrasted interactions of system size and technology, by replacing the variables technology and size from Table 6 with their interaction. Only interactions with more than 30 observations were kept, and only the most contrasted pairs are shown. A positive value means that the interaction on the left has a higher probability of impact than the one on the right.

The two biggest contrasts are when comparing mini hybrid fossil projects with nano solar projects (+42.5 pp) and micro fuel projects with nano solar projects (+41.9 pp). As suggested from the separated effects in Table 6, solar per se may have a positive impact (although moderate), but solar projects are less efficient when the power is delivered through nano systems (i.e. in SHS), as compared to other combinations of size and technology of off-grid electricity production. The lower probability of positive impacts of nano solar projects is due to their size limitation: a too low supplied power does not permit access to all types of electrical appliances.

Half of the biggest contrasts between interactions of size and technology involve hybrid projects, with or without fuel. Deploying a micro size project with hybrid renewable technologies has +40.4 pp higher chance of generating positive impacts than a nano solar solution, and + 28.5 pp more than a micro hydroelectric power project. Even nano projects based on hybrid renewable electricity have a higher probability of having a positive impact than micro projects based on a hydroelectric power source (the "classic" SHP) (+24.8 pp)%) or nano projects based on solar source (+36.7 pp).

Documents de travail du Centre d'Economie de la Sorbonne -2018.39

Factors of success by type of effects

The distribution of effects observed with scientific data by type of effect is shown in Table 8. The highest concentration is for Energy & energy market and the lowest is for Environment. The relatively low number of scientific measurement of effects on the environment may be due to the relatively recent emphasis in development policies on this core aspect of sustainable development.

Economic transformation is also an effect for which observations are relatively few, perhaps due to the concentration of observations on SHS, which do not target productive uses of electricity. and 5%, respectively. For the other effect types the proportion of positive impacts is a little below the average (15%). Finally, the negative sign of a number of observations for Lifestyle & NICT suggests that these results could be affected by a publication bias.

Extending knowledge of effects with expert data

Given the shortage of information based on scientific data, it is tempting to try to expand our information base with expert data. Expert Data (ED) has two levels of data quality: effects that rely on observations without variance (N=1) (i.e. without confidence interval), and effects that are solely documented from other research papers, or simply mentioned in institutional reports (N=0).

In both cases, including ED in the estimation does not allow the separation of proven favorable (unfavorable) outcomes and unproven favorable (unfavorable) outcomes, as ED do not provide confidence intervals.

Hence the use of ED in a model, which enlarges the observation sample, limits the precision of the model because the number of possible outcomes is reduced to 3 (favorable, unfavorable, inconclusive). This modifies the results. Merging proven and unproven favorable effects hides the specific information provided by proven favorable effects. In our data unproven favorable effects are more numerous (940 versus 225), and Table 6 shows that proven and unproven favorable effects have different explanatory factors. As a result, as shown by the differences between Table 6 and As shown on Table 11, Solar technology would be beaten by most alternatives, except Wind (whose parameter could not be estimated in Table 6). The most effective technologies for development would be Geothermal, whose parameter also could not be estimated in Local decisions (whether from municipalities or districts) would be more efficient than any other level (col 1), and country levels of decision would bring more unfavorable effects (col 2).

These results are in contrast to the conclusions reached from Table 6. They are however fragile, as the significance of parameters is very sensitive to sample changes. When estimated on the subsample of scientific data, parameters associated with sources of energy and system size become non-significant (table not shown).

Discussion

Possible selection bias

The selection procedure with EBSCO ensures that no bias from a focus in search of a specific research frontier, which would result in a narrowly oriented selection of papers, remains. However, some selection bias could still occur.

First of all, impacts of DEPs can be evaluated only under 3 conditions:

-Effects arise from implemented projects -Researchers tested these effects -Effects were measured with observed data with heterogeneity.

Effects are not observable if researchers have not considered evaluating them: some effects were not considered relevant, or not of interest, at the time of evaluation, or not surveyed due to budget constraints on the collection of field data. This might be the case for environmental effects, which are rarely measured with field data in the papers entered in the CoSMMA. Some effects are only documented from other pieces of research, or quantified without any sample of observations. In both cases, the significance of an effect is not computable, and whether the reported effect is an evidence of impact remains unknown. Our results suggest that evaluations without statistical tests may lead to conclusions at odds with conclusions obtained from evaluations with statistical tests.

Resolving this issue is beyond the scope of this paper, but future developments of the CoSMMA, whose aim is to broaden the scope and depth of DEP evaluations, could contribute to the solution.

In addition, research on DEP impacts can be affected by publication bias, because research publications are driven by the need to show innovation and tend to favor significant positive results.

In our case, the relatively small proportion of positive impacts reported in evaluations may suggest that this publication bias is limited.

The publication bias can also be magnified by the possible lack of independence of the project assessor. Organizations implementing funded projects need to demonstrate ex-post that positive impacts occur, and might be tempted to resort to assessors dedicated to showing positive impacts.

This may explain why a large proportion of papers that we have collected in the CoSMMA report favorable effects without providing scientific evidence in support of their conclusions. At this stage Documents de travail du Centre d'Economie de la Sorbonne -2018.39

of development of the CoSMMA, our choice of using evaluations based on scientific data, and not on expert data, is the only way to deal with this issue and identify best practice.

Finally, sample size could also be the origin of a publication bias. The cost of field evaluations puts a budget constraint on the sample size that researchers might be able to collect: as a result, small studies with limited samples might show significant effects only for those studies with the largest magnitude effects. The reason for this is that the critical size sample is a convex decreasing function of the magnitude of the assessed effect Châtelain (2010).

Cycle between funding and evaluation

Because projects are risky, donors or lenders tend to commit funds to new projects that show comparable specifications to previous successful projects, especially when they have defined risk management policies based on project characteristics. Publication bias might therefore sustain conservative commitment strategies, repeating the funding of the same type of projects as those which have shown large effects with small samples. In the absence of any third-party evaluations of electrification projects, the cycle of decision/evaluation/judgment could continue unabated. There could be a virtuous/vicious circle between publication bias and project commitment, each nurturing the other, a cycle that is all but random.

Concluding remarks

This research is the first step of the CosMMA project, towards a better understanding of the potential contribution of DEPs to sustainable development, with the aim of identifying best practice. In this pilot CoSMMA we have assembled a database of the characteristics of 421 DEPs and their effects on sustainable development.

The results of our meta-regression highlight the roles of energy source and system size. There is clearly a trade-off between the choice of new sources of renewable energy, especially solar energy, and system size. Solutions relying on solar energy alone bring positive impacts, but these impacts are reduced because solar electricity is mainly delivered through nano systems, whose positive impacts are much less frequent than positive impacts of larger systems such as Mini-grids. Hybrid systems may provide an interesting compromise, because they can be larger than SHSs, and also help solve other technical issues such as intermittence. Our results also suggest an important role for the organizational characteristics, as evidenced by the U-shaped curve describing the influence of the decision level.

So far, we have been able to use only scientific data, and not expert data, for lack of comparable precision of data provided by experts and data reported in econometric research. As a consequence, the sample size available for meta-analysis is smaller than what we have in the CoSMMA, which limits the breadth of our exercise. In particular, data limitations prevented us from exploring systematically the different types of effects of DEPs. In particular, we could not reach any Documents de travail du Centre d'Economie de la Sorbonne -2018.39 conclusions in many fields that are critical for sustainable development, such as environment, health, and education. Also, we could not assess best practice related to the objective of economic transformation. Hopefully, the fast development of DEPs may remove these limitations in the future.

One possible direction for further research could be related to the assessment of different uses of electricity. The CoSMMA project could sustain further studies to measure which uses of electricity matter in terms of economic development, based on the proven effects of DEPs, in relation to the power (system size) which is supplied to customers. As a result, electrification project developers could quantify the need for power given the socio-economic conditions of the targeted off-grid area, and thus project sizing could be optimized, increasing the survival probability of projects.

Measuring the latent demand is important because the development path of electrical appliances that was followed by households in advanced countries cannot be replicated today in developing countries. European consumers started to buy fridges during the 1950's and mobile phones in the 2000's. In contrast, African households have reached a 60% equipment rate in mobile phones in the last 5 years, but rarely own a fridge. It is thus crucial to further analyze what will be the household preferences for electrical appliances.

Our meta-analysis of DEPs effects gives a preliminary contribution to the measurement of latent demand for electricity, as positive impacts of electrification may be considered as proxies of electricity uses in developing countries.

In addition, our meta-analysis emphasizes which project characteristics have the highest probability of having positive impacts on sustainable development, and this should help developers to relate project design to expected electricity uses.

Presenting best practices of decentralized electrification may both encourage better sizing of projects, and also provide first indications for further research on the measurement of latent demand for decentralized electricity.

Figure 1 -

 1 Figure 1 -Number of papers by publication period (4 years) ............................................................................

Figure 2

 2 Figure 2 Map of DEPs registered in CoSMMA ..................................................................................................17

Figure 1 -

 1 Figure 1 -Number of papers by publication period (4 years)

  1993 1997 2001 2005 2009 2013 2017 No. of articles (4 years) Current period (2017-2018) Documents de travail du Centre d'Economie de la Sorbonne -2018.39

Figure 2

 2 Figure 2 Map of DEPs registered in CoSMMA



  Micro: 1 to 100 kW  Mini: 100 kW to 100 MW More than 70% are nano systems, and these nano systems are mostly solar based: the most frequent type is Solar Home System (SHS). Documents de travail du Centre d'Economie de la Sorbonne -2018.39

Table 2 -

 2 Review of literature or meta-analysis about electricity economics (as of Oct 12 th 2018) ....................

Table 3 -

 3 Listed studies in electricity economics (as of 12 th Oct 2018) .............................................................

Table 4 -

 4 Project inclusion criteria ...................................................................................................................

Table 5 -

 5 Scientific vs. Expert data ..................................................................................................................

Table 6 -

 6 Effectiveness characteristics of DEPs -Average Marginal Effects (AME) ..........................................

Table 7 -

 7 AME of impacts -Highest Significant Pairwise Comparisons ............................................................

Table 8 -

 8 Distribution of effects by type (scientific data) .................................................................................

Table 9 -

 9 Distribution of impacts by effect type (scientific data) ......................................................................

Table 10 -

 10 AME -Impacts -scientific data -separated regressions by subset of Effect Type ............................

Table 11 -AME-with expert data (N >= 0) .......................................................................................................

  Identifying best practice in DEPs requires an innovative methodology, because the focus on energy in Sustainable Development Goals (SDGs) is recent. Few DEPs have been assessed in a rigorous evaluation framework, although many observations of DEP effects are available in other areas of

Documents de travail du Centre d'Economie de la Sorbonne -2018.39

2.3. The potential contribution of a meta-analysis

Table 1

 1

	. The CoSMMA offers an original contribution, mapping a wide scope
	of DEP effects in developing countries with observed data.
	Special attention must be paid to SE4ALL (2017) and Katre and Tozzi (2018).
	SE4ALL (2017), Why wait?, was the first study to assess the effects of access to electricity appliances
	in developing countries on SDGs, using the multi-tier framework defined by ESMAP (2015). There
	was a similarity with our objectives, although CoSMMA covers more countries (69) and indicators
	(765). Furthermore, by considering DEP effects published by researchers, we evaluate DEP impacts
	without any preconceived assumption about which impact should be evaluated first or might be
	expected to arise initially. Effects in the CoSMMA were collected for 157 dimensions, grouped in 15
	topics and 8 types.
	Katre and Tozzi, (2018)propose a complete comprehensive scorecard for DEP evaluation, but do not
	use field data. Using observed or reported effects of DEPs, we are able to feed a large
	Documents de travail du Centre d'Economie de la Sorbonne -2018.39

Table 2 -

 2 Review of literature or meta-analysis about electricity economics (as of Oct 12 th 2018)

	Review	Key words	Response/reference	Title

[START_REF] Jamasb | A Quarter Century Effort Yet to Come of Age: A Survey of Electricity Sector Reform in Developing Countries[END_REF]

Creating a Smarter U.S. Electricity Grid Off-topic: US Davis (2012) Prospects for Nuclear Power Off-topic: US, Nuclear Power Borenstein (2012) The Private and Public Economics of Renewable Electricity Generation

Table 3 -

 3 Listed studies in electricity economics (as of 12 th Oct 2018)

	Reference	Number of listed studies
	Praktiknjo et al (2011)	16
	Jamasb et al (2017)	18
	Bonan et al (2014)	20
	Brenneman and Kerf (2002)	40	2
	Peters and Sievert (2015)	9
	Thopil and Pouris (2015)	15
	(van Gevelt (, 2014)	90
	3. Sources of data

3.1. Methods used to build the CoSMMA

Research papers used to document DEPs in the CoSMMA were taken from 4 economic research academic databases: Academic Search Premier, Business Source Complete, EconLit, GreenFILE.

Table 4 -

 4 Project inclusion criteria

	Projects in sample must -	Initial number of observations -2818	Number of excluded observations
		effects
	be operative (or eventually have been	Projects must have a commission	105
	operative)	date before 2018. Defaulted projects
		are accepted.
	not be in OECD and must have an	Effects from projects in OECD	15
	understandable continental location	countries or with unclear continental
		location ("worldwide" studies) are
		excluded
	deliver capacity below 100 MW	Application of Dillig et al (2016)	3
		criteria : projects with capacity below
		100 MW are not involved in
		balancing, nor in market exchanges
		nor in clearing
	use a clear specified technology	Effects from studies based on	85
		existing energy mix are excluded
	be deployed in rural area	Effects from projects in urban area	1
		are excluded
	be evaluated with samples that do	Observations above 5 times the usual	2
	not show unusually high extreme	threshold for Dfbeta were excluded
	values
	be evaluated with samples of normal	Samples with observation number	2
	size	larger than the 99% quantile of this
		number were excluded. The
		threshold was 352,800 observations
		Number in large sample -2,605	Total number of deleted
		observations	observations (effects) -213

Documents de travail du Centre d'Economie de la

Sorbonne -2018.39

Table 5 -

 5 Scientific vs. Expert data

	Denomination	Type	Number of obs. (N)
	Scientific data	Quantified effect with variance	n >1
	Expert data	Quantified effect without variance	n=1
	Expert data	Documented effect from Research	n=0
	Expert data	Unmeasured effect	n=0

Documents de travail du Centre d'Economie de la

Sorbonne -2018.39

Table 6 -

 6 Effectiveness characteristics of DEPs -Average Marginal Effects (AME)

	Positive	Unproven
		favorable

Documents de travail du Centre d'Economie de la

Sorbonne -2018.39

Table 8 -

 8 Distribution of effects by type (scientific data)

	Type of Effect
	No.	%

Table 9 -

 9 Distribution of impacts by effect type (scientific data) Documents de travail du Centre d'Economie de la Sorbonne -2018.39 Documents de travail du Centre d'Economie de la Sorbonne -2018.39Due to the limited number of observations, the set of controls was restricted to two essential controls, the number of observations (N) and the method of estimation.The parameters associated with technology in the Lifestyle & NICT effect type could not be computed. They were estimated in the Energy & energy market effect type, but none were significant.

	Type of Effect
	No.	%

Table 11 ,

 11 with Expert Data included, the unproven favorable effects dominate and lead to a totally different picture of DEP effectiveness.

Documents de travail du Centre d'Economie de la

Sorbonne -2018.39

Table 11 -

 11 AME-with expert data (N >= 0)

	Favorable	Unfavorable	Unknown
			direction

Table 6 (

 6 +10.8 pp, Table11,

col1) and Hybrid renewables (+3.7 pp).

Nano solutions would be a significantly better choice than Micro-grids (+6.3 pp, col 1) and would have a similar performance to Mini-grids. Documents de travail du Centre d'Economie de la

Sorbonne -2018.39

  Documents de travail du Centre d'Economie de la Sorbonne -2018.39 Annex 4 -Distribution of subsets of positive impacts Documents

		Technology
		No.	%	Cum. %
	Hydroelectric power source	5	2.2	2.2
	Solar	165	73.3	75.6
	Hybrid with Fossil fuel	6	2.7	78.2
	Hybrid renewables	32	14.2	92.4
	Biomass (and related tech.)	14	6.2	98.7
	Fossil Fuels	3	1.3	100.0
	Total	225	100.0
		System size
		No.	%	Cum. %
	Nano: <1 kW	198	88.0	88.0
	Micro: 1 to 100 kW	25	11.1	99.1
	Mini: 100 kW to 100 MW	2	0.9	100.0
	Total	225	100.0
		Decision Level
		No.	%	Cum. %
	Country	44	19.6	19.6
	Province	105	46.7	66.2
	County	16	7.1	73.3
	District	6	2.7	76.0
	Local	54	24.0	100.0
	Total	225	100.0
		Continent
		No.	%	Cum. %
	Africa	133	59.1	59.1
	Asia	66	29.3	88.4
	Latin America	26	11.6	100.0
	Total	225	100.0

de travail du Centre d'Economie de la Sorbonne -2018.39

Only papers about the impacts of energy are counted. Papers about the impact on growth are not counted. Documents de travail du Centre d'Economie de la Sorbonne -2018.39

The diagnosis of colinearity was made with the sub-sample of effects in Energy market, which is the largest sub-sample (384 observations before estimation).). The number of observations (N) had a first occurrence at second rank in the Condition Number matrix for the set of explanatory variables: it should thus not be dropped in first intention. The delay of evaluation was previously used as a qualification variable, not as a strong control: it was thus preferably dropped, as compared to and the variable Method was used. Documents de travail du Centre d'Economie de laSorbonne -2018.39

Acknowledgments

The authors thank Thomas André, Christophe Angely, Rémi Bazillier, Jean-Bernard Châtelain and Josselin Thuilliez for useful comments and discussions on this project; Fatou Cisse for her key support for data collection and data quality control; Olivier Santoni for his expertise dealing with geographical data. Both authors gratefully acknowledge support from Fondation pour les Etudes et Recherches sur le Développement International (Ferdi) and the French National Research Agency under program ANR-10-LABX-14-01.

As a consequence of this structure of effects by effect type, identifying factors that lead to positive impacts on the various types of effects is constrained by the small number of available observations for most of the types. Table 9 shows that only three effect types have more than 30 positive impacts in the CoSMMA. Due to the peculiar heterogeneity of these impacts in each effect type, estimates could be computed for only two types of effect: Energy & energy market, and Lifestyle & NICT.

Table 10 shows separated regression results. For each type of effects, only observations with effects of this type were selected, thereby defining separated multi-probit estimations. Only parameters related to the probability of positive impacts are reported.


01/16 0.4 UCLO exploitation plans and edits Canini 07/01/16 0.5 DE-CIX exploitation plans and edits Bleidner, Dietzel 20/01/16 0.6 Final version based on internal reviews Owezarski H2020-ICT-2014

During the first year, ENDEAVOUR has to focus on spreading the project ideas and use case scenarios through publishing news on different project / platform websites, forums, etc. Specifically, the main dissemination goal is to create a group of followers (network providers, network operators, IXPs, end-users) eager to use our results for their own network management and operation activities, services and deployment. A secondary goal is to collect requirements express by IXP operators and IXP end users, and issue related use cases. To this end, the dissemination plan of the partners of the ENDEAVOUR project consists of the following activities.

• Publishing premier high-quality papers in major international conferences and journals in the area of networking to promote new ideas and concepts stemming from project activities and outcomes.

• Gaining requirements from IXP operators and members.

• Dissemination of results and prototype implementations to companies for possible industrial exploitation.

Logo

To clearly identify the project in all dissemination activities, the name EN-DEAVOUR is used as a reference. Also, a logo has been designed (Fig. 1). The purpose of our logo lies on the target we want to achieve. The cross in the middle of the logo represents the new IXP fabrics that ENDEAVOUR aims to build, and that interconnects members, collects the traffic and forwards it to its destination. The boat wheel around explains that the main part of the work deals with finding out new ways for managing the IXP fabrics, and for driving the whole IXP ecosystem.

Project presentations

Project presentations include poster, flyer, fact-sheet, press releases, and announcements. This activity has the twofold objective of developing contacts in the research community involved in activities within the scope of the project, namely academics and data scientists working on IXP, and reaching a wider audience potentially interested in applying the outcomes of the project in possibly different contexts. H2020-ICT-2014-1 Project No. 644960 Figure 1: The ENDEAVOUR logo First, a poster, a flyer and a fact-sheet (ENDEAVOUR deliverable 5.1) presenting the main motivations behind the project have been produced and are available for download at https://www.h2020-endeavour.eu/press-desk.

For reaching a wider audience, DE-CIX issued a press release on its EN-DEAVOUR activity for announcing its research work for renewing its IXP fabric and management. The press release is available at https://nyc.decix.net/news/latest-news/news/article/de-cix-develops-new-tools-for-contro lling-data-traffic-1/ or from the ENDEAVOUR website at https://www.h202 0-endeavour.eu/press-desk.

Three other press releases have also been issued since the beginning of the project, but this time by companies not involved in the ENDEAV-OUR project, and Bloggers in the networking area. It includes the press release from the Pica8 US company (http://www.pica8.com/company/pressreleases/pica8-powers-sdn-driven-internet-exchange.php), the Reyno report (https://www.sdxcentral.com/articles/rayno-report/pica8-pushes-out-ciscoat-touix/2015/06/), and the LightReading blog (http://www.lightreading. com/carrier-sdn/sdn-technology/french-exchange-sees-classic-sdn-benefits /a/d-id/716901).

Finally, a Twitter account has been created and is regularly updated in order to reach a wider public and let followers of the project stay up to date. It is available at https : //twitter.com/h2020 endeavour.

Project Website

ners, reviewers (European Commission) and anybody else interested in the ongoing research carried out at the ENDEAVOUR project. A detailed description of this portal can be found in the ENDEAVOUR deliverable D5.2. As a key aspect, the ENDEAVOUR portal contains links to the main communication channels for dissemination and feedback. This website, publicly accessible at https://www.h2020-endeavour.eu/, describes the main goals of the ENDEAVOUR project. It also features a collaborative tool for the consortium members. The portal is based on Drupal, an open source CMS tool. or categories which are, (see Fig. 2):

• Home

• Objectives • Consortium • Deliverables • Dissemination • Events • Resources • Contacts • Reviewer Area

Conferences, workshops, and journals

One of the important dissemination activities deals with publishing premier high-quality papers in major international conferences and journals, and giving talks in workshops in the area of networking to promote new ideas and concepts stemming from project activities and outcomes. This dissemination activities also include giving talks, making demos, or presenting tutorials. Currently published papers are listed below:

• Canini, M., Kuznetsov, P., Levin, D., Schmid, S. (2015 Talks given are listed below:

• Antichi, G., "OSNT: A Community-owned Platform for High-performance and Low-cost Network Testing", RIPE Meeting (Open Source Working Group), May, 2015. Amesterdam, NL.

• King, T., "jFlowlib", RIPE Meeting (Open Source Working Group), May, 2015. Amesterdam, NL.

• Dietzel, C., "Measuring Delay and Packet Loss at an IXP", RIPE Meeting (MAT Working Group), May, 2015. Amesterdam, NL.

• King, T., "Making Route Servers Aware of Data Link Failure at IXPs", RIPE Meeting (Open Source Working Group), May, 2015. Amesterdam, NL.

• J. Rabadan, Ed.: S. Sathappan, K. Nagaraj, W. Henderickx, T. King, D. Melzer, "Operational Aspects of Proxy-ARP/ND in EVPN Networks", BESS Workgroup, July, 2015.

• Bruyere, M,. "SDN with Pica8", France-IX General Meeting, 18th September 2015, Paris, France.

• Harald A. Summa, Arnold Nipper, "DE-CIX update and outlook", DE-CIX summit 2015.

• King, T., "Software-Defined Networking", Insider Talk Episode 3, 2015.

H2020-ICT-2014-1 Project No. 644960

• Canini, M., "ENDEAVOUR: Towards a flexible software-defined network ecosystem", SWITCH SDN Workshop, 11th June 2015, Bern, Switzerland.

• Canini, M., "NetPaxos: Consensus at Network Speed", Microsoft Research, 16th June 2015, Redmond, WA, USA.

• Canini, M., "NetPaxos: Consensus at Network Speed", VMware Research, 18th June 2015, Palo Alto, CA, USA.

• Canini, M., "iSDX: An Industrial-Scale Software-Defined IXP", SWI-TCH SDN Workshop, 13th November 2015, Zurich, Switzerland.

• Canini, M., "ENDEAVOUR: Towards a flexible software-defined network ecosystem", RIPE Meeting (Plenary Session), 17th November 2015, Bucharest, Romania.

• Canini, M., "NetPaxos: Consensus at Network Speed", KTH, 27th November 2015, Stockholm, Sweden.

• Canini, M., "ez-Segway: Decentralize for Faster Consistent Updates", Telecom ParisTech, 29th November 2015, Paris, France.

Two conferences tutorials have also been given in two conferences:

• Zilberman, N., Antichi, G., Rotsos, C., "Open Hardware Networking", Tutorial in The 2015 ACM Conference on Special Interest Group on Data Communication, ACM.

• Zilberman, N., Antichi, G., Rotsos, C., "Open Source Networking", Tutorial in Network Softwarization (NetSoft), 2015 1st IEEE Conference on, IEEE.

Open source releases

The software developed within ENDEAVOUR will be released as open source as much as possible to foster dissemination.

During the first year of the project, jFlowLib has been issued. JFlowLib is a Java library to parse and generate sFlow and IPFIX data. It is freely available at https://github.com/de-cix/jFlowLib, or from the ENDEAV-OUR website at https://www.h2020-endeavour.eu/software. H2020-ICT-2014-1 Project No. 644960

ENDEAVOUR Workshops

The ENDEAVOUR consortium has been involved in the organization of four workshops, in which the ENDEAVOUR activities and results have been promoted. The rest of this section just lists these four workshops contribution without entering into details. For a detailed presentation, interested readers can refer to ENDEAVOUR deliverable D5.3.

Workshop DE-CIX Customers

During the first workshop [START_REF]Workshop: The Next Generation of Interconnection[END_REF], ENDEAVOUR met with DE-CIX customers in Cologne on the 19 th of February 2015. With more than 600 customers, DE-CIX is one of the largest IXPs worldwide, and hence an ideal player for ENDEAVOUR. During this workshop, the ENDEAVOUR consortium presented the project to a group of ISPs and explored their interest and expectations on how to benefit from the outcomes of the project.

1.6.2 12-14 April 2015: 26 th Euro-IX Forum

The Euro-IX Association [START_REF]Euro-IX: European Internet Exchange Association[END_REF] was formed to develop and strengthen the IXP community. Declared objectives are to promote open exchange of ideas and experiences, develop common procedures, gather information on regulatory issues affecting IXPs, and develop common procedures.

The Euro-IX Association organizes meetings for its members to contribute and benefit from the community's expertise and experience. The members meet twice a year at the Euro-IX Forums and gather a large and heterogeneous audience. While the Euro-IX meetings take place at different European locations it includes IXPs from all over the world. The 26 th Forum took place in Marseille [START_REF]26th Euro-IX Forum[END_REF] and we seized this opportunity to present ENDEAVOUR in an interactive plenary talk. The audience we addressed during this meeting is shortly described within the next section.

At the 26 th Euro-IX Forum, 126 attendees from 68 organizations came together. Dominated by European IXP operators, the audience of this second ENDEAVOUR workshop, was highly complementary to the one of the previous workshop and benefited from the comprehensive scope of the Euro-IX Forum, bringing together IXPs from all over the world, but also IXP members such as Microsoft, Google or Schneider Electrics, and equipment vendors such as Alcatel-Lucent or Cisco. The nature of the discussion reflects the heterogeneity of the audience, and while dominated by technical topics, non-technical ones such as regulation, are also within the scope of the forum. H2020-ICT-2014-1 Project No. 644960 1.6.3 24 September 2015: 6 th BNIX Networking Event Belnet organized a meeting with more than 75 attendees representing the customers and users of BNIX (Belgian National Internet eXchange). During the event, the ENDEAVOUR consortium introduced the project in an informal settings.

1.6.4 25-27 October 2015: 27 th Euro-IX Forum Same objectives as for the 26 th Euro-IX Forum, the 27 th edition was organized in Berlin from the 25 th to 27 th of October 2015. The 27 th Euro-IX was a great success with a record of 140 attendees representing 65 organisations (including 47 IXPs from 36 countries).

20 November 2015: ENDEAVOUR BoF session at RIPE71

RIPE 71 was organized in Bucharest on the 20 th of November 2015. BoF (Birds of a Feather) sessions are held at RIPE Meetings to start an informal discussion on a topic of shared interest among RIPE Meeting attendees. If a BoF attracts sufficient interest, it might become a RIPE Task Force or Working Group. The ENDEAVOUR project organized one of the BoF sessions.

Outlook

With the first ENDEAVOUR workshops we seized the opportunity to involve small to medium IXP members as well as IXP operators. Additionally, our direct contacts enriched our perspective by reaching to other Internet stakeholders. This valuable input will be used to align our use cases even further to their needs and thus make them more relevant for practical application in the future. Notably, WP4 benefits directly from the workshops and will feed its outcomes directly into the desired SDN architecture of WP2.

Furthermore, we brought ENDEAVOUR widely to the attention of the IXP community. We arouse awareness of the benefits that SDN has to offer and got feedback on the concerns regarding its implementation. Our experience proves that it is important to keep the community in a close feedback loop to achieve the desired impact. The rather big players have not been in the focus of our dissemination yet, however this remains one goal for the near future. H2020-ICT-2014-1 Project No. 644960

Standardization

ENDEAVOUR also contributed to a standardization working group at the IETF: the BESS WG. It leads to issuing an Internet draft:

• Rabadan, J., Sathappan, S., Nagaray, K., Henderickx, W., King, T., Melzer, D., "Proxy-ARP/ND function in EVPN networks", IETF Internet draft, BESS WorkGroup, 2015

Other activities

ENDEAVOUR consortium launched a survey for network operators. As part of our effort to bring SDN into IXPs, we want to create a broad picture of the operational challenges currently faced when peering at IXPs. Specifically, our goal is to understand:

• what are the current limitations at IXPs?

• what services do you expect to see at IXPs in the next years?

Survey results will help us to drive our efforts in characterizing the use cases of the ENDEAVOUR project. This survey also serves to help us to understand our audience better; by indicating where network operators consider their core business, this helps us to understand if we are talking to the right people and using the right language.

The survey is anonymous unless answering organizations desire to reveal their identity. The data we acquire will be treated confidentially and only reported in aggregate statistics and anonymized format.

The survey is available at: https://www.h2020-endeavour.eu/launchingsurvey-network-operators.

Exploitation plans 2.1 Academic Partners

Academic partners have common objectives in terms of dissemination and exploitation. As such, for all academic partners involved in the ENDEAV-OUR project, dissemination and exploitation plans include:

• Publishing premier high quality papers in major top conferences and journals in the area of networking, security, and QoS to promote our new disruptive ideas and concepts.

H2020-ICT-2014-1 Project No. 644960

• Gaining significant skills in the area of IXP fabrics building and management, and all related applications and services. Such skills and knowledge will lead to new courses to be held to PhD students thanks to summer schools, or to undergraduate students of universities and engineering schools.

• Finding new research directions and prospecting for new academic and industrial partners for future research projects based on skills and results gained in ENDEAVOUR.

• Disseminate our upcoming results and prototype implementation to company for possible industrial exploitation.

Details on the status of this dissemination and exploitation plans for each academic partner are given in the following.

Queen Mary University London

The Queen Mary University of London exploitation plans include:

• The publication of academic papers, some of which will be submitted soon.

• Talks on topics related to ENDEAVOUR.

• Collaboration with potential industrial partners: the Queen Mary University of London collaborates with the TouIX IXP in Toulouse, CNRS and Cambridge in the design of the UMBRELLA architecture.

Université catholique de Louvain

For its exploitation purposes, the UCL plans include:

• Papers that have been published and other papers that will be submitted soon.

• Talks on topics related to ENDEAVOUR.

• Achieve a positive impact on IXP operations at DE-CIX.

• Collaborations with new partners: the UCL has collaborated with Princeton University and ETH Zurich on the design of iSDX, a scalable SDX design and open source implementation that has already spurred early adoption in a US government agency. The UCL has also started H2020-ICT-2014-1 Project No. 644960 collaborating with Università della Svizzera italiana and attraced the interest of a potential industrial partner, Netronome, for the work on NetPaxos.

University of Cambridge

University of Cambridge started gaining contributions for its exploitation purpose:

• Papers are in the process of being submitted shortly.

• Talks, Tutorials and Demo have been carried out successfully.

• Dealing new possible industrial partners: University of Cambridge together with CNRS collaborate with the TouIX IXP in Toulouse. In particular, the design of the UMBRELLA architecture, where UCAM has been actively involved, helped to create a strong relationship with the aforementioned IXP. While the ENDEAVOUR consortium can benefits demonstrating its outcomes on a small environment, TouIX can benefit demonstrating new technologies thus potentially attract new customers.

CNRS

As shown in this deliverable, CNRS started gaining contributions for its exploitation purpose:

• Papers have been published or are in the process of being submitted shortly.

• For academic teaching, even if no specific lecture has been yet included for students at university or engineering schools, the way networking is taught takes into consideration the new requirements that network operators, IXP operators, and end users expressed.

• Dealing with prospection for new industrial partners, CNRS is now involved in a project with the TouIX IXP in Toulouse. The development objectives of TouIX and ENDEAVOUR being very close, and CNRS being involved in both projects, we are trying to set-up a win-win strategy between the two projects. Thus, ENDEAVOUR consortium largely benefits from being able to take advantage of a small and open IXP for testing some of the solution that it designed (for instance the UMBRELLA architecture). On the other side, TouIX can demonstrate H2020-ICT-2014-1 Project No. 644960 the benefits of such new technologies to its members, and potentially gain new members.

• CNRS and TouIX have been selected for organizing the Euro-IX event in 2017. This will be a premier exposure for ENDEAVOUR results.

Industrial Partners

Beside the industrial focus, ENDEAVOUR's industrial partners also have a strong research interests, which is in-line with the academia exploitation plan including:

• Publication of high-quality papers in major top conferences and/or workshops.

• Talks, tutorials and/or demos on topics related to ENDEAVOUR's goals.

• Exploiting new research directions in the area of SDN at IXPs.

• Collaborations with new partners: establish close cooperations with academic partners for ongoing research activities as well as future EUfunded projects, for instance in the field of networking, SDN and monitoring.

DE-CIX

The DE-CIX exploitation plans include:

• Extend DE-CIX's peering platform, services, and development activities with the ideas, concepts, and prototypes developed by ENDEAV-OUR. DE-CIX focuses on bringing those prototypes to a productionready state for commercial deployment. DE-CIX operates IXPs at various locations at different scales, thus enabling to deploy SDN at smaller IXPs first, while exploring possibilities to scale the technology to larger IXPs successively.

• Contribute to standardization processes at the IETF to make the outcomes of the ENDEAVOUR project available to a wider audience with a strong practical focus.

• Giving talks at operator centric conferences, such as RIPE, NANOG, or EURO-IX to promote the results of ENDEAVOUR.

H2020-ICT-2014-1 Project No. 644960

• Discuss the ENDEAVOUR outcomes with IXP members to align with their requirements for the future. Furthermore, DE-CIX targets SDN deployments in particular with a direct benefit for its members in mind.

• Closely cooperate with hardware vendors to support them building SDN capable hardware suited for the needs of IXPs, especially in terms of scalability and feature support.

IBM Research

The IBM Research exploitation plans include:

• Exploring potential use cases and applicability of ENDEAVOUR to IBMs product portfolio, in the areas of datacenter networking, Cloud and High-Performance Computing (HPC) interconnects. IBM may exploit the ENDEAVOUR results for their developments of IT systems for Cloud, datacenter, virtualization, and Big Data appliances, potentially also including emerging trends in custom HPC.

• Provide input to SoftLayer and IBM labs (Cloud Innovation Lab, Watson) on ENDEAVOUR-related topics such as: 1) off-/on-line monitoring products, 2) improved management tools for multitenant datacenters, 3) the next-next-generation of virtualization extensions to be added to the currently limited SDN/OpenFlow, as well as monitoring and sampling (sFlow, AQM/RED) networking standards and 4) potentially helping IEEE, IETF, ITU and the other standardization bodies with the results acquired in the area of SDN and distributed high speed (beyond 10Gbps) monitoring.

The commercial impact of these plans could be witnessed in:

• Lower IBMs Cloud TCO and power consumption.

• Increased security of the multitenant datacenter, while improving the workload/traffic observability -hence securing better QoE per tenant and lower management costs for Cloud operators.

• Enabling the standardization of IBM's next generation of monitoring for 100/400Gbps/1Tbps Converged Enhanced Ethernet architectures.

H2020-ICT-2014-1 Project No. 644960

Figure 2 :WP5

 2 Figure 2: Main menu of ENDEAVOUR website

  Contents 1 Dissemination Plan 1.1 Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Project presentations . . . . . . . . . . . . . . . . . . . . . . . 1.3 Project Website . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Conferences, workshops, and journals . . . . . . . . . . . . . . 1.5 Open source releases . . . . . . . . . . . . . . . . . . . . . . . 1.6 ENDEAVOUR Workshops . . . . . . . . . . . . . . . . . . . . 1.6.1 Workshop DE-CIX Customers . . . . . . . . . . . . . 1.6.2 12-14 April 2015: 26 th Euro-IX Forum . . . . . . . . . 1.6.3 24 September 2015: 6 th BNIX Networking Event . . . 1.6.4 25-27 October 2015: 27 th Euro-IX Forum . . . . . . . 1.6.5 20 November 2015: ENDEAVOUR BoF session at RIPE71 . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6.6 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Standardization . . . . . . . . . . . . . . . . . . . . . . . . . . 1.8 Other activities . . . . . . . . . . . . . . . . . . . . . . . . . . Industrial Partners . . . . . . . . . . . . . . . . . . . . . . . .

	WP5 / D5.4	ENDEAVOUR	Version 0.6
	1 Dissemination Plan
	3 Acronyms
		H2020-ICT-2014-1 Project No. 644960

2 Exploitation plans 2.1 Academic Partners . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Queen Mary University London . . . . . . . . . . . . . 2.1.2 Université catholique de Louvain . . . . . . . . . . . . 2.1.3 University of Cambridge . . . . . . . . . . . . . . . . . 2.1.4 CNRS . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 2.2.1 DE-CIX . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 IBM Research . . . . . . . . . . . . . . . . . . . . . . .

  Antichi, G., Rotsos, C., Moore, A. W. (2015, August). "Enabling Performance Evaluation Beyond 10 Gbps". In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (pp. 369-370). ACM.

). "A Distributed and Robust SDN Control Plane for Transactional Network Updates". In The 34th Annual IEEE International Conference on Computer Communications (INFOCOM 2015). • Dang, H. T., Sciascia, D., Canini, M., Pedone, F., Soul, R. "NetPaxos: Consensus at Network Speed". In The Symposium on SDN Research (SOSR 2015). • Capelle, M., Abdellatif, S., Huguet, M. J., Berthou, P. "Online Virtual Links Resource Allocation in Software-Defined Networks". In The 17th Annual IFIP International Conference on Networking (NET-WORKING 2015). H2020-ICT-2014-1 Project No. 644960 • Castro, I., Panda, A., Raghavan, B., Shenker, S., Gorinsky, S. "Route Bazaar: Automatic Interdomain Contract Negotiation". In 15th Workshop on Hot Topics in Operating Systems (HotOS XV). USENIX Association. The current list of demos performed is listed below: • • Han, J. H., Antichi, G., Zilberman, N., Rotsos, C., Moore, A. W. (2015, April). "An Integrated Environment for Open-Source Network Softwarization". In Network Softwarization (NetSoft), 2015 1st IEEE Conference on (pp. 1-2). IEEE.

The ENDEAVOUR public portal (https://www.h2020-endeavour.eu/) offers an extensive compilation of relevant information for project members, part-H2020-ICT-2014-1 Project No. 644960

-1 Project No. 644960 WP5 / D5.