Maarja Toots

Tarmo Kalvet

Robert Krimmer

Success in eVoting -Success in eDemocracy? The Estonian Paradox

Keywords: e-democracy, e-participation, e-voting, Estonia, case study

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Throughout time, democracy has continuously evolved and even undergone drastic changes -from face-to-face, via territorial to transnational societies. Most recently, the Internet fosters this transformation as it challenges the concept of state sovereignty and need for representation. Arguably, e-democracy as a transnational, location independent way for citizens to interact with their state and be able to communicate and deliberate in the way of a strong democracy, can be considered the concept for a third transformation following Dahl [START_REF] Dahl | Democracy and its critics[END_REF]. Consequently, there is a need for e-democracy instruments that help facilitate and shape such an e-democracy. Estonia has been a pioneer in developing electronic public services and today all public services include an e-service component. The eID card (the primary identification document for citizens and permanent residents) has enabled digital signing of documents since 2002 and remote e-voting in nationwide elections since 2005. Internet penetration has constantly increased -while in 2005, 58% of the population used the Internet, today 88% are internet users [START_REF]IC321: Computer and Internet users aged 16-74 by place of residence[END_REF]. Estonia undertook first steps to develop e-democracy in the early 2000s, creating the first e-participation platform in 2001 and holding electronic elections since 2005. However, while these early efforts placed the country among the top ten in the UN e-participation index from 2008 to 2012, Estonia seems to have fallen behind since then, now ranking 22nd out of 193 [START_REF]E-Government Survey[END_REF].

While Estonia has acquired the reputation of a successful e-voting country, we set out to ask if Estonia's success in e-voting also implies its success in e-democracy in a broader sense. To answer this, we developed an evaluation framework for the case study of e-democracy instruments in Estonia, by combining some of the success criteria often employed in information systems, e-participation and e-democracy literature, focusing on user acceptance on the one hand and the aspect of democratic legitimacy on the other. Thus, we are looking at the following criteria: 1) level of use; 2) user diversity, 3) stakeholder satisfaction with the system and 4) impact on the political process. While the first and the third category are typical IS success measures, the second and fourth aim to incorporate the component of democratic legitimacy, which is considered the overarching aim of e-democracy projects [START_REF] Prosser | eParticipation -Did We Deliver What We Promised?[END_REF]. The perceived legitimacy and success of a democratic exercise has been associated with not only engaging a sufficient number of participants but also reaching a diverse group of participants [START_REF] Prosser | eParticipation -Did We Deliver What We Promised?[END_REF][START_REF] Oecd | Promise and Problems of E-democracy[END_REF][6][START_REF] Karlsson | Democratic Legitimacy and Recruitment Strategies in eParticipation Projects[END_REF]. Similarly, the actual impact of e-democracy tools on political processes is considered a key measure of their success [START_REF] Oecd | Promise and Problems of E-democracy[END_REF]6].

The paper is structured as follows. Section two gives an overview of our research methodology. Section three establishes the theoretical framework for the analysis based on two relevant and complementary literature streams -public sector innovation generally and e-participation literature more specifically. This is followed by a summary of the Estonian e-voting system in section four and three major nationwide e-participation projects in section five. In section six we discuss the outcomes of the four e-democracy instruments in relation to the factors that have affected their success, followed by a few concluding remarks on the possible reasons why e-voting has worked more effectively in Estonia than e-participation.

Methodology

The aim of this study is to explore the steps that Estonia has undertaken in order to build e-democracy by analyzing the e-democracy instruments, such as TOM, Osale.ee, People's Assembly (Rahvakogu), most recently Rahvaalgatus.ee / Citizen OS1 , as well as e-voting, that have been implemented since the transformation of the public sector based on the use of the Internet caught on in the early 2000s. In particular, we are interested in identifying why e-voting works in Estonia and why e-participation does not.

The in-depth study of a contemporary phenomenon using multiple sources of evidence in its real-life context is a typical application of case-study methodology. Also, it is an area where there is traditionally -up to today -not enough empirical research [START_REF] Benbasat | The Case Research Strategy in Studies of Information Systems[END_REF] undertaken in the IS field.

Yin distinguishes between exploratory, descriptive and explanatory case study types [START_REF] Yin | Case Study Research. Design and Methods[END_REF]. As we intend to gain new insight in how an e-democracy is (not) being built, the exploratory approach is selected. Due to the unique situation Estonia is in -it is to date the only country in the world that offers e-voting in all its elections without any restrictions to all eligible voters [START_REF] Krimmer | The 2016 World-Map of E-Voting Activities[END_REF], it was abstained from choosing a comparative multiple case study setup and focus solely on the Estonian case. For conducting the actual case study research we follow Yin's three phases (i) define and design; (ii) prepare, collect; and (iii) analyze and conclude [START_REF] Yin | Case Study Research. Design and Methods[END_REF].

The data for compiling the case study were collected mainly through desk research in 2015, including existing studies, policy papers, reports, press releases, articles in the media, use statistics, legislative acts and government strategies.

Conceptual framework

Driven by the question why some e-democracy instruments, such as e-voting, seem to work better than others, we focused on studying the factors that make for a successful e-democracy tool. As e-democracy instruments can be viewed as a particular kind of public sector innovation, we found some useful guidance in public sector innovation research as well as e-participation and e-democracy literature.

Key success factors in public sector innovation

E-democracy instruments can generally be treated as public sector innovation concerned with "the creation and implementation of new processes, products, services and methods of delivery which result in significant improvements in outcomes efficiency, effectiveness or quality" [START_REF] Albury | Fostering innovation in public services[END_REF]. The emergence of literature on innovation genuinely attributable to the public sector can be observed since around 2000, with a focus on innovations in public services and governance [e.g., 13, 14 -16].

One of the most recent systematic accounts of public sector innovation is a literature review by De Vries, Bekkers and Tummers [START_REF] Vries | Innovation in the Public Sector: A Systematic Review and Future Research Agenda[END_REF] that maps influential factors in public sector innovation at different levels and in different stages, from idea-generation to adoption and diffusion. They distinguish key innovation drivers and barriers along four 2 For Osale.ee, an important information source was an evaluation report of the usage and usability of Osale.ee published by the Government Office (manager of the system) in spring 2015 (quoted in this paper as [START_REF]Osalusveebi ja valitsuse eelnõude infosüsteemi kasutatavuse analüüs[END_REF]), which relied on focus group interviews with Osale's managers and key user groups (ministry officials, civil society organizations, interest groups, and individual citizens). In addition, six semi-structured personal interviews were undertaken with key idea champions, IS managers and active users in April and May 2015, several of whom had also been involved in the creation of Osale's predecessor TOM.

For e-voting the experience as part of the OSCE/ODIHR election related activities on Estonia were of particular importance where one of the authors was able to participate. The opinions put forward in this article are all of the author's alone and should not be attributed to the OSCE/ODIHR or any other institution.

For Rahvakogu we were grateful to have had access to a forthcoming study thereon.

main categories: (1) environmental level, including regulatory pressures, environmental pressures (media attention, political and public demand), participation in networks and inter-organizational relationships;

(2) organizational level: resources, leadership styles, risk aversion, incentives/rewards, organizational structures, etc.; (3) characteristics of the innovation itself: ease of use, relative advantage, compatibility, cost, trustworthiness, etc.; (4) characteristics of innovators, including employee autonomy, tenure, mobility, knowledge and skills, creativity, commitment, etc. Some factors, such as leadership, were found to be important in all stages of innovation. The European Commission's report "Powering European Public Sector Innovation" [START_REF]Powering European Public Sector Innovation[END_REF], perhaps the most influential policy document on the topic, highlights similar barriers. Some of the key impediments to success are related to scattered competences, lack of coordination, unfavorable administrative and organizational culture, lack of resources, lack of leadership, risk-aversion and failure-avoidance, lack of collaboration and limited knowledge on how to apply and measure the outcomes of innovative processes.

Case studies of e-government innovation support these findings. Political will and innovation acceptance at all levels of the organization have been found to be key drivers of successful e-government innovations, while different stakeholder agendas, political turbulence and resource issues act as barriers [START_REF] Angelopoulos | Emerging Barriers in E-Government Implementation[END_REF]. Similarly, a case study of the Estonian e-government evolution identified leadership and public sector competencies, availability of resources, legislative and regulatory support, and the existence of (strategic) IT infrastructure as important drivers. The study also emphasized the importance of frequently underestimated factors: the competencies of the private sector, publicprivate collaboration and the actual process of technology transfer, including support mechanisms to public procurement of innovation [START_REF] Kalvet | Innovation: a factor explaining e-government success in Estonia[END_REF].

Key success factors for e-democracy instruments

The success and failure of e-democracy instruments has been associated with a variety of factors similar to those outline above for public sector innovation. A key success/failure factor seems to be their level of integration into organizational procedures and political processes [e.g., [START_REF] Oecd | Promise and Problems of E-democracy[END_REF]6,[START_REF] Freeman | Understanding E-Democracy: Government-Led Initiatives for Democratic Reform[END_REF]. E-democracy methods should have a clear mandate [6] and involve decision-makers from the outset [START_REF] Scherer | Hands-On Guideline for E-Participation Initiatives[END_REF]. Integration can be seen as a key prerequisite for impact, which to date seems to be limited at best [START_REF] Oecd | Promise and Problems of E-democracy[END_REF][START_REF] Scherer | Regional Participation Model to Engage Citizens in Distant Decision-Making[END_REF]. Another set of factors can be associated with organizational culture, attitudes and political support. In addition to organizational culture, broader cultural preconditions for e-democracy include a developed civil society, social trust and an open political culture [6,[START_REF] Freeman | Understanding E-Democracy: Government-Led Initiatives for Democratic Reform[END_REF].

The failure of many e-government initiatives has been attributed to overlooking the demand side and citizen's perspective [START_REF] Hsiao | Study of Factors Influencing e-Government Service Acceptance Intention: A Multiple Perspective Approach[END_REF]. Empirical evidence of e-participation tools suggests that their take-up has thus far been globally low [START_REF] Edelmann | Collaboration for Open Innovation Processes in Public Administrations[END_REF]. Neither have e-participation initiatives brought more people in decision-making, engaging just a narrow "elite" of politically active citizens [START_REF] Karlsson | Democratic Legitimacy and Recruitment Strategies in eParticipation Projects[END_REF][START_REF] Hindman | The Myth of Digital Democracy[END_REF][START_REF] Lidén | Qualities of E-Democracy: Examples from Sweden[END_REF]. Variables explaining participation include prior interest in politics, internet skills, younger age and high level of education [START_REF] Lidén | Qualities of E-Democracy: Examples from Sweden[END_REF], which is very similar to participation patterns in offline contexts [START_REF] Navarro | The Biased Inclusiveness of Local Democratic Innovations: Vehicles or Obstacles for Political Equality?[END_REF]. The challenge of attracting users implies the need to reckon with their needs and capabilities by engaging users in designing the e-participation tools [6,[START_REF] Talpin | When Deliberation Happens. Evaluating Discursive Interactions among Ordinary Citizens in Participatory Budgeting Institutions[END_REF]. Effective participation in the democratic debate also presumes particular requirements to system design, such as information accessibility and competent moderation [START_REF] Oecd | Promise and Problems of E-democracy[END_REF][START_REF] Venkatesh | User Acceptance of Information Technology: Toward a Unified View[END_REF]. Finally, the acceptance of any ICT-based democracy tool tends to be determined by their perceived usefulness and ease of use, the two central concepts in technology acceptance theories [START_REF] Held | The Transformation of Political Community: Rethinking Democracy in the Context of Globalization[END_REF]. It is assumed that user acceptance is higher for systems that require less effort, while demonstrating clear benefits for the user.

Either way, the development of an electronic democracy with transnational character [START_REF] Heindl | Elektronische Demokratie-"Dienstleistungen" des Staates: E-Voting, E-Legislation und E-Participation[END_REF] needs the further development of e-enabled instruments of democracy, i.e., e-initiatives, e-referenda and of course also e-voting instruments [START_REF] Krimmer | Evaluierungsbericht: E-Voting bei den Hochschülerinnen-und Hochschülerschaftswahlen 2009[END_REF]. E-voting takes a special role within this set of e-democracy instruments. Not only is it one of the most visible e-government projects which sometimes receive all the attention of the public, it also is often one of the most discussed and debated [START_REF] Caarls | E-Voting Handbook: Key Steps in the Implementation of E-Enabled Elections[END_REF].

The success of e-voting is often linked with an incremental, step by step, implementation [START_REF]Guidelines of the Committee of Ministers of the CoE on Transparency of E-enabled Elections[END_REF], careful consideration of stakeholders' interests [START_REF] Krimmer | The Evolution of E-voting: Why Voting Technology is Used and How it Affects Democracy[END_REF], as well as a holistic, interdisciplinary, approach [START_REF] Krimmer | e-Voting.at: Elektronische Demokratie am Beispiel der österreichischen Hochschülerschaftswahlen[END_REF]. It can be noted that e-voting is more focused on technological issues than other e-democracy instruments, partly due to the inherent paradox between unequivocal identification of voters on the one side and must not being able to establish a link between the vote and the voter, essentially keeping the vote secret and hiding the identity of the voter (preserving anonymity) [START_REF] Alvarez | Point, Click, & Vote[END_REF]. Due to the fact that to date most e-voting undertakings do not follow classical experimental setups [START_REF] Svensson | E-voting in Europe: Divergent democratic practice[END_REF] and are embedded in their national context [START_REF] Drechsler | Electronic Voting in Estonia[END_REF] it is hard to draw comparative conclusions and provide learning to others. We therefore decided to change the approach and conduct an in-depth analysis of a country's efforts around all kind of e-democracy instruments.

4

The case of e-voting in Estonia

The Estonian efforts around e-voting started in 2001 with a plan to introduce e-voting, allowing to cast votes remotely via the Internet (often also called "Internet voting") already for the Estonian 2003 parliamentary elections. Following the e-government logic this seemed liked the logical next step after e-tax reporting, e-banking and a paperless cabinet meeting of the government's ministers [START_REF] Alvarez | Internet voting in comparative perspective: the case of Estonia[END_REF]. It took two more years until e-voting become a reality, due to discussion around its constitutionality. With the first ever, countrywide, unrestricted, remote e-voting channel offered in legally binding elections, the 2005 municipal elections, Estonia manifested its narrative of being an ecountry. To date it remains the only country with such a universal approach to e-voting.

Several articles have been written about the Internet voting experience in Estonia [START_REF] Kalvet | Management of Technology: The Case of e-Voting in Estonia[END_REF][START_REF] Drechsler | Dispatch from the Future[END_REF][START_REF] Madise | E-Voting in Estonia 2005. The first practice of country-wide binding Internet voting in the world[END_REF][START_REF] Vinkel | Remote Electronic Voting in Estonia: Legality, Impact and Confidence[END_REF], but probably the most comprehensive overview can be found in [START_REF]Election Assessment Mission Report on the 6 March 2011 Parliamentary Elections in Estonia[END_REF]. Here Vinkel classified the development of Estonian e-voting in three stages: (i) setup period (2002-2005), (ii) growth period (2005-2011), (iii) maturity period (from 2011). In the first period the main technological decisions were taken (usage of the card; doubleenvelope algorithm). In the second phase, a continuous exponential increase in usage was experienced, while the actual application was not changed in functionality, design nor usability. The ongoing third developmental phase was started by a security incident during the 2011 Riigikogu elections 3 . It was followed by an electoral reform with the introduction of individual verifiability as its main result [START_REF] Solvak | E-voting in Estonia:Technological Diffusion and Other Developments Over Ten Years[END_REF]. 5

Estonian e-participation projects

TOM

Estonia's first national-level e-participation project TOM or Täna Otsustan Mina (meaning "Today I Decide") was launched at the initiative of Prime Minister Mart Laar and his IT advisor as early as in 2001, possibly making it one of the first of its kind in the world. The online platform, administered by the Government Office, allowed citizens to make proposals for new legislation and policies and discuss and vote upon them. More popular ideas would be forwarded to relevant government officials, who then would have one month to post a formal response. Despite a relatively lively public interest in TOM, the project soon encountered challenges, such as a limited number of active users, low quality of ideas, limited impact of citizens' proposals and the prevalence of formalistic responses by officials over an open attitude to dialogue [START_REF] Postimees | Eesti e-riigi kuulsamaid lipulaevu kukkus läbi[END_REF]. By TOM's third birthday in 2004, e-democracy enthusiasts had declared it a failure [START_REF] Rugam-Rebane | Interview on 2015-05-15[END_REF]. According to interviews with implementers and idea champions of the project, TOM seemed to be ahead of its time. Government institutions lacked an understanding of how to integrate TOM-generated ideas into their work process and citizens lacked the knowledge and skills to formulate their ideas in sufficient quality and formats that officials could work with. According to TOM's administrator, there was a gap in the regulatory, strategic and political context -as government-wide discussions on citizen engagement policies only started around 2004-2005, the ground for e-participation was not yet fertile [START_REF] Tom Survey | TID+ website[END_REF]. 3 A student managed to program a Trojan horse that would cast a different vote than the one intended by the voter. He consequently filed a complaint to the election management body but this was eventually turned down. [START_REF] Vinkel | Presentation to the OSCE Human Dimension Committee[END_REF] Citizens were equally dissatisfied. In a survey involving 25 active users, a number of ideas for improvement were voiced, such as the need for more active promotion of the project, improved information accessibility (e.g. systematizing citizens' ideas according to topics), design updates, involvement of experts and moderators to increase the quality of debate, and integration with other government information systems [START_REF] Simson | Saada parimad ideed valitsusele[END_REF]. However, instead of re-designing TOM, the government decided to build a new e-participation tool (later named Osale.ee) and migrate TOM to the new platform. By that time, TOM had more than 7,000 registered users, who had generated 1187 ideas in total [START_REF] Hinsberg | Interview on 2015-04-30[END_REF] out of which no more than 1% were actually implemented by the government [START_REF] Postimees | Eesti e-riigi kuulsamaid lipulaevu kukkus läbi[END_REF].

Osale.ee

The idea for developing Osale.ee (www.osale.ee) emerged around 2004-2005 during the process of designing a government-wide policy for citizen engagement. The process brought together government officials and civil society activists and led to ideas for a new e-participation tool, which would address the shortcomings of TOM by better integration into formal rule-making processes [START_REF] Tom Survey | TID+ website[END_REF][START_REF] Hinsberg | Osalusdemokraatia veebi kaudu. Infotehnoloogia avalikus halduses[END_REF]. Consequently, the Government Office took the decision to develop a new e-participation portal which would enable officials to engage civil society in legislative drafting. The goal of the portal was to enhance the transparency, openness, quality and legitimacy of decision-making [START_REF] Gramberger | Citizens as Partners[END_REF].

Osale.ee was launched in 2007 as a platform for public consultations on legislative drafts. A year later, the system was upgraded with the functionality of an "improved TOM", which allowed citizens to propose ideas to the government, and gather comments and votes in support. Despite all the criticism of TOM, its functionality was preserved in Osale.ee because of TOM's high symbolic value, pressure from civil society and the wish to signal that the government had not lost interest in citizens' ideas [START_REF] Tom Survey | TID+ website[END_REF]. Osale.ee also included a third function -a search engine of government documents. It thus aimed to enable all three types of government-citizen interaction: information, consultation and active participation [START_REF] Runnel | The Estonian Tiger Leap from Post-Communism to the Information Society[END_REF].

Osale.ee intended to engage three kinds of users: officials of government institutions (mostly of the executive branch), individual citizens and their institutionalized representatives. In practice, however, the tool soon became criticized for failing to attract users and lack of impact [START_REF] Rahvakogu | People's Assembly[END_REF]59]. The portal is still operational today but largely considered failed in terms of adoption and outcomes [START_REF]Osalusveebi ja valitsuse eelnõude infosüsteemi kasutatavuse analüüs[END_REF].

People's Assembly

In 2013 Estonia experimented with a post-Parliamentary democracy tool, the People's Assembly (Rahvakogu). The initiative came from the President of Estonia and several civil society organizations as a response to a public trust crisis. It consisted of an online platform for crowdsourcing proposals to amend Estonia's electoral laws, political party law, and other issues related to the future of democracy in Estonia. After three weeks of online crowdsourcing, the ideas were debated during a one-day 'delib-eration day' involving a stratified random sample of members of the public to proportionally represent different regions, age groups and gender [START_REF] Hinsberg | Rahvakogu tulemus: võim kaotas, kuid inimesed lähenesid[END_REF]. The process resulted in 15 proposals that were presented to the parliament.

A year later, three proposals out of the 15 became laws and several more have by now been partly implemented or re-defined as commitments in the government coalition program. However, as the organizers admitted, the exercise failed to achieve its main goal -to increase trust in institutions of representative democracy [START_REF] Kalvet | E-hääletamine (e-voting)[END_REF].

Discussion

The four e-democracy instruments that have been implemented in Estonia -e-voting, TOM, Osale.ee and People's Assembly -have met mixed success. E-voting, despite some initial barriers, has by now been adopted by a considerable share of voters (close to 32% in the latest elections) and is generally regarded as an effective tool [START_REF] Glencross | E-Participation in the Legislative Process[END_REF]. At the same time, the e-participation projects have only been able to engage a narrow group of active users and are largely perceived as lacking impact. Although TOM had close to 7,000 registered users, only 9% of them actually posted an idea [START_REF] Postimees | Eesti e-riigi kuulsamaid lipulaevu kukkus läbi[END_REF]. Osale.ee, the only ongoing e-participation project, has been reported to have no more than 5-10 committed active users [START_REF]Osalusveebi ja valitsuse eelnõude infosüsteemi kasutatavuse analüüs[END_REF]. While the People's Assembly online platform succeeded in attracting a high number of proposals and comments (close to 4,800) was still heavily dominated by a homogenous user group -middle-aged well-educated ethnic Estonian men [START_REF] Kalvet | E-hääletamine (e-voting)[END_REF]. It is therefore fair to conclude that none of the e-participation projects has been particularly successful in enhancing e-democracy in the sense of fostering an active engagement of all parts of society in shaping public decisions.

Based on theoretical and empirical literature, e-democracy innovations are challenged by various barriers. These include a lack of administrative and political championing, poor integration into organizational procedures and broader political processes, lack of easily demonstrable impact, unfavorable cultural context, hostile attitudes to citizen engagement, and the difficulty of matching different expectations and capabilities in designing systems intended to engage diverse user groups. Therefore, the success of e-voting compared to e-participation projects could at least partly be associated with its inherently high integration into policy processes and administrative routines, high political interest and support to the instrument, its immediate and easily demonstrable impact on the constitution of elected bodies, clear mandate and a clear procedure for translating input to outcomes.

E-voting has demonstrated clear benefits related to convenience to users -if voting on paper would take more than half an hour, voters are more likely to prefer the electronic channel over the traditional one [START_REF] Glencross | E-Participation in the Legislative Process[END_REF][START_REF]Eesti valitsusasutuste e-kaasamispraktikate analüüs[END_REF]. Looking more closely at the development path of e-voting, one can identify that the service has been characterized by a relative stability as the application remained relatively unchanged within the first six years of operation -and this change only happened due to external pressure (an attack) [START_REF]Election Assessment Mission Report on the 6 March 2011 Parliamentary Elections in Estonia[END_REF]. Initially the take-up of e-voting was relatively low (only 2% of the voters chose the electronic channel in the 2005 municipal elections first offering e-voting) and focused on early adopters of the ID card, similar to e-banking. There was thus in fact a high barrier of learning to be overcome. Once this barrier was mastered, the immediate return was incurred: convenience. With e-voting people potentially save time, while with other e-democracy instruments the actual impact has often not been clearly visible.

Unlike e-voting, Estonian e-participation projects have never achieved true integration with existing political processes and their mandate has remained unclear. In the case of TOM and Osale.ee, government institutions do not seem to have found a way to accommodate unsolicited ideas from citizens into their daily routines. Although Osale.ee aimed to fix the shortcomings of TOM by better integration into the policymaking process, it largely seems to have failed in this respect. Part of the problem has been the ambiguity of the concept of public consultations, which is undefined in legislation [START_REF]Osalusveebi ja valitsuse eelnõude infosüsteemi kasutatavuse analüüs[END_REF][START_REF] Hänni | Interview on 2015-04-02[END_REF]. Hence, the extent to which the government should take citizens' input into account has remained unclear. Similarly, as the Parliament's policy-making routine included no procedures for working with crowdsourced proposals, the Parliament was generally reluctant to discuss the ideas put forward by the People's Assembly [START_REF] Kalvet | E-hääletamine (e-voting)[END_REF].

The latter also has to do with political interest, which seems to have been relatively low for all e-democracy instruments but e-voting. As the Prime Minister's project, TOM clearly had political backing at the highest level, albeit limited to the Prime Minister's office, while its successor Osale.ee never saw any political champions [START_REF] Tom Survey | TID+ website[END_REF][START_REF] Mahrer | Towards the enhancement of e-democracy: identifying the notion of the'middleman paradox[END_REF]. Political interest was more mixed for People's Assembly -although the process was formally initiated by the President, it did not meet similar interest on the part of the Parliament, the group of politicians whose support would have been key to the project's impact.

Due to loose integration and low political interest, all three e-participation projects have lacked the expected impact. The actual implementation of citizens' ideas proposed through TOM and Osale.ee has remained marginal [START_REF] Postimees | Eesti e-riigi kuulsamaid lipulaevu kukkus läbi[END_REF]. Public consultations on Osale.ee have not yielded better outcomes -both government officials and interest groups criticize the platform for low usefulness [START_REF]Osalusveebi ja valitsuse eelnõude infosüsteemi kasutatavuse analüüs[END_REF]. The People's Assembly, despite several positive effects, did not bring the government closer to citizens and failed to stimulate a fundamental reform of political institutions it originally intended to [START_REF] Kalvet | E-hääletamine (e-voting)[END_REF]. Therefore, considering the effort that is required from citizens, administrators and politicians to engage in a complex political dialogue and the absence of immediate benefits such as time savings, it might well be argued that it is inherently more difficult for eparticipation projects to repeat the success story of e-voting.

Conclusions

Estonia's success in e-voting does not mean the country has been successful in promoting and enabling e-democracy in general. Somewhat paradoxically, the country that has been a champion of e-government and a pioneer in e-voting has not quite been a success story in e-participation and has consequently failed to develop a full-fledged edemocracy as some had initially hoped. However, it is not only that the politicians lack out on supporting this transformation (which could be due to the fear of losing power; compare [65]) but also citizens themselves. Contrary to the hopes of many early Internet enthusiasts, citizens do not appear to be particularly interested in taking advantage of all the opportunities for direct access to decision-making that contemporary technologies can offer, especially if the benefits are not immediately evident. As it seems, the third transformation of democracy towards a fully developed e-democracy still has to happen and not even Estonia can help out with this one -for now.

2

 2

3

 3

Figure 1 :

 1 Figure1: Share of E-voters out of Voters in Per Cent[START_REF] Glencross | E-Participation in the Legislative Process[END_REF]

Rahvaalgatus.ee is only a very recent development in

It was not further analyzed as part of this study and is only mentioned here for completeness.

Acknowledgements. This work was supported by the European Commission (Open-GovIntelligence H2020 grant 693849), Estonian Research Council (grants IUT19-13, PUT773) and Tallinn University of Technology Project B42.


Aggeliki Androutsopoulou

Francesco Mureddu
email: fmureddu@uoc.edu

Euripidis Loukis
email: eloukis@aegean.gr

Yannis Charalabidis
email: yannisx@aegean.gr

Passive Expert-Sourcing for Policy Making in the European Union

Keywords: crowd-sourcing, citizen-sourcing, public policy, technocracy, policy network, reputation management, opinion mining

The public sector gradually starts exploiting the crowdsourcing ideas initially developed in the private sector. However, there is much less knowledge on efficient and effective methods and practices for public sector citizen-sourcing in comparison with private sector crowdsourcing, so extensive research is required in this area. This paper contributes to filling this research gap, by presenting an ICT-based method for 'passive expert-sourcing', with the latter term denoting the collection of policy relevant information, knowledge and ideas from experts, which aims at supporting policy making by the European Union (EU) by leveraging its large policy community. Its theoretical foundation is previous theoretical work on the relationships between democracy and technocracy, and also on policy networks. The main technological pillars of the proposed method are: EU policy experts' profiling and reputation management, relevant documents' opinion mining and relevance rating, and finally advanced visualized presentation of them. Finally, a first evaluation of the proposed method is presented, leading to encouraging results.

Introduction

Crowd-sourcing has been initially developed in the private sector, in order to exploit external information, knowledge and ideas possessed by 'crowds' of individuals for problem solving or for the development of innovations [START_REF] Howe | The Rise of Crowdsourcing[END_REF][START_REF] Howe | Crowdsourcing, why the power of the crowd is driving the future of business[END_REF], [START_REF] Brabham | Crowdsourcing as a Model for Problem Solving: An Introduction and Cases[END_REF][START_REF] Brabham | Crowdsourcing: A Model for Leveraging Online Communities[END_REF][START_REF] Brabham | Crowdsourcing[END_REF]. It is defined as 'a new web-based business model that harnesses the creative solutions of a distributed network of individuals, in order to exploit 'collective wisdom' and mine fresh ideas from large numbers of individuals' [START_REF] Brabham | Crowdsourcing as a Model for Problem Solving: An Introduction and Cases[END_REF]. There has been extensive research and practice in this area, which has led to the development of efficient and effective crowd-sourcing methods and practices, and also to gaining a deeper understanding of them, their value, advantages a disadvantages, and also the specific contexts and kinds of problems for which each of them is more appropriate for; comprehensive reviews of this research are provided by Rouse [START_REF] Rouse | A Preliminary Taxonomy of Crowdsourcing[END_REF], Hetmank [START_REF] Hetmank | Components and Functions of Crowdsourcing Systems -A Systematic Literature Review[END_REF], Tarrell et al. [START_REF] Tarrell | Crowdsourcing: A Snapshot of Published Research[END_REF], Pedersen et al. [START_REF] Pedersen | Conceptual foundations of crowdsourcing: A review of IS research[END_REF] and Rechenberger et al. [START_REF] Rechenberger | Utilizing the Crowd -A Literature Review on Factors influencing Crowdsourcing Initiative Success[END_REF].

The public sector, motivated by the multiple 'success stories' of this new knowledge sourcing and innovation paradigm, and also by the increasing complexity of social problems and needs, has started making some first steps in this direction, by introducing forms of 'citizen-sourcing' in their policy making processes (see section 2.1 for a brief review of relevant literature). However, there is much less knowledge on efficient and effective methods and practices for public sector citizen-sourcing in comparison with private sector crowd-sourcing. It is therefore necessary to conduct extensive research in this area, in order to develop highly efficient ICT-based methods for this purpose, which enable the effective collection of policy relevant information, knowledge and ideas from citizens, and then the advanced processing of them in order to calculate useful policy analytics, which can provide substantial support for public policy making, and addressing its specific needs. In general, it is important in the area of public sector citizen-sourcing to reach a level of knowledge, efficiency, effectiveness and maturity, comparable to those of the private sector crowd-sourcing.

This paper contributes to filling this research gap, by presenting an ICT-based method for 'passive expert-sourcing', with the latter term denoting the collection of policy relevant information, knowledge and ideas from experts, in order to support policy making by the European Union by leveraging its large policy community. The first evaluations of citizen-sourcing initiatives [START_REF] Ferro | Policy Making 2.0: From Theory to Practice[END_REF], [START_REF] Loukis | An Analysis of Multiple Social Media Consultations in the European Parliament from a Public Policy Perspective[END_REF], [START_REF] Androutsopoulou | Using Social Media Monitoring for Public Policy Making -An Evaluation[END_REF] have shown that they can provide valuable insights into the perceptions of the general public concerning important social problems and government activities for addressing them, and also existing and proposed public policies for addressing them. However, it is recommended that in order to collect information and knowledge of higher quality, it is necessary to target more knowledgeable communities having strong interest and good expertise on the particular topic/policy under discussion. On this question there has been extensive political sciences research, which has revealed the importance of both democracy (i.e. political consultation with all stakeholder groups) and technocracy (i.e. specialized knowledge of experts) for the development of effective public policies for addressing the complex problems of modern societies [START_REF] Fischer | Technocracy and the Politics of Expertise[END_REF], [START_REF] Harcourt | Limits to EU technocratic regulation?[END_REF], [START_REF] Brown | Science in Democracy: Expertise, Institutions, and Representation[END_REF]. At the same time this research highlights the need of a relationship and balance between them (see section 2.2 for more details on 'Democracy vs Technocracy in Public Policy Making'). Furthermore, another stream of political sciences research has examined the emergence of policy networks, as a result of the increasing complexity of social problems, in which participate various both governmental actors and non-governmental actors (such as associations of various businesses, professions, labor unions and other interest groups), and their increasing importance for the design and implementation of public policies; each of these actors has developed valuable expertise, usually focused on its particular perspectives and concerns, which can be quite useful for public policy making [START_REF] Skogstad | Policy Networks and Policy Communities: Conceptual Evolution and Governing Realities[END_REF], [START_REF] Rhodes | Policy Network Analysis[END_REF][START_REF] Rhodes | Understanding Governance: Ten Years On[END_REF] (see section 2.3 for more details on 'Public Policy Networks'). So, our research has as theoretical foundation this abovementioned previous work on the relationships between democracy and technocracy, and also on the public policy networks.

The research presented in this paper has been conducted as part of the European research project 'EU-Community' (project.eucommunity.eu/), partially funded by the 'ICT for Governance and Policy Modelling' research programme of the EU. This paper is structured in six sections. In the following section 2 the background of our research is outlined (on public sector citizen-sourcing, democracy vs technocracy in public policy making, and also public policy networks). Then in section 3 is presented the methodology we adopted for the design of the proposed method and then for a first evaluation of it. In section 4 the proposed ICT-based method of passive expert-sourcing is described, followed by the results of a first evaluation of it in section 5. Finally, in section 6 conclusions are summarized and directions for further research are proposed.

Background

Public Sector Citizen-Sourcing

As mentioned previously in the Introduction, for public sector citizen-sourcing there is a lack of research similar to the one that has been conducted for private sector crowdsourcing, having similar levels of breadth and depth, probably because the former is a more recent phenomenon than the latter. Limited research has been conducted concerning the application of crowd-sourcing ideas in the public sector, the development of efficient and effective methods and practices for this purpose, and the evaluation of them from various perspectives [START_REF] Lukensmeyer | Citizensourcing: Citizen participation in a networked nation[END_REF], [START_REF] Hilgers | Citizensourcing: Applying the concept of open innovation to the public sector[END_REF], [START_REF] Linders | From e-government to we-government: Defining a typology for citizen coproduction in the age of social media[END_REF], [START_REF] Nam | Suggesting frameworks of citizen-sourcing via Government 2.0[END_REF], [START_REF] Seltzer | Citizen Participation, Open Innovation, and Crowdsourcing: Challenges and Opportunities for Planning[END_REF], [START_REF] Mergel | Implementing Open Innovation in the Public Sector: The Case of Challenge.gov[END_REF], [START_REF] Ferro | Policy Making 2.0: From Theory to Practice[END_REF], [START_REF] Prpić | The Fundamentals of Policy Crowdsourcing[END_REF].

Most of the existing literature on ICT-based methods for citizen-sourcing by government agencies is focusing on 'active citizen-sourcing', which aim at the use of government agencies' web-sites or social media accounts, in order to pose a particular social problem or public policy direction, and solicit relevant information, knowledge, opinions and ideas from citizens. In this direction Mergel and Desouza [START_REF] Mergel | Implementing Open Innovation in the Public Sector: The Case of Challenge.gov[END_REF] describe and analyze the Challenge.gov initiative the U.S. Office of Management and Budget, which aimed at applying private sector crowdsourcing ideas in the public sector, in order to source from citizens ideas, knowledge, and solutions for specific challenges that government faces. This initiative was based on an ICT platform allowing U.S. federal agencies to launch contests, and at the same time citizens to find appropriate contests and participate in them providing solutions, or reviewing and evaluating solutions provided by others, voting on solutions, and even getting involved in the implementation of solutions and subsequent evaluation of them. Charalabidis and Loukis [START_REF] Charalabidis | Participative Public Policy Making Through Multiple Social Media Platforms Utilization[END_REF] and Ferro et al. [START_REF] Ferro | Policy Making 2.0: From Theory to Practice[END_REF] propose a method for the systematic, intensive and centralized exploitation of web 2.0 social media by government agencies on public policies (existing or under development). This method is based is on a central ICT platform, which a) publishes automatically various types of policy-related content (e.g., short text long text, images, video) in multiple social media accounts of a government agency, using the application programming interfaces (API) of these social media, soliciting citizens' feedback on them; and b) similarly collects automatically from them data on citizens' interactions with this content (e.g., views, comments, ratings, votes, etc.), and makes advanced processing of them. However, more recently, there has been some interest in 'passive citizen-sourcing', which aims to exploit political content that has been developed by citizens freely, without any direct stimulation or direction by government, in various external (= not belonging to government agencies) web-sites or social media, such as political fora, news web-sites, political blogs, Facebook, Twitter, etc. accounts [START_REF] Charalabidis | Passive Crowdsourcing in Government Using Social Media[END_REF] [START_REF] Wandhöfer | Engaging Politicians with Citizens on Social Networking Sites: The We-Gov Toolbox[END_REF].

Therefore, extensive further research is required in the area of public sector citizensourcing, in order to develop a considerable knowledge base on it, and through it a level of efficiency and effectiveness, comparable to those of the private sector crowd-sourcing area. Our research makes a contribution in this direction, by developing an ICTbased method of passive expert-sourcing.

Democracy vs Technocracy in Public Policy Making

The increasing complexity of the problems of modern societies, the globalization of the economy and the development of technology have increased the need for and importance of knowledge and expertise for the design and implementation of public policies [START_REF] Fischer | Technocracy and the Politics of Expertise[END_REF], [START_REF] Radaelli | The Role of Knowledge in the Policy Process[END_REF][START_REF] Radaelli | The Public Policy of the European Union: whither Politics of Expertise[END_REF], [START_REF] Harcourt | Limits to EU technocratic regulation?[END_REF], [START_REF] Brown | Science in Democracy: Expertise, Institutions, and Representation[END_REF], [START_REF] Kurki | Democracy Through Technocracy? Reflections on Technocratic Assumptions in EU Democracy Promotion Discourse[END_REF]. This has led to the establishment of various expert bodies (in government agencies competent for the formulation of public policies, and also the other public policy stakeholders, such as associations of professions, labor unions, businesses and other interest groups), which can have various forms, from committees to separate organizations (e.g. economic institutes). These expert bodies have become today highly important for and influential on the formulation of public policies, and this is termed as 'technocracy' [START_REF] Fischer | Technocracy and the Politics of Expertise[END_REF], [START_REF] Kurki | Democracy Through Technocracy? Reflections on Technocratic Assumptions in EU Democracy Promotion Discourse[END_REF]. So today it is widely recognized that the two fundamental bases of public policy making are democracy (i.e. political consultation with stakeholder groups) and technocracy (i.e. knowledge of experts).

However, political sciences research in this area has highlighted the need of a relationship and balance between them, as each of them needs inputs from the other. In particular, participants in the democratic processes (various stakeholder groups, and even active citizens) need relevant knowledge and expertise, and the lack of them can have quite negative impacts [START_REF]Technocratic Policy Making and Democratic Accountability[END_REF]. At the same time the experts dealing with a particular social problem/public policy also need inputs from the political process, concerning diverse values and concerns of different stakeholder groups, and also their diverse perspectives, approaches and ideologies. For the above reasons Brown [START_REF] Brown | Science in Democracy: Expertise, Institutions, and Representation[END_REF] argues that democracy and technocracy are not in conflict, and their combination is a necessity in todays' realities of highly complex social problems and needs, and globalization; they generate different kinds of knowledge, which are both necessary for public policy making. The ICT can be very useful for the required exchange of knowledge between democracy and technocracy. Our research contributes in this direction, developing an ICT-based method for transfer of knowledge from the latter to the former.

Public Policy Networks

The increased complexity of social problems and needs also led governments to realize that their 'classical' unilateral modes of governance are insufficient, and they need knowledge resources and cooperation of non-state actors (initially economic actors and later other social actors as well) in order to design and implement effective policies, and this resulted in the development of public policy networks [START_REF] Skogstad | Policy Networks and Policy Communities: Conceptual Evolution and Governing Realities[END_REF], [START_REF] Rhodes | Policy Network Analysis[END_REF][START_REF] Rhodes | Understanding Governance: Ten Years On[END_REF]. They are defined as sets of formal and informal institutional linkages between various both governmental actors and non-government actors (such as associations of businesses, professions, labor unions and other interest groups) structured around shared interests in public policy-making and implementation. In public policy networks the non-state actors provide to the state actors on one hand information, knowledge and expertise, and on the other hand support for the design and implementation of public policies, and legitimization of them; in return the former have the opportunity to influence the public policies (e.g. legislation, allocation of government financial resources) towards directions that are beneficial to them [START_REF] Atkinson | Strong States and Weak States: Sectoral Policy Networks in Advanced Capitalist Economies[END_REF], [START_REF] Howlett | Do Networks Matter? Linking Policy Network Structure to Policy Outcome: Evidence from Four Canadian Policy Sectors 1990-2000[END_REF], [START_REF] Van Waarden | Dimensions and types of policy networks[END_REF], [START_REF] Marsh | Understanding Policy Networks: towards a Dialectical Approach[END_REF].

A critical characteristic of a network is the density of interactions among its participants; according to [START_REF] Marsh | Understanding Policy Networks: towards a Dialectical Approach[END_REF] higher density of interactions has positive impact on the timewise stability of the network, the development of shared values and beliefs concerning desirable policy objectives and instruments, and finally the effectiveness and outcomes of the network. Therefore, ICT can be very useful for increasing this critical characteristic at a low cost, and supporting the exchange of diverse expertise and knowledge among participants. Our research makes a contribution in this direction, developing an ICT-based method that supports the exchange of expertise and knowledge between the actors participating in a policy network.

Design and Evaluation Methodology

In order to design this ICT-based method of expert-sourcing and its supporting ICT platform thirteen workshops have been organized, with the first five of them aiming to gain a better understanding of the structure of the EU policy community, and then the next eight aiming to collect the requirements of potential users of our method and ICT platform, as part of the preparation and the implementation of the abovementioned EU-Community project. The EurActiv.Com (a leading EU policy online media network (www. euractiv.com), which participates as partner in the EU-Community project, and the Fondation EurActiv Politech (a public service foundation (www.euractiv.com/fondation) having as main mission 'to bring together individuals and organizations seeking to shape European Union policies', participating also as partner in this project, were the organizers of these workshops. The participants were representatives of important EU policy stakeholders (such as industry federations), members of the advisory boards of EurActiv.Com and Fondation EurActiv Politech, thematic experts in various EU policies, policy analysts, registered users of EurActiv.Com portals, and also permanent staff of various hierarchical levels from the European Commission.

In order to conduct an initial evaluation of the proposed method and its supporting ICT platform an evaluation session was organized with the participation of similar potential users. During this session the proposed method was introduced to the audience, together with the supporting ICT platform, and some first applications with their results. Then the participants had the opportunity to interact with the ICT platform by executing a set of predefined usage scenarios, under the observation of the organizers who supported them, and recorded any comments or difficulties, and as well feedback on possible improvements. Finally, we collected evaluation data from the participants in this session using mainly quantitative techniques, as they enable condensing and summarizing a large quantity of evidence in a few numbers that enable the easier drawing of conclusions [START_REF] Maylor | Researching Business and Management[END_REF], [START_REF] Ragin | Constructing Social Research: The Unity and Diversity of Method 2nd Edition[END_REF]. We developed our evaluation framework based on the models developed in previous technology acceptance research [START_REF] Charalabidis | Opinion Mining and Sentiment Analysis in Policy Formulation Initiatives: The EU-Community Approach[END_REF], [START_REF] Schepers | A meta-analysis of the technology acceptance model: Investigating subjective norm and moderation effects[END_REF], [START_REF] Hsiao | The intellectual development of the technology acceptance model: A co-citation analysis[END_REF], which has concluded that the intention to use a new technology, is determined mainly by two factors: its perceived 'ease of use' (= the degree to which potential users believe that using it would require minimal effort) and its perceived 'usefulness' (= the degree to which potential users believe that using it will enhance their job performance). So our evaluation framework has been based on these three factors, which have been elaborated and analyzed into several detailed questions, taking into account the particular objectives and specificities of the proposed method.This elaboration has been made separately for each the two main components that the users of the ICT platform can access: the Euractory (which enables users' registration and reputation calculation, rating other users and also searching for experts on a topic) and the PolicyLine (which provides a visualization of document search results). Based on the above evaluation framework a questionnaire was designed to be filled by the session participants; the questions of the framework were converted to positive statements, and the respondents were asked to provide the degree of their agreement/disagreement with each of them in a five-levels scale (1 = totally disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = totally agree). The questionnaire is shown in Table 1. The data collected through the questionnaire were processed using Excel. Furthermore, after filling this questionnaire a qualitative discussion was conducted with the participants on the same questions, in order to get a deeper insight of their perceptions.

A Passive Expert-sourcing Method

Description

From the interviews we conducted (see first paragraph of section 3) it was concluded that the main need of EU policy stakeholders is to be better informed on the most knowledgeable and credible experts on a policy related topic they are interested in, and also the most relevant documents on such a topic; it will also be useful if these documents are associated with the various stages of the E.U. policy processes. Since experts usually do not have time to generate new content on a topic (social problem or public policy -existing or under development) they are interesting in, the use of 'active citizen-sourcing' would not be possible. Therefore, a 'passive citizen-sourcing' approach should be adopted, based on the retrieval, processing and exploitation of already existing experts' generated content. So, the proposed method is based on retrieving automatically information from various sources about experts on policy related topics, and then collecting the knowledge and opinions they share online through texts and postings in multiple web-sites and social media they are using. This can be achieved by crawling at regular time intervals the most relevant external sources of knowledgeable and credible experts on EU policies, and also of relevant documents of various types, and update automatically the corresponding databases of a supporting ICT platform. The practical application of this method will lead to the collection of a large amount of information concerning policy experts and content generated by them. So, it is important to apply automated state-of-the-art techniques for processing and classifying this content, in order to extract interesting insights and knowledge from it concerning social problems and public policies. This textual content of documents, articles and social media posts will be processed using opinion mining and sentiment classification methods, in order to identify subjective information, extract opinions, identify the polarity of their orientation (positive, negative or neutral) and assess the relevance of them with relation to a topic (see [START_REF] Charalabidis | Opinion Mining and Sentiment Analysis in Policy Formulation Initiatives: The EU-Community Approach[END_REF] for more details). Furthermore, for the experts it is necessary to apply digital reputation techniques for assessing their reputation/credibility and provide a ranking of them per topic of interest. By storing the above data in a common database, and enabling search of it by the users and visual presentation of the results, public policy stakeholders will be able to find useful expert knowledge on complex policy debates, e.g. the most reputable/credible experts or the most relevant documents on a specific topic. An overview of the proposed method is shown in Figure 1. The proposed method aims to foster collaboration and knowledge sharing among the different policy stakeholders on E.U. policy topics. For this purpose, in order to enable focus on a particular policy topic of interest, our method uses the concept of 'policy process', under which all relevant information on experts and documents is collected and clustered. In particular, as a policy process can be modelled any ongoing or completed EU legislative procedure, or political debate in general, on a topic, while each topic can be associated with one or more policy processes. Policy processes can be initiated by any policy stakeholder in order to enable the interconnection and presentation of all relevant information and aspects of policy consultations in a structured way.

ICT Platform

An ICT platform has been designed for supporting the implementation of the above method. It consists of two main components accessible by the users, called 'EurActory' and 'PolicyLine', with each of them including several sub-components, and also an additional component called 'CurActory', which is not directly accessible by the users, and includes the database storing the above information and the functionality for updating it by crawling pre-defined external sources at regular time intervals.

The EurActory component collects and maintains a directory of profiles of people with high levels of knowledge, expertise and credibility in one or more topics related with EU policies, usually having an active role in policy making processes at European level. According to their role they are categorized into the three types, which have been identified though our workshops: influencers, analysts and institutional decision makers. These people are included in the people database of the system in the CurActory component automatically by the crawlers sub-component, or manually by using the self-registration capabilities provided by the EurActory component. The crawlers component crawls at regular time intervals various external sources, which can be pre-defined websites (e.g. Euractiv.com, EUR-Lex, Europa Whoiswho directory, RSS Feeds, blogs and news sites) or social media accounts (e.g. LinkedIn, Twitter), updating the CurActory database, which makes the relevant discovered information available through the EurActory or the PolicyLine.

Furthermore, this EurActory component provides rankings of the expert profiles according to their expertise on a set of topics of interest, through the 'reputation score' calculated by the Reputation Management sub-component for each expert per topic, based on the following criteria: • Self-evaluation: direct input from the user on his/her own area of expertise.

• Peer-assessment: based on endorsements from other users made through EurActory • Business Card Reputation: based on the reputation ranking of the organization and the user's position in the organization's hierarchy • Document Assessment: results of authored documents' assessment by their readers • Network Value: level of influence as the sum of network connections • Proximity trust: level of connection in social media • Past Measurements: taking into account reputation in previous months (its stability means credibility). • Offline Reputation: manually added for persons with no online presence Also, the EurActory provides the following capabilities to registered users: -Search for an expert profile, by name, EU policy or topic, which returns experts found in descending reputation score order (i.e. the most reputable first).

-View an expert profile; the profile pages can also be shared on social media.

-Create own profile and curate personal information, connect social media accounts, claim expertise topics.

-Activation of an expert profile that has been already created by the system administrators, after the discovery of it by the crawlers, and also update of profile details.

The 'PolicyLine' component provides state-of-the-art visualization of policy relevant documents, which are structured according to policy processes, aiming to provide to the user a better understanding of the multi-actor processes related with the EU decision making procedures and policy debates. Therefore, in the core of PolicyLine functionality is the concept of 'policy process' (described previously in 4.1). We can have documents manually attached by users of the ICT Platform, to the specific 'policy process', as well as automatically discovered ones by the abovementioned crawlers subcomponent, which searches on regular basis multiple web-sites and social media accounts in order to find significant documents published online (media articles, reports, tweets, policy proposals, legislative documents), authored by the experts' categories mentioned above.

In particular, PolicyLine provides statistical information for each policy process selected by the user, such as the total number of relevant documents and the number of visits of users on the specific policy process page. For a more detailed view, PolicyLine offers a timeline visualization (see Figure 2), which structures the main documents (based on relevance as well as author's reputation) associated with this policy process in a temporal order, and clusters them under a set of user defined stages of the particular policy process. It also provides information with respect to their authorship (colors are used for this purpose to reflect different authors' categories and sub-categories), and also shapes (such as rectangles and circles) to reflect different types of documents (e.g. rectangles reflect the proposal documents, while general documents are represented by circles). Documents sub-categories are defined concerning the type of organization from which each document is originated (e.g. European Institution, National/Local Governance, Academic Institution, Civil Society Organization, Media, etc.). The sizes of the shapes representing these documents reflect their relevance and author's reputation (so more relevant documents written by more reputable authors are shown bigger). Moreover, a user can select a particular document in order to view more details about it, including the results from the sentiment classification provided by the opinion mining sub-component (a linguistic analysis of the textual content of a document leads to an estimation of the polarity of the underlying text -see [START_REF] Charalabidis | Opinion Mining and Sentiment Analysis in Policy Formulation Initiatives: The EU-Community Approach[END_REF] for more details) and the relevant input provided by other platform users. In particular, PolicyLine for each document provides an interface where users can rate its accuracy, value, relevance and timeliness, and also enter comments on the document, so that an informal discussion on it can be stimulated.

Evaluation Results

In Table 2 are shown the results of the processing of the data collected through the evaluation questionnaire (average ratings for all questions) for the two main components that can be accessed by the users, EurActory and PolicyLine. We can see that the respondents find the ease of use of the EurActory component high (the average rating of relevant questions is 3.9), and for the PolicyLine component moderate to high, but closer to the latter (the average rating of the relevant questions is 3.67). Slightly lower are their perceptions with respect to usefulness, which it is perceived as moderate to high for the EurActory component (average rating of relevant questions 3.5), and moderate to high, but closer to the former, for the PolicyLine component (average rating of relevant questions 3.3). Finally, high is the intention to use again the PolicyLine component again (average rating of relevant questions 3.9), and slightly lower for the EurActory component (average rating of relevant questions 3.75).

In the qualitative discussion with the participants of the evaluation session the latter agreed that this ICT platform, and the whole method behind it, constitute an easy to use tool for finding quickly high quality information and opinions on important policy related topics and policy formulation processes, authored by knowledgeable experts, and also debate over them with other users. Furthermore, it enables and promotes communication and exchange of knowledge among EU policy stakeholders. It also allows awareness of and also debate and criticism on policy initiatives carried out by the European Institutions. The usefulness of the EurActory component was assessed a bit higher than the PolicyLine; this probably reflects that the former is easier to use and exploit its capabilities than the latter. The participants were in general satisfied with the proposed method and its supporting infrastructure, and expressed interest in using again the functionalities of both components. However, future improvements were suggested, concerning the graphical interface and especially the timeline visualization.

Conclusions

In the previous sections of this paper an ICT-based method for 'passive expert-sourcing' was presented, which allows the collection of high quality policy relevant information, knowledge and ideas from knowledgeable experts, aiming at supporting policy making in the European Union (EU) by leveraging its large policy community. Its theoretical foundation is previous theoretical work on the relationships and the required balance between democracy and technocracy, and also on policy networks. The proposed method is based on EU policy experts' profiling and reputation management, relevant documents' opinion mining and relevance rating, and finally advanced visualized presentation of them. At the political level, its objective is to enable a better interconnection of the two important bases of modern public policy making, the democratic processes and the technocratic expertise (which is of critical importance, as discussed in more detail in section 2.2), by supporting the transfer of knowledge from the latter to the former. In particular, it aims to support the efficient and effective retrieval by various actors of the democratic processes (e.g. representatives of stakeholder groups, journalists, government employees, active citizens, etc.) of diverse expert information, knowledge and ideas on a specific topic/policy, which is included in postings and texts authored by knowledgeable experts and published in various web-sites and social media. Furthermore, the proposed method of passive expert-sourcing aims to increase the density of interactions among the actors participating in public policy networks, which is highly important for their stability, the development of shared values and beliefs, and finally the effectiveness and the outcomes of such networks (as discussed in more detail in section 2.3), by supporting the exchange of expertise and knowledge between network participants.

Also, a first evaluation of the proposed method has been presented, which gave encouraging results, with respect to its ease of use and usefulness. However, further evaluation of this method is required, based on realistic pilot applications of it, in order to assess better its value and potential with respect to its abovementioned ambitious objectives: to what extent it enables and supports the transfer of information, knowledge and proposals from experts to the participants in the democratic processes of modern policy making, and under what conditions? to what extent it can enable and support the exchange of information, knowledge and proposals among the participants in public policy networks, and under what conditions ? to what extent can this method can assist the EU institutions to collect high quality information, knowledge, opinions and proposals from their policy networks? Research in this directions is already in progress as part of the abovementioned EU-Community project. Also, further research is required concerning the use of ICT for the transfer of knowledge in the opposite direction, from the democratic processes towards experts/technocracy (e.g. concerning diverse needs, values and concerns of different stakeholder groups on the particular social problem/public policy the experts analyze, and also existing diverse perspectives, approaches and ideologies). This is quite important for the construction of better and more multi-dimensional comprehensive and inclusive expert analyses and plans, which do not miss or neglect important aspects of the social problems or public policies they are dealing with and might be quite important for large social groups, and also do not underestimate existing diverse perspectives, approaches and ideologies.

Fig. 1 .

 1 Fig. 1.An overview of the proposed method of passive expert-sourcing

Fig. 2 .

 2 Fig. 2. PolicyLine timeline visualization

3 . 43

 343 Intention to useI expect to use PolicyLine on a regular basis in the future I will advise colleagues to use PolicyLine 4

Table 1 .

 1 Evaluation Results

		Ease of Use Perspective
		EurActory can be easily used without assistance	3.46
		Creating a profile is easy	4.08
		It is easy to access topic listings	4.15
		It is easy to rate peers	3.75
		Using EurActory has been a positive experience	4.08
		Usefulness
	EurActory	EurActory puts together information not found or collected under one roof elsewhere EurActory allows me to be more productive EurActory improves the quality of my work	3.15 3.38 3.46 3.85
		EurActory assists me in identifying relevant experts	3.54
		EurActory provides me with all the needed information on relevant	3.54
		experts
		EurActory enables me to reinforce my expert positioning
		Intention to use
		I expect to use EurActory on a regular basis in the future	3.85
		I will advise colleagues to use EurActory	3.62
		Ease of Use Perspective
		PolicyLine can be easily used without assistance	3.64
		I can easily create a 'policy process'	3.69
	PolicyLine	I can easily add a document in the 'policy process' I can easily rate/comment a document I can easily get an overview of the process Using PolicyLine has been a positive experience	3.79 3.5 3.73 3.71
		Usefulness
		PolicyLine puts together information not found or collected under one	3.29
		roof elsewhere	3.29