Shefali Virkar
email: shefali.virkar@donau-uni.ac.at

Gabriela Viale Pereira
email: gabriela.viale-pereira@donau-uni.ac.at

Michela Vignoli
email: michela.vignoli@ait.ac.at

Investigating the Social, Political, Economic and Cultural Implications of Data Trading

Keywords: data market, data trading, social and cultural aspects

Data market initiatives have, by assigning monetary value to data, and connecting the various actors responsible for its efficient production and consumption, far reaching consequences for national economies. The Data Market Austria (DMA) project represents a unique opportunity for Austria to leverage the enormous potential socio-economic benefits accruing from increased trade of data. At the same time, however, a number of key challenges to the successful uptake of the project needs to be considered, and new problems emerging from this new form of digital commercial infrastructure need to be anticipated and addressed. This study aims to examine how the benefits accruing to increased participation in a data-driven ecosystem can be applied to tackle the long-standing socio-cultural challenges and the possible societal and cultural impediments to the successful unfolding out of a data market. Theoretical discussions framed from arguments obtained through a systematic review of academic and scholarly literature are juxtaposed with empirical data obtained from data science experts and DMA project personnel to test whether they stand up to real-world practicalities and to narrow the focus onto the Austria-specific context. Our findings reveal that data is a dual-purpose commodity that has both commercial value and social application. To amplify the benefits accruing from increased data trading, it is vital that a country establishes a sound open data strategy and a balanced regulatory framework for data trading.

Introduction

A flood of structured information or data [START_REF]Data' -Glossary of Statistical Terms[END_REF] is created every day through the interactions of people using computers, mobile phones, GPS, and other Internet-enabled devices. This all-pervasive data has become the life-blood of the modern global economy [START_REF] Brown | Are You Ready for the Era of 'Big Data'?[END_REF], and it is widely accepted that, if harnessed appropriately, such open dataobtained from both public and private sourcescan add a new dimension to existing business analytics and give rise to novel, data-focused innovations [START_REF] Manyika | Open Data : Unlocking Innovation and Performance with Liquid Information[END_REF]. Data is often also placed at the heart of strategic decision-making; be it in business firms or within government organisations [START_REF] Marr | Big Data in Practice: How 45 Successful Companies Used Big Data Analytics to Deliver Extraordinary Results[END_REF]. The buying and selling of data generally occurs in what is commonly known as a data marketplace [START_REF] Deichmann | Creating a Successful Internet of Things Data Marketplace[END_REF], a specific ecosystem created for the purpose by either vendors or consumers of large quantities of information or by third party organisations. A succinct definition of the concept is provided by [START_REF] Schomm | [END_REF], who define a data marketplace as "…a platform on which anybody (or at least a great number of potentially registered clients) can upload and maintain data sets. Access to and use of the data is regulated through varying licensing models." Further, a 'data marketplace' or a 'data market' may be considered as the virtual space where high quality digital data is offered by data producers as "products" or "services" to potential consumers for payment or for free [START_REF] Li | Pricing Aggregate Queries in a Data Marketplace[END_REF]. Strategically using insights from big data to address many of the challenges that face societies has become an integral part of European Union policy in recent years [START_REF]European Commission: Building a European Data Economy[END_REF]. The approach developed by the European Commission emphasizes the importance of addressing relevant socio-economic implications of big data processing and analytics, such as respect of ethical principles and related legislation during the implementation of flagship policies (cf. personal data protection and privacy, ensuring informed consent, dual use and potential misuse of the research results, fair benefit sharing when developing countries are involved, and environment protection) [START_REF]Horizon 2020, Work Programme[END_REF]. It is vital, therefore, that the benefits and challenges associated with the trade of big data and the degree to which the concept has meaning for the general public are better understood. Key questions to pose in this context are: What impact will the trade of big data have on society, the polity, the economy, and on culture? And who are the winners and losers of the modern knowledge economy?

This study aims to critically examine, within the context of the Data Market Austria (DMA) project, how the benefits accruing to increased participation in a data-driven ecosystem can be applied to tackle the long-standing socio-cultural challenges and the possible societal and cultural impediments to the successful unfolding out of a data market. The research paper is structured sequentially in five parts. The first chapter introduces the relevant fundamentals of data trading. Section 2 describes the findings of a systematic literature review. Section 3 then presents and discusses empirical data obtained from two qualitative research tools, a research workshop and a semi-structured research questionnaire. The results are subsequently synthesised and analysed in Section 4. The final section brings together the key observations, lessons learned, and recommendations arising from these and earlier discussions.

Research Design

In order to identify and better understand the benefits resulting from the trade of digital data and the potential challenges that could hinder the successful development of a data market, with a focus on the Austrian context, this study adopted a three-part research design to collect research data: a systematic literature review, followed by a pre-conference workshop, and finally a questionnaire.

Context of the Project. The Data Market Austria (DMA) project seeks to establish a data services ecosystem in Austria through the creation of a significantly improved technology base for secure data markets and cloud interoperability, and through the development of a data innovation environment [START_REF] Ivanschitz | A Data Market with Decentralized Repositories[END_REF]. The DMA does not host data, but instead is a facilitator platform that provides a catalogue for registered data sources. Developers currently work with open data sets from Kaggle and the European Data Portal. Use of industry data is currently minimal for -besides those obtained from the pilotsthese are not currently publicly available. Main DMA stakeholder groups can be classified as data providers; service providers; infrastructure providers; brokers; data market customers; research, education and development; and end-users [START_REF] Thurner | Data Market Austria Deliverable -D2.1 Community Building Coordination, Dissemination & Exploitation Plan[END_REF]. The target groups of DMA comprise several different categories of data driven organisations and individuals, each with differing requirements for data. These have been broadly identified through initial research as Government, Industry, Research, Academia, the Public, the Media, and the Community [START_REF] Thurner | Data Market Austria Deliverable -D2.1 Community Building Coordination, Dissemination & Exploitation Plan[END_REF]. The DMA research project is funded by the Austrian Ministry for Transport, Innovation and Technology via the Austria Research Promotion Agency (FFG). A Beta working version is expected to be ready at the end of the project, in September 2019.

Systematic Literature Review. A systematic literature review [START_REF] Kitchenham | Procedures for Performing Systematic Reviews[END_REF] was conducted to identify and discuss the benefits accruing and challenges posed to successful data trading. The Scopus database was selected as the primary source of research material, as it was noticed that this database encompasses many relevant papers in the fields Business, Economics, Political Science, and Information Science. Based on the research questions and after discussing about the keywords, it was decided to use <data market> OR <data-driven> OR <open data> OR <digital data> AND <social impact> OR <societal impact> OR <social implications>OR <societal implications> OR <soci-ety> to conduct the search in the title, abstract, keywords and full-text of publications. Specific keywords for each topic were also combined with the aforementioned ones to find related and relevant sources. It was decided to mainly focus on publications of the past ten years.

Workshop. An interim report on the socio-economic, political and cultural implications of data trading showed the need for a deeper, evidenced-based study of how the impacts of a data market are perceived from a variety of perspectives worldwide. In order to determine the prevailing global attitude towards the data market concept, and to deepen our evidence-based investigation of the socio-economic, political and cultural implications of data trading, a workshop was organised under the auspices of a major international conference [START_REF] Viale Pereira | Exploring the Political, Social and Cultural Challenges and Possibilities Associated with Trading Data: The Case of Data Market Austria (DMA)[END_REF]. Guided by a semi-structured protocol, three moderators from the project under study conducted and reported the discussion. In recognising that a workshop is a form of focus group, the design of proceedings was based on [START_REF] Morgan | Focus Groups[END_REF], as a research technique that uses group interaction for collecting data on a specific topic. The design can be considered as a mixed structured and unstructured approach as two overarching questions were discussed over the course of one hour, which allowed the participants to get deeper into each discussion.

Questionnaire. To obtain deeper insight into the central research problem, and to validate the findings of the pre-conference workshop, primary data in the form of attitudes, opinions, and perspectives was gathered from individual members of the Austrian Data Market Austria (DMA) project partners. The questionnaire method was selected for data collection as it facilitates the thorough exploration of existing patterns and trends required to effectively describe the social, economic, and political impacts of the DMA project. A self-administered [START_REF] Wilson | Asking Questions[END_REF] and semi-structured questionnaire [START_REF] Adejimi | Employing qualitatively enriched semi structured questionnaire in evaluating ICT impact on Nigerian 'construction chain integration[END_REF] consisting of a mix of closed questions and open questions ordered in a structured sequence and with a pre-determined focus was used. In order to maximize coverage within the shortest period of time, we chose to administer the questionnaire online [START_REF] Kumar | Research Methodology. A Step-by-Step Guide for Beginners[END_REF]. The questionnaire was developed by this research team following conventional best practices [START_REF] Kumar | Research Methodology. A Step-by-Step Guide for Beginners[END_REF], and the finished survey was hosted on Google Forms. An invitation containing a link to the questionnaire was e-mailed to potential participants, and the number of responses tracked through the in-built counter.

Systematic Literature Review

This chapter summarises the key results of the systematic literature review, outlining the potential socio-economic, political and cultural of implications of data markets and data trading. From available scholarly and practitioner literature, six key domains have been derived: economic growth, employment and job creation, ageing populations, migration, climate change, and small-and medium-sized enterprises (SMEs). From further examination of the literature, four critical societal and cultural impediments to the successful unfolding out of the DMA were identified: privacy and trust, public decision making structures, social exclusion, and gender inequality.

Benefits of Data Trading to Society

Economic Growth. A wide range of positive economic impacts result from the free flow of digital data across borders. A report by the Strategic Policy Forum on Digital Entrepreneurship [START_REF]Strategic Policy Forum on Digital Entrepreneurship: Big Data and B2B Digital Platforms: The Next Frontier for Europe's Industry and Enterprises[END_REF] predicts that big data and digital platforms will bring enormous benefits to the European economy, as firms across many industries experiment with data-driven business models and use these as tools to drive product innovation. Further literature suggests that a unified digital market platform that facilitates the trade of data may revolutionize the way in which different forms of digital data are accessed by a wide range of stakeholders from across different sectors of the national and global economies; thus fostering increased business activity [START_REF] Bhimani | Exploring big data's strategic consequences[END_REF][START_REF] Eygm | Big Data. Changing the Way Businesses Compete and Operate. Insights on Governance, Risk and Compliance[END_REF], supporting streamlined business operations [START_REF] Arellano | Making decisions with data: How to put different data sources together[END_REF][START_REF]Business Roundtable: Putting Data to Work: Maximizing the Value of Information in an Interconnected World. Business Roundtable[END_REF], spurring data-driven innovation [START_REF] Zillner | Big Data-Driven Innovation in Industrial Sectors[END_REF][START_REF] Parmar | The New Patterns of Innovation: How to Use Data to Drive Growth[END_REF], enhancing the competitiveness of individual entrepreneurs and SMEs [START_REF] Manyika | Digital Globalization: The New Era of Global Flows[END_REF][START_REF] Oxera | Benefits of Online Platforms[END_REF] and by connecting market participants otherwise separated by physical distance [START_REF] Duch-Brown | The Competitive Landscape of Online Platforms[END_REF][START_REF] Libert | 3 Ways to Get Your Own Digital Platform[END_REF].

Employment and Job Creation. The participation of businesses in a data-rich environment wherein knowledge and information are easily obtained can affect a labour market in several ways. Data analytics and related capabilities can improve business competitiveness and become an engine of labour productivity and job creation [START_REF]SIIA): Data-Driven Innovation -A Guide for Policymakers: Understanding and Enabling the Economic and Social Value of Data[END_REF]; either by facilitating strategic business decision-making that improves firm performance in increasingly competitive national and global markets [START_REF] Wamba | How 'big data' can make big impact: findings from a systematic review and a longitudinal case study[END_REF], or directly improving labour productivity by making available a variety of relevant information and knowledge accessible to employees at all levels via a dedicated portal [START_REF] Koegler | The Empowered Employee: How 6 Companies Are Arming Their Teams with Data[END_REF]. Increased focus on labour productivity through data-driven value creationa direct result of the commodification of data via regular trading across the data marketcould also result in a direct reduction of the number of blue-collar or manual jobs needed within a par-ticular industry [START_REF]OECD: Data-Driven Innovation: Big Data for Growth and Well-Being[END_REF]. As more economic actors choose to operate in the data marketplace, the demand for so-called high-income 'knowledge workers' capable of taking advantage of the available resources would outstrip that for manual labour. Similarly, highly-skilled workers capable of manipulating data using 'smart' machines and artificial intelligence systems would, in the future, be in demand, while those lacking the necessary skills would be left behind [START_REF] Brynjolfsson | Race against the Machine: How the Digital Revolution Is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy[END_REF]. Ageing Populations. Nowadays, people over the age of 60 constitute the fastest growing population group with their numbers set to increase from 164m to 222m by 2030 [START_REF] Dobbs | Urban World: The Global Consumers to Watch[END_REF]. In rich countries like Austria, older consumers are also amongst the most affluent in society, are more educated than previous generations, and are expected to become one of the few engines of growth driving an otherwise sluggish global economy [START_REF]The Grey Market[END_REF]. This reflects a significant business opportunity for those companies willing to address the needs of people in later life. Further, ageing people require sustained access to a wide variety of highly specialised goods and services [START_REF] Kuenen | Global Aging: How Companies Can Adapt to the New Reality[END_REF]. The growth in the number and proportion of older people is likely, therefore, to strongly impact and change the focus of business activity [START_REF]A Silver Opportunity? Rising Longevity and Its Implications for Business[END_REF]. Finally, as younger employees become required to care for their ageing relatives, companies need to both understand how to compensate for employee absenteeism and target this emerging segment of specialised consumers [START_REF] Mochis | Targeting the mature market: opportunities and challenges[END_REF].

Better product insight as a direct consequence of increases in the collection of data on what the elderly and their caregivers need, want, and are using by way of goods and services would also result in greater returns to scope [START_REF]OECD: Data-Driven Innovation: Big Data for Growth and Well-Being[END_REF], as the data market platform would begin to facilitate data linkages between various suppliers of information and knowledge. The elderly and those who care for them would therefore benefit from network effects [START_REF]OECD: Data-Driven Innovation: Big Data for Growth and Well-Being[END_REF] generated by the frequent trading of information and knowledge: as the providers of goods and services to the elderly connect with the data market network, they by default connect their client base of users to the data market community. Migration. Migration is a complex global phenomenon with important policy ramifications for both Europe and for Austria. General consensus in policy and academic circles is that robust and accurate data is key to effective migration governance, and can positively impact both governments and individual migrants [START_REF]More than Numbers. How Migration Data Can Deliver Real-Life Benefits for Migrants and Governments[END_REF]. On the one hand, reliable migration data can support decision-making by informing policy makers on the impact that migration has on society and the economy [START_REF] Laczko | Improving Data on International Migration and Development: Towards a Global Action Plan?[END_REF]. For the individual migrant, on the other hand, timely access to relevant data, coupled with appropriate policy interventions based on such information, can mitigate the vulnerabilities and risks associated with emigration and integration [START_REF]More than Numbers. How Migration Data Can Deliver Real-Life Benefits for Migrants and Governments[END_REF]. Despite the great potential of reliable migration data to deliver insights into complex societal and economic problems, its current availability is still recognized as being very limited [START_REF]Safety and Dignity: Addressing Large Movements of Refugees and Migrants. Report of the Secretary-General[END_REF]. Therefore, a centralised data platform can help interested stakeholders to manage existing data, and to make better use of it [START_REF]More than Numbers. How Migration Data Can Deliver Real-Life Benefits for Migrants and Governments[END_REF]. Climate Change. Global warming has a range of potential ecological, societal and health impacts. Big climate data is, therefore, an essential resource for climate change estimates, which are a valuable input for policy makers, aid agencies, and industry representatives involved with building national resilience to climate change and developing adaptation strategies [START_REF] Ford | Big data has big potential for applications to climate change adaptation[END_REF]. [START_REF] Augur | Big data plays surprising role in fight against climate change[END_REF] identifies three ways in which climate-related big data can be used to fight climate change. The business case for climate-related data is made by [START_REF] Carney | Better Market Information Can Help Combat Climate Change[END_REF], who argues that financial markets have the potential to help in the fight against climate change in situations where investors have access to accurate information on how companies are anticipating and responding (or not) to climate risks and opportunities. Climate change data, especially when mixed/overlaid with other data sets, is an important source for data-driven decision making [START_REF] Faghmous | A big data guide to understanding climate change: the case for theory-guided data science[END_REF][START_REF] Burke | Global non-linear effect of temperature on economic production[END_REF] Small-and Medium-scale Enterprises (SMEs). The importance of big data for economic growth is already widely recognised within the European Union [START_REF] Sen | An overview of big data for growth in SMEs[END_REF]. According to [START_REF] Ogbuokiri | Implementing Bigdata Analytics for Small and Medium Enterprise (SME) Regional Growth[END_REF], however, SMEs often fail to successfully implement big data technology and analysis frameworks. Small-and medium sized enterprises are often slow adopters of big data analytics technologies, and it is essential to ease the adoption of such technologies by SMEs in order to support economic growth through big data. A data market infrastructure addressing the challenges to SME uptake of big data analytics has the potential to become an important instrument for fostering a sustainable, inclusive and growing data-driven economy [START_REF] Coleman | How can SMEs benefit from big data? Challenges and a path forward[END_REF]. [START_REF] Rossi | How to Make Big Data Work for SMEs[END_REF] argues that big data for SMEs is all about identifying, joining up and consolidating the various available sources of data, and then facilitating the analysis of that information to extract business value. Next to this, it is important to create the right framework and conditions for realising inclusive growth of the digital economy [START_REF] Morovan | Data The Fuel of the Digital Economy and SME Growth[END_REF]. Industry representatives have echoed these findings [START_REF] Burns | Why Haven't SMEs Cashed in on Big Data Benefits Yet?[END_REF].

Potential Impediments to the Successful Uptake of a Data Market

Privacy and Trust. Privacy and security can be considered among the main challenges when dealing with the digital ecosystem. In particular, the sharing of data in a data marketplace, particularly personal data that is tied to individuals, raises legitimate concerns that must be addressed [START_REF] Mundial | Big Data, Big Impact: New Possibilities for International Development[END_REF]. [START_REF] Immonen | Requirements of an Open Data Based Business Ecosystem[END_REF] affirm that, in the case of open data, only when private elements are added to the data sets does open data become private data. The management of data privacy thus is an important element, considering that private data are very often the case available in ecosystems such as the data market. There is also an identified risk of violating legislation (for instance the data protection law) by sharing and releasing data that could not be otherwise made available [START_REF] Zuiderwijk | The Negative Effects of Open Government Data-Investigating the Dark Side of Open Data[END_REF]. This scenario includes, for example, datasets which contain privacy identifying variables, sensitive variables; or have been created by multiple organizations that have different levels of security, policies, and comply with different laws [START_REF] Zuiderwijk | The Negative Effects of Open Government Data-Investigating the Dark Side of Open Data[END_REF]. According to [START_REF] Schomm | Marketplaces for data: an initial survey[END_REF], trustworthiness of vendors is an important dimension of the data market, which can be assessed based on the origin of the data and on how it is processed.

Public Decision-making Structures and Social Exclusion.

To create transparent public decision-making structures and enable active citizenship in a data market it is crucial to release information, data, and services that are usable for, and can be applied by all relevant stakeholders, including citizens (e.g. data related to social justice, public safety, and health). If the data market should be inclusive and not exclude any member of the society, going "beyond [the] rationales of increased efficiency, reduced costs, increased productivity, and economic growth" is essential [START_REF] Howard | More than Economics: The Social Impact of Open Data[END_REF]. In recognising the social aspects of the data economy, the European Commission aims to create a whole "digital society" founded on the training of citizens in digital skills, and the promotion of concrete actions to support projects related to jobs, employability, training, and social issues [START_REF]European Commission: New Skills Agenda for Europe[END_REF]. The NGO and charitable sectors are, thus, also important players in a data market wherein Big data offers many opportunities for social enterprises [START_REF] Small | Believe the Hype: Big Data Can Have a Big Social Impact[END_REF] (e.g. see the Nominet Trust 100 Social Tech Guide). Gender Inequality. Usually, the choices of women are restricted by not only their educational pathways but also social factors, such as starting a family and the workplace environment [START_REF]Women in Science[END_REF], cultural barriers, gender stereotype, or misinformation [START_REF] Wang | Gender gap in science, technology, engineering, and mathematics (STEM): current knowledge, implications for practice, policy, and future directions[END_REF]. Quantitative numbers show the underrepresentation of women in science, technology, engineering and mathematics (STEM) worldwide, and there is a need to identify the qualitative factors that shape women's decisions to pursue STEM careers. Socio-cultural factors like societal beliefs and expectations of male/female differences in ability, and cultural pressures to pursue traditionally masculine or feminine interests, are shown to have a significant impact on career decisions [START_REF] Wang | Gender gap in science, technology, engineering, and mathematics (STEM): current knowledge, implications for practice, policy, and future directions[END_REF]. The fact that sociocultural factors have such a strong influence over individual career decisions also means that we may intervene to alter these outcomes [START_REF] Wang | Gender gap in science, technology, engineering, and mathematics (STEM): current knowledge, implications for practice, policy, and future directions[END_REF].

Discussion of Findings

This section presents a synthesis of the salient points of the research findings obtained with empirical research tools. The workshop took place in 2018, as part of a conference on Digital Government. Nine participants from eight countries and three moderators attended. The participants comprised of researchers, students, and practitioners from the e-government field, none of them specialists of data trading or related domains. The data was registered by the moderators and crosschecked for analysis, and the findings are presented in a narrative form [START_REF] Bertrand | Techniques for analyzing focus group data[END_REF]. The questionnaire was administered online over the period of March -July 2018. Potential respondents were initially invited to participate by e-mail, and contacted via the DMA project mailing list. To increase response rates, organization team leaders were contacted individually and were encouraged to circulate the questionnaire amongst members of their team. It is estimated that 80 people were invited to participate in the activity, out of which 21 responded.

Benefits of Data Trading to Society

Economic Growth. Empirical data indicates that research subjects not only endorsed this view, but also considered economic growth to be the primary benefit to Austria as a direct consequence of the DMA initiative. However, respondents did not believe that any industry or sector stood to gain advantage from increased data trading. Questionnaire respondents argue instead that the benefits arising from business participation in the DMA would be distributed evenly across different sectors of the Austrian economy.

Employment and Job Creation. Although the responses of practitioner and experts underline the fact that increased data trading can boost employment and create new jobs, respondents indicate that only modest benefits will result from participation in the Austrian data market. A large number of research subjects were confident that the DMA would result in the creation of new types of jobs in Austria, and a rise in the number of people employed in data science and data protection-related fields, as businesses participating in the project become encouraged to adopt data-driven business models. Ageing Populations. A large majority of project partners surveyed do not feel that the Austrian DMA project would substantially impact the lives of the elderly in Austria at all. This is because the current format of the DMA is not structured to target directly to the needs of this target population. Empirical research data further suggests that the DMA is not perceived as being as mechanism that affects the nature and supply of goods and services catering to an elderly population. This does not, however, foreclose for some respondents the potential for the platform to facilitate the provision of goods and services for senior citizens, provided that the current structure of the system is sufficiently altered.

Migration. Empirical research data indicates that research subjects do not believe that significant benefits will arise from the DMA project in the domain of migration. Responses suggest that migrant populations stand to receive almost no direct benefit from the platform in its current format. Most questionnaire respondents indicated that serving migrant communities did not fall within the current purview of the project. Some research subjects did signal, however, that this was a desirable objective in the long run, and that targeted relevant information in the form of aggregated data and visualisations on pertinent topics could be made available. Climate Change. Evidence suggests that research subjects appear optimistic that the DMA project will result in a significant positive impact within the climate change domain. This sentiment was underlined by workshop and questionnaire participant responses, wherein individuals identified and deliberated upon how the current DMA structure and format could be improved to make climate change data more easily available. Research subjects also discussed in-depth the data analytics tools and methods that could be incorporated into the DMA portal to aid in better climate change governance. The role of the DMA as a data market in this context was also underlined.

Small and Medium-sized Enterprises (SMEs). Empirical data suggests that participation in a unified data market project like the DMA will greatly improve the competitiveness of SMEs and individual entrepreneurs. Research subjects see the platform as making available data previously held by larger firms, lowering cost barriers to data access, and connecting small players to other like-minded market participants. However, it was felt that the DMA needs to be restructured to be more "small companyfriendly", and to be geared towards offering small businesses more comprehensive access to data products and services. The role of the Data Broker was highlighted as being central to the effective mapping of SME data requirements with appropriate data sources.

Challenges to Effective Data Trading

Privacy and Trust. Empirical research suggests that issues surrounding privacy and trust are considered by experts and practitioners to pose the most significant threat to the success of the DMA. Better internal regulation of market participants, and the provision of robust and stable basic system functionalities, emerged from responses as the key means by which the perceived trustworthiness of DMA data providers can be improved and incentives for increased participation can be created. System process transparency as a means of improving project trustworthiness was also highlighted. Public Decision-making Structures. While research subjects did not believe that a lack of citizen engagement in data trading initiatives posed a significant threat to the success of the DMA, they were categorical in their opinion that a lack of public sector involvement was a very serious problem. The role of government as a catalyst, preserver, and vital link in the data trading infrastructure was repeatedly emphasised. The initiative, it was felt, would support the Austrian national open data strategy. However, questionnaire respondents were sceptical of whether the DMA would make a significant contribution to the opening of previously closed public sector data, as they felt the current format does not support the enforcement of public sector openness. Social Exclusion. Empirical results suggest that social exclusion is not considered a serious risk to the success of the DMA project. Research subjects involved with the project argued that, as a business-to-business platform, the DMA was not structured to directly involve particular individuals or groups in society, and that firms and entrepreneurs would drive the platform's success by using its resources to fuel data-driven innovation. However, it was noted that that the third sector could and would benefit significantly if NGOs chose to participate as business actors over the platform. Gender Inequality. Results from the empirical studies suggest that gender inequality is not considered a significant threat to the successful roll-out of the DMA in practice.

But can the DMA be used as a mechanism to correct prevailing gender inequalities in Austria? Research subjects seem to think notin its current format the DMA is not perceived to promote indiscriminate, gender-neutral access to data. Responses to the questionnaire indicate that the DMA format could be restructured to promote affirmative gender-related action. From a technical standpoint, too, system design elements could be incorporated to analyse and rectify gender bias in traded datasets.

Emerging Themes and Issues

A fundamental idea emerging from our research is that data is no longer just an economic asset, it has also become a social good. When traded over a data market platform, being ascribed with a definite monetary value and possessing definite pecuniary advantages for those business entities that use it effectively, there is no doubt that data has substantial economic value. However, this same commodity often loses its freemarket value when applied to tackle long-standing social problems and empower the vulnerable within a society. Conflict has already emerged between business priorities and the social imperative, and the success of a data market will hinge on whether or not data producers can look beyond monetary incentives and share their data for the benefit of society. Questions also arise about how value generated from the trade of data can be distributed within society. Evidence suggests that the nature and extent of economic and social value created through data trading is largely dependent on the actors within the ecosystem themselves, and the manner in which they interact. Co-creation, sharing, and collaboration are three mechanisms by which significant value can be extracted from data. Sectoral differences also determine the manner in which different businesses respond to, and generate value out of, traded datasets. The role of Government within the context of a national data ecosystem, therefore, becomes critical on a number of levels. First, as government policy forms the backbone of any national open data strategy or digital innovation agenda, public agencies can be instrumental in encouraging market participants to open and share data. Second, the role of the public sector can be considered central to the diffusion within society of the benefits arising from data trading: this may occur largely through Government taking the lead in the regulation of data trading spaces; it may result from policies that promote the empowerment of marginalized or vulnerable groups of peoplethe elderly, migrants, or women; or, it can arise through taxation policy, where the economic value generated by businesses is channeled into priority development areas.

Value is also more easily extracted from data when the data ecosystem is mapped out, and the roles of the various actors within itdata providers, data consumers, data broker, etc.are more clearly defined and legalised. This delineation makes it easier to know, and to understand, the different players that make up each data transaction; their particular characteristics, their specific interests, their relationship to other actors, and the value they ascribe to different types of data. Determining the ownership of data, and creating awareness about the latent monetary value of proprietary data, can also be considered important determinants of data market success. In order to correctly value traded data, it is important to ascertain the degree to which data owners have control over their data, as well as identify the economic incentives open to producers of data to share data products over a marketplace platform. Creating a sense of data ownership also contributes towards better data protection, and safeguards privacy, by encouraging more responsible data handling and data management practices.

Setting out the system-and national-level business case for data trading is central to creating a critical mass of data that can be used to drive innovation over the platform. In particular, it is important to clearly identify, amplify and promote those incentives that will encourage data producers to part with their resource holdings. Encouraging unfettered supply-driven innovation is also important in this context. To realise the benefits from increased data trading, and to mitigate the potential for conflict between freemarket principles and notions of data as a public good, it is also vital for a country to establish a sound open data strategy and a balanced regulatory framework for data trading. These must take into account both national public priorities and free-market economic imperatives in order to help steer the development of a national data trading infrastructure in a direction that is at once socially desirable and profitable.

The final observation arising from this research is that, without the appropriate incentives and safeguards in place, the unfettered trading of data can result in the amplification of prevailing inequalities and biases. A data market project like the DMA needs to be carefully planned out, and its potential consequencesintended and unintended thoroughly analysed from multiple perspectives, before it can be put into practice. It would be useful in this context, therefore, to compare social and cultural implications of a data market based on existing examples from around the world.

Conclusion

From a systematic review of relevant scholarly and practitioner literature, and a critical analysis of empirical research results obtained from workshop deliberations with data science experts and questionnaire responses from DMA project partners, a number of key themes, perspectives, lessons, and questions emerge and a number of conclusions arrived at. This research has identified benefits of a data-driven ecosystem especially related to long-standing socio-cultural challenges in the areas of economic growth, employment and job creation, ageing populations, migration, climate change, and smalland medium-sized enterprises (SMEs); and potential societal and cultural impediments to the success of a data market within the areas of privacy and trust, public decision making structures, social exclusion and gender inequality. Our findings reveal that data is a dual-purpose commodity that has both economic value and social application. Tension arises when its monetary value as a public good is not always the same as, indeed less than, its business or free-market value. In a commercial setting, data producers are less likely to share vital resources over a data market platform if they do not perceive significant economic returns. Further, without the appropriate incentives and safeguards in place, the unfettered trading of data can result in the amplification of prevailing societal inequalities and biases. Government, therefore, has a central role to play in reducing this friction. To realise the benefits from increased data trading, and to reduce the conflict between free-market principles and notions of data as a public good, it is vital for a country to establish a sound open data strategy and a balanced regulatory framework for data trading. Although presenting a comprehensive literature review and empirical analysis, the review is not exhaustive. Additional benefits and challenges of data-driven economy from areas beyond business, Economics, Political Science and Information Science could be identified with a different selection of database and keywords. Data collection in complementary data market initiatives could also bring light to contextual differences, adding value to the understanding of social, political, economic and cultural implications of data trading.

  . A number of data portals exist in the climate change domain -leading examples being the World Bank's Climate Change Knowledge Portal [47] and NASA's Climate Change Portal [48].

Acknowledgements. The Data Market Austria project is funded by the "ICT of the Future" program of the Austrian Research Promotion Agency (FFG) and the Federal Ministry of Transport, Innovation and Technology (BMVIT) under grant no. 855404


Athanasia Routzouni
email: nroutzouni@aegean.gr

Athanasios P Deligiannis

Vassilios Peristeras

Stefanos Gritzalis

An intercountry survey of participatory practices used for Open Government Partnership National Action Plan development

Keywords: open government, public policy co-creation, e-participation, Open Government Partnership

This paper maps the participatory process applied during the development of open government commitments by twenty-nine Open Government Partnership countries. It investigates the role and perceived value of e-participation practices and their relationship to inclusive interaction with stakeholders. The insights of the paper on the perceived value of the tools and methods used and the main impediments to the open government commitment creation process may help governments more effectively design their public participation efforts and increase the level of civic engagement in policy making. Although the results of this analysis derive from activities related to open government reforms, they may also be applicable to introducing participatory public policy formulation approaches in a broader range of policy domains.

Introduction

This paper attempts to map participatory practices that are used by governments when designing public policy, specifically open government commitments. It analyzes public participation practices and tools that have been applied by twenty-nine governments that were active in the Open Government Partnership (OGP) during the development of National Action Plans (NAPs) with specific commitments on open government issues. The presented practices have been applied to support an effective dialogue between government and civil society for policy formulation. Successful ones can be used as a tool to strengthen public accountability and transparency in government operations in an era characterized by an erosion of trust in government [START_REF] Janssen | Benefits, adoption barriers and myths of open data and open government[END_REF]. The analysis of the perceived value of the applied practices along with the identification of the main impediments to the process may also help governments more effectively design their relevant public participation processes and increase the level of public participation in policy making.

Although the results of this analysis derive from processes related to open government reforms, they may also be applicable to introducing participatory public policy formulation approaches in a broader range of policy domains.

1.1

The concept of participation in public policy making

Embedding the principle of participation in public policy making is not a new concept.

Arnstein [START_REF] Arnstein | A Ladder Of Citizen Participation[END_REF] introduced in 1969 the "ladder of citizen participation" outlining different levels of citizen participation ranging from pretextual to truly meaningful. Medimorec [START_REF] Medimorec | Vitalizing Democracy through E-participation and Open Government: An Austrian and Eastern European Perspective[END_REF] added a level, "codetermination", in which citizens co-decide, usually with politicians and administrators. The concept of participation evolved to Manzini's "map of participation" [START_REF] Manzini | Design, when everybody designs: An introduction to design for social innovation[END_REF] which further describes the involvement of different actors in the design process along with their interaction. In light of research that points to a loss of social capital [START_REF] Putnam | Tuning in, tuning out: The strange disappearance of social capital in America[END_REF] combined with and a decline in trust of government [START_REF] Nye | Why people don't trust government[END_REF], civic engagement is seen as a necessary prerequisite and central component of vital democracies. The most common approaches to citizen engagement include: electoral approaches (running for office, volunteering at campaigns), legislative and administrative information exchange approaches (public hearings), civil society approaches (participation via volunteer organizations, social clubs, and other forms of association) and deliberative and consensus-based approaches (joint action across sectors of society, classes of people, or types of individuals) [START_REF] Cooper | Citizen-centered collaborative public management[END_REF].

An open, participatory approach to government is suggested to restore legitimacy to government and improve trust in its operation by enhancing accountability [START_REF] Meijer | Open government: connecting vision and voice[END_REF][START_REF] Janssen | Benefits, adoption barriers and myths of open data and open government[END_REF] and attenuate corruption [START_REF] Bertot | The impact of polices on government social media usage: Issues, challenges, and recommendations[END_REF]. By consulting with citizens, government can improve transparency, access and service quality [START_REF] Hilgers | Citizensourcing: Applying the concept of open innovation to the public sector[END_REF].

Enabling meaningful civic participation is seen as a fundamental requirement of well-functioning democracies and should be a priority for governments [START_REF] Peña-López | Citizens as Partners. OECD Handbook on Information, Consultation and Public Participation in Policy-Making[END_REF]. The OECD recognizes three levels of participation: information provision, which although a one way flow from the government to a two-way interaction that enables citizens to provide feedback; and active citizen participation, in which citizen's engagement is requested for defining and shaping policies leaving the final decision under the responsibility of the government [START_REF] Macintosh | Characterizing E-Participation in Policy-Making[END_REF][START_REF] Ahmed | An overview of e-participation models[END_REF].

An alternative citizen participation spectrum is offered by the International Association for Public Participation (IAP2) [START_REF]IAP2 spectrum of public participation[END_REF] with five levels: 1) Inform, which entails providing the public with the necessary information on a given issue; 2) Consult, describing efforts at getting simple feedback from the public; 3) Involve, which describes efforts to engage with the public in more interactive ways to more deeply address their concerns; 4) Collaborate, which entails a meaningful partnership with the public in each aspect of the decision making process utilizing co-creation methods; and 5) Empower, a stage that includes co-creation activities and puts the final decision-making in the hands of the public as well [START_REF] Wimmer | Ontology for an e-participation virtual resource centre[END_REF]. The IAP2 public participation spectrum [START_REF]OGP Website, OGP Co-creation Standards[END_REF] is used by OGP to classify the quality of public participation.

E-participation in public policy making

There is significant evidence that the use of Information and Communications Technology (ICT) can expand opportunities for civic engagement and increase opportunities for people to participate in decision-making processes and service delivery [19]. For these purposes e-participation can be defined as the "process of engaging citizens through ICTs in policy, decision-making, and service design and delivery to make it participatory, inclusive, and deliberative" [20].

According to OECD guidelines [START_REF] Peña-López | Citizens as Partners. OECD Handbook on Information, Consultation and Public Participation in Policy-Making[END_REF] e-participation is dependent on three main conditions: (i) a clear focus on policies, decisions and governance practices to align them with to people's needs; (ii) particular attention to the means of interactioncommunication channels should be readily made available for people to offer their views and communicate among themselves as peers and with public authorities as partners; and (iii) clear focus on the characteristics of the interaction process between citizens and government to ensure the quality and legitimacy of e-participation efforts [START_REF] Macintosh | European e-participation Summary Report[END_REF].

The participation principle in formulating open government public policies

Open and participative governance has evolved into a central pillar in the effort to modernize public administration at a global level [START_REF] Yu | The new ambiguity of open government[END_REF]. Formulating public policies on empowering citizens, enhancing transparency, promoting integrity and accountability are central to government efforts for administrative reforms. In parallel, many international, European and national initiatives assist governmental efforts to promote horizontal government policies on Open Government and Innovation [START_REF] Piotrowski | The "Open Government Reform" movement: The case of the open government partnership and US transparency policies[END_REF]. Public policy formulation is part of the pre-decision phase of public policy making. This phase involves identifying a range of policy alternatives and specifying policy tools and solutions to address the challenges within the public policy agenda [START_REF] Sidney | Policy formulation: design and tools[END_REF].

One of the most notable initiatives to support the global open government agenda is the Open Government Partnership (OGP), a multi-stakeholder platform launched in 2011, which in 2019 brings together 79 countries that have committed to making their governments more open and accountable [START_REF]About OGP[END_REF]. Recognizing that the process of formulating open government policies involves a wide network of stakeholders and policymakers, OGP promotes effective collaboration between governments and civil society organizations in open government policy making [START_REF] Frey | Open government partnership four-year strategy 2015-2018[END_REF].

Each OGP participating country delivers bi-annually a National Action Plan (NAP) [START_REF]OGP Website, OGP Co-creation Standards[END_REF] which is the product of a co-creation effort between the government and civil society to develop concrete, time-bound and measurable open government commitments. Engaging civil society, citizens, and other stakeholders is considered a core element throughout the developing, implementing, monitoring and reporting processes of the action planning cycle [START_REF]OGP Articles of Governance[END_REF].

The action plan evaluation process is comprised of two activities; progress reports produced by country local researchers working for the Independent Reporting Mechanism of the OGP and self-assessment reports produced by the participating governments. OGP's Participation and Co-creation Standards set out requirements for engaging civil society, citizens, and other stakeholders throughout the OGP process. These standards also guide the content to be included in Self-Assessment Reports and include feedback mechanisms, space for co-creation, and facilitating a multi-stakeholder forum [START_REF]OGP Website, OGP Co-creation Standards[END_REF].

OGP also uses the International Association for Public Participation's (IAP2) "Participation Spectrum" to define the levels of citizen participation in developing an open government action plan. As already mentioned, the spectrum's participation levels are: Inform, Consult, Involve, Collaborate and Empowergovernment and civil society make joint decisions [START_REF]OGP Website, OGP Participation and Co-Creation toolkit[END_REF].

Methodology

The methodology of the present study includes analysis of the responses to a survey questionnaire which collected information on practices and tools applied during the development of OGP National Action Plans. These responses have also been compared to the results of the assessment made by the OGP Independent Reporting Mechanism on the levels of participation during action plan creation, as well as indicators such as the IAP2 spectrum for each country.

Participants

The participants of this study were members of the official network of National Points of Contact (PoC) in OGP participating countries. In each participating government, the PoC is the person responsible for coordinating the government's activities in relation to OGP [START_REF]OGP Support Unit: Government Point of Contact Manual[END_REF]. PoC's are working-level counterparts to a ministerial-level representative. They are direct advisors to the national representative of each country to OGP and are usually drawn from the civil service of each country. During the development of a National Action Plan, the PoC engages with civil society and other stakeholders and cooperates with Ministries and other government organizations involved in the agenda that emerges during the co-creation process. A PoC also engages with OGP Support Unit to receive information on the OGP Participation and Co-creation Standards and on international best practices that could be applicable to the national policy agenda. Twenty-nine PoCs responded to the questionnaire which corresponds to 36.7% of the total OGP countries in 2018. The countries represented are: Argentina, Armenia, Brazil, Chile, Colombia, Finland, France, Germany, Greece, Israel, Italy, Jordan, Lithuania, Luxembourg, Malawi, Malta, Mongolia, Nigeria, Panama, Paraguay, Philippines, Republic of North Macedonia, Romania, Serbia, Sierra Leone, Slovakia, the Czech Republic, the Netherlands and Ukraine.

2.2

Research sources and procedures

Survey questionnaire on the co-creation of OGP National Action Plans An original electronic survey questionnaire based on the OGP Participation and Cocreation Standards [START_REF]OGP Website, OGP Co-creation Standards[END_REF] and recent literature on co-creation methods and tools [START_REF] De Koning | Models of co-creation[END_REF][START_REF] Spagnoli | The Paradigm Shift of Living Labs in Service Co-creation for Smart Cities: SynchroniCity Validation[END_REF][START_REF] Zolotov | E-participation adoption models research in the last 17 years: a weight and meta-analytical review[END_REF][START_REF] Porwol | An ontology for next generation e-Participation initiatives[END_REF] was created to examine the co-creation approaches that OGP participating countries have applied to develop open government National Action Plans. A pilot study was conducted to verify the validity of the structure and content of the questionnaire. Five officials from OGP, working for the Support Unit and the Independent Reporting Mechanism, examined the questionnaire. The comments provided were taken into consideration for the final version. The online questionnaire was subsequently distributed to the official network of the PoCs in all 79 OGP participating governments.

The questionnaire explored the latest NAP creation period (2016-2018) and aimed to collect information related to the following key factors of the co-creation process: who initiated the co-creation process within government; the profile of participating stakeholders and the level of their contribution; the process objectives in each government; the characteristics of the engagement approach; the co-creation methods and tools employed; the number of participants in the co-creation activities; the impediments to the process; and the perceived contribution of the co-creation process on improving the completion rate of the Action Plan reforms.

Use of e-participation methods and tools in the co-creation process

The questionnaire also aimed to identify ICT-supported methods or tools that were used in the co-creation process. Specifically: use of a virtual platform for stakeholder engagement; use of ICT-supported co-creation methods (including social network analysis, online public consultations); use of ICT-supported co-creation tools (including consultation platforms, social media accounts, mobile contribution application, web discussion forums, online surveys); and value of including face to face meetings versus virtual / distance approaches in the co-creation process.

Comparisons between questionnaire responses and OGP assessment results

OGP evaluates country performance on their action plan creation process using both its Independent Reporting Mechanism (IRM) as well as other openly available indicators. OGP then publishes independent progress reports for each participating country via the IRM. For each National Action Plan, the report summarizes the assessment of the development process and the country's progress in fulfilling key open government principles. OGP also releases, in open formats, the data underlying these reports [START_REF]IRM Frequently Asked Questions[END_REF][START_REF]OGP Website, Explorer and IRM data[END_REF]. OGP data include an assessment of how each country has implemented the OGP consultation guidelines and to what extent each government works together with the civil society to develop the National Action Plan. OGP has published co-creation process assessment data for eighteen out of the twenty-nine countries that filled-in the OGP Co-Creation Questionnaire. The responses have been compared to the following indicators derived from already available OGP assessment data: Level of public influence using IAP2 spectrum; "Open participation" or "Invitation only" approach during consultation; and Consultation steps followed via six indicators that represent the main steps to be followed during Action Plan development (Availability of timeline prior to the consultation; Adequate advance notice; Awareness raising activities; Online consultation; In person consultations with the national community; Online publication of a summary of the received comments).

Results

Analysis of the survey questionnaire responses

According to the responses to the questionnaire, the surveyed PoCs indicate that in 55 percent of the cases the government itself initiated the Action Plan development process while in 7 countries (24%) the process has been initiated by the government together with the Civil Society. In the rest of cases (6 countries, 21%), the process was started by a multi-stakeholder forum.

The top identified reasons for using co-creation methodologies are related to strengthening the role of the stakeholders in the process (Engage stakeholders 97%; Increase trust between stakeholders 83%; Identify stakeholder needs 72%; Create a sense of ownership with stakeholders 66%). Raising awareness and communication have also been important factors in the process (Raising the level of awareness on different topics 59%; Improving communication processes 56%). It is worth noting that only one country replied that the required OGP co-creation criteria have been a reason for using co-creation methodologies in the process.

A significant percentage of countries (41.4%) have involved more than 100 participants, while 37.9% have involved more between 31 and 100 participants and 17.2% involved smaller groups of 1-30 people. The participating PoCs (72%) stated that civil society had a high contribution during the co-creation phase (in a scale from 1.Very much to 5.Not at all, 1: 48%; 2: 24%; 3: 24%; 4:3%, 5:0%).

The majority of participating PoCs (69%) stated that the co-creation process improves the content of the action plan (in a scale from 1.Very much to 5.Not at all, 1: 55%; 2: 14%; 3: 17%; 4:7% , 5:7%). More than half of the participants (58%) agreed that the co-creation process also improves the level of completion of the National Action Plan reforms (in a scale from 1.Very much to 5.Not at all, 1: 24%; 2: 34%; 3: 28%; 4:10%, 5:3%).

In terms of involving specific stakeholder groups, all PoCs (100%) responded that they usually engage civil society organizations and government departments. PoCs also involve academics (72%), citizens (55%), as well as the private sector and subnational government representatives (both groups at 72%). Parliament (41%) and the OGP Support Unit (38%) were less frequently involved while political parties have been totally absent from the co-creation process in all 29 countries.

Less than half of the countries (38%) have used e-participation platforms to involve stakeholders in the co-creation process. Except for one country that used a financial award scheme for the most active participating civil society organization, no rewards were used for co-creation participants.

The most frequently used co-creation methods include: SWOT workshops (43%), strategic roadmaps (29%), Living Labs (29%), appraisal interviews (14%), Personas (14%) and social network analysis (14%). To support the co-creation methods, the following tools have been primarily used by the countries included in the survey: Social media (83%), Crowd-mapping (35%), Mobile contribution applications (26%), Mind maps / Collaboration maps (17%), Issue cards (13%) and Motivational matrices (13%).

The participating PoCs indicated that the main impediments in the co-creation process have been the limited resources (66%) and the lack of time (52%) on the part of stakeholders as well as the difficulty in engaging stakeholders (38%). Other impediments include discontinuities caused by government changes and the electoral cycle (28%), difficulties in achieving consensus (21%) and a lack of trust in the effectiveness of the co-creation process (17%).

The OGP PoCs have also been asked to assess the significance of key factors in the co-creation process. The existence of permanent structures and processes, specifically the existence of permanent government agencies responsible for co-creation as well as the existence of a permanent stakeholder forum have been highlighted as key factors for the effectiveness of the process as shown in Figure 1. Almost a quarter of countries introduced ICT-supported methods in the OGP action plan co-creation process (24%). The main e-participation methods used are online public consultations and social network analysis. Less than half of the countries (38%) have used virtual platforms to engage with stakeholders but more countries have used eparticipation tools (69%) in the co-creation process. The main e-participation tools used are social media (68%) and mobile contribution applications (21%). Some PoCs have also reported the use of web discussion forums (4%), online surveys (4%) and consultation platforms (4%).

PoCs report that use of a virtual engagement platforms does not make a significant contribution to NAP content quality. The countries that used a virtual platform, (38% of the total) reported 1: 55%; 2: 9%; 3: 18%; 4:9%, 5:9% while the rest reported 1: 56%; 2: 17%; 3: 17%; 4:6%, 5:6% (in a scale from 1.Very much to 5.Not at all).

Adding face to face meetings during the co-creation process is reported by most PoCs (86%) as being of importance, more so than including virtual approaches in the co-creation process (55%).

3.2

Comparing questionnaire responses and OGP assessment results

Out of the 29 countries participating in this study, 18 participating countries have thus far been up assessed by OGP under the five levels of the IAP2 spectrum. They are classified in Table 1 Comparing IAP2 levels with the use of e-participation tools reveals that of those countries belonging to lower IAP2 levels (Consult and Involve) half (50%) have used e-participation tools while 75% of the countries in the upper IAP2 levels (Collaborate Empower) have done the same. OGP PoCs have been asked to indicate whether the co-creation process improves the content of the Action Plan 1 . Half of the countries (50%) belonging to the lower IAP2 levels (Consult and Involve) stated that the process improves the content of the Action Plan (in a scale from 1.Very much to 5.Not at all, 1: 40%; 2: 10%; 3: 30%; 4:10%, 5:0%). A higher percentage (75%) of the countries belonging to the upper IAP2 levels (Collaborate and Empower) agreed that co-creation improves the content of the Action Plan (1: 50%; 2: 25%; 3: 13%; 4:0%, 5:13%).

When asked if the co-creation process improves the level of completion of the included commitments, 40% of countries in the lower IAP2 levels (Consult and Involve) responded positively (1: 30%; 2: 10%; 3: 50%; 4:10%, 5:0%). More countries (63%) of the ones belonging to the upper IAP2 levels (Collaborate and Empower) have done the same (1: 13%; 2: 50%; 3: 13%; 4:13%, 5:13%).

PoCs have also been asked to assess the contribution of civil society organizations during the co-creation phase. Half of the countries (50%) belonging to the lower IAP2 levels (Consult and Involve) agreed that CSO's contributed highly (1: 30%; 2: 20%; 3: 50%; 4:0%, 5:0%). Most of the countries (88%) of the countries belonging to the upper IAP2 levels (Collaborate and Empower) agreed on the significant contribution of the CSO's in the process (1: 25%; 2: 63%; 3: 0%; 4:13%, 5:0%).

Regarding possible impediments to the co-creation process, half of the countries (50%) that are classified under the upper levels of public influence (Collaborate, Empower) have identified "Difficulties in achieving consensus" as an impediment to their 1 All the results reported in this section of the paper are on a scale from 1: Very much to 5: Not at all, unless otherwise specified in the text. process. On the other hand, no countries (0%) from the group of countries belonging to the lower IAP2 levels (Consult and Involve) recognized this impediment.

In terms of accessibility of the stakeholders to the consultation process, the 18 participating countries that have been assessed by OGP are classified under two relevant categories as follows: Open Participation: Armenia, Brazil, Chile, Greece, Italy, Lithuania, North Macedonia, Paraguay, Romania, Serbia, Sierra Leone and Ukraine, while Invitation Only: Czech Republic, Jordan, Malawi, Malta, Mongolia, Netherlands.

For countries that limited the process to invited stakeholders, the perceived contribution of the civil society organizations is lower compared to countries that applied open consultation procedures (responses for open participation are 1: 33%; 2: 50%; 3: 17% and for invitation only 1: 17%; 2: 17%; 3: 50%; 4:17%).

Out of the countries responding to the survey that have been classified by the OGP (18 countries) on their completion of the prescribed six consultation steps (timeline availability; advance notice; awareness raising; online consultation; in person consultations; summary of comments) most (67%) have followed all six. 2According to the OGP assessment, all countries have performed in-person consultations. (timeline availability: 78%; advance notice: 94%; awareness raising: 72%; online consultation: 83%; in-person consultations: 100%; summary of comments: 78%).

Discussion

According to the paper's key findings, a participatory, open, public policy design process is usually initiated by the government itself with a view to strengthen the role of the stakeholders in the process. Governments usually engage with stakeholders that represent civil society organizations, government departments and academic institutions while parliaments and political parties are less involved in the process. Most countries decide to design consultation procedures that are open and accessible, instead of limiting the process to invited stakeholders. They manage to reach the fourth highest level (Collaborate) in the IAP2 spectrum of public participation which points to a meaningful partnership with between stakeholders in the policy formulation and decision-making process. Governments recognize the effective contribution of civil society organizations during the co-creation phase, especially in countries with a high IAP2 level of public participation. Some governments do introduce virtual platforms for stakeholders' engagement. However, it is not perceived as a practice that increases the effectiveness of the co-creation process or the quality of the process outputs.

Governments apply various co-creation approaches but the wide inclusion of e-participation practices is comparatively low. E-participation is enabled mainly through online public consultations and social network analysis. E-participation tools used mainly include consultation platforms, social media, mobile contribution applications, web discussion forums and online surveys. It is worth noting that e-participation tools are mostly used in co-creation processes in countries that have reached a significant IAP2 level.

Co-creation processes usually favor in-person meeting approaches such as SWOT workshops, strategic roadmaps and Living Labs. These processes might be assisted by ICT participation tools such as crowd-mapping, mind maps and issue cards but they usually require the participants to be present in the same physical setting.

Governments, especially in countries with a comparatively mature participatory process as indicated by their IAP2 level, consider that co-creation can highly improve the contents of the action plan and the level of commitment completion. Additionally, the existence of permanent structures and processes in the government along with the involvement of a permanent stakeholder forum are considered key factors for the effectiveness of co-creation processes. Although virtual approaches are considered effective co-creation processes, governments give higher priority to face-to-face interaction with stakeholders to maximize the effectiveness of the process. In a similar vein, limited human resources and lack of time are considered key impediments to such processesnot lack of ICT tools.

Establishing central agencies that are responsible for citizen feedback and participatory process adoption and co-ordination can improve feedback quality, relevance and utility and significantly aid meaningful citizen participation. Participation efforts themselves could be submitted to stakeholders' review and examination and perhaps be cocreated through citizen feedback. In some cases, meaningful engagement with citizens could suffer by the use if certain 'arm's length' ICT tools such as surveys, quick polls and others.

The key findings of the paper provide insights into the public participation practices used by governments when designing public policy in an open government setting. They point to a realization by government officials involved in supporting open government commitments and policies that citizen participation is understood not simply as an e-government or e-participation project but as an equal access and direct democracy issue. In such a scenario, meaningful participation and inclusive interaction with stakeholders is considered more important than the availability of ICT tools.

Limitations and proposed future research

An analysis of the remaining OGP countries could address limitations arising from the fact that OGP had completed their assessment for a subset of the participating countries in this survey. This could improve conclusions based on comparing data obtained from survey responses and OGP assessment. By the end of 2019 OGP should officially publish all the evaluation reports for the relevant NAPs. These will provide further insights into the effectiveness of the Action Plan implementation in all twenty-nine countries. Future work could focus on further analyzing the data underlying these reports to identify possible associations between National Action Plan development practices and qualitative factors related to the completion and effectiveness of the Action Plan implementation. A follow-up study targeting the remaining OGP countries could also address the fact that of the twenty-nine countries that responded to the survey the representation of European states is comparatively high (52% of European OGP members, 32% of OGP countries in the Americas, 27% of Asia-Pacific countries and 14,3% of the African OGP countries responded).

Fig. 1 .

 1 Fig. 1. Assessment of key factors in the co-creation process (Five-point Likert scale: 0=not important, 4=very important)

Table 1 .

 1 : IAP2 Level of public influence in the co-creation process of OGP countries

	IAP2 level	Countries
	Inform	No countries
	Consult	Jordan, Lithuania, Malawi, Malta, Netherlands, Serbia
	Involve	Armenia, Czech Republic, Italy, Sierra Leone
	Collaborate Brazil, Chile, North Macedonia, Mongolia, Paraguay, Romania, Ukraine
	Empower	Greece

The full classification is as follows: 1 step: Malta; 2 steps: Malawi;

steps: Jordan, Lithuania

; 4 steps:

Netherlands; 5 steps:

Czech Republic; 6 steps: Armenia, Brazil, Chile, Greece, Italy, North Macedonia, Mongolia, Paraguay, Romania, Serbia, Sierra Leone, Ukraine.