Alexis Tsoukias

Gilberto Montibeller

Giulia Lucertini

Valerie Belton

POLICY ANALYTICS: AN AGENDA FOR RESEARCH AND PRACTICE

Keywords: analytics, policy analysis, decision analysis, policy cycle, decision support

In recent years the field of decision analysis has been heavily influenced by the "analytics'' perspective, which integrates advanced data-mining and learning methods, often associated with increasing access to "Big-Data", with decision support systems. This rapidly growing and very successful field of Analytics has been strongly business-oriented since its origin and is typically focussed on data-driven decision processes. In public decisions, however, issues such as individual and social values, culture and public engagement play a much bigger role and, to a large extent, characterise the policy cycle of design, testing, implementation, evaluation and review. From this perspective public policy making seems to be a much more socially complex process than has hitherto been considered by most analytics methods and applications. In this paper we thus suggest a framework for the use of analytics in supporting the policy cycle -and conceptualise it as "Business Analytics".

Introduction

Policy making has been a traditional domain of research and practice, where decision analysts1 have introduced formal methods aimed at helping policy makers improve their decisions (for a recent survey see De [START_REF] Marchi | From Evidence Based Policy Making to Policy Analytics[END_REF]. In recent years the field of decision analysis has been heavily influenced by the "analytics'' perspective, which integrates advanced data-mining and learning methods, often associated with increasing access to "Big-Data", with decision support systems. This rapidly growing and seemingly very successful field of Analytics has been strongly business-oriented since its origin (see [START_REF] Davenport | Analytics at Work: Smarter Decisions, Better Results[END_REF] and is typically focussed on data-driven decision processes. However, in public decisions issues such as individual and social values, culture and public engagement play a much bigger role [START_REF] White | Voices and values: Linking values with participation inOR/MS in public policy making[END_REF] and, to a large extent, characterise the policy cycle of design, testing, implementation, evaluation and review. From this perspective public policy making seems to be a much more socially complex process than has hitherto been considered by most analytics methods and applications [START_REF] Almquist | Public sector governance and accountability[END_REF][START_REF] Juntti | Evidence, politics and power in public policy for the environment[END_REF]. This is not to deny the potential for analytics to contribute throughout the policy cycle, including through the growing area of text analytics to analyse social media, as well as the increasing public access to data and tools for analysis, e.g. Google Public Data Explorer 2 , Guardian Data blog 3

). The intention of this paper is to propose a framework for Policy Analytics within which approaches such as these can be appropriately harnessed.

Analysing and supporting the design, implementation and assessment of public policies is not really a new domain. Political scientists have worked in this field for decades [START_REF] Moran | The Oxford Handbook of Public Policy[END_REF]. Equally, economists have focussed much of their research on rational theories of public decision making and formal methods for the ex-ante and ex-post evaluation of public policies (see for instance [START_REF] Dollery | The Evaluation of Public Policy: Normative Economic Theories of Government Failure[END_REF]. Cost-Benefit Analysis (see [START_REF] Dasgupta | Cost-benefit analysis: Theory and practice[END_REF]Nas 1996) is widely used and, perhaps, the best known method for evaluating public policies among both practitioners and researchers. However, Cost-Benefit Analysis is not without contemporary critics (for example, [START_REF] Ackerman | Priceless: On Knowing the Price of Everything and the Value of Nothing[END_REF][START_REF] Adler | New Foundations of Cost Benefit Analysis[END_REF] and many other approaches have been developed including, in recent years, Real Options Analysis (see [START_REF] Smit | Strategic Investment: Real Options and Games[END_REF][START_REF] Trigeorgis | A Real Options Application in Natural Resource Investments[END_REF]). Decision Analysis and Operational Research have also developed methods which aim at addressing public policy making (see for instance [START_REF] Pollock | Operational Research and the Public Sector[END_REF]. At the same time "analytics" has developed as increasingly an independent domain, both for practice and research, growing out of and bringing together more traditional fields such as statistics, data analysis, data mining, knowledge extraction and machine learning [START_REF] Davenport | Competing on Analytics: The New Science of Winning[END_REF][START_REF] Davenport | Analytics at Work: Smarter Decisions, Better Results[END_REF][START_REF] Liberatore | The Analytics Movement: Implications for Operations Research[END_REF].

An important question arising from the above observations is whether the field of Decision Analysis is giving appropriate consideration to the specific requirements for Analytics within the wide area of supporting Public Policy Making. Thus, the main motivation of this paper is to raise a number of questions and to prompt discussion about the skills and awareness that Decision Analysts need to operate effectively in this domain. In particular:

-Are practitioners involved in supporting decision processes in the public domain appropriately using analytics and, if so, what type of analytics could fit the type of support they use? -Is decision analysis training appropriate for helping the integration of traditional decision analytic tools and analytics in the area of public policy making? -Are researchers considering appropriately the relationships between the specificities of public policy making and analytics?

Our paper does not seek to provide an exhaustive reply to the above questions, but will try to provide our perspective on them. We also consider it important to raise and discuss the following assertions:

-Policy Making is a type of decision process with specific characteristics, thus demanding dedicated analytical methodologies. -Business Analytics does not always fit the requirements most policy cycles demand, particularly regarding the creation of knowledge to support such cycles. -In order to improve both policy making processes and decision aiding processes we need to integrate data-driven decision making with value-driven decision making. -Currently, most analytics methods are based on benchmarking and descriptive approaches, while supporting the activities occurring within a policy cycle needs to focus on constructive approaches, including constructive benchmarking and learning.

The paper has the following structure. In section 2, we discuss some key characteristics of policy making processes, distinguishing them from other decision processes. Section 3 reviews the types of policy analyses available to decision analysts and policy makers. We then define what we mean by Policy Analytics in section 4 and suggest opportunities that it can offer to decision analysts. The paper concludes with some suggestions on how this emerging field can be further developed.

Key Characteristics of Policy Making

What makes a policymaking process different from any other decision process? Why is policy making not just common decision making (but perhaps a little more complex)? We need to understand what characterises policy decisions, making them unique. First of all, we must recognise that speaking about policy-making means speaking about a "policy cycle" (Lasswell 1956). The policy cycle consists of a set of sequential actions linked by some main "goal" or, more generally, by some common public issue. This cycle is composed by eight major steps: issue identification, defining policy objectives, policy design, policy testing, policy finalisation, policy implementation, policy monitoring & evaluation, policy readjustment & innovation.

Not only are policy makers engaged(or should be engaged) in this policy analysis cycle, they are also confronted by five major complexities inherent to public decision making, which we describe next (see also [START_REF] Dunn | Public Policy Analysis[END_REF][START_REF] Parsons | An Axiomatic Approach to Robustness in Search Problems with Multiple Scenarios[END_REF][START_REF] Hill | The Public Policy Process[END_REF][START_REF] Kraft | Public Policy. Politics, Analysis and Alternatives[END_REF][START_REF] Davies | Is evidence-based government possible? Jerry Lee Lecture[END_REF]) .

Use of public resources. Although the use of public resources may appear obvious in this context, it also has many implications. First, many resources, tangibles or intangibles, used within the policy cycle are provided by the government or other public institutions and, under such a perspective, are considered public goods. Second, during a policy cycle, policy makers use resources provided also by those who are not involved in the cycle. Third, public policies allocate (or redistribute) resources and among the beneficiaries we can find people who are not implied in the policy cycle and are potentially considered not to be concerned with it. Fourth, decisions (and thus policies) are "irreversible allocations of resources" opening the way to ethical, moral, intergenerational, social justice, environmental issues of very basic nature (almost ideological).

Multiple stakeholders. Policy cycles are "de facto" participative, as many different actors (being citizens, groups or organisations) feel to be naturally involved, as soon as they discover that one of their private concerns can be associated with that policy cycle. Of course participation can be structured or not, allowed or not, visible or not, formal or informal -but it happens independently from the willingness of who "promotes" that given policy cycle. The result is that within the cycle we need to take into account the different concerns carried by the multiple stakeholders and the resources they carry with them, but we also need to handle the different "languages" practiced by all these stakeholders, the distinctive perceptions they might have of the policy cycle, and the different expectations they present. Last, but not least, we need to consider the potential confusion that occasionally may occur, when a policy cycle does not have a clear "decision maker" due to institutional rules, inconsistent legal frameworks and asymmetries in decision power distribution.

Long-time horizon. A policy cycle usually takes a considerable amount of time in order to be developed and this holds even for non-strategic policies. Moreover, the effects and consequences of a policy cycle may become visible only a long time after the cycle occurred and can hold for even longer periods. This may conflict with the agendas of different types of stakeholders. Policy makers usually have short-term agendas due to the timing of politics. Experts and/or analysts may have medium to long-term agendas due to the specific knowledge they have about the policy issue. Citizens may have agendas varying from very short term to very long ones, depending on the concerns they carry in the policy cycle. Such conflicting agendas and different time frames add further structural uncertainty to the policy cycle besides that generated by the difficulty to predict how social and economic scenarios may evolve in the future.

Legitimation and Accountability. What do policy makers look for while engaged in a policy cycle? They look for legitimation: legitimation for themselves, for their actions within that policy cycle specifically, for the outcomes of the policy making process and for the policy making process itself.

Legitimation can be obtained from different sources (the law, tradition, moral standards, knowledge, practice etc.) and is the cornerstone of the rationality developed by each stakeholder involved in the policy cycle (see [START_REF] Habermas | Logic of the Social Sciences[END_REF]. Under such a perspective, a key aspect in gaining legitimation by the policy making process itself are requirements of increasing the level of participation in the policy cycle, of providing transparency for the decision process (explanations and justifications of the outcomes, clear argumentation etc.), and of demanding more accountability (in the sense of "providing an account of") from policy makers.

Deliberation. Policy cycles occur in the public domain and, at least part of them, are "public decision processes". In order to be "public", decision processes need to establish "deliberations": those moments of the process where "decisions" are formally adopted, become officially known in the public domain, can be enforced by law, and the allocation of resources linked with the decision becomes irreversible. Deliberations are a crucial part of the policy cycle because they structure the timeline where the cycle occurs. However, this is not linearly perceived by the stakeholders: it becomes more "dense" when deliberations are expected to occur (and immediately afterwards).

Why should decision analysts pay special attention to the characteristics mentioned above? It is easy to note that such characteristics are strictly inter-related. They contribute in characterising how the policy making process is structured and, potentially, allow us to understand how it could be conducted. Under such a perspective, decision analysts are expected to provide some decision support exactly in these cases where one or more stakeholders need to establish their position and actions in the policy cycle, an issue captured by the concept called "Action-Arena" [START_REF] Ostrom | An agenda for the study of institutions[END_REF] or "Interaction Space" [START_REF] Ostanello | An explicative model of `public' interorganisational interactions[END_REF].

In other terms, decision analysis is expected to be used along the whole policy cycle, from the agenda setting to the assessment of alternative actions and their consequences, up to ex-post evaluation of the whole policy. Under such a perspective, we need both a richer toolkit of methods, which support decision makers along the process, as well as a comprehensive methodology allowing for a coherent structuring of the decision aiding process (see [START_REF] Bouyssou | Evaluation and Decision Models: a critical perspective[END_REF][START_REF] Belton | Muliple Criteria Decision Analysis: An Integrated Approach[END_REF][START_REF] Tsoukiàs | On the concept of Decision Aiding Process[END_REF][START_REF] Tsoukiàs | From Decision Theory to Decision Aiding Methodology[END_REF].

Analysis for Policy Makers

In this section we suggest that there is a growing demand for policy analysis and briefly review different types of modelling already developed with this intent. We then review, again briefly, the main components of the analytics movement.

Demand for Policy Analysis

Policy decisions impact large numbers of citizens on many different aspects of our social, economic and cultural life. As for any type of decision process, policy makers have to assume the full responsibility of their policies with respect to both private and public stakeholders. Under such a perspective, as we argued before, policy-making is a complex decision process and has always been a field where decision support has been sought: from using statistical information to applying decision analytic and operational research methods [START_REF] Larson | The Decision Process: Seven Categories of Functional Analysis[END_REF][START_REF] Rosenhead | Operational Research in Urban Planning[END_REF][START_REF] Pollock | Operational Research and the Public Sector[END_REF][START_REF] Dorling | Statistics in Society[END_REF].

However, in recent years we can observe a number of trends affecting the nature of demands policy makers address to analysts of any type (for a detailed discussion see [START_REF] Hill | The Public Policy Process[END_REF][START_REF] Kraft | Public Policy. Politics, Analysis and Alternatives[END_REF][START_REF] Moran | The Oxford Handbook of Public Policy[END_REF][START_REF] Nutley | From knowing to doing: Aframework for understanding the evidence-into-practice agenda[END_REF]:

-an increasing demand for participation in the policy making process, coming from opinion groups and single citizens; -an increasing mistrust between citizens and policy makers, as well as between citizens and "experts";

-an increasing social fragmentation, resulting in a loss of representativeness by traditional political parties and social organisations (such as the trade unions); -an increasing mistrust of science and consequent limits on its ability to convince citizens about policy consequences and impacts; -an exponential increase to the amount of information to which citizens can have access, information provided (most of the times) without any check on reliability and truthfulness..

The result is that the policy making process became ever more challenging, since the demand for accountability and legitimation, for both the process and its outcomes, becomes stronger.

To some extent this situation is captured by the appearance in recent years of several manuals and guidelines concerning the assessment of public policies at national and European level, such as:

-the Cost-Benefit Analysis manual of the European Union

Modelling for Policy Making

The use of formal analysis in supporting policy making has a long tradition in the literature. Besides the traditional field of Public Policy Analysis, Operational Research (OR) has also been often used to analyse public policies and recommend actions. A relatively recent push for evidence-based management [START_REF] Pfeffer | Hard Facts, Dangerous Half-Truths and Total Nonsense: Profiting from Evidence-based Management[END_REF], which has also been extended to public decision making [START_REF] Tavakoli | Decision analysis in evidence-based decision making[END_REF], has increased the attractiveness of formal policy analysis. In this section we briefly review such developments.

Public Policy Analysis

The field of Public Policy Analysis (for an overview see the classic textbook by Dunn 2012 and also [START_REF] Parsons | An Axiomatic Approach to Robustness in Search Problems with Multiple Scenarios[END_REF] has its roots in Economics and in Political Science. Economists' main focus is on how market-structures can better allocate scarce public resources, while political scientists are mostly concerned with understanding the roles of politics and the government in public policy processes [START_REF] Lindblom | The Policy-Making Process[END_REF].

A large body of public policy analysis is devoted to retrospective (ex post) analysis, which tries to understand the causes and consequences of policies after they have been implemented. Equally relevant in policy analysis is the role of prospective (ex ante) analysis, which encompasses the forecasting of consequences if policies were to be implemented and prescriptions about which policies should be implemented [START_REF] Dunn | Public Policy Analysis[END_REF].

Common tools for prescriptive analysis in this field are net present value assessments of costs and benefits of potential public policies, as well as cost-benefit analysis [START_REF] Munger | Cost-benefit analysis: Theory and application[END_REF], and cost effectiveness analysis, the latter often employed for military expenditure decisions [START_REF] Dunn | Public Policy Analysis[END_REF]. Challenges in those analyses involve how to properly monetise all benefits and how to set up an adequate discount rate for public goods without market prices [START_REF] Ackerman | Priceless: On Knowing the Price of Everything and the Value of Nothing[END_REF][START_REF] Adler | New Foundations of Cost Benefit Analysis[END_REF][START_REF] Montibeller | Raising the bar: strategic multi-criteria decision analysis[END_REF].

The Evidence-Based Policy Making approach, introduced by the UK government in the 1990s, represents the most recent "practice oriented" attempt to strength the policy making process: " … evidence-based policy helps people make well informed decisions about policy, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation" [START_REF] Davies | Is evidence-based government possible? Jerry Lee Lecture[END_REF]). This attempt extends the idea of governing based on "facts" (instead of ideology) typical of the European culture since the enlightenment (for a discussion see [START_REF] Dryzek | Policy Analysis as critique[END_REF]. However, this approach, criticised under many aspects (see [START_REF] Almquist | Public sector governance and accountability[END_REF][START_REF] Marchi | From Evidence Based Policy Making to Policy Analytics[END_REF], failed to become a standard, perhaps the main reason was its inability to convince policy makers that the typical difficulties of legitimating public policies could be overtaken by using it.

Public Sector Operational Research

After the early developments of OR, and successful military applications during World War II, business applications followed, particularly in the USA. In the 1960s RAND started using OR methods for public decisions, initially only dealing with hard data, but later on developing methods for policy analysis which could take into account soft factors, such as values and subjective judgments, as well as future scenarios (RAND 1996). In the late 1960s the British central government implemented an OR group, with the intention of promoting the use of OR in decision and policy making [START_REF] Kirby | Operations Research Trajectories: The Anglo-American Experience from the 1940s to the 1990s[END_REF]. Some limitations of traditional OR methods, such as an over-reliance on quantitative data and the use of an expert mode of analysis [START_REF] Franco | Facilitated modelling in operational research[END_REF], led to the development, in Britain, of Problem Structuring Methods (for an overview [START_REF] Mingers | Rational analysis for a problematic world revisited: problem structuring methods for complexity, uncertainty and conflict[END_REF]. These methods rely heavily on participative engagement with decision makers, adopting a facilitative mode of engagement [START_REF] Franco | Facilitated modelling in operational research[END_REF], and simple, often qualitative, models. Such methods have been used extensively in public sector decisions, particularly in Britain and continental Europe [START_REF] Rosenhead | Into the Swamp: The Analysis of Social Issues[END_REF][START_REF] Shaw | Problem structuring methods: new directions in a problematic world[END_REF].

Another important source of policy analysis support was the development of decision analysis in late 1960s [START_REF] Raiffa | Decision analysis: introductory lectures on choices under uncertainty[END_REF]), with the use of expert judgement in defining subjective probabilities of outcomes, and further extensions to decision with multiple objectives in mid 1970s [START_REF] Keeney | Decisions with multiple objectives: preferences and value tradeoffs[END_REF]. Decision Analysis has been used extensively since then for the analysis of many important policy decisions, particularly in the USA (e.g. [START_REF] Merkhofer | A Multiattribute Utility Analysis of Alternative Sites for the Disposal of Nuclear Waste[END_REF]Keeney 1987, Rosoff and[START_REF] Rosoff | A Risk and Economic Analysis of Dirty Bomb Attacks on the Ports of Los Angeles and Long Beach[END_REF]. A more recent stream of development is the use of decision analysis embedded in recurrent processes of public prioritisation (e.g. [START_REF] Del | An integrated process and management tools for ranking multiple emerging threats to animal health[END_REF][START_REF] Bana E Costa | Development of Reusable Bid Evaluation Models for the Portuguese Electric Transmission Company[END_REF]).

Last, but not least we should mention the literature about Data Envelopment Analysis and more generally about performance measurement in the public sector (initiated by an application about the effectiveness of public schools, see [START_REF] Charnes | Measuring the efficiency of decision-making units[END_REF]). (The reader can have a more comprehensive idea of the subject in [START_REF] Thanassoulis | Assessing police forces in England and Wales using data envelopment analysis[END_REF][START_REF] Emrouznejad | Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA[END_REF]

Business Analytics

Business analytics has been initially developed mainly for the private sector (Davenport andHarris 2007, Davenport et al. 2010). Seen from a very pragmatic point of view, "Analytics" is an umbrella term under which many different methods and approaches developed in the past converge. These include statistics, data mining, business intelligence, knowledge engineering and extraction, decision support systems and, to some extent, operational research and decision analysis. The key idea consists in developing methods through which it is possible to obtain useful information and knowledge for some purpose, this typically being conducting a decision process in some business application.

The smart part in developing "Analytics" has been to merge different techniques and methods in order to optimise both the learning dimension as well as its applicability in real decision processes. In recent years "Analytics" has been associated to the term "Big Data" in order to take into account the availability of large data bases (and knowledge bases as well) possibly in open access (Open Data organisations are now becoming increasingly available). Such data come in very heterogeneous forms and a key challenge has been to be able to merge such different sources, in addition to solving the hard algorithmic problems presented by the huge volume of data available.

However, the mainstream approach developed in these years is based on two limiting hypotheses. The first is that the learning process is basically "data driven" with little (if any) attention paid to the values which may also drive learning. This is with regard both to "what" matters and also to "why" it matters, potentially incorporating considerations of the extent to which different stakeholder perspectives are valued or trusted. The second is that, in order to guarantee the efficiency of the learning process seen, from an algorithmic point of view, as a process of pattern recognition, it is necessary to use "learning benchmarks" against which it is possible to measure the accuracy and efficiency of the algorithms. While this perspective makes sense for many applications of machine learning it is less clear how it can be useful in cases where learning concerns values, preferences, likelihoods, beliefs and other subjectively established information, which is potentially revisable as constructive learning takes place.

Policy Analytics

A Definition of the Concept

To support policy makers, decision analysts need to use existing information (facts, science, local knowledge, best practices, etc.), need to constructively model opinions, values, and judgments of the stakeholders, and need to do so in a meaningful, operational and legitimate way. We denote this set of skills under the term of "policy analytics'': i.e. the use and/or development of adequate methods and suitable processes for creating purposeful knowledge, aiming at supporting those specific type of decisions processes occurring within a policy cycle.

What Can Policy Analytics Offer?

So what can Policy Analytics offer to policy makers dealing with complex policy decisions? We suggest some avenues that might be explored by decision analysts.

In terms of data requirements, we see two major roles for Policy Analytics, in the same way that there are for Business Analytics. The first one is to explore databases which already exist. The second one is to gather data and create new databases, to explore particular issues relevant to policy makers. However, the nature of policy decisions, which we described earlier, makes these two roles very distinct from Business Analytics.

In exploring data which already exists (such as citizens' votes, preferences and demands, their relation to demographics, etc.) decision analysts must recognise that the multiple stakeholders involved and/or affected by a decision, and the multiple objectives pursued by the policy makers, mean that benchmarks cannot be easily defined or arbitrarily set. Policy Analytics thus needs to focus on constructive approaches, including constructive benchmarking and learning, to support decisions in the policy cycle. The need to construct benchmarks rather than simply basing them on measured performances or behaviours, or arbitrarily defining them, requires an understanding of the stakeholders' values and their power structure and, often also calls for their engagement in the design of the analysis. The need to promote constructive learning requires a focus on models for learning rather than models to give the "right" answer [START_REF] Geus | Planning as learning[END_REF]) and a recognition that the role of the analytical models is a constructive one [START_REF] Watzlawick | Change; principles of problem formation and problem resolution[END_REF][START_REF] Roy | Decision science or decision-aid science?[END_REF], For example, consider a situation in which healthcare providers are assessed on the basis of the average time a patient stays in the hospital. This apparently simple indicator is subject to many contextual factors alongside differing social and political interpretations. For example, in some countries "long stays" may be considered by patients as an indicator of "ineffective" and "expensive" health services, in particular where payment is direct or through purchased insurance, as in the USA. On the other hand, in countries where healthcare is provided by the state "long stays" could be considered as high quality care, indicating a health service which does not discharge a patient until assured of a full recovery. Yet another perspective is that long stays are indicative of inadequate provision of continuing care, resulting in frustration for the patient who cannot be discharged from hospital because the appropriate support is not available in the community. We are not going to discuss whether such approaches to health are justified or not. What is important is to note that the same statistical information (data) is likely to be interpreted differently depending on context, values and culture. Now consider the different stakeholders involved in health care management: the patients and their families, the health workers, the managers of the service and the political authorities under which health care is delivered in terms of public policy. Once again the same information (average time a patient stays in a hospital) will be considered differently depending on the stakeholder's objectives; for instance, the general manager of a hospital under pressure to increase the intake of new patients versus the patient wishing to be fully treated versus the politician wanting to be re-elected. In using this information such as this to inform any policy decision we need to consider the purpose, what is valued by whom and how, the context and culture where the policy is being developed and where it will be implemented. In order for a policy to be "legitimated" it will need to address such multiple (possibly conflicting) concerns or to be imposed by a policy maker owning a legitimation obtained beyond that precise context. In the first (and in practice more usual) case, legitimation is obtained by exchanging resources: a critical resource being information and knowledge. It is at this stage that "analytics" becomes crucial, since it should provide not just supporting information, but legitimating information 11 (showing for instance that reducing the average stay in hospital within a welfare state context will in the long term result in either worsening the quality of the service or in increasing its long term costs). The construction of such information requires taking into account long term effects and consequences consistent with the timing of most policy cycles. Finally it should be noted that relevance is perceived, registered (and measured) in specific moments where decisions are deliberated.

Perhaps an even more important role for Policy Analytics is its potential for creating new databases, gathering information which is relevant for the analysis and decision making, and exploring the data to support the policy-making cycle. Again constructivism [START_REF] Watzlawick | The invented reality: how do we know what we believe we know? Norton[END_REF][START_REF] Roy | Decision science or decision-aid science?[END_REF]) provides a proper conceptual background for this role, in our view, given the complex nature of societal problems and the need to understand multiple perspectives, consider multiple impacts on different sectors of the society, and assess options under multiple and often conflicting objectives.

Another key aspect, is that Policy Analytics must emphasise value-driven analysis which can support value-driven decision-making [START_REF] Keeney | Value-Focused Thinking: A Path to Creative Decisionmaking[END_REF], rather than being highly data-driven as is often the case in the context of business analytics. A value driven analysis understands that the alternative policies are means to achieve the values and objectives that society is pursuing. Different policies will have different impacts on the extent to which such values are achieved or upheld and, as these values often represent objectives held by different stakeholders, may impact unevenly on different segments of society.

Related to this, another opportunity which has to date been often neglected by policy analysis and public sector OR, is in supporting the design of better policies with a value-driven analysis perspective for example [START_REF] Gregory | Creating Policy Alternatives Using Stakeholder Values[END_REF][START_REF] Montibeller | Raising the bar: strategic multi-criteria decision analysis[END_REF]. Indeed within this perspective, the analysis is seen as supporting pro-active policy making, which tries to address problems and improve society, instead of reactively coping with public dissatisfaction and complaints.

We envisage several opportunities for the use of Policy Analytics in organisational contexts. A major opportunity is to embed analyses to inform decisions in the policy analytic cycle, for instance, in helping the identification of issues, in predicting impacts of possible policies, in policy design, in simulating policy implementation, and in helping the evaluation and monitoring of implemented policies. In Table 1 we suggest how Policy Analytics can support each step in the policy cycle and try to distinguish it from how Business Analytics would provide support in a similar business setting.

Conclusions and Further Directions of Development

In this paper we have suggested a conceptualisation for policy analytics and proposed a framework for it use in supporting public decisions processes occurring within a policy cycle. We argued that there is demand for policy analysis, and reviewed briefly methods that provide formal analysis in this context, such as public policy analysis, public sector operational research, decision analysis, and data development analysis applied to assessing public organisations.

We also argued that business analytics, which was developed mainly for supporting decisions in the private sector, while powerful and increasingly applied, has two limiting hypothesis: it is data driven and needs clear benchmarks to be set. We believe that these are challenging if one wants to apply analytics in supporting the policy cycle.

Let us now discuss the three motivations for which we started this discussion:

-Are practitioners involved in supporting decision processes in the public domain appropriately using analytics and, if so, what type of analytics could fit the type of support they use? Only to some extent. First, the types of analytics readily available have been designed for business purposes. Despite being generally helpful (in allowing data to support more effectively decision processes), business analytics may not fully fit the requirements of the policy cycle as these have been presented in this paper. Second, policy making is, and will remain, essentially a value driven decision process. Besides learning from data we need to learn from values: under such a perspective practitioners need tools, methods, and models allowing the consideration of this perspective. Furthermore, the clients (policy makers) need also to understand the difference between data versus value-driven analysis: if offering simple data driven decision support can be misleading, demanding simple data driven policy support can also be misleading. Policies are not in the data, they are in the values. Third, such tools, methods, and models need to support the whole policy cycle.

-Is decision analysis training appropriate for helping the integration of traditional decision analytic tools and analytics in the area of public policy making? We think that perhaps not in full. Most of the training of decision analysts does not address the specific requirements of the policy cycle, while most of the training of policy scientists does not address the issue of using formal methods of decision support. We need to establish training for decision analysts that could cover the whole spectrum of policy cycle issues, potentially from an interdisciplinary perspective and incorporating awareness of the social and political contexts of policy making. In doing so we must address decision support as a methodology, not just as a collection of tools and methods. We need decision analysts able to understand the complexity of the policy cycle, to be flexible enough to shift from one tool to another one (from problem structuring methods to quantitative modelling, from learning procedures to justification construction), to be aware of the possibilities offered by the new technologies but also to be clever enough to construct new paths within it.

-Are researchers giving appropriate consideration to the relationships between the specificities of public policy making and analytics? The above discussion introduces a number of challenges for researchers. If we think that such challenges need to be pursued, then our research needs to address them. In the following we present an agenda of issues we consider relevant (but not it is not an exhaustive list).

• Preference Learning. If policy making should reflect societal values, then we need to learn about them (the policy makers ' values, stakeholders' values, etc.). Values are often operationalized as preference statements, either comparative or absolute ones. The field of preference learning is increasingly becoming an important research area (see Fürnkranz and Hüllermeier 2010) addressing such issues, although more from a machine learning perspective; [START_REF] Belton | Interactive Multiobjective Optimization from a Learning Perspective, in Multiobjective Optimization -Interactive and Evolutionary Approaches[END_REF] discuss the interaction between individual learning and model learning and associated research challenges in the context of interactive decision making. A more constructive learning approach needs to be developed (see [START_REF] Bruner | Actual Minds, Possible Worlds[END_REF][START_REF] Mousseau | A general framework for constructive learning preference elicitation in multiple criteria decision aid[END_REF]. • Scenario Planning. The long-term implications of policy making imply the need to consider the range of possible futures, sometimes characterised by deep uncertainties and calling for the development of future scenarios (see [START_REF] Godet | The art of scenarios and strategic planning: tools and pitfalls[END_REF]Montibeller et al. 2006;[START_REF] Schroeder | Scenario-based multiple criteria analysis for infrastructure policy impacts and planning[END_REF][START_REF] Ram | Exploring the impact of evaluating strategic options in a scenario-based multi-criteria framework[END_REF][START_REF] Thj | Integrating multicriteria decision analysis and scenario planning: Review and extension[END_REF]. Further research on how scenarios are constructed (from paths along a decision tree to precise configurations of interaction spaces or arenas) and how to address issues of robustness in scenario planning would be welcome (see [START_REF] Roy | A missing link in {OR-DA}: Robustness analysis[END_REF][START_REF] Ph | Robust solutions and methods in decision aid[END_REF][START_REF] Levy | Using environmental indicators to quantify the robustness of policy alternatives to uncertainty[END_REF][START_REF] Wong | A rigorous definition of robustness analysis[END_REF][START_REF] Parsons | An Axiomatic Approach to Robustness in Search Problems with Multiple Scenarios[END_REF]Liesö et al. 2007).

• Argumentation Theory. As already introduced by [START_REF] Habermas | The Theory of Communicative Action[END_REF], legitimated policies are the ones which are appropriately explained, justified, supported and not sufficiently confuted (i.e. argued). Argumentation theory (Aristotle 1990;[START_REF] Schopenhauer | Dialektik in Schopenhauers handschriftlicher Nachlaß[END_REF][START_REF] Toulmin | The Uses of Argument[END_REF]) establishes a formal and rational framework for how to construct, use, exchange and confute arguments. Although hardly used as support in policy analysis (for exceptions see [START_REF] Atkinson | Parmenides: Facilitating democratic debate[END_REF][START_REF] Rehg | Computer decision-support systems for public argumentation: assessing deliberative legitimacy[END_REF][START_REF] Cartwright | Political Engagement Through Tools for Argumentation[END_REF][START_REF] Modgil | A general account of argumentation with preferences[END_REF] there is scope for further investigating several of its dimensions: the construction of explanations and justifications; the construction of argumentation scenarios; the issue of legitimate arguments (in collective debate);the relationship between game theory and argumentation theory.

• Support for Problem Structuring and Formulation. A large part of the decision support activities occurring within a policy cycle are about understanding, formulating and structuring "problems". Problem structuring methods (see [START_REF] Franco | Special Issue: Problem Structuring Methods I[END_REF][START_REF] Shaw | Special Issue: Problem Structuring Methods II[END_REF]) are now widely acknowledged as part of decision analytic tools; and there is a growing but still small body of research and practice on how to integrate such methods with other formal and/or quantitative methods (for examples see [START_REF] Belton | Integrated support from problem structuring through to alternative evaluation using COPE and V•I•S•A[END_REF][START_REF] Bana E Costa | Decision Support Systems in action: Integrated application in a multicriteria decision aid process[END_REF][START_REF] Montibeller | Causal Maps and the Evaluation of Decision Options[END_REF]Montibeller et al. 2007a;Montibeller et al. 2007b;[START_REF] Belton | Problem Structuring for Multiple Criteria Analysis, in New Trends in Multicriteria Decision Analysis[END_REF][START_REF] Howick | Mixing OR methods in practice: past, present and future directions[END_REF] for a survey of practice in OR). More importantly, from an analytics point of view, there are challenges on how problem formulation and problem structuring can be conducted in the absence of small-groups of decision makers, which has been typically the case in traditional applications of problem-structuring methods.

• Reformulation of Decision Problems. Policy cycles involve, often, long decision processes.

They also are "learning processes" for stakeholders implied in the cycle. Updating, revising and reformulating decision models is a regular activity within policy cycles and are all activities which could potentially be supported. This is an issue already addressed in the literature, both from a general point of view (see Gärdenfors 1988;[START_REF] Katsuno | On the difference between updating a knowledge base and revising it[END_REF] and for specific decision support purposes (see [START_REF] Tsoukiàs | Preference Modelling as a Reasoning Process: a new way to face Uncertainty in Multiple Criteria Decision Support Systems[END_REF][START_REF] Liberti | Reformulations in mathematical programming: definitions and systematics[END_REF]), but we are still far from having a comprehensive framework which could be also practically applied (although we have now specific formal languages for some classes of reformulation problems, see [START_REF] Liberti | The Reformulation-Optimization Software Engine[END_REF]).

• Design of Alternatives. Most decision problems discussed in the literature consider the set of alternatives on which they apply as "given", although we know that in practice frequently such a set needs to be constructed. There is little in the literature addressing this problem (see [START_REF] Belton | Muliple Criteria Decision Analysis: An Integrated Approach[END_REF] for a brief overview), despite the awareness of it (for example, [START_REF] Keeney | Value-Focused Thinking: A Path to Creative Decisionmaking[END_REF][START_REF] Goodwin | Decision Analysis for Management Judgment[END_REF][START_REF] Keller | Decision Problem Structuring: Generating Options[END_REF]. However, policy makers rarely come with established alternatives. Most of the policy cycle is about designing or constructing alternatives. Actually, most of "smart" policy making is about "innovative design" of "innovation policies", that [START_REF] Montibeller | Raising the bar: strategic multi-criteria decision analysis[END_REF]) is designing alternatives considered unconceivable at that moment in time (creative design). [START_REF] Simon | A behavioral model of rational choice[END_REF] had already discussed this cognitive activity in his seminal work, without providing operational and/or formal methods for addressing it. More recently [START_REF] Hatchuel | C-K design theory: An advanced formulation[END_REF] introduced C-K theory (C-K = concept-knowledge) as a general design theory opening a way to extending decision analysis by operationally addressing this issue which merits further exploration. There have also been suggestions for value-focused brainstormings of decision alternatives [START_REF] Keeney | Value-Focused Brainstorming[END_REF][START_REF] Montibeller | Structuring Resource Allocation Decisions: A Framework for Building Multi-criteria Portfolio Models with Area-grouped Options[END_REF], an approach which is resonant with [START_REF] Corner | Dynamic decision problem structuring[END_REF] dynamic decision problem structuring.

• Decision Aiding Practice. The great majority of research in our field concerns "theory". As emphasised in [START_REF] Tsoukiàs | On the concept of Decision Aiding Process[END_REF] it focuses on how decisions are "taken", underestimating both the theoretical and practical problems of how to support decision processes. Moreover, while the issue of introducing decision models within an organisational context has been discussed in the literature [START_REF] Nutt | The formulation processes and tactics used in organizational decision making[END_REF], very little has been done as far as inter-organisational contexts are concerned (see [START_REF] Munda | Social Multi-Criteria Evaluation for a Sustainable Economy[END_REF]. Given the specificities of the policy cycle, exploring the practical aspects of providing decision support remains a key research issue.

• Interdisciplinary research. The above mentioned research priorities clearly call for more interdisciplinary research engaging disciplines such as Artificial Intelligence, Computer Science, Sociology, Policy Analysis, Decision Analysis, and Cognitive Sciences. The reader will note that the references list of this paper already cites contributions from all such areas. Either Policy Analytics will emerge as a strong interdisciplinary research area or it will never succeed.

Concluding, we hope to have highlighted that Policy Analytics represents a key opportunity for the future of Decision Analysis, but poses challenges with implications for research, training and practice. Perhaps most importantly, it may provide an opportunity for how the world around us can be improved by decision analysts.

  manuals concerning the assessment and use of the European Social Fund 5 ; -the Green and Magenta Books of the UK Government 6 ; -the Public Policy Assessment Book of the UK Government 7 ; -the Italian law concerning the environmental assessment of Public Works 8 ; -the French law concerning the Technological Risk Assessment Plans 9 . These documents extend a well-established tradition in the USA related to the use of cost-benefit analysis on policy decisions (for an account of using CBA in the United States regulatory process see Shapiro and Morrall 2012; see also the World Bank manuals about CBA 10 ).

We use the term decision analyst in the broad sense of any operational research/management science professional.

www.google.co.uk/publicdata/directory

http://www.guardian.co.uk/news/datablog

http://ec.europa.eu/regional_policy/sources/docgener/guides/cost/guide2008_en.pdf

http://ec.europa.eu/regional_policy/sources/docgener/evaluation/evalsed/guide/index_en.htm

http://www.hm-treasury.gov.uk/d/green_book_complete.pdf; http://www.hm-treasury.gov.uk/d/magenta_book_combined.pdf

http://www.bis.gov.uk/assets/biscore/better-regulation/docs/i/11-1111-impact-assessment-guidance.pdf

http://www.minambiente.it/export/sites/default/archivio/biblioteca/dsa_linee_guida_vas_1999.pdf

http://www.installationsclassees.developpementdurable.gouv.fr/IMG/pdf/Guide_PPRT_tbd_complet.pdf

World Bank (2010) Cost-Benefit analysis in world bank projects. Washington DC

Supporting information is any argument supporting a claim. Legitimating information is any argument allowing a claim to be legitimated by the participants of a discussion.