Mark Hartswood
email: mark.hartswood@cs.ox.ac.uk

Marina Jirotka
email: marina.jirotka@cs.ox.ac.uk

SmartSociety: Collaboration between humans and machines, promises and perils

Keywords: Collaboration Between Humans and Machines, Promises and Perils SmartSociety, Responsible Research and Innovation, Panel Session, Techno-Social Visions, Brokerage

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

SmartSociety 1 is a 4 year EU funded project that is at the forefront of how digital technologies are transforming our lives. SmartSociety builds on existing trends towards increasingly closely coupled systems of people, devices, data and algorithms designed to guide people in their everyday activities. SmartSociety aims to leverage collectives of people and machines to provide Smart City services in ways that satisfy individual goals while simultaneously tackling societal challenges such as sustainability. It welcomes the increasing interconnectedness of our physical and digital existences to support new ways for people to collectively solve problems by connecting them to remote pools of expertise and resources possessed by participating humans and machines [START_REF]Social Collective Intelligence: Combining the Powers of Humans and Machines to Build a Smarter Society[END_REF].

SmartSociety aims to assist people in their everyday activities, while using the collective intelligence of the system as a whole to protect and preserve our shared resources. A traffic system enabled by SmartSociety would give advice to individuals on how to speedily complete their journey, while at the same time influencing overall flows of traffic to minimize pollution, reduce congestion, and reduce the impact of traffic incidents. Similarly, a SmartSociety tourism solution would utilize local knowledge to provide a customized experience for the individual user while at the same time smoothing the impact of tourism on the local infrastructure.

A SmartSociety keynote speech and panel session were included as part of the 2015 IFIP Summer School on Privacy and Identity Management2 with the aim of presenting the project's vision and eliciting feedback from the privacy community concerning privacy and other ethical challenges that the project faces. This approach of engagement forms part of the Responsible Research and Innovation agenda within SmartSociety, where issues pertaining to privacy and other social values are addressed as an integral part of realizing the project's goals. The panel session was recorded and transcribed so as the consortium could benefit fully from the discussion. An analysis of this transcript forms the main contribution of this chapter.

In this chapter we briefly introduce the emerging paradigm of Responsible Research and Innovation, and how this is being realized within SmartSociety. We recap some of the themes of SmartSociety that were featured in the keynote presentation under the rubric of the 'Promises and Perils' of a SmartSociety. Finally, we present an analysis of the panel session discussion, which touched on several key issues, including the relationships between technology and society, what we should expect from a 'SmartSociety', barriers and horizons in managing ethical issues, and brokerage as a methodological approach to weaving multiple perspectives into design.

Responsible Research and Innovation in SmartSociety

Responsible Research and Innovation (RRI) seeks to open up innovation processes so that they incorporate a broader range of perspectives of the techno-social futures that innovation may bring about. RRI aims to achieve this by coordinating a varied range of multi-level activities 3 undertaken by multiple actors at multiple points with the research / innovation lifecycles towards ensuring the outcomes of research and innovation are 'socially acceptable' and 'socially beneficial'. These may include traditional ethics processes, risk assessments and foresight procedures, as well as more innovative activities tailored to specific domains or research streams that emphasize multi-stakeholder involvement [START_REF] Stahl | From computer ethics to responsible research and innovation in ICT: The transition of reference discourses informing ethics-related research in information systems[END_REF].

RRI has been incorporated into funding bodies' research agendas. For the EU it is an integral part of the H2020 programme both as standalone actions as well as being integrated within other research themes 4 . In the UK, the Engineering and Physics Sciences Research Council (EPSRC) expects (but does not mandate) that RRI activities be built into research processes, offering the AREA 5 framework as guidance for researchers wishing to pursue an RRI approach. While the stances of the EU and UK differ in detail, they both place considerable emphasis on consultation and engagement across a broad range of publics as a central pillar of RRI.

An obvious quandary for RRI is how to decide which outcomes are actually socially beneficial -especially since innovation events themselves tend to alter our values and perspectives as a basis for judgment. Two proposals for solving this problem have been proposed. The first, promoted by von Schomberg, is to envisage RRI as encoding existing values enshrined within national or international charters, such as the European Union treaty, as providing "normative anchor points" for shared values such as the right to privacy, enhancing sustainability, promoting equality and so on [START_REF] Von Schomberg | A vision of responsible innovation[END_REF]. The second is less prescriptive of specific values and focuses instead on opening up spaces for reflection and dialogue where processes of value formation (including elements of consensus and conflict) can be played out [START_REF] Owen | Responsible research and innovation: From science in society to science for society, with society[END_REF]. Our approach draws on both of these proposals by following Brey [START_REF] Brey | Values in Technology and Disclosive Computer Ethics. The Cambridge Handbook of Information and Computer Ethics[END_REF] in framing issues in relation to values of democracy, fairness, autonomy and privacy. At the same time we treat each of these categories as a 'discursive space' where the implications of a SmartSociety for a given social values can be explored. This blended approach works especially well, since there is actually no single version of, for example, democracy and democratic values. Instead, many versions of democracy, democratic structures, processes and values are possible in different combinations that each has varying implications for participation and governance. Thus we may agree that democracy is important, but if we do so this is only a starting point for a more detailed conversation about what shape of democracy might be desirable or effective within Smart Societies.

To give a schematic overview of RRI, the following shows elements of RRI which have been proposed within the various sources cited above:

• [Upstream] Starting early in the research and innovation process.

• Anticipating the transformations and impacts of new products and processes.

• Giving a voice to multiple publics and stakeholders to explore the consequences of research and its desirability. Responsible Research and Innovation (RRI) was built in as an integral feature of the SmartSociety. The premise was that existing SmartSociety-like systems are often ad hoc and not as powerful as they could be, or else organized and powerful, yet poorly adaptive and often seemingly socially irresponsible. Examples include the emergent and entrenched asymmetries in power over personal data obtained though use of digital services such as Google and Facebook [START_REF] Lanier | Who owns the future?[END_REF]. SmartSociety aims to improve this situation by creating the tools to build more powerful systems that have considerations of ethics and social values 'built in'.

We have developed a dedicated RRI procedure for SmartSociety that we are currently writing up for a separate publication. In essence, this process has four steps [START_REF]Social Collective Intelligence: Combining the Powers of Humans and Machines to Build a Smarter Society[END_REF] Case studies and consultations to tease out issues with existing SmartSociety-type applications; (2) Synthesis of issues relevant to individual technical work-packages within the SmartSociety into a series of challenge documents that lays out the case study context and outlines the challenge; (3) Facilitating the technical work-package to respond to the challenge and (4) Structuring the responses into a series of project wide design guidelines and operating procedures. The IFIP Summer School keynote speech and workshop reported here forms part of step 1 in the above process. That is to say it has been a means of consulting the privacy research community about the ethics, privacy and social values challenges posed by SmartSociety. In the following section we spell out our consultation approach in greater detail.

Consultation approach

Part of the RRI process within SmartSociety has been to undertake a series of consultative exercises with stakeholders in a variety of domains to create a tapestry of varied perspectives on the SmartSociety vision and possible consequences of SmartSociety technologies. The aim has not been to undertake a systematic and exhaustive consultation -mainly because SmartSociety technologies and ideas are still forming and still quite fluid, and may be applied across a diverse range of application areas -but rather to enrich the project with a series of external perspectives that may not otherwise feature endogenously.

In the closing sessions of the 2015 IFIP Summer School on Privacy and Identity Management, we were given an opportunity to present to the members of the privacy community represented at the workshop one version of the SmartSociety vision, and to elicit feedback from that community.

The following format was used. A keynote speech was given by the RRI researcher which covered the following topics: (a) An overview of SmartSociety; (b) an overview of RRI. This included the screening of a video created as part of an earlier project (FRRIICT) that demonstrates the issues for RRI of innovating in ICT6 ; (c) the SmartSociety vision as portrayed by a cartoon movie of imagined participants in a SmartSociety using SmartSociety services; and (d) a presentation of some of the presumed ethical and societal issues posed by the SmartSociety vision, which were framed in terms of 'the perils and promises' of a SmartSociety.

After a short break, the presentation was followed by a panel session in which panelists gave their response to the presentation, and thereafter addressed questions raised by members of the audience. Prior to the event, the consent of panelists and the audience was obtained to audio-record the panel session discussion. An analysis of the recording forms the basis of this chapter, which will be also used internally within SmartSociety to raise social values and ethics related issues posed by the panel and the audience.

The four members of the panel consisted of (a) the leader of the one of the SmartSociety technical WPs; (b) a Social Scientist and Policy consultant; (c) a computer scientist and privacy advocate and (d) one of the co-authors of the SmartSociety proposal.

4

The promise and perils of a SmartSociety 7SmartSociety builds on seven technical work-packages each supplying a sociotechnical component that contributes an important capability in order to realize a SmartSociety system as a functioning whole. These are:

• Provenance and Trust Provenance is a data trail that supports audit. Trust mechanisms include reputations systems such as those commonly found on internet platforms.

• Sensor fusion Sensors in the environment, or worn by the user, are interpreted by computers to give computers access to a high-level description of what is happening within a SmartSociety system. • Peer profiles Data stores for information about people who are participating in a SmartSociety system. • Social Orchestration and algorithms Providing the mechanisms by which SmartSociety tasks are composed and the algorithms that support the activities of the participating collectives. • Incentives How to deliver incentives within a SmartSociety system to make participating more attractive and to direct the actions of the collective. • Programming framework To give the application programmer pro-gramming constructs that apply directly to social entities such as col-lectives. • Platform The infrastructure that ties the technical elements of a SmartSociety system into a cohesive whole.

Many familiar social web platforms, or cyber-social systems, such as Amazon or Uber (a controversial Ride Sharing platform), already include the key SmartSociety elements of reputation, incentives, algorithms and collectives etc. The SmartSociety vision is to provide better engineered components so that 'application developers' can rapidly build these classes of systems at will. Two advances unique to SmartSociety assist the engineering approach. One is the abstraction and modularization of key social web functionalities -e.g. reputation (e.g. ratings and reviews), incentives (e.g. badges and other rewards), embedded sensor systems (such as those present in phones and Google Glass) and user profiles. The second comprises three new technologies for combining the above components into new types of application. The first is a 'social orchestrator', that allows the specification and enactment of social activities such as negotiation. The second is a 'programming framework', which includes programming primitives that can be used to invoke collectives of people and/or machines and to give them the resources they need to undertake some task. The third is a platform that knits all of the above elements together.

In the SmartSociety keynote given at the IFIP workshop, the SmartSociety concept was demonstrated via a video 8 that was created within the project to show off a vision for the 'Ride Share' system called 'Smart Share'. Smart Share is an early demonstrator of SmartSociety capabilities, and ties together aspects of peer profiling, algorithms, reputation to enable collective sharing of individually owned resourcesnamely spare capacity in cars. Algorithms find matches between those wishing to travel. Peer profiles assert preferences to assist making a match. Reputation systems help establish trust. Incentives and 'gamification' mechanisms, such as 'badges', are used to encourage involvement.

The keynote was adapted from a talk given at the ICT Days meeting in Trento in 2014 9 . The aim, on that occasion, was also to be provocative and to spur discussion of the ethical issues relating to SmartSociety-like systems. The slides used for ICT Days were actually authored by technologists within the project, who were responding to their own interest in stimulating debate around ethical concerns.

The slides, aiming to be provocative, outlined the 'promises' and 'perils' of a future envisioned by SmartSociety. They achieved this by framing a series of utopias and dystopias -juxtaposing various promises of the SmartSociety project with various perils. One example is the promise of collective intelligence to solve previously unsolvable problems, versus the peril of assimilation within a collective where autonomy is erased through totalizing mechanisms of automated influence. The slides employed illustrations and imagery from popular culture that play on such fears and dystopian possibilities, including, for example, a reference to the Star Trek entity 'The Borg', which assimilates individuals into its 'hive mind' collective with the slogan "Resistance is futile".

The polarity of promise and peril was used as an alliterative device to high-light some of the issues, and should not be read as a perspective literally held by the slides authors' on each of the issues raised. Tables 1-4 below showed the various promises and perils juxtaposed within the presentation.

Promise Peril Control over personal data

Being able to specify who can access data and for what purposes.

Useless information

Incomplete and unrepresentative collective data sets because of opt-outs.

Anonymity

Privacy enhancing technologies may provide various ways of acting anonymously, but accountably within a system.

Unmanageable complexity

Leading to increased technical, organizational (e.g. trusted third parties) and social complexity.

Controlling disease

Collective pooling of data may be of huge social benefit.

All-knowing state

Amassed data also has a huge surveillance potential and people may experience temptations to use it beyond its original purposes.

Table 1. Privacy and governance

Promise

Peril Augmented users Benefiting from bodily monitoring and worn sensors.

Amplified data flows

New flows of personal data streaming out of any conceivable situation.

Context-based services

Providing advice based upon an interpretation of what a person is doing.

Invasive or intrusive

Misreading the context and interrupting at inopportune moments.

Proactively Acting on a person's behalf in helpful ways

Wrong data interpretation

Computers making mistakes that put people at a disadvantage.

Table 2. Augmented but self-determined users

Promise Peril Man-machine collaboration

Benefiting from machine intelligence that can check countless options.

Manipulation

'Silently' embedding agency and interests that shape a person's actions or capacity to act. Personalization Knowing a person well enough to give him/her what s/he wants.

Surveillance

Knowing a person too well and acting inappropriately on that knowledge.

Collective Intelligence

Benefiting from the wisdom of the crowd.

Humans as cheap labor

Exploiting the cloud and undermining traditional labor rights.

Table 3. Embedded algorithms

Promise

Peril Better health and care Critical mass for rare diseases or im-

Facebook replacing your social life

Virtual venues impoverishing the physi-proved coordination of professional and informal care.

cality of conventional activities

Smarter use of natural resources (energy, water)

Better management of the commons via collective management.

A terawatt of power to win Jeopardy

Increasing energy consumption of pervasive ICT infrastructures.

Knowledge economy

Greater independence and control over work.

Job destruction, eSlavery. Erosion of careers and deterioration of working conditions

Table 4. Collective People-Machine Intelligence

5

Interpretative approach

An anonymized transcript was made of the workshop recording as the basis for the analysis. A thematic analysis approach was used whereby emergent themes were identified and iteratively refined [START_REF] Guest | Applied Thematic Analysis[END_REF]. In addition, anonymized excerpts from the panel members' statements were presented in a data session involving wider members of the research group so as to elicit further interpretations of the material. The workshop approach is close to a focus group format where insights emerge discursively through the interplay of perspectives and experiences of the participants, yet it differs in that there are elements of performance and staging, and an interrogative style, which are typically absent from focus groups. The analysis undertaken resembles a continuation of this process, picking out the emergent themes, but also intertwining further perspectives available to the researchers who have the benefit of a lengthier reflection unavailable to the panelists who responded on the spot. Hopefully it is clear in the discussion below where the researcher's voice is more prominent. Hopefully, too, this voice is not perceived as being critical of participants or their views, but rather as taking those views as starting points for further deliberation. Inevitably, since the authors have a background in computer science, sociology and anthropology, as well as being deeply embedded within SmartSociety project as RRI researchers, their perspective on the issues raised in this chapter will reflect the perspectives, experiences and agendas of these a priori commitments.

6

Issues raised: managing ethical issues and social hazards of a SmartSociety.

This section details the issues raised by the panelists and the questions addressed to the panel by the workshop audience.

Panel members' responses

Our four panelists included two men and two women. Female pronouns are used throughout this chapter to help hide panelist's identities. In the first part of the above quote -technology is conceived as an instrument to bring about certain social outcomes that are desired by the sponsor of the technology (perhaps tackling what are more broadly agreed concerns). The second part of the quote acknowledges that, at the same time, technology carries social dangers through unwanted side effects, which necessitate regulation and oversight. The dilemma outlined is that of wanting to use technology to do helpful things but, at the same time, introducing all sorts of new complexities, some of which have negative implications.

"And the question is, which of these irresistible services and temptations that the data world offers are you going to turn down -which social processes are you going to stay out of -what is the price you pay for that -socially" Yet, building the appropriate values and safeguards into the system is hard, especially since people may be complicit in bringing about harmful side-effects, as it can be very difficult to resist data-based services and the social web, and the costs of opting out of these solutions may be high.

A first reading of these quotes suggests that they are describing a technology-led process directed by a technical elite that frames societal problems and enacts a vision of how computers can help. But we have to acknowledge that technologists are society members too (!) and bring their life-perspectives to bear and enact values that have currency and are shared more broadly in society, even if they may only be partially representative of the wider population. That is to say, RRI researchers should not make a knee-jerk assumption that the values held by technologists may not be more broadly shared. On the other hand, an RRI approach would advocate consultative processes to widen the perspectives that are drawn upon in creating new techno-social visions.

A second possibility is to read within these quotes an implicit distinction between technology on the one hand (which is prior to, and acts upon, society), and society on the other (which has to respond, or resist, or cope with technology). But this distinction is also hard to sustain because the technology itself emerges from existing ideas that form part of the cultural zeitgeist. This presumed distinction between technology / society turned up a number of times within the workshop dialogue and represents an important theme in this chapter.

Panelist SocSci. SocSci pointed out that becoming reconciled to technology has been an ever-present issue for humankind. She applauded the interdisciplinary approach within SmartSociety, and saw as valuable the internal dialogues that have been initiated in the project.

"We too [referring to the organization which she is affiliated] have worked with, if you want, instruments or approaches that have ranged from use of theatre, media, games, and have particularly placed an emphasis on dialogue -I'm think-ing about the 'Court of Ethics' which is a wonderful piece of drama, a play around robotics. This dialogue, about technology and society is as old as we are as human beings and I mean that as many ten thousands of years old ... I'm pretty sure there might have been debates about the perils of fire versus the warmth and the cooking benefits that it could bring -it wouldn't have surprised me if there were not a debate and dialogue as long ago as that."

It is patently true that such an ongoing dialogue exists, and has existed across various epochs, as well as being enacted at different levels and locations throughout society, and through different cultural forms (including theatre, as indicated above). Dialogue is also a hugely important component of RRI as well as being a vital mechanism by which cultures anticipate and come to terms with new techno-social eras.

We can extend SocSci's contribution to consider several further facets to dialogue that are important. One is that the presence of dialogue is not by itself a guarantor of beneficial outcomes, with much depending on how any given dialogue is geared into political processes. Another is that what is at stake, and what dialogues are possible and effective in any given historical epoch, may vary considerably. Habermas, for example, identified the coffee house culture of the Enlightenment as leading to the emergence of a 'public sphere', which had not existed in such a cohesive and egalitarian fashion prior to that moment in time [START_REF] Habermas | The structural transformation of the public sphere: An inquiry into a category of bourgeois society[END_REF]. A third is the hegemonic aspects of dialogue that determine the legitimacy of content, participants and venues within particular dialogical spheres. Fourth and finally, there is an issue of the implied separation between "technology and society" which carries over from the previous panelist, Tech. We might want to keep in mind how technology itself underpins and transforms dialogic possibilities, with the social web as an example of this.

Panelist PrivAdv. This panelist questioned more fundamentally the presumptions underpinning the concept of a SmartSociety, and posited an alternative set of values and meanings that may be attached to being 'Smart'. Do our technologies that promote convenience make us Smarter -or does 'Smartness' come instead from a focus on empowerment, education, participation in decisions and the capacity to opt out of the presumed benefits of digital living? Actually, the smartness of society may be reduced through technology, but improved through education and thoughtfulness: "So Smart is a buzzword, coming from SmartCards, SmartCars, Smart-Everything, and usually what used to be intelligent, or networked, or computer-aided or -supported -but what really is needed I think is a SmartSociety -smart people, educated people -and people who don't have to rely on technology." "So is SmartSociety now really something that is [heading?] to educated people who can join the discourse about SmartSociety, for example, who are better off than before, but not only because of convenience but also [because] of participating in decisions but also opting out, or refraining from cooperating with others?" Interestingly, when these views were further discussed within our research group (as part of this analysis), then opinion was divided between those who were more sympathetic to the panelist's position, and those who more wholeheartedly embraced the role of technology to help people to collectively share resources and solve problems. Those viewing the SmartSociety ambition in a positive light also rejected commonly suggested negative trajectories, such as job erosion or loss of autonomy. Discussion around the SmartSociety vision evidently contributes to a broader debate over how, collectively, we identify and wish to tackle contemporary social problems, and what forms of living we aspire to. Thus the panelist provides a helpful challenge to SmartSociety and asserts values that SmartSociety represents less well. This question of what values the SmartSociety project upholds is another recurrent one within this paper.

This panelist, picking up on the project's focus on SmartCities, also identified presumed values within the SmartSociety project about what is desirable about participating in city life -for example, that everyone should be enthusiastically sociable in every situation: "So I come from (place) and we are said to be very stubborn. I think that (place) should be the same. (It's not, I don't know.). Where it is, for many people, fully ok to live on your island and not to interact with so many people. Those who you are choosing to interact with, they are your really good friends really. So it takes a longer time, but then you choose, these are my small group of people I want to interact with, and not I'm choosing from a big list of people who want to talk about Jazz all the time in the car. Although it might be my interest to meet people at some part of the time, but not this is perhaps (the) first thing (that I wish to do). I don't think that (people from place) and others here are anti-social and don't want to socialize -it's more that there should also be the opportunity not be part of a SmartSociety which forces you to play according to those rules." Our interpretation of these comments is that PrivAdv is not referring to the privacy aspects of being sociable per se, but rather the presumption (visible from the keynote presentation) within SmartSociety solutions that sociability is always a desirable quality. One concrete way these comments have been figured into the project is within a 'tourism scenario' that is under development as part of demonstration of SmartSociety ideas and technologies. We have pointed out that tourists are a diverse group with some wishing to solitary, as opposed to sociable, experiences. These perspectives have also contributed to internal debates within the project concerning diversity, and in particular how far SmartSociety systems can cater for diverse sensibilities.

This panelist questioned the how Smart Society may interfere with the autonomy of its participants by the sorts mechanisms proposed to shape the behavior of participants towards global objectives: "for example, that I can go through Edinburgh without having a navigation that not only tells me where -what's the quickest way, but also, which already anticipates I where I am going or that I should go -or where I get the best vouchers for the 100th ride or so -so the incentives -the persuasion -in a direction where many people probably want to take it don't think that this may be manipulation -so not personalization but manipulation. And if you think about the interests and incentives of the stakeholders -big companies -their incentive has to be to maximize the money they make out of that. This is the reason for their existence. If they don't think about that, but only about how to empower peoplesmart people -making people Smart -that would be a different storyhowever, that's not the task of the company" Usefully she highlights the importance of ownership of a SmartSociety application, and how ownership plays a strong role in dictating whose interests it ultimately serves, and how the mechanisms of the platform may be the (perhaps silent) bearers of those interests. These comments have assisted us to develop a framework where we interpret hybrid systems (i.e. ones involving people and machines) by analyzing of how interests are represented and balanced within the system.

Panelist SmartSoc. This panelist re-grounded SmartSociety by proposing the serious role of counter-balancing the effect in the West of ever rising demands upon already overstretched care and medical systems. She suggested that SmartSociety-inspired approaches could increase the capacity of care systems by supporting local collectives of lay and professional carers, whilst simultaneously delivering improved outcomes for patients. Moreover, she contends that privacy and ethical issues are more easily solved when contained within local communities, but presumes that data can be contained at this level: "So, for example, in Sweden at the moment there are some hospitals where people with kidney disease -they go in, and they self-dialyze. They go in and they connect themselves up to the machines, and the dialysis work gets done when they want it to rather than when the hospital system wants it to. What we see when that happens is there is a huge amount of sharing of information be-cause there's a local context -the collective around self-dialysis -where the sharing of information, and the information adapts and changes to the circum-stances. What you also see is a huge reduction in cost to the hospital -50% less money needs spent on doing that dialysis -and the number of infections goes down -the number of infections and errors is much smaller in the self-dialysis community than in the professionally dialyzed community." "And for me, at any rate, when I start talking about the big story of privacy, for example, and data protection and so on -I think top-down is just foolish. Particularly for these kinds of situation. You have to do it bottom-up. And these things can be built in from the bottom. And we have to understand, actually, a much much more radical notion of what privacy by design is because actually the governance model, the local governance model, for how that information is interpreted and understood, has to be built into the development process -has to be built into the developer culture and the user culture, and those two overlap, and are built in from the bottom."

In the second quote the panelist is alluding to how 'top-down governance', such as via the application of privacy laws, still somehow fails to protect people in how their data is used by large institutions such as governments and corporations. The 'bottomup' model that is advocated instead posits community ownership and the co-design of technologies to secure data locally within these 'data communities'.

However, this is an artful re-framing of the SmartSociety vision that circumvents some of the issues raised or hinted at by prior speakers. In this version of a SmartSociety, factors previously seen as matters of preference and privilege (deciding to be sociable or not; having time and resources to be a tourist) are replaced in this new context by matters characterized by hard choices and necessity. By positioning SmartSociety within would generally be considered a worthy application context, and as solving a serious societal problem, it becomes much harder to formulate a critique. One way of regaining a critical stance is to point at how technological approaches to dealing with issues of care and caring often focus too narrowly on the mechanics of care, as opposed to the emotional, spiritual or social needs of a dependent person, e.g. [START_REF] Thieme | Enabling empathy in health and care: design methods and challenges[END_REF].

This panelist 'solved' some of the ethical and privacy concerns posed within a SmartSociety by including elements of co-production, and elements of community and individual control over personal data. Although this seems like an important strategy, it perhaps has the disadvantage that it repositions SmartSociety further away from the grander vision contained within the project's original grant proposal -which actually does propose 'internet-scale' systems. Perhaps many of the difficulties attached to SmartSociety arise because of the scale of the systems involved. While coproduction, and minimizing scale and scope, render the issues more manageable, in doing so does the original SmartSociety vision remain intact?

Questions to the panel

Following the panelists' responses roughly eight questions were put to the panel by audience members. Instead of going through each question sequentially, as we have done for the panelists' individual contributions, this section pulls out some recurrent themes that were identified in the questions and the panelists' responses.

In the main, questions focused less on SmartSociety and its facets, but more on general issues around ICT that have a social impact, and the difficulties inherent in tackling these issues. In the following sections we consider the problems the audience and panelists described with identifying and understanding issues and solutions, anticipating the problems that new technologies may bring about, and in embedding known solutions into policy processes. We then explore how one panelist's suggestion of brokerage provides a possible solution to these issues.

Horizons and barriers.

On several occasions discussion referred to several types of horizons and barriers that were perceived to constrain how the ethical social issues raised by SmartSociety may be addressed. 'Horizons' is taken to mean some limit to appreciation or perception form a given perspective -such as how far into the future a person can see, or how far their knowledge extends. One such horizon related to limits to our understanding: "What happens where the functions of a SmartSociety [are] so complex they are beyond the comprehension of most people? People are 'not smart enough' to make sense of the complex systems in which they are enmeshed."

"I think we have a fairly poor understanding of the informal processes that go on in these communities -and turning them into something that we can actively support with technology..." [Responding to an audience asking about horizontal processes that connect disparate communities.] The first quote above (from an audience member) expresses a concern that people may not understand the technical systems in which they participate, implying that the future may be only comprehensible to a small number of technical elites. The second quote (from a technologist) expresses the reciprocal concern that technologists themselves may lack knowledge of social processes from the perspective of being able to develop technologies to support those processes. Taken together, these statements raise the specter of technology proceeding without a full social understanding to create something that members of society themselves cannot comprehend. These concerns resonate with the earlier theme of a supposed separation between technology and society. Yet we know that technology and society are not separate in the ways that some of the panelists may have been construed as implying. As new sociotechnical systems emerge from a given socio-technical milieu, then both technologists and non-technologists will already have some grasp of each other's perspective, even if initially these shared perspectives may be limited. While horizons may exist at specific moments, over time, and through processes of social learning, the technology and its deeper historical roots or distant locales of production often become even more comprehensible, either through experience, education, through voluntary disclosure, or via processes of investigation and revelation. Similarly, there are very well established design approaches for opening up understandings of social processes and feeding them into system development, which are able to extend the designers' horizon of how certain social processes function.

Another is horizon is the future and our ability to anticipate outcomes:

"Lawmakers are in a much more difficult position … because this is information technology and it applies to all kinds of applications -it's very difficult for them to anticipate the consequences."

"Basically the changes we propose with big data and these kinds of analytics, say, in the long run, they will change society tremendously and we currently we cannot really anticipate how."

These statements express the paradox that in controlling the negative aspects of the techno-social we must act on what is essentially an unknowable future. This resembles Collingridge's dilemma that indicate at which point we are most in control and able to shape a technology, is precisely the point where we lack information about its consequences (see [START_REF] Croy | Collingridge and the control of educational computer technology[END_REF] for a discussion of Collingridge). However, the RRI literature draws a distinction between the possibility of anticipating, as opposed to the intracta-bility of prediction, and how anticipation and prediction may be conflated [START_REF] Guston | Understanding 'anticipatory governance[END_REF]. Thus, it may be hard (if not impossible) to predict precisely what future data and analytics will actually herald, but relatively easy to imagine (anticipate) alternative technosocial futures that involve them.

Perhaps these epistemological and temporal horizons, which are perceived to limit how ethical and privacy issues may be addressed, have little to do with absence of the necessary skills and knowledge from society as a whole. Locally, hazards and potential remedies may be well understood, but also this knowledge may be compartmentalized and hard to assimilate into new settings. Indeed, another barrier alluded to during the panel session relates precisely to this difficulty of embedding values and safeguards into processes and systems: "So even to get to the point where -the next stage -where you have to frame a procurement so that it is privacy-enhancing -well, how do you do that when you have no expertise about privacy-enhancing in people who are writing the procurement document and there are no products which can meet things that you might want to make mandatory requirements?" "So at the moment we are very far away from actually being able to say we can provide guarantees and safeguard that people are being treated fairly, and equitably, and that the values they care about are reflected in those systems." Barriers to assimilation were also identified in relation to political processes that were perceived as failing to ensure the appropriate policy and governance environment for emerging technologies: "-it's a debate that's a social debate -and what I've seen of this debate so far hasn't been very well informed when politicians are involved." ... "[politicians at a Smart City event] agreed on the common vision that having real-time data on every citizen and everything that is going on, and everything that every citizen wants, is for them the perfect democracy." "..but usually -politicians for example, … they have not (inaudible words?) been helpful in the last time [i.e. in recent times] -they also want to stay in the position -they may do something for long effect but very often it's more -well -that -they decide what is good for people and not that the people are part of that." An interesting question concerns how far these barriers and horizons represent hard, global constraints, and how much they are real, but only within a certain framing or context. Any given 'knowledge horizon' may only be local, and even seemingly 'hard' barriers, such as influencing policy makers, may be more tractable (for some) than the quotes above suggest. For example, the panel session discussion took an interesting turn when an audience member familiar with the lobbying process showed how this particular barrier could be overcome.

Brokerage. One audience member contrasted 'top down', 'bottom up' approaches with the idea of influencing a project "middle-sideways" through the process of brokerage: "About creating brokerage between different kinds of lateral developments ... and disseminating learning about good and bad approaches and so forth. That's a very creative role of brokerage."

In a sense this statement provides an answer to some of the issues identified as barriers and horizons above. To overcome barriers, or to extend horizons, one needs new knowledge, expertise, processes and routines. Often the relevant experience already exists, but is hard to access. Brokerage plays the role of connecting otherwise compartmentalized knowledge, and creating supportive circumstances for knowledge to flow and be assimilated into a new locale. In many ways brokerage is a key activity within RRI. It was actually part of what was happening in the panel session itself as the session created a space for reflection and cross-fertilization between people with different kinds of experience and expertise.

Brokerage was demonstrated by this specific audience member as she (in asking her question) directed SmartSociety's attention to the SWAMI project 10 that had previously considered ethical issues in relation to ambient Intelligence -a precursor of SmartSociety technologies. Another panel member recommended the work of Henry Mintzberg (e.g [START_REF] Mintzberg | Of strategies, deliberate and emergent[END_REF]), who's approach from the field of management science resonates with the brokerage approach.

Brokerage is visible in these simple acts of recommending and sharing references to reports or academic papers. The recommendation is as important as the actual reference, as a key challenge for brokerage includes establishing trust and relevance because of the vast array of perspectives available for people to draw upon.

The audience member recommending the brokerage approach did so in response to a point made in the keynote presentation that SmartSociety aims to address ethical issues by designing components in ways that avoid unethical outcomes. Brokerage was suggested as an alternative to this 'prior ethical design' as it was seen as better supporting elements of 'creativity' and 'spontaneity' that may be lacking in either 'top down' or 'bottom up' design approaches. Again, this resonates with ideas within RRI of the importance of engagement -with publics and other stakeholders -not only to encourage new lines of influence in technology formation, but also to foster the creativity that comes from the cross-fertilization of perspectives. Influencing politicians as an example of brokerage. Following several (mainly negative) comments about how politicians are unapproachable and how they fail to understand important techno-social issues, one audience member was moved to declare her own expertise in liaising with and informing politicians: "You need to understand what politicians do when they work -and when can you influence them and with what can you influence them. Basically they don't respond well to ethical issues because they ask you "who should do what"? Because that's the kind of things they can demand in their political arena. So you have to really translate also those issues about who should do what, and then what is the role of the governance in making the person do that. Timing is extremely important, so a politician, or at least a member of parliament, usually can only talk about things when it's on the political agenda." This audience member is evidently a skilled mediator on the topic of 'ethical concerns' and politicians. She offered a very different perspective to the exasperation expressed towards the political process documented in earlier quotes. Her contribution demystifies how political influence may be achieved, in particular, by revealing the rather mundane series of practical steps and pragmatic considerations involved in it. It is also another example of brokerage in action, where the venue / mechanism of the summer school panel session itself created opportunities for opening up otherwise compartmentalized knowledge.

Conclusions

SmartSociety carries with it its own vision, which is continuously undergoing evolution and redefinition to take on shapes and directions that are not always aligned across all project partners. This is a healthy state of affairs for a large, complex, cutting-edge research project. Indeed, the Summer School panel session made a significant contribution to further exploring what a SmartSociety might aim to be. Various versions of the SmartSociety vision were on show during the keynote talk, panel session and in the questions and answers that followed. Summer School attendees saw SmartSociety as part of digitally enhanced everyday living, as a solution to profound social problems, as being smarter through less engagement with technology, and more through learning and thoughtfulness. They saw the grand, allencompassing, societal level vision shine through the keynote presentation, as well as a more low-key reimagining of a localized community-level SmartSociety in the reply by one of the panelists. Whilst this contestation and exploration of the SmartSociety vision has positive aspects, there are downsides too, especially if the vision is so fluid that it proves always to be slippery and impossible to pin down. Indeed, the SmartSociety system itself has a chameleon-like ability to fit within practically any application context and, as it shifts between domains, the complexion of relevant values changes too. This leads to another problem whereby it becomes easy for advocates of SmartSociety to evade any given critique by giving it a new guise within an alternative setting. Also, different values come into play depending on scale and the application area, and repositioning SmartSociety solutions to operate over a smaller scale diffuses many of the ethical issues posed when it is articulated in terms of its grander ambitions. These observations suggest that, on the one hand, it could be easy for the SmartSociety project to deflect various ethical concerns by deftly repositioning itself but, on the other, that it is genuinely hard to pin down the complexities of values attached to the various guises of SmartSociety.

One way forward may be to explore actively how the SmartSociety vision fits within the field of existing and already deeply considered visions and critiques of techno-social futures. These may include ones that are more naturally allied to the SmartSociety vision, such as 'Social Physics' [START_REF] Pentland | Social Physics: How Good Ideas Spread-The Lessons from a New Science[END_REF], and others that may be more oppositional, such as Morozov's critique of 'Technological Solutionism' [START_REF] Morozov | To save everything, click here: The folly of technological solutionism[END_REF]. As part of this process, it would be useful to strive to articulate the possible combinations of values that may attach to SmartSociety in its different guises, to discover on which occasions they are complementary and able to co-exist, and when they are actually antagonistic or truly mutually exclusive. These measures may help the consortium to articulate SmartSociety values in a clearer, more contextualized way, as well as to stimulate reflection upon and refine which values it ultimately wishes the SmartSociety system to uphold.

There were several moments in the workshop discussion where arguments hinged around a distinction between technology and society, such as where technology may be seen as coming to the aid of societal problems, with society needing to respond to unwanted side-effects. Such issues were often couched in terms of barriers or horizons. While these are real and limiting in each individual case, their relative severity seems also to depend on the situation or context in which they are experienced. For example, one audience member, steeped in the mechanisms underpinning political processes, hardly felt this to be a barrier at all, whilst others experienced the political world as frustrating and troubling.

A key contribution came from an audience member who highlighted an important role for brokerage. Brokerage involves creating networks of connections and flows of experience and expertise between otherwise discrete and compartmentalized communities to stimulate creativity and gain answers to issues where approaches already exist 11 . Brokerage resonates strongly with the processes involved in RRI, which also seek to draw multiple perspectives into innovation processes. Brokerage may be a way of positioning RRI as a source of creativity and an enabler of innovation, as opposed to the perception that RRI sometimes attracts of seeking to constrain and regulate. If we shift our perspective from thinking of technology and society as being separate entities (where society has to cope with more and more varied technologies) and instead consider the direction of travel as towards weaving ever more dense technosocio hybrids, then to manage these trends effectively perhaps we do need processes similar to brokerage that help synthesize trans-disciplinary perspectives.

Finally, it is worth reflecting on the value of the panel session as a venue for stimulating debate. One aspect to highlight is that the discussion typically did not dwell on the implications of the core features of SmartSociety itself -such as algorithms, incentives, collective intelligence and so on, but rather focused on higher-level issues relating to the SmartSociety vision and the complexities of addressing ethical concerns more generally. Whilst these were useful discussions to have, in a future staging of the panel session we would consider adjusting the approach and reiterating key questions from the initial keynote to refocus any discussion around those core elements.

Overall, we believe that holding this workshop session was a valuable experience, both to enrich the summer school and to assist with the reflective process within SmartSociety project itself. We are grateful to the panelists and the summer school attendees for consenting to participate, and for their highly insightful and stimulating contributions.

  • [Midstream] Being responsive to the dialogue. • Adjusting the trajectory of research. • [Downstream] Creating the right policy and regulatory environment for the technology to emerge into.

  Distinct themes can be attributed to each of the panelists' responses. Panelist Tech (leader of a SmartSociety technical workpackage) raised the difficulty of providing guarantees that the interests of users are respected in SmartSociety-like systems. Panelist SocSci (a social scientist and policy consultant) drew attention to the ever-present dialogue around humans' ambivalent relationship to technology. Panelist PrivAdv (a computer scientist advocating the importance of privacy) raised fundamental questions over what the vision for a 'SmartSociety' might be. Finally, panelist SmartSoc (one of the original authors of the SmartSociety proposal) provided a context for SmartSociety's capacity to solve increasingly urgent social and economic problem around care. Therefore, two of the panelists were closely associated with the SmartSociety project, whereas the other two were not.Below we consider each of these contributions in more detail.

Panelist Tech. Tech drew attention to the way that computer systems have a social effect, opening up a thread of discussion in this paper about the different ways that technology may stand in relation to social processes: "What we have seen in the [SmartSociety] project is that constantly there are -there's a tension -between what you try and do with technology -because basically you want as much data -and you want to manipulate... you want to determine the outcome of social interactions with machine support so that you can introduce more intelligence into the system and help optimize -let's say help solve the travel problem in a city. And on the other hand, of course, you have the privacy concerns -more than just privacy, I think it's also accountability, transparency and governance -because what we've seen in our project is that all these algorithms essentially [they] all introduce biases."

http://www.smart-society-project.eu/

http://www.ifip-summerschool.org/

E.g. in the planning, conduct and dissemination of research; during the formulation of research programmes and by policy makers in anticipating the regulatory requirements of innovation emerging from research.

http://ec.europa.eu/programmes/horizon2020/en/h2020-section/responsible-researchinnovation

[START_REF] Brey | Values in Technology and Disclosive Computer Ethics. The Cambridge Handbook of Information and Computer Ethics[END_REF] The AREA acronym stands for: Anticipate, Reflect, Engage and Act. https://www.epsrc.ac.uk/research/framework/area/

https://www.youtube.com/channel/UCKITrA6PaVRkTsfdJtVP41w

This section draws on a presentation authored by the following members of the SmartSociety consortium which is used with their permission. Michael Rovatsos (University of Edinburgh), Daniele Moirandi (U-Hopper), Vincenzo Maltese (University of Trento), Ronald Ronald Chenu-Abente (University of Trento), Alethia Hume (University of Trento).

https://www.youtube.com/watch?v=SAhWCaCsXrA

http://2015.ictdays.it/en

Interestingly enough, brokerage could be couched in SmartSociety parlance as 'leveraging diversity in a collective social intelligence to create enriched hybrid perspectives'.

Acknowledgements

This work was supported by EU FP7 FET SmartSociety project (http://www.smartsociety-project.eu/) under the Grant agreement n.600854.


David Derler

Christian Hanser

Henrich C Pöhls

Daniel Slamanig

Towards Authenticity and Privacy Preserving Accountable Workflows

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

To efficiently handle frequently recurring processes within enterprises, it is advantageous to define standardized business processes. An ICT supported technical realization of a business process is usually denoted as a workflow [START_REF] Jablonski | On the complementarity of workflow management and business process modeling[END_REF]. Such a workflow can be seen as an abstract process, which defines a certain sequence of tasks as well as conditions on how participating entities have to complete these tasks. In such a context, workflows may span various departments within an enterprise or even various enterprises (interorganizational workflows).

To always have an overview of the current state of concrete workflow instances and to be able to react to deviations from the defined workflows, it is important to document each step and to report it to some entity. Thereby, an inherent requirement is that these reports allow to verify whether delegatees acted within their boundaries and that each task can be attributed to a certain delegatee. This shall hold true especially if the process is interorganizational. In addition, it is desireable to automatically derive information, e.g., to issue warnings if certain constraints in a workflow are not met. Furthermore, it is often required that documentations of certain workflows are retained in an unforgeable recording for auditing or legal purposes. For example, the European data protection law requires an organization to document the usage of data [START_REF] Bier | How usage control and provenance tracking get together -a data protection perspective[END_REF]. However, the boundaries within each participating entity can act in a workflow might already be a sensitive business internal. Hence, it should not be disclosed to other parties (e.g., other enterprises in interorganizational workflows). Thus, an additional requirement is that the defined boundaries are not revealed to entities verifying a report, i.e., to ensure privacy, while still being able to check whether delegatees acted within their boundaries. We stress that this goal is in contrast to confidentiality of task reports. In particular, privacy requires thateven when the task reports are available in plain-the defined boundaries are not recoverable.

A suitable application is outsourcing inter-or intra-enterprise workflows to some environment that is not under full control, e.g., to the cloud. An automated process outsourced to the cloud may then run on behalf of the participants to carry out a task within such a workflow. A participating enterprise will be interested in the correctness of the workflow, the compliance with associated privacy requirements and to hold the cloud accountable. We note that especially in context of accountability there are significant efforts to provide and standardize frameworks for cloud accountability, e.g., as demonstrated within the A4Cloud project [START_REF] Pearson | Accountability for cloud and other future internet services[END_REF]. We note that we are interested in a more abstract view on workflows and cryptographic tools that allow to realize the aforementioned requirements.

Related Work

Besides [START_REF] Montagut | Traceability and integrity of execution in distributed workflow management systems[END_REF][START_REF] Montagut | Enforcing integrity of execution in distributed workflow management systems[END_REF][START_REF] Lim | Workflow signatures for business process compliance[END_REF], not much attention has been paid to cryptographically enforcing certain properties of workflows. Subsequently, we review the existing approaches and other related concepts.

In [START_REF] Montagut | Traceability and integrity of execution in distributed workflow management systems[END_REF][START_REF] Montagut | Enforcing integrity of execution in distributed workflow management systems[END_REF], the authors investigate traceability and integrity aspects of decentralized interorganizational workflow executions. This work focuses on preserving authenticity and integrity with respect to logical relations (AND, OR, XOR) among certain tasks in a priori defined workflows, while the concrete agents executing the workflow tasks do not need to be pre-specified (these could be dynamically chosen with the help of some discovery service). To do so, they use policy-based cryptography [START_REF] Bagga | Policy-based cryptography and applications[END_REF], where every agent gets issued credentials from some central authorities (specifying attributes that the agent satisfies). Then, for each workflow step a policy defines what needs to be satisfied for the execution of the respective task (basically the required decryption keys can only be obtained if the policy is satisfied). In addition they use group signatures to guarantee anonymity of honest agents, but support traceability of malicious ones.

In contrast, [START_REF] Lim | Workflow signatures for business process compliance[END_REF] allows to dynamically define those workflows during the workflow execution. That is, they map the workflow to a (dynamically extendable) tree, where each node in the tree is interpreted as one particular workflow task. Then, building upon the hierarchical identity based signature scheme in [START_REF] Lim | Multi-key hierarchical identity-based signatures[END_REF], one can build a hierarchy of signing keys (i.e., each node in possession of a signing key can issue signing keys for its child nodes). These signing keys are then used to sign some task-dependent information and, due to the hierarchical nature of the underlying primitive, this delivers an authentic documentation of the workflow execution regarding the logical relations among subsequent tasks.

Orthogonal to our goals of authenticity, accountability and privacy, variants of attribute-based encryption were used for cryptographically ensured access control with respect to some policy in [START_REF] Paterson | Cryptography from pairings: A snapshot of current research[END_REF][START_REF] Al-Riyami | Escrow-free encryption supporting cryptographic workflow[END_REF][START_REF] Ferrara | Cryptographically enforced RBAC[END_REF]. Recent work [START_REF] Ferrara | Policy privacy in cryptographic access control[END_REF], thereby, also considers the possibility to hide the access policy.

Somehow close to our goal is [START_REF] Herkenhöner | Towards automated processing of the right of access in inter-organizational web service compositions[END_REF], but it does not target the enforcement of properties of workflows. However, the authors use malleable signatures to allow to remove (potentially confidential) information from signed data, while not influencing source authentication in service oriented architectures (SOAs). In their approach, workflow participants exchange signed data based on predecessorsuccessor relationships. This is not what we are looking for in this case.

Finally, the work done in this paper relates to data provenance, which deals with identifying the origins of data and also giving a record of the derivation [START_REF] Moreau | The provenance of electronic data[END_REF]. More precisely, this work relates to the aspect of process documentation found in data provenance, i.e., the proposed solutions will allow to verify whether a certain workflow was carried out as intended. This and other aspects of data provenance have been surveyed and studied in the literature, for example in [START_REF] Simmhan | A survey of data provenance in e-science[END_REF][START_REF] Moreau | Special issue: the first provenance challenge[END_REF][START_REF] Freire | Provenance for computational tasks: A survey[END_REF]. Our work may be considered as realizing some aspects of provenance with cryptographic guarantees, i.e., to ensure that any deviation from a planned workflow will be detectable and that each workflow participant can be held accountable for it's actions.

Motivation and Contribution

The few existing approaches to authenticity, accountability and privacy in workflows [START_REF] Montagut | Traceability and integrity of execution in distributed workflow management systems[END_REF][START_REF] Montagut | Enforcing integrity of execution in distributed workflow management systems[END_REF][START_REF] Lim | Workflow signatures for business process compliance[END_REF] rely on rather non-standard and often complex schemes. Given the importance of outsourcing computations and processes to cloud providers, it is thus an interesting challenge to look for simpler and more efficient solutions that rely on standard cryptographic primitives.

We propose two generic patterns to document the workflow executions, which can be instantiated using various different signature primitives. These patterns follow the well-known delegation-by-certificate approach from proxy signatures [START_REF] Mambo | Proxy signatures for delegating signing operation[END_REF], and-in contrast to existing solutions-allow to obtain particularly efficient schemes which only make use of standard cryptographic primitives with multiple efficient instantiations. In addition to existing work, which only considers tasks from an abstract point of view, we also consider the outputs of tasks and their corresponding documentation (reports). 3 In this context, we discuss means to predefine the structure of reports to ease an automated processing and also cover related privacy issues. We develop a set of requirements for workflow documentation systems and analyze possible instantiations of our generic patterns from different types of signature schemes with respect to these requirements. Finally, we discuss open problems and future directions.

Preliminaries

Throughout the paper we require the notion of digital signature schemes, which we recall subsequently. A digital signature scheme (DSS) is a triple (KeyGen, Sign, Verify) of efficient algorithms. Thereby, KeyGen is a probabilistic key generation algorithm that takes a security parameter κ ∈ N as input and outputs a secret (signing) key sk and a public (verification) key pk. Further, Sign is a (probabilistic) algorithm, which takes a message M ∈ {0, 1} * and a secret key sk as input, and outputs a signature σ. Finally, Verify is a deterministic algorithm, which takes a signature σ, a message M ∈ {0, 1} * and a public key pk as input, and outputs a single bit b ∈ {0, 1} indicating whether σ is a valid signature for M under pk.

A digital signature scheme is required to be correct, i.e., for all security parameters κ, all (sk, pk) generated by KeyGen and all M ∈ {0, 1} * one requires Verify(Sign(M, sk), M, pk) = 1. Additionally, for security one requires existential unforgeability under adaptively chosen-message attacks (EUF-CMA) [START_REF] Goldwasser | A digital signature scheme secure against adaptive chosen-message attacks[END_REF].

Workflow Model

In the following we align our notation largely with the one used in [START_REF] Lim | Workflow signatures for business process compliance[END_REF]. A workflow W comprises some central entity called the workflow manager (WM) who wants to outsource a workflow to some set A of entities denoted as agents. Thereby, every workflow can be decomposed into single atomic tasks t i ∈ T , where every task is executed by some agent. For instance, task t i ∈ T may be executed by agent A j ∈ A, which we denote by A j (t i ).

As it is common when modeling workflows (e.g., [START_REF]Information technology -Object Management Group Business Process Model and Notation[END_REF]), we define a workflow as a directed acyclic graph W = (T, E), where each vertex t i ∈ T represents one particular task and edges e j ∈ E ⊆ T × T represent task dependencies, i.e., a vertex (t u , t v ) ∈ E means that task t v follows after the completion of task t u . Now, we augment such a simple workflow by the following semantics and in the remainder of the paper we always mean such an augmented workflow when we speak of a workflow. Each vertex t i ∈ T with at least two outgoing edges (i.e., where outdegree deg + (t i ) ≥ 2) is called a split and each vertex t i with at least two incoming edges (i.e., where indegree deg -(t i ) ≥ 2)) is called a join. Each split and join is associated with a logical type {AND,OR,XOR}. In case of an AND split all edges are executed in parallel; in case of an XOR split exactly one edge must be executed; and in an OR split at least one edge needs to be executed. To illustrate this idea, we present an example of a simple workflow in Figure 1. For ease of presentation we label each outgoing edge with the respective type. To distinguish between successful and unsuccessful workflow executions, we need the notion of a trace. A trace τ of a workflow is a sequence of tasks in the order of their execution and a trace is called valid if it is compatible with the workflow. Let us look at the example in Figure 1. For instance, the trace τ

= (t 1 , t 2 , t 3 , t 5 , t 7 , t 8 , t 9 ) is invalid, but τ = (t 1 , t 2 , t 3 , t 4 , t 5 , t 7 , t 8 , t 9 ) is a valid trace.
Another issue that needs to be addressed is that not every agent may be allowed to execute every task. Consequently, we use assignment α(τ ) to denote the sequence indicating which agents have executed the respective task. For instance, we may have α(τ ) = (A 1 , A 2 , A 3 , A 4 , A 5 , A 6 , A 7 , A 8 ). Furthermore, either the WM may specify which potential set of agents is allowed to execute each task (static assignment) or each agent may dynamically decide which agents may execute the subsequent task(s) (dynamic assignment). In case of a static assignment, we call an assignment α(τ ) valid if it is compatible with the restrictions set by the WM. We note that our above notation deviates from the one in [START_REF] Lim | Workflow signatures for business process compliance[END_REF] who only consider dynamic assignments. Also, in contrast to [START_REF] Lim | Workflow signatures for business process compliance[END_REF] who solely look at the tasks in a workflow from a very abstract level, we are also interested in properties of the outputs of the tasks and thus get a bit more concrete. Therefore, we introduce the notion of the documentation of one particular task in a workflow and denote it as the report of a task, or report in short.

Subsequently, we introduce desirable properties for the documentation of workflow executions. Firstly, the most crucial requirement in our setting is that reports are protected against unauthorized modifications. Recall, that we do not consider the orthogonal feature of providing confidentiality for workflow data.

Requirement 1. The integrity of the reports needs to be ensured. Furthermore, it is required that only the workflow manager and the execution agent, which is actually performing a certain task, can produce a valid report.

Requirement 2. For a particular task t i , no one except the workflow manager and the agent(s) assigned to t i is/are capable of creating task reports that are accepted by an auditor.

In this context, it is also important that each report can be used to identify the respective execution agent (workflow manager), i.e., to ensure accountability. Requirement 3. The execution agent (workflow manager) that performs a certain task can be held accountable for its actions.

However, as long as the work is done correctly, a delegator might want to account for the work of a delegatee, while still being able to accuse the delegatee in case of a dispute. 4Requirement 4. One can not publicly verify whether a delegator or a delegatee created a certain report, while it is still possible to provide a proof assigning the task execution to one of the aforementioned parties.

In addition, it is desirable to automatically derive information, e.g., to issue warnings if certain constraints in a workflow are not met.

Requirement 5. Task reports allow to derive the order of the tasks in a certain workflow instance.

Bringing Signatures to Workflows

We can model workflows using the well-known delegation-by-certificate approach from proxy signatures [START_REF] Mambo | Proxy signatures for delegating signing operation[END_REF]. Subsequently, we describe two useful patterns.

Static assignment. Figure 2 illustrates the pattern for a statically assigned workflow. Here, the workflow manager computes a signature σ 0 on a sequence of (sets of) public keys PK together with the respective split/join operations. Then, for each task, (one of) the authorized agent(s) can sign the respective report using its secret key sk i corresponding to the public key pk i in PK. To be able to reconstruct the order of the task executions, agent A j also includes the signature(s) of the agent(s) executing the preceding tasks in its signature σ i . Dynamic assignment. Figure 3 describes the pattern for dynamically assigned workflows. In this approach, the workflow manager only delegates to the first agent within the workflow and the agents can further delegate the execution rights for subsequent tasks to subsequent agents.

Structuring Task Reports

Orthogonal to the requirement to ensure logical relations among tasks, it might also be interesting to automatically verify certain constraints regarding particular decisions upon execution of a task t i . For instance, it would be convenient to predefine certain sets of possible actions of an agent per task. As a simple realization one can think of a form containing several multiple-choice fields, where each multiple-choice field corresponds to a subtask of a specific task in a workflow. Then, an application monitoring reports can easily define constraints in Workflow manager (WM): setup workflow W = ({t1, t2, t3, t4}, {(t1, t2), (t1, t3), (t2, t4), (t3, t4)}) Workflow manager (WM): setup workflow W = ({t1, t2, t3}, {(t1, t2), (t2, t3)})

PK = pk 1 ||OP||{pk 2j } 2 j=1 ||pk 3 , σ0 ← Sign(PK, skWM)
PK = pk 1 , σ0 ← Sign(PK, skWM) Agent 1: A1(t1) r1, σ1 ← Sign(r1||pk 2 ||σ0, sk1) Agent 2: A2(t2) r2, σ2 ← Sign(r2||pk 3 ||σ1, sk2) Agent 3: A3(t3) r3, σ3 ← Sign(r3||σ2, sk3) PK, σ0
Setup Fig. 3. Pattern for dynamically assigned workflows. The tuples (ri, σi) denote the task reports corresponding to Ai(ti). For simplicity, we omit split/join (cf. Figure 2).

the fashion of: if Option A was chosen in Subtask 1.1 and Option B was chosen in Subtask 1.2 then issue a warning. If required, this can easily be extended to arbitrarily complex forms per task. Adding a structure to the task reports, suggests to introduce the following additional requirements. Requirement 6. It is possible to predefine the structure of task reports.

Besides addressing the structure of the report, allowing the delegatee to predefine sets of admissible choices for certain parts of task reports would help to improve the quality and help to automate the processing. Requirement 7. It is possible to predefine sets of admissible choices for certain fields in the task report.

However, such detailed workflow reports also impose privacy requirements, since it is crucial that business internals remain confidential, e.g., when reports are revealed for auditing purposes. Requirement 8. Task reports do not reveal additional information that is available to the delegator and/or the execution agent (e.g., the unused choices of the predefined sets of admissible replacements).

Instantiations

Using standard digital signatures, one can straightforwardly instantiate the patterns in Figure 2 and Figure 3. Subsequently, we revisit the instantiation of these patterns with other variants of digital signatures. We stress that we provide algorithmic descriptions for the schemes as we believe that this makes the presentation unambiguous and clearer than any informal textual description.

Append-only Signatures. Append only signatures [START_REF] Kiltz | Append-only signatures[END_REF] allow to publicly extend signed messages and to update the signature correspondingly. An append only signature scheme (AOS) is a tuple of efficient algorithms (Setup, Append, Verify), which are defined as follows:

Setup : On input of a security parameter κ, this algorithm outputs a keypair (sk, pk), where sk constitutes the signature on the empty message. Append : On input of a public key pk, a signature σ n-1 on a message (m 1 , . . . , m n-1 ), and a message m n , this algorithm outputs a signature σ n on the message (m 1 , . . . , m n ). Verify : On input of a public key pk, a signature σ and a message M = (m 1 , . . . , m n ), this algorithm outputs a bit b ∈ {0, 1}, indicating whether σ is valid.

For security, AOS are required to provide AOS-unforgeability under chosen message attacks. Informally this means that the only way of creating a valid signature of length n on a message M = (m 1 , . . . , m n ) is to extend a valid signature on message M = (m 1 , . . . , m n-1 ).

Application to Workflows: Using append-only signatures, the workflow manager creates a signature on the empty message and each agent can append its documentation. Due to their public-append capabilities, AOS are suited for unauthorized delegations, which only ensure the integrity of the signed reports.

Redactable Signatures. Informally, redactable signatures [START_REF] Johnson | Homomorphic signature schemes[END_REF][START_REF] Miyazaki | Digitally signed document sanitizing scheme with disclosure condition control[END_REF][START_REF] Steinfeld | Content extraction signatures[END_REF] allow to sign documents, where certain predefined parts can later be blacked out (or cloaked) without signer interaction and without invalidating the signature. A redactable signature scheme (RSS) is a tuple of efficient algorithms (KeyGen, Sign, Verify, Redact), which are defined as follows (using the notation of [START_REF] Derler | A general framework for redactable signatures and new constructions[END_REF]):

KeyGen : On input of a security parameter κ, this algorithm outputs a key-pair (sk, pk). Sign : On input of a secret key sk, a message M and admissible redactions ADM, this algorithm returns a message-signature pair (M, σ) (where ADM can be derived from σ). Verify : On input of a public key pk, a message M and a signature σ, this algorithm outputs a bit b ∈ {0, 1}, indicating the validity of σ. Redact : This algorithm takes a public key pk, a signature σ, a message M and modification instructions MOD, computes an updated signature σ and outputs an updated message signature pair (MOD(M ), σ ).

Essentially the redaction can be done by everyone, meaning that (1) the entity that performs the redaction is not accountable for the changes and ( 2) one is only able to black out certain document parts. For security, redactable signatures are required to be unforgeable and private.

Unforgeability captures the infeasibility to output a valid message signature pair (M, σ) without knowing sk, unless (M, σ) was obtained by redaction. Privacy requires it to be infeasible for every efficient adversary to reconstruct the redacted message parts, given the redacted message and its signature.

See [START_REF] Derler | A general framework for redactable signatures and new constructions[END_REF] for a formal security model. Besides these properties, the security model for RSS has been refined and extended several times. Firstly, [START_REF] Pöhls | Accountable redactable signatures[END_REF] introduced the notion of accountability, which requires that signers and redactors can be held accountable for their signatures/redactions. Secondly, [START_REF] Pöhls | Accountable redactable signatures[END_REF] and [START_REF] Camenisch | Composable & modular anonymous credentials: Definitions and practical constructions[END_REF] independently introduced unlinkability for RSS as an even stronger privacy notion.

Unlinkability essentially requires multiple redactions of the same document to be unlinkable. We, however, note that we do not further consider unlinkability here, since privacy already provides the required security guarantees in our context. We also mention that redactable signatures are related to the more general framework of P -homomorphic signatures [START_REF] Ahn | Computing on authenticated data[END_REF].

Application to Workflows: In context of workflows, RSS can be used in two different ways:

(1) One uses RSS in the same way as conventional DSS. Then, when it is required to publish reports (e.g., for auditing purposes) it can be useful to redact certain confidential parts of the reports. ( 2) Provided that all potential reports are known prior to designating a task to an agent, one could enumerate all variants of the reports and sign this list using an RSS. The agent then simply redacts-thus removes-all reports that are not required. While conventional RSS do not provide accountability in this setting, accountable RSS (ARSS) [START_REF] Pöhls | Accountable redactable signatures[END_REF] can be used to additionally provide accountability.

Sanitizable Signatures. Sanitizable signatures [START_REF] Ateniese | Sanitizable signatures[END_REF][START_REF] Brzuska | Security of sanitizable signatures revisited[END_REF][START_REF] Brzuska | Sanitizable signatures: How to partially delegate control for authenticated data[END_REF][START_REF] Brzuska | Unlinkability of sanitizable signatures[END_REF][START_REF] Brzuska | Non-interactive public accountability for sanitizable signatures[END_REF][START_REF] Brzuska | Efficient and perfectly unlinkable sanitizable signatures without group signatures[END_REF][START_REF] Pöhls | Sanitizable signatures in XML signatureperformance, mixing properties, and revisiting the property of transparency[END_REF] split messages in fixed and variable message parts and allow to issue signatures on them. A designated party (the sanitizer) is then able to modify the variable parts of the message without invalidating the signature. A sanitizable signature scheme (SSS) is a tuple of efficient algorithms (KeyGen sig , KeyGen san , Sign, Sanit, Verify, Proof, Judge). Subsequently, we recall the definitions from [START_REF] Brzuska | Security of sanitizable signatures revisited[END_REF]:

KeyGen sig : On input of a security parameter κ, this algorithm outputs a signer key-pair (sk sig , pk sig ). KeyGen san : On input of a security parameter κ, this algorithm outputs a sanitizer key-pair (sk san , pk san ). Sign : On input of a message M , corresponding admissible modifications ADM, the keypair of the signer (sk sig , pk sig ), as well as the verification key of the sanitizer pk san , this algorithm outputs a message-signature pair (M, σ), where it is assumed that ADM can be reconstructed from σ. Sanit : On input of a valid message-signature pair (M, σ), modification instructions MOD, some auxiliary information aux, the verification key of the signer pk sig and the secret key of the sanitizer sk san , this algorithm outputs an updated message signature pair (MOD(m), σ ) and ⊥ if the modification instructions are incompatible with ADM. Verify : On input of a message-signature pair (M, σ) and the verification keys of the signer pk sig and the sanitizer pk san , this algorithm outputs a bit b ∈ {0, 1} indicating whether σ is a valid signature on M . Proof : On input of a message-signature pair (M, σ), q message-signature pairs (M j , σ j ) q j=1 created by the signer, the keypair (sk sig , pk sig ) of the signer and the verification key of the sanitizer pk san , this algorithm outputs a proof π. Judge : On input of a message-signature pair (M, σ), the verification keys of the signer pk sig and the sanitizer pk san , and a valid a proof π, this algorithm outputs a bit b ∈ {sig, san}, indicating whether the respective signature was created by the signer or the sanitizer.

Subsequently, we informally discuss the security properties of sanitizable signatures (introduced in [START_REF] Ateniese | Sanitizable signatures[END_REF] and formalized in [START_REF] Brzuska | Security of sanitizable signatures revisited[END_REF]):

Unforgeability requires that only honest signers and sanitizers are able to produce valid signatures. Immutability requires that malicious sanitizers are not able to modify fixed message parts. Transparency requires that no one (except the signer and the sanitizer) can distinguish signatures of the signer from signatures of the sanitizer. Privacy requires that no one (except the signer and the sanitizer) can recover sanitized information. Signer-/Sanitizer-accountability Requires that no signer can falsely accuse a sanitizer of having created a certain signature and vice versa.

The above properties have seen some refinement and gradual extension since their formalization in [START_REF] Brzuska | Security of sanitizable signatures revisited[END_REF], e.g., by [START_REF] Klonowski | Extended sanitizable signatures[END_REF][START_REF] Gong | Fully-secure and practical sanitizable signatures[END_REF][START_REF] Canard | Sanitizable signatures with several signers and sanitizers[END_REF][START_REF] Brzuska | Non-interactive public accountability for sanitizable signatures[END_REF][START_REF] Brzuska | Efficient and perfectly unlinkable sanitizable signatures without group signatures[END_REF][START_REF] Pöhls | Accountable redactable signatures[END_REF][START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF].

In [START_REF] Klonowski | Extended sanitizable signatures[END_REF], among others, an extension that additionally allows to define sets of admissible replacements per message block (LimitSet) was introduced and later formalized in [START_REF] Canard | On extended sanitizable signature schemes[END_REF] (henceforth called extended sanitizable signatures or ESSS). Their formalization, however, does not require the sets of admissible modifications to remain concealed upon verification, and, thus, does not define privacy in the original sense. Thus, [START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF] introduced the notion of strong privacy, that additionally covers this requirement. In [START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF], it is also shown that ESSS providing strong privacy can be black-box constructed from every secure SSS in the model of [START_REF] Brzuska | Security of sanitizable signatures revisited[END_REF] and indistinguishable accumulators [START_REF] Derler | Revisiting cryptographic accumulators, additional properties and relations to other primitives[END_REF].

Orthogonal to that, [START_REF] Brzuska | Non-interactive public accountability for sanitizable signatures[END_REF] discusses that accountability can be modeled in two ways: non-interactive or interactive. The model presented above is tailored to interactive (non-public) accountability. In contrast, non-interactive (public) accountability requires that Judge works correctly on an empty proof π. 5 We emphasize that non-interactive accountability might be helpful in workflows, where the original signer can not be involved for certain reasons, e.g., efficiency.

Application to Workflows: By definition, SSS include a delegation mechanism, i.e., a signer grants a sanitizer permission to modify certain parts of a signed message without invalidating the signature. Thus, using this primitive, one can not only pre-specify the execution agent, but also the structure of the report. 6 In other words, SSS allow to split the report into several fields; then, according to the pre-defined workflow, one specifies which agent (i.e., by specifying the sanitizer) is allowed to put arbitrary content into certain fields of the report. In addition, SSS provide transparency, which is useful if it is required to hide whether a certain task was outsourced or not. In case of a dispute, the {Proof, Judge} algorithms still guarantee accountability. In case the additional level of privacy given by transparency is not needed, one can use non-interactively (publicly) accountable SSS, e.g., [START_REF] Brzuska | Non-interactive public accountability for sanitizable signatures[END_REF][START_REF] Brzuska | Efficient and perfectly unlinkable sanitizable signatures without group signatures[END_REF].

ESSS [START_REF] Canard | On extended sanitizable signature schemes[END_REF]: Extended sanitizable signatures, as defined in [START_REF] Canard | On extended sanitizable signature schemes[END_REF], extend SSS by the possibility to limit the admissible modifications per message block to sets of allowed messages. This allows for an even more fine grained definition of the report structure. However, the model of [START_REF] Canard | On extended sanitizable signature schemes[END_REF] does not require the unused choices in the sets of admissible modifications to remain hidden upon verification. While this extension eases the automatic processing of reports, the limited privacy features limit the practical applicability of this instantiation. ESSS [START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF]: Extended sanitizable signatures, as defined in [START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF], fix the aforementioned privacy problems, which, in turn, extends their applicability to workflow documentation systems.

Proxy Signatures. Proxy signature (PS) schemes, introduced in [START_REF] Mambo | Proxy signatures for delegating signing operation[END_REF] and formalized in [START_REF] Boldyreva | Secure proxy signature schemes for delegation of signing rights[END_REF] allow a delegator to delegate the signing rights for a certain message space M to a proxy. A proxy can then produce signatures for messages m ∈ M on behalf of the delegator. Subsequently, we recall the definitions from [START_REF] Boldyreva | Secure proxy signature schemes for delegation of signing rights[END_REF]:

(D, P) : The originator and the proxy jointly compute a delegation for the message space M as well as a proxy signing key skp. The originator runs D and outputs the delegation σ computed using its signing key sk i , whereas the proxy verifies the delegation and obtains the proxy signing key skp, which consists of its private signing key sk j and the originators delegation. Sign : This algorithm computes and outputs a proxy signature σ P for message m ∈ M using the proxy signing key skp. Verify : This algorithm verifies whether proxy signature σ P is a valid proxy signature for message m under pk j , delegated by pk i . On success, this algorithm outputs 1, and 0 otherwise. ID : This algorithm outputs the identity j of the proxy, when given a proxy signature σ P .

For security, proxy signatures are required to be unforgeable, which informally means that no one can produce valid signatures for messages m / ∈ M and only the designated proxy can produce valid signatures for m ∈ M. In [START_REF] Hanser | Warrant-hiding delegation-by-certificate proxy signature schemes[END_REF], the model was extended by introducing privacy, which essentially requires thatupon verification of a signature σ P on a message m ∈ M-the verifier learns nothing about M (except that m ∈ M). Signatures secure in this model are called warrant-hiding proxy signatures (WHPS). We note that proxy signatures are one instantiation of the more general concept of functional signatures [START_REF] Bellare | Policy-Based Signatures[END_REF][START_REF] Boyle | Functional Signatures and Pseudorandom Functions[END_REF].

Application to Workflows: Here, a delegator grants a proxy the signing rights for messages out of a certain message space M. This delegation mechanism can be used to predefine all possible reports and the executing agent only chooses the suitable report. In addition, WHPS additionally provide privacy with respect to the unused reports in the designated message space.

Blank Digital Signatures. Blank digital signatures, introduced in [START_REF] Hanser | Blank digital signatures[END_REF], allow an originator O to define and sign forms (so-called templates T ) consisting of fixed and exchangeable (multiple-choice) elements. These forms can then be filled in (instantiated) by a designated party (the proxy P). Upon verification, the verifier only learns the values chosen by the designated party. A blank digital signature scheme (BDSS) is a tuple of efficient algorithms (KeyGen, Sign, Verify T , Inst, Verify I ), which are introduced subsequently. Thereby, we assume that DSS signing keys for the originator (sk O , pk O ) and the proxy (sk P , pk P ) already exist.

KeyGen : On input of a security parameter κ and an upper bound for the template size t, this algorithm outputs public parameters pp. We assume pp to be an implicit input to all subsequent algorithms. Sign : On input of a template T , the signing key of the originator sk O and the verification key of the proxy pk P , this algorithm outputs a template signature σ T and a secret instantiation key sk T P for the proxy.

Verify T : On input of a template T , a template signature σ T , the instantiation key of the proxy sk T P and the public verification keys of the originator pk O and the proxy pk P , this algorithm outputs a bit b ∈ {0, 1}, indicating whether σ T is valid. Inst : On input of a template T , a template signature σ T , an instance M, the signing key sk P and the instantiation key sk T P of the proxy, this algorithm outputs an instance signature σ M on M if M is a valid instance of T and ⊥ otherwise. Verify I : On input of an instance M, an instance signature σ M and the verification keys of the originator pk O and the proxy pk P , this algorithm outputs a bit b ∈ {0, 1}, indicating whether σ M is valid.

The security requirements for BDSS are (informally) defined as follows:

Unforgeability requires that only the honest originator and proxy can create valid signatures. Immutability requires that even malicious proxies cannot create instance signatures for invalid instances M of T . Privacy requires that no one (except the proxy and the originator) can recover the unused choices for the exchangeable elements.

Application to Workflows: BDSS are-up to the missing transparency and accountability properties-similar to ESSS in [START_REF] Derler | Rethinking privacy for extended sanitizable signatures and a black-box construction of strongly private schemes[END_REF] and can, thus, be used for similar purposes. We note that all known instantiations of BDSS [START_REF] Hanser | Blank digital signatures[END_REF][START_REF] Derler | Privacy-enhancing proxy signatures from non-interactive anonymous credentials[END_REF] provide public accountability, since they require an explicit signature of the delegatee (proxy).

Comparison and Discussion

In Table 1, we bring the various possible instantiations discussed above into the context of the previously defined requirements, where we exclude naive instantiations. Depending on the used scheme, we can cover different subsets of previously posed requirements. While choosing a concrete instantiation always depends on the requirements, we note that ESSS and BDSS seem to be particularly well suited for the considered applications. Note that-mainly due to the imposed overhead-we do not consider naive solutions such as achieving Requirement 6 and 7 by enumerating all possible task reports. We again stress that the instantiations discussed in this paper are very simple and only make use of standard cryptographic primitives with multiple efficient instantiations. Thereby, we only require to assume the existence of some public key authority.

Outlook. In this paper we have discussed potential solutions for authentic and accountable, yet privacy maintaining documentation of outsourced workflows. While we, thereby, followed a rather high-level and informal approach, it would be interesting to model the desired security properties more formally (as for instance done in [START_REF] Lu | Secure provenance: the essential of bread and butter of data forensics in cloud computing[END_REF] for cloud provenance). Furthermore, it would be interesting to evaluate the practical value of our proposed solutions in a real world setting. Finally, we note that it seems to be straight forward to extend our approach by cryptographic access control solutions (e.g., [START_REF] Ferrara | Policy privacy in cryptographic access control[END_REF][START_REF] Ferrara | Cryptographically enforced RBAC[END_REF]) to restrict the access to task reports. We leave these points as future work.

Fig. 1 .

 1 Fig. 1. A simple workflow example.

Task 1 :Task 3 :Task 4 :Fig. 2 .

 1342 Fig. 2. Pattern for statically assigned workflows. The tuples (ri, σi) denote the task reports corresponding to Ai(ti). If OP = AND then σ ← σ2||σ3, if OP = OR then σ ← σ2, σ ← σ3, or σ ← σ2||σ3, if OP = XOR then σ ← σ2 or σ ← σ3.

Fig. 4 .

 4 Fig. 4. A simple task report for a task t1 = (t1.1, t1.2, t1.3), composed of three multiple choice elements.

Table 1 .

 1 Requirements covered by the respective instantiations. Legend: . . . supported, A . . . supported by ARSS[START_REF] Pöhls | Accountable redactable signatures[END_REF], † . . . if scheme is transparent

	Inst.	R1 R2 R3 R4 R5 R6 R7 R8
	DSS
	AOS
	RSS (1)
	RSS (2)	A A †
	SSS	†
	ESSS [15]	†
	ESSS [20]	†
	PS
	WHPS
	BDSS

The authors have been supported by EU H2020 project Prismacloud, grant agreement n • 644962.

This could also be interesting in the context of data provenance.

When following the paradigm in Figure2, the workflow manager is the delegator, whereas the agents are the delegatees. In contrast, following the paradigm in Figure3, agents act as both, delegatees and delegators, while only the first delegation is performed by the workflow manager.

Note that this obviously contradicts transparency, meaning that no scheme can be transparent and non-interactively accountable at the same time.

Using SSS supporting multiple sanitizers[START_REF] Canard | Sanitizable signatures with several signers and sanitizers[END_REF], one can even pre-specify multiple possible agents for a single task.