Laurent Dupont
email: l.dupont@univ-lorraine.fr

Joëlle Mastelic

Nathalie Nyffeler

Sophie Latrille

Eric Seulliet

LIVING LAB AS A SUPPORT TO TRUST FOR CO-CREATION OF VALUE: APPLICATION TO THE CONSUMER ENERGY MARKET

Keywords: Trust, Open Innovation, Living Lab, Co-creation of Value, Co-design O31, O33, L17

Open Innovation is widely explored, and many technologies are developed to support the involvement of stakeholders in its distributed co-design process, i.e. when actors work asynchronously and at a geographical distance. One of the fundamental parameters for the success of distributed collaborative approaches is the trust that the actors have in each other, in the current process and in technology. However, practitioners make little use of trust as a parameter for piloting and supporting co-creation of value. The lack of understanding of the mechanisms involved seems to explain this situation. Using a multiple-case-study analysis of co-design in the field of energy, this paper proposes to identify the levers in Living Lab favouring the trust between stakeholders. In addition to practical illustration, this paper provides a first co-design project management framework for practitioners, through the development of the "Co-coon Matrix".

INTRODUCTION -THE CHALLENGE OF CO-CREATION OF VALUE

Chesbrough (Chesbrough, 2003, p. 43) introduced and defined the concept of Open Innovation (OI): "Open Innovation means that valuable ideas can come from inside or outside the company and can go to market from inside or outside the company as well." OI is a company-centric paradigm, the aim of which is to open the boundaries of companies' R&D departments to the outside world. This concept and its related practices are now widespread in the socio-economic world.

In this context, the challenge is no longer to disseminate OI but rather to identify the levers to strengthen it, relying in particular on the areas linked to this flagship concept. Different evolving literature streams co-exist in parallel with OI, such as User Innovation (UI) or User-Driven Innovation, which we will address through Living Labs (LL) [START_REF] Schuurman | Bridging the gap between Open and User Innovation? Exploring the value of Living Labs as a means to structure user contribution and manage distributed innovation[END_REF]. User Innovation is clearly consumer-centric [START_REF] Von Hippel | Democratizing innovation[END_REF]. The aim is to co-design products and services with its users. As proposed by Schuurman, the Living Lab concept is at the intersection of OI and UI. To co-create value with the external world, the company's boundaries must be porous enough to facilitate the exchange of ideas, concepts or prototypes, whether it is an outside-in or insideout exchange. Tools and techniques provided by UI are used in OI contexts.

Within an organization, OI underlies a twofold process of openness. (1) The first level of openness concerns project teams in the upstream phase of innovation, whose members are expected to interact regularly in an iterative way with the end customer, or even the user of the product or service that is in the co-design process [START_REF] Sanders | Co-creation and the new landscapes of design[END_REF]. In addition, the organization itself must be part of an OI process requiring interaction between teams and all or part of the organization with its ecosystem (suppliers and economic, institutional and academic partners etc.) It is no longer a bilateral relationship between the company and its users but numerous interactions in a complex ecosystem of actors. In LLs, the model used to analyse a typology of actors is named the Quadruple Helix, composed of companies, public authorities, academics and citizens [START_REF] Arnkil | Exploring Quadruple Helix: Outlining user-oriented innovation models[END_REF]. Thus, the increased recognition of inventiveness and innovation as essential differentiating factors in an increasingly competitive world, the rise of the collaborative economy involving sustained exchanges between peers and the strong interest in introducing a principle of co-design into innovation projects strengthens, disseminates and even invites us to rethink the OI, interpersonal relations and governance.

In such a complex system [START_REF] Kurtz | The new dynamics of strategy: Sense-making in a complex and complicated world[END_REF], in order for the co-design process to function and develop in an optimal and fluid way, it is useful to identify what can stop or -on the contrary -stimulate the motivation of individuals to engage as co-designers or at least contributors with a process over which they seem to have no control. This paper focuses on the links between trust, the co-design process and co-creation of value. Indeed, research shows that trust is a key success factor in the development of organizations and the integration of users into OI processes, as we will see in the next section. In addition, [START_REF] Greco | Open innovation in the power & energy sector: Bringing together government policies, companies' interests, and academic essence[END_REF] highlight the potential contributions of OI to the energy sector with the key role of five variables: (1) government involvement, (2) university involvement, (3) customer and supplier involvement, (4) absorptive capacity and (5) innovation novelty. More specifically, in the world of energy distribution, co-design (for example, the design of new technology or new services) represents an opportunity for the development of customer relations and value co-creation. Numerous benefits of co-design have been mentioned by (Steen et al., 2011, p. 58), who categorize it in four groups: "Improving idea generation, improving the service, improving project management and improving longer-term effects". The challenge is to put the user at the centre of energy innovation, at the beginning of the value chain, in trades and practices that are more oriented towards technological innovation [START_REF] Dupont | Collaborative Innovation Projects Engaging Open Communities: a Case Study on Emerging Challenges[END_REF]. This evolution of the context offers the opportunity to understand, and even measure, as soon as possible in the arrival of new practices, the contribution of user integration to the innovation process in the field of energy services. Given the novelty of this field, as described in the research design part, we have decided to adopt a research approach based on multiple-case-study analysis. Thus, through two energyrelated case studies, we will illustrate how trust is a major issue, particularly in situations where projects are based on research by a wide diversity of actors, including users, in both quantity and quality (i.e. fields of expertise, focus of interest). The discussion part proposes to highlight and articulate specific Living Lab functions that promote trust in the co-design process to support the co-creation of value. The challenge is to verify that an idea is appropriate or rejected for what it is, and not because of the co-design process followed, the technology used to bring it out and represent it or because of the initial actor who carries it. Through the outline of a functional specification [START_REF] Pohl | Requirements engineering: fundamentals, principles, and techniques[END_REF], we will more specifically highlight the characteristics that a co-design process must offer to stakeholders.

THE CHALLENGE OF TRUST FOR INNOVATION

Trust: an essential lever for inter-and intra-company innovations

As already explained by Koschatzky (2001, p. 6), "firms which do not cooperate and which do not exchange knowledge reduce their knowledge base on a long-term basis and lose the ability to enter into exchange relations with other firms and organizations". This observation is now widely shared by the OI scientific community. Nevertheless, the quality of this cooperation remains a challenge for companies to increase innovativeness and further reduce time to market. Working on the robustness of the process can be beneficial, for example by ensuring that it is more in line with users' needs. The co-design process can also become more eco-responsible by mobilizing processes that respect the principles of sustainability by favouring short channels, socially responsible choices, the well-being of employees, etc. One of the common denominators of all these areas for improvement is trust. It is found at all levels of an organization and can be a lever for development: trust between peers in a community of codesigners, trust between companies and external contributors, trust between a company and its employees, trust between partners, trust in the methods and tools underlying the innovation process. Trust contains three dimensions [START_REF] Mayer | an Integrative Model of Organizational Trust[END_REF]: (1) benevolence (the perception of a positive orientation from one person towards another); (2) ability (group of skills, knowledge, know-how in a particular domain); (3) integrity ("relates to the perception that the other party adheres to a set of principles and values that the trustor finds acceptable, such as delivering on promises" [START_REF] Shazi | Trust as a predictor of innovation network ties in project teams[END_REF]). Taking these principles into account, it can be seen that several levels of maturity in terms of trust can be observed and developed within companies and their partnership relations. Thus, researchers propose a model that ranges from "limited trust" to "collaborative trust" [START_REF] Fawcett | Supply chain trust: The catalyst for collaborative innovation[END_REF]. Within companies, the research of Pirola-Merlo (2010), using West's "innovative team climate" model, shows that there seems to be a positive correlation between a team's climate and the speed of completion of a research and development project. Four dimensions characterize this climate [START_REF] West | The social psychology of innovation in groups[END_REF]: (1) the sharing of clear and valued objectives; (2) a non-threatening environment where members can influence discussions and decisions; (3) the pursuit of excellence through quality work and critical evaluation; (4) the valuing of innovation and supporting of working practices to achieve innovation. [START_REF] West | Flourishing in teams: Developing creativity and innovation[END_REF] recall that some authors consider that learning and innovation can only take place when group members trust the intentions of other members.

In 2014, a study conducted on a panel of 48 member companies of the German Maintenance Services Association showed that three elements are needed to achieve a medium to high level of innovation: contracts, trust in a supplier to respect collaboration and ability to fulfil the obligations agreed between the parties [START_REF] Van Der | When are contracts and trust necessary for innovation in buyer-supplier relationships? A Necessary Condition Analysis[END_REF]. This same study showed that in order to reach certain levels of innovation, certain thresholds must be respected for each of these three parameters. Under certain conditions, innovation performance depends very directly on the level of detail of contracts and the quality of trust established between partners. At the enterprise level, the establishment and maintenance of trust-based relationships in B2B reduces risks, transaction costs and long-term relationships [START_REF] Dovey | The role of trust in innovation[END_REF]. This is particularly true for innovation and creativity processes [START_REF] Shazi | Trust as a predictor of innovation network ties in project teams[END_REF]. Innovation requires creativity and a certain amount of risk taking, so the results are uncertain and unpredictable, making it impossible to draft precise terms and clauses of a contract in advance. More flexible mechanisms are needed to guide a partnership or collaboration, in which it is necessary to balance trust between partners and the establishment of contracts. In addition, a 2011 survey [START_REF] Wang | The impact of trust and contract on innovation performance: The moderating role of environmental uncertainty[END_REF] of 315 Chinese companies shows that while, in certain circumstances, contracts and trust can replace each other and guarantee the same performance, in situations of uncertainty trust between the parties is much more effective than the (costly) implementation of contracts. Managing an OI project requires the ability to understand and anticipate the environment in which the project will be deployed. Furthermore, researchers underline the role of trust at the macro level. Indeed, they demonstrate that societal trust promotes firm innovative efficiency in an open innovation context [START_REF] Brockman | Societal trust and open innovation[END_REF]. Plus, according to Nestle et al., "trust turns out to be a significant facilitator of open innovation cultures" in firms of a German high-tech cluster (Nestle et al., 2018, p. 7). This research on systems of innovation suggests that adequate infrastructure supported by local governments and a dedicated cluster manager could promote network activities and establish mutual trust between actors of firms. Finally, the literature highlights the importance of addressing trust to promote business innovation processes, the culture of open innovation, and potential economic dynamism. At the same time, the state of the art shows that little work is being done on this topic. Thus, the relationship between users, trust and open innovation would benefit from being better studied and understood to improve the process of co-creation of value.

Trust at the heart of user engagement and collaboration in Living Lab

Living Lab is a recent phenomenon. Research on Living Labs still needs structuration and an "Organizing Proposition", which is true of all newly researched phenomena, as mentioned by Davis in his seminal paper, "That's interesting" [START_REF] Davis | That's Interesting! Towards a Phenomenology of Sociology and a Sociology of Phenomenology[END_REF]. In 2015, Schuurman proposed a structuration different from Pallot et al. [START_REF] Pallot | Living Lab research landscape: from user-centred design and user experience towards user co-creation[END_REF] LL research landscape, with an interesting model separating three layers: the macro, meso and micro level, as shown in Table 1. The macro level of Table 1 refers to the "Open Innovation" stream, with the constellation of actors in a public-private-people partnership. The meso level represents the innovation projects. The micro level refers to the "User innovation" stream and is composed of the LL methodology. Definitions of LLs vary, as the research is still emerging to study the phenomena. Mastelic proposes the following definition of Living Labs in her doctoral thesis on the subject (Mastelic, 2019, p. 58). A Living Lab is an innovation intermediary, which orchestrates an ecosystem of actors in a specific region. Its goal is to co-design products and services in an iterative way, with key stakeholders in a public-private-people partnership and in a real-life setting. One of the outcomes of this co-design process is the co-creation of social value (benefit). To achieve its objectives, the Living Lab mobilizes existing innovation tools or develops new innovation tools. Without a co-design process, the phenomena cannot be termed "Living Lab". Observational studies can be mobilized to better understand the social practices but, by themselves, they cannot be considered as innovation tools used in LLs. Without the Quadruple Helix represented in the partnership [START_REF] Arnkil | Exploring Quadruple Helix: Outlining user-oriented innovation models[END_REF], a project is not considered a Living Lab. Public authorities, academics, private companies and citizens must be part of the ecosystem of actors that co-design products and services. Each actor has a specific role and does not participate in each phase of the co-design process (idea, concept, prototype, product); a selected group of actors are involved, depending on the goals of each step, coordinated by the LL manager. The notions of trust and contract are strong issues in LL. Indeed, LLs as innovation networks require the establishment of strong links of cooperation and collaboration. In the context of LL, this cooperation is done with the users who are at the heart of the projects. From experience, [START_REF] Guidat | Lorraine Smart Cities Living Lab: white paper on living labs[END_REF][START_REF] Dupont | Study case: Living Lab Mode for urban project design: Emergence of an ad hoc methodology through collaborative innovation[END_REF]Dupont et al., , 2015;;Dupont et al., 2015;[START_REF] Mastelic | Energy management in a public building: A case study co-designing the building energy management system[END_REF], we can point out that projects in LL mode induce tacit contracts between participating users, project leaders and potential method leaders without being able to guarantee the intellectual (and/or industrial) property of what could be produced. In addition, the mobilization of actors is generally based on mutual trust, which must be built and nurtured. We suggest transferring the work of [START_REF] Shazi | Trust as a predictor of innovation network ties in project teams[END_REF] on innovation networks within business projects to LL projects. Thus, we recognize that users bring different skills depending on the subject and the time of mobilization (diagnosis, idea generation, feedback, etc.) to provide expertise and know-how useful for innovation. On the other hand, in all cases, their benevolence is required and necessary. It remains a nodal point in the relationship being built. Research shows that integrity is also a fundamental characteristic. A competent employee who lacks integrity will be actively avoided in the setting up and development of innovation projects. Our experience with LL projects has confronted us with some recurring questions from user participants in relation to project sponsors: What will be done with our contributions? Do we serve the general interest or a particular interest? Will our expression be respected? Conversely, project leaders who adopt a LL approach aimed at involving users in their approach sometimes wonder about the integrity of the actors who will join them: will they respect a certain confidentiality if it is required? Will they be in line with our values? Will they be "sincere" or "authentic"? In practice, since it is impossible to verify a priori the full interoperability between participants, facilitators and project leaders, it is necessary to accept that actors withdraw or are excluded during the process.

Co-creation of value and trust: forgotten by collaborative environments?

What do we mean by co-creation of value? When a company decides to open the boundaries of its R&D department to the outside world (OI), and to put in place a co-design process with its users (UI), the goal of this process is to co-create value. We see the co-creation of value as the outcome of a co-design process. As Vargo and Lusch express it, there is no value created until the service is consumed [START_REF] Vargo | Evolving to a New Dominant Logic for Marketing[END_REF]. The consumer is always a co-creator of value. In a co-design process, the user becomes a co-designer and is integrated at the beginning of the value chain, in the "fuzzy front end" [START_REF] Sanders | Co-creation and the new landscapes of design[END_REF]. A study conducted in 2018 on collaborative immersive environments [START_REF] Dupont | A Universal Framework For Systemizing the Evaluation of Immersive And Collaborative Performance[END_REF] provides interesting insights into the notion of trust and co-creation of value. Collaborative immersive environments include tools, methods, digital systems (from Web 2.0 to immersive devices), or even physical spaces that allow and facilitate the work of groups composed of eclectic people, distributed or not, in synchronous or asynchronous work [START_REF] Recker | How collaborative technology supports cognitive processes in collaborative process modeling: A capabilities-gains-outcome model[END_REF][START_REF] Tran | Quelle contribution des technologies collaboratives à la configuration des organisations ?[END_REF][START_REF] Bayrak | Identifying collaborative technology impact areas[END_REF][START_REF] Dupont | A Universal Framework For Systemizing the Evaluation of Immersive And Collaborative Performance[END_REF]. The analysis, based on 23 scientific articles published since 2000 in international journals and one thesis, highlighted four recurring dimensions for Collaborative eXperience: (1) Sense Making; (2) Trust Building; (3) Shared Meanings; (4) Mutual Understanding. This study also identified two properties by dimension, respectively: context understanding and relevance; interpersonal relationships and trust; knowledge sharing and knowledge creation; group dynamics and collective intelligence. Dupont et al. [START_REF] Dupont | A Universal Framework For Systemizing the Evaluation of Immersive And Collaborative Performance[END_REF] reveals that two properties are not very well researched: trust and knowledge creation. The small number of studies on collaborative immersive environments that focus on these properties raises questions about the importance given to concepts that are at the heart of collaborative processes, which are by nature open and by nature part of a knowledge creation process, consubstantial with the innovation process.

The various elements from the literature underline the fundamental nature of trust for each member of a company, for example through the belief in his or her own abilities. Trust between the members of a company is also necessary to ensure balanced and profitable relationships. Trust must therefore be found between the hierarchical and functional levels. Finally, loyalty is an essential factor in an organization's relations with its partners and other stakeholders in its environment, particularly when it comes to involving users in the process. More generally, it is also a question of having confidence in your market and your ability to adapt to more or less predictable changes. Despite the relevance and clarity of these findings, it still seems difficult to establish and maintain relationships of trust. Although context and technology can respond to a number of challenges on an ad hoc basis, there are still obstacles to overcome in order to strengthen trust at the different levels we have been able to identify. It seems necessary to understand and equip the processes of the upstream phase of innovation to ensure an OI process that fills the gap.

RESEARCH DESIGN

Multiple-case studies from longitudinal exploratory research

In this article, we adopt a multiple-case-study approach [START_REF] Yin | Case study research and applications Design and methods[END_REF]. Indeed, we focus on contemporary events over which we have no control and our research questions are more explanatory and deal with the tracing of operational processes over time. In fact, we want to know "how" and "why" our living labs had worked (or could work better). In particular, we focus on the "trust" variable in two Living Lab projects. Thus, this method allows us to shed light on the question of trust through the complementary analysis of two case studies of OI applied to the field of energy services for two European countries (Switzerland and France), both of which have been the subject of a Living Lab initiative over a relatively similar period. Fell proposes the following definition: "Energy services are those functions performed using energy which are means to obtain or facilitate desired end services or states" (Fell, 2017, p. 137). The innovative nature of the deployment of OI approaches for companies in this industrial sector encourages us to capitalize and analyse concrete experiences to develop and characterize transposable methodologies. This work also allows us to provide additional analyses to the research of Greco, Locatelli and Lisi [START_REF] Greco | Open innovation in the power & energy sector: Bringing together government policies, companies' interests, and academic essence[END_REF] who deplore the small number of studies on OI for companies in this sector. Europe is putting in place an energy transition based on two main pillars: (1) energy efficiency and (2) the development of renewable energies. 40% of the energy consumption is related to the building sector in Europe (European Commission, 2013). Ambitious goals are set at the European level: "by 31 December 2020, all new buildings are nearly zero-energy buildings" (European Commission, 2013). The energy sector is still traditional and focused mainly on technological innovation. Cases of OI in the sector are emerging, closely linked to Smart City initiatives in Europe. In the LL network, for example ENoLL, actors are developing action research projects such as the Lorraine Fab Living Lab ® and the Energy Living Lab. In this research, we use three data collection sources: (1) individual perceptions from researchers; (2) individual and collective behaviours and attitudes observed during the project; (3) process outcomes (how and why it works). The data collection used different sources of evidence: (1) documentation such as research papers or reports from the projects; (2) archival records as capitalized ideas; (3) direct observations from researchers; (4) participant-observation as researchers involved as makers; (5) physical artefacts as well as digital and physical platforms. The first case presents part of the results of a longitudinal exploratory study conducted during the creation of the Energy Living Lab (ELL) in Switzerland, whose challenge is to create an innovation ecosystem to promote the co-design of energy services for one of the 700 energy distributors. The second case presents the French project Link by Makers (LbM) of the University of Lorraine. This project, driven by academics and makers (i.e. members of a FabLab), received the support of ENEDIS, the French electricity distributor, and Université de Lorraine (Chair REVES project), as a way to experiment with some form of OI in connection with the smart meter installed by the distributor who manages 95% of the French grid. Two different teams of researchers designed these cases independently. Nevertheless, they adopted the same living lab approach on the same topic. The first team, involved in a Swiss LL, conducts research on marketing and innovation management. The second, engaged in a French LL, does research on industrial engineering and innovation management. The two case studies that we will detail below emerged in different local contexts, but in both cases the researchers considered that the stakeholders concerned in each situation trusted each other. They also assumed that users trusted the process set up by academics to generate a real co-design dynamic. The first work presented independently validates the ability of processes to generate co-creation of value. The cross-referenced analysis of the two teams' feedback on the implementation and management of these two cases allows us to return to the premise of the trust placed a priori in the process by the stakeholders. The latter is not automatic. Barriers to change can be avoided. Good practices have been identified to establish a climate of trust between stakeholders in an innovation process that is open over periods of six to eighteen months. Finally, it seems possible to use technological levers to accelerate this innovation process and guarantee its agile nature.

Case study 1: The Energy Living Lab, lack of trust as a limit to the co-design of energy services

This case study describes one of the first projects of the Energy Living Lab. It began in autumn 2013 and lasted two years, in a context of transition in the Swiss energy sector. Indeed, during this period, distributors will move in the near future from a monopolistic status in their jurisdiction, with captive customers, to a competitive context, in which customers are free to choose their energy distributor. The challenge of this ELL project is to use co-design to develop energy services and co-create value for the partner company. The method followed is structured in four steps: (1) the formulation of the challenge, (2) the co-creation platform (www.ibrain.ch), (3) the target audiences, (4) the nature of the data collected, as well as the successive steps of data processing and analysis. This case has been described in a previous paper [START_REF] Dupont | Confiance et Technologie : Deux Dimensions de l'Innovation Ouverte et Agile appliquée à l'Énergie[END_REF] and Figure 1 shows the co-design process and its outcome (co-creation of value) stage by stage, supported by specific technologies. Users' interest in co-creation of value was measured by the number of ideas submitted by users in the different contexts (physical and virtual interfaces). The number of responses desired by the economic partner was achieved, but it was necessary to switch from the virtual interface to a physical interface in order to stimulate face-to-face users (see Table 2). The challenge presented had to be explained during these interactions, which could suggest that the question was too technical or complex, and that it was not appropriated by users, which could have been a barrier to participation via the virtual interface. The research team hoped that the participation rate through virtual interfaces would be higher. Their confidence in the ability of users to address this issue individually and virtually was overstated. The ideas selected by the company nevertheless raise several technical, legal or operational issues and the company carries out further analyses to confirm their feasibility and, if necessary, the implementation modalities. It should be stressed, however, that no ideas proposed were considered to be really new and many of them were redundant, which is not surprising in an idea generation phase where the aim is to understand the latent needs of users. These raw ideas must then continue a long way in the innovation process to be transformed into concepts, then into prototypes and finally into services. The critical phase is to continue the design of the service after this ideation phase within the company, at which point the risk of the "not invented here syndrome" [START_REF] Chesbrough | Bringing open innovation to services[END_REF] may appear, which is characterized by the rejection of ideas not generated within the company and blocking the process.

The company's approach underlines the company's clear desire to integrate the user at the heart of the discussions; the profound changes that the energy market in Switzerland is undergoing have a direct impact on the company's internal culture and a genuine global opening strategy makes it possible to prepare it for market liberalization (the implementation of an Internet exchange platform, which is at the same time informative, incentive and collaborative, is now a testament to this global approach which has been initiated).

Physical interfaces seem to yield a greater number of ideas than purely virtual interfaces. Three hypotheses could explain this point: (1) the energy is not "top of mind" with the users questioned and it is necessary to stimulate the group face-to-face so that it gives ideas, (2) the challenge posed was badly formulated and not very comprehensible for non-experts, (3) the virtual interface does not offer enough guarantees in relation to the uncertainty generated by the approach (need to reassure and to quickly respond to the questions or misunderstandings that direct physical relations generate). The group solicited seems to have an influence on the number and quality of ideas submitted.

The student group submitted more ideas in proportion to the number of people invited. Ideas from this group were considered to be of better quality (more ideas selected on a pro rata basis in proportion to the ideas submitted). The company that participated in the research has recently begun to change its relationship with its customers in view of the forthcoming liberalization of the electricity market for households in Switzerland. It is possible that the relationship of trust and loyalty is not yet strong enough for customers to engage in co-creation of value via a digital platform. Because, as Cova and Cova (2009, p. 95) highlight, "the process of governance underlying the new consumer's discourse must remind companies that they must not believe that all their customers have succumbed to the formatting of the creative consumer and that they all now have the skills to dialogue, play a role and integrate the company's offers", this process takes time and requires several iterations before they can transform their customers into change agents. The divergent opinions of the expert groups can be explained by various hypotheses; the internal jury is under the influence of a latent internal strategy, a realism as to the applicability of the ideas suggested or a "time-to-market" reflection. However, there is no evidence that the composition of two external and homogeneous juries would have led to selections of converging ideas: the flexibility left to the juries as to the organization of the selection process accentuates the exploratory process of such a study. This may also be due to the "not invented here" syndrome mentioned earlier. However, the interest in seeking ideas from outside the company, as Chesbrough points out, is to overcome this syndrome by working on openness, particularly in a monopolistic environment. We therefore come back to the question of the trust of the company's members in the outside world.

If the Swiss case study allows us to understand the intangible dimension of creativity through a process of generating raw ideas, the French case study on the smart meter allows us to deepen the transition from idea to prototyping in a context of mobilizing very eclectic actors among a population that is not clearly identified.

Case study 2: Linky by Makers, building trust between industry, academia and makers

Between 2013 and 2022, ENEDIS is to deploy 35 million smart meters in France. These meters are the property of the electricity distributor. The meter itself is in the process of being installed for commercial operation. The "smart meter" product can be described as "finished" but the smart grid system will only be fully operational when all the meters are installed in French territory. Furthermore, the ongoing technological and industrial transition and the issues it raises generate controversy (Assemblée Nationale, 2016) that we will not study here. Linky by Makers (LbM) examines the use of smart meters and smart grids (http://linkybymakers.fr/in-english/). This national project began at the end of 2015 during an exchange between industrialists, academics and makers within the Lorraine Fab Living Lab ® (LF2L). At the instigation of Nancy-based makers networking with other French makers in five French regions via five "Regional" FabLabs (FLR), this project, initially planned for six months, was extended by 12 months in part because of trust issues.

LbM is based on the principle that academics, consumers and suppliers can work together [START_REF] Carayannis | Mode 3 Knowledge Production in Quadruple Helix Innovation Systems, Mode 3 Knowledge Production in Quadruple Helix Innovation Systems[END_REF], in particular to understand the uses of a technology that is still little known to the greatest number of people, even though we are all electricity users. The experimental project posits the following hypothesis: makers who are used to "tampering with" could: (1) be an avant-garde community that enriches the understanding of the possible uses of the meter (via additional developments in open hardware);

(2) support reflection on new forms of production/consumption/use of electricity from so-called smart grids. The project's objective was then to allow the French FabLabs, or those who considered themselves makers, and who were interested in the question of energy, to take up this challenge themselves. The University positions itself as a research support to help stakeholders communicate with each other and share ideas, as well as capitalize on and analyse the experience. Different aspects of this case have been described in two previous papers [START_REF] Dupont | Collaborative Innovation Projects Engaging Open Communities: a Case Study on Emerging Challenges[END_REF][START_REF] Dupont | Confiance et Technologie : Deux Dimensions de l'Innovation Ouverte et Agile appliquée à l'Énergie[END_REF]. Figure 2 focuses on the co-design process and its outcome (co-creation of value) stage by stage, supported by digital and physical platforms. LbM's main challenge is to get actors from different parts of France to collaborate with each other, with different timeframes (full-time and/or occasional contribution, paid or voluntary, etc.), different practices, etc. In other words, it is a question of articulating four dimensions [START_REF] Dupont | Collaborative Innovation Projects Engaging Open Communities: a Case Study on Emerging Challenges[END_REF]: (1) Geography with FLRs in four different regions, themselves composed of actors distributed throughout their territory, leading to a lack of face-to-face interactions (lack of spontaneity); (2) Conceptual because the terms "creativity", "energy" and "open source/hardware" do not refer to the same representations and uses according to the actors; moreover, the project started with an approach to creativity (i.e. producing abstract concepts with an academic method) while the makers are in the "make" (disassemble, reassemble, reproduce, transform, etc.); finally, LbM brought together and confronted structured organizations and very agile or fuzzy organizations with sometimes contradictory governance modes;

(3) Technology with a strong initial limit -the distributor did not want to lend smart meters and the participants had no in-depth technical knowledge of this technology or of other aspects of the project (measuring electricity, developing objects in open hardware, working in open source, etc.); (4) Time, which is expressed in terms of the time available to participants according to whether they can invest themselves over six months, two years, the time of a weekend, etc. Time also concerns the management of the knowledge produced: how to leave contributions that others can take up and enhance? Beyond these four barriers to be overcome to ensure collaboration, this case study highlights three observations related to the process of identifying and co-creating the uses of electricity distributed by smart grid: (1) Each has its own practices and objectives: an "engineering" company with an industrial strategy, academics with a research project, FabLabs with their specific history, logic of action and different interests, etc. (2) The makers (at the time of the LbM project) are ultimately not very interested in or accustomed to energy issues; moreover, it is necessary to learn to give concrete expression to intangible elements (energy, electricity, trust, data, communication etc.). ( 3) The integration of users in the product launch phase, when irreversible strategic decisions have been taken, is too late, which inhibits self-determination and the motivation to be part of a co-creation process. It is a question of co-constructing a vision shared by all the actors in the upstream phase.

Like the Swiss ELL context, the French context has largely conditioned the predispositions of the actors to collaborate, requiring the transcendence of individual and specific practices and centres of interest strongly rooted in each other to build a community of interest in a vague and tense context (Assemblée Nationale, 2016). Finally, we can summarize the management of the LbM project in 5 challenges that had to be overcome to create a favourable climate of collaboration between the actors: (1) Having a common representation of the project; (2) Ensuring interactions between communities at the right time in a process under construction; (3) Developing governance compatibilities through peer-to-peer negotiation; (4) Managing internal and external communication; (5) Giving a concrete explanation to intangible elements and using different media to materialize both the project process and its outputs. By proceeding in an iterative and constructivist manner, the research team, in interaction with the participants, deployed and accompanied or adapted the use of the following collaborative technologies. We are aware of the non-exhaustive nature of the approach to be taken, particularly with regard to energy transition issues. The work of [START_REF] Koirala | Trust, awareness, and independence: Insights from a sociopsychological factor analysis of citizen knowledge and participation in community energy systems[END_REF] highlights in particular the importance of local communities for involving citizens and users in the development of new responses such as community energy systems. This study also shows that trust in the local community is one of the key factors for engagement.

TOWARDS A SUPPORT SYSTEM FOR TRUST IN THE CO-CREATION OF VALUE

Findings from the two case studies

The two experiments we have carried out and studied allow us to highlight the challenges related to building stakeholder confidence and trust at the different levels of a Living Lab project. Beyond the limitations encountered in the implementation of projects, their observation and analysis give us the opportunity to better understand the mechanisms in place between communities that are distributed and solicited to contribute to a common project. The researchers' commitment to the design, installation and activation of the processes and the extensive nature of the analyses we were able to produce from these two projects, 24 and 18 months respectively, allow us to access both a quantitative and qualitative reading of these original research materials. Figures 1 and 2 thus present the main technological elements deployed to support co-design in ELL and LbM projects and foster trust between stakeholders.

The physical or virtual platforms mobilized are classified in order of appearance in the process of each project. These diagrams also indicate the quantitative productions resulting from each of them.

Based on the observations and analyses reported in the section dedicated to research design, we can improve each of these processes in order to strengthen co-creation of value between the actors (Figure 3). It appears that two functions are not yet fulfilled by any of the projects: the appropriation within companies of ideas generated outside and the ability to retain and engage consumers or communities in a co-design process involving a company. In both cases, it should also be pointed out that academics have probably played a role as trusted third parties in the process and in the issues submitted to users. The proposal to support service design conceived by ELL can be assimilated to the practice of LF2L and implemented for the LbM project, and finally, the prospect of the emergence of a platform connecting buildings, consumers and producers is specific to the French case, and therefore not yet generalizable.

For each platform and technology used during the co-design process, it is necessary to consider the trust that users place in them, as well as the role that these technologies play in the level of trust that stakeholders place in each other. How does the integration and implementation of these technologies generate a favourable climate among stakeholders to build mutual trust and collaborate together? In other words, by drawing on the elements of the literature relating to trust [START_REF] West | The social psychology of innovation in groups[END_REF][START_REF] Mayer | an Integrative Model of Organizational Trust[END_REF], how to ensure that these technologies encourage an innovative climate within an Open Innovation process involving users: (1) benevolence;

(2) the ability to fulfil the "obligations" agreed upon between the parties; (3) sharing and adhering to clear and valued objectives; (4) the emergence of a non-threatening environment where the parties can contribute and influence decisions by being recognized and valued in their contribution;

(5) the pursuit of excellence through quality work and critical evaluation; (6) the enhancement of innovation and support for working practices to achieve innovation.

Outline of an integrated process for co-creation of value

In the light of the previous findings, we can outline a process with new technological responses. Figure 4 below thus proposes an integrated vision of good practices to establish a climate of trust between the stakeholders in a Living Lab process. This potential process is built by aggregating and scheduling the various data and feedbacks from the two case studies. We have identified eight main functions (Figure 4) that a Living Lab must perform to strengthen the trust in a co-design process: (a) Communication, defined by "to promote and support physical and virtual communication"; (b) Ideation defined by "to submit, select and evaluate ideas"; (c) Materialization defined by "to support collaborative prototyping of products and services"; (d) Contribution defined by "to ensure the sharing of co-creations in open source hardware"; (e) Appropriation defined by "to make externally generated ideas appropriate within companies"; (f) Validation defined by "capitalize on tests by use (and feedback)"; (g) Compensation defined by "to engage communities and make them loyal"; (h) Publication defined by "to guarantee transparency". The definitions we use for these functions come from the cases studied and our experience in the field of project co-design as a project manager for 15 years. The sequential presentation of functions follows the order of occurrence observed in the practical cases, without freezing these steps, which can sometimes be carried out concurrently. Feedback loops, or at least iterations, should also be considered. To characterize and refine the eight functions that a Living Lab must perform, we have crossed them with the determinants of trust from literature within a co-design process including users. This method allows us to sketch a matrix, called "Co-coon" (for co-creation, confidence and trust), which we have completed from the work presented in the previous sections. The content of the matrix constitutes exploration work for researchers and practitioners as a guide to good practices and possible avenues for implementation. (5-a) underlines that a mediator has to dispel misunderstandings. Thirteen (grey) boxes are filled with data from literature and professional experiences. In 1-d, literature suggests to use agile methods popularized by the notion of agility in the IT field [START_REF] Highsmith | The agile manifesto[END_REF]. Agility is available at different scales, from the small team within a company to complete organizations, whether industrial or public. For example, [START_REF] Dikert | Challenges and success factors for large-scale agile transformations: A systematic literature review[END_REF] specifically study agile transformations at the level of large industrial scales. [START_REF] Mergel | Agile innovation management in government: A research agenda[END_REF] focuses its work on governments.

Limits

These works are pioneering in two ways: in terms of Living Lab, trust, and co-creation of value, through an original reading of literature and a linking of concepts; in terms of field of observation and implementation because the energy sector is currently undergoing change. In this paper, we assume the iterations and exploratory logic between the action-research of researchers engaged in Living Lab projects on energy and a more classical state of the art work.

The involvement in our long-term projects enabled us to identify and validate the research gaps.

We were also able to build a first project management model. Nevertheless, it is now necessary to test and validate or improve the Co-Coon matrix by building new projects applied to the consumer energy market and other topics. It could also be relevant to measure the creation of potential value for each box or to assess the risk of destruction of potential value in the event of non-implementation. New resources are therefore needed to engage many actors in such an approach over a long period of time.

CONCLUSION AND PROSPECTS

In this article, we have recalled the fundamental aspect of developing trust within innovation processes to guarantee and amplify their open character and strengthen users' involvement. Nevertheless, the literature highlights the lack of research to understand and accurately characterize the mechanisms underlying the establishment of shared trust in collaborative processes, such as co-design. Two case studies based on a greater consideration of users in the field of energy distribution and management allow us to illustrate and highlight some of the determinants of trust for the co-creation of value. This longitudinal study on projects that took place between January 2014 and July 2017 would need to be supported by the exploration of new data from specifically oriented work on this subject. Putting the user or consumer at the centre of an innovation process and at the heart of service development, for example, does not guarantee their loyalty and commitment. On this point, seeking to test the statistical correlation between the implementation of co-design process methods and the appropriation of the services thus developed by users would legitimize this approach. Because, as Gassmann and his colleagues observe [START_REF] Gassmann | The future of open innovation[END_REF], there is a lack of measurement of the effectiveness of this process. It would also be relevant to conduct research in other industrial fields to validate the generic dimension of the proposed model to support the co-design process (Figure 5). We have deliberately not integrated the technologies of the smart meter into the latter. These make it possible to collect usage data in near-real time, opening up new perspectives in the relationship with the user and the development of service offers for customers. The creation of these data by users, their collection and use is the subject of a democratic debate in France (Assemblée Nationale, 2016). The controversies in several European countries about smart meters also underline a certain distrust of these newly deployed technologies. Moreover, the recent a priori diffusion of the OI paradigm in the energy sector [START_REF] Greco | Open innovation in the power & energy sector: Bringing together government policies, companies' interests, and academic essence[END_REF] may generate different practices from industrial sectors already largely engaged in this logic. A comparative approach would undoubtedly strengthen the Co-coon Matrix we are proposing. Finally, the intangible, even abstract, nature of energy and electricity seems to make it difficult to mobilize neophyte actors with low energy literacy on issues that can quickly become technical or at least require the assistance of specialists to shed light on particular points. A greater effort in communication, vocabulary creation and shared representations seems necessary to overcome this difficulty. The development and use of innovation spaces that bring stakeholders together and collaborate could provide a favourable framework for the emergence of collective intelligence [START_REF] Morel | Espace d'innovation : de nouveaux lieux pour l'intelligence collective ?[END_REF] in the service of projects and ecosystems. To enhance the efficiency of our model, the time dimension will also need to be studied. Indeed, our two case studies describe projects that span almost two years. Is it a constant or a contest of circumstance? Minimum durations should be measured to ensure the success of each step of the Co-coon Matrix -and to evaluate how the industrial domain concerned, the diffusion of open innovation, and societal engagement influence the temporality of the process.

More generally, all the elements of the Co-coon Matrix can be the subject of an experimental program to consolidate and enrich this first proposal with a better consideration of trust in codesign processes. In addition, Co-coon Matrix proposes functional building blocks from a multitude of physical and virtual platforms. Based on this model, it would be interesting to study the possibility of deploying a digital technology that integrates all these functionalities and that would be the "digital twin" of a physical platform dedicated to collaborative innovation, generating a framework where the appetite for creativity is truly released. The recent emergence of the blockchain [START_REF] Nakamoto | Bitcoin: A Peer-to-Peer Electronic Cash System[END_REF] in support of the decentralized system of trust seems to offer new perspectives in many fields, such as democracy [START_REF] Caseau | La blockchain ou la confiance distribuée[END_REF] and manufacturing. This technology also seems to provide specific answers to the challenges of co-creation of value [START_REF] Seulliet | Open innovation, co-création: pourquoi la blockchain est une petite révolution[END_REF][START_REF] Patrick Duvaut | La blockchain pour redonner le pouvoir aux individus[END_REF]. Technological building blocks combined with a relevant organizational model could therefore offer a climate of trust in the communities of co-designers and give meaning and motivation to the actors involved in OI processes.

Figure 1 -

 1 Figure 1 -Sequential vision of the complementarity of the support platforms for co-creation within the ELL project (adapted from (Dupont et al., 2017))

Figure 2 -

 2 Figure 2 -Sequential vision of the complementarity of the support platforms for co-creation within the LbM projects (adapted from (Dupont et al., 2017))

Figure 3 -

 3 Figure 3 -Proposed evolution of technological supports to strengthen co-creation of value for each of the ELL and LbM projects

Figure 4 -

 4 Figure 4 -Eight main functions for a Living Lab supporting trust

Figure 5 -

 5 Figure 5 -Co-coon Matrix to strengthen co-creation, confidence and trust in a Living Lab process

Table 1 -

 1 Different levels of analysis in LLs, adapted from[START_REF] Schuurman | Bridging the gap between Open and User Innovation? Exploring the value of Living Labs as a means to structure user contribution and manage distributed innovation[END_REF]

Level of analysis Definition in LL terms Literature streams Macro (system)

		Living Lab constellation consisting of actors	Open Innovation
		(PPP partnership) and infrastructure
	Meso (project)	LL innovation project	Open & User Innovation
	Micro (stage of the	LL methodology	User Innovation
	project)	consisting of different research steps

Table 2 -

 2 Synthesis of the results of the co-design process implemented by ELL

Distributor's customers Members of the energy efficiency community HES-SO Bachelor's and Master's students

					Total
	Number	30,000	5,000 (some of which may be from customers)	4,707	between 30,000 and 39,707
	Participants		Around 500		Around 500
	Ideas submitted
	via Internet platforms		73	21	530
	3 Dedicated events		436	Not Concerned
	Origin of the selected ideas		21	9	30
	Average of the
	scores assigned		7.59 points	8.12 points	7.74 points
	(via Student test)

ACKNOWLEDGEMENT

The authors would like to thank the anonymous voluntary participants in the ELL and LbM projects, as well as the institutions, companies and innovation spaces that have supported this long-term work. L. Dupont would like to thank Prof. M. Camargo, Dr A. Gabriel and the Daisee network for their involvement in LbM, their advice and ideas. J. Mastelic thanks the companies and individuals who took part in the ELL project.


Srdjan Vesic
email: vesic@cril.fr

Bruno Yun
email: bruno.yun@abdn.ac.uk

Predrag Teovanovic
email: teovanovic@fasper.bg.ac.rs

Graphical Representation Enhances Human Compliance with Principles for Graded Argumentation Semantics

Keywords: Argumentation, Human Reasoning, Principles, Graded Semantics

We examined principles of graded argumentation semantics (independence, anonymity, void precedence, and maximality) to explore if (a) they realistically model human reasoning, (b) graphical representation of arguments facilitates compliance with the principles, (c) there is a positive correlation between compliance with different principles, and (d) this compliance is related to cognitive reflection, need for cognition and faith in intuition. The participants (𝑁 = 96) were randomly assigned to one of two experimental conditionsthe graph group was presented with textual and graphical representations, while the second group was presented only with textual arguments. Our results indicate that there are major differences in the compliance with the several argumentation principles studied in this paper. However, compliance with argumentation principles was consistently better and more consistent in the graph group. Moreover, cognitive reflection correlated with compliance to some principles, but only in the graph group.

INTRODUCTION

Group decision-making and negotiation are fundamental issues for modern multi-agent systems [START_REF] Ito | Persuasion among Agents: An Approach to Implementing a Group Decision System Based on Multi-Agent Negotiation[END_REF][START_REF] Panisson | Argumentation Schemes and Enthymemes in Multiagent Systems[END_REF]. In such a context, information exchange between agents is crucial for agents to coordinate and cooperate. Argumentation theory offers an intuitive interface for stating and explaining agents' positions, allowing them to share or withhold their goals or intentions during the negotiation process.

Researchers in AI often point out that one of the strong points of the argumentation approach is that it uses the format which is intuitive and easy to grasp by humans [START_REF]Handbook of formal argumentation[END_REF][START_REF] Besnard | Elements of Argumentation[END_REF][START_REF] Van Eemeren | A systematic theory of argumentation the pragma-dialectical approach[END_REF][START_REF] Yun | Representing Pure Nash Equilibria in Argumentation[END_REF]. This is because arguments are often constructed by humans to defend or challenge a viewpoint in a debate or a dialogue. We often make decisions by relying on arguments in favor or against a particular action (e.g. buying an object). The scholars in artificial intelligence (AI) developed formal models of reasoning, dialogue, and decisionmaking based on argumentation.

Most of the recent approaches in argumentation are based on the formalism introduced by Dung in 1995 [START_REF] Minh | On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games[END_REF]. He represents the arguments as nodes of a graph where each directed edge models an attack between two arguments. Once an argumentation graph is generated, one can make use of one of the so-called "argumentation semantics". A semantics allows calculating the set of extensions, which are the sets of arguments that can be accepted together. Those extensions represent the possible viewpoints in an argumentation debate. As a result, arguments can have three levels of acceptability. An argument is skeptically accepted if it belongs to all the extensions; credulously accepted if it belongs to at least one extension; rejected if it does not belong to any extension.

Researchers also introduced the so-called "ranking-based semantics" as a way to compare arguments from the least to the most contested. To navigate the plethora of the existing ranking-based semantics, desirable principles have been defined to classify them [START_REF] Amgoud | Ranking-Based Semantics for Argumentation Frameworks[END_REF][START_REF] Bonzon | A Comparative Study of Ranking-Based Semantics for Abstract Argumentation[END_REF][START_REF] Yun | Sets of Attacking Arguments for Inconsistent Datalog Knowledge Bases[END_REF]. For example, the void precedence principle claims that every non-attacked argument is stronger than any attacked argument; the independence principle claims that two arguments that are not connected by attacks should not have any influence on each other's ranking. It should be noted that there are more than 20 principles in ranking-based semantics [START_REF] Amgoud | Acceptability Semantics for Weighted Argumentation Frameworks[END_REF][START_REF] Bonzon | A Comparative Study of Ranking-Based Semantics for Abstract Argumentation[END_REF][START_REF] Yun | Sets of Attacking Arguments for Inconsistent Datalog Knowledge Bases[END_REF].

We tackle three research questions in this paper. First, to the extent of our knowledge, the principles we mentioned have never been experimentally evaluated to check whether they are intuitive or acceptable for humans (and especially non-experts). Hence, we aim to examine if the argumentation principles proposed by the AI researchers realistically model human reasoning. In other words, if a given principle prescribes that a certain argument, say 𝑋 , should be stronger than another argument 𝑌 in a certain context (i.e., argumentation graph), do the human participants also rank 𝑋 as stronger than 𝑌 ? The first goal of our experiment is to verify this.

Second, it is often claimed by the researchers in AI that argumentbased models are easier to grasp by humans than logical formulas. In particular, they hypothesized that the graph-based representation is helpful to understand the problem and facilitate reasoning. Thus, the second aim of our study was to examine whether the participants who see graph-based arguments draw better conclusions than those who do not have access to the argumentation graph.

Third, we investigated if there is a positive manifold among compliances with different normative principles of ranking-based semantics, i.e., whether people who more frequently behave in accordance to expectations that derive from one principle (e.g., anonymity) are also prone to behave more frequently in accordance to expectations that derive from other principles such that independence, maximality, and void precedence among others [START_REF] Amgoud | Ranking-Based Semantics for Argumentation Frameworks[END_REF][START_REF] Bonzon | A Comparative Study of Ranking-Based Semantics for Abstract Argumentation[END_REF][START_REF] Yun | Ranking-Based Semantics for Sets of Attacking Arguments[END_REF]. Finally, we aimed to examine if these individual differences can be predicted by more traditional psychological measures such as the Need for cognition, Faith in intuition, and Cognitive reflection.

This paper is structure as follows. In Section 2, we recall the argumentation setting and formally introduce the principles for graded semantics as well as the necessary notations. In Section 3, we provide the reader with details on the design of our experiments, including explanations on how the principle compliances and the cognitive styles were measured. In Section 4, we thoroughly present our results via item-level, scale-level, and correlational analyses. Lastly, in Section 5, we discuss our findings and reflect on their wider impact on the argumentation community and AI in general.

BACKGROUND

We start this section by recalling the definition of an argumentation graph as defined by Dung in his seminal paper [START_REF] Minh | On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games[END_REF]. Definition 2.1 (Argumentation graph). An argumentation graph is a pair AS = (A, C), where A is a finite set of arguments and C ⊆ A × A is a set of binary attacks between arguments. The set of attackers of

𝑎 ∈ A is 𝐴𝑡𝑡 (𝑎) = {𝑏 ∈ A | (𝑏, 𝑎) ∈ C}.
Given an argumentation graph AS = (A, C), an undirected path in AS is a sequence (𝑎 1 , . . . , 𝑎 𝑛 ) such that for all 𝑖 ∈ {1, . . . , 𝑛}, 𝑎 𝑖 ∈ A and for all 1

≤ 𝑗 ≤ 𝑛 -1, (𝑎 𝑗 , 𝑎 𝑗+1 ) ∈ C or (𝑎 𝑗+1 , 𝑎 𝑗 ) ∈ C.
The connected components of AS, denoted 𝑐𝑐 (AS), is the set of largest subgraphs of AS, such that two arguments are in the same component of AS iff there is an undirected path between them.

In Dung's approach of argumentation [START_REF] Minh | On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games[END_REF], the acceptability of arguments is based on their belonging to some sets, called extensions [START_REF] Baroni | An introduction to argumentation semantics[END_REF][START_REF] Martin | Semi-stable semantics[END_REF][START_REF] Minh | On the Acceptability of Arguments and its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and n-Person Games[END_REF][START_REF] Sarah | The cf2 argumentation semantics revisited[END_REF]. Different semantics are used to calculate the extensions; we refer the reader to the work of Baroni et al. for an introduction on argumentation semantics [START_REF] Baroni | An introduction to argumentation semantics[END_REF]. A different approach consists in ranking the arguments from the most to the least acceptable ones depending on how much they are contested. The latter semantics are called ranking-based semantics and have been widely explored in the literature [START_REF] Amgoud | Ranking-Based Semantics for Argumentation Frameworks[END_REF][START_REF] Bonzon | A Comparative Study of Ranking-Based Semantics for Abstract Argumentation[END_REF][START_REF] Yun | Ranking-Based Semantics for Sets of Attacking Arguments[END_REF]. Indeed, there are two similar approaches: the ranking-based approach and the graded approach. A ranking-based semantics provides a rank (or an order) on arguments, for example 𝑎 ≺ 𝑐 ≺ 𝑏, whereas a graded semantics provides the degrees (sometimes called scores), for example 𝐷𝑒𝑔(𝑎) = 0, 𝐷𝑒𝑔(𝑐) = 0.8, 𝐷𝑒𝑔(𝑏) = 1, meaning that e.g. 𝑎 has the acceptability degree 0. Note that each graded semantics naturally induces the corresponding ranking-based semantics.

Definition 2.2 (Graded semantics).

A graded semantics is a function 𝜎 that takes as input any AS = (A, C) and returns a function

𝐷𝑒𝑔 𝜎 A S : A → [0, 1]. The notation 𝐷𝑒𝑔 𝜎 A S (𝑎) ≤ 𝐷𝑒𝑔 𝜎 A S (𝑏) means that 𝑏 is at least as acceptable as 𝑎 w.r.t. 𝜎.
To perform a more systematic study of graded semantics, researchers in argumentation have defined several desirable principles that should govern the behaviour of these semantics. Since this is the first study that aims at assessing to which extent the principles are compatible with human reasoning, we focus on a limited subset of principles. To test the compliance we need several examples (at least five) per principle and to keep the experiment length within reasonable limits, we decided not to exceed five principles in this study. We choose the principles that were considered the simplest to model using up to four arguments to avoid large graphs, which are difficult to grasp and evaluate by non-experts. Note that although there are more restrictive versions of the anonymity and independence principles, where the isomorphic images of an argument need to have the same acceptability degree, we decided to encode the "ranking-based" version of the principles since humans tend to compare arguments between them, rather than associate absolute scores to each argument.

To define anonymity, we first introduce the notion of isomorphism between argumentation graphs. Anonymity states that the names of the argument should not influence their acceptability. Definition 2.4 (Anonymity). We say that a graded semantics 𝜎 satisfies anonymity iff for every two argumentation graphs AS = (A, C) and AS ′ = (A ′ , C ′ ) such that AS ′ = 𝛾 (AS), it holds that for all 𝑎, 𝑏 ∈ A, 𝐷𝑒𝑔 𝜎 A S (𝑎) ≤ 𝐷𝑒𝑔 𝜎 A S (𝑏) iff 𝐷𝑒𝑔 𝜎 A S ′ (𝛾 (𝑎)) ≤ 𝐷𝑒𝑔 𝜎 A S ′ (𝛾 (𝑏)). As stated before, there is a more restrictive version of anonymity exists where the degree of each argument must be exactly the same as its isomorphic image, i.e., for all argument 𝑎 ∈ A, 𝐷𝑒𝑔 𝜎 A S (𝑎) = 𝐷𝑒𝑔 𝜎 A S ′ (𝛾 (𝑎)). The independence principle states that the acceptability degree of an argument should only affected by other arguments in its connected component. Definition 2.5 (Independence). We say that a graded semantics 𝜎 satisfies independence iff for every argumentation graph AS, for every AS ′ = (A ′ , C ′ ) ∈ 𝑐𝑐 (AS) and for every 𝑎, 𝑏 ∈ A ′ , if 𝐷𝑒𝑔 𝜎 A S ′ (𝑎) ≤ 𝐷𝑒𝑔 𝜎 A S ′ (𝑏) then 𝐷𝑒𝑔 𝜎 A S (𝑎) ≤ 𝐷𝑒𝑔 𝜎 A S (𝑏). Void precedence states that an unattacked argument is more acceptable than an attacked one. Definition 2.6 (Void precedence). We say that a graded semantics 𝜎 satisfies void precedence iff for every argumentation graph AS = (A, C) and 𝑎, 𝑏 ∈ A such that 𝐴𝑡𝑡 (𝑎) = ∅ and 𝐴𝑡𝑡 (𝑏) ≠ ∅ then 𝐷𝑒𝑔 𝜎 A S (𝑎) > 𝐷𝑒𝑔 𝜎 A S (𝑏). Maximality states that an unattacked argument should have the maximum acceptability degree. Definition 2.7 (Maximality). We say that a graded semantics 𝜎 satisfies maximality iff for every argumentation graph AS = (A, C) and 𝑎 ∈ A such that 𝐴𝑡𝑡 (𝑎) = ∅ then 𝐷𝑒𝑔 𝜎 A S (𝑎) = 1. We now present the details of the experimental setting.

EXPERIMENTAL DESIGN

We recruited a total of 98 participants (91 females) from a pool of master's students that attended statistics course at the Faculty for Special Education and Rehabilitation at the University of Belgrade.The mean age was 25.92 years (𝑆𝐷 = 4.25), the participants did not have any previous knowledge about computer science nor argumentation theory, and all participants received course credit for taking part in the study. ments (A, B, C, andD) about the Zuber transportation company (which graphical representation is provided in Figure 1).

A : The train 4147 at 7.45 am for Paris will be on time, I checked this morning on the Zuber transportation website. B : The train 4147 at 7.45 am for Paris will be 20 minutes late, I checked this morning on the Zuber transportation mobile app. C : The Zuber company has been reported to show incorrect information recently due to hackers' activity that targeted their database. D : The Zuber company stocks have dropped in value since the recent hacker attack.

For each use-case, we designed the corresponding argumentation graph by following the well established rules from argumentation theory. In particular, we chose the types of attack relations (e.g. attacking an explicit generic) that are the easiest to understand by humans [START_REF] Cramer | Directionality of Attacks in Natural Language Argumentation[END_REF] to insure that their mental representation matches the graph displayed. The experiment was split in four major parts:

( (2) Second, we gave the participants three control questions to test their understanding of the notions of argument and attack. In each question, the participants were first shown a text (in natural language) describing three arguments as well as three different argumentation graphs. Then, they were asked to choose which graph corresponded to the given text. In Example 3.3, we present one of the control questions.

Example 3.3. Let us consider the following three textual arguments (A, B, and C) about the whereabouts of Sophie Schmoe. The three graphical representations given to the participants are represented in Figure 3. A : Joe Bloggs made a statement that he saw Sophie Schmoe at the Palais Garnier yesterday at 8 pm. B : Romana Leech says that there was a protest in front of the Eiffel tower yesterday. C : Dick Harry declared that he was with Sophie Schmoe at the Champs Elysée yesterday at 8 pm and that Romana Leech is a compulsive liar so her statements cannot be trusted. (3) Third, the participants were asked to perform 16 different tasks (𝑇 1 to 𝑇 16 ). Our goal was to check the participants' level of compliance with the different argumentation principles. More details about how each individual principle was assessed will be given in Section 3.1. In each task, the participants were shown between two and four textual arguments and the corresponding argumentation graph if the participant was assigned in the graph group, otherwise, the participant was only shown the textual arguments (no-graph group). Then, the participants were asked to estimate the strength of each argument by using a 4-point Likert scale: 1 (very weak), 2 (weak), 3 (strong), and 4 (very strong). (4) Lastly, participants were presented with three cognitive reflection test tasks [START_REF] Frederick | Cognitive Reflection and Decision Making[END_REF] (see Section 3.2.1) following five statements assessing need for cognition (𝑁 𝐹𝐶 1 to 𝑁 𝐹𝐶 5 ) and five statements assessing faith in intuition (𝐹 𝐼 1 to 𝐹 𝐼 5 ) that were part of the rational-experiential inventory [START_REF] Pacini | The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon[END_REF] (see Table 1).

𝐴

Evaluating Principle Compliance

In this subsection, we give more details on how the participants' compliance with the graded semantics principles studied in this paper have been assessed.

3.1.1 Anonymity. This principle claims that the ranking of arguments will remain the same, independently of the arguments' content, as long as the structure of the argumentation graph is preserved. We study two variants of anonymity 1 . The first variant, which we call anonymity between tasks is more general and requires an isomorphism between any two graphs with the same structure. Five pairs of tasks were used to measure this principle (ABT_1 to ABT_5; see first row of Table 2). For each pair, both tasks consist of the same number of arguments (ranging from two to four) such that the topology of the graph is preserved. Formally, the argumentation graphs can be represented using the same graph if the arguments are renamed. To illustrate, let us present one pair of tasks (see Example 3.4 and 3.5 below). ments (A, B, andC) about the painter Philippe Zhao (which graphical representation is provided on the left side of Figure 4): A: The historian Petrović deduced from the recently found letters of the painter Philippe Zhao that he only had one wife and she was Serbian. B: The historian Albin deduced from the recently found letters of the painter Philippe Zhao that he only had one wife and she was Polish. C: Roger says that Philippe Zhao had multiple wives in his life.

Note that both A and B attack C since each of them contains a justification, and also since a historian is more trustworthy than an average person (Roger). 1 This distinction is new and does not exist in the literature. Note that the graphs in the two tasks are "the same" (two mutually conflicting arguments attacking the third argument) except for the arguments' names and contents. The anonymity principle says that the evaluation of the arguments should depend only on the graph and not on the arguments' names and their content. For instance, if a participant claims that A and B are equally strong and are both stronger than C in the first task, according to anonymity, they should claim that in the second task, B and C are equally strong, and are both stronger than A.

The second variant of the anonymity principle, called anonymity within tasks, requires a graph when there exists an isomorphism (different from identity) from the graph to itself. Fourteen tasks were employed to measure this principle (AWT_1 to AWT_14; second row in Table 2). For each task, we compared the acceptability values attached to the arguments by the participants and compared them with the different arguments that are isomorphic within the graph. For instance, in Example 3.4, A and B mutually attack each other and they both attack C, anonymity within tasks says that A and B should have the same acceptability degree.

3.1.2 Independence. This principle claims that two arguments that are not connected by attacks should not have any influence on each other's ranking. Five pairs of tasks were used in our study (IND_1 to IND_5; third row in Table 2). For each pair of tasks, one task consisted of two or three arguments, whereas the other task consisted of the same arguments with one or two additional arguments that were not conflicting with (and thus, not connected to) the existing arguments. Formally, there was no attack between any of the existing arguments from the first task and the new arguments introduced in the second task -therefore, this principle states that the ranking of the arguments from the first task should remain unchanged after the introduction of new unrelated arguments.

In Example 3.6, we illustrate one pair of tasks. The first task was composed of two mutually conflicting arguments A and B. In the second task, the participant is shown the previous two arguments 2: Percentage of correct answers in two groups and tests of difference. ABT_1 stands for the first pair of tasks that is used to check the compliance with anonymity between tasks. 𝑇 3 -𝑇 10 means that tasks 3 and 10 are used in ABT_1. 𝐴 = 𝐵 means that we check whether the strength of argument 𝐴 is equal to the strength of argument 𝐵. The notation 𝐶 > 𝐴, 𝐵 means that we check if 𝐶 is stronger than 𝐴 and 𝐵. Regarding maximality, 𝐶 = 4 means that we check if 𝐶 has the maximal strength. and the following additional arguments C and D. Since newly introduced arguments C and D are not related to A and B, the rating of A and B should remain unchanged, i.e. if a participant claimed that A is as strong as B in the first task, they should claim that A is still as strong as B in the second task (after C and D are added).

Example 3.6. Let us consider the following four textual arguments (A, B, C and D) about Lady Gaga (which graphical representation is provided in Figure 5).

A : Benedicte says that the dress worn by Lady Gaga this morning is green. B : Michael says that the dress worn by Lady Gaga this morning is red. C : Lady Gaga was nominated for the Liechtenstein Neuroscience Association award this year. D : Liechtenstein Neuroscience Association committee announced that only scientists can be nominated for their award.

𝐴 𝐵 𝐴 𝐵 𝐶 𝐷

Figure 5: Two graphs shown in two different tasks. Independence states that the ranking between A and B should be the same in the two graphs.

Void precedence.

This principle claims that every non-attacked argument should be stronger than any attacked argument. Ten tasks were employed to measure this principle. For each task, we checked that arguments that were not attacked were ranked higher than any arguments that were attacked by at least one other argument.

Example 3.7. Let us consider the following four textual arguments (A, B, C and D) about tennis (which graphical representation is provided in Figure 6).

A: John thinks that each tennis game should end after one player wins three sets. B: Pierre thinks that each tennis game should end after one player wins two sets. C: Gerhard claims that the players will be too tired at the end of the season if all the tournaments are played on three sets. D: Ichiro says that the Association of Tennis Professionals (ATP) should provide more money for young players since the sponsors have too much impact.

𝐷 𝐶 𝐴 𝐵

Figure 6: Void precedence states that 𝐶 and 𝐷 should be ranked higher then 𝐴 and 𝐵.

Here, in addition to the mutual conflicts between A and B, C attacks A but is not attacked by the other arguments. The void precedence principle states that non-attacked arguments, here C and D, should be ranked strictly higher than A and B.

3.1.4 Maximality. This principle claims that non-attacked arguments should have the maximal acceptability value. Ten tasks were employed to measure this principle. In each task, we checked whether all non-attacked arguments had degree 4 (very strong). For instance, in Example 3.7, since arguments C and D are not attacked, we checked if the participant ranked C and D as "very strong". [START_REF] Frederick | Cognitive Reflection and Decision Making[END_REF]), consisting of three items that cue a fast but incorrect response. One example of an item is the following question: "A racket and a ball cost 1100 RSD (Serbian currency) in total. The racket costs 1000 RSD more than the ball. How much does the ball cost?". Although the correct answer is "50 RSD" approximately 60% of participants answered "100 RSD" (an incorrect but intuitive answer). A total score was calculated as a sum of intuitive responses (Cronbach's 𝛼 = 0.73 indicating fair level of internal consistency).

Measures of Cognitive Styles

Need for cognition and faith in intuition.

Those styles were assessed by employing a short, 10-item form of rational-experiential inventory [START_REF] Pacini | The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon[END_REF] (see Table 1). Each subscale consists of five items, and participants were instructed to assess statements by using a 5-point Likert-type scale ranging from 1 (completely disagree) to 5 (completely agree). Scores for each subscale were calculated as average item values after reversely coding data for negatively formulated items. Cronbach's 𝛼 = 0.87 for faith in intuition subscale was quite high, but it was much lower for need for cognition subscale (Cronbach's 𝛼 = 0.46) indicating suboptimal level of internal consistency for these measures. 2 relate to the item-level of analysis. As described in Section 3, for each of the five principles, several (pairs of) tasks, ranging from five to fourteen, were used to examine if people behave in accordance with these principles. For three principles (independence, anonymity between tasks and anonymity within tasks), consistency of responses was used as a criterion of normative behavior, while response accuracy was used as a criterion for two other principles (void precedence and maximality). Principles and related (pairs of) items are presented in the first two columns of Table 2. In the third and fourth columns of Table 2, we show the percentage of normatively correct responses in the no-graph and graph groups respectively. Results indicate that anonymity between tasks in the no-graph group was the hardest principle to comply with; with a percentage of correct answers ranging from 7.3% to 48.8%, while participants behaved the most frequently in accordance to anonymity between tasks in the graph group; with a percentage of correct answers ranging from 68.4% to 100%. Furthermore, the percentage of normatively correct responses was higher in the graph group in comparison to the no-graph group on all 44 items, and this difference was statistically significant (𝑝𝑠 < 0.05) on 27 items. The mean effect of graph representation on the participants' performance across items was 𝑟 φ = 0.27.

EXPERIMENTAL RESULTS

Results presented in Table

Results of scale-level analyses, presented in Table 3, further demonstrate that graphical representation of argument structure significantly enhances participants' performance. The effect of graphical representation was consistently significant across scales (values of t statistics ranged from 4.04 to 6.41, all p-values were equal or less than 0.001) and its size ranged from 0.70 (medium effect) to 1.37 (very large) in terms of Cohen's d statistic, i.e., from 0.33 to 0.55 in terms of point-biserial correlation coefficient.

Results of the correlational analysis are shown within Table 4 (no-graph group) and Table 5 (graph group) with Cronbach's alpha coefficients on diagonal lines. Presented results indicate that internal consistency of responses was somewhat higher in the group that was presented with graphs, and this difference was statistically significant for anonymity between tasks (χ 2 (1) = 8.11, 𝑝 = 0.004) and void precedence (χ 2 (1) = 11.38, 𝑝 < 0.001) scales.

Measures of individual differences in participants' propensity to comply with different normative principles showed positive manifold, but this tendency was far more pronounced in the graph group, in which correlation coefficients ranged from 0.38 to 0.67. Consequently, a single latent factor (𝜆 = 3.17), that loaded highly on each scale (all 𝑟𝑠 above 0.70) and accounted for 63.4% of the total variance, was extracted. In other words, participants who were more prone to comply with one of principles were also more prone to comply with all the other principles, which indicates that these behaviors are rooted in more general ability to comply with argumentation principles. This ability was significantly related to cognitive reflection (𝑟 = 0.34, 𝑝 < 0.001), but not to faith in intuition and need for cognition (𝑟𝑠 < 0.20). This indicates that propensity to resist reporting the response that first comes to mind and to engage in further reflection is highly relevant for understanding the general ability to comply with argumentation principles. On the other side, two latent factors were retained in the no-graph group and rotated using the Promax procedure. The first latent factor accounted for 46.3% variance and it loaded highly on maximality (0.88), void precedence (0.78) and independence (0.77), while the second explained an additional 30.3% of the variance and loaded on two anonymity scales -between (0.94) and within tasks (0.87). These two factors were practically independent (𝑟 = 0.19, 𝑝 = 0.24) and neither one correlated significantly with cognitive reflection, faith in intuition and need for cognition (𝑝𝑠 > 0.05).

DISCUSSION

We showed that participants exhibit a higher level of compliance with the principles when they are provided the graphical representation of the arguments (see Table 3). We now provide some more detailed comments. We start by considering the principles that were the least complied with and analyse possible reasons.

Anonymity between tasks is the least satisfied principle for the no-graph group. Furthermore, there is a huge difference in satisfaction between the graph and the no-graph group. We hypothesise that this is because participants had difficulties in observing that there is the same structure of conflicts without formalizing the structure via an argumentation graph or another formalism. This would explain both the low degree of compliance in the no-graph group and the difference between the graph and the no-graph group.

The second least satisfied principle in the no-graph group -and the least satisfied principle in the graph group -is maximality. We hypothesise that maximality is just too strong a principle. Let us illustrate this on the task 𝑇 6 , in which there was the least degree of compliance with maximality for both groups. This task is shown in Example 3.7 and Figure 6. Consider argument 𝐷; even if it is not attacked in this task, the participants do not necessarily completely agree with it due to their beliefs and their background knowledge. They can consider the strength of 𝐷 to be less than 4 (for example, 3). However, maximality states that since 𝐷 is not attacked, it should have the maximal strength, which in our experiment was 4.

The data from other empirical studies on argumentation [START_REF] Cerutti | Formal Arguments, Preferences, and Natural Language Interfaces to Humans: an Empirical Evaluation[END_REF][START_REF] Polberg | Empirical evaluation of abstract argumentation: Supporting the need for bipolar and probabilistic approaches[END_REF][START_REF] Rosenfeld | Providing Arguments in Discussions on the Basis of the Prediction of Human Argumentative Behavior[END_REF] provides possible reasons of non-compliance with some of the principles. For instance, Polberg and Hunter [START_REF] Polberg | Empirical evaluation of abstract argumentation: Supporting the need for bipolar and probabilistic approaches[END_REF] observe that "the data shows that people use their own personal knowledge in order to make judgments". It is obvious that this can yield noncompliance with anonymity and maximality. Rosenfeld et al. [START_REF] Rosenfeld | Providing Arguments in Discussions on the Basis of the Prediction of Human Argumentative Behavior[END_REF] showed that in the context of argumentative conversations, people do not always choose arguments that are justified according to the given semantic choice. Cerutti et al. [START_REF] Cerutti | Formal Arguments, Preferences, and Natural Language Interfaces to Humans: an Empirical Evaluation[END_REF] showed that although there is a "correspondence between the acceptability of arguments by human subjects and the justification status prescribed by the formal theory in the majority of the cases, there are some significant deviations, which appear to arise from implicit knowledge". We refer the reader to the survey of empirical cognitive studies about formal argumentation by Cerutti et al. [START_REF] Cerutti | Empirical Cognitive Studies About Formal Argumentation[END_REF].

In future work, we plan to explore whether weighted argumentation graphs [START_REF] Amgoud | Evaluation of arguments in weighted bipolar graphs[END_REF] could better model human reasoning. The idea is to consider not only arguments and attacks, but also arguments' initial weights. An initial weight represents the intrinsic strength of an argument which does not take into account the graph and the attacks but only internal factors, for example, the trust in the source providing the argument or the participant's belief in an argument.

We already explained that participants exhibit a higher level of compliance with the principles when they are provided the graphical representation of the arguments. Furthermore, graphical representation of arguments not only enhances participants' performance in a way that it amplifies compliance with principles, but it also increases the reliability of people's behavior (see Tables 4 and5) . Namely, their responses were more consistent in two ways. First, as shown by Cronbach's alpha coefficients, internal consistency of measures of anonymity between tasks and void precedence was much higher in the graph group. In other words, a participant who solves correctly one of void precedence tasks (resp. tasks intended to measure anonymity between tasks) has higher chance to solve correctly other void precedence tasks (resp. tasks intended to measure anonymity between tasks). Second, as indicated by the coefficients of correlations and results of exploratory factor analysis, overall strength of relations between measures of adherence to different principles was much higher in the graph group. In other words, the person who complies with independence (or any other principle) has a higher chance to comply with all other principles. Taken together, correlational results indicate that people were far more consistent in their evaluation of arguments (i.e., less prone to respond randomly) when arguments were accompanied by graphical representations.

Conclusions presented here should be taken with some caution considering the exploratory nature of our study and scarcity of similar previous research. One might ask if results would be replicated under other conditions. For example, participants could be instructed to rank arguments in each task (instead of rating each of 5: Correlations in the graph group. Cronbach's 𝛼s are presented on a diagonal line. * 𝑝 < 0.05, * * 𝑝 < 0.01. them). They could also be presented with all possible combinations of pairs of arguments in each task and instructed to decide if one of the two arguments is stronger and, if yes, which one. Robustness of results could also be tested by varying instruments and material. Future research might also include additional tasks that would allow for research of other principles, or which would examine reasoning on more complex structures of arguments [START_REF] Hecham | Argumentation-Based Defeasible Reasoning For Existential Rules[END_REF][START_REF] Modgil | The ASPIC+ framework for structured argumentation: a tutorial[END_REF][START_REF] Toni | A tutorial on assumption-based argumentation[END_REF][START_REF] Yun | Argumentation techniques for existential rules[END_REF]. It would also be interesting to examine if solely graphical representation of arguments structure (without the text) would lead to different results. Finally, the different populations, beside university students, should be also included in further studies in order to extend the generalisability of the obtained results.

Given the empirically proven usefulness of the graphical representation, one can ask the question of how to use this fact in practice. Suppose that the people are discussing a question online in the framework of e-democracy (e.g. whether to build a swimming pool, a park, or a railway station). We know that they will have a more insightful discussion and a better mutual understanding if they have access to the graph corresponding to their discussion. Our results are aligned with other studies recommending the usage of collaborative computer-supported argument visualisation tools [START_REF] Iandoli | Socially augmented argumentation tools: Rationale, design and evaluation of a debate dashboard[END_REF], such as Rationale (https://www.rationaleonline.com/) or Kialo (https://www.kialo.com/).

One remaining open question is, however, how does one provide such a graph in a real-life setting? Are the participants able to provide it themselves and what mechanisms can we use to ensure the correctness of the created graph, i.e., that the graph models the problem in question. We plan to address those questions as a part of future work. Namely, we plan to investigate how well people translate arguments and attacks into a graphical representation and whether drawing the graph themselves improves their compliance with the reasoning principles. Note that although a similar approach was used by Cramer and Guillaume [START_REF] Cramer | Directionality of Attacks in Natural Language Argumentation[END_REF], they only focused on the human perception of the directionality of attacks and not on how drawing the graph could improve cognitive reasoning. From the control tasks, we showed that roughly 75% of the participants choose the appropriate graph among the three offered.

Definition 2 . 3 (

 23 Isomorphism). An isomorphism between two argumentation graph AS = (A, C) andAS ′ = (A ′ , C ′ ) is a function 𝛾 : A → A ′ such that for every (𝑎, 𝑏) ∈ C iff (𝛾 (𝑎), 𝛾 (𝑏)) ∈ C ′ .With a slight abuse of notation, we use AS ′ = 𝛾 (AS).

Figure 1 :

 1 Figure 1: Graphical representation of natural language arguments and their attacks for the Zuber use-case.

1 )Example 3 . 2 .Figure 2 :

 1322 Figure 2: Graphical representation of natural language arguments and their attacks for the Staphylococcus use-case.

Figure 3 :

 3 Figure 3: Graphical representation of the three graphical representations (top, middle, bottom) proposed in a control question.

Example 3 . 4 .

 34 Let us consider the following three textual argu-

Figure 4 :Example 3 . 5 .

 435 Figure 4: Anonymity states that the strength of C in the graph from Example 3.4 (left) should be the same as the strength of A in the graph from Example 3.5 (right).

3. 2 . 1

 21 Cognitive reflection. Cognitive reflection was assessed via the cognitive reflection test (CRT

Table 1 :

 1 𝑁 𝐹𝐶 1 I do not like to have to do a lot of thinking. (R) 𝑁 𝐹𝐶 2 I try to avoid situations that require thinking in depth about something. (R) 𝑁 𝐹𝐶 3 I prefer to do something that challenges my thinking abilities rather than something that requires little thought. 𝑁 𝐹𝐶 4 I prefer complex to simple problems. Statements shown to participants. (R) is used to denote reversely scored items

	𝑁 𝐹𝐶 5 Thinking hard and for a long time about something
		gives me little satisfaction. (R)
	𝐹 𝐼 1	I trust my initial feelings about people.
	𝐹 𝐼 2	I believe in trusting my hunches.
	𝐹 𝐼 3	My initial impressions of people are almost always right.
	𝐹 𝐼 4	When it comes to trusting people, I can usually rely on
		my "gut feelings".
	𝐹 𝐼 5	I can usually feel when a person is right or wrong even
		if I can't explain how I know.

Table 3 :

 3 Descriptive statistics for normative principles scales in two groups

	Scale	No-graph (𝑛 = 57) Graph (𝑛 = 41) 𝑀 𝑆𝐷 𝑀 𝑆𝐷	𝑡 (96)	Test of difference 𝑝 𝑟	𝑑
	Independence	73.1	21.7	88.8	16.5			4.04 < 0.001 0.38 0.81
	Anonymity between tasks 24.9	20.1	63.5	34.3			6.41 < 0.001 0.55 1.37
	Anonymity within tasks 71.5	18.6	91.4	14.5			5.93 < 0.001 0.52 1.19
	Void precedence	47.3	24.7	73.0	32.0			4.29 < 0.001 0.40 0.88
	Maximality	38.8	25.0	57.2	27.2			4.42		0.001	0.33 0.70
		1	2	3	4	5			6	7	8	9
	1. Independence	0.46
	2. Anonymity between tasks	0.14	0.29
	3. Anonymity within tasks 0.40 * * 0.68 * *	0.76
	4. Void precedence	0.53 * *	0.23	0.29	0.68
	5. Maximality	0.46 * * -0.16	0.06	0.51 * * 0.73
	6. Control tasks	0.43 * * 0.36 * 0.49 * * 0.47 * * -0.08 0.37
	7. Cognitive reflection	0.23	0.07	0.19	0.21	0.05	0.20	0.74
	8. Need for Cognition	-0.17 -0.17 -0.06	0.02	0.02 -0.04 -0.11 0.32
	9. Faith in intuition	-0.10 -0.04 -0.07 -0.23 -0.21 0.04	0.05 -0.03 0.84

Table 4 :

 4 Correlations in the no-graph group. Cronbach's 𝛼s are presented on a diagonal line. * 𝑝 < 0.05, * * 𝑝 < 0.01.

		1	2	3	4	5	6	7	8	9
	1. Independence	0.40
	2. Anonymity between tasks 0.46 * *	0.76
	3. Anonymity within tasks 0.49 * * 0.67 * *	0.81
	4. Void precedence	0.54 * * 0.67 * * 0.64 * *	0.90
	5. Maximality	0.38 * * 0.43 * * 0.44 * * 0.65 * * 0.78
	6. Control tasks	0.26	0.32 *	0.19	0.47 * * 0.32 * 0.47
	7. Cognitive reflection	0.28 *	0.24	0.27 * 0.37 * * 0.18	0.20 0.82
	8. Need for Cognition	0.10	0.02	0.10	0.14	0.19 0.27 * 0.05 0.50
	9. Faith in intuition	0.32 *	0.04	0.11	0.13	0.13	0.11 0.02 0.34

* 0.89 Table

Proc. of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2022), P. Faliszewski, V. Mascardi, C. Pelachaud, M.E. Taylor (eds.), May 9-13, 2022, Online. © 2022 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.