Thomas Baudel
email: baudelth@fr.ibm.com

Manon Verbockhaven

Victoire Cousergue

Guillaume Roy

Rida Laarach

ObjectivAIze: Measuring Performance and Biases in Augmented Business Decision Systems

Keywords: Business decision systems, Decision theory, Cognitive biases

Business process management organizes flows of information and decisions in large organizations. These systems now integrate algorithmic decision aids leveraging machine learning: each time a stakeholder needs to make a decision, such as a purchase, a quote, or hiring someone, the software leverages the inputs and outcomes of similar past decisions to provide guidance, as a recommendation. If the confidence is high, the process may be automated. Otherwise, it may still help provide consistency in the decisions. Yet, we may question how these aids affect task performance. Can we measure an improvement? Can hidden biases influence decision makers negatively? What is the impact of various presentation options? To address those issues, we propose metrics of performance, automation bias and resistance. We validated those measures with an online study. Our aim is to instrument those systems to secure their benefits. In a first experiment, we study effective collaboration. Faced with a decision, subjects alone have a success rate of 72%; Aided by a recommender that has a 75% success rate, their success rate reaches 76%. The human-system collaboration had thus a greater success rate than each taken alone. However, we noted a complacency/authority bias that degraded the quality of decisions by 5% when the recommender was wrong. This suggests that any lingering algorithmic bias may be amplified by decision aids. In a second experiment, we evaluated the effectiveness of 5 presentation variants in reducing complacency bias. We found that optional presentation increases subjects' resistance to wrong recommendations. We intend to leverage these findings to guide the design of humanalgorithm collaboration in financial compliance alert filtering.

Introduction

For the past 20 years, Business Process Management (BPM) [START_REF] Jeston | Business Process Management[END_REF] has streamlined processes and operational decision-making in large enterprises, transforming work organization. BPM organizes information flows, from input events such as a purchase order or a hiring request, through chains of stakeholders involved in various parts of the process, to reach some outcome. In a nutshell, a BPM system allows programming an enterprise like one would a robot.

Because all inputs and operational decisions are stored, recent improvements involve applying machine learning techniques on past inputs and outcomes, to automate decision processes or to assist decision makers by providing suggestions on the likely outcome of each decision, assuming similar causes produce similar effects [START_REF] Maggi | Predictive monitoring of business processes[END_REF][START_REF] Halle | Business Rules Applied[END_REF]. This technology ought to be largely beneficial, reinforcing the consistency of decisions and improving productivity, enabling "extended cognition", or "active externalism" as described by Chalmers [START_REF] Chalmers | The Extended Mind, Philosophy of mind: classical and contemporary readings[END_REF], or as envisioned as "Human-Centered AI" by Schneiderman [START_REF] Shneiderman | Human-Centered Artificial Intelligence: Three Fresh Ideas[END_REF]. Now, augmenting decision processes is not without risks. There is a large institutional community focusing on the area of AI Ethics, that stresses the requirements of fairness, transparency, explainability, accountability [START_REF] Hleg-Ai | Ethics guidelines for trustworthy AI[END_REF]… In particular, the prevention of algorithmic biases is a major concern [START_REF]Algorithms : mind the bias ! Report of the Institut Montaigne thinktank[END_REF], which needs to be addressed by technology [START_REF] Bellamy | AI Fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias[END_REF], best practices [START_REF] Amershi | Guidelines for Human-AI Interaction[END_REF] and regulations [START_REF] Lemaire | LOI n° 2016-1321 du 7 octobre 2016 pour une République numérique[END_REF]. There is less institutional visibility on the changes to work practices and human decision-making these tools introduce. Cognitive biases induced by decision support systems are largely studied, identifying patterns such as automation and complacency biases [START_REF] Bahner | Misuse of automated decision aids: Complacency, automation bias and the impact of training experience[END_REF] or, on the contrary, algorithm aversion [START_REF] Burton | A systematic review of algorithm aversion in augmented decision making[END_REF] and decision fatigue [START_REF] Hirshleifer | Decision fatigue and heuristic analyst forecasts[END_REF][START_REF] Anderson | The Psychology of Doing Nothing: Forms of Decision Avoidance Result from Reason and Emotion[END_REF] when higher productivity is expected. There is less work directly applicable to the context of business decision-making, in our present situation, where decision aids can be provided as a generic, system-level feature, regardless of their relevance for the task being performed by the human agent.

To ensure this type of assistance can be safely and profitably incorporated in business decision support systems, we first narrow down the type of tasks we are interested in augmenting, and the type of aids we wish to evaluate. Then, we review the literature to guide our designs. We present a performance and cognitive biases evaluation model, as a set of metrics that can be evaluated empirically for various kinds of decision aids and tasks. We conduct a study to evaluate the ability of our model to measure performance and biases. Finally, we propose a methodology to incorporate bias measurement and compensation in business decision support systems, to ensure they provide their expected benefits with minimal inconvenience.

Augmented Business Decision Making

A business process is modeled in a flowchart. In this model, decision steps take as input some information, leverage external information, such as regulations or resource constraints, presumably only known to the decision maker, to advance the process through a predetermined set of possible outcomes. The type of decision tasks we in-vestigate are constrained, leave little room for creativity and presumably rely on a combination of explicit rules and heuristics.

When the decision logic can be formally expressed, decisions can be automated deterministically, via deduction [START_REF]A Brief History of the Business Rule Approach, 3rd ed[END_REF]. When sufficiently robust heuristics are available, scoring methods or more sophisticated algorithms may automate the decision via induction, with an escalation process to handle exceptions. Finally, many operational business decisions involve complex tradeoffs, which, we assume, involve so many factors that automation is for now out of question. In these cases, a decision aid fed with previous decisions and possibly external data sources, such as revenue targets for the team and staffing statistics, may provide information that can be useful, for instance to a manager who considers granting a hiring request.

More formally, we are interested in measuring, and possibly improve, the effects decision aids may have in the following circumstances:

• Some information regarding a case is available and is assumed to be reliable.

• A choice, among a predefined set of alternatives, needs to be made.

• The choice is partly constrained by explicit constraints (regulations, resource limitations…) • Some contextual information, exact or probabilistic, explicit or intuitive, is available to the decision maker (e.g. guidelines & priorities regarding the business context). Those allow the decision maker to form an internal decision model, which is a non-explicited procedure a priori followed consistently by the decision maker. • Other information is known only by the system, such as a history of cases. • There is a best possible choice, but it may not be knowable in advance, if ever, for a given case. • It is possible to provide, a posteriori, exact or approximate measures of which choices tend to perform better for which types of cases, for instance as an actuarial report. These measures can be used to create a computable decision model.

Numerous business situations implemented with BPM match this description, ranging from the mundane, such as a purchase decision, to more serious decisions such as hiring someone, granting a loan or selecting the winner of a bid, finally to morally heavy situations like deciding of a prison sentence [START_REF] Taddeo | How AI can be a force for good[END_REF]. Still, much decision-making activity, such as medical diagnostic or investment decisions, allow more creativity in choice-making and falls outside the scope of our work. What matters in our circumstances is that no algorithm may deliver a choice with 100% accuracy. An algorithm may be better than humans in general, but there is no substitute for human judgment and liability, when it is not possible, even a posteriori, to know if a particular decision was the right one.

A variety of decision aids can be automatically provided to users in this context, which we categorize, inspired from Miller's description of explanations [START_REF] Miller | Explanation in artificial intelligence: Insights from the social sciences[END_REF]:

-Attributes deemed most important in the decision (inputs) -Scoring (computed by deterministic rules, deduction) -Comparable cases and their outcomes (nearest neighbors, induction) -Decision tree branch and weights (probabilistic deduction).

-Counterfactuals (abduction). While there may be more types of decision aids, these cover the use cases we have reviewed in the literature that can be implemented generically, without specific tuning for the decision task, for instance creating a custom visualization. Finally, the presentation of these decision aids may influence their usage. They may be provided as plain recommendations, inciting the user to follow them, or available only on request, after a delay, or even after the decision has been made, as a verification step.

Our goal is to assess if decision aids improve the decision-making process, moving it towards a definition of rational decision-making suitable to our context. For now, we focus on performance metrics:

-Can we provide a methodology to measure decision-making performance in our contexts of use?

-Can the combination of human and algorithmic aids outperform both the human and the algorithm taken alone? How can we reach this stage of human-machine "collaboration"?

-Machine learning biases are a major concern. Even if this human-machine collaboration is an improvement, it seems inevitable that underlying algorithmic biases may taint decisions. Various cognitive biases may interfere. Can we identify and separate those to design correction strategies targeted at each of them?

To address those questions, we propose a model, as a set of metrics, to define performance, various biases and resistance, and carry an experimental study on a simple decision task meeting our requirements to assess the capacity of the model to discriminate various presentations modes. The aim is to generalize this study in our contexts and provide continuous monitoring of decision tasks and decision aids. But before, we must acknowledge that there is a large body of literature addressing similar issues, which has guided our research.

3

Related Work

Decision Theory

Understanding decision-making is a full research area in psychology. For a start, there are several positions regarding the notion of "correct" decision. For Rational Decision Theory, a rational choice maximizes an expected utility function [26, p.237]. Non rational theories [START_REF] Gigerenzer | Decision Making: Nonrational Theories, Editor[END_REF] consider effects such as risk aversion or naturalistic viewpoints. Ultimately, these approaches can be reconciled when assumptions are clearly stated [START_REF] Kahneman | Conditions for Intuitive Expertise, A Failure to disagree[END_REF]. We take mostly a Rational Decision Theory standpoint: simple hypotheses are acceptable in our context. We also assume good will: the decision maker and the decision stakeholders (the company) share the same utility. Under stress, pressure, or poor motivation, we may find a divergence, which leads to a complacency bias or decision fatigue [START_REF] Anderson | The Psychology of Doing Nothing: Forms of Decision Avoidance Result from Reason and Emotion[END_REF]. Within decision theory, our work falls into the area of judge-advisor sys-tems [START_REF] Bonaccio | Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences[END_REF]. Although much of the literature in this area is focused on human advisors, we retain the importance of advice presentation on the decision outcomes [START_REF] Logg | Algorithm appreciation: People prefer algorithmic to human judgment[END_REF].

A major lesson of Decision Theory is that performance varies between individuals: experts and novices approach problems differently, and personality traits can have a strong influence [START_REF] Frederick | Cognitive Reflection and Decision Making[END_REF]. Classical cognitive biases such as the order effect can be significant [START_REF] Romanov | The Presence of Order-Effect Bias in Moscow Administration[END_REF]. Finally, the availability of more information does not necessarily lead to better decision-making [START_REF] Dijksterhuis | On making the right choice: The deliberation-without-attention effect[END_REF]. For fast and frugal or other recent approaches [START_REF] Hafenbrädl | Applied Decision Making With Fast-and-Frugal Heuristics[END_REF][START_REF] Gigerenzer | Decision Making: Nonrational Theories, Editor[END_REF], human decision making does not rely so much on risk -when the decision rules and probabilities of outcomes are well modeled -than on uncertainty -where the indecisiveness is not just a consequence of unknowns quantities, but also of unknows on the suitability of the decision process itself -. Hence, making a decision heuristically, based on limited information, may yield better results than paying close attention to possibly irrelevant details. Burton [START_REF] Burton | A systematic review of algorithm aversion in augmented decision making[END_REF] convincingly defends that, by design, algorithmic decision aids operate under models of risk, while humans need to consider the uncertainties of a situation. Consciously leveraging this difference may provide the means to make the best of human and algorithm complementarity.

Decision Support Systems

Algorithmic decision aids have existed for a long time, in slightly different contexts.

Semi-automation/process control: Our early focus was on measuring, and possibly reducing, automation bias, which we felt should occur in our context. A large portion of the literature on these biases focuses on tasks that involve less dedicated attention and analysis than business decisions. For instance, [START_REF] Parasuraman | Complacency and Bias in Human Use of Automation: An Attentional Integration[END_REF] finds attentional deficits at the onset of complacency biases. [START_REF] Bahner | Misuse of automated decision aids: Complacency, automation bias and the impact of training experience[END_REF] identify this bias in process control tasks that involve verification rather than true decision making. Automation bias is clearly related to complacency bias. It can also result from attention deficits, which are even more prevalent in assisted driving tasks [START_REF] Endsley | From Here to Autonomy: Lessons Learned From Human-Automation Research[END_REF]. [START_REF] Alexander | Why trust an algorithm? Performance, cognition, and neurophysiology[END_REF] finds a conformity bias: use by others increases trust in an algorithm. Finally, [START_REF] Gombolay | Decision-making authority, team efficiency and human worker satisfaction in mixed human-robot teams[END_REF], [START_REF] Yetgin | Complacency and Intentionality in IT Use and Continuance[END_REF] and [START_REF] Alberdi | Why Are People's Decisions Sometimes Worse with Computer Support?[END_REF] describe a major cause of biases: when the algorithm outperforms the human most of the time, motivation necessarily dwindles. Conversely, [START_REF] Onnasch | Crossing the boundaries of automation-Function allocation and reliability[END_REF] provides some evidence that decision aids lose all usefulness when they provide less than 70% accuracy, which indicates that providing those in generic business processes requires some prior assessment of relevance.

Recommender Systems are algorithmic decision aids, but the tasks they support does not meet our focus: they don't help making a choice among predetermined outcomes, and decision quality is elusive: we often assimilate decision performance with user satisfaction [START_REF] Knijnenburg | Explaining the user experience of recommender systems[END_REF]. Several types of cognitive biases can interfere, such as exposure bias [START_REF] Khenissi | Modeling and counteracting exposure bias in recommender systems[END_REF]. Interestingly, recent literature in decision theory for recommender systems focuses on algorithm aversion, the opposite of automation biases: subjects avoid following recommendations, even when the algorithms perform better than humans [START_REF] Burton | A systematic review of algorithm aversion in augmented decision making[END_REF]. The literature stresses the need to provide explanations [START_REF] Yeomans | Making sense of recommendations[END_REF][START_REF] Rader | Explanations as Mechanisms for Supporting Algorithmic Transparency[END_REF], especially for expert users [START_REF] Knijnenburg | Each to his own: how different users call for different interaction methods in recommender systems[END_REF]. We also notice that trust in the system degrades when bad recommendations occur [START_REF] Prahl | Understanding algorithm aversion: When is advice from automation discounted?[END_REF], or when the task is highly subjective [START_REF] Castelo | Task-dependent algorithm aversion[END_REF]. These findings should apply to our context of use.

Visual Analytics for decision making is an entire class of algorithmic decision aids. Visualization tools are geared towards exploratory analysis, which involves open decisions [START_REF] Dimara | Conceptual and methodological issues in evaluating multidimensional visualizations for decision support[END_REF], thus is not in our focus. Still, work on identifying and reducing cognitive biases, such as the attraction effect [START_REF] Dimara | Mitigating the Attraction Effect with Visualizations[END_REF] is relevant. More significant, the identification of the numerous biases [START_REF] Dimara | A Task-Based Taxonomy of Cognitive Biases for Information Visualization[END_REF] that may be found in visual decision aids provides a useful guiding framework.

Impact of algorithmic decision aids on work practices, and ethical considerations

Medical decision support systems include decision aids and are extensively studied. Once again, they do not enter our scope, as the type of decision aids they provide are highly customized for specific purposes, and a medical decision can hardly qualify as a choice among predetermined possible outcomes. They support critical decisions, which explains why decision aids may be met with suspicion and some level of algorithm aversion [START_REF] Cabitza | Biases Affecting Human Decision Making in AI-Supported Second Opinion Settings[END_REF]. Even in successful propositions [START_REF] Cai | Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making[END_REF], suspicion may not arise from the tool itself, but from the way it transforms work practices, perhaps for the better, but in directions which open avenues for high uncertainty [START_REF] Char | Implementing Machine Learning in Health Care -Addressing Ethical Challenges[END_REF]. At this point, addressing both algorithmic and cognitive biases in decision aids reaches beyond the scientific undertaking, into the realm of professional ethics. Determining the role of algorithmic decision aids requires a rigorous assessment of their power and their limitations in benefiting society.

From scientific to ethical considerations. Health professions are not the only industry questioning the impact of decision aids on work practices. Morris [START_REF] Morris | Ethical Obligations and Decision-Making in Accounting: Text and Cases, Chapter 2: Cognitive Processes and Decision Making in Accounting. 4 th Edition 2017[END_REF] devotes a whole chapter on cognitive biases in decision-making for accounting, and how overcoming those biases is an ethical issue. Legal and administrative professions raise similar concerns [START_REF] Coglianese | Regulating by Robot: Administrative Decision Making in the Machine-Learning Era[END_REF], including how proper explanations improve the perception of legal decisions [START_REF] Binns | It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions[END_REF]. Decision automation also produces surprising adaptive behaviors to circumvent the loss of control associated with algorithm-driven activities. For instance, [START_REF] March | The Behavioral Economics of Artificial Intelligence: Lessons from Experiments with Computer Players[END_REF] shows that human players learn to control computer players in computer games. Kyung Lee [START_REF] Kyung | Working with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers[END_REF] describes how Uber drivers use collaboration to understand and regain some level of control over the dispatch algorithms. One of the possible shortcomings of algorithm-driven decision making (and decision aids) is the potential to induce unwanted behaviors by reverse engineering the decision aids' logic: the decision maker becomes 'controlled' by the subjects impacted by those decisions.

To address these concerns, expert groups and regulatory bodies provide guidelines [START_REF] Hleg-Ai | Ethics guidelines for trustworthy AI[END_REF][START_REF] Amershi | Guidelines for Human-AI Interaction[END_REF] to design trustworthy decision support systems. But in our context, we need more than design guidelines and regulations. We need metrics, and possibly a partially automated method to ensure those metrics stay within safe boundaries in the variety of contexts were algorithmic decision aids will be generically embedded in business decision support systems. As a first step towards this ambitious goal, we propose some metrics applicable to business contexts, and we have conducted an empirical study on a simple task to assess their discriminatory power, in the hope of generalizing them to real-world tasks.

A performance model for decision-making

In machine learning, performance is most often synthesized by the Fβ score, where the β term translates the cost differences between false positives and false negatives, and F is the harmonic mean of the precision and recall of the classifier. In a given situation, the equation should be weighted by the expected occurrence ratio of positives and negatives, or other priorities, such as keeping decisions homogenous rather than maximizing value. In our HCI context, the only parameters we may control are the error rates, and thus, we will assimilate performance with a strict reduction in the error rates, while neglecting, for now, 𝐶 𝑡 .

Decision aid effectiveness & collaboration

Because we are interested in measuring the performance of decision aids, we define first a measure of human-algorithm collaboration. This is quantified as the ratio: M1 = performance with a decision aid / max (performance without a decision aid, performance of the classifier).

This ratio depends on a lot of factors:

• If a classifier clearly outperforms humans, there is little chance for this ratio to be > 1, full decision automation is most likely the best solution. • Conversely, a classifier will likely be useless when its accuracy is < 70% [START_REF] Onnasch | Crossing the boundaries of automation-Function allocation and reliability[END_REF].

• If the classifier and the humans tend to make the same type of mistakes, typically because they leverage the same information, then there is no reason for M1 to be above 1. • Finally, when the internal decision model (of the user) and the computable decision model (algorithm) leverage different sources of information, or prioritize the components of expected utility differently, we may find a measure above 1, demonstrating a collaboration effect.

Automation bias & resistance

Collaboration is easy to assess, explain, and it provides a nice metric to guide augmented Business Decision system design. It tells however little about how decision aids influence decision-making. We need to consider how a wrong recommendation influences the subject to let them lose their rationality, i.e. Automation bias, which can be one of two sub-categories:

• Authority bias: the subject follows (wrongly) the algorithm instead of their own reasoning because they perceive themselves as less accurate • Complacency bias: the subject follows (wrongly) the algorithm out of a lack of motivation. While we cannot distinguish the reason for the presence of an automation bias, we may still define it as the probability that a subject who sees a wrong recommendation will make a non-rational decision, considering he would have made a rational decision had he not seen this recommendation. We can define a dual measure of resistance as the probability that the subject makes a rational decision when given a bad recommendation, knowing that if the recommendation had been correct, he would have made a rational choice.

More rigorously, we can define a linear panel model [START_REF] Frees | Longitudinal and Panel Data: Analysis and Applications in the Social Sciences[END_REF]. Subjects are indexed by i, αi corresponds to individual effects, while decision number are indexed by t. Y'i,t = 1{response of(I,t) is rational}. We compare the control group with the treatment group that received wrong recommendations: Xi,t = (1, 1{false recommendation}) T . The linear panel model Yi,t is defined as: With this panel model, automation bias is defined as: While resistance is defined as:

Estimating those probabilities can be performed with pooled ordinary least squares (pooledOLS) when performing a between-groups study (individuals are randomly assigned to the control and treatment groups and trials are randomly distributed). Other biases and effects, such as decision fatigue, order effects, timing effects, expertise effect, can be measured with the same econometric tool, provided our experiments record the appropriate information. To the reader unaccustomed to econometric tools: in a trivial panel, B is simply the extra error rate introduced by false recommendations, while C is the error rate when bad recommendations are given over the error rate when no recommendation are provided.

The goal of our research is to embed this evaluation model in various business decision systems, to automatically assess the proper decision aids to provide users, on a case by case basis.

Study

Before we may consider embedding those metrics in an augmented BPM system, we need to assess their effectiveness and discriminatory power. To this effect, we have conducted an online study based on a real use case but gamified so as to attract a large and varied population of subjects. Our study relies on a simple decision task: some information describing a case, a choice to be made among predefined possible outcomes, a right choice that depends on some rules and heuristics presumed known to the user (with an acceptable degree of variability). We test a single decision aid, presented as a recommendation from a generic classifier, under a variety of presentation modes. Sometimes the recommender will be misleading. We want to measure the success rate of the subjects, the impact of "wrong" recommendations as well as other measures such as decision fatigue or time taken to reach a decision. The deviation from rationality is attributed to an automation bias which denotes that the subject choses to follow the recommender over their own reasoning or intuition. Once we have obtained measures for a control condition (without decision aids), we provide decision aids with various presentations and observe the performance variations.

Choice of a decision task to evaluate

The task inspiring our use case is a fraud detection task: in large financial institutions, tens of thousands alerts on transactions are raised everyday as potential frauds. Scores of analysts review each of these alerts for further inspection, spending an average of 16 seconds on each alert. ML tools are currently being studied to help with this task, but actual deployment raises reliability and regulatory issues. Proper performance measures are needed to determine the optimal combination of algorithm and humans to handle this alert filtering. Resorting to a real-world task to define and assess our metrics is difficult: it requires extensive domain knowledge, and access to many expert users. Early pilots aimed at presenting a simplified version of the real task with a tutorial failed to provide the type of engagement we thought was needed: subjects would fall for a type of "impostor's" syndrome, and blindly follow recommendations rather than give a chance to their own intuition and their own internal decision model.

Others have followed this path [START_REF] Green | The Principles and Limits of Algorithm-in-the-Loop Decision Making[END_REF] and failed to demonstrate nuanced results regarding the possibility of actual complementarity of human and algorithm decision processes.

Instead, we propose a simple decision task, for which a large population can create their own internal decision model easily, and will therefore have some autonomy in deciding whether to trust their judgment or the algorithmic recommendation. We leverage the well-known Titanic dataset1 , a database of passengers on the Titanic ship that sunk in 1912. This dataset is widely used to teach classification algorithms, because it exhibits some obvious patterns: most women in 1 st and 2 nd class survived, while most men in 3 rd and 2 nd class died, and other attributes such as number of relatives on board have a significant but lesser influence on the fate of each passenger. Simple machine learning classifiers, as well as humans after a short study of the data, exhibit success rates of 70-80% in guessing correctly if a passenger survived or not.

The decision task consists in, upon being presented with a passenger's information, choosing if it is more likely to have survived or died. The task is repeated 20 times.

To create an incentive, the presentation is somewhat gamified: subjects are enticed to maximize their score of correct guesses. Unbeknown to the subjects, the passengers presented all follow the expected distribution of survivors: a logistics regression classifier has >70% chance of correct classification on those passengers. Hence, we are asking the subject to make a rational choice -maximizing their probability of scoring high -, not a chance guess. This means reaching a perfect score is possible and even likely. Obtaining less than 50% (less than random chance) is a sure sign the subject is not properly committed to the task.

Stimulus presentation

The experiment starts with a few demographic questions: age range, level of studies and type of studies (humanities, business, engineering/science or other). Then the subject is presented with the goal, as well as some interactive visualizations (treemap) that let them create their internal decision model. We do not present explicit decision rules so as not to taint subjects (Figure 1). Next, we introduce the task, indicating that the recommender (in the experimental condition) has about 76% success rate of guessing correctly. Then, we present the stimuli (Figure 2). In accordance to the stated success rate of the recommender, 5 times in the run of 20 trials, the recommendation is wrong: it says "survived" when the subject has died or vice-versa. Stimuli in the experimental condition: passenger data, followed by a recommendation (dies or survives), and, on the right, 2 buttons "survives" and "die". In the control condition, the recommendation panel is not shown.

Finally, after 20 trials, we ask a few experience questions: estimated success rate, did they choose intuitively or using self-made rules (or don't know), how many times they think the recommender provided the wrong answer, and a free comment box. Finally, we provide their score, and an invitation to an event that presented the early results. We set a cookie on their browser so they can't repeat the experiment for a few hours, so as not to taint our data collection with repeated trials by the same subject.

Second run: presentation variants

The first run was meant to compare a control condition (no decision aid) with an experimental condition. In a second run, conducted two weeks later on a different population, we tested 5 different presentations:

• Control: the same presentation as the first run • Optional display: Instead of displaying the classifier result, the user has to click on a button "show recommendation" to display its result. • Forced acknowledgement: the subject is presented with the passenger data, then must click a button to make the recommendation appear, and only then, after a small delay, they enter their choice. • Reminder of 75%: instead of being only stated at the beginning, the subject is reminded that the decision aid has a 75% chances of success next to the recommendation. • 80% success rate: instead of "guessing wrong" 75% of the time (5/20), the decision aid "fails" only 4 times out of 20. Of course, this is stated to the user, so it should not impact a rational behavior.

Each subject was assigned to the same condition for all their trials (between-groups design). The goal of this run was to assess how different presentation strategies and recommender reliability affect the measures of performance, authority and resistance (defined below) in a significant way. This would be a strong indication that our metrics can be generalized to other context to drive the design of augmented business decision support systems.

Subjects recruitment and filtering

After a pre-run to calibrate expectations with ~20 subjects, we recruited subjects online. We did not want to use a survey service such as Amazon Mechanical Turk as we felt the subject's engagement would be distorted by a financial reward. Instead, we presented the experiment as a "fun and useful challenge". We announced the challenge on a variety of venues, starting in the company and student's forums and slowly extending our call to a wider audience, such as focused reddit and facebook groups, over a 2 months span. 75% of the incoming participants completed their trials run, which we take as indicative of the motivation we had managed to induce from our anonymous subjects (Figure 3). A few participants [START_REF] Binns | It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions[END_REF] either failed to grasp the task or wanted to introduce noise and had less than 50% success (less than random chance), and we discarded them. In total, the first run had 231 participants and 155 usable trial runs, the second had 302 participants and 250 usable trial runs. The demographics reached is a mix of students and educated professionals, with more of an engineering/science background, roughly equally distributed in the 20-55 age range (Figure 3). 6 Results

Decision aid effectiveness

The first run shows a significant collaboration effect: subjects in the control condition obtain a score of 72.3%, while subject in the experimental condition (with decision aid) have a 76% success rate, giving M1 = 1.014. While this improvement may appear modest, it is statistically significant, and it should be reminded that our experiment is tuned to obtain average scores in the realistic range of 70%-80%. Finally, in the condition where the recommender has a 80% success rate, we have a coefficient of 0.7822 (p-value < 10 -4 ). Hence the collaboration effect disappears (M1 = 0.977), indicating that automatic classification would be better suited to the task. While we have not tested a recommenda1tion with 70% or less success rate, we can assume from [START_REF] Onnasch | Crossing the boundaries of automation-Function allocation and reliability[END_REF] that we would find a similar negative impact on the collaboration.

These measures of M1 allow deciding which type of presentation and decision aid to choose for a given task and a given effectiveness of the algorithm. However, the cost of wrong decisions, particularly if they involve an algorithmic bias, may not be symmetrical: false positives and false negatives may have different costs in different scenarios. Hence, the measure of biases is important to assess the full cost/benefit analysis of choosing appropriate decision aids.

Quantification of the automation bias

Coefficients β1 and β2 of the panel model are significantly nonzero at 5%. We can therefore reject the hypothesis that displaying a recommendation has no effect on the rationality of subjects. The model provides us with a metric that can be applied to various groups of the pool of subjects and compare their relative rationality (B(αi), 0= little influence) and resistance (C(αi), 1=maximal resistance). We can apply this model to study trends between different demographic classes recorded at the start of each run or presentation variants, between individuals, between trials (passengers) or any other available criteria. For instance, we display here the authority bias and resistance by type of studies (Table 4).

Study type

Authority While the bias and resistance differences are small, those measures can be useful to apply to varying levels of expertise on a real task. Data and detailed results are available in the supplementary material.

Comparison between presentation variants

To choose the most effective presentation mode, depending if our goal is to maximize collaboration effectiveness, or to minimize authority bias while maintaining a high collaboration effectiveness, we compare the distributions of success under several conditions in Figure 4. In Figure 4, we see success rate in the control condition as a horizontal line of blue marks, as a reference. Also for reference, the continuous horizontal line marks the "success rate" of the algorithm taken alone. The black marks indicate the success rate of the presentation mode. If the black marks are above both the blue marks (human alone) and the line ("computer alone"), then we can say the decision aid is effective in improving decision performance. This happens in all conditions but the last one, where the recommender has a much higher "success rate" of 80%. Based on this figure, the best performance is achieved with the "Forced acknowledgement" presentation mode (the subject must click to see the recommendation, then they can make their decision), although other presentations are quite close. This is the same information as tables 1 and 2.

But more importantly, the red marks show the distribution of success with a wrong recommendation. We can see that in the "optional display" presentation mode, the authority bias is weaker. This suggests using the presentation mode "optional display" over "Forced acknowledgement" if one is particularly concerned about avoiding underlying algorithmic biases.

6.4

Qualitative feedback Time spent on this experiment is not relevant enough to justify elaborate statistics: we needed a task that could be completed fast to reach a large population and match our target use case of alarm filtering. Aggregated data shows that, by and large, our assumptions hold. The average time to answer a trial is 10.3s in the control condition vs. 9.5s in the 1 st experiment condition, with medians at 6.3s and 5.4s respectively, suggesting a small performance improvement with the decision aid. In the experiment condition, there is a slightly longer average time (9.6s vs 9.1s) when the recommendation is wrong than when it is correct, suggesting that subjects perceive the need to reflect when presented with a counter-intuitive proposition. Performance time by demographic category does not vary much, and finally, the time spent on the experiment influences very little the success rate (Figure 5). Subjects feedback. Subjects showed a variety of reactions. Some explained their reasoning: "I only looked at sex and class, taking more things into account was too confusing", "The pattern is pretty obvious. First class was a high chance of survival. Female is a high chance of survival. Children did better than adults. I didn't trust that the algorithm would do better than saying survive for 1st class and female.". Others detailed their frustration at various constraints we had voluntarily set for the task: "I wish the algorithm had provided me with some explanation about its recommendations. Typically, when I disagreed with the recommendation, I would have loved to ask "why do you recommend this?"", "If the algorithm is only 80% accurate, why show us the algorithm answer before we make our decision?", "I would have needed a few explanations on how the algorithm works before the experiment and while doing it some feedback on how it decided would have been helpful.", "I would have preferred to answer with probabilities rather than a binary choice.", "I would be interested to know the AI learning method".

Finally, many showed an understanding and appreciation for the study, noting that wrong recommendations could indeed affect their judgment: "Interesting experiment. Would love to see the end study!", "Funny (not topic, itself) and interesting", "AI = Random?", "I am very curious to understand the analyses process and the results.

Would it be possible to receive the paper when published? Thanks". In debriefing interviews, subjects indicated that the test had made them aware of the complexity of the thought process at play when deciding to trust an algorithmic recommendation, which indeed, was the primary motivator of this study.

Discussion

Limitations

Our study results align with [START_REF] Onnasch | Crossing the boundaries of automation-Function allocation and reliability[END_REF], [START_REF] Gombolay | Decision-making authority, team efficiency and human worker satisfaction in mixed human-robot teams[END_REF], [START_REF] Yeomans | Making sense of recommendations[END_REF] and [START_REF] Alberdi | Why Are People's Decisions Sometimes Worse with Computer Support?[END_REF]. Taken together, they suggest that decision aids are useful only when the "algorithm alone" success is within a certain range, which we can roughly estimate as [70%..human success rate + constant]. Still, our measurements on this task may not be generalizable to other decision tasks. The causes of the uncertainty in a decision-making task may vary widely, the bias patterns may equally vary. Our contribution lies in the metrics of collaboration, automation bias and resistance, how to assess them and put them in production for our specific context, it is not a contribution to Decision Theory.

The task we have evaluated is not an expert task, it is more comparable to a routine managerial decision than to a complex decision such as medical diagnosis. Even though the metrics we have defined can be applied to those more complex contexts, experimental setup and access to many experts should make this very difficult, justifying more longitudinal approaches such as [START_REF] Cai | Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making[END_REF]. As mentioned in the introduction, our focus is in providing generic decision aids in the context of business decisions, and our task matches this context.

Finally, the effects we have observed, while statistically significant, may seem quite modest. We have shown a significant advantage of augmented decision-making in certain circumstances, and a significant difference in several presentation modes to contain, or, on the contrary, increase, automation bias and resistance. The small amplitude of those effects is only a consequence of the narrow window in which augmented decision-making has its usefulness. As our results suggest, when the algorithm clearly outperforms humans, it is probably better to rethink the role of the human. Conversely, when the algorithmic aid is of poor relevance, it is likely not useful, and, on the contrary, may lead to unwanted propagation of algorithmic biases through authority or complacency cognitive biases. This, by the way, has been confirmed a posteriori by the collaborative design sessions of the fraud detection workflow that inspired our study.

Towards a methodology and embedded measures of performance in augmented decision making

Now that our model has shown its explanatory and discriminatory power, our next goal is to apply the model we have defined to real usage scenarios, first in the use case described before: fraud alert detection. Another important direction is to apply our methodology to other decision aids, such as nearest-neighbors' methods, that shows the data and outcomes of cases closely related to the present case.

In the longer run, we envision that decision aids will be generalized in business decision systems, provided they are instrumented with tooling that continuously assesses their relevance and performance so as to mitigate risks while improving the productivity and quality of decision-making.

Conclusion

We have presented a model, as a set of metrics to include in A/B testing of decision aids used in business decision tasks to assess their usefulness and control the biases a particular decision aid and its presentation may induce on the decision maker. Applied to a simple decision task, our metrics can be used to show the possibility of humansystem collaboration (72% of success for humans alone, vs 76% for a human assisted by an algorithm that gives the correct answer 75% of the time). We have also defined measurements of automation biases and resistance to this bias. Applied to our decision task, we found a significant effect of wrong recommendations on the rationality of subjects (-5%), indicating that underlying algorithmic biases in the decision aid may be propagated in the decision process instead of compensated by the human.

Testing several presentation variants, we found that a technique that presenting the recommendation only on request (optional) was effective in increasing the resistance of the subjects, all the while preserving the performance of decision-making.

Our measurement system is meant to be embedded in A/B testing of generic decision support system used in many contexts of business decision management systems and business processes. Augmented decision systems shift part of the responsibility of the decision maker to the system designer. Introducing systems that may induce authority, complacency or other cognitive biases creates liabilities for the system designers. Therefore, tools to objectivize the performance and impact of biases are needed to alleviate this liability, replacing impressions and subjective design guidelines with objective measures. We believe furthering this work is important to shape the future of augmented business decision-making.

Fig. 1 .

 1 Fig. 1. interactive visualization (treemap), helping the user form an internal decision model of who predominantly survived on the Titanic: Females from 1 st and 2 nd class, and who tended to die: males of 2 nd and 3 rd class.

Fig. 2 .

 2 Fig.2. Stimuli in the experimental condition: passenger data, followed by a recommendation (dies or survives), and, on the right, 2 buttons "survives" and "die". In the control condition, the recommendation panel is not shown.

Fig. 3 .

 3 Fig. 3. Distribution of the subjects by age range, years of study after high school, and type of study (self-reported).

Figure 4 :

 4 Figure4: comparison of success rates with various stimuli presentations (black marks), success rates with wrong recommendation (red marks), success rate with good recommendations (green). Blue marks indicate the success without decision aid, and the line represent the "success rate" of the algorithm alone.

Figure 5 :

 5 Figure 5: 2D plot time spent x success rate.

  Our target users, actuaries, financial analysts, budget planners and managers are more accustomed to a simple cost model, that includes cost of processing, and follows an equation similar to (for a binary decision): P = (1-𝐸 𝑛 ) 𝐺 𝑛 + (1-𝐸 𝑝 ) 𝐺 𝑝 -𝐶 𝑝 𝐸 𝑝 -𝐶 𝑛 𝐸 𝑛 -𝐶 𝑡 Given a cost matrix: 𝐶 𝑝 : 𝐶𝑜𝑠𝑡 𝑜𝑓 𝑚𝑖𝑠𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝑎 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝐶 𝑛 : 𝐶𝑜𝑠𝑡 𝑜𝑓 𝑚𝑖𝑠𝑐𝑙𝑎𝑠𝑠𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝑎 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝐺 𝑝 : 𝑃𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑖𝑑𝑒𝑛𝑡𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝑔𝑎𝑖𝑛 𝐺 𝑛 : Correct n𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝑖𝑑𝑒𝑛𝑡𝑖𝑓𝑖𝑐𝑎𝑡𝑖𝑜𝑛 gain 𝐶 𝑡 : 𝐴𝑣𝑒𝑟𝑎𝑔𝑒 𝑐𝑜𝑠𝑡 𝑜𝑓 𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 (typically human time and amortized time to develop the decision aid solution). And some error terms: 𝐸 𝑛 , 𝐸 𝑝 ..: 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑒𝑟𝑟𝑜𝑟 𝑟𝑎𝑡𝑒 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑟𝑜𝑝𝑜𝑠𝑒𝑑 𝑐𝑜𝑛𝑓𝑖𝑔𝑢𝑟𝑎𝑡𝑖𝑜𝑛

Table 1 :

 1 decision aid effectiveness (first run result)

		coefficient	95% confidence interval
	Control condition (human alone)	0.7230	[0.6948, 0.7512]
	With decision aid	0.7604	[0.7530, 0.7682]
	"Algorithm alone"	0.75	--
	This collaboration effect is most useful to compare decision aids presentation syn-
	thetically. Our second run shows some modest but consistent variations:
		coefficient	M1
	Control condition (human alone)	0.7230	1
	With decision aid (new run)	0.7651	1.020
	Optional display	0.7655	1.020
	Forced acknowledgment	0.7660	1.021
	Reminder of 75%	0.7619	1.016

Table 2 :

 2 measures of collaboration for various presentation modes of the recommender

Table 4 :

 4 authority bias and resistance for different demographic groups.

			bias	Resistance	95% conf. B(αi), 95% conf. C(αi)
		B(αi)		C(αi)
	Engineering/science 0.0666	0.8581	[0.0626, 0.0705] [0.8152, 0.9011]
	Business	0.0708	0.8423	[0.0663, 0.0753] [0.7898, 0.8947]
	Humanities	0.0684	0.8471	[0.0641, 0.0726] [0.7970, 0.8971]
	other	0.0737	0.8423	[0.0687, 0.0787] [0.7900, 0.8946]
	Age range	B(αi)	C(αi)	Level	of	B(αi)	C(αi)
				study
	15-25	0.0661	0.8606	2-		0.0688	0.8553
	25-40	0.0687	0.8473	4		0.0681	0.8552
	40-55	0.0663	0.8522	5		0.0657	0.8537
	55+	0.0713	0.8548	8		0.0694	0.8545

https://www.kaggle.com/c/titanic

Acknowledgments

We thank Grégoire Colombet and François Jaquin for introducing us to their client's decision problems that led to this research work. We also thank Pranivan Baudouin and Christopher Dolloff for their precious assistance along this project.


Stephane Dupas
email: stephane.dupas@ird.fr

Cesar Alejandro Antolinez

miCampoApp, a Participatory Research Tool for Agroecology

Keywords: agroecology, participatory research, web application, traceability

Agroecology, defined as the ecological science of food production is also as practical approach to design food production systems based on local concerted solutions that aim to promote synergy among the diversity of human and non human food systems elements. These two facets makes agroecology a good candidate for participatory research. Information technology should help using this information for the production of structured scientific knowledge. In this respect, there is a need for information technology that is adaptive to encompass the diversity of within and between systems and that provide benefit to farmers that feed it with data. We present MiCampoApp, a webapp that aims to join participatory research and certification in agroecology, with two roles, the farmer and the administrator. The idea to join participatory research for decision support with certification for market differentiation in single system is that much of the certification data harbors information for scientific research, and this incentivize data collection for the interest of the farmer and the community. The administrator create a model for data collection in order to solve a research question of interest for a communities or to produce traceability information to feed participatory guarantee system in a community. The farmer collects information using simple icons and produces traceability pages for research or certification purposes.

Introduction

Agroecology, is a discipline and a movement [START_REF] Wezel | Agroecology as a science, a movement and a practice[END_REF].As a discipline is can be defined as the ecological science of food production [START_REF] Francis | Agroecology: The Ecology of Food Systems[END_REF]. Agroecology is also a movement and a set of principles of action aiming to drive the transformation of food system toward sustainability [START_REF] Wezel | Agroecology as a science, a movement and a practice[END_REF]. Among its principles of action is the choice to build concerted solutions, promoting diversity and synergy among ecosystem components, based on local observations and knowledge [START_REF] Fao | Scaling Up Agroecology Initiative Transforming[END_REF]. Agroecological diversification and local solutions therefore harbors high value in terms of participatory research on agroecosystems components and their interactions. Yet putting together this knowledge and the diversity of local solutions, and testing agroecological hypotheses regarding synergies and diversity requires organizing a common agroecological dataset [START_REF]TAPE Tool for Agroecology Performance Evaluation 2019 -Process of development and guidelines for application[END_REF]. There is however a lack of traceability of agroecological practices [START_REF] Warner | Agroecology as participatory science: Emerging alternatives to technology transfer extension practice[END_REF] [START_REF] Warner | Agroecology as participatory science: Emerging alternatives to technology transfer extension practice[END_REF]. This lack of traceability is not only a problem for science, it is a problem of market differentiation. Agroecology local solutions do not necessarily meet certification standards, and costs of standard certification are unaffordable in particular for low income farmers [6? ,7](FAO 2007, Altieri andNicholls, 2008, IFAD-ORG 2004). Agroecology to the difference of organic agriculture is not a production system but a set of principles to build cooperative diversified and resilient food systems. It is therefore difficult to standardize data to be collected to certify agroecological farms and products. In order to make certification and traceability more affordable to small holder, and to assess agroecology principles, participatory guarantee systems have been promoted [START_REF]Participatory Guarantee Systems (PGS) for sustainable local food systems[END_REF] and developed from users, based on sharing ethical values and certifying by the active participation among farmers or between farmers and consumers rather than indirectly and independently form third party company [START_REF] Ponce | Aporte potencial de los sistemas participativos de garantía (SPG) a la soberanía alimentaria: caso SPG Caldas 2016-2017[END_REF]. This partially reduces the cost of agro-ecology certification but still demands organizational costs. These two issues of participatory research and market differentiation agroecology can share the common objective of identifying indicator variables linked to sustainability of food systems. The possibility to collect and share information by the internet can be seen as an opportunity to address traceability problems [START_REF] Karippacheril | Global Markets, Global Challenges: Improving Food Safety and Traceability While Empowering Smallholders Through ICT[END_REF]. Yet there is a need for information systems that allow a diversity of production and transformation, distribution and consumption models to be implemented and traced for market and research purposes. There is a need of a data platform able to adapt to each agroecological system and practices to combine agroecological principles and certification needs. We propose in this article a web application, a generic tool that permits the user to develop its own data collection system with uploaded icons to simplify interface to be able to feed with participatory data that can be tagged and published in traceability web pages links for research evaluation and market differentiation. The app includes modules for sensors and prediction of risks.

Web app functionalities

The web app exchanges information between tree user types, administrator that create models, the farmers and the public (Figure 1).

Administrator functionalities

The administrator (Figure 2) can log in to the web page and,

1.

Edit his profile 2.

Create data models ("Cultivos"), categories within data models ("categorias"), and create variables within categories. Available variable types are : Boolean, numeric, text, percent, range, categories selection, categories based on image selection, categories associated with quantities. Data models, categories, and variables are attributed simple icons uploaded to the system. The system count with a large number of icons that allow to represent agroecological information in its different dimensions of practices, crops (89 crops are graphically represented), farm characterization for certification, and crop pest and disease observations for research. Each variable is associated with help pages (links, images, text alimented by administrator) to accompany users data collection.

3.

Activate or deactivates variables and categories so that users can or cannot collect data for these variables.

4.

Generate TAGs ("Etiquetas") and add these tags in the variable edition panel. TAGs are selection of variables that can be published as web pages by users for traceability purposes (research, or market differentiation). 5.

Create models of sensors with the format of data transmission that can be used by users to connect IOTs to the system . 6.

Activate or desactivate sensors registered by users. 7.

Generate posts in the BLOG web pages for the system that can be used as help pages for users of the system. And activate or desactivate web pages of any user. 8.

Provide registered user administrator role Register as user 2.

Register and edit its production unit ("Finca") (Figure 3)

3.

Register and edit its functional units ("Parcela") (Figure 4)

4.

Visualize, edit and create (Figure 5) production cycles as defined as data models by administrator. The following models are already available:

(a) Crop production cycles. They include data on management practices associated to these crops (planting, hilling, pruning, spraying, harvesting...), and observations including lists of principal pests and diseases observed on these crops.

i.

The vegetables associations (hortalizas) cycles (Figure 6) ii.

The cocoa (cacao) cycles (Figure 7). The cacao model aims to collect information of cacao crops and contains tags for certification and evaluation of practices. Cacao certification costs is a major limitation for improving practices and have access to appropriate market prices in south american countries (IFAD-ORG 2004). iii.

The potato (papa) cycles (Figure 4c). The potato model aims to collect information of potato crops and epidemiological modelling. Potato crops are under strong epidemiological pressures that leads to overuse of pesticides. The models to predict risks of disease have been developed on temperate countries. We published in the web site prediction on potato blight units to aid farmer reducing pesticide use (agromakers.org:3838/interactiveBlightMap/). The potato cycle is aimed at improving this model through participatory science. iv.

The canna (achira) cycles (Figure 9), allowing to include transformation tasks. Our tool aims at alowing to build data systems for certification on demand. We implemented a good farming practices (Buenas Prácticas agrícolas) cycle. It include 3 categories : category risks evaluation (peligros) includes icons for climatic risks, chemical risks, physic risks and biological risks evaluation ; category planification (planeación) includes icons for registering certification for water access, existence of risk map, water plan, aptitudes analysis, historical record of soil use; category infrastructures (instalaciones) includes caracterization of areas for workers, inputs storage, input preparation, product gathering, waste management. Add sensors that can be connected to the data base. 6.

Generate traceability webpages on from cycles using variables selected in the tags. These can have certification, research or logistic purpose. The variables include in the tag 7.

Generate BLOG web pages to produce more personal information on their production unit, their practices or other relevant information for the community.

Code source

The web app is developed in codeIgniter 2, available at github . upon request The current version is in spanish.

Research protocol for participatory epidemiological modeling of potato blight

We present a research protocol to use the web app for participatory research on potato blight epidemiological modeling for decision support on reduction of pesticide use. The irrational use of pesticide to control potato blight leads to major public heath problems in the Andean region [START_REF] Pradel | Use of the environmental impact quotient to estimate impacts of pesticide usage in three Peruvian potato production areas[END_REF]. Modeling can be used in temperate countries to develop early alert systems and reduce pesticide use ( [START_REF] Small | Development and implementation of the BlightPro decision support system for potato and tomato late blight management[END_REF]). Other simple decision support can be used to accompany farmers decisions based on simple observations ( [START_REF] Pérez | A simple, hand-held decision support designed tool to help resource-poor farmers improve potato late blight management[END_REF]). However, such models may be misleading depending on microclimatic conditions, and when transferred to tropical regions due to differences in practices and climatic seasonality not accounted in the models ( [START_REF] Batista | Validation of decision support systems for tomato early blight and potato late blight, under Brazilian conditions[END_REF]). There is therefore a need for adaptation of models. The potato cycle and its traceablity pages (Figure 8) can be used in this purpose. This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation as well as the experimental conclusions that can be drawn.

Discussion

The web app is considered as an advance in agroeoclogical science and traceability in agroeoclogy since the administrator can adapt data collection system to the diversity of local solutions developed in agroecology, and allow cooperation between farmers and researchers in order to produce scientific knowledge. There does not seem to exist other such tool at the moment the that accomplish this function. This tool would allow to valorize the numerous research experiments performed by agroeocogical farmers. Nevertheless the application is a web app, that requires internet connection. This limits its use in rural due to gap in internet access ( [START_REF] Ziegler | Rural connectivity in Latin America and the Caribbean: A bridge for sustainable development in a time of pandemic[END_REF]). Its use as a traceability tool for market differentiation purpose is still in a research state since the traceability pages lack design work. Small agroecology farming, can have major contribution on the achievement of sustainable development goals (SDG), ( [START_REF] Peterson | Scaling up agroecology to achieve the SDGs: A political matter[END_REF]). The principle of agroecology that most contribution has on SDG, that emerges from the principle of building from local, is the diversification of its products an practices (Millenum Institute 2018). This diversification stabilizes small producers income related to variations in production and market prices ( [START_REF] Rodriguez | Sustainability of Diversified Organic Cropping Systems-Challenges Identified by Farmer Interviews and Multi-Criteria Assessments[END_REF]). Our application aims to account for this diversity linking valorization in research and could be used for market differentiation in an adaptive way.

Acknowledgments

Figure 1 .

 1 Figure 1. Workflow of miCampo App.

Figure 2 .

 2 Figure 2. System administration. The letters represent hyperlinks 1.2. Farmer functionalities The farmer can 1.Register as user 2.Register and edit its production unit ("Finca") (Figure3)

Figure 3 .

 3 Figure 3. Production unit (Finca) registering.

Figure 4 .

 4 Figure 4. Functional unit (Parcela) registering.

Figure 5 .

 5 Figure 5. Cycle selection for editing (Editar), registering (Registros), sensor edition (Sensores), labelling (Trazabilidad), or creating (crear cycle).The letters represent hyperlinks.

Figure 6 .

 6 Figure 6. Vegetables cycle categories with variables within the two first categories, and registering of plant pruning variable.The letters represent hyperlinks.

Figure 7 .

 7 Figure 7. Cacao cycle categories with variables within the three first categories.The letters represent hyperlinks.

Figure 8 .

 8 Figure 8. Potato cycle categories with categories and variables within the categories a, b, c and d.The letters represent hyperlinks.

Figure 9 .

 9 Figure 9. Canna cycle categories with categories.The letters represent hyperlinks.v.Other possible cycles. We created cycles for 84 cultures with their icons.Categories an variables can be added to create traceability tools.(b) Certification is a bottlneck of agroecology scaling (Altieri and Nicholls, 2008).Our tool aims at alowing to build data systems for certification on demand. We implemented a good farming practices (Buenas Prácticas agrícolas) cycle. It include 3 categories : category risks evaluation (peligros) includes icons for climatic risks, chemical risks, physic risks and biological risks evaluation ; category planification (planeación) includes icons for registering certification for water access, existence of risk map, water plan, aptitudes analysis, historical record of soil use; category infrastructures (instalaciones) includes caracterization of areas for workers, inputs storage, input preparation, product gathering, waste management.

Figure 10 .

 10 Figure 10. Certification cycle categories and variables 5.Add sensors that can be connected to the data base. 6.Generate traceability webpages on from cycles using variables selected in the tags. These can have certification, research or logistic purpose. The variables include in the tag 7.Generate BLOG web pages to produce more personal information on their production unit, their practices or other relevant information for the community.

Author Contributions:

  Stéphane Dupas conceptualized the application and wrote the article, Alejandro Antolinez coded the source and mounted the webapp on the server Funding: This work has been supported by IRD EGCE, International mixed laboratoy BIO I NCA, andCGI ARPr Strengtheningtheevidencebase f oraclimateresilientandlowcarbonsmallholderagriculturethroughagroecology Institutional Review Board Statement: Informed Consent Statement:

Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 17 December 2021 doi:10.20944/preprints202112.0292.v1

Journal Not Specified 2021, 1, 0

Data Availability Statement:

Code source is available upon request at upon request Acknowledgments: In this section you can acknowledge any support given which is not covered by the author contribution or funding sections. This may include administrative and technical support, or donations in kind (e.g., materials used for experiments).

Conflicts of Interest: Declare conflicts of interest or state "The authors declare no conflict of interest." Authors must identify and declare any personal circumstances or interest that may be perceived as inappropriately influencing the representation or interpretation of reported research results. Any role of the funders in the design of the study; in the collection, analyses or interpretation of data; in the writing of the manuscript, or in the decision to publish the results must be declared in this section. If there is no role, please state "The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results".