Ismael Rafai
email: ismael.rafai@gmail.com

Zakaria Babutsidze

Thierry Delahaye

Nobuyuki Hanaki

Rodrigo Acuna-Agost

No evidence of attraction effect among recommended options: A large-scale field experiment on an online flight aggregator

Keywords: Digital nudge, E-commerce, Recommender system, Flight booking, Behavioral bias, Attraction Effect Digital nudge, E-commerce, Recommender system, Flight booking, Behavioral bias, Attraction Effect

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

With the proliferation of the Internet, purchases are increasingly made through online marketplaces. These outlets offer consumers a wider variety of products than most brick-and-mortar stores can afford. However, this increase in product range is only beneficial to consumers if they can identify the products they prefer from among the set available. To address this issue, a large body of the information systems literature has been dedicated to the study and improvement of recommender 2 systems [START_REF] Bobadilla | Recommender systems survey[END_REF][START_REF] Lu | Recommender system application developments: A survey[END_REF]. Recommender systems are "programs which attempt to recommend the most suitable items (products or services) to particular users (individuals or businesses) by predicting a user's interest in an item based on related information about the items, the users and the interactions between items and users" [START_REF] Lu | Recommender system application developments: A survey[END_REF]. Recommender systems, thus, aim to assist the users of digital platforms, by identifying and highlighting the alternatives that are most likely to match users' needs.

If consumers were utility maximizers, as assumed by the standard rational choice theory, this matching would be assured by identifying the distribution of users' preferences in order to recommend the products that are most likely to be preferred. However, there is plenty of evidence from economics, psychology and management science arguing that humans do not behave as simple preference maximizers, but use various heuristics [START_REF] Babutsidze | How do consumers make choices? a survey of evidence[END_REF] and are subject to decision biases [START_REF] Tversky | Judgment under uncertainty: Heuristics and biases[END_REF]. These departures from rationality reflect the fact that preferences, or at least their expression through choices, are not neutral with respect to the choice set presentation. Applied to information systems research, the interaction between preferences and the choice presentation should be considered when designing a recommender system [START_REF] Adomavicius | Reducing recommender system biases: An investigation of rating display designs[END_REF]: this interaction could be either limited -to avoid negative side-effects -or on the contrary enhanced, to improve recommender system performance.

Since choices are impacted by the way they are presented, the choice architect -i.e., the designer of the choice environment -may nudge users. By this we mean that the choice designer is able to change "the aspect of the choice architecture [to] alter people's behavior in a predictable way without forbidding any options or significantly changing their economic incentives" [START_REF] Thaler | Nudge: Improving decisions about health, wealth and happiness[END_REF]. This concept of nudges, coming from psychology and behavioral economics, has been recently introduced and is becoming popular in information systems research [START_REF] Arnott | Behavioral economics for decision support systems researchers[END_REF][START_REF] Jameson | Choice architecture for human-computer interaction[END_REF][START_REF] Weinmann | Digital Nudging[END_REF].

Despite digital nudging gaining popularity among practitioners, the vast majority of empirical studies to date were conducted in laboratory settings. Laboratory studies are ideal to prove the existence and assess the internal validity of behavioral biases, and to identify variables and conditions that moderate their effects. However, once internal validity of the bias is assessed, and once it has been replicated several times in the laboratory, it is also important to test if the bias is relevant in the field. Indeed, the relevance of a behavioral bias in the (digital) field cannot simply be assumed as demonstrated by the results presented in this article.

The digital nudge we are interested in, has principally been tested in situations where the researcher can design an ideal environment where all alternatives' attributes are defined and controlled to maximize the likelihood of observing the effect. On the contrary, in the field, many recommender systems cannot control product and service attributes, but rather can only control how they are disclosed/presented. Moreover, in the laboratory, decision makers (i) know they are participating in an experiment, (ii) choose from a very limited number of alternatives, with limited consequences, and (iii) are "captive", in the sense that they do not have many outside options (including the possibility to delay their choices or to choose from another, competing, platform). It is, therefore, fundamental to test the conceptual replicability of lab-identified behavioral biases in the field in order to assess their practical relevance [START_REF] Hüffmeier | Reconceptualizing replication as a sequence of different 29 studies: A replication typology[END_REF].

In this article, we present the first test of the effectiveness of a "digital decoy nudge" in a real recommender system setting. We rely on a large-scale field experiment and observe more than 140,000 searches made on an online flight aggregator, where we are able to manipulate the set of recommended itineraries. The digital decoy nudge we are interested in is based on a decision bias known as the asymmetric dominance effect, or attraction effect (hereafter AE). It consists of recommending one "decoy" itinerary that is dominated by one of the former recommended itineraries (the target), but not by others, in order to increase market share of the target recommendation. Our paper, therefore, contributes both to the literature on digital nudges, and to the literature on the AE. In addition, while many behavioral biases are conceptually easy to understand, operationalizing and testing them in the digital field represents a significant organizational and cultural challenge from a company point of view [START_REF] Gupta | Top Challenges from the first Practical Online Controlled Experiments Summit[END_REF].

As a result, this article may interest companies aiming to implement digital nudges in their recommender systems, by providing an example of how a digital decoy nudge could be operationalized into a search engine.

In our results, we do not detect the presence of the attraction effect on recommended options in the specific setting studied. However, our large dataset allows us to control for session characteristics and (at least partially) rule out potential explanations for the absence of the effect. For example, we found that manipulating the size of the choice set does not affect the size of the attraction effect, nor does the similarity between the target and the competitor options, the stakes of the decision, or the device used (computer vs smartphone) in the search process. While questioning the practical relevance of the effect for recommender systems, additional exploratory results suggest that it could be possible to identify the source of effect heterogeneity in future research, paving the way to the implementation of personalized digital nudges in recommender systems.

The rest of the paper is organized as follows. The next section reviews existing literature on digital nudges and on the AE, discusses the relevance of the effect for recommender systems, and presents our hypotheses. We then present the details of our experiment and its results. The last section concludes with a discussion and a presentation of potential avenues for future research.

Literature review

Digital nudges

In theory, the digital world is particularly suited for designing, testing and evaluating the impact of nudges, since the choice architect can easily manipulate "user-interface design elements to guide people's behavior in digital choice environments" [START_REF] Weinmann | Digital Nudging[END_REF] and can observe nearly immediately the impact of the digital nudge on behaviors. 1 There are many examples of the use of digital nudges. For example, in e-commerce, [START_REF] Adomavicius | Do recommender systems manipulate consumer preferences? A study of anchoring effects[END_REF] report the presence of anchoring effects; [START_REF] Camilleri | The importance of online reviews depends on when they are presented[END_REF] highlights an order effect between reviews and product presentation on purchase intentions; [START_REF] Shen | Facilitating Complex Product Choices on E-commerce Sites : An Unconscious Thought and Circadian Preference Perspective[END_REF] show that introducing a distraction task between products' presentation may facilitate complex product choices by evoking unconscious 1 Originally, in behavioral economics, nudges designated interventions that were intended to improve individuals' welfare, and the term "sludge" was preferred when similar techniques were used to increase a seller's profit by negatively impacting consumers. However, in the information system literature, the term "digital nudge" has been introduced to designate small modifications of the choice environment that influence users' decisions, regardless of their impact on welfare [START_REF] Schneider | Digital Nudging -Influencing Choices by Using Interface Design[END_REF]. thoughts; and [START_REF] Sheng | Effects of Web Atmospheric Cues on Users' Emotional Responses in E-Commerce[END_REF] show that purchase intentions can be enhanced by the presence of web atmospheric cues on an e-commerce website. Similarly, [START_REF] Liu | The dark side of reviews: The swaying effects of online product reviews on attribute preference construction[END_REF] show that online reviews influence the weight of attributes in product evaluation; [START_REF] Aerts | The platform shapes the message: How website design affects abstraction and valence of online consumer reviews[END_REF] show that website design may nudge reviewers to deliver higher quality reviews and [START_REF] Craciun | Discrete emotions effects on electronic word-of-mouth helpfulness: The moderating role of reviewer gender and contextual emotional tone[END_REF] show that the perception of a review is impacted by the presentation of the reviewer's gender. [START_REF] Schneider | When the Stars Shine Too Bright : The Influence of Multidimensional Ratings on Online Consumer Ratings When the Stars Shine Too Bright : The In fl uence of Multidimensional Ratings on Online Consumer Ratings[END_REF] report that the overall rating of a product can be manipulated by proposing multi-dimensional rating systems, where consumers are also asked to rate precise attributes of the product.

In reward-based crowdfunding, [START_REF] Wessel | The impact of sold-out early birds on option selection in reward-based crowdfunding[END_REF] demonstrate that unavailable sold-out early bird options impact option choices. [START_REF] Adomavicius | Effects of online recommendations on consumers' willingness to pay[END_REF] show that the willingness to pay for digital songs is impacted by systems' recommendations. In the context of mobile phone recommendations, [START_REF] Ho | Nudging moods to induce unplanned purchases in imperfect mobile personalization contexts[END_REF] found that hiding products' attributes increases consumers' urge to buy and thus their unplanned purchases, and that this effect interacts with consumers' mood. [START_REF] Kroll | Digital nudging and privacy: improving decisions about self-disclosure in social networks *[END_REF] show that digital nudging can be used to increase self-disclosure in social networks. [START_REF] Sivan | Do search engines influence media piracy? Evidence from a randomized field study[END_REF] report that the position of legal alternatives in a recommender system reduces piracy behavior.2

Attraction Effect

A particular digital nudge on which we focus in this paper is based on a decision bias known as the asymmetric dominance effect, or attraction effect. The AE was introduced by [START_REF] Huber | Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis[END_REF] and predicts the increase of the choice probability of one alternative (the target) caused by the introduction of a new alternative (the decoy) in the choice set. For this to happen, the decoy should be dominated by the target but not by competing alternatives (that is, the decoy is inferior in at least one characteristic and is not superior in any characteristic to the target; but the decoy is superior to other competing alternatives in at least one characteristic), see Figure 1 for an example. The AE has received great attention from both professionals (because of its marketing possibilities) and academics (as it constitutes a major departure from the standard rational choice theory).

Explanations and moderators of the attraction effect

The AE has been widely documented in the laboratory, as well as in survey settings, including purchasing goods [START_REF] Doyle | The robustness of the asymmetrically dominated effect: Buying frames, phantom alternatives, and in-store purchases[END_REF], lottery choices [START_REF] Castillo | The attraction effect and its explanations[END_REF], perceptual tasks [START_REF] Trueblood | Not Just for Consumers: Context Effects Are Fundamental to Decision Making[END_REF], and even animal decision making [START_REF] Shafir | Context-dependent violations of rational choice in honeybees ( Apis mellifera ) and gray jays ( Perisoreus canadensis )[END_REF]. 4 Several explanations of the effect have been proposed, which may be grouped into compensatory and non-compensatory decision strategies. Compensatory strategies assume that individual decisions are based on a trade-off between the product attributes [START_REF] Lee | A Comparison of Compensatory and Non-Compensatory Decision Making Strategies in IT Project Portfolio Management[END_REF]. Within this framework, the AE can be explained by a change in the weighting of the attributes, induced by the introduction of the decoy. 3 From a theoretical point of view, the AE implies a violation of the Regularity condition (the probability of choosing an alternative cannot increase when the choice set is extended), which is incompatible with any random utility model [START_REF] Dasgupta | Regular'choice and the weak axiom of stochastic revealed preference[END_REF].

Indeed, the introduction of the decoy can either (i) extend the range of attribute's values, narrowing the perceived distance between options within the range (range theory) or (ii) increase the number of levels available to define the attribute, increasing its weight in the decision (frequency theory). However, explanations based on compensatory choices cannot explain why individual differences moderate the effect, and in particular, why the AE is stronger for younger individuals, those with lower levels of literacy, cognitive ability, and higher faith in intuitions [START_REF] Kim | The attraction effect in decision making: Superior performance by older adults[END_REF][START_REF] Mao | The attraction effect is more pronounced for consumers who rely on intuitive reasoning[END_REF]. Neither are they able to explain why the AE does not systematically increase with increasing range of the decoy attribute [START_REF] Castillo | The attraction effect and its explanations[END_REF].

In contrast, other explanations of the AE are based on a modification of the decision process: the introduction of a decoy, and the asymmetric dominance relation it creates, offers individuals new ways to make and justify their decisions. These explanations can be related to non-compensatory strategies.

Non-compensatory strategies are based on heuristics rather than attribute trade-offs. For example, a consumer may decide to restrict the choice only to the N alternatives with the lowest price, or with the highest quality, or to the options that are unambiguously dominating other options. In those conditions the introduction of the decoy may imply that a competing alternative, which would have been preferred

to the target if the consumer was using a compensatory strategy, would be excluded from the consideration set altogether. Overall, while there is no single (type of) explanation that can encompass all moderators of AE that have been identified [START_REF] Yang | AN OVERVIEW, ITS FRAGILITY[END_REF], the AE suggests that the decision process does not simply consist in revealing existing preferences. In particular, having strong priors, and the difficulty of perceiving the dominance relationship are two factors that crucially limit the AE.5

Ecological validity of the effect

Recently, some failures to replicate the AE, and the restrictive conditions under which the AE is usually obtained have raised a debate on the ecological validity of these studies. This questions the relevance and economic significance of the AE [START_REF] Frederick | The limits of attraction[END_REF][START_REF] Yang | More evidence challenging the robustness and usefulness of the attraction effect[END_REF][START_REF] Huber | Let's be honest about the attraction effect[END_REF]. To the best of our knowledge, there is only one article that directly operationalizes the attraction effect in the field, with real customers [START_REF] Doyle | The robustness of the asymmetrically dominated effect: Buying frames, phantom alternatives, and in-store purchases[END_REF]. The authors investigated the AE in a local grocery store, by creating dominated baked beans (either by introducing a more expensive brand of the same quality as the higher quality baked beans, or lowquality beans with damaged cans into the choice set) and observing purchases from real customers. 6Other studies investigate the AE by using a hypothetical two-alternative (plus a decoy) forced-choice paradigm, where alternatives are defined through two numerical attributes (e.g., price and quality) upon which consumers' preferences are straightforwardly defined (almost all consumers prefer low price and high quality). In such a design, real incentives are absent [START_REF] Lichters | What really matters in attraction effect research: when choices have economic consequences[END_REF], the size of the choice set is limited [START_REF] Dimara | The attraction effect in information visualization[END_REF],

more subjective attributes are absent, and it is impossible to abstain from choice [START_REF] Dhar | The effect of forced choice on choice[END_REF]. These are all factors that are not applicable to most marketplaces but have been suspected to be necessary for consumers to identify dominance and to act on it [START_REF] Bettman | Constructive consumer choice processes[END_REF]. Following this debate, [START_REF] Lichters | On the practical relevance of the attraction effect: A cautionary note and guidelines for context effect experiments[END_REF] have proposed guidelines to conduct experiments on the AE to increase both the likelihood of obtaining the effect, and the external validity of the studies. These guidelines include the presence of economic consequences, a realistic and meaningful product presentation, presence of a no-buy option, relevance of the sample, and the avoidance of artificially repeated choices.

Testing the AE in a real recommender system

While [START_REF] Huber | Let's be honest about the attraction effect[END_REF] admitted that computerized laboratory studies can lack external validity to explain purchases made in brick-and-mortar stores, where customers can physically interact with the products they want to purchase, they argued that the criticism is less true for digital marketplaces. In digital marketplaces, product descriptions are closer to how choices are presented in computerized experiments, where product quality is often reduced to a numerical rating. Therefore, the emergence of digital marketplaces should increase the likelihood of situations occurring that are favorable to the appearance of the AE. In this desire to increase external validity, some papers proposed to test the AE with experiments that closely follow the guidelines by [START_REF] Lichters | On the practical relevance of the attraction effect: A cautionary note and guidelines for context effect experiments[END_REF]. These experiments are referred to as "artificial field experiments" (online experiments conducted on a more representative sample than students) and "framed field experiments" (online experiments that mimic or replicate the actual design of existing digital marketplaces). For example, [START_REF] Weinmann | The Attraction Effect in Reward-Based Crowdfunding: Evidence from Four Experiments[END_REF] highlight the AE in reward-based crowdfunding by running experiments on Prolific and by replicating a Kickstarter campaign, with economic consequences. [START_REF] Gomez | The attraction effect in mid-involvement categories: An experimental economics approach[END_REF] mimic websites like Phillips.com and Mediamarkt.com for choices among portable speakers and Groupon.com for choices among restaurant menus. Those experiments strongly suggest that the AE could appear in the digital field.

However, despite their high level of external validity, framed field experiments cannot reach the same level as real field or natural experiments. The respondents of framed field experiments are volunteers aware of their participation in an experiment. Therefore, they can be subject to experimenter demand effects [START_REF] Zizzo | Experimenter demand effects in economic experiments[END_REF]. Even if the choice is made real by the presence of economic incentives, and the participant is not forced to buy thanks to the presence of a no buy option, the context of the choice remains artificial in the sense that the participants are not real customers interested in buying the product outside of the study, and they may use different decision processes than the ones they would use in their everyday lives.

In addition, there are many features of online marketplaces that are absent from both offline marketplaces and framed field experiments. In online marketplaces, consumers are less captive and are free to search on other websites, implying that the set of outside options is much wider than the one proposed in the existing studies (given that only few studies actually proposed an outside option [START_REF] Lichters | What really matters in attraction effect research: when choices have economic consequences[END_REF][START_REF] Dhar | The effect of forced choice on choice[END_REF]). Another distinguishing feature is the dramatically high number of alternatives that users of online marketplaces have to search through. For instance, the only previous study testing AE in the field was conducted in a local grocery store [START_REF] Doyle | The robustness of the asymmetrically dominated effect: Buying frames, phantom alternatives, and in-store purchases[END_REF] and was concerned with three types of baked beans (one target, one competitor and one decoy) that were available in the store. Importantly, in such a setting, consumers can be considered as (more) captive, very differently from digital marketplaces --it is costlier to move from a store to another just because the choice set of baked beans is not fully satisfying to a consumer, than moving from a website to another.

Moreover, the AE has always been investigated in situations where the choice architect can freely define or control product attributes. One strength of such laboratory experiments is that the researcher can introduce the decoy that is expected to maximize the attraction effect [START_REF] Kaptein | Tracking the decoy: maximizing the decoy effect through sequential experimentation[END_REF]. In contrast, many recommender systems cannot directly change the choice set by creating new options or redefining their attributes, but acts only on the saliency of the existing options (e.g., by highlighting or sorting those options on top) to increase their likelihood of being considered. Since being considered is a necessary step for an option to be chosen, highlighting existing decoys (that would have not been perceived previously) should have qualitatively similar effects than introducing new decoys in the choice set [START_REF] Brady | Menu-Dependent Stochastic Feasibility[END_REF]. In particular, the existence of "phantom decoy effects" (the decoy alternative is presented but unavailable) proves that the availability of the decoy is not a necessary condition for the AE to be observed [START_REF] Pettibone | Testing alternative explanations of phantom decoy effects[END_REF]. However, one may expect the effect to be reduced if compared to the standard decoy experiment because (i) some of the users may consider the decoy even when it is not recommended, and (ii) the attributes of the recommended decoy cannot be defined to maximize the effect size. On the other hand, in the situations where many alternatives are available, which are the ones where recommender systems are the most useful, consumers are suspected to adopt more noncompensatory choice processes, and heuristics, and the AE is thus expected to increase [START_REF] Banerjee | When should we not expect attraction effect? The moderating influence of analytic versus holistic thinking[END_REF].

Finally, most of the studies investigating the AE focused on the impact of the decoy on the probability of choosing the target, relative to a specific competitor. This makes sense for sellers who are only interested in their own sales. However, from the point of view of a recommender system, it is interesting to study the impact of recommending a decoy, not only on the probability of choosing the target, but also on the probability of choosing the other recommended or available options. Therefore, in this paper, we will test the following three hypotheses:

H1: recommending a decoy option increases the probability that the user will choose one of the available options out of all options available on the platform.

H2: recommending a decoy option increases the probability that the user will choose one of the recommended options.

H3: recommending a decoy option increases the probability that the user will choose the targeted option (i.e., a recommended option that dominates the decoy).

The study

Our study aims to investigate the external validity of the AE on recommended options in a real online marketplace: namely online flight booking. More precisely, we focus here on a consumer's decision to book (or not) one of the available alternatives, from a menu comprising all flight itineraries from airport A to airport B on date T (and potentially a return on date T') proposed and ranked by a search engine. As discussed above, such an environment contrasts with the standardized paradigm that has been previously used to study the AE. The consumer faces a large range of alternatives, which vary in numerous vertical (e.g., price and duration) and horizontal (e.g., airline and time of departure) attributes, and is not forced to make a choice. If we set aside consumers' preferences among the horizontal dimensions and focus on price and duration, consumers typically face a time-money trade-off whenever the cheapest available itinerary is not the fastest one. Usually, those two options are recommended to help consumers to make their choice and we can thus investigate whether the likelihood of choosing a target itinerary -e.g., the fastest or the cheapest option on the list -is increased when a decoy itinerary (i.e., an itinerary asymmetrically dominated in price or duration by the target itinerary but not by its competitors) is also recommended.

Field experiment on an online flight aggregator

We conducted a large-scale field experiment in 2019 (from April 14 th to May 24 th and from July 31 st to November 1 st ), on a real online flight aggregator. As in other flight aggregators, users indicate a departure date D, a departure airport A and a destination airport B (and potentially return date D') to receive a list of all (relevant) available itineraries, offered by most online travel agencies and airlines worldwide. Above this list, a "highlighted panel" is displayed to recommend two itineraries -the "Cheapest" and the "Fastest". 7 Users are free to choose several itineraries from the list, or from the highlighted panel by clicking on the "Select" Button (Figure 2, left). If they do so, more information and a list of sellers (airline and/or travel agency) is provided, and the user can choose one of those sellers and be redirected to its website to perform a booking. We do not observe in our experiment the final booking decision, but the redirection to the sellers' website is considered as a "conversion" from the perspective of the aggregator. Figure 2: Screenshots of a search on the website. 7 In cases where the cheapest (respectively the fastest) itinerary was not unique, the recommender system highlighted the fastest of the cheapest itineraries (respectively the cheapest of the fastest itineraries).

13

Note: Left (right) figures show the NoDecoy (Decoy -here targeting the Fastest) versions of the website.

More information about an itinerary is disclosed when clicking on the "Flight details" button. The "Select" button opens a list-box with several links to sellers' websites for booking. The exact design depends on users' browser and graphics settings. Users have to scroll down to see more alternatives and may apply a filter to mask or change the ranking of the itineraries. This does not affect the appearance of the highlighted itineraries.

All in all, we observe a total of 200 992 search sessions, but focus the main analysis on the 146 851

sessions where at least one potential decoy itinerary was available (the analysis of the sessions where no potential decoy was available is presented in Appendix A). The next subsection explains how we identify the decoy and our experimental manipulation.

Decoy identification and experimental manipulation

Our experimental manipulation consists in recommending a third "Featured" itinerary in the highlighted panel, as shown in Figure 2, which compares the "NoDecoy" setup, with two highlighted options, to the "Decoy" setup, with three highlighted ones.

This Featured itinerary is selected from the list of available options to be a decoy that targets either the Fastest (i.e., it is an itinerary more expensive than the Fastest, but faster than the Cheapest) or the Cheapest option (i.e., it is slower than the Cheapest, but cheaper than the Fastest), with the following procedure.

a) Selection of the potential decoys targeting the Cheapest and the Fastest.

Using the two recommended itineraries (the Cheapest and the Fastest), we identified two sets of potential decoy itineraries: (i) the set of decoy itineraries (with price and duration )

targeting the Cheapest, and (ii) the set of decoy itineraries (with price and duration )

targeting the Fastest. The itineraries in the two sets satisfy the following conditions:

• ≤ < & < ≤
for the set of decoys targeting the Cheapest, and

• < ≤ & ≤ <
for the set of decoys targeting the Fastest, where ( , ) and ( , ) denote the price and duration of the Cheapest and Fastest itineraries, respectively.

In the case where more than one decoy was targeting the Fastest (Cheapest), we select as a potential decoy targeting the Fastest (Cheapest) the decoy which was closest in departure time to the Fastest (Cheapest). Indeed, because departure time appeared to be one of the most important horizontal characteristics, we aimed to control for this dimension as much as possible.

b) Selection of the Featured alternative.

• Case 1: no potential decoy is targeting the Cheapest and no potential decoy is targeting the Fastest. In this case, the "Featured" itinerary was randomly selected from the set of all available itineraries. Those search sessions were different from the sessions where decoys were available and thus were removed from the main analysis. We analyze those search sessions separately in the appendix to assess the impact recommending a third alternative had on the conversion rates and found no impact of recommending a third alternative.

• Case 2: There is one potential decoy targeting the Cheapest OR one potential decoy targeting the Fastest, but not both. In this case, we choose this potential decoy as the Featured itinerary.

• Case 3: at least one potential decoy is targeting the Cheapest AND at least one potential decoy is targeting the Fastest. In this case, we randomly choose the Featured itinerary between the potential decoy targeting the Fastest and the potential decoy targeting the Cheapest with probability one half.

We identified a Featured itinerary for all search sessions, but it was highlighted only in the Decoy version of the website. We examine the impact of this experimental manipulation on three distinct conversion rates:

• the "overall conversion rate" (CR), which measures the probability that a user selects at least one of the available itineraries;

• the "highlighted conversion rate" (HCR), which measures the probability that a user selects at least one itinerary among the Faster or the Cheapest;

• the "targeted conversion rate", which measures the probability that a user selects the itinerary targeted by the decoy.

A statistically significant increase in those conversion rates, when recommending a decoy, would be interpreted as an evidence of the AE in online flight booking.

Control for session characteristics

The heterogeneity of search sessions allows us to study in which context the AE is the strongest.

For example, [START_REF] Gaudeul | Fast then slow: A choice process explanation for the attraction effect[END_REF] show that the AE is more likely to occur when the competitor and the target alternatives are close in terms of utility and when the decisions are made quickly decisions, suggesting the AE is due to an intuitive cognitive process. Therefore, we can investigate if the AE is more likely to occur when the competitor and the target itineraries are similar or dissimilar, and when the travel is being planned in advance or not. We also investigate the impact of the size of the results list on the AE, since having more choice may either prevent the identification of the dominance relation (reducing the AE) or may incite the user toward more intuitive thinking (increasing the AE). Since we can identify if the users made their research using a mobile (tablet or phone) or a computer, we can also test if the AE is dependent on the device used. In particular, we collected and studied the impact of the following characteristics: (i) the "device used": if the research was made on a computer or on a mobile; (ii) if the user was looking for a one-way or a return flight; (iii) the duration of the stay (only relevant for return travel), that is the difference between the departure date and the date of return; (iv) the number of days before the departures (i.e., the difference between the departure date and the date of the search); (v) the menu size, that is the total number of itineraries presented by the flight aggregator; (vi) the duration and the price of the cheapest and the fastest itineraries. Table 1 presents the characteristics of the sessions and shows that those characteristics are not statistically significantly different between the two versions of the website.

Table 1: Mean and standard deviation of the session characteristics for both versions of the website.

Note: p-values indicate the p-value for a z-test comparing the proportion of (highlighted) conversion for each characteristic between the two versions of the website.

Results

At least one itinerary was selected in 10 877 of the 146 851 search sessions and at least one of the recommended itineraries was selected in 2 789 of the search sessions. We were able to identify requests made from the same web browser and, although we do not have the means to identify or track the person actually making the searches, we therefore consider searches on the same browser to be made by the same user. We identified [START_REF] Zizzo | Experimenter demand effects in economic experiments[END_REF]

Main effect

To assess statistical significance, we conducted two-sample, two-tailed z-tests and report

Cohen's d as a measure of effect size. We also report Bayes Factor ( ) in favor of the null hypothesis. 8 We find that highlighting a decoy does not increase the probability that a user will convert, that is, choose at least one option in at least one of their search sessions (CR= and the Fastest itineraries were available, and thus had equal chance of being highlighted. This condition is necessary for the targeted option to be randomly balanced between the Cheapest and the Fastest options and with respect to other attributes. We find that the probability of a user choosing the target option is not higher than the probability of choosing the highlighted competitor option (CR(target) = 2.08% vs CR(competitor) = 2.03%, z=-0.27, p= 0.788, Cohen′s d=0.00344, = 39.23).

Test of moderators of the effect

To investigate whether the AE is more likely to appear in particular settings, we conduct a similar analysis, based on subsample of the sessions, splitting according to: (a) the menu size, (b) the time pressure (measured by the number of days before the departure), (c) the similarity between the Cheapest and the Fastest highlighted itineraries, (d) the economic stakes (measured by the price of the Cheapest itinerary), and (e) the device used (either mobile or computer). More precisely, for each of the following characteristics: size of the menu, time pressure, duration similarity, price similarity, and economic stakes, we separated the sessions according to their median values:

• High (low) "Size of the menu" sessions correspond to the half of the sessions with the highest (lowest) number of available itineraries.

• High (low) "Time pressure" sessions correspond to the half of the sessions where the date of departure is the closest (farthest) from the date of the search.

• High (low) "Duration similarity" sessions correspond to the half of the sessions where the ratio of the durations between the Cheapest and the Fastest is the highest (lowest).

• High (low) "Price similarity" sessions correspond to the half of the sessions where the ratio of the prices between the Fastest and the Cheapest ! ! is the highest (lowest).

• High (low) "economic stakes" sessions correspond to the half of the sessions where the price (in USD) of the Cheapest option is the highest (lowest).

For each of those subsamples, we then conducted the AE analysis following the same methodology as described previously. We report those results in We found no significant AE effect on conversion rate, highlighted conversion rate nor on target vs competitor conversion rate for any of the subpopulations, except a very small and limited effect in half of the sessions with the lowest time pressure (i.e. sessions where the departure date is more than 28 days ahead from the search). In those sessions, the alternatives are more likely to be chosen when they are targeted by a decoy (CR(Target)=2.26%), than when a decoy is targeting another alternative (CR(Comp)=1.72%). [Table 3, z=2.197, p=0.028, Cohen's d=0.0385, =

.

Robustness test

As a robustness check, we conducted four logistic regressions at the session level, with the overall conversion rate (CR, Model 1), the highlighted conversion rate (HCR, Model 2), the conversion rate of the cheapest (CR(Cheapest), Model and of the fastest (CR(Fastest), Model 4) as dependent variables. To limit the endogeneity issue, we only selected the first search session for each user. For Model 1 and Model 2, the AE effect was estimated using a dummy indicating if a decoy was highlighted. For Model 3 (Model 4), the AE was estimated using a dummy indicating the presence of the decoy targeting the Cheapest (Fastest) and the analysis was restricted to the subset of sessions where a decoy was highlighted and where decoys targeting both cheapest and fastest itineraries were available. Indeed, those conditions are necessary for the target to be exogenously and randomly decided with probability one half. The AE estimated in Model 3 and 4 therefore measures the impact of being the target vs being the competitor. In all the models, we use the abovementioned session characteristics (size of menu, Day to departure, Price similarity, Duration similarity and Used device), as moderators of the AE, as well as other session characteristics (price and duration of the fastest and cheapest itineraries, a dummy indicating if the itinerary is one way or return, and the duration of the stay for the return itineraries) as controls. The results are presented in

Exploratory analysis

As exploratory analyses, we tested other cut-off values for the characteristics presented in Tables 2 and3, and conducted similar analyses for other subpopulations. For each of the selected characteristics (size of the menu, time pressure, duration similarity, price similarity, and economic stakes), we divided the search sessions into subpopulations corresponding to deciles of those characteristics, and estimated the effect size of the AE on conversion rate, highlighted conversion rate and target vs competitor conversion rates. The detailed results are presented in Appendix B (Figures B1-B5). We also performed an analysis by country, restricted to the countries where we observed most of our search sessions (Appendix B, Figure B6). Since we conducted a large number of tests and only a very limited fraction of them reached significance, we will refrain from overinterpreting the significant results that are reported here.

Concerning time pressure, we found no significant effect when dividing the sessions into deciles (Figures B1 (a-c). However, we found opposite significant effects of highlighting a decoy on the probability of choosing the target compared to the competitor, when search sessions were divided according to being made a day, a week, or a month before travel. More precisely, the alternatives targeted by the decoy have less chance of being selected when the search is made between a week and a month before travel [Figure B1 (d), Cohen's d=-0.048, p=0.02], and more chance of being selected when the search session is made more than a month before the travel [Figure B1 (d) Cohen's d=0.036, p=0.045]. We found no impact of the menu size (Figure B2).

Concerning duration similarity, we found a negative effect on the overall conversion rate for the sessions corresponding to the first decile in terms of duration similarity $ ≥ 4.28 × )

[Figure B3

Discussion

In this article, we studied and did not find evidence for the existence of the attraction effect (AE)

for the recommended options in online flight booking. In general, our data did not provide support for the following three hypotheses: H1: recommending a decoy option increases the probability that the user will choose one of the available options out of all options available on the platform; H2: recommending a decoy option increases the probability that the user will choose one of the recommended options; H3: recommending a decoy option increases the probability that the user will choose the targeted option (i.e., a recommended option that dominates the decoy).

The typical paradigm used in the studies documenting AE is based on hypothetical forced choices among two or three alternatives. These alternatives are usually characterized by two or three numerical attributes and presented in a simple, minimalistic design. In contrast, our flight aggregator offers the users the possibility to search and gather information about numerous alternatives, defined by numerical attributes for which preferences are universal (such as price and flight duration), as well as ones for which preferences could be potentially heterogenous across users (e.g., departure time or airline). In addition, implementation in the field cannot prevent users from soliciting additional information from other search engines. In this setting, we decided to manipulate which options were presented in the highlighted panel at the top of the list of all available alternatives. Because of its saliency, we considered that this highlighted panel constitutes (to some extent) a refined design, embedded in a more complex environment. We, therefore, expected to detect an AE, but at somewhat weaker level. Potential difficulties in identifying dominance relationships could be attributed to the presence of certain flight attributes that obscure the user search process. Moreover, the richness of the choice set, along with the option to completely abstain from choosing, repeat the search at a later stage, or use another flight aggregator, could also influence the construction of user preferences. Our manipulation, therefore, represents a relatively less pronounced modification of the user experience, compared with previous (laboratory) studies. In particular, a user who repeated their search multiple times over an extended period and investigated extensive set of alternatives, would form much more precise and stable preferences than in a more artificial laboratory setting, significantly diminishing (and potentially annihilating) the effect of the nudge. One would expect this limiting effect on the AE to be smaller in users' early searches. However, our robustness analyses did not detect the AE in the subset of data consisting of the search sessions by novice users either (Table 4). In addition to the points discussed in the previous paragraph, there are a number of other possible explanations for the lack of AE observed in our setting in contrast to previous studies.

First, in our experiment, decoy alternatives were included in the choice set in both versions of the website and our manipulation consisted only in recommending the decoys in one of the versions.

Therefore, if users were attentive and considered all the available alternatives, the choice set was similar in both versions of the website, preventing the AE from occurring. However, this interpretation seems unlikely since search sessions returned on average more than 38 possible itineraries (preventing users from full consideration). Moreover, when controlling for the size of the choice set, we find no evidence for the effect, regardless of the number of available alternatives. This suggests that the number of available alternatives was not the main reason why we were unable to replicate the effect.

Another explanation could be that most consumers have strong preferences toward one of the proposed itineraries, or toward the outside option. The AE has mostly been observed when the target and the competitor were close in terms of utility. However, our analyses do not suggest that a lack of similarity in terms of utility was the main reason for not finding the effect. Indeed, we were not able to observe any monotonic relation between the similarity in terms of price and duration of the competitor and the target and the strength of the effect. If similarity does play a role in the field, its role is thus less straightforward than the one identified in laboratory settings.

Another particularity of flight booking decisions is to involve stakes much higher than in most incentivized experiments, leading to more deliberative thinking and preventing the appearance of the AE [START_REF] Gaudeul | Fast then slow: A choice process explanation for the attraction effect[END_REF]. However, we conducted analyses on subpopulations to control for economic stakes, measured by the price of the cheapest of the available alternatives, and did not find that the stakes of the decisions were moderating the effect in the field.

Another reason for the field replication failure could be that users are less captive than in laboratory experiments, in the sense that they have access to richer outside options (other platforms, possibility to delay their choices, etc.) Unfortunately, we were not able to measure user's activity outside of the flight aggregator and cannot directly test the captivity hypothesis (e.g., by controlling the number of websites visited). The closest proxy of captivity we could get from our data was time pressure, measured by the delay between search and travel. Indeed, one may assume the richness of the outside options decreases as the departure draws nearer, increasing both user captivity and time pressure. Interestingly, we observed a positive (although very small in size) attraction effect, only when we restricted our analyses to the half of the sessions where search was the most in advance of the travel.

While this result goes against previous findings, which suggest that time pressure enhances AE [START_REF] Gaudeul | Fast then slow: A choice process explanation for the attraction effect[END_REF],

and thus should be interpreted carefully, at least it does not suggest that captivity is a necessary condition for obtaining the effect.

Finally, one crucial difference is that in our experiment, the recommended decoy was chosen from among the set of potential decoys. Other studies instead created and offered a specific decoy, that was expected (by the choice architect) to maximize the probability of observing the AE. Indeed, [START_REF] Kaptein | Tracking the decoy: maximizing the decoy effect through sequential experimentation[END_REF] have shown that the size of the AE is a (non-monotonic) function of the distance between the decoy, the target and the competitor, and have proposed an algorithm to find the decoy that maximizes the AE.

They conclude that the failure to replicate the AE can be attributed to a "misplaced" decoy. Applying their algorithm in our case was not possible, due to the heterogeneity of the sessions and to the fact that we cannot freely manipulate the price and duration of itineraries. This absence of latitude is shared by many recommender systems: the choice architect does not always have the ability (or the right) to create new alternatives or to alter existing ones. This last explanation suggests that digital decoy nudges could be more suited to situations where the choice architect has the flexibility to create and modify the proposed alternatives (e.g., a producer or a reseller, who can decide the price or other attributes of products), rather than those where the main strategic variable is the decision of which and how existing information is disclosed (e.g., a search engine or a marketplace).

On the other hand, additional exploratory analyses have suggested possible heterogeneity in the AE. This conjecture, however, needs to be confirmed by confirmatory research based on new samples (due to the exploratory nature of our analyses). In particular, we found a significant effect only for very specific levels of similarity, countries, and time pressure. Because a high number of statistical tests were performed to find those results, it is likely they were obtained by chance, so we refrain from interpreting them. Nevertheless, being able to find (some) effect heterogeneity in subpopulations suggests that it could be possible to design algorithms that learn to identify which session features support the effect the most and thus offer personalized digital nudges. For example, one of our exploratory results suggested that opposite effects could be found depending on how far in advance of the travel date the flight search was conducted. Recommending a decoy favored the target alternatives (as predicted by AE) when the search preceded the travel by more than a month, but the decoy favored the competing alternative (repulsion effect) when the search preceded the travel by a shorter period. If this finding were robust, a recommender system could be designed to exploit this feature and adapt digital nudges to session characteristics. For example, if the goal of the flight aggregator presented here were to promote faster itineraries over cheaper ones, it could recommend decoys that target the fastest when the day of departure is more than a month in the future, but recommend decoy options that target the cheapest for searches preceding the departure by shorter periods.

To conclude, more research is needed to confirm these findings. In particular, we call for more conceptual replications of the AE within real digital marketplaces and recommender systems and with a larger variety of goods. If other researchers fail to replicate the AE in recommender systems, then laboratory experiments could also be useful to identify and test how the specific features of recommender systems can moderate the AE. For instance, one could test how the recommendation model (e.g., recommending options based on compensatory versus non-compensatory models [START_REF] Lin | Non-compensatory psychological models for recommender systems[END_REF]) or the interface (allowing or not the filtering of the alternatives with respect to attributes) of the recommender system interacts with the AE. Another interesting feature to explore would be the absence of captivity of recommender system users who are free to search on other websites. For instance, one could imagine an experiment where the participants have access to several platforms from which they can make their choice, and the study investigates whether the presence (or recommendation) of a decoy option on one platform, but not on the others, impacts how choices are made between and within platforms. Such studies would not only guide managers in devising more efficient recommender systems, but also contribute to advancing our understanding of human decision-making in digital environments.

Acknowledgment

3

 3

Figure 1 :

 1 Figure 1: Example of potential decoys' positions for itinerary choices. Note: In this itinerary choice example, alternatives are defined by only two attributes: price and duration (the cheaper and the faster, the better). Alternatives located in the region are dominated by the Cheapest itinerary (they are more expensive but not faster than the Cheapest), but not by the Fastest itinerary (they are slower but cheaper than the Fastest) and are potential decoys targeting the Cheapest itinerary. Symmetrically, the alternatives located in the region are potential decoys targeting the Fastest.

  a) Screenshot of the baseline version. b) Screenshot of the decoy version. c) Highlighted panel of the baseline version. d) Highlighted panel of the decoy version.

  (a) Cohen's d = -0.051, p=0.022]. For the sessions corresponding to the 7th decile $1.33 × ≤ < 1.51 × ) we found a positive effect on the probability of choosing a highlighted option [Figure B3 (b) Cohen's d = 0.052; p=0.014], however the probability of choosing the target is lower than the probability to choose the competitor [Figure B3 (c): Cohen's d= -0.058, p= 0.05]. As for price similarity, highlighting a decoy has a significant effect only if we restrict the analysis to the 7 th decile $1.25 × ≤ ≤ 1.35 × ), and reduces the overall conversion rate [Figure B4 (a): Cohen's d = -0.041; p=0.044]. Concerning economic stakes, when looking at Figure B5a, the effect size of the impact of highlighting a decoy on the probability of choosing any alternative seems to increase between the 4 th to the 10 th decile, although none of the effects is statistically significant. Finally, concerning countries' subpopulations, the only significant effect was obtained for the sessions made from Germany: highlighting a decoy increases the probability of choosing the target, compared to the competitor [Figure B6 (c): Cohen's d = 0.129, p=0.004].

Table 2

 2 (for the conversion rate and the highlighted conversion rate) and in Table3(for the Target and Competitor Conversion rate).

				NoDecoy	Decoy	z	p-value Cohen's d
	All sessions	CR HR	17.26% 5.15%	16.92% -0.978 5.04% -0.501	0.328 0.617	-0.00917 46.73 -0.0047 66.49
	Size of the	high	CR HCR	15.23% 2.42%	15.20% -0.059 2.50% 0.415	0.953 0.678	-0.00068 61.25 0.00479 56.20
	menu	low	CR HCR	13.15% 5.62%	13.77% 1.533 5.76% 0.535	0.125 0.592	0.01796 18.64 0.00627 52.30
	Time	high	CR HCR	14.76% 4.35%	14.93% 0.377 4.40% 0.222	0.706 0.824	0.00473 52.52 0.00279 55.01
	pressure	low	CR HCR	17.03% 4.99%	17.39% 5.14% 0.553 0.75	0.453 0.581	0.00954 41.96 0.00703 47.70
	Duration	high	CR	15.08%	14.94% -0.304	0.761	-0.00404 50.78

Table 2 :

 2 The Attraction Effect on the overall and highlighted conversion rates separated by session characteristics.

			Competitor	Target	z	p-value	Cohen's d
	All sessions		2.08%	2.03%	-0.27	0.788	0.00344	39.23
	Size of the	high	1.14%	1.00%	-0.894	0.371	-0.01372	21.87
	menu	low	2.48%	2.75%	0.95	0.342	0.01693	17.88
	Time	high	2.03%	1.63%	-1.78	0.075	-0.03034	6.03
	pressure	low	1.72%	2.26%	2.197	0.028	0.0385	2.56
	Duration	high	2.37%	2.04%	-1.111	0.266	-0.02268	13.23
	similarity	low	1.45%	1.64%	1.107	0.268	0.01592	18.87
	Price	high	2.10%	2.10%	0	1	0	27.91
	similarity	low	1.40%	1.51%	0.575	0.566	0.00879	27.71
	Economic	high	2.18%	2.28%	0.444	0.657	0.00729	27.66
	Stakes	low	1.41%	1.38%	-0.151	0.880	-0.00268	27.81

Note: "CR" (respectively "HCR") indicates the overall (resp. highlighted) conversion rate, that is the probability of a user selecting at least one option (resp. to select the Fastest or the Cheapest highlighted itinerary) in at least one search session.

Table 3 :

 3 The Attraction Effect on target and competitor conversion rates.Note: The target (competitor)'s conversion rate represents the probability of a user choosing thehighlighted alternative (not) targeted by the decoy, in at least one of the search sessions.

Table 4 .

 4

		CR	HCR	CR(Cheapest)	CR(Fastest)
		(Model 1)	(Model 2)	(Model 3)	(Model 4)
	AE	-0.001	-0.012	0.259	-0.107
		(0.041)	(0.099)	(0.311)	(0.454)
	AE x Days to	0.012	-0.034	0.015	-0.079
	departure	(0.034)	(0.065)	(0.155)	(0.312)
	AE x Menu	0.003	0.209	-0.637	0.498
	size	(0.034)	(0.166)	(0.414)	(0.345)
	AE x	-0.060	-0.157	0.862	-1.264
	Duration similarity	(0.043)	(0.081)	(0.522)	(0.706)
	AE x Price	0.291	0.424	2.439	-3.062
	similarity	(0.304)	(0.936)	(2.615)	(3.909)
	AE x Mobile	0.024	0.138	-0.054	-0.885
	Device	(0.083)	(0.155)	(0.425)	(0.671)
	Constant	-1.430 ***	-2.821 ***	-3.605 ***	-4.944 ***
		(0.104)	(0.208)	(0.667)	(0.942)
	Days to	-0.091 ***	0.021	0.025	0.060
	departure	(0.027)	(0.052)	(0.108)	(0.236)
	Menu size	0.011	-0.881 ***	-0.687 **	-0.401
		(0.026)	(0.137)	(0.235)	(0.332)
	Duration	-0.116 **	-0.033	-1.490 **	-0.320
	similarity	(0.037)	(0.062)	(0.511)	(0.372)
	Price	-1.508 ***	-4.025 ***	-4.015	0.123
	similarity	(0.298)	(0.886)	(2.678)	(2.192)
	Mobile	-0.280 ***	-0.434 ***	-0.688 *	0.553
	Device	(0.066)	(0.123)	(0.292)	(0.438)
	One-way	-2.473 ***	-2.031 ***	-1.478	-2.892
	itinerary	(0.255)	(0.416)	(1.062)	(1.642)
	Stay duration 1.133 ***	0.937 ***	0.615	1.370
		(0.111)	(0.180)	(0.459)	(0.712)
	Price fastest	0.0001 *	-0.0001	0.00002	-0.0003
		(0.00003)	(0.0001)	(0.0003)	(0.0004)
	Price cheapest -0.293 ***	-0.143	-0.567	0.385
		(0.053)	(0.118)	(0.361)	(0.392)
	Duration	0.038	0.214 **	0.554 *	-0.926
	fastest	(0.046)	(0.083)	(0.248)	(0.515)
	Duration	-0.002	-0.010 *	-0.009	0.028
	cheapest	(0.002)	(0.004)	(0.014)	(0.024)

Table 4 :

 4 Regression analysis

	Note: standard errors are in parenthesis. *: p<0.05; **: p<0.01; ***: p<0.001

Other examples can be found in[START_REF] Arnott | Behavioral economics for decision support systems researchers[END_REF].

See[START_REF] Lichters | On the practical relevance of the attraction effect: A cautionary note and guidelines for context effect experiments[END_REF] for a review.

Experimental manipulation and product presentation can moderate the attraction effect: the relevance and meaningfulness of the product decreases the AE[START_REF] Mishra | Antecedents of the Attraction Effect: An Information-Processing Approach[END_REF], while the use of pictorial and tactile forms[START_REF] Simonson | Choice in Context: Tradeoff Contrast and Extremeness Aversion[END_REF], video demonstrations[START_REF] Slaughter | Decoy effects and attribute-level inferences[END_REF] or graphical representations[START_REF] Choplin | Magnitude comparisons distort mental representations of magnitude[END_REF] enhance the AE.

Another study conducted by[START_REF] Li | Retraction of "The Decoy Effect as a Nudge: Boosting Hand Hygiene With a Worse Option[END_REF] proposed to test the attraction effect in three field experiments to increase hand hygiene in a food factory, but has been retracted due to anomalies and untraceable data collection processes.

Bayes Factor t-tests are performed following[START_REF] Rouder | Bayesian t tests for accepting and rejecting the null hypothesis[END_REF]. For each test, we report the "Scaled-Information Bayes Factor", based on a normal prior distribution of effect size, with scale factor r=0.70.

This research was supported by the French National Research Agency (ANR) (GRANT: ANR-15-IDEX-01). Data for this research was provided by Amadeus, a major IT provider for the global travel and tourism industry. We thank the anonymous reviewers, for their useful comments that helped improve this work. We thank Catherine Buchanan for proofreading this article.

This research was supported by the French National Research Agency (ANR) (GRANT: ANR-15-IDEX-01). Data for this research was provided by Amadeus, a major IT provider for the global travel and tourism industry. We thank the anonymous reviewers, for their useful comments that helped improve this work. We thank Catherine Buchanan for proofreading this article.