Xavier Blanc
email: xavier.blanc@labri.fr

Thomas Degueule
email: thomas.degueule@labri.fr

Jean-Rémy Falleri
email: jean-remy.falleri@labri.fr

Diffing E2E Tests against User Traces for Continuous Improvement

Keywords: end-to-end testing, web application testing, trace differencing

End-to-end testing (E2E) is a popular approach to uncover faults in web applications by mimicking the behavior of real users. Using dedicated frameworks (e.g., Playwright, Cypress), testers translate high-level validation scenarios into concrete browser interactions to evaluate the application's behavior. Testers are forced to make implementation choices when writing their tests: choosing one of several valid selectors to click a button, focusing an input field using a Click or Tab event, etc.

In this paper, we show that the resulting tests may not always represent the behavioral of real users-some action sequences may even be infeasible for users. These discrepancies between tests and users hurt the very premise of E2E testing.

To address this issue, we propose a novel method that enables testers to understand the true behavior of their E2E tests, and help them improve their tests by pinpointing when, where, and in which way they differ from the behavior of real users.

Using a set of E2E tests provided by our industrial partner, we show that our tool is indeed able to identify discrepancies between the tests and the users and that, in several cases, the testers decided to update their tests to better match the behavior of their users.

CCS CONCEPTS

• Software and its engineering → Software testing and debugging.

INTRODUCTION

A popular approach to web application testing is to define and execute end-to-end (E2E) tests to uncover faults by mimicking the behavior of real users [START_REF] Leotta | Chapter Five -Approaches and Tools for Automated End-to-End Web Testing[END_REF]. E2E testers typically leverage the capabilities of popular testing frameworks such as Playwright, Cypress, or Selenium to translate high-level validation scenarios (e.g., purchasing a product in a webshop) into concrete interactions within the browser where the web application runs.

Using these frameworks, simple steps in a scenario (e.g., clicking a button or filling a form) can be realized in multiple ways. Indeed, a modern stylized button may consist of several DOM elements, each with its particular CSS selector, and the click itself may be realized in various ways: clicking the button directly, focusing it and pressing the Enter key, etc. Similarly, one may jump from one field in a web form to the other by pressing the Tab key or by focusing each field with a Click event before inputting a value. Although there are many ways to realize the same scenario, only one (or a handful) is eventually implemented and verified.

For E2E testing to be successful, it must simulate the behavior of real users. However, since simple interactions may be realized in many different ways, it is difficult for testers to assess whether the interactions realized by their tests match what a user would do. Worse, our exploratory analysis reveals that, out of the box, test frameworks sometimes interact with web applications in a way that can never be reproduced by actual users (Section 2). For instance, when filling a web form, Playwright may use the JavaScript API without performing any Click or Keystroke. This contrasts with real user interactions that would inevitably require the use of the mouse or the keyboard to focus on the fields and to set their values. We believe that these discrepancies between tests and users hurt the very objective of E2E testing and may prevent the tests from revealing faults encountered by the users.

In this paper, our aim is twofold: assist the testers in understanding the true behavior of their E2E tests, and help them improve their tests by pinpointing when, where, and in which way they differ from the behavior of a real user. To this end, we first present a non-intrusive tool that automatically records every relevant interaction on a web application (clicks, keystrokes, etc.). We use this tool to record the traces resulting from test executions, as well as the traces of real users interacting with the application (Section 3). Then, we present a diff algorithm and interface that pinpoints and highlights the differences between the execution(s) of a test and the same scenario realized by one or several users (Section 4).

We hypothesize that whenever the trace produced from an automated test and from a user realizing the same scenario differ, the testers may decide to update their tests to match the user's trace better. In our evaluation (Section 5), we find that, indeed, our tool can identify new interactions that are of higher quality than those used in the tests (e.g., because the user clicked on a different area or used some keyboard shortcuts) and identify ill-formed traces that prompted the testers to reconsider their tests (e.g., because the test filled two input fields in a form without switching focus in between). This leads us to conclude that our tool can nicely complement the usual E2E testing toolkit by improving the quality of E2E tests implemented using the most popular frameworks.

BACKGROUND & EXPLORATORY ANALYSIS

Let us consider a straightforward web application composed of two pages: a landing page and a login page (see Figure 1). Alice, a junior tester, is tasked with testing this simple web application by implementing a new E2E test. Her objective is to assess the following scenario:

(1) Go to the landing page (2) Navigate to the login page (3) Fill the username and password fields (4) Submit the form (5) Check that the user has been successfully logged in To implement her scenario, Alice uses the Playwright framework and writes the source code presented in Listing 1. She implements all the scenario steps and makes some design choices to specify how the test interacts with the browser. For instance, she decides that moving from the landing page to the login page is performed by clicking the DOM element identified by goto-login-page-link. She also decides to rely on the fill() method of the Playwright framework to implement step (3). Alice's design choices aim at realizing the scenario in a similar manner as a real user. Her choices are also driven by her knowledge of the framework and her willingness to make the source code as clean as possible.

Bob, a senior tester who is also familiar with the playwright framework, explains that the fill() method relies on the internal JavaScript API of the browser to focus the text fields and set their values. As a consequence, the test does not actually perform any realistic user interaction (clicks or keystrokes) in step (3) of the scenario. He then proposes a new version where the fill() methods are each replaced by a click() and a type() (see Listing 2). This new version is therefore much closer to the behavior of a real user. Finally, Jenny, another senior tester, proposes a third version claiming that some users usually use the Tab key to navigate through the fields in a form. She also argues that the form > div.button selector is too vague and may accidentally match another button in the form if it were to evolve in the future. She then proposes another selector that uniquely identifies the submit button. Listing 3 shows the relevant lines in this third version. This simple illustrative example shows that there are many ways to implement the same E2E test. Some implementations, such as the first version of this simple test, emulate a behavior that cannot be realized by real users, which directly contradicts the purpose of E2E testing. We argue that these implementations should be avoided in tests, but that testers do not have the necessary tool support to easily detect these cases.

There might also be several implementations that represent the behavior of real users. It then becomes difficult for testers to decide which design choices should be taken (e.g., here, a Click or a Tab). In such a context, we argue that tests should be as close as possible to real users, with the main motivation to uncover faults that may be encountered by them. For instance, if most users use the Tab key to switch among the fields, then the test should do the same.

In this paper, we aim at helping testers such as Alice, Bob, and Jenny towards that objective. We propose an approach that identifies the real interactions made when executing an E2E test. We then propose to compare these interactions with the ones realized by real users of the application. We argue that highlighting the differences in interactions between tests and users enables testers to improve their E2E tests and, ultimately, their application's quality. This overall process is depicted in Figure 2 and detailed in the following sections.

RECORDING INTERACTIONS TRACES

In this section, we explain how we record the interactions performed by tests and real users when using a web application. The section starts by presenting the set of interactions we are interested, and then presents how we build sequences of interactions (i.e., traces) by monitoring the browser and the web application.

User Interactions

In any web application, most interactions are performed thanks to a pointing device (e.g., a mouse) and a keyboard. An interaction mainly means to (1) focus on a specific element and (2) actuate it. Some interactions simultaneously focus on an element and actuate it (e.g., a mouse click). Others only focus an element (e.g., the Tab key), or only actuate (the Enter key). Interactions have a target, which is the element on which the interaction occurs and that can be uniquely identified by a CSS selector [START_REF] Leotta | Robula+: an algorithm for generating robust XPath locators for web testing[END_REF].

Several interactions can be performed with the mouse: leftclicking, double-clicking, right-clicking, selecting a text, moving the mouse, etc. We choose to only consider left-click on an element (mouse down), as the other mouse interactions rarely lead to application-specific interactions in most web applications. We further choose to abstract any left-click interaction by the word Click$TargetCSSSelector (where TargetCSSSelector is the CSS selector of the target element of the interaction). For instance, a click on an element identified by the CSS selector #goto-login-page-link is abstracted by the word Click$#goto-login-page-link.

Several interactions can be performed with the keyboard: editing a text field, pressing the Enter key to submit a form, pressing the Tab key to focus the next element, etc. When a user is editing a text field, we consider that all the interactions they perform can be abstracted by only one interaction: Edit$CSSSelector, as it is uncommon for web applications to have custom interactions triggered by entering a specific text value. When a user is not editing a text field, the keyboard may be used to focus and actuate an element thanks to keyboard shortcuts. Unfortunately, there is no standard defining which keyboard shortcuts are generally available and supported. The WAI-ARIA Authoring Practices 1.1 defines some principles and some guidances but does not specifies a list of keyboard shortcuts that any browser and any web application should respect. 1 The W3C defines a list of keyboard shortcuts that are commonly supported by browsers, but these shortcuts are not for focusing on and actuating elements. 2 Finally, the gray literature contains some lists of keyboard shortcuts for interacting with any web application. 3We then consider that the following shortcuts are supported by most the browsers and web applications: key interaction Tab focus on the next element in the DOM ShiftTab focus on the previous element in the DOM Space actionate the focused element Enter actionate the focused element Arrow select an option in a select box Esc close a popup or a dialog We choose to abstract these interactions using the word Key$TargetCSSSelector, where Key represents the keyboard shortcut and TargetCSSSelector the target's CSS Selector. In our example, Tab$#username abstracts the fact that the Tab key has been pressed when the element identified by #username was focused. Note that this interaction makes the #password element focused.

Our set of interactions is relatively small but captures the majority of the interactions that are performed by a user. For example, running the test depicted in Listing 3 will yield the following trace:

nb interaction 1

Click$#goto-login-page-link

2 Click$#username 3 Edit$#username 4 Click$#password 5 Edit$#password 6 Click$#submit
By contrast, running the test depicted in Listing 1 highlights that the test is not well designed as it does interact with the browser like a real user (the fill() method uses the JavaScript API and does not emulate clicks nor keyboard keys):

nb interaction 1

Click$#goto-login-page-link 2

Click$#submit

Recording traces

To record a trace of interactions, we monitor the web application and listen to the mouse and keyboard events. More precisely, we built a simple JavaScript script that listens to the mousedown and keydown events. Each time such an event occurs, the script creates the corresponding interactions and sends them to a server responsible for storing the trace. Thanks to the Chrome extension API, this script is injected into any page and executed just after loading. Listing 4 presents the source code of the script for mouse interactions. This script generates an interaction for each mousedown event and sends it to the server.

DIFFING TRACES

A trace captures the interactions performed through the web interface (clicks and keyboard strokes). It can be recorded when a test is run (test trace) or when a user browses the web application (user trace). Diffing a test trace with a user trace highlights their differences in terms of interaction. It therefore pinpoints the interactions that are missed by the test, the ones that are done by the test whereas the user does not perform them, and any other kinds of discrepancies.

This section presents our approach for diffing two traces. It starts by explaining how we use Myers' algorithm to diff two traces. Then, it describes our graphical visualization of the diff that uses screenshots attached to the trace. Our approach is designed to help testers understand the differences between the tests and the users to help them decide whether-and how-the tests should be updated to match the users.

Myers' Algorithm for Diffing Interactions Traces

A trace represents a sequence of interactions. From a syntactical point of view, each interaction is represented by a token, and the trace can therefore be seen as simply a sequence of tokens. Diffing two traces is, therefore, the same as applying a sequence differencing algorithm such as the one proposed by Myers [START_REF] Myers | An O(ND) Difference Algorithm and Its Variations[END_REF]. The principle of Myers's algorithm is to find the longest common subsequence (LCS) between the two sequences. Using this longest common subsequence, it deduces the tokens that are conserved between the two traces (those belonging to the LCS), the deleted tokens (tokens of the left trace not belonging to the LCS), and the inserted tokens (tokens of the right trace not belonging to the LCS). For instance, if we diff the traces obtained by running the tests of Listing 1 and Listing 2, we obtain the diff shown in Table 1. In this table, the conserved tokens are represented by the white line containing two equal (=) signs. The inserted tokens are represented by the green lines with a plus (+) sign. The deleted tokens are represented by the red lines with a minus (-) sign. In the diff terminology, a sequence of consecutive red and green lines between white lines is called a hunk. In Table 1, there is only one such hunk consisting of 4 green lines (lines 2 to 5). This hunk indicates that in the second trace, First trace Second trace = Click$#goto-login-page-link = Click$#goto-login-page-link there are four interactions that were not performed in the first trace.

+ Click$#username + Edit$#username + Click$#password + Edit$#password = Click$#submit = Click$#submit
If we now compute a diff between test runs of Listing 2 and Listing 3, we obtain the diff shown in Table 2. This diff contains

Second trace

Third trace

= Click$#goto-login-page-link = Click$#goto-login-page-link = Click$#username = Click$#username = Edit$#username = Edit$#username - Click$#password + Tab$#username = Edit$#password = Edit$#password - Click$#submit + Click$form > div.button
Table 2: Myers' algorithm output for the traces of 2 and 3 two hunks. The first hunk contains one green line and one red line (lines 4 and 5). This hunk indicates a Click interaction in the second trace, while in the third trace, there was a Tab interaction. The second hunk also contains one green and one red line (lines 7 and 8). This hunk indicates a Click interaction in both traces but on a different target, as we can deduce from the different selector value.

More generally, there are three types of hunks. Firstly, hunks consisting only of green lines, such as the one in Table 1. These hunks represent interactions performed in the right trace but not in the left trace, i.e., interactions performed by the users only, not by the tests. Conversely, there are also hunks consisting only of red lines. These hunks represent interactions performed in the left trace but not in the right trace, i.e., interactions performed by the tests only, not by the users. Finally, there are hunks composed of both red and green lines, such as the ones in Table 2. These hunks represent interactions performed differently by tests and users, for instance because they interacted with different elements or because they interacted with the same element with a different action.

Visualizing the diff

In the previously described presentation of the diff, using the tokens is not very readable for testers. We then propose a graphical visualization that uses the tokens and the screenshots that were recorded at the time the interaction took place as well as the bounding rectangle highlighting which specific element was the target of the interaction. We still use the green and red notations to display the hunks. As an example, Figure 4 displays the beginning (lines 1 and 2) of the diff shown in Table 1. This visualization shows that the first interaction is the same for the two traces and that the trace of the second version has an interaction that is not present in the trace of the first version. The main difference with Table 1 is that the tester now clearly sees in the screenshot which actions took place when the two traces diverged. 2. One notable difference to the table is that we pack the red and green lines to save vertical space. The hunk displayed in Figure 5 clearly shows that in the second trace, a click was made on the password input, while in the third trace, the tab key was pressed while the focus was on the username input.

EVALUATION

This section presents the evaluation of our approach using a field experiment [START_REF] Stol | The ABC of Software Engineering Research[END_REF]. We conducted this experiment with our industrial partner Mr Suricate, a company specialized in implementing and running E2E tests for external customers. 4 The goal was to use our approach on real test scenarios chosen by the company to answer this research question: is our approach effective in pinpointing relevant differences in the test traces?

Context

Mr Suricate covers the entire scope of its clients' test campaigns, including creating the E2E test suites and scenarios, alert notification, test execution, and the creation of incident sheets in the event of a test failure. It is a young growing company with many clients in different business domains (eCommerce, banking, energy, etc.). In a nutshell, their clients express their test objectives, and Mr Suricate creates the test scenarios, implements them, runs them every day, and reports anomalies whenever they arise.

To optimize the creation and the implementation of the test scenarios, Mr Suricate has developed a graphical domain-specific language (DSL), built atop Blockly, 5 with a dedicated runtime engine. The DSL mainly consists of basic building blocks (e.g., for clicking on elements, filling text fields, navigating to a page) and composite blocks (e.g., for executing actions in sequence or in parallel). Based on the Playwright framework, the runtime engine compiles the scenarios written in this DSL into JavaScript files and executes them. 3: The six employees who participated to our experiment

Corpus and Methodology

We worked closely with two test managers of Mr Suricate to conduct our field experiment. They gave us five real test scenarios coming from five different external clients. They also asked the clients to open their scenario and to accept to publish some information: two accepted (Manitou Group 6 and Zadig & Voltaire7 ) and the three others preferred to remain anonymous.

The test managers provided us with a virtual machine running the runtime engine which allowed us to run the tests at any time. After instrumenting their runtime with our trace recorder (cf. Section 3.2), we then ran the five scenarios to obtain the five corresponding traces generated from the execution of the tests.

Then, six employees of Mr Suricate (five testers and one product owner) participated in our experiment. Each of them acted as a real user that would attempt to realize the actions described in the scenarios. They carried out the scenarios using an instrumented version of the Chrome browser that captured their traces in the same way we recorded the tests. We decided to assign each tester to a different scenario and the PO to the Zadig & Voltaire scenario. We thus obtained six user traces. Table 3 lists the participants, their age, their experience, their role in the company, and the scenario they were assigned (𝑆_𝐴, 𝑆_𝐵, and 𝑆_𝐶 denote the three scenarios for which the corresponding company preferred to remain anonymous).

Finally, we invited the testers and the PO to use our diff visualization to compare their own trace with that of the corresponding test, to know whether the diff helps them to better understand and possibly improve the automated test. We asked the testers and the PO to mark each interaction in each hunk as either "Of Interest" or "Not Interesting" (cf. Figure 6). If an interaction is marked as "Of Interest", it means that the difference triggered the tester to investigate the test and potentially improve it.

Our field experiment is then composed of six cases (one for each of the six employees). A case is composed of three parts: a user trace, the corresponding test trace, and the diff with the marks of interest. Thanks to these six cases, we first answer our research question through quantitative analysis. When a hunk is marked as "Of Interest", it means that the diff visualization pushed the testers to question their tests and, potentially, to improve them so that they can be closer to the behavior of a real user. The more hunks end up being marked as "Of Interest", the more tests may be improved. Section 5.3 presents the metrics we measured and the results of our quantitative analysis.

To better understand the differences between the test traces and the user traces, two of the authors inspected all the hunks and built a taxonomy of their underlying causes. As it is a subjective task, the two authors worked together to reach a consensus. Finally, an interview session was conducted with the two test managers of Mr Suricate where we shared with them the results of our experiment and asked them to give their opinion on the taxonomy. Section 5.4 presents this analysis.

Quantitative analysis

We first made some descriptive statistics by counting:

• the number of interactions in the test trace (#TestI),

• the number of interactions in the user trace (#UserI).

Table 4 presents all the metrics we measured. First, it shows that the number of interactions in the test traces (#TestI) ranges from 21 to 69. This highlights the fact that the five scenarios are quite complex as they consists of a significant number of steps. It should be noted that the Zadig & Voltaire scenario is the most complex, with a total of 69 interactions. This scenario consists in choosing several products in the webshop and then buying them after creating a user account.

The descriptive statistics shows that some user traces have more interactions than the test traces, while others have less. This depends on the scenario but also on the experience of the participants. As a matter of fact, the less experienced participant did not strictly follow the steps of the scenario, and performed much more interactions than the other participants. Two other participants, 𝑆 𝐴 and 𝑆 𝐸 , performed less interactions than the test. In the case of 𝑆 𝐴 , the tester did not manage to finish the scenario before the session was over. In the case of 𝑆 𝐸 , the test performs interactions that can be avoided by a real user; we discuss this point later in Section 5.4.

We then measured the following metrics on the diff:

• the number of interactions common to the test and user traces (#Sim), • the number of hunks (#Hunks),

• the number of hunks that contain interactions of the test and the user traces (#HTU),

• the number of hunks that contains only interactions of the test trace (#HT_), • the number of hunks that contains only interactions of the user trace (#H_U), • the number of interactions contained in the biggest hunk (HChurn).

The metric #Sim shows that the test trace and the user trace share a lot of interactions. The ratio #Sim / #TestI ranges from 0.40 (Zadig & Voltaire) to 0.86 (𝑆 𝐴 ), which is not surprising as Zadig & Voltaire is the most complex scenario. The metric #Hunks shows that the discrepancy is high. The metrics #HTU, #HT_ and #H_U give more information. First of all, #HTU shows that most of the hunks contain interactions of the test and the user traces. This means that the traces diverge in a lot of ways. Some hunks (#HT_) only contain interactions of the test trace. We show in Section 5.4 that they highlight cases where the test performs interactions that are not useful anymore. Some hunks (#H_U) only contain interactions of the user trace. We show in Section 5.4 that they either identify cases where the user performed additional extra interactions or highlight cases where the test misses to perform some interactions. The metric HChurn show that the biggest hunk belongs to the Zadig & Voltaire scenario, which is not surprising as it is the most complex scenario. Finally, the #Mark metric shows that there are hunks that contain at least one interaction of interest. If we consider that the object of interest is the hunk, 12% of the hunks (8/64) are interesting. Their existence asks for a deeper investigation of the tests to check whether they may be improved. This low score shows that our approach lacks precision as it produces many hunks that are not interesting. If we now consider that the object of the interest is the couple (user trace, test trace), then 66% are interesting. Finally, the two cases of the Zadig & Voltaire scenario show that the results of our approach also vary with the participant.

As a short conclusion, this quantitative analysis shows that our approach pinpoints discrepancies between the user and test traces. Participants expressed that some of these discrepancies need more investigation and may end with some test improvement. The analysis also shows that our approach identifies many hunks that are not considered to be of interest (86%) and that the results are pretty subjective.

Hunks analysis

Two authors analyzed the 64 hunks of the experiment and identified six main causes of discrepancy. They then interviewed the test managers and asked them to get their opinion on the causes.

Tab vs click.

Focusing on an element of a web application can be done either by clicking on it or using the Tab key. The test traces of our experiment mostly contain clicks; tabs rarely happened. The user traces contain clicks and tabs, depending on the participants. This difference in practice is, therefore, a cause of the discrepancy. For example, Figure 7 shows such a hunk in the Manitou Group scenario. The test focuses on the password field by clicking on it, whereas the user uses the Tab key. The two test managers agreed that this case of discrepancy is important as the test should be as close as possible to the user's behavior. For instance, a button is frequently composed of a surrounding bloc (<div>), an image (<img>) and a legend (<span>). However, depending on which element is clicked, our approach returns different interactions (the CSS selector is unique for each element), which is a cause of discrepancy. For example, Figure 8 presents such a difference in the Manitou Group scenario. The test clicked on the legend, whereas the user clicked on the surrounding block. The two test managers agreed that this case discrepancy is not as important. They argue that the widget itself is of importance, not the technical details of its implementation.

Missing Interaction.

A testing framework may sometimes use the JavaScript API to focus and interact with the elements. The test trace does not contain the corresponding interactions in such a case, whereas the user trace does. Indeed, a user cannot use the JavaScript API and always uses the mouse or the keyboard. Figure 9 presents an example on the Zadig & Voltaire scenario. The Figure shows a form with several fields. The user trace (on the right) registered a click on the element, whereas the test trace (on the left) did not (there is no interaction). The two test managers said that this cause of discrepancy is very important and asked for an improvement of the testing framework. Their framework already extends the fill() method of the Playwright Framework. Their extension modifies the default behavior of this method by making an explicit click on the fields and by using the keyboard to fill them. Our experiment made it possible to identify new situations where their framework still uses the JavaScript API. Therefore, they wanted to investigate these situations with the objective to potentially improve their framework.

Web Application

Changes. There are some changes, which are performed on a web application, that do not break the tests. For example, this is the case when the changes aim to ease the user experience by automatically filling the fields. These changes do not have any impact on the test trace as the test always fills the fields. They, however, have an impact on the user trace as the user is not required to fill the fields anymore. Figure Figure 10 presents an example on the Zadig & Voltaire scenario. The form is now automatically filled, but the test still explicitly fills it. This is to be contrasted with the user trace, where no such action happened. The two test managers were interested by this case of discrepancy, as the tests should remain consistent with the web application's behavior, even when it does evolve. 5.4.5 User-Side Interactions. Participants frequently clicked or used their keyboard aside from the scenario. The test never performs these interactions. Figure 11 presents an example on the Manitou Group scenario: the figure shows that the participant clicked somewhere on the form, even though it was not required 5.4.6 User Deviations. Participants of our evaluation sometimes deviated from the scenario, leading to hunks that may be large. For example, Figure 12 presents an example on the Zadig & Voltaire scenario. The figure shows that the participant chose a different value than the test for a text field. This difference is due to the sometimes loose steps in a scenario (here, "create an account"). The two test managers said that this case might be interesting even if they agreed that most of the hunks would lead to nothing. They argue that this kind of hunks may help to update a fuzzy test in some very rare situations.

Threat to validity

Our approach aims at measuring and comparing the clicks and the keyboard strokes performed by automated tests and by real users. Our field experiment on five industrial scenarios with six testers confirms that our approach accurately abstracts the interactions, and there is no threat to construct validity.

In total, 629 interactions have been realized in our evaluation. We cannot assume that all kinds of interactions have been realized. However, we can consider that all of the commonly performed interactions have been observed, limiting the threat to content validity.

We agree that the results we obtained are influenced by the scenarios, the participants, and the managers, which is a threat to internal validity. Furthermore, the results we obtained cannot be generalized much beyond the specific web applications we have been studying, which is a threat to external validity. To limit these threats, we conducted a field experiment with a company that built a considerable experience in E2E testing. We also worked on large industrial scenarios representing complex E2E tests.

RELATED WORK

In this section, we describe the related work on the domain of E2E web testing. E2E web tests are known to be fragile [START_REF] Hammoudi | Why do Record/Replay Tests of Web Applications Break?[END_REF], therefore researchers have extensively worked on means to mitigate this issue. To the best of our knowledge, no other approaches have been introduced to precisely compare two interactions traces like ours. However, there is a body of complementary work in three main domains, as follows.

Automated web test repair

Researchers have investigated how to repair broken web tests [START_REF] Stocco | Visual web test repair[END_REF][START_REF] Brisset | Erratum: Leveraging Flexible Tree Matching to repair broken locators in web automation scripts[END_REF][START_REF] Choudhary | WATER: Web Application TEst Repair[END_REF][START_REF] Hammoudi | WATERFALL: An Incremental Approach for Repairing Record-replay Tests of Web Applications[END_REF][START_REF] Chen | An Improving Approach for DOM-Based Web Test Suite Repair[END_REF]. The objective of these approaches is to automatically repair web tests that are broken due to a variety of reasons, such as layout changes in the web application. They usually use a DOM analysis or a visual analysis (using computer vision algorithms) to identify and repair the cause of the breakage. In contrast to these approaches, we do not aim at automatically repair web tests, but rather to pinpoint subtle differences between the execution of a test and the behavior of a real user.

Improving the robustness of web tests

Several approaches have been described to decrease the odds of a test being broken by a change in the web application. There are two main categories of approaches. Approaches from the first category aim at improving the robustness of the CSS selectors used in the tests [START_REF] Leotta | Robula+: an algorithm for generating robust XPath locators for web testing[END_REF][START_REF] Brisset | Erratum: Leveraging Flexible Tree Matching to repair broken locators in web automation scripts[END_REF][START_REF] Leotta | Reducing web test cases aging by means of robust XPath locators[END_REF][START_REF]Using Multi-Locators to Increase the Robustness of Web Test Cases[END_REF][START_REF] Zheng | A Method of Optimizing Multi-Locators Based on Machine Learning[END_REF][START_REF] Bajaj | Synthesizing Web Element Locators (T)[END_REF]. Approaches from the second one aim at relying on visual cues to guide the test instead of CSS selectors [START_REF] Alégroth | JAutomate: A Tool for System-and Acceptance-test Automation[END_REF][START_REF] Leotta | Automated generation of visual web tests from DOM-based web tests[END_REF][START_REF]Pesto: Automated migration of DOM-based Web tests towards the visual approach[END_REF]. In our work, we leverage these approaches to synthesize selectors when the test or the user are interacting with the web application.

Web test debugging

Many approaches have been introduced to help developers to debug web pages [START_REF] Burg | Explaining Visual Changes in Web Interfaces[END_REF][START_REF] Chi | Doppio: Tracking UI Flows and Code Changes for App Development[END_REF][START_REF] Oney | FireCrystal: Understanding interactive behaviors in dynamic web pages[END_REF][START_REF] Mickens | Mugshot: Deterministic Capture and Replay for Javascript Applications[END_REF]. Two main challenges are addressed by these approaches. First, they aim at recording as precisely as possible what happened during a browsing session to enable developers to replay the session and reproduce the behavior. Second, they usually try to establish links between the DOM elements manipulated and the source code written by the developers. Nevertheless, they are not designed to quickly compare two different browsing sessions, like we do. However, in our approach we borrow the idea of the recording web session traces with a lower level of precision to support comparing two different sessions.

CONCLUSION

E2E tests are meant to reflect the behavior of real users to uncover the faults they might encounter using a web application. As testers are forced to make implementation choices when translating high-level validation scenarios into concrete automated tests using popular E2E frameworks such as Playwright or Cypress, the resulting tests may not always represent the behavioral of real users. In particular, we show that some interactions resulting from these tests may not even be feasible by real users. These discrepancies hurt the very premise of E2E testing.

In this paper, we proposed a novel method to record the interaction traces of both real users and E2E tests. Using a novel diff algorithm and visualization for interaction traces, we have shown that it was able to pinpoint when, where, and in which way the tests and the users differ.

Our field experiment with the Mr Suricate company showed that, while our approach may sometimes show information to testers that they do not find interesting, it was also able to pinpoint in several places that the tests were either out-of-sync with the application or too far from a real user's behavior. This prompted managers at the company to investigate some of their E2E tests, as well as their testing framework, which highlights the usefulness of our approach and tool.

For future work, we would like to expand our field experiment by including more users for each scenario to soften the subjectivity of our results. Ideally, we would like to record the behavior of real users interacting with the web application in production, and use these traces in the comparison with the E2E tests. To lessen the number of false positives in the diff, we also plan to exploit the information contained in the screenshots to match widgets more accurately. When the user and the test click on two different element, and thus two different selectors, the screenshots should allow us to recognize that they actually click on the same widget, and that no diff should be reported.

  (a) Landing Page. (b) Login Page.

Figure 1 :

 1 Figure 1: A simple web application with two pages: a landing page and a login page.

Listing 1 :

 1 Source code of the simple E2E test (version 1). const browser = await chromium.launch({ slowMo:1000, headless:false }); const page = await browser.newPage(); await page.goto('http://localhost:8090/'); await page.click('#goto-login-page-link'); await page.fill('#username', 'Jane'); await page.fill('#password','pass'); await page.click('form > div.button'); const testResult = await page.$eval('#logname', el => el.value === 'Jane');

Listing 2 :

 2 Source code of the simple E2E test (version 2). const browser = await chromium.launch({ slowMo:1000, headless:false }); const page = await browser.newPage(); await page.goto('http://localhost:8090/'); await page.click('#goto-login-page-link'); await page.click('#username'); await page.type('#username', 'Jane'); await page.click('#password'); await page.type('#password','pass'); await page.click('form > div.button'); const testResult = await page.$eval('#logname', el => el.value === 'Jane');

Listing 3 :

 3 Source code of the simple E2E test (version 3).

  await page.click('#username'); await page.type('#username', 'Jane'); await page.keyboard.press('Tab'); await page.type('#password','pass'); await page.click('#submit');

Figure 2 :

 2 Figure 2: Approach overview.

Listing 4 :

 4 A script for recording mouse interactions document.addEventListener('mousedown', mouseDown, true);

Figure 3 :

 3 Figure 3: Visualizing the interactions realized by the test depicted in Listing 2. The red rectangles highlight which element, identified by the corresponding CSS selector, is the target of the interaction.

Figure 4 :

 4 Figure 4: Visualization of lines 1 and 2 of the diff shown in Table 1

Figure 5

 5 Figure 5 displays another example based upon the lines 3 to 5 of Table2. One notable difference to the table is that we pack the red and green lines to save vertical space. The hunk displayed in Figure5clearly shows that in the second trace, a click was made on the password input, while in the third trace, the tab key was pressed while the focus was on the username input.

Figure 5 :

 5 Figure 5: Visualization of lines 3 to 5 of the diff shown in Table 2

Figure 6

 6 Figure 6

Figure 7 :

 7 Figure 7: A Click vs Tab hunk in the Manitou Group scenario

Figure 8 :

 8 Figure 8: A example of hunk in the Manitou Group scenario with a click on a same widget that yields two different interactions.

Figure 9 :

 9 Figure 9: An example of hunk in the Zadig & Voltaire scenario with no focus on the element.

Figure 10 :

 10 Figure 10: An example of an evolution of the Zadig & Voltaire web application that may ask for an update ot the test.

Figure 11 :

 11 Figure 11: An example of a side click made on the Manitou Group scenario.

Figure 12 :

 12 Figure 12: An example of a use deviation in the Zadig & Voltaire scenario. The test and the user chose different values for the same field.

Table 1 :

 1 Myers' algorithm output for the traces of 1 and 2

Table 4 :

 4 Tester -Scenario #TestI #UserI #Sim #Hunks #HTU #HT_ #H_U HChurn #Mark 𝑇 𝐴 -𝑆 𝐴 Experimental quantitative results

		54	43	29	9	5	2	2	10	1
	𝑇 𝐵 -𝑆 𝐵	22	22	19	3	3	0	0	4	0
	𝑇 𝐶 -𝑆 𝐶	61	68	45	14	8	3	3	8	4
	𝑇 𝐷 -𝑀𝑎𝑛𝑖𝑡𝑜𝑢𝐺𝑟𝑜𝑢𝑝	21	22	17	5	4	0	1	4	1
	𝑇 𝐸 -𝑍𝑎𝑑𝑖𝑔&𝑉 𝑜𝑙𝑡𝑎𝑖𝑟𝑒	69	63	28	16	13	1	2	18	0
	𝑇 𝐹 -𝑍𝑎𝑑𝑖𝑔&𝑉 𝑜𝑙𝑡𝑎𝑖𝑟𝑒	69	114	41	17	10	0	7	30	0

https://www.w3.org/TR/wai-aria-practices-1.1/

https://www.w3schools.com/tags/ref_keyboardshortcuts.asp

https://www.barrierbreak.com/have-you-ever-tried-to-access-the-webpage-usingkeyboard/

https://www.mrsuricate.com/

https://developers.google.com/blockly

https://www.manitou.com/

https://zadig-et-voltaire.com/

Conference'17, July 2017, Washington, DC, USA Xavier Blanc, Thomas Degueule, and Jean-Rémy Falleri mouseDown(event) { let prefix = 'Click'; let suffix = generateCSSSelector(event.target); let interaction = prefix + '$' + suffix; sendAction(interaction, SERVER _ URL); } Listing 5 presents the source code of the script for keyboard interactions. This script checks whether the target is editable or not. It then generates an interaction (a word) depending on the key that has been pressed and whether the target is editable. Finally, the word that abstracts the interaction is sent to the server. To ease the understanding of a recorded trace, we further attach a screenshot of the page at the time the interaction took place. This can be done thanks to the chrome extension API. We also attach the coordinates of the bounding rectangle of the target element. The screenshot and the bounding rectangle allow us to build a visualization of the traces highlighting the targets of the interactions with red rectangles. Figure 3 presents our visualization for three interactions of the trace obtained by running the test of Listing 2.


Y Khalaf

P El Hage

R El Hage

EDST-PR2N Lcpm

J Dimitrova Mihajlova

A Bergeret

P Lacroix

NEW FORMALDEHYDE-FREE PARTICLE PANELS MADE FROM AGRICULTURAL WASTES AND CHITOSAN

Keywords: Particleboards, miscanthus, spent mushroom substrate, olive pomace waste, chitosan

Current environmental problems, deforestation, and tensions over raw materials, are pushing manufacturers to substitute wood in the production of wood-based panels by other alternative lignocellulosic sources. In addition, the replacement of synthetic binders based on non-renewable, petro-sourced, and carcinogenic formaldehyde with other bio based and biodegradable materials is being sought with the aim of pollution reduction and environmentally friendly materials production. Thus, the present work is devoted to the development of new panels using a polysaccharide binder, chitosan, in presence of reinforcements based on miscanthus or spent mushroom substrates combined with olive waste residues.

METHODS

Miscanthus × giganteus (M) from the summer 2018 harvest comes from the EARL Ar Gorzenn Company (Pont Croix, France). The spent mushroom substrate (SMS) is recovered from the "Gourmet mushroom" farm located in Byblos, Lebanon. These resources underwent grinding followed by automated sieving before use. Olive pomace waste is collected at the "Ghaoui-Ghaoui" olive mill in Darbechtar located in northern Lebanon. This waste has undergone crushing and manual sieving to separate the Olive pomace (OP) from the Olive Stones (OS) waste. Chitosan having a degree of deacetylation greater than 90%, a viscosity of 30 to 100 cps and an average molecular weight of 250,000 g mol -1 was purchased from Glentham Life Sciences (Corsham-Wiltshire, United Kingdom).

Four panels were prepared from the different components as shown in Table 1. The binder solution is prepared by dissolving chitosan (4% w/v) in aqueous glacial acetic acid solution (2% v/v) at room temperature using a mechanical stirrer. 24 g of OP or OS are added to the chitosan solution followed by mechanical stirring for 5 minutes. The mixture is then poured onto 36 g of M or SMS particles followed by manual mixing. The mixture is left at room temperature for 30 min to promote reinforcements wetting. Preheating is then applied for 60 min at 105 °C. The partially dried mixture is placed in a Teflon-coated stainless steel mold (180 × 50 × 70 mm 3 ) and compacted using a backing mold and a hydraulic press to a thickness of 1 cm. Compaction is maintained overnight in an oven at 105 °C. The panels are recovered after cooling and demoulding.

The bulk density, porosity, morphology, water absorption capacity by immersion and mechanical compressive strength before and after immersion are evaluated for the various panels.

RESULTS

Density and Porosity

The average bulk density and the porosity rate are presented in Table 1. Despite the similarity of the total initial weight and the fixed mold dimension, the bulk density of the materials was different depending on the reinforcement type. Thus, the SMS-based panels are much denser than the M-based ones because of significant volume shrinkage of the samples which are probably related to the collapse of the microtubes inside the cells and a reaction of mycelium present in SMS [START_REF] Khoo | Development of formaldehyde-free bio-board produced from mushroom mycelium and substrate waste[END_REF]. The porosity rate decreases with increasing bulk density, and conversely. The most porous panels are MOS and MOP. However, SMSOP has the lowest porosity rate. This phenomenon could be explained by the softening and shrinking of the mycelium [START_REF] Khoo | Development of formaldehyde-free bio-board produced from mushroom mycelium and substrate waste[END_REF] after cooking/drying of the panels at 105 °C inducing a decrease in the intrinsic porosity of the SMSs.

Optical Microscopy

Fig. 1 presents the superficial observations of the different panels using a "Ladybird MZ1240 Trinocular" stereomicroscope. The shiny layer on the surface of the fibres corresponds to the presence of the chitosan binder. This can be considered as an indicator of a good wetting of the different particles by the binder. It is visible that the M and SMS are well dispersed and homogeneously oriented parallel to the surface. It appears that OS are dispersed as aggregates. A gap is clearly visible between the particles indicating the presence of pores.

Fig 1. Microscopic observations of the panels surface according to the formulations

Absorption Capacity

The absorption capacity (Δm %) of the panels versus time is presented in Fig. 2. It seems that Δm is independent of the nature of the reinforcements. SMSOS panels have the highest absorption capacity for the first 30 minutes, while for the same duration a low absorption capacity (Δm <10%) is noted for the other panels. After immersion for 5 days, the MOP panels exhibit significant weight gain at saturation, reaching the threshold of 40%. This agrees with the photos of the microscope which show the presence of the large-diameter pores for the MOP panel. The rest of the panels seem to have an intermediate weight gain (32-35%).

Mechanical Behaviour by Compression

Fig. 3 shows the compressive strength (MPa) values of the panels before and after immersion in water. SMSOP panels show the highest strength followed by M panels. This is explained by the presence of minor porosity for SMSOPs and the presence of mycelium. After immersion/drying, the compressive strength appears to be deteriorated for all panels. Only MOP panels seem to have close value to that obtained before immersion.

CONCLUSIONS

In this study chitosan-based panels using miscanthus and/or spent mushroom substrate with olive pomace waste was produced. Preliminary results show that panels are successfully prepared with an average density between 665 and 899 kg/m 3 . The absorption capacity of all panels is less than 40% by weight despite the extended immersion time. To complete this study, additional mechanical bending and internal bonding strength analysis, contact angle analysis and microscopic observations after immersion will be performed. The aim of this study is the production of fully bio-based particleboards using local and/or neglected materials for interior design applications.

Fig 2 .

 2 Fig 2. Absorption capacity of the panels versus time (A-for 30 min; B-for 5 days)

Fig 3 .

 3 Fig 3. Compressive strength (MPa) of the panels before and after immersion/drying depending on the formulations

Table 1 .

 1 Average bulk density and porosity rate of the panels as a function of the formulations

Porosity (%) Average ρapparente (kg/m 3 ) Fibbers (36 g /24 g) Sample name

	MOP	Miscanthus / Olive	665 ±80	16.2
		Pomace
	MOS	Miscanthus / Olive	685 ±9	19.4
		Stones
	SMSOP	Spent mushroom	899 ±47	10.0
		substrate / Olive
	SMSOS	pomace Spent mushroom	806 ±14	15.8
		substrate / Olive
		Stones

*For all panels: mchitosane = 4.5g; Vacetic acid 2% = 108 mL; ratiofibres/binder (%/% m/m) = 93/7