Jérémy Frey

Martin Hachet

Fabien Lotte

EEG-BASED NEUROERGONOMICS FOR 3D USER INTERFACES: OPPORTUNITIES AND CHALLENGES

3D user interfaces (3DUI) are increasingly used in a number of applications, spanning from entertainment to industrial design. However, 3D interaction tasks are generally more complex for users since interacting with a 3D environment is more cognitively demanding than perceiving and interacting with a 2D one. As such, it is essential that we could evaluate finely user experience, in order to propose seamless interfaces. To do so, a promising research direction is to measure users' inner-state based on brain signals acquired during interaction, by following a neuroergonomics approach. Combined with existing methods, such tool can be used to strengthen the understanding of user experience. In this paper, we review the work being undergone in this area; what has been achieved and the new challenges that arise. We describe how a mobile brain imaging technique such as electroencephalography (EEG) brings continuous and non-disruptive measures. EEG-based evaluation of users can give insights about multiple dimensions of the user experience, with realistic interaction tasks or novel interfaces. We investigate four constructs: workload, attention, error recognition and visual comfort. Ultimately, these metrics could help to alleviate users when they interact with computers.

INTRODUCTION

Interactive systems that rely on 3D User Interfaces (UI) are increasingly used in a number of applications including industrial design, education, art or entertainment [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF][START_REF] Jankowski | A survey of interaction techniques for interactive 3d environments[END_REF]. As such, 3DUI and interaction techniques can be used by many different users with many varying skills and profiles. Therefore, designing them requires adequate evaluation tools to ensure good usability and user experience for most targeted users [START_REF] Bowman | A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods[END_REF][START_REF] Jankowski | A survey of interaction techniques for interactive 3d environments[END_REF]. To do so, a number of evaluation methods has been developed including behavioral studies, testbeds, questionnaires and inquiries, among others [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF][START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF][START_REF] Jankowski | A survey of interaction techniques for interactive 3d environments[END_REF]. This resulted in the design of more relevant, efficient and easy-to-use 3DUI.

Nevertheless, there is still a lot of room for improvements in the currently used evaluation methods. In particular, traditional evaluation methods could either be ambiguous, lack real-time recordings, or disrupt the interaction [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]. For instance, although behavioral studies are able to account in real-time for users' interactions, they can be hard to interpret since measures may not be specific, e.g. a high reaction time can be caused either by a low concentration level or a high mental workload [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research[END_REF]. Questionnaires and other inquiry-based methods such as "think aloud" and focus group all suffer from the same limitation: resulting measures are prone to be contaminated by ambiguities [START_REF] Nisbett | Telling more than we can know: Verbal reports on mental processes[END_REF], social pressure [START_REF] Picard | Affective computing[END_REF] or participants' memory limitations [START_REF] Kivikangas | Review on psychophysiological methods in game research[END_REF]. There is therefore a need for more objective (or more precisely "exocentric", as defined in [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]) and continuous measures of the usability qualities of 3DUI that do not interrupt the user during interaction.

In order to obtain such measures of the user's inner-state during interaction, a recent promising research direction is to measure such states based on brain signals acquired from the user during interaction [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]. In other words, 3DUI could be evaluated by following a neuroergonomics approach [START_REF] Parasuraman | Putting the brain to work: Neuroergonomics Past, Present, and Future[END_REF]. So far, neuroergonomics have been applied mostly to study and assess 2DUI, see, e.g. [START_REF] Parasuraman | Putting the brain to work: Neuroergonomics Past, Present, and Future[END_REF]; [START_REF] Anderson | A user study of visualization effectiveness using EEG and cognitive load[END_REF]; [START_REF] Peck | Using fNIRS brain sensing to evaluate information visualization interfaces[END_REF]. 3D interaction tasks are more complex for the user since interacting with a 3D environment is more cognitively demanding than perceiving and interacting with a 2D one. Indeed, it requires the user to perform 3D mental rotation tasks to successfully manipulate 3D objects or to orientate him/herself in the 3D environment. As such it involves cognitive mechanisms not solicited, or barely so, in classical 2D interaction tasks, such as spatial cognition. Moreover, 3D environments being usually much richer and complex to interact with than desktop 2DUI, they also generally lead to more pressure on the user's cognitive resources. Finally, perceiving 3D environments is also different from 2D ones, due to their 3D content, this content being potentially displayed in stereoscopy, leading to different perceptual mechanisms.

Therefore, in this paper, we show how 3DUI can benefit from neuroergonomics, and how new neuroergonomics tools and research are needed for this purpose. We notably focus on electroencephalography (EEG)-based neuroergonomics, since EEG are both accessible, portable, non-invasive and provide a high temporal resolution. This makes them suitable to measure a number of cognitive phenomenons and constructs that are relevant to assess 3DUI, as we will see later in this manuscript. Overall this paper aims at identifying 1) the opportunities in EEG-based neuroergonomics for 3DUI, i.e. how useful metrics to assess 3DUI can be derived from EEG signals and 2) the remaining challenges, i.e. what still needs to be done to reach usable and useful neuroergonomics for 3DUI. This paper is organized as follows. Section Opportunities identifies the opportunities in EEG-based neuroergonomics for 3DUI, i.e. what can we learn from EEG signals about a given 3DUI. In particular, we will see that analyzing EEG signals during 3D interaction tasks can provides us with information about the ergonomics of both the input devices and interaction techniques (Subsection Input devices), and the output, e.g. the display of the 3D environment (Subsection Output devices). We will illustrate these opportunities with some of our recent works on this topic. Then, Section Perspectives and Challenges identifies the scientific challenges that we will have to solve to further advance the field, at the hardware (Subsection Hardware), software (Subsection Software), neuroscientific (Subsection Neuroscience) and humancomputer interaction (HCI, Subsection Human-Computer Interaction) levels

OPPORTUNITIES

As mentioned earlier in this paper, 3DUI requires the user to perform generally complex 3D interaction tasks (3D objects manipulation, navigation [START_REF] Bowman | 3D User Interfaces: Theory and Practice[END_REF]) as well as to perceive a generally complex 3D environment. This is a typical interaction loop, with the user sending commands to the application, here 3D interaction tasks, and the system providing feedback to the user, here displaying a 3D environment. Interestingly enough, EEG signals can be used to study both aspects, i.e. how suitable is the input device/technique provided to the user, and how suitable is the output, typically the way the 3D content is displayed. More precisely, we and others have shown that we can estimate from EEG signals, more or less accurately, whether a 3D input device or technique is cognitively difficult and intuitive to use. It is also possible to infer from EEG signals whether a 3D display is comfortable for the user. We illustrate these two points with examples taken from our recent works in the following sections.

II.1 INPUT DEVICES

Recently we have been studying 3DUI through 2 separate studies. We started to investigate the use of physiological sensors to assess mental workload over the course of a 3D manipulation task in [START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF]. Then, in [START_REF] Frey | A framework for electroencephalography-based evaluation of user experience[END_REF], we used specifically EEG in order to extend those results with more detailed measures and by encompassing more constructs, i.e. attention and error recognition. The realistic tasks that we have investigated demonstrate how EEG could be used to analyze complex 3DUI and give insights to designers while they develop new input modalities for HCI.

WORKLOAD DURING 3D MANIPULATION TASKS

We proposed in [START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF] to use EEG and physiological signalselectrocardiography (ECG), electrodermal activity (EDA) -to study mental workload during a 3D docking task with a novel input device that possesses multiple degrees of freedom.

Figure 1. Schematic view of a user performing 3D manipulations tasks with the CubTile input device.

Participants manipulated a device known as the CubTile, developed by the Immersion company 1 . The CubTile is made of 5 orthogonal touch surfaces (Figure 1). Thanks to those many planes, the CubTile enables users to manipulate 3D objects with more degrees of freedom than with traditional input devices; they can easily choose the axis onto which they perform scaling (with a "pinch" movement), rotation (when at least 2 fingers draw such motion) or translation (when fingers slide on one of the surfaces).

The task used in our experiment was previously developed to demonstrate such capabilities, where users had to assemble one by one parts of a 3D bridge. Participants had to complete a set of 7 docking tasks while their physiological signals were being monitored, by translating, rotating and scaling the bridge parts, in order to put them at the correct location. In particular, during the second task a time constraint was added and during the fifth task the controls were inverted on purpose, in order to test if we could measure a difference within physiological signals that would concur with such increases of difficulty.

With our approach, a calibration phase shall occur so that the system could learn which features are associated to a specific individual, during a task that is known to induce the studied construct. For workload estimation, we used a standard task known 1 http://www.immersion.fr/ as the N-back task [START_REF] Kirchner | Age differences in short-term retention of rapidly changing information[END_REF], that here required the user to watch white letters on a black screen and to perform mouse-clicks. In the "0-back" (easy) condition participants simply have to discriminate between a target letter and distractors; in the "2-back" (difficult) condition, participants have to decide if the letter they see is the same as the one that appeared two times before -see [START_REF] Mühl | EEG-based Workload Estimation Across Affective Contexts[END_REF] for this implementation of the task. Hence, the N-back task plays on working memory, and it was used two induce two levels of workload -low and high. Once the calibration is completed, the machine could then use this knowledge to gain insights about a new context, here the 3D docking task (for comments about signal processing robust to context change between calibration and real task, see section Software).

Among the physiological signals that we investigated, EEG was the best able to discriminate workload levels reliably on the N-back task, with performances close to 90% on average for our 8 participants. This score is far beyond chance, which is 50% for a 2-class problem. In contrast, our analyses yielded poor performances with EDA (64.5% on average) and the classification accuracy of ECG signals was not significantly different from random. Hence, we focused our efforts on EEG while we pursued our evaluations. Once the baselines for the easy and difficult tasks were set, we processed the EEG activity recorded while participants were completing the 3D docking task. The results we obtained gave us unique insights into how much mental effort the participants were devoting to each task, and how this mental effort evolved over time (Figure 2).

When a constraint appeared concurrently with the second task -pillars were "falling" continuously from the sky and had to be positioned quickly before they touched the ground -the workload index increased substantially. This is consistent with the sudden pressure that was exerted on users. As one could expect, the mental workload lowered and settled in tasks 3 and 4, during which there was no more time pressure. The inverted commands during the fifth task is the moment when the workload index was the highest on average among all participants. Then, after this sudden surge of mental stress, once again the measured workload has been reduced in the two subsequent tasks, where the commands were back to normal -see [START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF] for further discussions.

Overall, the mental workload that was measured with EEG along the course of the interaction matches the design of the tasks. Those results suggest that EEG may be a suitable tool for evaluating complex 3D interaction tasks, even when an interaction technique as novel as the CubTile is involved. We built upon this study to create an assessment of mental workload more precise and to integrate new dimensions of the user experience, such as the measure of attention.

FRAMEWORK FOR AN EEG-BASED EVALUATION OF USER EXPERIENCE

In Frey et al. (2016) we went beyond the results obtained previously with 3D manipulation tasks. Here the objective is to validate the use of EEG as an evaluation method for HCI in a controlled and carefully crafted environment, with various interaction techniques (Figure 3). We contribute a set of methods to estimate continuously the user's mental workload, attention level and recognition of interaction errors during ecological and realistic interaction tasks. Indeed, such constructs are traditionally evaluated during controlled lab experiments based on protocols from psychology that are vastly different from an actual interaction task, see, e.g. [START_REF] Grimes | Feasibility and pragmatics of classifying working memory load with an electroencephalograph[END_REF]. The virtual environment takes the form of a maze where players have to learn and reproduce a path. A character displayed with a third person perspective moves by itself at a predefined speed inside orthogonal tunnels. Soon after the character enters a new tunnel, symbols appear on-screen, that users have to select timely in the right order. We crafted 4 levels of difficulty by tuning the speed or the number of items to remember. In order to make sure that we could elicit various mental states, we validated those levels in a pilot study using the NASA-TLX questionnaire [START_REF] Hart | Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research[END_REF], a well established questionnaire that accounts for workload.

Figure 4. The virtual environment, where players control a character that moves by itself inside a 3D maze. Players have to select directions using the character as a frame of reference. Hence, when the character is on the right ceiling (right image), user have to press right in order to go up.

Beside learning a sequence, the principal challenge comes from how the directions are selected. The input device that users are controlling is mapped to the character position (Figure 4). Indeed, the character slides by itself from the bottom of the tunnel to one of the walls or to the ceiling from time to time. When the character is upside down for example, commands are inverted compared to what players are used to. This discrepancy between input and output is a reminder of the problematic often observed with 3DUI, where most users manipulate a device with 2 degrees of freedom (DOF), such as a mouse, to interact with a 6 DOF environment.

In particular, in this study we compared two interaction techniques: a keyboard and a touchscreen. We chose to use the particularity of the touch screen to make the task more difficult. Indeed, while we used a touch screen for its possibility of direct manipulation, we kept the character as a frame of reference, resulting in non-intuitive input commands, that were not co-localized with output directions. Applying the same workflow as in the previous section, we managed to measure through EEG a workload index that varied depending both on the level of difficulty and on the interaction technique. The differences were significant and matched the intended design: workload increased with the difficulty and was overall more important when participants used the touchscreen, more complicated to handle on purpose. Notably, we were able to measure a near-continuous index of workload with EEG (2second time windows). E.g. the best performer during workload calibration shows patterns that clearly meet the expectations concerning both difficulty levels and interactions, as pictured in Figure 5.

Moreover, we were able to measure two other constructs alongside workload during this study, attention and error recognition. Attention refers to the ability to focus cognitive resources on a particular stimulus [START_REF] Kivikangas | Review on psychophysiological methods in game research[END_REF]. In HCI, measuring the attention level could help to estimate how much information users perceive. In the present work the measure of attention relates to inattentional blindness; i.e. it concerns participants' capacity to process stimuli irrelevant to the task [START_REF] Cartwright-Finch | The role of perceptual load in inattentional blindness[END_REF]. We calibrated and measured attention by playing "odd" sounds (high pitched bell) in the background, based on [START_REF] Burns | Use of auditory event-related potentials to measure immersion during a computer game[END_REF]. Results showed that in the high difficulty condition the attention level of participants toward external stimuli was significantly lower -i.e. inattentional blindness increased (Figure 6, left).

Error recognition relates to the detection by users of an outcome different from what is expected [START_REF] Nieuwenhuis | Errorrelated brain potentials are differentially related to awareness of response errors: evidence from an antisaccade task[END_REF]. We focused on interaction errors [START_REF] Ferrez | Error-related EEG potentials generated during simulated brain-computer interaction[END_REF], which arise when a system reacts in an unexpected way, for example if a touch gesture is badly recognized. Interaction errors enable to assess how intuitive a UI is, and they are hardly measurable by another physiological signal than EEG. We calibrated error recognition using the task described in [START_REF] Ferrez | Error-related EEG potentials generated during simulated brain-computer interaction[END_REF]. Maybe because said task was too simplistic compared to our virtual environment -moving a 2D cursor on screen vs navigating in a 3D maze -, the number of interaction errors differed only by a tendency when we processed EEG signals, with the touchscreen that raised slightly more interaction errors than the keyboard (Figure 6, right). While tenuous, this result is coherent with our hypotheses.

One should acknowledge that when it comes to recordings as sensitive as EEG, artifacts such as the ones induced by muscular activity are of major concern. The way we prevented the appearance of such bias in this study was threefold. 1) The hardware we used -active electrodes with Ag/AgCl coating -is robust to cable movements, see e.g. J. [START_REF] Wilson | BCI hardware and software[END_REF]. 2) The classifiers were trained on features not related to motion artifacts or motor cortex activation. 3) The position of the touch screen minimized participants' motion, and gestures occurred mostly before the time window used for detecting interaction errors. To further control for any bias in our protocol, we ran a batch of simulations where the labels of the calibration tasks had been randomly shuffled. Should artifacts bias our classifiers, differences would have appeared between the keyboard and touch screen conditions, even with such random training. Among the 20 simulations that ran for each of the 3 constructs (workload, attention, error recognition), none yielded significant differences.

Thanks to the ground truth obtained during the pilot study with the NASA-TLX questionnaire, these results validate the use of a workload index measured by EEG for HCI evaluation and set the path for two other constructs: attention and error recognition. Beside the evaluation of the content (i.e. difficulty levels) we were able to compare two interaction techniques. These are promising results for those who seek to assess how intuitive a UI is with exocentric measures, or for people who design new interaction techniques.

II.2 OUTPUT DEVICES

EEG is not only an opportunity to frame more precisely the quality of input devices, it can assess dimensions of the user experience that relate to output devices as well. For instance, in Frey, Appriou, [START_REF] Frey | Estimating Visual Comfort in Stereoscopic Displays Using Electroencephalography: A Proof-of-Concept[END_REF]; Frey, Appriou, Lotte, & Hachet (2016) we explored how EEG can be used to monitor visual comfort during the display of stereoscopic images. We present the first system that discriminates comfortable conditions from uncomfortable ones during stereoscopic vision using EEG.

Indeed, with stereoscopic displays a sensation of depth that is too strong could impede visual comfort and may result in fatigue or pain -whenever devices use shutter or polarized glasses (respectively "active" and "passive" displays), parallax barrier ("autoscopic" displays, e.g. Nintendo 3DS) or head-mounted displays (as with the Oculus Rift) to produce pairs of images. Some viewers could even feel pain [START_REF] Lambooij | Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review[END_REF].

In order to mitigate those symptoms and adapt the viewing experience to each user, we propose an innovative method that can discriminate uncomfortable situations from comfortable ones. It reacts quickly (within 1s), without calling upon users, so it does not disrupt the viewing. Our solution is versatile because all stereoscopic displays use the same mechanism to give the illusion of depth. They send a different image to the left and right eyes. As with natural vision, the visual fields of our eyes overlap and the difference between the two images helps our brain to estimate objects' distance. However, as opposed to real-life, where eyeballs rotate and eyes focus accordingly to the distance of objects, with stereoscopy the eyes remain focus on a fixed plane (the display) while the eyeballs rotate depending on the virtual depth. This discrepancy between vergence (eyeball rotation) and accommodation (crystialline lenses that focus light beams) is known as the "vergence-accommodation conflict" (VAC).

VAC causes stress on users [START_REF] Lambooij | Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review[END_REF]. The closer or further a virtual object gets compared to the display plane, the stronger this conflict is. When it is too important or lasts too long, visual discomfort occurs. VAC is one of the major causes of the symptoms associated to visual fatigue in stereoscopic displays [START_REF] Hoffman | Vergenceaccommodation conflicts hinder visual performance and cause visual fatigue[END_REF][START_REF] Lambooij | Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review[END_REF]. Shibata et al. [START_REF] Shibata | The zone of comfort: Predicting visual discomfort with stereo displays[END_REF] established a "zone of comfort" using questionnaires, a zone within which the apparent depth of objects should remain to avoid discomfort for viewers. Unfortunately, because of individual differences [START_REF] Lambooij | Visual Discomfort and Visual Fatigue of Stereoscopic Displays: A Review[END_REF], it is hard to generalize such recommendations and use them as is. Besides, viewing conditions vary. Ambient light, screen settings, viewing angle and stereoscopic techniques are parameters among others that influence the rendering and as such alter visual strain [START_REF] Bangor | Display technology and ambient illumination influences on visual fatigue at VDT workstations[END_REF]. It may then be interesting to back up the VAC with another type of measure than sporadic and disruptive questionnaires. Finding a way to measure continuously such conflict would help to prevent the appearance of symptoms or to adapt on-the-fly displays to reduce stress when it is needed.

Figure 7. In order to investigate visual comfort with stereoscopic displays, still objects were presented at various virtual depths for a few seconds. C: comfortable condition; NC: non-comfortable condition.

In our experiment, we studied the appearance of virtual objects, still primitives such as cubes, cylinders or icospheres. We studied two conditions: objects appeared at a comfortable position ("C" condition, slightly before or behind the screen) or at an uncomfortable position ("NC" condition, very far away from the screen or very close to viewers) -Figure 7. Those conditions were validated with a questionnaire analogous to that of [START_REF] Shibata | The zone of comfort: Predicting visual discomfort with stereo displays[END_REF].

By studying the EEG activity that followed images' appearance, we were able to train a classifier to discriminate between both conditions with an average accuracy of 63.3% in single-trial (1-second time window), a percentage significantly different from chance, that went above 70% for one third of our participants. These scores, while not as high as some other established BCI systems, may be already sufficient to improve users' comfort. Indeed, on-the-fly correction of uncomfortable images can be seen as error correction, and in such settings detection rates from 65% are acceptable to improve interactions [START_REF] Vi | Detecting error-related negativity for interaction design[END_REF]. The reliability of the system improved by a fair margin when we simulated the aggregation of several trials -up to 90% with 5 trials -suggesting that in a real-life scenarios the performance would increase if the system waits for several measures before making an assumption.

Using short time windows (features were extracted over 1s), we set the basics of a tool capable of monitoring user experience with stereoscopic displays in near real time. A passive stereoscopic comfort detector could potentially be useful for multiple applications, as a tool to compare with exocentric measures (possibly offline) different stereoscopic displays; or to dynamically adapt discrepancy to avoid discomfort (e.g. during 3D movies, should the adaptation be smooth and not disturb viewing), among many others.

It is interesting to note that such measure of visual comfort could well be combined with the investigation of others constructs, for example the measure of workload or attention level as described in the previous section. Encompassing several dimensions at once will help to maximize further the overall user experience, let it be beforehand during HCI evaluation or over the course of an interaction with neuroadaptive technologies [START_REF] Zander | Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general[END_REF].

To summarize, EEG holds great promises to study the ergonomics pros and cons of both input and output devices and techniques for 3DUI. It can indeed provide continuous and exocentric measures of the usability of an input device, technique or task, by estimating mental states such as mental workload, inattentional blindness or error recognition, even during rich and complex 3D interaction tasks. It can also provide measures of the level of comfort or quality the user perceives with the output devices and software, e.g. 3D displays, for instance by estimating how comfortable stereoscopic displays are. Naturally, these are recent works and examples, and not yet full-fledged solutions. Much still needs to be researched and developed to make EEG a fully mature evaluation tool for 3DUI. It nonetheless shows that EEG is a relevant and useful technology to do so, that needs to be further investigated, as discussed in the next sections.

PERSPECTIVES AND CHALLENGES

These different works and opportunities clearly showed the promising potential of a neuroergonomics approach based on EEG to evaluate 3DUI. However, research works in this direction are relatively recent, and there are still a number of scientific challenges that remain to be solved to be able to apply this neuroergonomics approach in practice to the design of 3DUI. In particular, there are challenges at the hardware level -notably regarding the EEG sensors used -; at the software level -regarding EEG signal processing algorithms and the accessibility of software to use them -; at the neuroscience level -to understand further existing cognitive mechanisms and identify new ones relevant for 3DUI evaluation -; and finally at the HCI level -to use in practice neuroergonomics methods during the design of 3DUI. These various challenges are described in more details in the following sections.

III.1 HARDWARE

As with all physiological measures, one limitation of EEG comes from the hardware, mandatory to make the measures and yet not necessarily practical to use in the laboratory or in the field. In this section we summarize issues that arise from the use of EEG and we list the characteristics of a forthcoming "ideal" EEG device -see also [START_REF] Frey | Leveraging human-computer interactions and social presence with physiological computing[END_REF] for deeper insights.

New EEG devices appeared on the market in the last years, oriented toward a larger public, with a lower price tag and a more comfortable use. As opposed to "wet" electrodes employed for medical research, "dry" electrodes are faster to set-up (no more conductive gel) but are less sensitive -see [START_REF] Blankertz | The Berlin Brain-Computer Interface: Non-Medical Uses of BCI Technology[END_REF]; J. [START_REF] Wilson | BCI hardware and software[END_REF]. Hence, some companies, while trying to transform EEG into a mass-product, bring less reliable technology to the market. Those devices often posses fewer electrodes. Lastly, without a helmet the electrodes are difficult to place in a standardized position on the scalp.

While there are already many devices that are practical to use thanks to dry sensors and wireless connection to computer -a must-have for ambulatory use-cases -it is not always easy to verify their claims concerning their accuracy. The use of proprietary or close software also prevent users from freely choosing their workflow, e.g. one may have to switch to another operating system because of poor support from the manufacturer.

Having the possibility to place electrodes at specific locations on the scalp is essential, especially because it would require very expensive and cumbersome hardware in order to cover at once all the positions. Unfortunately, most of the EEG headsets that are appealing from a user perspective are an all-inclusive solution were we must comply with the choice of the manufacturer -that sometimes differ from standard positioning [START_REF] David Hairston | Usability of four commercially-oriented EEG systems[END_REF]. Often, as it is the case with motor imagery, it would be preferable to have 8 electrodes packed around a narrow area of interest (i.e. C3/Cz/C3) than 16 electrodes trying to cover the whole head. The poor spatial resolution of EEG could be solved in part by the versatility of electrodes position. This is why we defined three requirements for an ideal EEG solution:



Open-source driver in order to process freely and knowingly data, whatever the platform or the operating system one chooses to use at a specific moment.



Open-source hardware in order to acknowledge beforehand the reliability of measured signals. When the specification and the schematics of hardware are a known variable it is easier to assess the precision of the measures. Characteristics such as the signal-to-noise ratio of the amplifier or the resolution of the analog-todigital converter (ADC) are critical. Even though traditional manufacturers give the overall specifications of their amplifiers, the electronic circuitry is the result of complex interactions and in practice many components influence devices' performance (Usakli, 2010).



Customizable electrode positioning, that is to said the possibility to place electrodes anywhere on the scalp. Moreover, this positioning should follow the 10-20 international system, or even better the 10-5 system (more refined separations along the sagittal reference curve), so that measures and findings could be compared together and with the existing scientific knowledge.

To which we add new items for a practical solution:

 Affordable, not only for every lab to be able to use them, or to make possible to use many different EEG headsets at the same time, but also for the technology to reach the general public -we have seen how EEG could be effective for neuroadaptive technologies.

 Lightweight, so it could be used in ambulatory settings. That implies a technology that is not a hindrance nor a health risk when it is used during a prolonged period.



Wireless, as no cables will ensure that users can move freely while interacting.



No or little preparation. The setup time is one tedious step both for participants and for the experimenter: fill electrode one by one with gel or other conducting solution, wait for good measures, sometimes check the impedance and correct the position accordingly, etc. Such operation could last for dozens of minutes if there are many electrodes. As such, dry electrodes seem the obvious choice.

With all these observations in mind, the appearance of a community-driven initiative such as OpenBCI 2 is a step toward the dissemination of EEG devices. It is open-hardware, the firmware and the proposed software suite are open-source, it is nearly a hundred times cheaper than the traditional equipment found in the laboratory, and both the form-factor design and the proposed 3D printable headset oversaw many possible uses.

Such a solution has downsides, of course. First, this particular device possesses only 16 electrodes, which limits its use -most of the work we presented in this paper relied on a 32 channels amplifier. The more channels we have the better, since a better 2 http://openbci.com/ scalp coverage enables more accurate measures. However, as mentioned at the beginning of this section, when the EEG markers are known a good placement can overcome this difficulty. More importantly, the practicality of the device should not let one forget about the reliability of the measures. While the first comparisons we made are encouraging and suggest very close recordings between the OpenBCI board and a medical grade equipment [START_REF] Frey | Leveraging human-computer interactions and social presence with physiological computing[END_REF], more thorough examinations are required before we could exactly know the advantages and the limitations that more users-driven solutions could offer to neuroergonomics.

III.2 SOFTWARE

At the software level, the main challenges are related to EEG signal processing, and in particular about how to extract reliable estimates of cognitive processes and constructs from EEG signals, despite their noisy and non-stationary nature. These two issues with EEG are particularly prominent when using EEG to study 3DUI. Indeed, EEG signals are rather sensitive to noise and artifacts, see, e.g. [START_REF] Goncharova | EMG contamination of EEG: spectral and topographical characteristics[END_REF], such as motion artifacts (due to users' movements), muscle tension artifacts (e.g. from face muscles) or electromagnetic interferences from nearby electronic equipment. Contrary to a lab 2DUI experiment in which the user is quietly sitting in front of a standard screen, all these types of noise are likely to occur with 3DUI. Indeed, with touch devices to a minor extent or, to a larger one, with full virtual reality (VR) equipment, the user will be moving more, or even standing or walking while using the 3D device, hence resulting in potential motion artifacts. Then, the use of heavy 3D equipment such as VR caves, 3D shutter classes or unconventional interaction devices such as haptic input devices or trackers, may lead to stronger electromagnetic interference. Finally, the richness and complexity of the 3D environment may lead to stronger emotions in the user, e.g. frustration or joy, which in turns could lead to stronger facial expressions and thus facial muscle artifacts. Although, to the best of our knowledge, there have been no report of strong EEG signal degradation due to the first two types of noise (motion artifacts and electromagnetic interference) during 3D interaction, developing EEG signal processing tools to clean such artifacts or to estimate cognitive states while being robust to them, should improve the reliability of the whole system.

Another reliability issue with EEG signals is their non-stationarity. This can be due to the electrodes' gel drying, or to changes in the fatigue or attention level of the user over the course of the experiment [START_REF] Roy | Mental fatigue and working memory load estimation: Interaction and implications for EEG-based passive BCI[END_REF]. As far as 3DUI is concerned, there is another source of non-stationarity that is prominent and a major problem: non-stationarity due to changes in context. Indeed, designing a cognitive state estimator, e.g. a workload level estimator, requires calibrating such estimator on examples of EEG signals from the user. To do so, classical cognitive state induction tasks are used, such as the N-back task for workload (Mühl et al., 2014;[START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF]. Indeed, these validated tasks ensure a proper calibration of the system as they ensure the EEG signals collected really correspond to the cognitive state we want to estimate in the EEG signals. However, these tasks are very different from the context in which the cognitive state estimator will be used. For instance, whereas the N-back task only involves letters' appearance, during a 3D interaction task, the user will be much more active, and confronted to a much richer environment and stimuli, which will change her/his EEG signals. As such, directly using an estimator calibrated on an N-back task to estimate workload in a complex 3D interaction task may simply fail. To avoid this problem, we designed a specific EEG signal processing algorithm to calibrate spatial filters robust to such context changes for mental state estimation based on EEG oscillations (Frey et al., 2016;[START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF]. In the future, it would be necessary to design similar algorithms to estimate mental states based on event related potentials (ERP), as well as to calibrate not only spatial filters that are robust to context changes, but also temporal filters and classifiers.

The calibration of the cognitive state estimators is also an issue in itself. Indeed, this calibration must be performed for each cognitive state we want to study and for each user. This is very time consuming and limits the number of states that can be studied simultaneously (e.g. in Frey et al. (2016) we studied 3 states simultaneously, which was already more than most works which study a single one), and may exhaust the user due to their length. It is therefore necessary to design EEG signal processing tools with reduced calibration time, or, ideally, no calibration at all -a so-called subjectindependent estimator. Recent works in that direction in the field of EEG-based Brain-Computer Interfaces should therefore be pursued [START_REF] Lotte | Signal Processing Approaches to Minimize or Suppress Calibration Time in Oscillatory Activity-Based Brain-Computer Interfaces[END_REF].

Finally, to make these neuroergonomics technologies practical, it is necessary to make them easily accessible and usable. In other words, 3DUI designers should be able to use these technologies without deep understanding of EEG or signal processing. Ideally, they should be usable with just a few clicks. Research and Development works should therefore be conducted to create easily usable software to perform EEG-based neuroergonomics evaluations. Solutions such as the open-source, real-time neuroscience OpenViBE software, are a first step in that direction [START_REF] Renard | OpenViBE: An Open-Source Software Platform to Design, Test, and Use Brain-Computer Interfaces in Real and Virtual Environments[END_REF].

III.3 NEUROSCIENCE

At the neuroscience level, there is also necessary research to be conducted, in particular to know more about cognitive states that could be relevant to study 3DUI, and how they are reflected in the EEG signals. In particular, there are a number of cognitive states that are relevant to assess interaction, and that could be estimated to some extent in average EEG signals, at least during lab experiments [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]. It remains to be shown whether they could be also estimated continuously, in singletrial, during complex, real-life interaction tasks. Notably, there are some evidences that the extent of error recognition -i.e. how much the interaction outcome differs from the expected one, is reflected in scalp EEG signals [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]. It remains to be shown whether it can be estimated continuously for each interaction task, for real applications. If that is so, this could be a very interesting measure of how intuitive is a 3DUI (or any UI by the way). Similarly, being able to measure the user's emotion in EEG signals, could provide valuable insights into the user experience. Although, EEG-based emotion recognition is still a challenging task so far [START_REF] Mühl | A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges[END_REF], this could potentially inform us about how enjoyable or frustrating a UI is.

III.4 HUMAN-COMPUTER INTERACTION

In the field of Human-Computer Interaction, evaluation of interfaces and systems play a central role. It allows validating new concepts, and it guides designers in their general conception choices. To achieve robust assessments that are scientifically valid, evaluation protocols need to be designed with great care. This notably concerns the tested population, the definition of the experimental conditions, the sequence in which different parameters are tested, and so on. It also concerns the data that is stored, and the way it is analyzed. Even if each experiment is different, there are general rules and practices that are widely shared among the community.

In the future, with the emergence of new physiological assessments as the ones we described above, the standard evaluation protocols will evolve. First experiments that are done in this direction are highly customized. A big challenge will be to standardized the protocols based on EEG ergonomics for tomorrow evaluations. A set of guidelines and standard procedures should be defined to help researchers in their evaluation work.

Compared to standard evaluation protocols, EEG-based protocols have several specificities that need to be taken into account. First, as stated in Subsection Software, the recorded data is strongly subject dependent, meaning that at the moment it is hardly possible to transfer a calibration from one participant to another -or even from one session to another -without great loss of accuracy. Even when gamification is used to make calibration tasks less tiresome [START_REF] Flatla | Calibration games: making calibration tasks enjoyable by adding motivating game elements[END_REF] or when calibration tasks are alternated with the main interaction (Frey et al., 2016), this requirement is tedious for participants.

Second, the need for an accurate synchronization between, from the one hand, EEG recordings and, from the other hand, the interactions performed by participants may also necessitate to tune the software that is being evaluated. For instance, each event of interest -e.g. target selection, beginning of a new task -should be associated to timestamps for offline analyses, or should "ping" the program that is recording EEG for online analyses. The need for such precise logging information is nowadays easier to fulfill thanks to protocol alike LSL 3 , that posses simple bindings in many programming languages, but it should still be considered beforehand when experiments are designed.

EEG-based evaluations bring new constraints. They also open new opportunities for HCI. In particular, they will allow fine assessments of the user interaction. Hence, UI designers will be able to inject the results of the experiments in the developed technique at an early stage of the creation process. We may assume that, with such kind of EEG-based evaluation -if they are powerful enough -, several cycles of standard evaluations, which would require a large number of users, could be saved. These new practices that redefine standard evaluation and design approaches should progressively appear in the research community, in conjunction with more traditional approaches.

CONCLUSION

We demonstrated how brain signals -recorded by means of electroencephalography -could be put into practice in order to gather insights about multiple dimensions of the user experience, whether input or output devices are considered, with realistic interaction tasks or novel interfaces. As a supplementary evaluation method, EEG brings continuous and exocentric measures that does not disrupt tasks performed by the users. So far we have investigated workload, attention and visual comfort, and we got encouraging results with interaction errors, a metric that relates to how intuitive an interface is.

While challenges lie ahead and need to be overcome for the methodology to truly disseminate -e.g. features robust to context changes or faster setup times -the increasing availability of EEG devices opens new possibilities. Thanks to a deeper understanding of users, EEG-based neuroergonomics will lead to better HCI in general, and, since 3D interactions are more demanding, to better 3DUI in particular.

Figure 2 .

 2 Figure 2. Workload index measured from EEG during various 3D docking tasks -the red cross pictures an outlier. A time constraint was added during task 2 and commands were inverted during task 5.

Figure 3 .

 3 Figure 3. A keyboard and a touchscreen are compared using EEG measures during realistic interaction tasks.

Figure 5 .

 5 Figure 5. Workload index over time for participant 3. Left: "keyboard" condition, right: "touchscreen" condition. Background color represents the corresponding difficulty level.

Figure 6 .

 6 Figure 6. Left: The attention index significantly differs across difficulties. Right: The number of interaction errors differs by a tendency between interaction techniques.

https://github.com/sccn/labstreaminglayer


Jérémy Frey

Martin Hachet

Fabien Lotte

Recent advances in EEG-based neuroergonomics for Human-Computer Interaction

Human-Computer Interfaces (HCI) are increasingly ubiquitous in multiple applications including industrial design, education, art or entertainment. As such, HCI could be used by very different users, with very different skills and needs. This thus requires user-centered design approaches and appropriate evaluation methods to maximize User eXperience (UX). Existing evaluation methods include behavioral studies, testbeds, questionnaires and inquiries, among others. While useful, such methods suffer from several limitations as they can be either ambiguous, lack realtime recordings, or disrupt the interaction. Neuroergonomics can be an adequate tool to complement traditional evaluation methods. Notably, Electroencephalography (EEG)-based evaluation of UX has the potential to address the limitations above, by providing objective, realtime and non-disruptive metrics of the ergonomics quality of a given HCI [START_REF] Frey | Review of the use of electroencephalography as an evaluation method for human-computer interaction[END_REF]. In this abstract, we present an overview of our recent works in that direction. In particular, we show how we can process EEG signals in order to derive metrics characterizing 1) how the user perceive the HCI display (HCI output) and 2) how the user interacts with the HCI (HCI input). First, at the output level, we conducted some experiments to study the visual comfort experienced by users of stereoscopic displays (e.g., 3D TV). We showed them several objects presented at different stereoscopic depths in front or behind the screen while recording their EEG signals. Some of these depths were voluntarily uncomfortable (either too close or too far from the eyes). By analyzing Event Related Potentials (ERP) following a stereoscopic object appearance using advanced EEG signal processing techniques, we showed we could discriminate, in a single trial (i.e., in 1s of signal), ERP corresponding to comfortable displays versus ERP corresponding to uncomfortable ones (Frey 2016a). Second, we asked our users to perform different cognitive tasks involving different levels of cognitive workload (i.e., mental efforts), while recording their EEG signals. Here as well, by using signal processing and suitable machine learning, we could discriminate low mental workload from high mental workload in short windows (2s of signal) of EEG (Mühl 2014). By refining these algorithms we showed we could then discriminate mental efforts in complex interaction tasks, notably during 3D object manipulation tasks [START_REF] Wobrock | Continuous Mental Effort Evaluation during 3D Object Manipulation Tasks based on Brain and Physiological Signals[END_REF], as well as during navigation tasks In a video game [START_REF] Frey | A framework for electroencephalography-based evaluation of user experience[END_REF]. We also showed we could use such methods to compare the mental workload induced by different interaction devices or techniques. Overall, these recent results suggest that EEG can be used as an objective and complementary evaluation technique to characterize and assess different HCI. This thus opens the door to a new generation of HCI, designed by exploiting EEG-based neuroergonomics.


Meredith Cox

Dong Wang

Jonathan Barés

Robert P Behringer

Self-organized magnetic particles to tune the mechanical behavior of a granular system

Keywords: 47, 57, Gc -Granular flow 81, 05, Rm -Porous materials; granular materials 78, 20, Fm -Birefringence

Above a certain density a granular material jams. This property can be controlled by either tuning a global property, such as the packing fraction or by applying shear strain, or at the micro-scale by tuning grain shape, inter-particle friction or externally controlled organization. Here, we introduce a novel way to change a local granular property by adding a weak anisotropic magnetic interaction between particles. We measure the evolution of the pressure, P , and coordination number, Z, for a packing of 2D photo-elastic disks, subject to uniaxial compression. A fraction Rm of the particles have embedded cuboidal magnets. The strength of the magnetic interactions between particles is too weak to have a strong direct effect on P or Z when the system is jammed. However, the magnetic interactions play an important role in the evolution of latent force networks when systems containing a large enough fraction of the particles with magnets are driven through unjammed to jammed states. In this case, a statistically stable network of magnetic chains self-organizes before jamming and overlaps with force chains once jamming occurs, strengthening the granular medium. This property opens a novel way to control mechanical properties of granular materials.

Above a certain density granular materials jam [START_REF] Liu | Nonlinear dynamics: Jamming is not just cool any more[END_REF], i.e. become mechanically stable. As the medium transitions from unjammed to jammed, its behavior changes from fluid-like to solid-like. From a theoretical perspective, jamming has typically been considered for systems of particles that only interact via contact forces. But little is known for the case we consider here, when there are additional weak particle interactions even when the particles are not in contact. From a more practical viewpoint, jamming has important consequences. Often it is a drawback as in a silo or in a funnel, but it can be used constructively as in recent innovative architecture constructions to create stable structures [START_REF] Dierichs | Aggregate structures: Material and machine computation of designed granular substances[END_REF][START_REF] Zhao | Packings of 3d stars: stability and structure[END_REF] and soft robotic actuators [START_REF] Brown | Universal robotic gripper based on the jamming of gran-ular material[END_REF][START_REF] Cheng | Design and analysis of a robust, low-cost, highly articulated manipulator enabled by jamming of granular media[END_REF][START_REF] Steltz | Jsel: Jamming skin enabled locomotion[END_REF] to reversibly change a soft material into a solid strong one. These studies highlight several ways to create or destabilize mechanically stable states [START_REF] Jaeger | Celebrating soft matter's 10th anniversary: Toward jamming by design[END_REF][START_REF] Keller | Aleatory architectures[END_REF]: changing the system volume/density [START_REF] Brown | Universal robotic gripper based on the jamming of gran-ular material[END_REF]; applying system-wide shear strain [START_REF] Bi | Jamming by shear[END_REF]; and locally, changing grain shapes [START_REF] Zhao | Packings of 3d stars: stability and structure[END_REF][START_REF] Miskin | Adapting granular materials through artificial evolution[END_REF][START_REF] Athanassiadis | Particle shape effects on the stress response of granular packings[END_REF], organization [START_REF] Dierichs | Aggregate structures: Material and machine computation of designed granular substances[END_REF], or friction [START_REF] Zhao | Packings of 3d stars: stability and structure[END_REF]. Reversibly changing the system proper-ties at the global scale is routine, but changing properties at the grain scale is much more challenging. We take a first step toward a novel manner to affect the macroscopic behavior of a granular packing by changing grain-scale properties. We do so by adding magnets to a controlled fraction of the grains. The magnetic interaction strength at the local scale is very weak compared to interaction forces that are typical of jammed states. Despite the low magnetic forces involved, the fluidity of the granular system below jamming provides a mechanism for self-organization of structures that is reminiscent of magneto-rheological fluids [START_REF] Weiss | Viscoelastic properties of magneto-and electro-rheological fluids[END_REF][START_REF] Hagenbüchle | Chain formation and chain dynamics in a dilute magnetorheological fluid[END_REF][START_REF] Ukai | Patterns formed by paramagnetic particles in a horizontal layer of a magnetorheological fluid subjected to a dc magnetic field[END_REF]. This organization can persist above jamming providing the concentration of magnetic particles is high enough.

The experiment consists of unixial compression (see Fig. 1-(a)) of a mixture of 645 bidisperse (60% small, 40% large) photo-elastic disks with either diameter 12.7 or 15.9mm, and thickness 6mm. This ratio prevents crystalline ordering. A 4 × 4 × 4mm 3 neodymium magnet was embedded in the bottom of a fraction R m of the disks as in Fig. 1-(b-Top), and a UV-fluorescent bar was drawn along the attractive direction of the magnet on the top of each particle. The particles were cast in a reverse-image chemically-resistant silicone mold [START_REF]Smooth-On[END_REF] that was formed over a high resolution 3D printed base. The disks were cast from shore stiffness 50 liquid polyurethane [START_REF]Smooth-On[END_REF] and then cured. For some particles, a magnet was centered at the bottom of the mold before pouring the polyurethane. This process avoided residual stresses in the polymer around the magnet. The friction coefficient between particles is 0.62 ± 0.07.

The disks were placed in a 44 × 40cm 2 (x, y) rectangular cell, in a spatially random stress-free compact rectangle with the long direction corresponding to the direction of compression. Initially, there was a ∼ 3cm gap between the particles and the cell boundaries to let particles rearrange freely before reaching the jammed state. The particles rested on a smooth Plexiglas R sheet lubricated with talc, yielding a particle-base friction coefficient of 0.36 ± 0.04. The setup was illuminated from below by a circular polarized uniform white light source, and from above by less intense UV lights. A 20 megapixel SLR camera, mounted 2m above the particles, recorded views with and without a crossed (with respect to the source) circular polarizer (see Fig. 1-(a)). After each 1mm compression step, the system was imaged (i) without the second polarizer, (ii) with the second crossed polarizer (see Fig. 1-(b)), and (iii) with the white light off and UV light on. The normal light pictures gave the particle positions. The UV light pictures gave the magnetic orientations. The pressure (P ) inside each disk was computed from the squared gradient of the photo-elastic image intensity G 2 m (for magnetic particles) and G 2 nm (for non-magnetic particles) following [START_REF] Howell | Stress fluctuations in a 2d granular couette experiment: A continuous transition[END_REF][START_REF] Junfei Geng | Footprints in sand: The response of a granular material to local perturbations[END_REF][START_REF] Ren | Reynolds pressure and relaxation in a sheared granular system[END_REF][START_REF] Behringer | Statistical properties of granular materials near jamming[END_REF]. The P ∼ G 2 values were corrected using the inset graph of Fig. 2 to account for differences in the different particle types, as suggested by Fig. 1-(b). That is, the calibration between squared gradient and P is different for magnetic, G m , and non-magnetic, G nm , particles primarily because the stiff magnet deforms the photo-elastic response. Contacts between particles are also accurately detected using positions and the photo-elastic response, as in Majmudar et al. [START_REF] Majmudar | Jamming transition in granular systems[END_REF]. The sensitivity of the polyurethane permits observation of a photo-elastic response of the particles for applied forces as low as 0.015N. This is sufficient sensitivity to allow the detection of contacts due to magnetic attraction of the particles.

The inter-particle interaction forces have been measured for pulling and pushing different combinations of disks (small/large/magnetic/non-magnetic) away/toward each other and measuring the resulting force f and edge-toedge distance between disks δ in a micro-strain analyser (TA Instruments RSA III). Fig. 2 shows that during compression, magnetic and small particles are sensibly stiffer. In every case the particles exhibit Hertzian-like behavior and the magnetic attractive forces follow f ∼ -e -0.35•δ . Also, the highest attractive magnetic forces (0.05N) are greater than forces due to basal friction (∼ 0.005N considering friction coefficient and particle weight), and both are much lower than repulsive forces measured in strong force chains (∼ 2N). This implies that neither the magnetic forces nor stiffening due to the magnets plays a direct role in the modification of the macroscopic behavior of the granular material once it is jammed.

However, the magnetic particles clearly influence the evolution of the system and the medium response. Fig. 3-(a) shows the evolution of the global pressure computed as the sum over G 2 (with separate calibrations by particle type) over all the disks when the granular system is compressed for different magnetic particle ratios R m , from 0% (no magnetic particles) to 100% (only magnetic particles). Each curve represents an average over three compression runs. Above the fluid-like unjammed states, two different types of response arise, one for low R m and another for high R m . The data for R m = 0% is characteristic of all the lower R m response curves, and R m = 100% is characteristic of the higher R m responses. In the first case, P stays near zero for moderate strains, and particles rearrange in a liquid-like way to relax forces in the system; P then increases rather rapidly above a critical-like ε. In the second case, pressure increases more steadily with initial ε, and a sharp increase occurs at a lower ε.

The distinction between low and high R m is particularly significant for the evolution of Z vs. ε, or packing fraction φ, for different ratios of magnetic particles. The responses for all R m < 50% are nearly the same, and separately the responses for all R m > 50% are all similar, with For the different combination of particles (small/large/magnetic/non-magnetic) the interaction force is plotted as a function of distance δ or overlap (δ < 0) between particle boundaries. Force scale for positive δ (right panel) is not the same as for negative (left panel). Non-magnetic particles are slightly softer than magnetic once. For the latter, a maximum attractive force of 0.05N is measured. Inset: for an equal compression force, G 2 (averaged over all pixels inside the photo-elastic materials of the particle) is plotted for magnetic G 2 m and non-magnetic G 2 nm particles, for small and large particles. A linear variation is found:

G 2 m = k • G 2 nm with k > 1.
a rapid crossover for R m 50%. Specifically, as in Fig. 3-(b), for R m < 50% (and as previously observed in [START_REF] Majmudar | Jamming transition in granular systems[END_REF] for purely non-magnetic systems), Z is initially very low (Z < 1) and increases over a modest range of compressions to the jamming point Z 3 which corresponds to a sharp increase in P . In contrast when R m > 50%, the coordination number is always higher than in the non-magnetic case due to contacts caused by magnetic attraction, which leads to Z 2. Then, when the system is compressed, Z increases roughly linearly until the rate of increase accelerates near the jamming point. We believe that this change in response with R m is due to a percolation transition of the magnetic interactions, which we will describe in more detail elsewhere. The jamming packing fraction, φ J , shifts from ∼ 0.728 when the system is purely magnetic to ∼ 0.742 when the system is completely non-magnetic. Note that all the jamming points are significantly lower than 0.84, the isotropic jamming point in 2D, because uniaxial compression is anisotropic, and friction between particles allows for shear jamming. (Also note that uniaxial compression is a mixed shear and isotropic compression mode.) The transition between these two response types as R m varies is sharp, which means that near the transition, an important change of the material response occurs with only a very small change in the fraction of magnetic particles.

Although the macroscopic response when magnetic particles are introduced changes significantly, as in Fig. 3-(a), the magnetic forces are very small (< 0.05N) compared to typical strong repulsive forces (∼ 2N) in the jammed state. Hence, the difference in P and Z induced by increasing R m cannot be directly related to the weak magnetic forces. To Fig. 3: (color online) Evolution of the pressure, measured as corrected G 2 (a) and of the coordination number, Z (b) as a function of the applied strain ε and of the packing fraction φ for different ratios of magnetic particles, from 0% (only nonmagnetic) to 100% (only magnetic). The higher the percentage of magnetic particles, the more resistant the system and the faster pressure increases. The number of contact per grains is naturally higher for magnetic particles and the system jams faster. In (b) a dashed line for Z = 3 marks the jamming transition. (a)-inset: zoom of the pressure evolution after the fluid-like rearrangement behaviour. Mechanical behaviour sensibly varies as a function of Rm.

understand the changes in material response with R m we focus on the network structures formed by the magnetic disks. As shown in the inset of Fig. 4-(b), neighbour magnetic particles tend to align in the same magnetic direction to attract each other, and hence to form magnetic chains. A pair of particles in a magnetic chain are identified as two nearby particles when the distance between the ends of their magnetic bars is less than 3mm. These chains can consist of up to 12 particles, and can exhibit ramifications and loops (see inset of Fig. 4-(b)). We define the length, , of a magnetic chain as the number of member particles, and we consider only chains that have ≥ 3. Fig. 4-(a) shows the evolution of the probability density function (PDF), P ( ), for different compression steps for an R m = 100% packing. The statistical distribution of is constant during the compression even if chains break and merge during the process (see video in supplementary material). P ( ) follows an exponential law with coefficient 0.72 ± 0.01:

P ( ) ∼ e -0.72• (1)
This statistical distribution remains the same within the errorbars for all R m > 0 for the present experiments (not shown here). This implies that whatever their density and the constraints inside the system, magnetic particles selforganize to maintain a magnetic chain network of the same statistical nature. Also, the average number of magnetic chains is roughly constant during the compression process but as shown in the inset of Fig. 4-(a) it increases linearly with R m .

The external load on a granular medium is not supported homogeneously by all the grains of the system but along forces chains forming a sparse network [START_REF] Majmudar | Jamming transition in granular systems[END_REF][START_REF] Radjai | Bimodal character of stress transmission in granular packings[END_REF][START_REF] Kondic | Topology of force networks in compressed granular media[END_REF]. The force network presented as a black shadow in the inset of Fig. 4-(b) is formed by particles having a G 2 value higher than a certain threshold (8a.u., 60% of the maximum G 2 value in Fig. 4-(b)). The threshold is far above the pressure that could result from magnetic interactions.

There are several seemingly contradictory facts that need explanation: 1) the magnetic forces are too small to directly affect the system response, except below and near jamming; 2) P and Z as functions of strain depend significantly on R m ; 3) the magnetic particles self-organize into statistically stationary networks. A reasonable way for the magnetic particles to strongly affect the pressure would be if they were associated with the strong force networks. To test this hypothesis, we investigated the effect of the magnetic networks on the force network organization. Fig. 4 (b) shows that when force chains begin to form, they grow preferentially from particles that are part of magnetic chains. We define R 1 to be the ratio of the number of particles in magnetic chains that are also in force chains to the number of particles in force chains. We compare this ratio to R 2 , defined as the ratio of particles in magnetic chains to all particles in the total system:

Here, N means particle number of the specified type. If force chains form everywhere in the system without regard to magnetic chains, then those two ratios should be equal. But as shown in Fig. 4-(b), initially, R 1 is much higher than the ratio of magnetic chain particles in the total system: R 1 R 2 . The difference between both ratios vanishes when the pressure increases, and the network becomes denser after the jamming transition. This means that the first force chains to form, which constitute the initial backbone of the system and that often will be the most loaded when the pressure increases, form mainly from particles belonging to a magnetic chain, whatever R m is.

Hence, the force chains that constitute the mechanical structure of a granular system under loading are im-posed by a magnetic chain structure that self-organized and was statistically stationary during the early evolution of the packing. The nearly binary response of the granular medium (e.g. either the R m = 100% or 0% responses) suggests that for low R m , the density of magnetic chains, even if force chains form along them, is too low to be able to setup a strong structure below jamming, and hence dominate the macroscopic response of the material. The sharp transition as R m is varied smoothly (around R m = 50%) to a very different response strongly suggests a percolation of the magnetic chains. This property could be used to tune the rigidity of a granular material by changing the magnetic nature of a small number of grains, and could be carried out by various processes (e.g. temperature change, external magnetic field change) that would change the magnetic state of a small number of particles.

To conclude, we observe that even though the magnetic inter-particle force for our particles is much lower than typical jammed-state repulsive forces, the presence of enough magnetic particles has an important impact on the response of the granular material to compaction, provided the fraction of magnetic particles is large enough. More precisely, a sharp transition in the pressure/coordinationstrain/packing fraction curves is observed around R m 50%. Also, self-organized magnetic chains occur in the packing at a very early stage. Even though the magnetic chains evolve during compression (breaking, merging) the statistics of their length remain the same throughout the process. This last point helps explain the first one, since we observe that force chains that support the applied load are formed preferentially by particles in the magnetic chains, particularly for smaller strains.When the magnetic chain fraction is too low, the magnetic chains cannot impose a mechanical network that is strong enough to affect the macroscopic behavior of the granular material, and the response resembles the R m = 0 case. We believe this property can be used in novel applications. An additional interesting direction involves the use of magnetic particles for a rapidly driven system. For each point, the average is done over all the compression steps for 3 runs with the same Rm. The standard deviation is too small to be visible on the graph. (b): Evolution of the ratios R1 (plain line, ratio of particles in a magnetic chain among particles in a force chain) and R2 (dashed line, ratio of particles in a magnetic chain in the whole system) during the compression for different mixes of magnetic particles. Force chains mainly form on top of magnetic chains (see text for details). (b)-inset: part of the system for a 100% magnetic particle experiment with ε = 16%, colored lines show magnetic chains (color scale for length) and dark lines shadow show force chains.

Fig. 1 :

 1 Fig. 1: (color online) (a): 3D schematic of the uniaxial compression experiment. A moving wall (blue arrow) compresses bidisperse magnetic photo-elastic particles by small steps. Each particle is marked on top by a bar, drawn in UV-fluorescent ink that gives the orientation of the embedded magnet. At each step the system is imaged in white light, (i) between crossed polarizers, (ii) without crossed polarizers, and (iii) in UV light. (b): top view with crossed polarisers showing magnetic and non-magnetic particles that have been compressed uniaxially. The presence of an embedded magnet modifies the inner mechanical and photoelastic response of a particle.

Fig. 2 :

 2 Fig. 2: (color online) Details of interactions between particles.For the different combination of particles (small/large/magnetic/non-magnetic) the interaction force is plotted as a function of distance δ or overlap (δ < 0) between particle boundaries. Force scale for positive δ (right panel) is not the same as for negative (left panel). Non-magnetic particles are slightly softer than magnetic once. For the latter, a maximum attractive force of 0.05N is measured. Inset: for an equal compression force, G 2 (averaged over all pixels inside the photo-elastic materials of the particle) is plotted for magnetic G 2 m and non-magnetic G 2 nm particles, for small and large particles. A linear variation is found:G 2 m = k • G 2 nm with k > 1.

R 1 =

 1 N (particles in force & magnetic chain) N (particles in force chains) R 2 = N (particles in magnetic chain) total number of particles

Fig. 4 :

 4 Fig. 4: (color online) (a): probability density function (PDF) of the length, , of magnetic chains, averaged over 3 100% magnetic particle experiments measured at different steps of the compression. Even if chains break and merge, the PDF still follows an exponential law (grey dashed line) with coefficient 0.72 ± 0.01. (a)-Inset: average number of magnetic chains for experiments with different percentage of magnetic particles.For each point, the average is done over all the compression steps for 3 runs with the same Rm. The standard deviation is too small to be visible on the graph. (b): Evolution of the ratios R1 (plain line, ratio of particles in a magnetic chain among particles in a force chain) and R2 (dashed line, ratio of particles in a magnetic chain in the whole system) during the compression for different mixes of magnetic particles. Force chains mainly form on top of magnetic chains (see text for details). (b)-inset: part of the system for a 100% magnetic particle experiment with ε = 16%, colored lines show magnetic chains (color scale for length) and dark lines shadow show force chains.

Acknowledgements: This work was supported by The NSF-Research Triangle MRSEC, NSF grant DMR-1206351, The William M. Keck Foundation, and a London post-doctoral fellowship for JB.