Pierre Bégout

Thierry Duval

Sébastien Kubicki

Bertrand Charbonnier

Emmanuel Bricard

WAAT: a Workstation AR Authoring Tool for Industry 4.0

Keywords: 3D authoring, AR Authoring tool, Virtual Reality, Augmented Reality, Mixed Reality, AR Training, AR guidance, Virtual environment, Industry 4.0, Human-computer interaction, 3D interaction

Figure 1. Operator on an assembly

Introduction

In a context of constant industrial evolution and a need of more industrial agility to answer the increasingly unpredictable customer requests, having well trained assembly line operators is extremely important. This operator training problematic is well known by manufacturers such as elm.leblanc, a boiler manufacturer for which this training problem is very important during each winter, when boiler orders rise a lot, as many temporary workers must be hired for a few months. The training of these new operators is an important process that involves a lot of time and human resources. For instance, one experimented operator is dedicated to the training of one new operator. During this time, the production lines output is not optimal because of the learning curve of the new operators as well as the attention they require from permanent staff decreasing the overall capacity.

One solution to this training issue is the use of AR in the factory (cf. Figure 1), to train new operators directly on the assembly line with less supervision from experimented operator. This tool could also be used by an experimented operator to train on a new position of the assembly line or a different assembly line.

Indeed, in the industrial context, AR can be used in many different ways . It can be used to make expert remote guidance, create new products using collaborative engineering methods, or realize digital inspection of new prototypes [START_REF] Fraga-Lamas | A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard[END_REF]. AR is also used to test the ergonomics of the workstations and the reachability of some equipment [START_REF] Berg | Industry use of virtual reality in product design and manufacturing: a survey[END_REF]. Error and default detection using AR are possible and very useful in Airbus factories [START_REF] Barbosa | Augmented Reality in Aerospace Manufacturing: A Review[END_REF]. These different uses have a lot of potential, but the most interesting feature of AR in our case and the most used is the assembly assistance, for training purpose [START_REF] Paelke | Augmented reality in the smart factory: Supporting workers in an industry 4.0. environment[END_REF][START_REF] Barbosa | Augmented Reality in Aerospace Manufacturing: A Review[END_REF]. Actually, assembly tasks training using AR has been researched for a long time, with the comparison of paper instructions sheets with VR and AR instructions, showing that AR instructions were the most efficient [START_REF] Boud | Virtual reality and augmented reality as a training tool for assembly tasks[END_REF]. More recently, new ways of using AR for assembly training are researched, by combining AR and other tools. It can be an intelligent tutoring system for error detection [START_REF] Westerfield | Intelligent augmented reality training for motherboard assembly[END_REF] or data from the factory [START_REF] Makris | Augmented reality system for operator support in human-robot collaborative assembly[END_REF][START_REF] Danielsson | Operators perspective on augmented reality as a support tool in engine assembly[END_REF].

These AR environments are created with AR authoring tools, and most of them are dedicated to 3D CAD experts or 3D AR experts. So, even if the endusers can participate to the modelling of the AR environment during its usercentred design with such authoring tools, they cannot use them to evolve the AR representation according to the real environment changes. This is why we make the assumption that to be efficient and to provide feedback on the operator in training, such authoring tools should be dedicated to line managers. It would permit the users with the real knowledge of the assembly line to model its different elements. These elements could be, for instance, 3D models or anchors 1 .

In our industrial context, which can occur quite often for factories, the 3D models of the parts of the assembly line are not available or could change dynamically according to the needs of the production. This is why we cannot use a classical 3D CAD tool to manage the placement of the augmentation relatively to the parts of the assembly line and we need an authoring tool to facilitate the configuration of the AR training tool.

Our contribution is to improve AR authoring by proposing a tool allowing line managers, who are not familiar neither with AR technology nor 3D CAD tools, to quickly create a 3D representation of the assembly line, by placing and manipulating 3D elements easily without the need of long training time to be efficient with AR technology.

The remaining of this paper is organized as follows: in section 2 we explain the industrial constraints for the use of this authoring tool. Section 3 contains the related work, justifying the choices made in section 4 which describes WAAT, our authoring tool. Section 5 describes the comparison between WAAT and Microsoft Guides on an authoring task. Finally, section 6 concludes this paper and discusses about future developments.

Industrial context

In an industrial context, we have a lot of constraints related to the nature of the workstations and the way boiler assembling is done. We have many problematic to take into account in order to make an authoring tool that allows line managers to quickly and easily create a 3D representation of the assembly line.

Time management

When working on an assembly line, every gesture is thought to accomplish the technical action in the smallest amount of time possible. Every workstation is thought to accomplish a certain number of technical tasks, the placement of each equipment is well thought. Therefore the authoring tool should help gaining time in the factory by quickly configuring the training tool. The final user of this authoring tool should be able to perform the authoring task quickly, otherwise the tool won't be used.

Managing evolution

The elm.leblanc company assembly lines are manageable and can be changed during the day to correspond with the current boiler model. Each workstation can be rearranged (some parts are on wheels to be easily movable). Some tasks can pass from a workstation to another when changing the boiler model currently assembled on the line.

This leads to the need of regular updates of the 3D model of the assembly line to correspond to the state of the real one and/or to new boilers models. The most suitable user to perform these updates is the line manager, who knows the assembly line state and each workstation very well.

Main features

Knowing these requirements, we have several uses for our authoring tool. First, the authoring tool must allow fast and easy placement of 3D models of the workstation (cf. Figure 2a). The AR guidance could consist of an information panel containing text, videos, pictures, or it could also be highlights of 3D models. This AR guidance must also be placed easily (cf. Figure 2b). The AR part of the tool must allow the user to compare the 3D models and the real objects (cf. Figure 2c), and to move or resize the 3D objects to match the real scene and the objects, and also to make sure the operator guidance panel is visible (cf. Figure 2d). The next section presents the state of the art about 3D authoring tools that we could use to meet all our requirements.

Tool evaluation criteria

Knowing our constraints and our context, we defined criteria to evaluate the different tools we studied. First, the tool must allow the placement of 3D models, and also the placement of AR guidance (3D animation, text, video) to check its placement and its visibility. As seen previously, the workstations can change quite often so we need to be able to do live edit while in AR. The industrial context prevents us to put tracking markers (detailed in section 3.2) everywhere to track the workstation elements, so we need to use only one marker for the whole workstation. The training required to use the authoring tool must be fast and simple. The tool's interface must be simple, a GUI authoring is preferable. Our final users are the line managers and experimented operators, so the authoring tool needs to be easy to use, and the AR part must be usable hands free to enable users to perform the technical tasks (head mounted display are probably the best solution to do that).

3 Related work

3D Authoring tools

3D models of components and workstations are widely used in the industry. Most of the time they are used to create the full product, change the layout of the factory, test the ergonomics of the workstations [START_REF] Berg | Industry use of virtual reality in product design and manufacturing: a survey[END_REF]. To design these 3D models, many tools are available since the 70's. Some of the most used tools to model industrial component are SolidWorks2 , Inventor3 , CATIA 4 , Pro/ENGINEER 5 and AutoCAD 6 . These tools offer precise modeling features. The problem with these CAD (Computer Aided Design) tools is that their learning curve is very steep and require dozens of hours to properly use them [START_REF] Kostic | Comparative Study of CAD Software, Web3D Technologies and Existing Solutions to Support Distance-Learning Students of Engineering Profile[END_REF]. It is also difficult to export files to AR engines like Unreal 7 or Unity 8 without the use of an other software. Usually, the exported object are opened in a DCC (Digital Content Creation) tool (Blender 9 for example) to be converted in a suitable format for AR engines.

Other digital content creation tools, such as 3dsMax 10 , Maya 11 , Blender, can be used to model our workstation, but just as the previous CAD softwares seen previously, their interface is very complicated, and the learning time is very high. They offer lots of options to create very precise 3D models. SketchUp 12 , another DCC tool, is easier to learn since it is based on line drawing but the 3D models obtained this way are less accurate than with the other tools [START_REF] Parisi | Programming 3D Applications with HTML5 and WebGL: 3D Animation and Visualization for Web Pages[END_REF].

These 3D modeling tools are great when accurate 3D models are required, but they target professional designers who can spend time to learn a new tool and realize accurate 3D models of industrial components and workstations. In our case, we don't need such accurate 3D models. Indeed we only need to represent the components of the workstation and their position to the AR guidance tool.

AR Authoring

To be able to use AR authoring tools in our assembly lines, we have to know where to locate the 3D models in the space, to place them at the right position, and which graphical assets to use. To achieve that goal, several tracking techniques are used, and several recommendations exists. These are detailed in the next subsection.

Graphic elements To use AR in an industrial context, we have to adapt our graphical assets to be easy to understand, create and render, and avoid occlusion [START_REF] Gattullo | Informing the Use of Visual Assets in Industrial Augmented Reality[END_REF]. To achieve that goal, it is better to use symbols and simple models [START_REF] Scurati | Converting maintenance actions into standard symbols for Augmented Reality applications in Industry 4.0[END_REF] to guide the user and represent instructions. Text can be used but has to be simplified [START_REF] Gattullo | Towards augmented reality manuals for industry 4.0: A methodology[END_REF] and write in a billboard to have contrast between the text and the background.

Tracking methods Tracking is a fundamental issue when using AR. Accurate and stable tracking techniques are important for using AR in an industrial environment. Most of the time, markers are used to locate the real objects and help positioning 3D objects. The capture of the marker is made by image recognition. After recognition, the 3D object linked with the marker appears [START_REF] Zhu | An authorable context-aware augmented reality system to assist the maintenance technicians[END_REF][START_REF] Mota | Augmented reality mobile app development for all[END_REF][START_REF] Westerfield | Intelligent augmented reality training for motherboard assembly[END_REF][START_REF] Yang | Mobile Augmented Reality Authoring Tool[END_REF][START_REF] Jung | Smartphone as an augmented reality authoring tool via multi-touch based 3D interaction method[END_REF][START_REF] Sambrooks | Designing HARATIO: A novice AR authoring tool[END_REF][START_REF] Gimeno | A new AR authoring tool using depth maps for industrial procedures[END_REF]. Markers can be pictures, QR code, colored paper, any kind of 2D unique-looking object. This technique is easy to implement and useful in an environment that can change depending on the assembly task such as our assembly line. An other tracking technique uses object recognition by comparing a 3D model of the object to the real object captured by the camera [START_REF] Erkoyuncu | Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality[END_REF]. This technique requires precise 3D models of our objects and is useful in a static environment, which is not the case with our assembly line.

AR Authoring tools

With the emergence of AR, many tools have appeared to build AR applications such as ARToolkit [START_REF] Kato | Marker Tracking and HMD Calibration for a Videobased Augmented Reality Conferencing System[END_REF] that provides a marker-based registration using computer vision. ARToolKit and its variations all require the user to have C/C++ skills. More recently, Vuforia13 a software development kit, provides computer vision technology to recognize and track planar images and 3D objects in real time. To use Vuforia, C# skills are necessary. This is the main problem with creating AR content. Most of the time low level AR libraries are used, requiring programming skills from the user.

The need for High level AR authoring tools for non-programmers is important. The earlier frameworks were often a plug-in of an other software, like DART [START_REF] Macintyre | DART[END_REF], a plug-in for Adobe Director 14 (not supported anymore). DART was supporting the 3D editing part of the software with visual programming, such as dragging a 3D object in a scene and adding behavioral scripts to the object. Another GUI-based visual AR authoring tool, AMIRE [START_REF] Grimm | AMIRE-authoring mixed reality[END_REF] allows the user to describe interactions between objects with visual representations, which can become complex. An other approach with a more rigid framework was APRIL [START_REF] Ledermann | APRIL: a high-level framework for creating augmented reality presentations[END_REF] based on XML descriptions. APRIL is a plug-in of their own platform containing XML parser. Using XML can be quite difficult for a user with no programming skills at all.

More recently, 3D GUI-based authoring tools have been studied. In ACARS [START_REF] Zhu | An authorable context-aware augmented reality system to assist the maintenance technicians[END_REF] the authoring part is realized on desktop using a 3D model of the equipment. The maintenance technician can do on-site authoring using a tangible object to change the information's position. The authoring in SUGAR [START_REF] Gimeno | A new AR authoring tool using depth maps for industrial procedures[END_REF] is also realized on desktop. The maintenance expert can place AR information around the equipment using depth map of the factory. The authoring is also made by an expert in ARAUM [START_REF] Erkoyuncu | Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality[END_REF], he gives information to the authoring tool about the procedures to perform, which will place automatically the AR content around the equipment using spatial, time and user context. Some authoring tools include the final user in the authoring process. In ARTA [START_REF] Gattullo | Towards augmented reality manuals for industry 4.0: A methodology[END_REF], an experienced worker is recorded performing a task using eye-tracking and a camera, but he also has to learn how to use Unity3D to split the recording in different steps. The final user can also intervene to verify the quality of the modeling in AR [START_REF] Geng | A systematic design method of adaptive augmented reality work instruction for complex industrial operations[END_REF], to add details in the instructions or edit the models, but the original modeling is done by an engineer. The target devices for these authoring tools are HMD (Head Mounted Display) devices, but a mobile platform (tablet or phone) can also be a good alternative. Using the VEDILS framework [START_REF] Mota | Augmented reality mobile app development for all[END_REF] allows any user to create AR content for mobile using a desktop application. It uses visual programming and behavioral blocks to add interactions to the 3D objects, but it requires knowledge in algorithmic to be used properly.

Using mobiles to visualize AR content leads to research on mobile AR authoring tools, reducing the material needed for the authoring task, allowing in-situ authoring, which can be really interesting in some situations where we don't have access to a computer. HARATIO [START_REF] Sambrooks | Designing HARATIO: A novice AR authoring tool[END_REF] allows the user to add 3D objects from a library in a scene, including behavior scripts. All actions are made with a radial menu. We can see limitations with mobile authoring tools like in [START_REF] Jung | Smartphone as an augmented reality authoring tool via multi-touch based 3D interaction method[END_REF] or [START_REF] Yang | Mobile Augmented Reality Authoring Tool[END_REF]. In the first one, the user interacts with the 3D object on the touch screen, whereas in the second one the interactions are done directly with the markers which can be combined. With these tools we can only associate one marker with one 3D object.

It seems that authoring tasks made using desktop applications are often realized by an expert on the industrial procedure or a designer while we need our tool to be usable by line managers. It also seems that most authoring tools don't allow users to directly edit the content while in AR to better match the real objects. Mobile authoring can be interesting in our case but the interaction limitations and the marker/3D object limit is a problem because we will have multiple 3D objects per workstation.

WAAT (Workstation AR Authoring Tool)

To meet all the requirements presented in section 2 and summarized in Table 1, we propose WAAT: an authoring tool designed to allow users to create 3D scenes of workstations by placing 3D objects representing the workstation. The originality of WAAT is that it makes it possible for the user to compare the 3D model with the real workstation. The authoring task in WAAT is easy to perform by production staff, only need a few 3D models, and does not require any engineer or designer.

System overview

WAAT is composed of two modules, a desktop 3D authoring module and an AR authoring module, as illustrated Figure 3. The desktop authoring module (see Figure 4) is used to create the 3D scene whereas the AR authoring module is used to modify and validate the placement and/or size of the 3D models. The two modules are connected by the server, with a JSON file for each scene allowing the user to modify the scene either on desktop or AR. WAAT is developed with Unity using Vuforia for the marker tracking in the AR module. The AR hardware currently used is the Microsoft Hololens 2 15 .

To deploy WAAT on another AR device such as the Magic Leap 16 we would just have to add to our system the Magic Leap extension of the SDK (Software Development Kit) we currently use.

3D authoring of the assembly line

The authoring task on the desktop application is done with simple interactions using a mouse and a keyboard. To select which object the user wants to add, he just has to choose the object in the bottom interface (cf. Figure 4) and define its position in the scene (we get the 3D objects from any 3D modelling or CAD tool). He can then move the objects to place them more precisely and change some of their properties (e.g., name, scale, . . . ). We chose drag and drop as the interaction method but it is not the only one available (i.e., keyboard or a graphic overlay for the axis and rotation), and testing will help to choose the best suited one. Then the user can save the scene he created or load a scene previously created. We tested WAAT on a workstation from the factory (cf. Figure 5). We used 3D models to represent the workstation and the placement of the boiler. We represented the boxes where the parts to assemble are kept and the tools to use with "spatial anchors", a 3D object allowing the AR training tool to know the position of the different part to pick (see Figure 8). We also placed the panel used to display AR assembly guidance.

We conceived WAAT to also be usable in an immersive mode, but we focused on the main features of the tool and our users want to use it in non-immersive mode for the moment.

Using AR to check the 3D modeling

To make sure the 3D model of the assembly line is on par with the real assembly line, the user will use the AR part of the tool and compare the position, rotation, size of the 3D objects with their corresponding real object.

If it is not on par with the real objects (as illustrated in Figure 6), the user can move the 3D objects by grabbing them and placing them at the correct position. Each 3D object refers to a real object and must be placed, rotated and scaled as its real counterpart to allow the AR guidance system to be well configured (as illustrated in Figure 9). The AR guidance system will use the position of the 3D objects to tell the operator where to grab or place technical parts.

After finishing all his modifications, the user can save his work, which updates the 3D model of the assembly line in the WAAT system. He can now load another scene to check the 3D modeling.

WAAT and AR guidance

WAAT is also used to place the marker position for further AR guidance of the operators. Each scene is associated to a marker allowing the system to load the correct objects related to a workstation. The 3D objects in WAAT are then used for the training or as anchors for the AR guidance system.

Placing AR markers in the model of the assembly line After the 3D modeling of the assembly line, the final step is to place the markers positions in the virtual scenes. This virtual marker is used to show the user in AR the position where he is supposed to place the real marker to use the system. It is placed in the same way than any other 3D object.

Placing AR markers in the real assembly line To be able to test the AR guidance (and to use it during the operator formation) the user has to place real markers at the same place as the "virtual" markers in the real assembly line. For every workstation, a specific marker is placed in a specific position. If the real marker is not at the same position as the virtual one, as illustrated Figure 7, all the 3D objects will be misplaced compared to the real objects. If the position of the virtual marker is too difficult to use in the assembly line, the user can move it to match the correct position and obtain the situation illustrated in Figure 8. AR Checking of the AR guidance The AR scene is also used to verify the positioning of the "operator guidance": the panels where the training instructions will be shown. The user can move these panels to make sure the instructions are visible and that there is no occlusion or other problems. The user will also check the positioning of all markers and objects and move them if needed since they will also be used by the AR guidance system. Here again, after finishing all his modifications, the user can save his work, this will update the assembly line model globally, and more precisely, the markers positions and AR guidance panels positions.

Comparison with Microsoft Dynamics 36Guides

We compared the use of our tool WAAT with the solution from Microsoft 17 . This solution is similar to ours, it is separated in two modules: the desktop module and the AR module. The desktop module is the place where the user creates the instructions and add the 3D models tied with these instructions. The difference with WAAT is that there is no placement of the 3D models with Microsoft Dynamics 365 Guides. The AR module is used for the placement of the 3D models in the real environment and to create the AR assistance (i.e., arrows showing what to do at each instruction, where to place the element). The tracking is made with a QR code.

We compared the authoring time to place the 3D models of the workstation at the right position between WAAT and Microsoft Guides. We evaluated the total authoring time (desktop + AR), with the same number of 3D models to place on the same workstation. The desktop authoring is faster with Guides, since there is no placement of the 3D objects in a virtual world. The 3D models are linked to a task and their characteristics can't be changed (position, name, scale).

For the AR authoring, our tool is much faster because the 3D models are already placed almost at the right position. The movement to achieve the right positioning is minimal. In Guides, however, the user has to get the 3D models in the instructions panel, requiring a lot of movement, and taking a lot of time.

As summarized in Table 2, we can see that WAAT is slightly faster than Guides on the environment authoring task. To improve our tool and reduce the authoring time, we have to improve the desktop authoring, and more precisely the placement of the 3D models. One of the considered solution is workstation templates placing the anchors automatically with only the references in entry. However in WAAT, knowing the names and positions of the 3D models and anchors allows the AR assistance creation tool to generate automatically the placement instructions by collision detection.

More testing is required, in which we will evaluate the authoring time, the number of error, the accuracy and fidelity. We will also evaluate the usability with the System Usability Scale, and a workload test with the Nasa TLX.

In this paper, we have presented WAAT, a 3D authoring tool allowing untrained users to create 3D models of the assembly line in a boiler factory. WAAT allows fast creation and AR comparison of the virtual and the real model. In case of a mismatch between both virtual and real models, the user can easily and quickly move the 3D virtual components to match the components of the real workstation. He can test the AR guidance used to train assembly line operators to make sure that everything is fully visible and doesn't interfere with the realization of the technical tasks.

Improving desktop authoring will be an important part of our future work to reduce the time spent on the desktop part. First, to adapt the interactions used to move/rotate/resize the 3D objects to our users, adding accessories such as a 3D controller will be explored. We will also test new AR systems to test their usability and native interactions to see if we can use them, or if we can adapt them to our users. Second, we have to make other tests with the line managers from the factory to enhance the usability and the intuitiveness of WAAT. Last, using 3D models of the workstations will also allow us to test the ergonomics of new workstations during the creation process with the use of a posture capture outfit.

Finally, the immersive mode of WAAT will allow us to visualize the augmented environment and test the placement of the operator guidance, even if it's not the focus right now. We will be able to simulate the AR test of the scenes without the need to be in the factory all the time.

Figure 2 .

 2 Figure 2. Needed features of the authoring tool. (a) 3D models placement with a motor. (b) AR guidance placement on the motor. (c) AR checking of 3D models and AR guidance on a real boiler. (d) AR validation of the placement on a real boiler

Figure 3 .

 3 Figure 3. WAAT system overview

Figure 4 .

 4 Figure 4. WAAT desktop module

Figure 5 .

 5 Figure 5. The workstation we want to model

Figure 6 .

 6 Figure 6. Bad placement and size of the 3D models

Figure 7 .

 7 Figure 7. Bad marker placement on the real object

Figure 8 .

 8 Figure 8. Good placement of the AR Guidance

Figure 9 .

 9 Figure 9. Good placement of the 3D models

Table 1 .

 1 Table summarizing the main features of 3D authoring tools suited for AR authoring.

Table 2 .

 2 Table summarizing the time (minutes) for the authoring of a workstation using WAAT or Guides.

	Tool	Desktop AR Total
	WAAT 6'30" 1'50" 8'20"
	Guides 3'19" 5'10" 8'29"

detailed in section 3.2.

https://www.solidworks.com/

https://www.autodesk.com/products/inventor/overview

https://developer.vuforia.com/

https://www.adobe.com/products/director.html

SolidWorksO X X X X O X X Inventor O X X X X O X X CATIA O X X X X O X X Pro/ENGINEER O X X X X O X X AutoCAD O X X X X O X X Blender O X X X X O X X 3dsMax O X X X X O X X Maya O X X X X O X X SketchUp O X X X X O X X ARToolKit O O X O X X O X Vuforia O O X O X X O X DART O O X O X O O X AMIRE O O X O X O O X APRIL O O X O X X O X ACARS O O O O X O O X SUGAR O O X O X O X X ARAUM O O X X X O O X VEDILS O O X O X O O X ARTA O O X X X O O O Geng et al. O O O O X O O X HARATIO O O O X O O X O Jung et al. O O O X O O X O Yang et al. O O

https://www.microsoft.com/en-us/hololens

Acknowledgements

This work is possible thanks to the workers of the Drancy elm.leblanc factory and of Saint-Thégonnec elm.leblanc factory for exchanging with us on the problematic of their work and helping us to better understand their needs in 3D modeling and AR guidance.


Abir Massaad

Aren Ayman Assi

Joe Bizdikian

Ziad Bakouny

Nour Khalil

Ismat Ghanem

Hélène Pillet

Xavier Bonnet

Wafa Skalli

A J Bizdikian

F Yared

O 095-How do postural parameters vary during gait in children with cerebral palsy? A 3D subject-specific skeletal segment registration technique

Keywords:

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Postural parameters, calculated in static position on frontal and sagittal radiographs, are altered in ambulant children with cerebral palsy (CP) and are known to be related to gait abnormalities [START_REF] Massaad | Gait and Posture[END_REF][START_REF] Aktas | [END_REF]. Most of these parameters are positional and could vary during gait. Their computation could be performed using gait simulation models that are usually generic or semi-personalized [START_REF] Aktas | [END_REF][START_REF] Bosman | Gait and Posture[END_REF]. There are currently no studies investigating how subject-specific postural parameters are modified during gait in children with CP.

Research Question

How do postural parameters vary during gait in children with CP?

Methods

14 typically developing children (TD-group, age:11.7 ± 4 year) and 14 children with cerebral palsy (CP-group, age:11 ± 4 year, Hemiplegia:N = 4, Diplegia:N = 10) underwent gait analysis using Davis protocol [START_REF] Bosman | Gait and Posture[END_REF] with additional markers on thighs and shanks. Subjects then underwent low-dose biplanar X-rays in standing position with the reflective markers still in place. Subject-specific 3D reconstruction of the spine, pelvis and lower limbs were performed with calculation of 3D radiological postural parameters in the static position: pelvic tilt(PT), sacral slope(SS), acetabular abduction(Acetabular_-Abduction), anteversion(Acetabular_Anteversion) and tilt(Acet-abular_Tilt) [START_REF] Davis | [END_REF] as well as anterior (Ant_Acet_coverage) and posterior (Post_Acet_Coverage) acetabular coverage over the femoral head. 3D bones were registered on each frame of the gait cycle [5] (Fig. 1). A new technique developed for this study, utilizing finite element modeling, was used to reduce soft tissue artefacts. The same postural parameters were then computed during the gait cycle, using the 3D registered bones, at each time frame: means and ranges of motion (ROM) were calculated then compared between TD and CP.

Results

The TD-group had an Acetabular_Anteversion of 14 ± 5°(CPgroup: 12.5 ± 4°) in static position that increased to an average of 18 ± 4˚during gait only in the TD-group (p < 0.001). During walking, the average over the entire gait cycle of the Acetabular_Anteversion and Acetabular_Tilt was decreased in the CPgroup (Acetabular_Anteversion: CP: 13 ± 6˚vs TD:18 ± 4˚, p = 0.001; Acetabular_Tilt: CP: 20 ± 8˚vs TD: 25 ± 6˚; p = 0.011, Figs. 2 &3 ). Moreover, the average of SS during gait was increased in the CP-group when compared to TD (44 ± 6˚vs. 34 ± 8˚, p = 0.002). The ROM of the PT was significantly larger in the CP-group compared to TD (9 ± 3˚vs. 6 ± 2˚, p = 0.05) (Fig. 3).

E-mail address: ayman.assi@usj.edu.lb (A. Assi).

⁎ Corresponding author.

Discussion

This is the first study to compute 3D subject-specific postural parameters during gait in children with CP. The CP-group presented a higher ROM of PT during walking which could be related to altered spinal movement and stability during gait. Children with CP seem to present significant acetabular alterations in static and dynamic that might alter lower limb kinematics during gait. Future studies should investigate acetabular postural parameters in patients with different gait profiles to elucidate postural and compensatory behaviors in children with CP. The subject-specific skeletal registration technique could provide dynamic quantitative data for gait biomechanical modelling.

Fig. 1 .

 1 Fig. 1. Registration technique of 3D skeletal segments in gait analysis.

Fig. 2 .

 2 Fig. 2. 3D acetabular and pelvic parameters calculated in standing position and during the gait cycle in CP and TD children.

Fig. 3 .

 3 Fig. 3. Variation of 3D pelvic and acetabular parameters during gait in CP and TD children.


Alexandre Kabil
email: alexandre.kabil@imt-atlantique.fr

Thierry Duval
email: thierry.duval@imt-atlantique.fr

Nora Cuppens

Alert Characterization by Non-Expert Users in a Cybersecurity Virtual Environment: a Usability Study

Keywords: Virtual Reality, Cyber Security, Usability Study

Although cybersecurity is a domain where data analysis and training are considered of the highest importance, few virtual environments for cybersecurity are specifically developed, while they are used efficiently in other domains to tackle these issues. By taking into account cyber analysts' practices and tasks, we have proposed the 3D Cyber Common Operational Picture model (3D Cyber-COP), that aims at mediating analysts' activities into a Collaborative Virtual Environment (CVE), in which users can perform alert analysis scenarios.

In this article, we present a usability study we have performed with nonexpert users. We have proposed three virtual environments (a graphbased, an office-based, and the coupling of the two previous ones) in which users should perform a simplified alert analysis scenario based on the WannaCry ransomware. In these environments, users must switch between three views (alert, cyber and physical ones) which all contain different kinds of data sources. These data have to be used to perform the investigations and to determine if alerts are due to malicious activities or if they are caused by false positives. We have had 30 users, with no prior knowledge in cybersecurity. They have performed very well at the cybersecurity task and they have managed to interact and navigate easily. SUS usability scores were above 70 for the three environments and users have shown a preference towards the coupled environment, which was considered more practical and useful.

Introduction

More and more human activities are digitized and connected in cyberspace. The cyberspace could be defined as Gutzwiller does as a space that 'comprises communications between computing devices, the devices themselves, and the interconnections of machines and networks to the physical world as both sensors and actuators' [START_REF] Gutzwiller | Situation awareness in defensive cyberspace operations: An annotated bibliographic assessment through 2015[END_REF]. In this context cybersecurity grows in importance, as a small security breach in a network can have a huge impact on organizations or countries, as with the WannaCry Attack in 2017. Visual Analytics solutions are now provided to analysts to help them to understand cyber situations [START_REF] Varga | The application of visual analytics to cyber security[END_REF]. These solutions offer efficient 2D visualizations coupled with advanced data processing capabilities, but few 3D visualizations or Immersive Analytics solutions are developed for cybersecurity, even though they can leverage some issues in data analysis and are getting attention in other domains [START_REF] Hackathorn | Immersive analytics: Building virtual data worlds for collaborative decision support[END_REF].

In the same way, training environments proposed for cybersecurity are still only developed for specific cases with expert tools or serious games for sensitization whereas Virtual Environments for Training have been used for years in the industry for different scenarios or practices [START_REF] Sebok | Improving safety and human performance in maintenance and outage planning through virtual reality-based training systems[END_REF].

In our opinion, Virtual Environments (VE) for cybersecurity should merge data visualization with contextual scenarios to be useful for both experts and non-experts [START_REF] Kabil | Why should we use 3D Collaborative Virtual Environments for Cyber Security?[END_REF]. Figure 1 shows in a Venn Diagram our position towards the usability of Virtual Reality (VR) for cybersecurity. To develop such environments, the analysis of cyber activities, tasks and practices must be performed to model relevant use cases. To face this issue, we have developed a VE instantiation model, the 3D Cyber Common Operational Picture (3D CyberCOP) [START_REF] Kabil | 3D CyberCOP: a Collaborative Platform for Cybersecurity Data Analysis and Training[END_REF], based on activities and task analysis of Security Operations Centers (SOC) operators. We also have proposed an alert analysis scenario based on the WannaCry ransomware, as a use case of our model. To assess the effectiveness of a VE for both data visualization and training for cybersecurity, evaluations and exercises should be performed with various users.

In this article, we will present the usability study we have performed with non-expert users, based on an implementation of the 3D CyberCOP model. We will not emphasize new or innovative interaction techniques but rather more on the adaptation of cyber practices into a VE.

In section 2 we will present the 3D CyberCOP model and our use case based on the WannaCry ransomware. This model was based on a study of Security Operation Center (SOC) operators' activities, tasks and practices and aims at adapting these practices into a VE.

Then in section 3, we will present an instantiation of our model we have developed for a usability study with non-experts users. We will detail our usability study in section 4 and the results in section 5, and we will conclude by showing perspectives of our approach.

3D CyberCOP Model and analysis scenario

There is a growing interest in human factor field and cognitive science for cybersecurity, which aim at easing the burden of analysts, as they have to deal with a huge number of data sources to develop a mental model of the cyber situation, called the Cyber Situational Awareness (Cyber SA) (Figure 2).

Defining a cybersecurity operation activity model can facilitate the understanding of operators' activities within organizations and the collaboration between these organizations [START_REF] Takahashi | Toward global cybersecurity collaboration: Cybersecurity operation activity model[END_REF]. We have developed our 3D Cyber Common Operational Picture (3D CyberCOP) model with this in mind. In this section, we will detail our model and the analysis of WannaCry ransomware we have made in order to create alert analysis scenarios.

Cyber Activity Modeling

Cyber activity studies take often place in Security Operations Centers (SOCs), which are well-defined structures where analysts are monitoring networks roundthe-clock, and where they have to investigate alerts for client companies or internal security [START_REF] Ámornik | Prerequisites of virtual teamwork in security operations centers: Knowledge, skills, abilities and other characteristics[END_REF]. There exist anthropological studies in SOC in order to understand the roles and tasks of operators [START_REF] Sundaramurthy | Turning contradictions into innovations or: How we learned to stop whining and improve security operations[END_REF], and taxonomies were developed to characterize them [START_REF] Evesti | Cybersecurity situational awareness taxonomy[END_REF]. Cognitive Tasks Analysis (CTA) were also performed to assess operator's sense-making capabilities [START_REF] Damico | Cyber operator perspectives on security visualization[END_REF] and to help to develop effective visualization tools via the EevI framework, for example [START_REF] Sethi | Expert-interviews led analysis of eevi -a model for effective visualization in cyber-security[END_REF].

We had had the opportunity to visit our industrial partners' SOCs and to perform the following activity analysis described hereafter [START_REF] Kabil | From Cyber Security Activities to Collaborative Virtual Environments Practices through the 3D CyberCOP Platform[END_REF]: after a preliminary visit to understand the context and to get authorizations, we had planned to interview SOC operators and to get information on their practices to develop adapted visualizations.

By using a SOC operator's roles model such as the one proposed by McKenna, Staheli and Meyer in [START_REF] Mckenna | Unlocking user-centered design methods for building cyber security visualizations[END_REF] and features from a Computer Supported Cooperative Work (CSCW) model proposed by Casarin, Pacqueriaud, and Bechmann in [START_REF] Casarin | Umi3d: A unity3d toolbox to support cscw systems properties in generic 3d user interfaces[END_REF] which are relevant in SOCs, we have defined an activity model which purpose is to adapt cybersecurity practices into a Collaborative VE. In this paper, we will focus only on single-user cyber activities and we are not taking into account collaborative ones. For each feature, we will describe how it is done in SOCs, how we could tailor it for Virtual Environments (VE), and an example of a possible instantiation in an alert analysis scenario, as summarized in Table 1. Roles Cyber activities are a trade-off between decisions and data analysis [START_REF] Staheli | Collaborative data analysis and discovery for cyber security[END_REF]. For example, an analyst has to execute the request of a coordinator by investigating alerts and sending reports, and the coordinator must take into account this analysis to make decisions.

To implement roles, we can define specific visualizations and interactions. For example, an analyst role could have a detailed view of an asset from an egocentric point of view whereas the coordinator role could have a global view of all assets of the network from an exocentric point of view (as in the Frame of Reference model from Salzman, Dede, and Loftin [START_REF] Salzman | Vr's frames of reference: A visualization technique for mastering abstract multidimensional information[END_REF]). To respect the hierarchies between roles, we could implement hierarchical interactions: analysts are given orders and send reports whereas coordinators give orders and read reports.

Tasks Cyber analysts have different levels of expertise and they perform tasks with respect to it. To coordinate their actions, they use a ticketing system that tracks their activities and helps them following procedures. Analysts can perform a lot of actions but they can be regrouped into categories such as Asset Analysis, Log Cleaning or File Disassembling. The National Institute of Standards and Technology (NIST) gives global information about tasks in cybersecurity 3 .

For our VE, we have modeled and simulated categories of actions such as inspection (getting information from a computer or an alert), analysis (performing anomaly analysis on a computer), or reporting (sending a report on alerts). The ticketing system has been implemented to keep the coordination between users' actions concerning existing procedures.

In an analysis scenario, we want users to perform simple yet relevant tasks by giving them contextual UIs. Tasks are represented by UI buttons and they are related to an incident analysis procedure.

Data As analysts have to face a lot of heterogeneous data from various sources and sensors [START_REF] Zhang | A visualization system for multiple heterogeneous network security data and fusion analysis[END_REF], aggregation and correlation tools such as Security Information and Event Management systems (SIEMs) or Intrusion Detection Systems (IDS) give them high-level events or alerts that are much more interesting to investigate than raw data [START_REF] Pahi | Data exploitation at large: your way to adequate cyber common operating pictures[END_REF].

In our VE, we have simulated events or alerts from high-level data sources and we have provided contextual metrics that can give global information about the security state of the system.

In an analysis scenario, we could generate alerts triggered by some metrics provided by the simulated system such as the entropy of filesystems or the network traffic.

Visualizations Usually, 2D dashboards are used to give general information and status reports to everyone in the SOC. Analysts use a lot of dedicated tools to correlate or monitor data. These tools are tailored for specific cases and Visual Analytics solutions are used only if regular ones are not giving enough information [START_REF] Varga | The application of visual analytics to cyber security[END_REF].

In our VE, we have developed both 2D dashboards and immersive interfaces for different roles and needs. In an analysis scenario, analysts could be immersed while the coordinator will have a holistic point of view from a 2D dashboard (Figure 3). To limit cognitive load while allowing data correlation [START_REF] Zhong | A user-centered multi-space collaborative visual analysis for cyber security[END_REF], we have provided separate views that give complementary information about the monitored systems. For example, one physical view and one cyber view could be provided. These views respectively contain data about the 'physical' environment (device settings, last user logged on the device, geographical position of computers in a building, etc) and the network environment (network flow and topology, IP addresses).

To implement these features, we have designed an alert analysis scenario based on the WannaCry ransomware.

Ransomware modeling

We have chosen to model the Wannacry Ransomware4 , as it has caused major troubles in May 2017 and as its behavior could be described simply [START_REF] Mohurle | A brief study of wannacry threat: Ransomware attack 2017[END_REF]. Figure 4 sums up our modeling of WannaCry ransomware.

When Wannacry infects a computer (by being downloaded and executed for example), it encrypts data on the computer and it propagates itself through the network by using Windows seven Operating System (OS) vulnerabilities. The user must pay a ransom to get her data back, as the encryption could not be broken. The entropy of filesystems and network traffic metrics are enough to describe Wannacry's high-level behavior: when it infects a computer, it raises the entropy metric (files are encrypted, resulting of high activity) and when it propagates, the network traffic increases as well. These metrics raise alerts when they reach thresholds and these alerts can be true or false positives (for example a data backup and encryption or a download activity from a computer will raise these metrics even if these activities are not caused by WannaCry). WannaCry can also raise only one metric, for example, if the computer is protected against data encryption but not patched (the ransomware can propagate), or if it is patched but not protected.

One specific analysis scenario will be presented in the following section.

Proposed environments and scenario for usability evaluation

To perform a usability evaluation, we have chosen specific features to implement into our environment. In this section, we will detail the VE and the alert analysis scenario we have designed for the evaluation.

Evaluated environment

As we want to assess the effectiveness of VE for cybersecurity, we have decided to implement the Analyst role of our 3D CyberCOP model. Users will be immersed in an environment containing a potentially infected network of computers and will have to gather information to characterize alerts. In order to get information from the environment, we have provided three views, namely the alert, the cyber and the physical view (Figure 5). Users have to switch between them in order to get relevant information on the situation. -The alert view contains data regarding the alerts status. If a computer is concerned with an alert, it will be highlighted by a particle system for example.

-The cyber view allows users to get information from the network architecture or network processes. -The physical view contains data related to the processes of computers, their operating system, and other information.

Navigation in the Virtual Environment is made by using the teleportation metaphor, but we also have let the possibility to the user to use a joypad to move.

Information is displayed through 2D interfaces that are accessible by selecting computers or relevant 3D objects. Selection is made by using the selecting ray metaphor.

Analysis scenario

The alert analysis task has been simplified to be achieved by users with no prior knowledge about cybersecurity: only a few data were available and the procedure was lightened. The analysis scenario was the following (Figure 6):

-Alerts appear on the alert view, and users have to follow an investigation procedure to resolve the case. -The investigation is done by switching views to gather relevant information about the alert (an entropy alert should be investigated in the physical view and a network alert in the cyber one). Specific actions will be available for users, such as 'get information from this computer' or 'perform an anomaly analysis' (these actions are simulated according to procedures). -Once the user thinks she has enough information, she has to characterize the alert by escalating or dismissing it. Once it is done, she has to select another alert and continue the investigation.

This scenario was successfully implemented into a VE, and we have proposed a usability study based on this implementation.

Usability evaluation protocol

As we wanted to evaluate the usability of a VE for an alert analysis task, we have designed three different environments and an evaluation protocol.

Virtual Environments for alert analysis

Abstract environments, as 3D graph visualization are more and more used for data analysis whereas concrete environments are usually used for training sessions. We have wanted to see if users would perform better in a classical 3D Graph visualization representing the network topology or in an office-based one where computers, desktops and so on are represented.

To go beyond this comparison, we have also proposed an environment containing both the network topology and the office assets. These environments are represented in the Figure 7.

Order of the three proposed environment was defined by two conditions: even users begin with the graph-based environment (Abstract-Concrete-Mixed condition) whereas odd users begin with the office-based one (Concrete-Abstract-Mixed condition). The third environment was always the mixed one.

Protocol

We thought that users would find the concrete environment more entertaining but that they would also be more efficient in the abstract environment while the mixed one would be considered too complex. We have defined three scenarios presenting similarities so that the tasks were different in each environment to avoid a learning effect. These scenarios contained the same infected assets but positioned differently into the environments.

We had performed the evaluation with an Oculus Rift CV1. VE was developed with the Unity Game Engine and the VRTK toolkit for interaction metaphors. An event-based architecture was used to implement the scenario and to facilitate action logging [START_REF] Kabil | From Cyber Security Activities to Collaborative Virtual Environments Practices through the 3D CyberCOP Platform[END_REF].

Users had to fill a consent form where we explained the purpose of the experiment. Then, they had to perform a tutorial task in order to get familiar with the immersive devices and the VE. Alert analysis tasks lasted fifteen minutes; there were eleven alerts to investigate, and two false positives among them.

In between immersive tasks, we asked users to fill a SUS usability questionnaire [START_REF] Brooke | Sus-a quick and dirty usability scale[END_REF] and to answer some questions about the task itself.

At the end of the third immersive task, after filling the SUS and task questionnaires, we asked users which interface they had preferred and why. We asked also some questions about the physiological effects of immersive experiments and technology adoption.

The experimentation protocol is summed up in the Table 2.

Results and Findings

We carried out our experiment for a month and collected the results of thirty users, mostly computer science students. Most users were unfamiliar with immersive technologies and cybersecurity but familiar with video games. The total average duration of a user's evaluation was one hour and thirty minutes, and all users passed the analysis scenarios. In the Figures that we are going to present, the margin of error comes from the standard deviation. The p-values given come from ANOVA-type variance analyzes at 95%.

Results

The abstract, concrete and mixed environments respectively obtained SUS usability scores of 76.41, 72.42 and 77.33 out of 100 (Figure 8), which means that they are considered usable (the usability threshold of an interface is set to 70/100 for the SUS test). There were no significant differences in the environmental usability scores between the users who started experimenting with the abstract environment (ACM condition) and users who started with the concrete environment (CAM condition) (Figure 9).

The results of ANOVA-type variance analysis at 95% between the two conditions and for the abstract, concrete and mixed environments give p = 0.81, p = 0.71 and p = 0.69 respectively. Users have preferred the mixed environment combining abstract and concrete data representation. The ACM and CAM conditions for passing the experiment (i.e. starting with the abstract or concrete environment) had no effect on user preferences (Figure 10). Contrary to what we thought, the complexity of this environment was not a limiting factor for most users, but the fact that it was always explored in the third session may have influenced their choice. The average score for the questions concerning physiological disorders was 3.08/10, which means that users did not experience any discomfort during the experiments (Figure 11), although we have found that six users (three out of five users older than thirty-five years old, two users under the age of twenty-five and one user between twenty-five and thirty-five years old) felt more dizziness than the others.

Users have considered that they could easily acquire the skills required to use this VE (average score of 8/10), with a significant effect of age (average score of 5.8 / 10 for those older than thirty-five years old).

Immersive devices were considered pleasant and potentially usable at work. Environments and immersive technology were generally found to be relevant, with scores on questions about technology adoption and user experience above 6 out of 10.

An analysis score was assigned to users based on the accuracy of their alert characterization. A score of eight out of eleven means, for example, that they correctly characterized eight alerts (including false positives) out of the eleven present in the scenarios. On average, users have performed well during the experiments (average score greater than 8.5/11) (Figure 12 on the left). We found out that alert analysis time was significantly higher in the concrete environment than in the abstract and mixed environments (p = 4×10 -4 ). Users have analyzed alerts more quickly in the mixed environment, although there were no signifi- cant differences in analysis times between abstract and mixed environments (p = 0.1).

As the mixed environment was always explored last, adaptation to the experiment as well as familiarity with the task can explain this result (Figure 12 on the middle). On the other hand, we did not find any differences concerning the distance traveled in different environments (p = 0.59) (Figure 12 on the right). There was barely any effect of age or familiarity on results, but our sample of users was too small to interpret significant differences between familiarities (for example we had only five users that were more than thirty-five years old). Some data were available to users to help them to characterize alerts (as the Operating System version or the IP address), and between each experiment, we asked users which data sources were useful. We have noticed that users tended to use more and more data sources to analyze alerts while they were gaining experience during the experiment (Figure 13). Fig. 13. Data usages for characterizing alerts. We can see a progression of data usages from scenario 1 to scenario 3 (scenarios were always made in the same order).

Findings

This experiment showed us that a VE for non-experts in cybersecurity is usable, even if results can be discussed.

Users' performance (e.g. score) was not better from the first try to the last one. Maybe the task was too simple so that most of the users had been performing well but have not progressed through experiments. Because we gave users barely any indication (except in the inter-experiment questionnaires in which was mentioned which data source could be used to conduct the analysis), they had sometimes no idea about their performances: they knew that they were correctly characterizing alerts, but sometimes they were having some doubts.

Globally, users did not face cybersickness, even if our five subjects who have more than thirty-five years old and were familiar neither with video games nor with immersive interfaces felt more dizzy according to the questionnaire answers.

The three views were well used by users, but some users pointed out that data were not so different between them, and that alert view was not useful. Maybe a 2D User Interface (UI) could be provided to users to get alerts information and to reduce views switching. Some users have regretted that any help was available during the experiment. This could be provided easily by adding a 2D UI attached to any user's hand and showing pieces of advice.

Instead of investigating alerts one after the other, users have tended to crossanalyze alerts to see if there were some redundancies in data or if computers close to each other in the network topology have the same processes and alerts.

The fact that users preferred the graph-office mixed environment is maybe due to that it was the last environment they explored, and that they were more familiar with the system. The fact that they have not preferred the office-based environment regardless of condition is an interesting result that we need to investigate.

Overall, users have considered that these VEs for cybersecurity were usable and they were ready to use them, but as they were not cybersecurity experts, we need to perform other studies to analyze this point. Immersive technologies were appreciated by users and they were interested in using them for their work or activities.

Conclusion

By studying Security Operations Centers (SOCs) operators' practices, we have designed an activity model, the 3D CyberCOP, which aims at translating cybersecurity activities into a Collaborative Virtual Environment (CVE). This model have allowed us to design a Virtual Environment (VE) in which users could perform cybersecurity alert analysis scenarios based on the WannaCry ransomware. In order to assess the effectiveness of merging both data visualization and training approaches for cybersecurity operators, we have conducted a usability study with non-expert users.

This study has shown that they have succeeded in interacting and navigating through the environments. They have managed to gather data and to understand the cyber situation only after few sessions with great scores, even if the task seemed difficult on the first try. Users have shown a preference to the environment that contained both graph and office visualizations and have found these environments useful for analyzing cyber threats, with a usability score superior to 70.

Further studies should be made to assess the effectiveness of such environments for cybersecurity experts.

In the future we will develop a 2D version of the experimentation, to compare the results between immersive and non-immersive interfaces. The usefulness of view switching will be evaluated maybe in a collaborative experiment. Linking our VE with an existing cyber data analysis tools will bring us more capabilities and more realistic cases, and we will evaluate the reliability of this approach by comparing training with a cyber training tool and with our VE.

Fig. 1 .

 1 Fig. 1. Venn diagram of our position about the usability of Virtual Reality Applications for cybersecurity, which is at the intersection of three distinct domains, Virtual Reality, Data Visualization and Cybersecurity.

Fig. 2 .

 2 Fig.2. Data sources treated by a cyber analyst to get Cyber SA, from[START_REF] Zhong | Studying Analysts' Data Triage Operations in Cyber Defense Situational Analysis[END_REF].

Fig. 3 .

 3 Fig. 3. Examples of visualizations developed from our activity analysis: a 2D monitoring dashboard (TOP) and an immersive interface (BOTTOM).

Fig. 4 .

 4 Fig. 4. WannaCry behavioral model used for alert analysis scenario creation.

Fig. 5 .

 5 Fig.5. From top to bottom, the alert view, the cyber view, and the physical view. Views are colocated in the VE, in order to facilitate the switching between them. Users select computer to access data relative to the current view.

Fig. 6 .

 6 Fig. 6. Alerts investigation simplified scenario.

Fig. 7 .

 7 Fig. 7. From top to bottom: concrete office environment, abstract graph-based environment, and mixed environment. In these environments, users had to analyze cybersecurity alerts by following specific analysis scenarios.

Fig. 8 .

 8 Fig. 8. Mean values (with standard deviation error) of SUS usability score for each environment. All environments gets a score higher than 70, which means that they are usable (from the SUS test perspective).

Fig. 9 .

 9 Fig. 9. Mean values of SUS usability score for each environment and for both experimentations' conditions. No significant differences between experimentations' conditions.

Fig. 10 .

 10 Fig. 10. Users' preferences for proposed environments. No significant differences between experimentations' conditions.

Fig. 11 .

 11 Fig. 11. Cybersickness questionnaire answers. Users have not felt particularly dizzy while performing the experimentation.

Fig. 12 .

 12 Fig. 12. Mean values of scenario scores, analysis time per alert and distance traveled for each environment.

Table 1 .

 1 3D CyberCOP model allowing the adaptation of SOC features into a Virtual Environment.

	FEATURES	ACTIVITY MODEL	DESIGN SOLUTIONS EXAMPLE SCENARIO
	Roles	-Trade-off between decision and data analysis -Hierarchical interactions	-Users with specific actions -Hierarchical interactions	-Analyst immersed in VR -Coordinator with 2D Dashboard
	Tasks	-Ticketing system -Specific tasks from defined roles	-Ticketing system -Simulated interactions or scenarios	-Simulated actions for incident analysis
			-2D, 3D or immersive	-2D and
	Visualizations	-Tools for monitoring, correlation and reporting	filtered visualizations -Integration of	immersive visualizations -Physical, Cyber
			existing tools	and Alert views
	Data	-Aggregated by SIEMs or IDS -From various sources	-Simulated data sets and metrics -Integration with existing tools	-High level events and alerts -Network & Entropy metrics

Table 2 .

 2 Usability experiment protocol

	Experimentation Step	Duration
	Greeting and consent form	15 min
	Tutorial and familiarization task	10 min
	Three VE experiments in two different orders 10 x 3 min
	Usability and task analysis survey	5 x 3 min
	User Experience and preferences survey	10 min

https://www.nist.gov/cyberframework

http://cert-mu.govmu.org/English/Documents/White%20Papers/White%20Paper%20-%20The%20WannaCry%20Ransomware%20Attack.pdf

Acknowledgments

This work was supported by the Cyber CNI Chair of Institute Mines Télécom, which is held by IMT Atlantique and supported by Airbus Defence and Space, Amossys, BNP Paribas, EDF, Nokia and the Regional Council of Brittany. It has been acknowledged by the Center of excellence in Cyber Security.