Esteban Perrotin
email: esteban.perrotin@airbus.com

Matthieu Roy

Ariane Herbulot

Michel Devy

Fabrice Bousquet

Towards the certification of vision based systems: modular architecture for airport line detection

This paper addresses the certification issues for vision based systems embedded on civil aircraft to assist pilots during the ground navigation phase. We propose a design methodology, through the example of a modular architecture to detect ground mark lines in a color image. Our detection method is based on the combination of classical image processing algorithms, deep learning methods and a particle filter algorithm. We argue that the main interest of this architecture is to ease the certification problem when compared to an end-to-end neural network. We discuss about difficulties around certification and propose some arguments.

Context

Computer vision applications have made considerable progress in recent years, with applications to many fields, from healthcare to autonomous vehicles. In this context, the civil aeronautic is considering using vision based systems to support pilots in their operations during the ground navigation tasks. For instance, functions such as obstacle detection, axis keeping, runway detection and others are currently studied. However, due to the complexity of such functions, it is challenging today to achieve the certification of systems largely based on computer vision algorithms in civil aeronautic field.

Actually, all functions embedded in civil aircraft have to be compliant to standard design processes defined by the authorities. These processes are described in documents such as Certification Specification [START_REF] Easa | Certification Specifications and Acceptable Means of Compliance for Large Aeroplanes CS-25[END_REF], the ARP-4754a [START_REF] Sae | ARP4754a -Aerospace Recommended Practices[END_REF] and DO-178-C [START_REF]DO-178C[END_REF]. These documents define Figure 1: General V-model and our scope. complete guidelines to design, develop and implement functions to be embedded on civil aircraft. This whole process is called certification. The standard certification process follows a V-model. It consists in designing, developing, implementing, verifying and validating a system by following specific steps described in guidelines. An example of V-model is shown in Figure 1. More details are provided in the section 2.1. Some functions embedded on civil aircraft are already using vision based systems (VBS). For example, the A380 from AIRBUS is equipped with a camera integrated in the aircraft fin. The video input can be sent to both passengers and pilots. The pilot can use the video output as an help to drive the aircraft during the taxi phase. The image is directly used and interpreted by the pilot without any algorithm used for its interpretation. However, actual certified functions based on vision systems let the pilot as the principal actor to extract and interpret information represented in each image. New functions, such as line detection, motion estimation or obstacle detection are using complex algorithms to extract and process features from the image in order to improve the pilot assistance. The certification of such algorithms has never been done in the European civil aeronautic field. That is why, previous studies [START_REF] Boniol | Challenges in the certification of computer vision-based systems[END_REF] highlighted the fact that the design of computer vision-based systems opens a new paradigm in certification. These difficulties are mostly due to the complexity of algorithms. For example, most of recent computer vision algorithms are based on machine learning (ML). Machine learning algorithms learn through examples how to analyze images in order to achieve a desired task. This approach is not compliant with standard design process [START_REF] Bhattacharyya | Certification considerations for adaptive systems[END_REF]. Actually, current standards require an explicit description of detailed requirements. This is the opposite of ML systems which they learn the operational behaviors through examples and not from requirements. In order to prepare a new standard, the European Union Aviation Safety Agency (EASA) is working on guidelines [START_REF] Easa | Concepts of design assurance for neural networks (codann) ii[END_REF] [START_REF] Easa | Easa concept paper: First usable guidance for level 1 machine learning applications[END_REF] to safely design such systems. There are also many public research projects on this topic. The DEEL project is also currently working on this problematic and and have published a White Paper [START_REF] Delseny | White paper machine learning in certified systems[END_REF]. They address many questions on the risks, challenges and potential solutions of using ML in systems submitted to certification constraints. However, vision based algorithms are not limited to machine learning.

In this paper, we present our position with regards to VBS certification issues. In order to illustrate how to solve these problems, we will take the example of line detection on airport runways/taxiways and describe our methodology to design the modular architecture of a specific line detection algorithm.

Line detection is a useful function that could make easier the ground navigation for the aircraft pilot. Due to its simplicity, it is probably one of the first task using vision based algorithms which could be certified. However, most of recent methods repose on end to end deep learning architecture [START_REF] Wang | Lanenet: Real-time lane detection networks for autonomous driving[END_REF]. Because of the certification aspects, we desire to construct a functional architecture without using end to end deep learning methods. Figure 2 shows an example of system architecture that could be used to guide an aircraft on a taxiway, and/or to display information to the pilot. We will focus our discussions on the design and tests of this line detection function architecture. We will not discuss the hardware implementation or The paper is organized as follows: Section 2 presents difficulties that could be encountered in the certification of computer vision based systems. Section 3 presents our proposed architecture in the specific line detection application. Section 4 explains our arguments to help certification of this architecture. Then the last section concludes.

General aspect of the certification of Computer Vision-based systems

Computer vision-based algorithms can be summarized in two main parts: extract from each image discriminant information (often called features) and then process this information to achieve the desired operation. Extracting pertinent information is a difficult task. An image can be seen as a 2D signal and the undesired information can be seen as a structured noise. In our application, we aim at extracting features that characterize lines. Hence, we desire to remove all other elements of the image.

In order to answer such problems, complex algorithms of computer vision are used. We will show in the following section that standard guidelines used for the development of civil avionic function are difficult to be applied on complex computer vision items.

In this section, we briefly present general aspect of certification, then we expose steps that could lead to difficulties in the certification of vision based algorithms and expose some elements from the state of art.

General aspects of certification

The principal guidelines of the civil aeronautics certification is provided by standards such as ARP-4754A [START_REF] Sae | ARP4754a -Aerospace Recommended Practices[END_REF] and DO-178C [START_REF]DO-178C[END_REF]. To design a new avionic high level function, the first point of the design process consists in analyzing the impact of this function in case of potential hazards. This step is called functional hazard analysis (FHA). For instance, if a failure of the function could lead to endanger the aircraft or passenger safety, the function will probably be classified as "catastrophic". It will be necessary to prove that the probability for the failure of this function during a flight hour is less than 10 -9 . This FHA step will drive requirements that should be verified by the implementation of the function.

Then, this high-level function is refined into lower level functions and a preliminary functional architecture is proposed. This architecture is submitted to a preliminary system safety assessment (PSSA During the design process, multiple tests have to be defined to guarantee and verify that the implementation of items is correct and fulfill requirements at each level. Tests are driven by the system safety assessment (SSA) to ensure the availability and the performance of the function in all identified scenarios.

This safety design process is used to develop high level safety functions for civil avionic system.

Difficulties around certification for VBS algorithms

The certification process ensures that the proposed high-level function is correctly designed and implemented following specific guidelines depending on the level of criticality of the function.

Each sub-level item should verify specific requirements. However, in the case of computer vision systems some aspects of the guidelines seem complicated to satisfy, in particular in systems using machine learning. The authors in [START_REF] Bhattacharyya | Certification considerations for adaptive systems[END_REF] and [START_REF] Mirzaei | Safety Cases for Adaptive Systems of Systems: State of the Art and Current Challenges[END_REF] discuss on this problematic for adaptive systems. In [START_REF] Boniol | Challenges in the certification of computer vision-based systems[END_REF] the authors detail the certification challenges on a specific VBS dedicated for two applications: visual odometry and obstacle detection.

Considering this previous work and with regards to the difficulties we encountered during our development, we precise some points that seem difficult to answer considering the current certification standards:

• Comprehensible requirements have to be written. In computer vision applications writing high level requirements -HLR (e.g. detect a specific object) could follow the usual process. However, writing lower level requirements could be harder (e.g, explicit evidences that characterize a given object in all kind of environments).

• Verifiable requirements: In order to verify that a computer vision algorithm is correctly working, we have to present different inputs in the system and verify that the outputs correspond to the desired ones. Ideally, we would like to test all potential inputs. But, due to the high dimensionality of images (255 1024×768×3 ≈ 10 4718592 combinations for 1024px ×768px color images) it is not realistic to test all possibilities. Most of these images are pure noise or not relevant. A solution could be to provide a minimal dataset, with a distribution of images close to the real application. Furthermore, the desired outputs have to be known for each image tested. Manual labelling on a huge dataset seems impossible and it raises an issue on the accuracy on the labelling process. When using automatic or semi-automatic labelling one has to prove that the algorithm used for labelling is robust with regard to the performance for the high-level function. Validating an automatic labelling process raises similar certification activities. Section 2.3 contain more details about the acquisition of a test data set.

• Robustness: The recent advances in computer vision (in particular with deep learning methods) have greatly enhanced performance for many applications. However,these performances must be defined and evaluated in particular in degraded conditions (fog, rain, low illumination, etc.). In such conditions with environmental hazards algorithms could perform poorly. Avionic systems have to operate correctly despite abnormal inputs and conditions.

In addition the stability of the results have to be considered. For a small variation in the input image, computer vision algorithms have to output the same value. There is no evidence that computer vision algorithms achieve this stability. In particular this case appears with deep learning methods and adversarial attack [START_REF] Lin | Adversarial attacks in modulation recognition with convolutional neural networks[END_REF].

• Traceability: During the design process, the requirements of a given level are broken down into one or more requirements of the next level. It has to be shown that each low level requirement (LLR) corresponds to a requirement of the higher level (HLR).

• Interpretability: This point is not essential for the process of certification but it argues about trustworthiness of the system and helps to convince authorities about the viability of the system. In order to trust a functionality, each step of a function has to be explained and must have its proper purpose based on rationals. This is not true for machine learning based applications. Machine learning learn to extract and process features from the image through examples. Many works have been done to explain which features are extracted, how an interpretation is generated from these features, especially for deep learning methods [START_REF] Molnar | Interpretable Machine Learning[END_REF]. However, this aspect should be accepted for classic computer vision algorithms.

Related works

To address these difficulties, many works have been done, for example to ensure robustness of algorithms, to discuss and propose methods for the creation of test data sets and develop solutions to interpret complex algorithms such as convolutional neural networks.

To ensure robustness of algorithms (accuracy and confidence on the output), methods have been proposed to compute bounces of confidence [START_REF] Germain | Risk bounds for the majority vote: From a pac-bayesian analysis to a learning algorithm[END_REF]. These methods could be based either on the demonstration of a formal proof that gives a guarantee about envelope of the algorithm output, or on the execution of enough tests to cover all potential use cases.

In computer vision, due to the complexity of algorithms, the large variety of input images and the lack of precise requirements, it is difficult to develop formal proofs for applications. However, this is already the case in many avionic systems. This is why functions are tested on large data tests. The construction of these test data sets raises many interrogations. Since it is not feasible to test every images, it is necessary to select pertinent scenarios to test algorithms. But algorithms have to be tested in many corner cases, so that acquiring real data for every desired scenario is impossible. A solution consists in generating synthetic images from simulators. Many works have been done to provide image generators useful to test computer vision algorithms for given applications. For instance, we use OKTAL-SE simulator. It generates color or infrared images based on a physical model. However, there is a gap between their simulated image and real image, in particular in the level of detail for the textures representation.

On the other hand, Zendel et al. [START_REF] Zendel | Cv-hazop: Introducing test data validation for computer vision[END_REF] have proposed a system of guide-words to quantify and qualify hazards that could appear in computer vision applications. Their works provided an answer to "Which situations should be covered by the test data and have we tested enough to reach a conclusion?". By using CV-HAZOP, in [START_REF] Zendel | Analyzing computer vision data -the good, the bad and the ugly[END_REF] and in [START_REF] Zendel | How good is my test data? introducing safety analysis for computer vision[END_REF] they propose a standard procedure devised by the safety community to validate complex systems.

In the next section, we describe our line detection algorithm, so that we could illustrate how to deal with the verification of the properties summarized here above.

Modular architecture of our line detection method

As said previously, constructing an argument of certification for VBS is a difficult task. This section describes our methodology to construct our line detector. We start by modeling the problem using prior information. Then a general architecture solving this problem. Finally we describe each item in this architecture. Our principal objective is to ease the certification process by diminishing the complexity of verification and validation on each item of our architecture.

Problem modeling

As said in the previous section, an important point to follow in the certification process is the definition of requirements. The objectives of the function (high level requirements -HLR) should be as clear as possible. In our application, we aim at detecting lines (ground marks). That is why, we define the objectives of our function as the following example: "The function has to detect ground marks that define lines for aircraft ground navigation. The input of the function is an RGB image acquired by an embedded camera (typically in the fin or the cockpit as shown in Figure 3) and parameters set from operational concept. Internal and external parameters of the camera are known. Lines have to be detected until a desired distance to the camera. Output lines will be defined by a set of points.".

Figure 3: Input RGB image from the fin camera.

We start by constructing a model of the lines that will help to fulfill HLR. We construct this model by following some key points:

• Use the maximum of prior information to select important regions of interest (ROI) in the image. These regions have a high probability of containing desired information (presence or absence of ground marks). A ROI is constructed by using the parameters of the camera and the desired maximum range. In practice, it will mask unwanted elements like the aircraft (if the image is from the fin camera) or the sky.

• Construct it in order to facilitate broken high level requirements into lower level requirements. Construct a set of information that represents the object and its evolution (spatial and/or temporal). In our case, the HLR specifies the output: a set of points. Thus, we consider this set of points as the spatial propagation of points that describe a line. So, we have to detect a first point, define a dynamic law that describes the propagation of points, and precise an end criteria.

• Find and write most of the properties verified by the desired object in the image (size, structure, texture, etc.). These properties will help to verify that extracted information is correct. These properties could also help in writing an end criteria. Following these points, we model a line as a spatial repetition of a pattern (Figure 4). The pattern is considered known. The pattern will repeat himself in the presence of some noise (degradation, orientation, illumination variation, deformation, etc.). This repetition is done in respect to some properties provided by the International Civil Aviation Organization (ICAO) [START_REF]AERODROMES: aerodromes design and operations[END_REF]: the curve made by central points of the patterns must satisfy constraints. In addition, depending on the illumination, the color of the painting on each pattern should be close to yellow [START_REF] Meymandi-Nejad | Colorimetric space study: Application for line detection on airport areas[END_REF]. The width of lines is known.

To describe a pattern in a thumbnail, we use a state vector

X k = {(x k , y k ), θ k , s k }. (x k , y k )
is the position of the center of the thumbnail in the image, θ k is the orientation of element in the thumbnail and s k is the size of the thumbnail (considered as a square). The evolution between state vectors is described by a function g, where g uses the current state vector X k to update the next state vector X k+1 . The update depends also on external factor noted in an unknown noisy term U k :

X k+1 = g(X k , U k ) (1)
This function is unknown and we consider a simplified model. We use the following equations to update the variables inside the dynamic model:

x k+1 = x k + s k cos(θ k ) (2a) y k+1 = y k + s k sin(θ k ) (2b) θ k+1 = θ k (2c) s k+1 = min(max(s k , a), b) (2d)
Where the constants a and b determine the minimal and maximum size of thumbnails. It models the line's trajectory as a linear trajectory and supposes the two variables s k and theta k are not evolving. This model is not quite precise. But the particle filter will correct the evolution of these parameters according to the observations from the image. In practice it is good enough to catch simple line patterns. In the future, we plan to add other variables to capture and describe environmental perturbations (such as the illumination, the principal color, etc.) and also enhance the dynamic model to improve performances.

General architecture

Using this model, we split our specification into three parts: find potential starting points of the line, recognize a pattern of the line from a thumbnail described by a state vector and predict the evolution of the pattern in the image. Each item has its own requirements driven from HLR and its own unit tests. The coverage of the HLR by LLR is ensured by the model. Figure 5 shows our design. Once the interface between modules is defined, it is important to note that each part can be developed independently. Following the key points expressed above and using our model, we propose a modular architecture to detect lines from an image. The architecture of the system is described in Figure 6. The main point of our architecture is to separate prediction from a model to information extracted from the image (since the image can be considered as a complex noisy signal). The proposed algorithm works as follows:

1. The image is acquired by the camera and considered as a representative data of the reality.

2. Using the image, an initialization based on simple assumptions (such as gradient, color and position) will propose some initial conditions (potential starts of lines) as an initial state vector X 0 .

3. Using the initial condition and the dynamic model of lines a filter will predict the next state of the line Xk+1 .

4. The prediction will be sent to an observation function that will attribute a measure corresponding to the probability of the correctness of the prediction Xk+1 with regards to the image and modeled feature of the desired pattern.

5. Using the measures, the filter will update his predictive model: Xk+1 → X k+1 .

6. Until an end criterion is verified, the program will repeat from step (3).

This algorithm detects one line. In practice, we applied this algorithm many times to detect many lines in the image. Some heuristics are used to discard some elements like the ridges of the lines (see Figure 3).

Description of modules

Now, we detail and explain our choice for each module.

Initialization module. The initialization function uses the image as input. It has to propose potential starting points of a ground line. To detect these starting points we developed an algorithm based on classical image processing, using gradient and color prior from the model of line. This algorithm is not detailed in this paper. The robustness of this function is low. However, in case of failure (wrong starting point), the proposed point might be discarded by the filtering and observation function. That is why we favor recall over to precision.

Filtering module. To predict the next state of the line, we have to estimate the solution from the equation 1. To solve this equation many filters can be used such as the Kalman filter. However, in practice, that model is limited due to the noisy term U k . If the distribution of U k was Gaussian, the Kalman filter would be a suitable solution. In our case, we choose to use a particle filtering algorithm similar to the one presented in [START_REF] Meymandi-Nejad | Lane detection and scene interpretation by particle filter in airport areas[END_REF]. Firstly because it can solve this problem without knowledge on the distribution of U k . Secondly, to provide a first analysis on problems encountered by the certification process of particle filters. These filters are useful for navigating, positioning and tracking [START_REF] Gustafsson | Particle filter theory and practice with positioning applications[END_REF], they could be used in more applications in the future. Since our project is to put on the table new technologies, we made the choice to construct our line detector using a particle filter.

The particle filters are a set of Monte Carlo algorithms. The objective is to compute the posterior distribution of a stochastic process. The first step is to create particles {X n k } N n=1 from an initial prior distribution. Each particle corresponds to an estimation of the state vector. A weight w n k is associated with each particle. This weight corresponds to the confidence in a particle to represent the actual real state. The closer the particle is to the real state the higher its weight is. In most systems using particle filters the weight is computed by using a measure provided from external sensors (such as GPS or inertial sensor in the case motion estimation). In our case, we define a sub-item called "observation module" that attributes a weight to a particle using as input the image and the variables contained in the particle. This observation module is detailed below. Once all particles have an associated weight, the particle filter algorithm will re-sample particles using the distribution created by {(X n k , w n k )} N n=1 . More information about particle filtering can be found in [START_REF] Ristic | Beyond the kalman filter: Particle filters for tracking applications[END_REF][22] and more about this particle filtering method are provided in a french paper [START_REF] Perrotin | Détection des lignes aéroportuaires par méthode de filtrage particulaire: Évaluation de fonctions d'observations[END_REF].

The principal advantage of particle filter algorithms is their ability to solve nonlinear problems tainted by a noise without prior knowledge about this noise. The principal inconvenience of these algorithms is the computing time. The more a problem is complex, the more it will require particles to compute a good estimation. We don't cover computation time problematic in our approach. However, it could be possible to cover this issue by adapting the amount of particles depending on the confidence in the observation module [START_REF] Zhang | Confidence-level-based new adaptive particle filter for nonlinear object tracking[END_REF].

Observation module. Using a state vector provided by the filter (through a particle), it selects a thumbnail in the image (between 15 and 51 pixels depending on the variable s k in the state vector), resize the thumbnail to 33×33 pixels and attributes it a score. This score is between 0 and 1 and evaluates the pertinence of components in the state vector in regard to the thumbnail. The score is the product of the output of tree subitems:

• The first one is a binary classifier. It scores the presence or absence of ground marks in the thumbnail. After a preliminary study [START_REF] Perrotin | Détection des lignes aéroportuaires par méthode de filtrage particulaire: Évaluation de fonctions d'observations[END_REF], we selected a small convolutional neural network (CNN) as classifier. This choice is also motivated to provide a real application to use CNN in a restrained context controlled by other elements. By using a CNN at this part of the line detection method, it permits to achieve correct result in the line detection without using end-to-end CNN. To train this CNN we constructed a dataset of 4000 thumbnails with manual annotation (50% positive and 50% negative). The score used is directly the output of the classifier. More details about the creation of the CNN are provided in [START_REF] Perrotin | Détection des lignes aéroportuaires par méthode de filtrage particulaire: Évaluation de fonctions d'observations[END_REF].

• The second item measures correlation between the orientation θ k in the state vector and the principal orientation in the thumbnail. We compute the orientation of the element in the thumbnail using a method based on the Gabor filter.

• The last item measures the centering of the line in the thumbnail using classical image processing techniques such as gradient, color processing, Otsu's binarization and Hough transformation. It extracts the principal straight yellow lines in the thumbnail. Then it computes a score (between 0 and 1) depending on the projection of the center of the thumbnail on the line equation.

The product of these scores produces the final score that will serve as the weight for the particle in the filter.

In the next section, we discuss the verification and validation of these modules.

Certification Arguments

This section presents our arguments and thoughts about the feasibility of the certification for this architecture. This architecture proposes to reduce the amount of tests required to verify the system by using unit tests on each item. The principal assumption is: "If each item works correctly the function works correctly". This assumption holds because high level requirements are covered by lower level requirements. Each item has a simple and comprehensive task. The next parts develop discussion and arguments about verifying whenever each item is doing its task correctly. This modular architecture should make the certification easier. The more an architecture is decomposed on items the more we can hope to reduce the complexity of each item and facilitate the creation of tests to verify performance of each item. It also helps to implement monitoring systems. Each item can have its own monitoring system and improve confidence on its results. In addition, when a module is changed (or improved) the certification should be done for this module only. It can significantly reduce certification cost when updating a system.

Filtering module

From the point of view of certification, Monte Carlo methods are often used to test algorithms but are generally discarded in embedded avionic systems. This is partially due to the fact that these algorithms use random number generation and one requirement for certification is repeatability. However, it is possible to generate offline a set of distributions that verify desired prior distribution. In addition, some particle filter algorithms use fewer random steps. A survey of recent advances in particle filtering can be found in [START_REF] Wang | A survey of recent advances in particle filters and remaining challenges for multitarget tracking[END_REF].

Errors in the particle filter could occur depending on three conditions. Firstly, if the dynamic model is incorrect and does not match the real trajectory of the line. Secondly, if the prior distribution is not realistic (e.g. if the distribution has a huge bias). Thirdly, if the measure provided by the observation module is wrong and does not bring information about the evolution of the dynamic system. Figure 7 shows these potential errors. To test and validate performances of the filter, we build a generator of lines. An example of the generated line is shown in Figure 8. In this generator, it is possible to compute the perfect measure (it is computed by checking if the position of the particle is on the line or not and if the orientation of the particle is correct or not). We selected a prior distribution as a Gaussian distribution, where the variance is fixed in regards to the width of the line. These tests allow validating that within our dynamic model and prior distribution this method can follow line until a certain curve and depending of the amount of particles. Furthermore, authors have been conducting research to prove the convergence of the particle filter [START_REF] Hu | A general convergence result for particle filtering[END_REF] or computing bound of errors in particle filtering algorithms depending on incorrect model assumption [START_REF] Vaswani | Bound on errors in particle filtering with incorrect model assumptions and its implication for change detection[END_REF]. Also, they have proposed solutions to enhance the algorithm in presence of noise and bias on the measure [START_REF] Abdallah | Box particle filtering for nonlinear state estimation using interval analysis[END_REF]. In regards to the state of the art, it should be possible in some situations to use formal proof to guarantee confidence of the particle filter. Otherwise, the feasibility of tests ensure the possibility to compute performance on such algorithms.

Observation module

The observation module obtains as input a state vector from the filtering module and an image from the camera acquisition. It has to judge the pertinence of the state describing a line pattern. The first step of this module is to create a thumbnail by selecting a specific region of interest in the image using information in the state vector.

Then it attributes a measure by combining the result of three sub functions. The first one is a classifier that predicts the presence or absence of the desired pattern in the thumbnail. The second one is a function that estimates the principal orientation of elements in the thumbnail. And the last is a function that estimates if the element is centered in the thumbnail.

Classification. We provide here a minimal scope to use deep learning methods. Instead of using them to achieve complex tasks in high dimensional images, we reduce the problem to a simple binary classification of small thumbnails.

In addition, the output of the classifier does not require a very high integrity. As said previously, the filtering module accepts a part of error from the measure. It is acceptable for the classifier to make some wrong prediction.

In this context, it is still not possible to test the classifier on every possible image (256 3×33×33 ≈ 10 7843 ). At the same time it should be possible to compute confidence bounds for the classifier by using methods proposed in the state of art. For example, the Pac-Bayes theory [START_REF] Pitas | Pac-bayesian margin bounds for convolutional neural networks[END_REF], adver-sarial method [START_REF] Lin | Adversarial attacks in modulation recognition with convolutional neural networks[END_REF] or bounce generalization [START_REF] Long | Size-free generalization bounds for convolutional neural networks[END_REF] seem promising. This part is still ongoing and at the moment the only confidence in our classifier comes from a data test consisting of approximately 4000 thumbnails (50% positive and 50% negative). This dataset is constructed by manual annotation and we lack arguments in the confidence of this test. Figure 9 shows examples of thumbnails of the dataset. Our convolutional neural network achieved 88.3% of accuracy on training and 87.2% on the test data set. Orientation and center measure. The orientation measure defines the general orientation of a line in the thumbnail. The orientation is found by filtering the thumbnail using the Gabor filter on every possible orientation. The maximum response of the filter appears when the direction of the line is in the same orientation that the filter. In the case of absence of line in the thumbnail, the response of the filter is not predictable. This function is checked from a dataset similar to the one built for the classifier. In addition, because of the pattern of line, we can ensure that the Gabor filter has one of the best response when the line is the principal element in the thumbnail. Another measure consists in determining whenever the line is or not in the center of the thumbnail. This is done by geometric image processing without much difficulty. It is verified on the same data set as for orientation measure.

Initialization

The initialization extracts features that lines should verify: color, gradient and position assumptions. From camera parameters we defined the region of interests where the starting points of the lines can be. In each region, we use color processing to separate "yellow" elements from others. Then we use the morphological operator and skeleton algorithm to select potential starting points. Due to the use of color processing, environmental hazards impact the result. To im-prove performance and robustness against illumination variation and other hazards, the defined color "yellow" is adapted in each region by using a method proposed in [START_REF] Meymandi-Nejad | Colorimetric space study: Application for line detection on airport areas[END_REF]. In case of a false starting point, other modules should detect and discard this point (absence of ground marks, length of line, etc.). It is still possible to miss a line because the minimal requirements are not fulfilled (illumination, camera resolution, etc.).

Verification

The final verification is done qualitatively on real images from different scenarios. Figure 10 shows results to illustrate the function. Points of the same color correspond to the result of the filter at each iteration. Red curves correspond to the quadratic regression of these points. Currently, the system is not designed to be robust to occlusion. As we can see, the initialization step captures only three lines. Lines far away from the camera are not detected because of this step. The architecture is designed to reduce false positives (erroneous detection).

Figure 10: Results with our architecture.

Conclusion and future works

This paper focuses on the design methodology of vision based algorithms. We proposed a modular architecture for line detection applications designed to ease the certification process. This proposition avoids the use of end to end deep learning methods. They seem to have better accuracy but they are not compliant with the actual certification process. The main interest of decomposing vision architecture into smaller parts is to facilitate the generation of tests. Each part of the architecture has its own specifications and its own test data set. In addition, it provides a minimal restrained context where complex algorithms, such as deep learning methods, could be studied. Also, it should be easier to update and improve components one by one.

To build this architecture, we propose a general method describing the spatial or temporal dynamic of the visual object and decompose the architecture into two parts. The first one will predict features that should describe the object.

The second one will check in the image if this feature corresponds to reality. This architecture should work for object tracking. We are currently working to build larger test data sets and a monitoring system using temporal information (video processing). The question about test data sets is still ongoing. The use of guidelines such as CV-HAZOP helps the identification and construction of potential hazardous scenarios. However, data from these scenarios are not easily acquired and we have to base tests from simulators. It raises the question about the certification of such simulators.

Figure 2 :

 2 Figure 2: Example of visual based functions integrated in a civil aeronautic system

Figure 4 :

 4 Figure 4: Model of a line.

Figure 5 :

 5 Figure 5: Design and test of our model.

Figure 6 :

 6 Figure 6: Simplified schema of our modular line detection architecture.

Figure 7 :

 7 Figure 7: Potential errors of the particle filter

Figure 8 :

 8 Figure 8: Example of test to validate the particle filter under the proposed dynamic model. In white the generated line. In green, particles with high weight (positioned on the line). In red particles with low weight (not on the line).

Figure 9 :

 9 Figure 9: First line shows thumbnails considered as positive. Second line shows thumbnails considered as negative.

ACKNOWLEDGMENT

This work was supported by AIRBUS Operations. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views AIRBUS Operations. We would like to thank the OKTAL-SE company for providing the simulation tool, used to build synthetic images.


Marius Beka

D Waroux

B E Nguema

C Kolski

N Malvache

Design of a human-error-tolerant interface using fuzzy logic

Keywords: Human-machine interfaces, human error, human behaviour, fault tolerance, human operator models, fuzzy logic

HAL is

Introduction

Today's increasingly complex industrial systems require highly skilled operators, who need to control several parameters at once. This implies that human reliability should be ensured [START_REF] Swain | Handbook of human reliability analysis with emphasis on nuclear power plant applications[END_REF][START_REF] Rasmussen | Information processing and human-machine interaction, an approach to cognitive engineering[END_REF][START_REF] Hollnagel | Human reliability analysis, context and control[END_REF][START_REF] Laprie | Guide de la sûreté de fonctionnement[END_REF][START_REF] Kolski | Interfaces homme-machine, application aux systèmes industriels complexes[END_REF].

Certain circumstances may bring about grave errors, even with reliable operators [START_REF] Reason | Human Error[END_REF]. One way of avoiding such errors is to develop specialised, intelligent (or adaptive) help systems (Hancock and Chignell, 1989;Schneider-Hufschmidt et al., 1993;[START_REF] Kolski | Implementation of AI techniques for "intelligent" interface development[END_REF][START_REF] Kolski | A Review of "intelligent" human-machine interfaces in the light of the ARCH model[END_REF]. The Human-Error-Tolerant Interface (HETI) corresponds to a special kind of help system [START_REF] Rouse | Conceptual design of human error tolerant interface for complex engineering systems[END_REF][START_REF] Coonan | An error tolerant interface using operator and system models[END_REF]; one that is aimed at minimizing the consequences of certain human errors by keeping human operators alert in the face of an unexpected event. In order to be truly efficient, the HETI has to understand the human actions, and correct them in cases of error. It is why the preliminary analysis and modelling of the human errors is a very important step in the design of the so-called "human error tolerant interfaces". The model must be coherent with what the human operator has to do in summing the application.

This paper is composed of four parts. In the first part, the global principles of the HETI are defined. The second part explains preliminary experiments aimed at studying and modelling human errors that would be tolerated by the HETI to the greatest d egree possible. Based on the data obtained from the preliminary experiments, a HETI is described in the third part; of course, this HETI must be considered as a laboratory prototype, aimed at proving the feasability of such an approach. This HETI was designed using fuzzy logic, which is the most suitable artificial intelligence method for operator-activity modelling [START_REF] Rouse | Analysis and classification of human error[END_REF][START_REF] Cacciabue | A model of operator behaviour for manmachine system simulation[END_REF][START_REF] Shaw | Fuzzy model of a human control operator in a compensatory tracking loop[END_REF]. The main appeal of fuzzy-logic models is that they take into account the imprecisions and uncertainty of human judgement [START_REF] Zadeh | Fuzzy sets[END_REF][START_REF] Kaufmann | Introduction à la théorie des sous-ensembles flous -Tome I: éléments théoriques de base[END_REF][START_REF] Pedrycz | Fuzzy control and fuzzy systems[END_REF][START_REF] Yager | Essentials of fuzzy modeling and control[END_REF]. The evaluation of the HETI, tested within a laboratory (controlled) environment, is explained in the last part of this paper.

Human-error-tolerant interface (HETI): global principles

The development of interfaces that are tolerant of human errors is, in practice, based on preliminary studies of the kinds of errors that humans make in simulated and/or real conditions. In these studies, errors are identified by recording actions that result in the behaviour of the humanmachine system failing to meet well-defined criteria of productivity or safety. The idea is to use such studies to develop ways which, in the real world, will make it possible to replace, improve or negate inappropriate human actions [START_REF] Rouse | Conceptual design of human error tolerant interface for complex engineering systems[END_REF][START_REF] Hollnagel | The design of fault tolerant systems: prevention is better than cure[END_REF][START_REF] Hollnagel | Human reliability analysis, context and control[END_REF]Beka Be Nguema et al., 1993;[START_REF] Masson | Human errors learned from field study for the specification of an intelligent error prevention system[END_REF][START_REF] Masson | Prévention automatique des erreurs de routine[END_REF].

It should be noted that, from the theoretical viewpoint, these studies are based on task analysis, and the modelling of human tasks related to how people reason, leading to human error taxonomies [START_REF] Norman | Design rules based on analysis of human error[END_REF][START_REF] Leplat | Erreur humaine, fiabilité humaine dans le travail[END_REF][START_REF] Rasmussen | Information processing and human-machine interaction, an approach to cognitive engineering[END_REF][START_REF] Rasmussen | Coping with human errors through system design: implications for ecological interface design[END_REF][START_REF] Reason | Human Error[END_REF][START_REF] Senders | Human error, cause, prediction and reduction[END_REF].

There is no unique or unified architecture for a HETI to be found in the research literature. A possible architecture of such an interface could consist of three major modules; see Fig. 1.

-There is a decoding module, which translates the human actions (i.e., the input commands of the human operator) into data that the HETI can use.

-A second module first identifies the human actions in all control situations. It is based on: (1) a human actions model, which describes what the human operator can do in all possible control situations, and (2) a model of the industrial application, which describes what should be done by the human operator in all possible control situations. This second module can then correct the actions in the event of human error. In the research literature, the human actions model and the application model can be combined into a so-called human operator model. This paper uses this terminology (human operator model).

-A third module is concerned with presentation of information on a graphical screen. It has two roles: it presents the state of the process variables, according to different presentation modes (bargraphs, trends, mimic displays, and so on, as in classical control rooms; see [START_REF] Rasmussen | Information processing and human-machine interaction, an approach to cognitive engineering[END_REF][START_REF] Kolski | Interfaces homme-machine, application aux systèmes industriels complexes[END_REF], and it explains to the human operator the problems that the HETI has diagnosed and the advantages to be gained from its proposed intervention (feedback from the module #2).

The research described below is particularly focused on the second module (programmed using fuzzy logic). The usefulness of a HETI is undeniable, but it cannot always be applied. Such a system requires an extremely reliable model of the application, both of the possible situations and of the possible actions. The latter will not be exhaustive, because it will probably be based upon well defined fault conditions. But, in reality, totally new situations often arise. In the latter case, neither the HETI nor the human operator will have the necessary knowledge to cope with the problem on the basis of past experience, and this type of system will then not be effective. This is why a relatively simple process has been chosen to illustrate the features of the HETI (see the next section).

Preliminary experiments aimed at studying and modelling human errors

One of the aims of the HETI is to identify the human operator's action. In the event of human error in context of the application, that action must be corrected. Whatever the application, it is necessary to be aware of what errors the human operator is likely to commit. This is made possible by carrying out preliminary experiments, in a real context or by simulation, with operators, and by observing the errors that they commit during the performance of their process-control tasks. Without knowledge of the possible errors, it is impossible to design the HETI.

Such experiments are explained in this section. It begins by explaining the simulated industrial process, as well as the tasks that several dozens of subjects (acting as human operators) had to perform.

The human behaviours and errors indentified during these experiments are then described. They are the results of two types of experiment: in the first, the human operators had one main task to perform (called a "single" task, in the research literature according to experimental tradition); in the second, the operators had a far more complex task, in that two tasks were to be performed in parallel (a "double" task context; see, for example, [START_REF] Sperandio | Charge de travail et variation des modes opératoires[END_REF]), bringing about significant performance variations.

Human tasks considered

The goal of the preliminary experiments was to define the specification for the human operator model, to be integrated into the HETI. To achieve this, a study of human operator behaviour during the course of a simple simulated process was conducted, under various task configurations. Analysis of the experimental data allowed classification of the various kinds of behaviour, as well as the kinds of errors encountered in each task configuration. The task configurations used in the study are: presence of thermal inertia, presence of graphic deterioration, and double tasking (with two different tasks).

The experiments consider different types of human task, as follows.

-A manual task of temperature adjustment, in which the simulated industrial process is a quadruple heat exchanger. This process consists of a cooling system that takes hot water at a temperature (T 1e ), and flow rate (Q 1e ), and then cools it using cold water at a temperature (T 2e ) and flow rate (Q 2e ). The system is made up of four heat exchangers: e 1 , e 2 , e 3 and e 4 . These are connected in series on the hot-water circuit, and are fed cold water in parallel (see Fig. 2). Each exchanger is controlled by an up-flow dispenser, respectively named d 1 , d 2 , d 3 and d 4 , which sends cold water into the exchanger and redirects it into a secondary pipe when switched off. A similar dispenser, called d 0 , controls the hot water input in the cooling system. This redirects hot water into a secondary pipe when switched off, as would be the case in an emergency shutdown. -A second task consists of classifying a series of pentagons (Fig. 3). In this classification task, 36 randomly selected pentagons, of any size, are displayed, one by one on the screen. From these, 11 belong to the "very large" category; 8 belong to the "large" category; 5 belong to the "medium" category; 4 belong to the "very small" category. Each display comes with a multiplechoice question and a space where the operator enters a self-evaluation of the certainty on a scale of 0 to 1, where 0 indicates null certainty and 1 indicates complete certainty about this classification.

-These two tasks can be combined under a so-called "double task", including both the temperature adjustment task and the classification task.

CLASSIFY THE PENTAGON 1 0 certainty very large large medium small very small the pentagon is : In order to prove the feasibility of the HETI design, using the fuzzy-logic approach, two relatively simple tasks have been chosen. These tasks are not an accurate reflection of the many complex situations found in industry, and particularly in the control rooms of dynamic processes; thus, the results cannot be directly extrapolated to such complex processes. These tasks have been chosen because they allow the human behaviours and errors to be exhaustively identified during the preliminary experiments (this is very important in such exploratory researches) ; these tasks are also sufficient to overload the human operators, and thereby test their ultimate capabilities as regards error generation. For more complex processes in which the situations can prove to be too numerous to be studied in an exhaustive manner, it is a matter of studying whether it is possible to decompose the process into k simpler sub-systems. In that case, it then becomes possible to apply the same approach to one or several of these sub-systems, with the aim of designing a HETI. This is a research line in its own right which, to the authors' knowledge, has not been studied at international level.

First experiment conducted with 44 subjects (single task)

In the first experiment, conducted with 44 subjects (also called "human operators" in this section, even though the subjects are not real operators, but university students), the main human task consists of keeping the outlet temperature constant. First, each human operator (i.e., each subject) is instructed to aim for a temperature of between 20°C and 30°C in the outgoing hot water (T 1s ). To achieve this, the operator may adjust the cold-water flow from Q 2e in increments of 10 m 3 /s. The operator also has control over the on/off switches of the main hot water dispenser (d 0 ) and the individual heat exchangers. The operator is provided with continuous temperature and flow-rate readings from the hot water and cold water circuits, as shown in The test consists of twenty iterations, each lasting twenty seconds. The number of temperature adjustment sequences has been fixed at twenty to provide a more easily exploitable scaled results assessment. Operator-performance evaluation is accomplished by counting the number of acceptable temperature adjustments achieved by the operator over the course of the 20 sequences that the operator undergoes. An acceptable temperature adjustment is one where the desired final temperature (20°C-30°C) is achieved in less than 20 seconds, as shown in Fig. 5. Any sequence where the desired temperature range cannot be reached within 20 seconds, or where the d 0 dispenser is used to stop the temperature-adjustment sequence, is deemed unacceptable. The human operator is unaware of the 20s time limit. This limit was decided upon following test trials, done to validate the protocol, where 20 seconds was sufficient time for any operator to perform the task under normal operating conditions (to be defined later). However, the operator was asked, at the beginning of the test, to achieve the very best possible results. c) A stage where the above conditions deteriorate due to the addition of error-inducing, graphicdata alterations. The aim here was to bring the human operator to produce an error behaviour.

During both stages, the cold-water inlet temperature is 15°C. Hot-water inlet parameters are shown in Fig. 6. During the second stage, graphic data alterations P 1 , P 2 , P 3 and P 4 are introduced. d) A stage similar to stage b), with the addition of thermal inertia in the outgoing hot water (T 2e ). This inertia was selected so that temperature would seem to change slowly. The temperature variation delay may be adjusted according to the intended goal. The optimal value, obtained after preliminary testing, is 0.25 s/°C.

Second experiment conducted with 28 subjects (double task)

For this second experiment, 28 of the 44 subjects were available. In the second experiment, each operator is to undertake the following tasks, illustrated in Fig. 6: -One temperature-adjustment task, as described above, -One pentagon-classification task (Fig. 3). This involves classifying 36 pentagons (appearing one by one on the graphic screen) according to pre-existing templates, then self-evaluating the certainty of this classification on a scale of 0 to 1. This gives an evaluation of the degree of confidence of the human operator in performing the task. This pentagon-classification task is an often-used and well-documented study in the authors' laboratory [START_REF] El Mechrafi | Analysis of human decision making in interaction with a team-mate during a classification task[END_REF][START_REF] Benhalima | Experimental protocol design system based on objectoriented approach: application to graphical communication[END_REF][START_REF] Desombre | Graphic communication and human errors in a vibratory environment[END_REF][START_REF] Louas | Human behavior assessment method for a detection task performed in a stressful environment[END_REF]. It was selected for the present study to increase the complexity of the human task. Moreover, it uses very different assessment skills from those used in the first task. The human operator influences the process manually by clicking icons, and enters answers for the pentagon-classification task in the same way. This second experiment is divided into three stages: (1) a training stage for the pentagonclassification task, (2) the pentagon-classification task, (3) double tasking, induced by the addition of the pentagon-classification task to the temperature-adjustment task. In every classification task, 36 pentagons are displayed, one by one, on the screen. This number was selected so that the two different tasks would take the same time. -P 1 : all thermometer outlines disappear, and outgoing hot water thermometer starts to behave erratically, -P 2 : outgoing hot water thermometer disappears altogether, -P 3 : outgoing cold water thermometers start to behave erratically, -P 4 : outgoing cold water thermometers disappear altogether.

During the course of the pentagon-classification tasks, the pentagon display rate regularly increases so as to further complicate the task, as shown in Fig. 8. For the HETI design, only the unaltered temperature adjustment task with thermal inertia, and the double task were used. The reasons for this selection are that the presence of a thermal inertia is closer to real-life conditions, and that double-tasking allows mental overload simulation, at least for the latter part of the experiment.

Results

Each experiment starts after the subject has completed an anthropometric identification questionnaire. This lasts from 40 to 60 minutes, according to the time needed by each operator to become familiar with the process. During the temperature-adjustment stage, the data collected are the variations in temperature, flow-rate and dispenser status parameters over time. At the end of the experiment, subjects are required to fill out another questionnaire; this time concerning the operator's perceptions about the experiment, data deterioration, and whether any available means were left either unused or little used by the operator during the experiment.

The results were processed using classical descriptive statistical methods [START_REF] Bouroche | L'analyse des données[END_REF][START_REF] Grais | Méthodes statistiques[END_REF]. Cumulative curves, histograms and hierarchical classification were used in the process. These results are fully described in [START_REF] Beka Be Nguema | Comportement de l'opérateur humain face à une situation dégradée et imprévue: contribution à la réalisation d'une interface homme-machine tolérante aux erreurs humaines[END_REF]. Forty subjects underwent the temperature-adjustment task experiment without data deterioration but with thermal inertia added; of these, 28 also underwent the double-task experiment. Some facts could be noted following the experiments.

-Subject reaction times and the duration of the adjustment were both longer in the case of an unacceptable adjustment than in the case of an acceptable adjustment.

-Subjects either used every adjustment parameter available, or used only the cold-water flow-rate, in the temperature-adjustment task.

-Some errors were due to the operator's inability to estimate the limits of the acceptable temperature range, thus causing slight 'oversteering'.

-An analysis of the subjects' answers in the after-experiment questionnaire showed that the subjects took into account only the outlet parameter and the adjustment variables when conducting the task.

Subjects were classified according to their performance, which was defined as the number of acceptable adjustments achieved over the total adjustment sequences. Only one subject had a performance of less than 10/20 when doing every temperature-adjustment sequence. Most subjects had a performance over 12/20. Two subjects' strategies gave good results. The first one, used by all but one of the subjects, was to use every available parameter: only the cold-water inlet at first, then the dispensers as needed. Another strategy, used by the remaining subject (who was the exception), was to use only the cold-water inlet, even if two sequences were then impossible to achieve.

Three kinds of behaviours were encountered among the subjects: (1) the "high-risk" takers: these continued the task even when insufficient information was available, or when they did not use "upstream" information; (2) the "measured-risk" takers: these continued the task until a certain critical point (varying from one subject to another) was reached, and then preferred to stop the process; (3) the "no-risk" takers: these stopped the process by activating the emergency d 0 dispenser as soon as something was amiss, especially during the data-deterioration stage.

Four main kinds of errors were observed. These are, from the most frequent to the least frequent, as follows.

-Errors caused by lack of attention [START_REF] Reason | Human Error[END_REF]: when the operator used the emergency shutdown during the temperature-adjustment task without thermal inertia and without data deterioration.

-Intended "errors": these are due to the operator's lack of motivation which can be seen during non-critical sequences of the first stage. The subjects concerned do not admit to these errors, which are therefore difficult to analyse.

-Errors caused by la ack of understanding [START_REF] Reason | Human Error[END_REF], which are typical of the beginning of the temperature-adjustment task without thermal inertia and without data deterioration. Behaviour is hesitant. These could also be delayed lack-of-attention errors.

-Errors due to poor estimation of the results [START_REF] Leplat | Erreur humaine, fiabilité humaine dans le travail[END_REF]: these occur when the operators poorly estimate the outlet hot water temperature or the size of the pentagon.

These errors have been considered in the HETI design, which is the subject of the following section.

HETI Design based on the data obtained from the preliminary experiments

Functioning modes of the complete system, including the HETI

The system can work using any of the following modes, seen in Fig. 9: (1) In the "strictly automatic" mode, an automatic process-control system is implemented by the HETI when requested by the human operator. Actually, the process-control system is the fuzzy controller introduced earlier. The human operator has no further direct control over the process when using this mode. The strictly automatic mode could be useful to an inexperienced operator, by indicating the appropriate method of handling the process.

(2) The "strictly manual" mode can only be activated on a request from the human operator. It gives the human operator total control over the process. When this mode is activated, the HETI is prevented from interfering with the process.

(3) The "temporarily manual" or "normal" mode is the default functioning mode of the system. In this mode the system is controlled by the human operator, but the HETI is active.

(4) The "temporarily automatic" mode can only be activated by the HETI, following a human error. The HETI leaves this mode as soon as the process reaches a non-critical state. It uses the same fuzzy controller as the "strictly automatic" mode.

(5) The "transitory" modes are temporarily activated during the transition from the automatic to the manual mode, or vice versa (Fig. 9). Specific parameters for these modes will be explained in detail later. In addition to the above modes, a help (advisory) function was defined. The purposes of this function are (1) to warn the human operator that an error has probably been made and (2) to give advice on the correct course of action. These actions are the same as those that would be taken by the human-operator model if the HETI were active.

The "strict" modes are permanent modes, where the HETI has a passive role towards the operator, and cannot initiate any change of modes. The "normal" and "temporarily" modes allow the HETI to take an active role in the process.

A three-button menu, related to the functioning modes, was defined. This is accessible via the graphic screen by the human operator. The three buttons are called respectively: AUTO, MANU and HELP, as illustrated in Fig. 10. The AUTO and MANU buttons are mutually exclusive, i.e. selection of the AUTO button deactivates the MANU button, and vice versa. The HELP button works independently of the other buttons.

Structure of the HETI

The structure of the HETI is shown in Fig. 11. Throughout each task, the human operator has a number of options about the functioning modes of the system. Information about the state of the process is received, and the operator gets help, as needed, when the "help" mode is activated. The dotted lines indicate the information output that can be used by the human operator. The bold lines show the input and the main outputs of the HETI. Finally, the fine lines show the processes mode within the HETI.

In this structure, a human operator model (concerned with possible human actions) is used. This model (along with fuzzy logic and fuzzy problem solving) comprises the fuzzy controller defined in subsection 4.3. In the event of human error, the fuzzy controller is designed to match the best operator strategy, which is correct: an efficient action is then applied to the process.

Description of the human operator model

The temperature-adjusting operator performs the role of a temperature-control device which is responsible for keeping the hot water outlet temperature within a given range. Similarly, the fuzzy model of this human operator is the equivalent of a fuzzy controller. Fuzzy logic was selected for this model because it takes into account human imprecisions and uncertainty. Moreover, it allows for descriptive modelling of knowledge and behaviour. The model's role in the HETI is threefold. -To provide assistance to human operators in overload situations. In this case the model calculates the preferred course of action, which is then indicated to the human operator.

-To assume control of the process when the operator is overwhelmed.

The fuzzy-logic reasoning controller selected here is similar to that designed by [START_REF] Sugeno | Fuzzy control of a model car[END_REF]. It allows direct output of the defuzzified control. Weights (W i ) are attributed to each rule (i). These weights are obtained from the premise of each rule. Every rule is systematically applied and used for control calculations. Fuzzification was performed using the simpler trapezoid function, to begin with. The rules and the fuzzy sets were determined according to five linguistic values:

VN -> very negative N -> negative Z -> zero P -> positive VP -> very positive

The fuzzy rules were set using the best operator's strategy. This operator's actions were used as a model for high-performance process control. In an ideal HETI, other (non-optimal) operator models must also be taken into account. In this case this operator's actions were observed during the temperature-adjustment task with thermal inertia, but without data deterioration. Indeed, preliminary testing has shown that the shortest possible procedures would give the best errorcorrection results from the HETI. A study of the operator's strategy highlighted two primary, logical principles. Whenever the hot water outlet temperature became higher than 30°C or lower than 20°C, the operator acted upon the number of in-use dispensers. However, if the temperature stayed within the desired range, the operator acted upon the cold-water inlet flow rate, which allows easier temperature control. This operator's strategy led to the design of five fuzzy rules, to be described in detail later.

The controller is composed of two fuzzy motors and one "strategy-choice device" (OCS), as shown in Fig. 12, so as to use both temperature-adjustment strategies: (1) acting upon the dispensers, and (2) acting upon the cold-water inlet flow rate. The strategy-choice device compares the outgoing water temperature with a set value of 25°C, which corresponds to a mid-range temperature. This 25°C value was used for stabilising and optimising the temperature control. Each fuzzy motor receives the  E ~ fuzzy variables (variation of the error between the outgoing hot water temperature and the mid-range value of 25°C, over time) and E ~(error between the midrange value of 25°C and the outgoing hot water temperature of the process). However, only one of the motors select,ed by the OCS, does the controlling calculations. Motor #1 generates a flowvariation command, whereas Motor #2 generates a command to either add or remove a heatexchanger.

The fuzzification.

The fuzzification of error and error variation was found using the best operator's strategy. This was done according to the five linguistic values introduced earlier (VN, N, Z, P and VP), as shown in Figs 13 and 14. The Z linguistic value corresponds to a membership function where a 0°C gap between 25°C and the outgoing water temperature gives an ordinate of 1, figure 13. Fuzzification of error and error variation was found using the best operator's strategy. This was done according to the five linguistic values introduced earlier (VN, N, Z, P and VP), as shown in figures 13 and 14. The Z linguistic value corresponds to a membership function where a 0°C gap between 25°C and the outgoing water temperature gives an ordinate of 1, see Fig. 13.

0 1 +6 + 10 -6 -10 VN N Z P VP Membership function of error (e)
error, e (°C)

-4 +4 Fig. 13. Membership function for error. The "no error" category (Z) was widened in order to avoid wobbling within the imposed range.

In the case of a 2.5 °C/s thermal inertia, for instance, error variation (De) can really take only three values: -2.5°C/s, 0°C/s or +2.5°C/s, as shown in Fig. 14. These three values correspond to the possible rates of temperature variation within the hot-water outlet. A W i weight, which is independent of the AND and OR fuzzy operators, is given to each "number i" rule (from 1 to 5). Weight calculation allows an estimation of the ratios in which the commands of each rule must be applied. The relative importance of each weight is related to the state of the parameters within the process to be regulated. W i weight values are given by [START_REF] Guerra | Analyse de données objectivo-subjectives: approche par la théorie des sousensembles flous[END_REF]:

W i = OR(AND[ E j e  ), E k e   (1)
where l is an index that takes into account the number of entry combinations yielding the same Du command (a u command is a command that is acceptable to the operative part, from the fuzzy command); k are indices for the linguistic variables that are taken into account, and m X (x 0 ) is the membership function of the fuzzy value to the X fuzzy set.

The Min/Max logical functions are associated to the AND/OR functions:

AND(A, B) = Min(A, B) OR(A, B) = Max(A, B) .
The weighting formula (1) comes down to a "maximum of minima" calculation, and becomes:

e e E E j k       0 0 ,   W Max = ( ) Min i .
(2) 4.3.3 The defuzzification. The controller output is obtained after calculating the weights of each rule. This can be done in many ways. If command variables u i are set at the maximum of their linguistic values, two defuzzifications are possible [START_REF] Buckley | Expert fuzzy controller[END_REF]: linear and non-linear defuzzifications. Non-linear defuzzification was used here:

(3)

where n is the number of rules (five in this case) and u i are the maximum values of u for the flow rate and dispenser commands, see The arbitrarily set breakpoints are +1 and +2 for an increase in the number of available dispensers, and -1 and -2 for a drop in that number.

4.3.4

Performances obtained by the model. Several preliminary trials (without a human operator interacting with the HETI) have been performed in ways that validate the model technically. For the 20 temperature-adjustment sequences of the experimental protocol with thermal inertia, the model achieved the following performances:

-For a 20/20 regulation performance, 20 acceptable adjustments were made over the 20 adjustments that had to be done.

-During a change of input variables in the simulated process, the controller reacts with a lessthan-one second delay.

-On average, a controller needs 3 seconds to find the next adjustment during each sequence change. This corresponds to the controlling program's execution time. The very best human operator's execution time was 6 seconds on average, whereas the overall mean (including all subjects) was 12 seconds. The gap between the best operator's time and the controller execution time is due to the human operator's delayed reaction (of 4 seconds on average).

-The controller is stable throughout the 20 sequences.

-A compromise was found so as to let the strategy choice device (seen in the centre of Fig. 12) use the best operator's temperature-adjustment strategy while keeping the system stable.

This technical performance is promising, and allowed the possibility of evaluating the HETI approach with real human operators. Such an evaluation is the subject of the next section.

Evaluation in laboratory with five experts

An evaluation was done by five experts in human-machine systems (between 8 and 15 years of experience each). The experts were all familiar with the research work being performed and the means being implemented. They were aware that, in principle, their performances were likely to improve with appropriate use of the HETI, and that this was one of the aims of the research project.

The aim of the evaluation was threefold here.

-The first aim was to check that, in situations when the HETI was in fact used, the expert's performance improved. (It should be noted that this aim was not truly crucial, given that many tests had been carried out previously in the laboratory with subjects who had already taken part in the preliminary experiments. These tests had already shown significant performance improvements when the HETI was used. A mere confirmation of this improvement was therefore expected.)

-The second was to study the behaviour of the experts in relation to the HETI. They could possibly (unlike the subjects of the preliminary experiments) try to catch the HETI out, or competing with it, or choose not to trust it, etc., whilst at the same time, obviously, they were attempting to obtain the best performance possible.

-The third aim was to collect remarks and criticisms before and after the experiment, according to technical and ergonomic criteria. (This evaluation principle is often used in real and/or simulated cases. It is fully explained and discussed in different versions by various authors: [START_REF] Ombredane | L'analyse du travail[END_REF], [START_REF] Molich | Improving a human computer dialogue[END_REF], [START_REF] Nielsen | Usability engineering[END_REF], Wilson and Corlett (1996), and so on.)

The tasks performed by the five experts

The HETI structure that was evaluated is the same as that described above (Fig. 11), except that during the evaluation the expert selects the HETI option on the menu (Fig. 10) only if that expert considers it necessary. The HETI was automatically activated when no temperature adjustment could be achieved within 13 seconds. This HETI structure allows the avoidance of any errors that would cause an unacceptable temperature adjustment. The evaluation was performed using the double task, including the no-deterioration temperature-adjustment task with thermal inertia, and the pentagon-classification task. The task was done first without, and then with, the HETI. As in the above experiments, the experiment started with a training stage and simple tasks. In the doubletask stages, the experts were required to complete the pentagon classification as a priority. The results are detailed in Table 1.

Remarks concerning the behaviour o f the experts

When the HETI is in the "momentary" mode and the expert does not act upon the process, the resulting performance is 20/20. The same results are obtained in "help" mode when the expert applies the suggested command. Behavioural differences induced variations in the experts' results. Indeed, each expert selected the HETI mode according to his own self-confidence about the task. However, each individual sometimes overestimated his/her potential performance.

Main results of the evaluation

The experts' comments can be divided into three categories.

Comments on the tasks included the difficulty of visualising circuit input and output at the same time, the difficulty of integrating dispenser functions, insufficient training, and the lack of real-time constraints and performance indication, which would have been more challenging.

Comments on the execution of the tasks included the difficulty of finding a heuristic or a function rule that would allow prediction of, rather than mere reaction to, each change in the input/output, the impossibility of getting a global view of the process, the difficulty of keeping one's mind on the priorities (even though subjects were required to give priority to the pentagon-classification task) and too much reading in the "help" mode to be able to react quickly, increasing the mental overload with the speeding-up of the pentagons' appearance rate. Whenever help could be invoked, it was appreciated, especially in the "active help" mode, because this allowed a lightening of the workload, while leaving subjects with a sense of accomplishment. There were no comments or suggestions about this help mode, except about the information display. After the experiment, the experts were asked to complete a questionnaire and give suggestions about the experiment.

The experts considered the experiment from different points of view, according to their own fields. Their comments included synopsis of the ergonomy and the degree of difficulty of the tasks, and not always direct comments on the usefulness and potential of the HETI. The tasks were designed precisely in order to provoke mental overload, and this is why training was limited and more time was needed to evaluate and execute the tasks; thus the desired overloading effects were achieved. However, the experts did not really evaluate the actual usefulness of the HETI, which seemed obvious to them.

The experts had striking differences in behaviour, as indicated in Fig. 18. Some were so enthralled by the tasks that they disregarded the HETI in order to concentrate on their performance. Three of the experts did not even use the HETI, either because they presumed that they could do the task without help, or because they activated it too late, or because they preferred to concentrate, almost solely, upon pentagon classification. This indicates that experts, when faced with an unfamiliar process, have unpredictable behaviours. This was the converse of the beginner subjects, who based their actions solely on their training and the stimulation they received, thus following the commands more closely. The experts appeared not to follow a simple deductive reasoning process but to execute seemingly thoughtless actions, eventually losing track of the task at hand. Contrary to accepted ideas, the experts did not make any fewer errors than the beginners, except those caused by lack of knowledge [START_REF] Prümper | Some surprising differences between novice and expert errors in computerized office work[END_REF]. However, they spent less time correcting these errors, once discovered. At this point, the facts described above go beyond the bounds of automation, the field in which this study was done. Instead, they belong to cognitive psychology and ergonomics.

Conclusion

With this specific type of industrial process, the use of fuzzy logic for human operator modelling seems to be a promising choice. The fuzzy-logic operator model was designed using an analysis of the best operator's actions after preliminary experiments; it includes both general and fine tuning.

In the "strictly automatic" mode, the controller designed in this way carried out perfect adjustments in every operating configuration. In the "temporary" modes, the efficiency of the human operator using the HETI increases, thanks to the HETI.

The HETI is basically composed of a human operator model and a solution module. Its validation by experts in human-machine interaction has shown, however, that some experts have unpredictable attitudes towards the help system. Some do not activate the system, or activate it too late. Such attitudes may be related to each expert's own perceived capacity to take control over the process and are comparable to the "no-risk" and "high-risk" behaviours found during the primary stages of the experiment. This indicates a strong need for training and more adaptation time for most of the human operators using a HETI.

Several perspectives are opened by this preliminary study, concerning:

-the necessity to undertake other technical and ergonomic evaluations with experts and novice operators, with different types of industrial processes, -the analysis of a manual take-over after control of a process by the HETI, -the use of non-optimal human operator models for the making of the HETI, -the development of a module that is able to identify the intentions of a human operator during specific tasks.

Fig. 1 .

 1 Fig. 1. Global architecture of a Human-Error-Tolerant Interface (HETI)

Fig. 2 .

 2 Fig. 2. Diagram illustrating the industrial process. This diagram appears on a graphic screen in front of the subject. The upper left-hand part is where instructions are given to the subject. Temperatures are represented by bar graphs; flow rates are represented by dials. The upper left-hand area is used for a pentagon classification task.

Fig. 3 .

 3 Fig. 3. Screen display for a pentagon-classification task.

Fig

  Fig. 4. Experimental system.

  Fig. 5. Acceptable versus unacceptable temperature adjustments

  Fig. 6. Experimental system.

  Fig. 7. Test sequences, showing changes in temperature and flow rate on the incoming hot water line. P, P 2 , P 3 and P 4 mark the points at which the graphical display undergoes progressive deterioration (during the second of the two test phases only).-P 1 : all thermometer outlines disappear, and outgoing hot water thermometer starts to behave erratically, -P 2 : outgoing hot water thermometer disappears altogether, -P 3 : outgoing cold water thermometers start to behave erratically, -P 4 : outgoing cold water thermometers disappear altogether.

Fig. 8 .

 8 Fig.8. Progress of the experiment. Pentagon appearances are synchronised with the beginning of the sequences; one new pentagon for every two sequences at first, then one pentagon per sequence, then two, then four.

  Fig. 9. HETI functioning modes.

  Fig. 10. Mode selection (directly on the graphic screen in a specific zone)

Fig. 11 .

 11 Fig. 11. HETI internal structure. The grey lines show the functioning mode options that the human operator may take.The dotted lines indicate the information output that can be used by the human operator. The bold lines show the input and the main outputs of the HETI. Finally, the fine lines show the processes mode within the HETI.

-

  To provide training to inexperienced human operators. During training, the right actions are shown to the human operator by the model.

Fig. 12 .

 12 Fig. 12. Diagram of the principle used for the fuzzy regulation with strategy choice.

Fig. 14 .

 14 Fig. 14. Membership function of error variation De, m(De).

4. 3 . 2 Fig. 15 .

 3215 Fig. 15. Regulation rules matrix after adaptation. For example, the rule yielding a very positive Du command is the following: (if e is VN AND De is Z) OR (if e is VN AND De is N) THEN (Du is VP).

Figs

  Fig. 16. Membership function of Du for fuzzy motor #1. This fuzzy motor is used to control the inlet cold water flow. Control variables Du i are set at their maximum linguistic values. The arbitrarily set breakpoints are +20 and +30 for a flow-rate increase, and -20 and -30 for a flow-rate decrease.

cold water inlet hot water inlet hot water outlet cold water outlets

								op tion s o ffered to the
								op erator; o perato r
								enters ch oice b y mo use
								click in g on ico ns
					E 2	Q 2e	2e T
				d 1		d 2		d 3		d 4
	E 1	Q 1e	d 0	e 1		e 2		e 3		e 4	Q 1s	S 1
										10 0
										50
				Q 21 s	21 s T	Q 22 s	22 s T	Q 23 s	23 s T	Q 24 s	0 24 s T	S 2

Table 1 .

 1 Experts' results. The result of each classification is expressed as a "good classification" rate. The results of the adjustment task are expressed as the number of successful adjustments out of twenty. Expert #1 estimated that he didn't need help during the second double task. Expert #4 chose the "strictly automatic" mode during the second double task.

	Task	Expert	Ex pert 1	Ex pert 2	Ex pert 3	Ex pert 4	Ex pert 5
	Pen tag one classification	84 .3 3 %	84 .2 1 %	79 .9 2 %	84 .6 1 %	84 .6 1 %
	Temperature ad ju stement	12 /2 0	14 /2 0	19 /2 0	2/20	10 /2 0
	Doub le task	Pen tag one classification	81 .5 7 %	86 .8 4 %	84 .1 7 %	10 .2 5 %	92 .3 0 %
	witho ut HETI	Temperature adjustemen t	16 /2 0	15 /2 0	17 /2 0	3/20	13 /2 0
	Doub le task with HETI	Pen tag one classification Temperature adjustemen t	81 .5 7 % 14 /2 0	89 .4 7 % 17 /2 0	94 .8 7 % 14 /2 0	92 .3 0 % 20 /2 0	82 .0 5 % 15 /2 0
			Seco nd	Seco nd do uble task	Use o f the	Use o f the	"HELP" mod e d urin g
			do uble task	in "HELP" mode,	"HELP" mod e	"Urg ent stop "	the second d oub le
			in "strictly	then "HETI	at the end o f	du ring the	task ;
			manual"	activ ated " mod e	the second	simp le task and	HETI activated
			mode	when the pentago ne	do uble task	the first do uble	witho ut co ntin uity ,
				freq uence increases		task. "Strictly	then HETI
						automatic"	cou tino usly activated
						mode du ring	when the pentago ne
						the second	freq uence increases
						do uble task .

The correspondance should be adressed to Prof. Christophe KOLSKI, LAMIH -UMR CNRS 8530, Université de Valenciennes et du Hainaut-Cambrésis -Le Mont-Houy, B.P. 311, F-59304 Valenciennes Cedex, FRANCE, E-mail : christophe.kolski@univ-valenciennes.fr