Hanen Lejmi-Riahi

Mouna Belhaj

Lamjed Ben Said

Studying emotions at work using agent-based modeling and simulation

Keywords: Social simulation, Agent-based modeling, Emotional agents, Organization psychology, Workplace

Emotions in workplace is a topic that has increasingly attracted attention of both organizational practitioners and academics. This is due to the fundamental role emotions play in shaping human resources behaviors, performance, productivity, interpersonal relationships and engagement at work. In the current research, a computational social simulation approach is adopted to replicate and study the emotional experiences of employees in organizations. More specifically, an emotional agent-based model of an employee at work is proposed. The developed model is used in a computer simulator WEMOS (Workers EMotions in Organizations Simulator) to conduct certain analyzes in relation to the most likely emotions-evoking stimuli as well as the emotional content of several work-related stimuli. Simulation results can be employed to gain deeper understanding about emotions in the work life.

Introduction

Researches that deal with emotions in the workplace have recently received extensive attentions after a long period of neglect. This is due to the increasing awareness of organizational both theorists and practitioners during the last years of the crucial role that emotions play in the worklife. In fact, emotions shape employees behaviours, alter their well-being, their job-satisfaction, their motivation and their organizational commitment; they also impact their social relationships and their performance at work. Moreover, built upon findings from neuroscience, psychology and cognitive science, emotions have critical impacts on cognitive tasks that human resources practice during their daily work activities. These include decision-making, attention, memory, perception and learning. Consequently, new sub-domains related to organizational researches such as organizational psychology and organizational behaviour start to account for emotions, to explore and try to understand their nature, their causes and their roles in shaping human behavior within the organizational environment. Nevertheless, researches in these fields face some challenges due to the complex nature of emotions, to their unpredictability and to the difficult task to measure them. Almost of these researches use either self-reports or physiological indicators in order to identify emotions. While the former method provides data that can be biased due to the subjective nature of emotions and to their ephemeral occurrence, the latter is, generally, expensive and data are altered by the 'unusual' environment participants are placed in. In this context, social computer simulation can represent a reliable alternative tool to study and explore the emotional experiences of employees in the workplace. According to [START_REF] Davidsson | Agent based social simulation: A computer science view[END_REF], this type of simulation uses computers to study social systems behaviours. It can be useful when we try to study phenomena on which conducting real life experiments is impossible or phenomena which are difficult to get access to or to observe directly. In particular, the agent-based approach of social simulation is suitable to model and explore the human resources behaviour and specifically emotion dynamics at work. It offers a platform to conduct deep bottom-up understanding of the organization psychology from individual-level characteristics of its entities. In this way, researchers can capture and gain more insights about the complexity of the worklife and the human behaviour in organizational settings without going to the trouble of carrying out real life experiments. It is also worth to highlight that agent-based social simulation and modelling have been successfully used to model a large variety of complex systems in different fields such as social science, economics, ecology and political sciences.

The structure of the paper is organized into five main parts as follows. First, a literature review in relation to emotional agent-based models of human behavior in the workplace is exposed. Second, a presentation of the agent-based model of the emotional experiences of employees in the workplace is carried out. Third, an agent-based simulator WEMOS (Workers EMotions in Organization Simulator) is introduced. It simulates a workplace environment in which employees are subject to emotions-evoking stimuli. The simulation generates several results that are exposed. Finally, a discussion is elaborated about the simulation results and the different findings they highlight and a conclusion is also presented.

Emotional agent-based models of human behavior in the workplace

A wide range of agent-based models have been proposed in relation to the workplace context. They have been used to model, simulate and shed the light on several aspects underlying the behavior of employees in the workplace (e.g. [START_REF] Page | Agent-Based Modelling of Stress and Productivity Performance in the Workplace[END_REF] and [START_REF] Chae | Task difficulty and team diversity on team creativity: Multi-agent simulation approach[END_REF]). However, using this paradigm to explore the emotional experience of employees in the workplace has not been given great attention. That is, few publications that discuss the issue of using emotional agent-based modeling and simulation in the organizational context can be available in the literature. Moreover, even the works that can be found represent major limitations that are discussed below.

In the context of underground coalmine environment, an integrative behavior framework for mine workers is presented in [START_REF] Cai | Modelling and simulating of risk behaviours in virtual environments based on multi-agent and fuzzy logic[END_REF]. To create more believable and realistic behaviors of mine workers, physiological and psychological attributes such as emotion, personality and motivation are taken into consideration. An-other work that is related to health care environments, can also be found in [START_REF] Jain | Emotion enabled model for hospital medication administration[END_REF]. Agent-based emotional models of patients and nurses in a simulated hospital environment have been developed. The simulated system aims to examine the effects of negative emotional stress on the reasoning process of the nurses while performing patient medication administration tasks. Additionally, and in the military context, an emotion enabled architecture for situated agents DETT (Disposition, Emotion, Trigger, Tendency) has been presented [START_REF] Van Dyke Parunak | A model of emotions for situated agents[END_REF]. Emotions do influence the way the agent intention tendency is directed. The proposed model was used to simulate combat scenarios in the warfare.

The main problem of the aforementioned models is their restriction to specific organizational environments (e.g. underground coalmines, hospitals and military contexts). This may limit their genericity. In this context, other emotional agentbased models of employees in the workplace have been proposed independently of any specific organizational context. In [START_REF] Martínez-Miranda | Teaks: Simulation of human performance at work to support team configuration[END_REF], authors propose an agent-based simulator as a decision support tool used by managers in the workplace. Managers tune the settings of a team configuration system to optimize the team performance. Emotions are used, with other parameters, to determine every worker agent performance. In this work, the focus is set on only four emotions which are desire, interest, disgust and anxiety. In [START_REF] Oswaldo | Identifying emotion in organizational settings[END_REF], an emotional engine has been injected into a framework devoted to study social organizations. The aim is to provide a deeper understanding of the functioning of organizations. In fact, using the simulated system, actors within organizations are able to capture the elicited emotions and to cope appropriately with the emotional dimension of their behaviors. In this work, only social emotions of the actors, such as pride, gloating and guilt, are considered. In [START_REF] Riahi | Agent-based modeling and simulation of the emotional experiences of employees within organizations[END_REF], an agent-based model of emotions at work is presented. The research adopts a cross-disciplinary approach and takes advantage of theoretical foundations from organization behavior and organization psychology to elaborate upon the emotion generation process. The simulation model remains very limited as it addresses only joy emotion.

Agent-based model of employees emotions

The proposed agent model corresponds to an employee in the workplace with his associated emotion generation mechanism (Figure 1). The latter is based on the OCC model [START_REF] Ortony | The cognitive structure of emotions[END_REF] that represents a computation-oriented psychological theory of emotions.

Based on the OCC model, emotions are the result of appraisal of one of the three main facets of the world: events, actions of agents, or aspects of objects. This is regarding to the goals, standards, and attitudes of the agent [45,58]. The appraisal of events by respect to the agent goals leads to events-related emotions that include: well-being emotions (i.e. joy and distress), prospect-based emotions (i.e. hope, fear, satisfaction, fear-confirmed, relief and disappointment) and fortune of others emotions (i.e. happy-for, resentment, gloating and pity). The appraisal of actions of agents (oneself and others) by respect to the agent standards gives rise to actions-related emotions (also called attribution emo-Fig. 1. Emotional agent model of employee emotions generation in the workplace tions) that incorporate pride, shame, admiration and reproach. In some cases, an action of agent is also considered as an event that impacts the agent goals and his standards. The appraisal of such situation elicits compound emotions that contain gratification, remorse, gratitude and anger. Finally, the appraisal of aspects of objects by respect to the agent attitudes triggers objects-related emotions that unfold love and hate emotions. In this research work, we deal only with twenty over twenty two emotions of the OCC model as we take into consideration events-related emotions, actions-related emotions and compound emotions.

Following the presented OCC model structural components of emotions, it is necessary to identify the list of agent perceived events and actions of agents as well as his work-related goals and standards. This list is referred in the emotional agent model (Figure 1) as the agent mental states. It is also necessary to explain how the agent mental states correlate and how their corresponding relationships can be presented computationally. Furthermore, it is required to highlight how the emotion elicitation process with its sub-processes operate and use these identified OCC model-related components to generate emotions.

Agent mental states

In this work, mental states of the employee agent refer to its sets of beliefs, work-related goals and work-related standards.

Beliefs An employee agent is continuously perceiving its environment and the various happenings that may occur in the workplace. These input information are used to update the agent beliefs that correspond to its view about its external environment. In the current work, the agent set of beliefs is related to various emotional circumstances that may take place at work and may have implications on the agent emotional experiences. They include a range of emotions-inducing stimuli that can be of positive or negative valence. They refer to either emotional events or emotional actions of agents that may occur in the organizational context. These stimuli have impacts on the emotional states of employees agents.

To identify the various emotions-inducing stimuli that may occur in the organizational context, an investigation in organizational behavior and organizational psychology literature is carried out. The investigation focuses on works that deal with work-related emotional experiences. A relevant work in the area, that is proposed by [START_REF] Ohly | What makes us enthusiastic, angry, feeling at rest or worried? development and validation of an affective work events taxonomy using concept mapping methodology[END_REF] in [START_REF] Ohly | What makes us enthusiastic, angry, feeling at rest or worried? development and validation of an affective work events taxonomy using concept mapping methodology[END_REF], has been the source to inspire the current work. It introduces a taxonomy that was built upon diary studies sampled from 218 full-time employees of different industries and professions in Germany. This taxonomy is used to identify the list of the affective events and the agent actions that employees experience at work. Examples of emotions-inducing stimuli that can be retrieved from the taxonomy are "got a new job offer" which is an event and "my supervisor thanked me" which is rather an agent action.

Work-related goals and standards In the work settings, every employee has a set of goals that he tries to achieve and a set of standards that are applied to its conduct. Following the guidelines of the OCC model, when impacted by certain stimuli, these goals and standards engender emotions generation.

Accordingly, to conduct a study that deals with emotions in the workplace, we need to define the list of the employee work-related goals and his work-related standards. While the former are coupled with Events-related emotions, the latter are coupled with actions of agents to generate Actions-related emotions.

In this work, we are mainly inspired by [START_REF] Furnham | A cross-cultural investigation into the relationships between personality traits and work values[END_REF] and [START_REF] Meglino | Individual values in organizations: Concepts, controversies, and research[END_REF] in order to identify this list. These two works from the organization psychology field provide a list of work-related values which can be used to find out the worker goals and his standards. Indeed, person values can be classified into two sets. The first is called terminal and they refer to the desirable end-states and goals that a person would like to achieve. The second set is called instrumental values and they refer to the preferable behaviors and the acceptable modes of conduct [START_REF] Rokeach | The nature of human values[END_REF]. One example of work-related goals that are considered in the model is "Advancement and promotion". According to work-related standards, "Fairness" is one example of this list.

Computational representation of the OCC model components and their corresponding relationships

As earlier specified, emotions are generated by respect to the OCC model. That is, they are the result of an appraisal of events and agents actions regarding to its goals and standards. In this vein, it is important to map the emotionsevoking events that occur in the workplace to the employee agent work-related goals. This is to generate events-related emotions. Likewise, it is necessary to map the emotions-evoking actions of agents to the employee agent work-related standards. This is required to generate actions-related emotions. For compound emotions, the mapping can be implicitly deducted from the relationships that map certain stimuli (i.e. agent actions which are also considered as events) to both of the agent work-related goals and its standards.

To represent computationally the various aforementioned kinds of mappings, Fuzzy Cognitive Maps (FCM) are used. In brief, this technique is used to represent the fuzzy causal reasoning between concepts and to measure the strength of the impact among them. Thus, FCM can be employed to model the causal relationships that link events occurring in the organization environment to workrelated goals of employee agents and their actions to their work-related standards. Moreover, FCM is also used to measure the strength of impact among these various concepts as well as to present the qualitative description of their presence degrees. These information are essential to elaborate on the emotions generation process as explained later. Figure 2 represents one example of fuzzy cognitive maps that are used in the current work. It shows the relationships between the work-related goal Achievement and its associated impacting events that can occur in the workplace.

Fig. 2. Fuzzy cognitive map of Achievement goal and its impacting emotional events

Notably, to define the causal relationships among the FCM concepts as well as the weights of their interconnections, experts from organizational psychology are asked to participate in this task. Experts generated individual FCMs, with the same nodes but different concepts interconnections or/and different weights. Then, to get the final FCM, a combination algorithm that takes into account the credibility weight of each expert was applied from [START_REF] Stylios | Modeling complex systems using fuzzy cognitive maps[END_REF].

Emotion generation process

The emotion generation process of the employee agent is centered around three main components, namely appraisal derivation, emotion elicitation and emotion intensity calculation (Figure1).

Appraisal derivation It is used to evaluate the relevance of stimuli (i.e. events or agent actions) to the agent and to determine the values of appraisal variables. These latter are dimensions which are used to assess the situation. They describe the individual subjective relevance that is attributed to the emotions-evoking stimuli. Following the OCC model [START_REF] Ortony | The cognitive structure of emotions[END_REF] and inspired by the computational model of emotions that is proposed in [START_REF] Riahi | Computational models of immediate and expected emotions for emotional bdi agents[END_REF], several appraisal variables are involved in the emotion generation process of the employee agent. These appraisal variables are used later either to derive the emotion type (i.e. the emotion label such as joy) by the emotion elicitation model, to calculate the emotion intensity (i.e. the strength of the emotional response), or to perform both of them (Figure 1).

The FCM that is presented above is used by the appraisal derivation component. In fact, the two central appraisal variables (i.e. desirability of events and praiseworthiness of agent actions) are used to appraise triggered stimuli. These two appraisal variables are evaluated based on the FCM parameters. More specifically, fuzzy values of events occurrences and the weights that are associated between events and agents goals are used to calculate the desirability of events. Likewise, fuzzy values of actions occurrences and the weights that are associated between actions and agents standards are used to calculate the praiseworthiness of actions.

Emotion elicitation and emotion intensity calculation After appraisal derivation, the appraisal variables are used to elicit emotions and to calculate their intensities. The emotion elicitation (Figure 1) defines a list of rules that specify the conditions under which each emotion type is triggered and map each situation to the corresponding elicited emotion label. In this work, in order to define these elicitation rules and to identify the list of appraisal variables to account for, the OCC model has been used. According to the emotion intensities calculation (Figure 1), it is used to determine the strength level of the elicited emotion. A sub-set of the appraisal variables, as proposed in the OCC model, were employed to calculate the emotions intensities. Moreover, the methods of their calculations were based on a computational model of emotions that can be found in [START_REF] Riahi | Computational models of immediate and expected emotions for emotional bdi agents[END_REF].

Simulation and results analysis

During the simulation runs, different types of events and actions of agents take place in the organization environment and corresponding emotions are generated. These stimuli are triggered in line with a taxonomy that associates for each cluster of workplace events or actions a certain frequency [START_REF] Ohly | What makes us enthusiastic, angry, feeling at rest or worried? development and validation of an affective work events taxonomy using concept mapping methodology[END_REF]. The taxonomy data are built upon diary studies that are sampled from 218 employees of different sectors in Germany in 2015. In addition, several random variables are included in the simulation such as the fuzzy degrees of events occurrences as well as the degree of friendship between each pair of employee agents.

As the main objective of the current research is to gain deeper insights about emotions in the workplace, the developed simulator WEMOS provides several analyzes in relation to the emotions-evoking stimuli as well as the emotional content of these work-related stimuli. While the former aims to identify the most likely stimuli that are the causes of a specific emotion, the latter seeks to explore the most likely emotions that are the consequences of a given stimulus. WEMOS takes as input the mapping among triggered events and performed actions of agents on the one hand and the generated emotions on the other hand in order to produce these insights.

Emotions-evoking stimuli

This part of analyzes concerns the most likely stimuli that are the causes of a given emotion in the organizational context.

Events-related emotions-evoking events Employee agents may perceive several emotions-evoking events such as receive promotion or got good pay raise.

Figure 3 shows a representative repartition example of events-related emotionsevoking events. This repartition concerns the most likely events that may trigger well-being emotions (i.e. joy and distress) in the workplace. Results reveal that was-given freedom to make a decision and got good pay raise are the most likely events that are responsible of joy emotion. They also show that technical problem is the most likely event to produce distress emotion. Fig. 3. Well-being emotions-evoking events Attribution emotions-evoking actions of agents Appraisal of the employee agent own actions or the actions of its co-workers by respect to its standards generates Attribution emotions. Examples of emotional stimuli that trigger Attribution emotions may include assisted my supervisor and felt competent and receive thanks from my supervisor.

Running the simulation produced results that indicate that both of receive a praise and receive thanks from my supervisor are the most likely agent actions that invoke emotions of Pride and Admiration in the workplace (Figure 4). In addition, negative discussion with a co-worker, conflict with a co-worker and unjustified reproach are the main causes of Reproach and Shame emotions. Compound emotions-evoking stimuli Some happenings in the organization are recognized as events and actions of agents that are impacting simultaneously the agent goals and its standards. They generate Compound emotions. The simulation shows that meet the deadline, solve a technical problem and finish a work task are the most common sources of Gratification and Gratitude emotions (Figure 5). Results also highlight that had a phone call with a short-tempered customer is the principle cause of Anger and Remorse emotions.

Pride/Admiration

Stimuli emotional content

In this class of analyzes, the simulation reveals the most likely emotions that are generated as a consequence of a certain stimulus occurrence. In other words, more insights can be gained in terms of the emotional content of a given workplace happening. Figure 6 is a representative example of results that are produced by the simulation. It represents the repartition of the most common emotions that are generated when an event that belongs to the cluster Managerial and internal problems, organizational climate takes place. Was overqualified for an assigned task is an event that belongs to this mentioned cluster. Results demonstrate that Shame, Reproach, Remorse and Anger are the most likely emotions that are triggered following a stimulus that is included in the cluster Managerial and internal problems, organizational climate occurs.

Discussion and conclusion

Simulation results that are related to emotions-evoking stimuli are of great importance for managers and supervisors. Indeed, through the insights they provide, strategy-makers in organizations can act appropriately to prevent or foster the occurrence of specific emotions. For instance, simulation results reveal that, while traditional financial-oriented rewards and recognition that are based on promotion and good pay and benefits are important, they are not more frequent and more related to positive emotional experiences (e.g. Joy, Hope and Satisfaction) than psychological-oriented rewards such as giving subordinates the freedom to make decisions and involving them in problem solving. In fact, according to the simulation results, the emotion-evoking event was given freedom to make decision is more likely to produce Joy emotion than got good pay raise or receive promotion. Moreover, results also highlight the relevance of recognition and giving thanks to subordinates in order to induce Pride and Admiration emotions. As such, a supervisor must find time to thank his team members for each job well done. This may make them feel valuated, boost their motivation and inspire them to do their best.

Notably, the simulation results that are linked to the stimuli emotional content are in line with other empirical studies that were conducted in other disciplines especially in organizational psychology and management. Indeed, a relevant research study that was published in [START_REF] Grandey | Affective states and traits in the workplace: Diary and survey data from young workers[END_REF] and was cited by 346 research papers is able to establish this agreement. In this work, authors found that Interpersonal mistreatment from customers is the most frequent source of Anger and Recognition from supervisors for work performance is the principle cause of Pride. These findings are in accordance with WEMOS simulation results. Indeed, according to the statistical insights it provides, Anger is most often to be felt when had a phone call with a short-tempered customer, which is a representative example of the cluster Problems in interactions with clients or patients, occurs. In addition, Pride is most likely to be generated as a consequence of receive a praise or receive thanks from my supervisor Another correspondence between the current work findings and other relevant work in organization psychology [START_REF] Basch | Affective events-emotions matrix: A classification of work events and associated emotions[END_REF] was also recognized. The research that is proposed in [START_REF] Basch | Affective events-emotions matrix: A classification of work events and associated emotions[END_REF] is among the few research papers that have investigated the mapping between work-related emotional events and the most likely associated experienced emotions. It is cited by 368 research papers. According to the authors, negative acts of management lead to Anger emotions in 16% of cases. In this work, 17% of cases Anger are experienced following the occurrence of an event that belongs to the cluster Managerial and internal problems, organizational climate. In addition, in [START_REF] Basch | Affective events-emotions matrix: A classification of work events and associated emotions[END_REF], personal problems engender Sadness emotion in 100% of cases. In the current study, Health problems and private issues lead to Sadness (i.e. Distress), Fear and Pity emotions. While the two works are congruent in relation to Sadness emotion, they differ in Fear and Pity. This is due to the fact that the current work deals with social emotions such as Pity and Happy-for which are not considered in [START_REF] Basch | Affective events-emotions matrix: A classification of work events and associated emotions[END_REF]. Besides, we also take into consideration the occurrence of some unconfirmed events that are related to Health problems and private issues. This triggers emotions such as Fear.

To conclude, an agent-based simulator WEMOS of employees emotions in the workplace is developed. This social simulator provides analyzes that explore the experience of emotions in organizations. Based on the above discussion, WEMOS offers several results that can help underpin our understanding about emotions at work. In particular, managers, supervisors and strategy makers can take advantage of WEMOS to foster their awareness in relation to the most-likely events that may engender specific emotion and the most-likely emotion that can be generated given a certain circumstance. They can consequently operate in descriptive and predictive modes in organizations. Finally, WEMOS can be extended by considering the emotional contagion mechanism that is able to spread an integrative positive or negative emotional climate among co-workers. It is also possible to incorporate mood and personality that have significant impact on human behavior in the workplace.


Tangyike Zhang

Songyi Zhang
email: zhangsongyi@stu.xjtu.edu.cn

Yu Chen

Chao Xia

Shitao Chen
email: chenshitao@stu.xjtu.edu.cn

Nanning Zheng
email: nnzheng@mail.xjtu.edu.cn

Mixture Modules Based Intelligent Control System for Autonomous Driving

Keywords: Mixture modules, Control system, Path tracking, Ride comfort

As a typical artificial intelligence system, a safe and comfortable control system is essential for self-driving vehicles to have the same level of driving ability as human drivers. This paper proposes a novel control system for autonomous driving vehicles based on mixture modules, which aims to ensure the accuracy of path tracking while meeting the requirements of safety and ride comfort. The mixture modules consist of a lateral controller to control the steering wheel angle of the vehicle for path tracking and a longitudinal controller to adjust the speed of the vehicle. We conducted a series of experiments on our simulation platform and real self-driving vehicles to test the proposed control system and compared it with the traditional methods which are widely used. The experimental results indicate that our control system can run effectively on real vehicles. It may accurately track the intended driving path and adjust the driving speed comfortably and smoothly, which demonstrates a high level of intelligence.

Introduction

Generally, an autonomous driving system can be divided into three subsystems: sensing, planning and control [START_REF] Thrun | Stanley: The robot that won the darpa grand challenge[END_REF] [START_REF] Urmson | Autonomous driving in urban environments: Boss and the urban challenge[END_REF]. The control system needs to control the actuators such as throttle, brake and steering wheel according to the smooth and drivable path generated by the planning system. The efficiency of control will directly determine the safety and comfort of the ride [START_REF] Hemami | On the steering control of automated vehicles[END_REF]. With the advancement of the commercialization of self-driving technology, people have put forward higher requirements for the safety and ride comfort of self-driving vehicles [START_REF] Watzenig | Automated driving: safer and more efficient future driving[END_REF]. However, autonomous vehicles are usually faced with complex road scenarios. The control efficiency is affected by numerous factors, including the vehicle dynamics, the change of path curvature and the road condition. Therefore, the design of control system is a key and challenging problem for self-driving technology.

Traditional control system typically consists of a lateral controller and a longitudinal controller [START_REF] Jo | Development of autonomous carpart ii: A case study on the implementation of an autonomous driving system based on distributed architecture[END_REF] [START_REF] Xu | Integrated longitudinal and lateral control for kuafu-ii autonomous vehicle[END_REF]. The lateral controller is used to calculate the steering wheel angle of the vehicle to achieve path tracking, and the longitudinal controller is used to control the throttle and brake pedal percentage to control the speed of vehicle [START_REF] Cai | An intelligent longitudinal controller for application in semiautonomous vehicles[END_REF]. Although the traditional control method can provide a well solution to the problem of tracking accuracy, it is often difficult to adjust the ride comfort. In addition, traditional methods tend to have poor control performance at a high speed. An important reason is that the vehicle models used in traditional methods ignore the influence of longitudinal speed on lateral control. While designing the control system, only by considering such kind of influence can the control system have good performance at different speeds.

Therefore, we make great efforts in tuning proper look-ahead distance and designing a vehicle model to improve traditional methods. Since the control performance is determined by the performance of modules, a better control strategy is automatically selected. In order to enable the vehicle to track the target path quickly and accurately, we propose a nonlinear lateral controller. It can fit a quintic spline curve that starts from the current pose of vehicle and converges to the trajectory to be tracked. The goal point of quintic spline curve is determined by the look-ahead distance. After generating the quintic spline curve, the required curvature towards the target point can be determined for the vehicle. Then the corresponding steering wheel angle can be calculated according to the curvature and the current speed. The quintic spline curve is updated in real time according to a certain frequency, which provides a new reference for the lateral controller.

In self-driving systems, the speed commands generated by planning system may not be smooth and continuous. If such speed commands are directly executed without processing, it will cause sharp changes to the output of longitudinal controller, which affects the ride comfort. Therefore, when designing longitudinal controller, we first determine the constraint of speed command through passenger's feedback on the ride comfort. After the speed command is input to the longitudinal controller, the speed command will be processed according to the constraint condition, and then executed by the controller. Different control strategies are adopted for different dynamic characteristics of the throttle pedal and brake pedal, and the throttle or brake commands that meets the comfort requirement is obtained.

Compared with traditional system, our control system is easy to implement, and it can accurately track the target path while controlling the speed comfortably and smoothly. The "Pioneer" self-driving vehicle, which is equipped with the proposed control system, had participated in the 10th IVFC [START_REF] Xin | China future challenge: Beyond the intelligent vehicle[END_REF] and won the first place in the total score.

In this paper, firstly we review some relevant research on self-driving vehicle control systems, then introduce our proposed control system and describe the specific details in its implementation. Following that, the performance of control system is verified through the simulated experiments and the experiments on "Pioneer" self-driving experimental platform, indicating that our control system can work effectively on real vehicles. A comparison with the widely used traditional methods is presented as well. Finally, we summarize our contributions and future work. Block diagram of the proposed control system. Our control system includes a lateral controller and a longitudinal controller. The lateral controller receives the target path and current pose of vehicle, converts the path to vehicle coordinate system firstly, then generates a quintic spline curve in real time, and finally calculates a steering wheel angle command by vehicle model. The longitudinal controller smoothes the speed command, then adopts different control strategies for throttle and brake pedal.

Related Work

A great deal of explorations has been conducted in the construction of selfdriving control systems. In aim of solving path tracking problems, which is a core issue of control system, there are quite a few available ways including geometric controllers [START_REF] Coulter | Implementation of the pure pursuit path tracking algorithm[END_REF][START_REF] Girbés | Continuous-curvature kinematic control for path following problems[END_REF][START_REF] Rossetter | A potential field framework for active vehicle lanekeeping assistance[END_REF], dynamic controllers [START_REF] Rossetter | A potential field framework for active vehicle lanekeeping assistance[END_REF], optimization controllers [START_REF] Kang | Design and testing of a controller for autonomous vehicle path tracking using gps/ins sensors[END_REF], model predictive controllers [START_REF] Yu | Mpc-based regional path tracking controller design for autonomous ground vehicles[END_REF], and adaptive controllers [START_REF] Hessburg | Fuzzy logic control for lateral vehicle guidance[END_REF]. The geometric controller is relatively simple to configure and implement while showing strong robustness in most driving scenarios, thus being widely used. One of the most classic geometric controller is based on the pure pursuit algorithm [6][19] for calculating the steering wheel angle according to the circular arc. Many selfdriving platforms, such as Autoware [START_REF] Kato | An open approach to autonomous vehicles[END_REF], integrate the pure pursuit algorithm or its modified versions in the control system. "Stanley Method" [START_REF] Thrun | Stanley: The robot that won the darpa grand challenge[END_REF] is based on the vehicle kinematics model, considering the cross track error and orientation error. Filho et al. [START_REF] Massera Filho | Longitudinal and lateral control for autonomous ground vehicles[END_REF] proposed a geometric controller based on the cubic Bezier curve, however, in his work, the author did not take much into consideration the influence of tire slip, and the cubic Bezier curve only guaranteed the continuity of orientation but that of curvature, so the curve does not exactly match the actual travel trajectory of the vehicle. Amidi [START_REF] Amidi | Integrated mobile robot control[END_REF] proposed a geometry controller based on a quintic curve that reduces the discontinuous motion of the steering wheel. Dominguez [START_REF] Dominguez | Comparison of lateral controllers for autonomous vehicle: Experimental results[END_REF] analyzed the performance of several geometric controllers and performed several comparative experiments.

In determining the geometric controller's look-ahead distance, a traditional method is to set the look-ahead distance as constant or a linear or quadratic function of speed, but usually fails to make the controller maintain an optimal performance. The core issue is that there is no considering the trend of curvature change of the forward path segment, making it unable to adjust the look-ahead distance in time while driving. Moreover, the relative position and orientation of the vehicle and the desired path are not considered. Ollero [START_REF] Ollero | Stability analysis of mobile robot path tracking[END_REF] analyzed the effect of look-ahead distance on stability in the pure pursuit algorithm. Chen et al. [START_REF] Chen | A robust look-ahead distance tuning strategy for the geometric path tracking controllers[END_REF] proposed a robust fuzzy logic-based look-ahead distance tuning strategy, which can operate by providing fewer related variables, but this method cannot guarantee the continuity of control thus bringing certain limitations in the real car applications.

In the construction of self-driving integrated control system, Xu et al. [START_REF] Xu | Integrated longitudinal and lateral control for kuafu-ii autonomous vehicle[END_REF] proposed a design scheme for self-driving vehicle control system, which integrates multiple adaptive and robust algorithms to improve the compatibility of control system. Kang et al. [START_REF] Kang | Design and testing of a controller for autonomous vehicle path tracking using gps/ins sensors[END_REF] improves the lateral safety of vehicle by combining the steering controller with a speed controller that maintains the lateral acceleration limit.

Lateral and Longitudinal Controller

Lateral Controller

In our control system, we define the lateral controller as a module that calculates steering wheel angle commands for a self-driving vehicle to keep on the target path based on the path information and the current state of vehicle (see Fig. 1), where the target path is defined as a set of ordered points. Each point contains the information about position, orientation and curvature.

Vehicle Model In our lateral controller, how to map the driving trajectory to the steering wheel angle of vehicle is a key issue. For easy implementation, the model should not be designed to be too complex. Thus, we apply the Ackermann model [START_REF] Mitchell | Analysis of ackermann steering geometry[END_REF] and its simplified version, the bicycle model [START_REF] Amer | Modelling and control strategies in path tracking control for autonomous ground vehicles: a review of state of the art and challenges[END_REF] (shown in Fig. 2). The Ackermann model is the most commonly used kinematic model in vehicle modeling, the centers of curvature circles that correspond to four wheels intersect at point O. The bicycle model is an effective way to simplify the Ackermann model, and its simplification is based on following assumptions: 1. Ignore the movement of vehicle in the vertical direction. 2. The front and rear tires of vehicle can be described by one tire each, and the center points of the front and rear wheels are the same respectively.

3. The rear wheel cannot deflects, the control input on steering wheel is mapped to the steering of front wheel, and the proportionality of the steering wheel angle and the front wheel angle of the vehicle is fixed.

According to the bicycle model, we can get relationship between the steering wheel angle and the curvature of driving path:

θ steering = k sf tan -1 ( L R ) = k sf tan -1 (Lγ), ( 1
)
where L is the wheelbase of vehicle, R is the radius of driving track, γ is the curvature of driving track, and k sf is the steering ratio coefficient from steering wheel angle to front wheel angle.

However, the bicycle model does not take into account the effects of lateral forces of wheels at different speeds. Therefore, when driving at high speeds, the slip of tire is increased, making control performance worse than normal conditions. So we introduce a curvature compensation coefficient, thus the relationship between the obtained steering wheel angle and curvature of the traveling track is calculated as follows:

θ steering = k sf tan -1 ((L + k γ v 2 c )γ), (2)
where k γ is the curvature compensation coefficient that is determined empirically, and v c is current speed of vehicle.

Tuning Look-ahead Distance The look-ahead maneuver simulates driving behavior of the human driver, which enables self-driving vehicle to calculate steering wheel angle commands in advance based on road information ahead, thereby improving the stability of control system. Our controller uses a singlepoint look-ahead model and the look-ahead point is determined by look-ahead distance l. Similar to most look-ahead-based algorithms, the look-ahead distance selection in our controller also has a large impact on control performance. In our lateral controller, the look-ahead distance is calculated as follows:

l = tanh(γavg(kvc+l0)) γavg
+ k e * cte, γ avg > 10 -4

(kv c + l 0 ) -γ 2 avg (kvc+l0) 3 3.0 + γ 4 avg (kvc+l0) 5 15.0

+ k e * cte, else , (3)
where v c is the current speed of vehicle, cte is the cross track error of vehicle, and γ avg is the average curvature of path within original look-ahead distance kv c + l 0 . When calculating look-ahead distance, we take multiple factors into account, such as current speed of vehicle, curvature of path in the front, and current cross track error.

Quintic Spline Curve Fitting In order to ensure the continuous movement of steering wheel when the vehicle tracks the path, we adopt a method of generating a quantic spline curve that converges from current state [x current , y current , θ current , γ current ] T to the state of the look-ahead point [x lookahead , y lookahead , θ lookahead , γ lookahead ] T on the target path. Compared to circular arcs and cubic Bezier curves, the curvature of quintic spline curve is always continuous, thus improving the stability of steering wheel.

Target Path

Look-ahead Point Fig. 3. Quintic spline curve generated by lateral controller, which converges from current state of vehicle to the state of look-ahead point on the path.

In the vehicle coordinate system, the current state of vehicle is [0, 0, 0, tanδc L+kγ v 2 c ] T , where δ c is the front wheel angle of vehicle calculated by

θsteering k sf
. And the state of look-ahead point is [x l , y l , θ l , γ l ] T , where x l , y l , θ l , γ l are the values of the position, orientation and curvature of the look-ahead point converted to vehicle coordinate system. According to the work [START_REF] Amidi | Integrated mobile robot control[END_REF], in solving the polynomial, we need to assume that the absolute value of θ l is less than π/2, and we can describe x as a quintic polynomial about y:

x(y) = a 0 + a 1 y + a 2 y 2 + a 3 y 3 + a 4 y 4 + a 5 y 5 .

(4)

According to the constraints of x(y), ẋ(y) and ẍ(y) on current state and the state of look-ahead points, the quantic polynomial can be solved as follows [START_REF] Amidi | Integrated mobile robot control[END_REF]:

x(y) = a 2 y 2 +(10k 1 -4k 2 + k 3 2 )y 3 +(-15k 1 +7k 2 -k 3 )y 4 +(6k 1 -3k 2 + k 3 2 )y 5 , ( 5
)
where

a 2 = -tanδc 2(L+kγ v 2 c ) ,k 1 = x l + tanδc 2(L+kγ v 2 c ) , k 2 = -tanθ l + tanδc L+kγ v 2 c , k 3 = -γ l sec 3 θ l + tanδc L+kγ v 2 c
. The solved quintic spline curve is as shown in Fig. 3.

Calculate the Steering Wheel Angle

After getting the quantic spline curve, the ideal driving curvature of vehicle needs to be determined. The curvature corresponding to a position in front of current pose is used as the driving curvature, so that the problem caused by mechanism delay can be solved. Then according to Eq. 2, we can get the steering wheel angle command.

Longitudinal Controller

The structure of our longitudinal controller is shown in Fig. 1. In our control system, the longitudinal controller is a module to receive speed commands sent by planning system and based on real-time state of vehicle, calculate throttle pedal percentage and brake pedal percentage and send them to vehicle in order to make the speed gradually approaches the required value with high ride comfort. The work [START_REF] Huang | Fundamental study of jerk: evaluation of shift quality and ride comfort[END_REF] pointed out that ride comfort has a strong correlation with acceleration and jerk. However, in experiments, we found that as long as the longitudinal acceleration and jerk are within a certain range, the ride comfort in most cases can be guaranteed. Therefore, we first preprocess the input raw speed command v cmd to get a new speed command v comf ort that meets the acceleration and jerk constraints:

v comf ort = a comf ort dt, (6)
where

a comf ort =    a ul , j comf ort dt > a ul a ll , j comf ort dt < a ll j comf ort dt, else , (7)
j comf ort =    j ul , vcmd > j ul j ll , vcmd < j ll vcmd , else , (8)
where a ul , a ll is the upper and lower limits of acceleration, j ul , j ll is the upper and lower limits of jerk, and the principle of selecting constraints is to take passenger's ride comfort requirements and controller's dynamic performance into account, in the meanwhile, to ensure safety. In practical applications, the constraints can also be flexibly switched according to driving scenarios. Since it is not allowed to simultaneously depress the throttle pedal and brake pedal, after inputting a comf ort as an input signal to the controller, we need to calculate e speed = v comf ort -v c . If e speed is greater than zero, the throttle sub-controller is used to calculate throttle pedal percentage, and the brake pedal percentage is not calculated. If e speed is less than zero, the brake sub-controller is used to calculate brake pedal percentage, and the throttle pedal percentage is not calculated. The resulting throttle pedal percentage and brake pedal percentage output are as follows:

u throttle = K pt ((K a e speed -a c ) + Kae speed -ac Tit dt), e speed > 0 0, else , (9)
u brake = 0, e speed > 0 K pb ((K T e speed -T c ) + K T e speed -Tc T ib dt), else , (10)
where K pt , K pb , T it and T ib are the parameters of PI controllers. a c is the current acceleration of vehicle, and T c is the current braking torque of vehicle.

4 Experiments and Results Our real car experimental platform "Pioneer" is shown in Fig. 4(a). "Pioneer" is based on 2017 Lincoln MKZ hybrid model. It is equipped with a drive-bywire system, communicates with an IPC through the CAN bus. Developers can use the IPC to send control commands to control the throttle pedal percentage, brake pedal percentage, steering wheel angle and gear status, while receiving the information of vehicle in real time. Our experimental platform is equipped with a high-precision integrated navigation system for accurate position estimation. The software platform is developed based on ROS (Robot Operating System). Specific parameters of our vehicle are as shown in Table 1.

Besides, we also conducted experiments in a simulation platform [START_REF] Chen | Autonomous vehicle testing and validation platform: Integrated simulation system with hardware in the loop[END_REF]. The simulation platform includes a simulated vehicle model and simulated roads to maximize the reality of real vehicle motion, as shown in Fig. 4(b). Similar to controlling a real vehicle, we can also control the driving of the simulated vehicle through the control system to achieve safe and efficient algorithm testing.

Path Tracking Experiment

In the path tracking experiment, we conducted a series of comparative experiments to verify the effectiveness of our proposed lateral control method. The experimental path on campus is shown in Fig. 5, suitable for testing the performance of controllers in curvature-changing scenarios. We chose two widely-used path tracking algorithms, Pure Pursuit [START_REF] Coulter | Implementation of the pure pursuit path tracking algorithm[END_REF] and Stanley Method [START_REF] Thrun | Stanley: The robot that won the darpa grand challenge[END_REF] to compare with our methods by controlling the real car running the entire road, the cross track error and orientation error box-plot of different methods and the excerpt of steering wheel angle while crossing U-turn are shown in Fig. 6 and Fig. 7. It can be found that when using our method, the distribution of cross track error and orientation error are more concentrated. In addition, our method produces less jitter, and the actual performance of the steering wheel movement is closer to that of a human driver. On the simulation platform, we also carried out the same comparative experiments. The experimental results are shown in Fig. 8 and Fig. 9. Similar to the real vehicle experiment results, our approach has the best performance.

Speed Control Experiment

The speed control experiment on real car is carried out on a straight two-lane road, the planning system sends changing speed commands to the longitudinal controller. The experimental results in the realistic environment are as shown in Fig. 10. Although the speed command issued by the planning system is drastically changing, the longitudinal controller can guide the vehicle to smoothly reach the reference speed. In the meanwhile, there is no significant overshoot in the process of controlling the speed. Similarly, the experimental results in simulation are shown in Fig. 11.

Conclusion

This paper proposes a mixture modules based self-driving control system that guarantees both accuracy and ride comfort, which includes a lateral controller that controls the steering wheel angle and a longitudinal controller that controls the vehicle speed. By improving the vehicle model and look-ahead distance tuning strategy, as well as implementing a geometric controller based on quintic spline curve, smoother output and better control effect have been achieved. In this paper, the implementation details of lateral controller and longitudinal controller are introduced, and the experimental verification is carried out both in realistic environment and in simulation. The experiment results that the performance of our control system is superior. In the future work, we plan to further improve the generality and robustness of the controller to better adapt to the requirement of different vehicles and different scenarios.

  Fig.1. Block diagram of the proposed control system. Our control system includes a lateral controller and a longitudinal controller. The lateral controller receives the target path and current pose of vehicle, converts the path to vehicle coordinate system firstly, then generates a quintic spline curve in real time, and finally calculates a steering wheel angle command by vehicle model. The longitudinal controller smoothes the speed command, then adopts different control strategies for throttle and brake pedal.

Fig. 2 .

 2 Fig. 2. Ackermann model and bicycle model. Bicycle model simplifies the four wheels of ackermann model into two wheels, centered on the front and rear axles of vehicle.

Fig. 4 .

 4 Fig. 4. (a) The self-driving experimental platform "Pioneer", based on 2017 Lincoln MKZ hybrid model. (b) The simulation platform based on Gazebo. Various experimental scenarios can be built in this simulation platform.

Fig. 5 .Fig. 6 .

 56 Fig. 5. Target path in path tracking experiment. The vehicle needs to track the path on which there are scenarios that require maneuvers including straight-line driving, quarter-turn, U-turn, etc..

Fig. 7 .Fig. 8 .

 78 Fig. 7. Excerpt of steering wheel angle while crossing U-turn in real environment.

Fig. 9 .

 9 Fig. 9. Excerpt of steering wheel angle while crossing U-turn in simulation.

Fig. 10 .

 10 Fig. 10. Experimental results of speed control in realistic environment.

Fig. 11 .

 11 Fig. 11. Experimental results of speed control in simulation.

Table 1 .

 1 Vehicle parameters.

	Parameter	Value
	Maximum horsepower	253P s
	Maximum brake torque	3400N m
	Range of steering wheel angle output	-470 • to 470 •
	Resolution of Steering wheel angle output	0.1 •
	Maximum rotation speed of Steering wheel	500 • /s
	Steering ratio	14.8 : 1
	Vehicle wheelbase	2850mm

This work was supported by the National Natural Science Foundation of China(NO.61773312,61790563).


Jiří Martínek

Ladislav Lenc
email: llenc@kiv.zcu.cz

Pavel Král

Training Strategies for OCR Systems for Historical Documents

Keywords: CNN, Historical Documents, LSTM, Neural Network, OCR, Synthetic Data

This paper presents an overview of training strategies for optical character recognition of historical documents. The main issue is the lack of the annotated data and its quality. We summarize several ways of synthetic data preparation. The main goal of this paper is to show and compare possibilities how to train a convolutional recurrent neural network classifier using the synthetic data and its combination with a real annotated dataset.

Introduction

The efforts to preserve the cultural heritage of historical documents has become a significant task in the document processing field. A reasonable information retrieval on scanned documents is possible only after the text recognition process though. Optical character recognition (OCR) and handwritten text recognition (HTR) need to be implemented with a respect to the historical domain. In such a domain we often struggle with the quality of historical document and with a lack of annotated data.

Line-based OCR approaches are currently very widespread and utilized by the state-of-the-art systems. They use a whole text line as an input instead of individual characters. Therefore, it is not necessary to perform character segmentation, which is error-prone. Within this paper, we focus on a historical printed German Fraktur dataset.

A combination of deep convolutional and recurrent neural networks (CRNN) is nowadays used for line-based OCR [START_REF] Breuel | High performance text recognition using a hybrid convolutionallstm implementation[END_REF]. Long short-term memory (LSTM) [START_REF] Hochreiter | Long short-term memory[END_REF] is often used within the CRNN model. For this task, we used the a CRNN [START_REF] Graves | Offline handwriting recognition with multidimensional recurrent neural networks[END_REF], [START_REF] Simistira | Recognition of historical greek polytonic scripts using lstm networks[END_REF] model inspired by the authors of [START_REF] Shi | An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition[END_REF].

To be able to train such a line-based OCR system, we need to provide a big annotated corpus. Manual annotation is a very time-consuming way of dataset creation and hence our annotated corpus consists of only ten manually annotated pages (1368 images of text lines). Since we split the above-mentioned dataset into training, validation and test sets, we consider the training set to be too small and inappropriate for training a neural network classifier. The lack of training data is a significant issue with regard to the ability to learn a language model. It was shown that an implicit language model is an important part of the OCR system [START_REF] Sabir | Implicit language model in lstm for ocr[END_REF].

Using synthetic data for OCR is a considerable step in historical documents processing. Jaderberg et al. present a framework for recognition and synthetic data generation [START_REF] Jaderberg | Synthetic data and artificial neural networks for natural scene text recognition[END_REF]. Margner et al. [START_REF] Margner | Synthetic data for arabic ocr system development[END_REF] present synthetic data for Arabic OCR and another synthetic data generation was proposed by Gaur et al. [START_REF] Gaur | Generation of synthetic training data for handwritten indic script recognition[END_REF].

There are a number of tools (i.e. tool for Arabic OCR in [START_REF] Margner | Synthetic data for arabic ocr system development[END_REF] or Aletheia proposed in [START_REF] Clausner | Efficient ocr training data generation with aletheia[END_REF]) that deal with the synthetic data generation and annotation. One of our synthetic datasets utilizes the Ocropus OCR System (more precisely OCRopy-linegen) [START_REF] Breuel | The ocropus open source ocr system[END_REF], which is another example of a tool for text recognition and synthetic data generation.

The main contribution of this paper is to give an overview of alternative training strategies with a small amount of annotated data available. Three learning strategies will be proposed for this purpose: the first one uses only a few annotated real pages, the second one considers only synthesized text lines and the third method combines generated data with the real ones. We will also propose two generation methods which compose generated text-lines from real characters. We will compare the performance of these methods with OCRopy-linegen tool. We will further experiment with different white-space padding at the beginning of a line image.

The paper is organized as follows. The following section describes our CRNN classifier. The Section 3 describes the dataset. In Section 4 we show the synthetic data creation process. The Section 5 analyzes the training strategies. Experiments and results are shown in Section 6. The final section concludes the paper.

CRNN Classifier

The CRNN classifier use connectionist temporal classification (CTC) output layer. CTC is an output layer designed for sequence labeling with RNNs [START_REF] Graves | Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[END_REF]. We utilize a CNN-LSTM line image classifier. It is capable of classifying unsegmented line images and retrieving the character sequences.

The input of our network is binarized line images with a height of 40 pixels. CNN and max-pooling layers reduce the dimension and extracts features, which are fed to the RNN, more precisely two bidirectional LSTM with 256 units. The output of the bidirectional LSTM layer is a set of dense layers with the softmax activation function. It represents a probability distribution of characters per each time frame. Let A be a set of symbols. The most probable symbol ât of each time frame t is determined as:

ât = argmax ai∈A p ai t (1)
where p ai t is a probability of observing character a i at a given time t. The final part of the classifier is a transcription layer, which decodes the predictions for each frame into an output sequence. The architecture of the classifier is depicted in Figure 1.

Dataset

Our manually annotated dataset consists of ten two-column pages of German newspaper from the second half of the nineteenth century. We chose two pages for testing, one for validation and the remaining seven for training. Table 1 shows detailed statistics of the dataset. A fragment of one page is shown in Figure 2.

Synthetic Data Creation

This section analyzes different types of synthetic data creation. Before getting to that, though, since the CRNN uses LSTM layers, the implicit language models issue is a well-posed question.

In [START_REF] Sabir | Implicit language model in lstm for ocr[END_REF] authors have shown that the implicitly learned language model can improve the recognition accuracy. Therefore, the texts used for synthetic data generation should be aimed at the domain, we work in. 3 .

There are many ways to create synthetic text lines. First of all, we must provide a text source in the target domain. Then the simplest way to create an image of the text line is to take images of individual characters and put them together. It must be done with a respect to the target font though.

It is very convenient to have several different examples of each symbol. Then we ought to use a randomly picked image for a given symbol, due to the greater diversity of the synthetic dataset. This way can be considered as a simple form of data augmentation technique (e.g. Perez et al. [START_REF] Perez | The effectiveness of data augmentation in image classification using deep learning[END_REF]) because from one source text line it is possible to create several different corresponding images.

Another thing to consider is gaps between characters. The gap should be chosen with regard to a font style. The simplest solution is to estimate a constant pixel value and use it as a gap for each pair of letters. Another way is to specify a range (e.g. 1 px -5 px) and use a random value within this range. Finally, it is possible to use real examples to provide precise values of gaps between each pair of characters. However, because of the quality of scans and noise, the extracted precise values could be distorted. Therefore we chose to use the random gaps approach with a range 1 px -5 px. In Figure 3 there is an example of generated random space data.

Synthetic Data Generator

An alternative way of creating such a synthetic dataset is to use a text generating tool (e.g. OCRopy-linegen [START_REF] Breuel | The ocropus open source ocr system[END_REF]). The tool usually has the possibility to choose a font, source texts and several types of image transformations (skewing, warping etc.) to make the desired image of the text line. Every rendered character is always the same and then it could be very useful in the case of printed text (see Figure 4). The images of annotated text lines are depicted in Figure 5.

Training Strategies

Within this section, we present three strategies for CRNN training. The first strategy is straightforward and does not use synthetic data. The second one is basically a comparison of the qualities of the synthetic data without the influence of the real dataset. Finally, the last strategy combines the synthetic and the annotated data and tries to find optimal settings (volume of pages and number of epochs).

For each strategy, there are more possibilities to enrich a training process, such as data augmentation, binarization or different white-space padding at a beginning of an image. The following experimental section covers the binarization and white-space padding issues.

Experiments

Each one of the previously introduced strategies is evaluated on the validation dataset. We developed two different types of synthetic data. The first one contains character images with random gaps and the second one uses the OCRopus (OCRopy-linegen).

Since there is only one text source, all these synthetic datasets should have the same setup from the point of view of the implicit language model.

Binarization and White-space-padding experiment

The original real images have an 8-bit grayscale png format. Using OpenCV4 we thresholded the images to make them binarized by the Otsu's method [START_REF] Otsu | A threshold selection method from gray-level histograms[END_REF]. After the first set of experiments, we discovered that binarized images obtained better results than grayscale ones, which is in compliance with Gupta et al. [START_REF] Gupta | Ocr binarization and image preprocessing for searching historical documents[END_REF]. Hence we used the binarized images for all experiments.

The white-space-padding experiment verifies the influence of a white-spacepadding at the beginning of the image (see Figure 6). The standard white-space padding is about 10 pixels (i.e. the white space between the leftmost edge of the image and the position of the first black pixel which is part of the text). We extended the padding by 50 pixels and we used the approach of Strategy 1 for the comparison (see Table 2). The results show that it is better to use the larger padding. For further experimental setups, all images are padded by 50 px white space to reach about 60 px padding in total, which has been determined as the appropriate value. 2. Comparison of the influence of white-space -100 epochs with real data

Annotated Dataset Experiment -Strategy 1

In this experiment, we tried to use seven pages of the annotated dataset to train a CRNN. The results of this baseline approach, as well as the course of training, are depicted in Figure 7.

The left image shows train and validation CTC loss, while the right one shows the character error rate (CER) and the word error rate (WER). Either of these images shows that after approximately 80 epochs of training the model reaches the best results and then starts to stagnate. The model is trained using the stochastic gradient descent (SGD) algorithm and the learning rate is set to 0.005. We tried to change the value of the learning rate and roughly speaking the results after 100 epochs were very similar and the only minor change was a slightly different course of training. Table 3 shows the results after 100 epochs. 3. 100 epochs experiments results

The experiment showed that the model reached excellent results on the validation data and no significant overfitting occurred. On the other hand, the main drawback is that limited implicit language model has been learned (only 955 unique text lines). Another important issue is a very long learning time (at least 80 epochs).

Synthetic Dataset Experiment -Strategy 2

This experiment compares the quality of our synthetic datasets. In the following Figures, we present the results of 25 epochs of training. The train loss of OCRopy-linegen dataset is after 25 epochs much lower and learning occurs faster, but on the other hand, the validation results (WER, CER and Edit distance) are better in the case of the Random space dataset. We can conclude that the Random space dataset is more similar to the real data. This section describes the third strategy to train the model. First of all, we tried to verify our presumption from the previous section. The training course has shown that the training longer than 10 epochs is not profitable (see Figures 8 and9). Therefore, we took the random space models based on 10, 25 and 50 epochs of synthetic training and then we added one page of real data for the additional 25 epochs of training (Figure 10). Our assumption was the more epochs of synthetic data training, the slower the learning occurs. Unfortunately, our assumption was not confirmed. But still, the model trained on 10 epochs of synthetic training is more suitable for additional real data training than the other ones and moreover, it has the lowest training demands. In the light of a discussion above, we chose 10 epochs as a base for the additional training, because it reached the best results with a respect to the previous experiment.

Based on this 10 epochs, we took gradually 1-7 pages of the annotated train dataset and then we extended the training process of 25 epochs of additional training. The summary of results is in Figure 11. This experiment confirms an assumption that the more pages and epochs, the better the results. All other synthetic datasets show similar dependencies as Figure 11, so we present just a summary of the results in Table 5.

Conclusions

In this work, we have studied and compared different methods of OCR training. Three learning strategies have been proposed for this purpose: the first one uses only a few annotated real pages, the second one considers only synthesized text lines and the third method combines generated data with the real ones. We have also proposed two generation methods which compose generated text-lines from real characters. We have compared the performance of these methods with OCRopy-linegen tool. We have further experimented with different white-space padding at the beginning of a line image.

We have shown that it is not possible to use only synthetic data to obtain sufficient OCR results. We have further reported that it can be sufficient to train the OCR models using only a small amount (about 7 pages) of real data to obtain a good recognition score. However, this process needs many training iterations and like we have indicated the implicit language model is limited. We have concluded that the best way is to combine both large synthetic and small real data to obtain the best recognition score at the low computation costs. We have also proven that our generation method obtains better results than the OCRopy-linegen tool. The last observation is that it is beneficial to use larger padding (about 60 pixels) at the beginning of the line image.

Fig. 1 .

 1 Fig. 1. CRNN Architecture

Fig. 2 .

 2 Fig. 2. Example of the historical text Page # Line # Word # Character # Train 7 955 7653 50 426 Val 1 138 1084 6 669 Test 2 275 2163 13 828 Table 1. Statistical dataset info

Fig. 3 .

 3 Fig. 3. Examples of generated data with random space.

Fig. 4 .

 4 Fig. 4. Examples of generated data by OCRopy-linegen.

Fig. 5 .

 5 Fig. 5. Examples of real dataset lines.

Strategy 1 :

 1 use only training part of the annotated real corpus for training Strategy 2: use only synthetic data for training Strategy 3: use synthetic data first and then gradually add pages of the training part of the annotated real corpus for training.

Fig. 6 .

 6 Fig. 6. Illustration of two different white-space-padding examples

Fig. 7 .

 7 Fig. 7. WER, CER, Train & Val loss after 100 epochs with the learning rate 0.005 from the training annotated real dataset

Fig. 8 .

 8 Fig. 8. WER, CER, Train & Val loss after 25 epochs from the random space dataset

Fig. 9 .

 9 Fig. 9. WER, CER, Train & Val loss after 25 epochs from the OCRopy-linegen dataset

Fig. 10 .

 10 Fig. 10. Random space additional training based on 10, 25 and 50 epochs of synthetic training

Fig. 11 .

 11 Fig. 11. Random space additional training

Table 1 .

 1 Statistical dataset info

		7	955	7653	50 426
	Val	1	138	1084	6 669
	Test	2	275	2163	13 828

Table 4 .

 4 Table 4 shows the results. Results of Strategy 2 6.4 Synthetic and Annotated Dataset Experiment -Strategy 3

		Train loss WER CER Edit distance
	Random space	7.630	0.655	0.237	10.732
	OCRopy-linegen	0.014	0.734	0.281	12.174

Table 5 .

 5 Summary of the results

		Train loss WER CER Edit distance
	Strategy 3
	Random space	0.069	0.065 0.012	0.428
	OCRopy-linegen	4.838	0.307	0.069	2.725
	Strategy 2 -Random space	7.630	0.655	0.237	10.732
	Strategy 1 -100 epoch	0.056	0.068	0.014	0.464

In our case historical German texts from the second half of the nineteenth century.

https://opencv.org/

Acknowledgement

This work has been partly supported by Cross-border Cooperation Program Czech Republic -Free State of Bavaria ETS Objective 2014-2020 (project no. 211) and by Grant No. SGS-2019-018 Processing of heterogeneous data and its specialized applications. Access to computing and storage facilities owned by parties and projects contributing to the National Grid Infrastructure MetaCentrum provided under the programme "Projects of Large Research, Development, and Innovations Infrastructures" (CESNET LM2015042), is greatly appreciated.