Wassim Swaileh
email: wassim.swaileh@irisa.fr

Florent Imbert
email: florent.imbert@irisa.fr

Yann Soullard

Romain Tavenard

Eric Anquetil

Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network

Keywords: Online Handwriting, Trajectory Reconstruction, Digital Pen, Temporal Convolutional Neural Network, Inertial Measurement Units

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Digital devices can help pupils and teachers in the learning process by promoting active learning techniques and providing immediate feedbacks [START_REF] Simonnet | Evaluation of children cursive handwritten words for e-education[END_REF]. The e-learning literature shows that computer-based analysis of handwriting can be really accurate, sensitive, and reliable to produce relevant and consistent feedbacks for correction or guidance. Several pen-based tablet applications have been designed in order to give immediate and personalized feedback to children [START_REF] Krichen | Combination of explicit segmentation with seq2seq recognition for fine analysis of children handwriting[END_REF]. Moreover, children still need to learn to write on paper to acquire different sensations because today, it is still by far the most widely used handwriting medium.

As an answer to that need, digital pens have been designed to allow for handwriting on paper while capturing the handwriting gesture. Here, we focus on such kinds of stylus with the goal to reconstructing the digital handwriting trajectory of the pen. The digital pen that we use in this work is the Digipen stylus developed by STA-BILO, which is equipped of kinematic sensors to track the pen movements. IMU sensors are utilized in other fields to recognize pre-defined movements [START_REF] Mcintosh | Sensir: Detecting hand gestures with a wearable bracelet using infrared transmission and reflection[END_REF][START_REF] Wilhelm | Perisense: Ring-based multifinger gesture interaction utilizing capacitive proximity sensing[END_REF] or recreate pedestrian trajectories [START_REF] Derrode | Unsupervised pedestrian trajectory reconstruction from IMU sensors[END_REF]. More closely related to this work, a wide range of studies have tackled the OH recognition task, which can be successfully accomplished despite the noisy nature of IMU sensor signals.

In this work, we focus on the more challenging task of trajectory reconstruction. Regarding OH recognition, there is one label for a global shape of handwriting. The model is trained to extract global displacement features to produce a prediction. This is in contrast with the OH reconstruction where there is one label per time frame. Thus, the model is trained to extract local displacement features based on a more or less local view to produce position predictions. In addition, hovering parts, i.e. when the pen is in air, have to be modeled in order to get a good position of the next touching stroke. This must be done without knowledge about the labeling, as there is no pen traces related to those parts. To the best of our knowledge, the recent work proposed by Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] is the second attempt to reconstruct OH trajectories from a digital stylus using deep neural networks. It generalizes the works from Ott et al. [START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF] to multiple writers. Older works explore more traditional approaches for this task such as Markovian models [START_REF] Pan | Handwriting trajectory reconstruction using lowcost imu[END_REF] or movement acceleration inference techniques [START_REF] Bu | Handwritingassistant: Reconstructing continuous strokes with millimeter-level accuracy via attachable inertial sensors[END_REF].

In terms of evaluation, some of the earlier works relied on a qualitative assessment of the reconstructed trajectories [START_REF] Nguyen | Online trajectory recovery from offline handwritten japanese kanji characters of multiple strokes[END_REF]. In other works, recognition performance is used as a proxy to evaluate reconstruction [START_REF] Huang | Agtgan: Unpaired image translation for photographic ancient character generation[END_REF][START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF]. Sometimes, additional reconstruction-only metrics such as the Root Mean Squared Error (RMSE) or the Dynamic Time Warping (DTW) are used [START_REF] Chen | Complex handwriting trajectory recovery: Evaluation metrics and algorithm[END_REF]. This work presents a novel handwriting trajectory reconstruction pipeline from IMU sensors.

Our main contributions are:

• preprocessing including an alignment strategy between input and ground-truth time series for training, as well as a learning pipeline based on touching strokes,

• a neural network architecture inspired by Temporal Convolutional Networks (TCN) to cope with noisy signals from the inertial sensors,

• a database that will serve as a benchmark to help advance research,

• an evaluation protocol based on the Fréchet distance to assess the quality of the reconstructed trajectories.

Both training and testing phases are described in details to allow the reproducibility of experiments.

The rest of the paper is organized as follows, related works are presented in the following section. Section 3 introduces the data acquisition protocol and challenges of this task. The proposed method is described in section 4. The experimental results are presented and discussed in section 5 before the conclusion and perspectives.

Related works

While the field of digital devices for note-taking, drawing, or handwriting learning is growing quickly, most of the systems use a screen for digital handwriting acquisition. Only few of them use a stylus that integrates motion tracking systems in order to reconstruct the handwriting trajectories. To the next, we first present related works on handwriting trajectory reconstruction, based on various devices. Then, we focus on the use of IMU sensors for handwriting recognition which is a task that have been widely studied. Another approach is to allow writing on paper, which is more natural, easier and more ergonomic.

Handwriting trajectory reconstruction

To do that, pens are equipped with specific tools to capture the handwriting gestures. One way is to embed a camera in the stylus, as for the Anoto.

Disadvantages are that these stylus are expensive and that they have to be used on special papers.

A second way is to equip the stylus with IMU sensors. This solution is surface free and low cost and one can write on tablet, paper, or on a board. However, in contrast to a pen based tablet, where one obtains absolute coordinates of the online trajectory, a signal from IMU sensors is only composed of relative pen displacements and it can be very noisy.

Based on the previous statement, we can distinguish between three categories of OH trajectory reconstruction tasks: i) trajectory reconstruction from offline handwriting image [START_REF] Chen | Complex handwriting trajectory recovery: Evaluation metrics and algorithm[END_REF]; ii) reconstruction from a pen-tip optical tracking system [START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF];

iii) trajectory reconstruction from IMU signals [START_REF] Bu | Handwritingassistant: Reconstructing continuous strokes with millimeter-level accuracy via attachable inertial sensors[END_REF][START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF][START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF]. In practice, only a limited number of works have focused on OH trajectory reconstruction from IMU signals. The works of Wehbi et al., [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] (multi-writer approach) and Ott et al, [START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF] (monowriter non-generic approach) used deep learning techniques to reconstruct the OH trajectories from IMU signals as a step towards the OH recognition task. Earlier works such as [START_REF] Pan | A noise reduction method for imu and its application on handwriting trajectory reconstruction[END_REF] used more traditional approaches (Hidden Markov models [START_REF] Pan | Handwriting trajectory reconstruction using lowcost imu[END_REF], pen-tip acceleration and angular velocity inference to multi-level writing plans [START_REF] Bu | Handwritingassistant: Reconstructing continuous strokes with millimeter-level accuracy via attachable inertial sensors[END_REF]) for the same task.

Bu et al. [START_REF] Bu | Handwritingassistant: Reconstructing continuous strokes with millimeter-level accuracy via attachable inertial sensors[END_REF] introduces an IMU system based on accelerometer and gyroscope data to reconstruct handwriting. The method infers the tip trace displacements (captured by the IMU) into several planes of OH trajectories using principal component analysis (PCA) and a set of empirically chosen parameters. This method works in optimal conditions of use but remains sensitive to rotations and translations of the pen.

Ott et al. [START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF] rely on multi-task learning for joint classification and trajectory reconstruction.

They show that a joint learning improved both the OH recognition and the trajectory reconstruction when using a suitable combination of loss functions. However, the proposed approach cannot be generalized since the proposed architecture is limited to produce trajectories of fixed length (100 points). Furthermore, the training and test sets were collected over a single writer.

Liu et al. [START_REF] Liu | Eavesdropping on stylus pen writing via magnetic sensing from commodity mobile devices[END_REF] focus on magnetic signals but they are sensitive to outer magnetic field. In [START_REF] Pan | Handwriting trajectory reconstruction using lowcost imu[END_REF], the authors use low cost IMU from a smartphone to recognize characters and perform trajectory reconstruction. Linear Discriminant Analysis (LDA) is used to detect movements which are estimated mathematically.

Recently, Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] have proposed an approach that deals with the Stabilo Digipen, on its earlier version 5.0. They create a first dataset by using the Digipen on a tablet via a Stabilo mobile application. This allows to acquire simultaneously input signals from the pen and OH trajectories from a tablet.

The gap between sampling rates coming from the pen and the tablet makes mandatory to align the two time series to get one label for each time frame. It is done through a preprocessing that will be discuss more in details in Section 3.3. To do so, Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] linearly interpolate the OH trajectory signal from the tablet. However, the linear interpolation may re-distribute the points equally in time, which result in the loss of the online writing dynamics of speeds and acceleration.

The authors observe that the linear interpolation on the whole output signal leads to erroneous alignments due to the variability of the transmission time delay. By applying the linear interpolation on strokes rather than on the whole label, better alignments are obtained at the cost of dropping out from the dataset any sample whose input and output signals have unequal numbers of strokes. The number of strokes is deduced from the force sensor signal (pen side) and the pressure signal (tablet side). Then, they use a Convolutional Neural Network (CNN) made of 3 convolutional layers with batch normalization layers in between to reconstruct the handwriting trajectory.

Using IMU sensors for handwriting recognition

As discussed before, only few works perform handwriting reconstruction from IMU sensors. On the other hand, the task of OH recognition from IMU sensor data has been tackled through the use of: i) the pen-tip trajectory signal given by a touchscreen device [START_REF] Mustafid | Iamonsense: Multi-level handwriting classification using spatio-temporal information[END_REF]; ii) signals coming from IMU sensors [START_REF] Bu | Handwritingassistant: Reconstructing continuous strokes with millimeter-level accuracy via attachable inertial sensors[END_REF][START_REF] Ott | Benchmarking online sequence-to-sequence Springer Nature 2021 L A T E X template and character-based handwriting recognition from imu-enhanced pens[END_REF][START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF][START_REF] Wehbi | Towards an imu-based pen online handwriting recognizer[END_REF][START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF].

Recently, [START_REF] Ott | Benchmarking online sequence-to-sequence Springer Nature 2021 L A T E X template and character-based handwriting recognition from imu-enhanced pens[END_REF] introduced a benchmark study that compares several neural networks architectures trained to recognize characters, symbols, words and equations from IMU signals.

A CNN/BLSTM architecture proposed in [START_REF] Ott | The onhw dataset: Online handwriting recognition from imu-enhanced ballpoint pens with machine learning[END_REF] obtains the best results. Other works intend to address both OH trajectory reconstruction and character recognition. For instance, Wehbi et al. and the remote device (e.g. tablet) [START_REF] Kreß | Hardware-aware workload distribution for aibased online handwriting recognition in a sensor pen[END_REF] or explore domain adaptation [START_REF] Klaß | Uncertainty-aware evaluation of timeseries classification for online handwriting recognition with domain shift[END_REF][START_REF] Ott | Domain adaptation for time-series classification to mitigate covariate shift[END_REF] and explainability [START_REF] Azimi | Improving accuracy and explainability of online handwriting recognition[END_REF] in the context of OH recognition from the Digipen.

In this section, we present the protocol applied for collecting a new benchmark IRISA-KIHT dataset.

We detail the data acquisition process and the protocol to generate the ground truth. For this, Stabilo 1 created the Digipen, a pen ball digital pen with a Wacom tip instead of an ink ball.

Several versions of such digital pen exist, such as Digipen kids version 5.0 that works at 100Hz sampling rates and Digipen basic version 6.0 that works at 400Hz (the one used in this work).

Acquisition tools description

Data acquisition protocol

The recording process (Figure 2(a)) starts by selecting one set of predefined scripts that will be written on the tablet surface using the Digipen.

One set consists of 34 samples that have to be written one at a time during a single recording session. It is composed of five groups: 15 characters, 10 words, 5 equations, 2 shapes and 2 word groups. While recording, a user holds the pen's on/off switch up, which is a natural way to take the Digipen due to grips designed on the pen to naturally position the fingers properly.

Data acquisition challenges

To create the IRISA-KIHT dataset the acquisition process is quite challenging due to several difficulties. First, the stylus and the tablet have different sampling rates, so the selection of both of them is an important step. As discussed before, we select the Samsung S7 FE tablet and Digipen v6 for data collection. Second, an important degree of freedom comes from the pen orientation that directly impacts pen sensor values. The Digipen design includes grips to naturally position the fingers properly with the stylus on the same orientation.

Another challenge concerns the recording of the hovering (pen up) movements. When the pen gets too far from the tablet, the tablet stops recording a trace. This results to parts of the data coming from the stylus that not match any ground truth from the tablet.

In addition, a drift in the accelerometer measurements is possible, and the kinematic signals In practice, we observe that aligning the timestamps using DTW as we do is more effective than relying on the DTW alignment of force

Cleaning and normalization

Neural Network model

We design a neural network architecture to predict the displacement vectors of the handwriting trajectory from the tablet, given a sensors signal of the Digipen (Figure 2(e)). The training and test steps of our proposed model are described in the following sub-sections.

Model architecture

In order to take into account the past and future states when mapping an input sequence toward an output signal, we propose to use a non-causal Temporal Convolutional neural Network (TCN) architecture as in [START_REF] Bai | An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[END_REF]. The choice of a TCN architecture is supported by the great success of the TCN for sequence-to-sequence tasks with few training samples, e.g. for weather prediction [START_REF] Yan | Temporal convolutional networks for the advance prediction of enso[END_REF],

traffic prediction [START_REF] Dai | Hybrid spatio-temporal graph convolutional network: Improving traffic prediction with navigation data[END_REF] and sound event localization

and detection [START_REF] Guirguis | Seld-tcn: Sound event localization & detection via temporal convolutional networks[END_REF].

The CNN architecture proposed in [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] captures the spatial features that refer to the arrangement of data points of a sequence, and the relationship between them within the sequence.

However, it lacks the hierarchical and distant dependencies that can be grasped with dilated convolutions and a deep architecture as in a TCN.

It was proved that TCN architecture is most suitable to extract relevant spatial and temporal features for a sequence of frames describing an action compared to an LSTM recurrent network [START_REF] Nan | Comparison between recurrent networks and temporal convolutional networks approaches for skeleton-based action recognition[END_REF]. Indeed, recurrent architectures are known to suffer from vanishing gradient problems in very long sequences, such as those produced by IMU sensors. Thus, TCN based systems outperforms their LSTM counterparts in different fields of application such as anomaly detection [START_REF] Gopali | A comparative study of detecting anomalies in time series data using lstm and tcn models[END_REF] or skeleton-based action recognition [START_REF] Nan | Comparison between recurrent networks and temporal convolutional networks approaches for skeleton-based action recognition[END_REF].

Our model receives 10 of 13 channels of the sensor data. Magnetometer data are neglected due to the interference of the tablet magnetic field with the signal. Figure 5 illustrates our network architecture. It is composed of TCN-like layers, a dense layer followed by a batch-normalization layer before the last dense layer that produces the output of the network (the two (∆x, ∆y) channels).

The TCN part consists of three stacked inner blocks of non-causal and dilated 1D-convolutions with kernel size of 5. The dilation rate applied to each block is increased with the depth of the network, from 1 to 4 and every inner layer is followed by a batch normalization layer. More details are given in appendix. At test time, to evaluate the shape of the prediction, we normalize and center the predicted trajectory and the reference trajectory using the Procrustes analysis to fit a uniform scale.

Model training and test

Experiments and results

This section presents the experimental framework, especially the evaluation protocol and the metric used to evaluate our model. Then, we investigate the benefit of the successive parts of our approach. First, we experiment variants of networks to explore various receptive field sizes.

Second, one compares our pipeline to a previous approach [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] and evaluates the benefit of using both our alignment method and our model. the goal is always to predict both touching and hovering strokes.

Evaluation protocol and metric

Some works perform trajectory reconstruction as a step towards recognition. Thus, final recognition performance can be used as an evaluation criterion [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] and, as discussed before, it cannot reflect well the quality of the reconstruction.

Here, the end goal is the trajectory reconstruc- In fact, due to the limited tracking capacity of the tablet over 15mm above the tablet screen, the handwriting signal is partially captured on hovering. For that, we only consider the touching parts (pen-down) for which we can compare a trajectory prediction to the handwriting trajectory from the tablet and compute the Fréchet distance.

We used the FAU-EINNS dataset, our IRISA-KIHT dataset, and a subset of it, IRISA-KIHT-S, to compare our processing pipeline to [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] IRISA-KIHT and IRISA-KIHT-S contain 38 and 30 writers' recordings, respectively. Each recording contains 34 characters, words, word groups, equations, and shapes. Section 6.2 of the appendix describes the datasets. The FAU-EINNS dataset consists of words written by 6 writers, and we take the samples of the user numbers 1, 2, 3, 5 and 6

for training (1774 samples) and testing on the 344 samples.

Receptive field effect

To Table 1 shows that the TCN model with the smallest receptive field size (49) performs better than the ones with greater receptive fields on FAU-EINNS (100Hz) dataset. On our IRISA-KIHT (400Hz) dataset, the TCN model with the greatest receptive field (373) generally outperforms the other sizes of receptive field as seen in Table 2.

With a big receptive field, a large context can be exploited to make the prediction as well as the signal noise of long hovering can be absorbed, with the cost of higher number of learnable parameters. Table 3 results on word groups. These two models are quite similar in terms of performance (Figure 6).

As one of our next goal is to embed the model inside the Digipen, we decided to keep the TCN-49 model to get a good trade-off between the model performance and the number of parameters.

Comparison with related work

We compare the performance of our approach against the one of Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] on both FAU-EINNS dataset and our own dataset, called IRISA-KIHT. We also provide results on the subset of the latter called IRISA-KIHT-S which is made publicly available and can hence be used as benchmark for future works. We applied the same evaluation protocol on both models and processing pipelines. Due to the comparatively smaller size of IRISA-KIHT-S, a 3 fold cross validation is used when this dataset is at stake, as shown in Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] approach (CNN model and linear interpolation alignment) on every dataset, both quantitatively as seen in Table 5 on the average Fréchet distance and qualitatively as illustrated in Figure 7 (Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] appears on the second line of the figure while our approach appears on the bottom line). Despite the difference in size between IRISA-KIHT and IRISA-KIHT-S datasets, one can observe that reported performance is comparable in both datasets for both our method and its competitor (see table 5).

This confirms that IRISA-KIHT-S is of sufficient size for the quantitative assessment of OH trajectory reconstruction from IMU sensor data. In the next section, we will evaluate each step of the processing chain.

Alignment methods and models

In the following, we compare two effective pro- 6, we observe that each step of our approach outperforms the counterpart of the Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] method on both datasets. The results show that our alignment is better whatever the model on all the datasets.

In contrast to the linear alignment, our alignment based on the DTW algorithm keeps the writing dynamic, which seems to be essential to reach quality trajectory reconstruction. the pen is hovering. Figure 8 illustrates the effect of over hovering on the alignment process.

As illustrated in Figure 8, we distinguish two tracking field). However, for sentences, the user needs to space words and he may raise the pen higher while searching where to put the pen on the screen again.

To avoid the untracked hoverings in the training datasets, we used the touching data only to train our model. The idea is to train the model on touching strokes only and test on touching and hovering strokes as well. We compared our model when it is trained on touching only or on touching and hovering strokes.

Table 7 shows that the strategy of learning on touching strokes only is generally better than the one of learning on touching and hovering strokes upon the distance of Frechet, and this is also the case for characters, words, equations and shapes categories. Results on the sentence category are very close. We think this is due to the correlated effect of the untracked hovering and the pen-movement hesitations of the writer between the words of the sentence. However, by looking at Figure 9, we observe that the touching model reconstructs better touching strokes that appear in the form of single character (e.g. "a") or shape (e.g. "rectangle"). It is also better with double strokes characters (e.g. "û", "ô") where a single tracked hovering can be observed. Similarly, we observe that our model reconstructs quite good the touching parts of the word groups but it fails to find where to start the reconstruction of the next stroke. This may happen due to untracked hoverings (that represents movement hesitations about where to put the pen again on the screen) like in "a -bed" where there is a long untracked hovering between "a" and "bed". A new dataset has been collected using a mobile application, a tablet and a Digipen and we provide benchmark results on it. To tackle with the sampling rate differences between the Digipen and the tablet, as well as the synchronization problem, we introduced a DTW based alignment approach.

Conclusion & perspectives

We show through experiments on several datasets that the TCN neural network architecture outperforms the CNN one proposed in Wehbi et al. [START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] as well as the DTW based alignment approach that enhances the performance of both CNN and TCN models. We also show that the model predicts better the touching and tracked hovering parts, when the model is trained on the touching strokes only.

The major limitation of our approach is the modelling of the untracked / complex hovering trajectories that is still an open research problem. This constitutes a first important track for future works and could be solved using a semisupervised or unsupervised deep learning framework. Moreover, in this paper, we only evaluated the shape of the reconstruction. We could extend our evaluation to the dynamics of the trajectory reconstruction such as speed, acceleration, etc.

RF = 1 + 2 • (k -1) • N • i d i (2)
The kernel size k is selected so that the receptive field covers enough context for predictions.

Fréchetdistance

The formal definition of the Fréchet distance is:

Let S be a metric space, d its distance function.

  Nowadays, a wide range of applications in the domain of remote sensing and tracking systems benefits from recent improvements in deep neural network architectures. Tracking systems are commonly based on Inertial Measurement Unit (IMU) due to the low cost of these sensors. However, IMU-based sensors are quite imprecise due to the poor quality of the IMU signals.

  There are several types of systems dedicated to acquisition of digital handwriting. The first one is based on screens with special pen-based tablet. Tablet manufacturers, based on different Wacom technology, have developed their devices -such as Samsung Spen, Apple Pencil and Microsoft Stylet Surface Pen -to write on a tablet and enable to have a digital OH. These pen based tablets provide a precise track, but they have to be used on a specific device.

  trained two neural networks in succession: first, a model for reconstructing the OH trajectories from IMU signals; second, a model for character recognition based on the reconstructed trajectories. They assess the quality of the reconstruction based on the character recognition rate, but a good recognition rate may not reflect the quality of the reconstructed trajectory. A multitask learning is proposed by Ott et al. [21] using a CNN-LSTM network. They show that both the OH trajectory reconstruction and the character classification benefit from a joint learning. As part of the UbiComp 2021 Challenge on OH recognition, Wegmeth et al. [27] trained a CNN/BLSTM to recognize mathematical expressions written with the Stabilo's Digipen. Their approach is based on a boundary feature extractor follow by a character classifier. As a consequence, the recognition performance depends on the quality of the label boundary splitting. Other studies minimize the link bandwidth between the Digipen

Fig. 1 ©

 1 Fig. 1 © STABILO International Digipen's sensor location

  are transmitted from the pen to the tablet in sets of six points through Bluetooth. Variable transmission time delay results in asynchronous timestamps for kinematic and tablet data.

4

 4 Trajectory reconstruction pipeline To reconstruct the handwriting trajectory from IMU signals, we present a complete pipeline which consists of training and test phases as illustrated in Figures 2, and 3. The training phase consists in: (i) a preprossessing process that produces cleaned and aligned input sensors signals and output handwriting trajectory signals (Figure 2(b) to (d)); (ii) a reconstruction training process, where a neural network model based on a Temporal Convolutional Network (TCN) architecture is trained on strokes using a dedicated loss function to predict the displacement vectors of the handwriting trajectory (Figure 2(e)); (iii) a post-processing to obtain the handwriting trajectory signal from the expected displacement vectors. (Figure 2(f)). The test phase comprises the three principal steps of the training phase. However, preprocessing is reduced to only cleaning and normalizing the input signals (Figure 3(ii)). In addition, the prediction model (Figure 3(iii)) is applied on the whole input signal of the target sample, including touching and hovering strokes of the sample. Postprocessing are the same as in the training phase followed by scaling and centering steps to compare the predicted trajectory with the ground truth.

Fig. 2

 2 Fig. 2 Training processing steps of our proposed handwriting trajectory reconstruction pipeline.

Fig. 3

 3 Fig. 3 Test processing steps together with post processing evaluation of the proposed handwriting trajectory reconstruction pipeline.

4. 1

 1 Data preprocessing Data prepossessing are required at the training phase due to the misalignment of the pen and tablet signals and to the noise in signals. First, signals are divided into spans that correspond to the written samples. One recalls that input signals captured from the Digipen are composed of 13 channels while signals from the tablet are made up of 3 channels (x, y and pressure). The timestamps of the Digipen and the tablet are added as an additional channel. Then, every sample is passed forward to the following process section 4.1.1.

  As part of the training phase, irrelevant parts of the input signals are removed. Those parts are the start and end pen-up movements that does not refer to pen-up and pen-down actions to write the given script (Figure 2(b)). Since the input sensor signals represent the relative values of the acceleration, speed and orientation of the Digipen, it is reasonable to map it to the displacement vectors of the handwriting trajectory signal rather than to the real values of handwriting trajectory. Thus, displacement vectors (∆x, ∆y) are computed from the (x, y) channels of the handwriting trajectory. In order to maintain the interoperability of the system between the different versions of Digipen, accelerometers, gyroscope, magnetometer and force signals are normalized by their maximum values (as reported by the manufacturer). In the test phase, the cleaning and normalization process concerns the sensors' signal only (Figure 3(ii)).

4. 1 . 2

 12 Input -Output DTW alignment Due to the different sampling rates between the stylus and the tablet, an alignment process is crucial to get a mapping between the input and output time series for training. In addition, the sensor data are acquired in packets of 6 data points and the recorded timestamps are not equally spaced due the Bluetooth transmission time delay. Due to this mismatch, the recorded input and output signals have different lengths and they are not synchronized. In order to respect the writing dynamics as much as possible, we propose an alignment approach based on the Dynamic Time Warping algorithm (DTW) (Figure 2(C)). The pen and tablet signals are recorded simultaneously. The transmission time delay is between 10 and 40 milliseconds in the Digipen version 6.0. Since the sampling rate of the sensor data is higher than the one of the tablet data, we need to up-sample the tablet data to match the sensors data length. First, the average transmission time delay is subtracted to the tablet data in order to approximately synchronize it with sensors data. Then, we use the DTW algorithm (for more details see the appendix) to find an alignment path between the timestamps of the stylus and the tablet.

Fig. 4

 4 Fig. 4 Our proposed DTW based alignment compared to the linear interpolation based alignment proposed by [29]. The left side image represents the raw force / pressure (sensor/online) signals, the middle image shows the interpolation alignment result and the right side image shows the DTW based alignment results. The red lines connecting the two signals represent the duplicate points added to the pressure signal and the grey ones represent the one-to-one alignment.

Figure 4

 4 shows a comparison between the alignment approach used in[START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] based on linear interpolation and our proposed approach based on DTW. Alignments are presented on the force (pen sensor) signal against the pressure signal (from the tablet). The icon images of Figure4shows that the DTW alignment (Figure4(III)) results in the same data distribution than the raw points while the linear interpolation one (Figure4(II)) spaces the data points equally.4.1.3 Splitting into strokesSplitting into strokes only concerns the training phase of the proposed pipeline. It enables to train the neural network on the touching parts (strokes) only (Figure 2(d)). The idea is to use the force/pressure signals of the sensors and handwriting trajectory signals to determine the touching strokes of the training samples, knowing that we consider every touching stroke whose force/pressure values is greater than a predefined threshold, namely 0.01 for the force and 0 for the pressure. We have the choice to either learn the neural network model on the entire training sample including the touching and hovering stroke, or to learn it on the touching strokes only. We will investigate it in the experimental part. However, we always test the model on the entire test samples (including the touching and hovering strokes).

  During the training phase, the model is trained to minimize the Mean Squared Error between the real and predicted displacement vectors of the handwriting trajectories. One uses the ADAM optimization, a batch size of 16 and a learning rate of 10e -3 . As stated before, the 10 input channels of the sensor signal are the front and rear accelerators, gyroscope and force channels of the sensor signal, obtained after the cleaning and normalization process. During the test phase, a cleaned and normalized signal is given as input of the model that predict the corresponding displacement vectors of the handwriting trajectory signal.4.3 Post-processingThis process is the same in training and test phases. It consists in reconstructing the predicted handwriting trajectories by computing the accumulated values from the predicted displacement vectors (Figure 2(f), and Figure 3(iv)).

Finally, one explores

  the impact of training our model both on touching and hovering strokes compared to training it on touching strokes only, when

Fig. 5

 5 Fig. 5 Our TCN model for handwriting trajectory reconstruction.

  tion and dedicated metrics should be used to assess similarity between ground truth and reconstructed trajectories. Root Mean Squared Error (RMSE) is a natural candidate[START_REF] Ott | Joint classification and trajectory regression of online handwriting using a multi-task learning approach[END_REF], but it can't handle temporal alignment. Dynamic Time Warping (DTW), allow this temporal alignment. Chen et al.[START_REF] Chen | Complex handwriting trajectory recovery: Evaluation metrics and algorithm[END_REF] created the Adaptive Intersection over Union (AIOU) metric, which uses the stroke width and thickness as well as the length-independent DTW to compare the trajectory-point alignment of two different handwriting trajectories of different lengths. In this work, we decide to rely on the Fréchet distance , whose formal definition is given in appendix section 6.4. This distance has benefits that we detail in the appendix section 6.4. The Fréchet distance measures the distance between curves, by taking into account the location and ordering of the points along the curves[START_REF] Har-Peled | New similarity measures between polylines with applications to morphing and polygon sweeping[END_REF]. This allows to capture both local and global information accurately, and correlates well with qualitative assessment in practice.

  design the TCN model architecture, we pay attention to the sampling rate of the input signal of sensors. It is obvious that the TCN based neural network may have the capacity to absorb the noise of such low quality and noisy signals, depending on the size of its receptive field. In order to evaluate the receptive field effect of our model, we trained and evaluated the model with different sizes of receptive fields equal to 49, 85, 168 and 373 on both FAU-EINNS (100Hz) and IRISA-KIHT (400Hz) datasets.

FAUFig. 6

 6 Fig. 6 Handwriting reconstructions with two TCN models with different receptive fields on the FAU-EINNS (left) and IRISA-KIHT (right) datasets. First line, ground-truth. Second, the 49-receptive-field TCN model. Third line: TCN model with 373 receptive field.

  cesses of the handwriting trajectory reconstruction pipeline; (i) the input-output alignment process and (ii) the reconstruction model performance. We provide a cross validation scheme between the two alignment methods (linear interpolation and DTW based alignment methods) and the two reconstruction models (CNN and TCN based models) on the FAU-EINNS and IRISA-KIHT datasets. From Table

  Figure 7 shows examples of reconstruction trajectories using the CNN or TCN models, when using the linear interpolation and the DTW alignments on the two datasets. Moreover, our TCN model outperforms the CNN counterpart whatever the alignment method, which shows that the TCN model benefits from a larger receptive field and a deeper network to model more complex patterns and to be less sensitive to the noise.5.5 Touching versus hovering trajectories reconstructionRecalling the definition of the touching and hovering strokes, a touching stroke consists of data points that are recorded while touching the tablet screen. Hence, hovering stroke is recorded partially due to the limited capacity of the tablet screen to track the pen when it goes up to 15mm 2 . As a result, only the Digipen continue to provide sensor signals when the pen is up over 15mm. In conclusion, there is no ground truth for such hovering movements. To offset the effect of over hovering on the data preprocessing, the proposed DTW alignment duplicates the end points of the period when 2 http://tennojim.xyz/article/wacom intuos pro l guide

Fig. 7

 7 Fig. 7 Comparison with [29] approach with our one. First line, [29] CNN model and linear interpolation alignment results. Second line, CNN model and the introduced DTW alignment results. Third line, our TCN model and linear interpolation alignment results. Last line, our TCN model and the proposed DTW alignment results.

Fig. 8

 8 Fig. 8 Tracked hovering and untracked hovering effects on the alignment process

flat doted lines of Figure 8 .

 8 (ii) untracked hoverings, when the Digipen goes too far from the tracking field of the tablet and consequently leading to lose the OH signal to be reconstructed by the model. At the same time, we observe a dual effect of missing the handwriting signal that can be correlated with the writer experience in writing. In fact, a fluent writer (adult) don't need to search where to start writing the next stroke of the same character or words. As a result, he don't need to raise the pen so high (outside the tablet'

  This work introduces a novel processing pipeline for reconstructing handwriting trajectories from kinematic sensor signals. Input signals come from a digital pen designed by the Stabilo company called Digipen. Our approach consists in training and testing phases, each one composed of three main processes; preprocessing, a neural network training/prediction process and post-processing.

  z), accelerometer rear (x, y, z), gyroscope (x, y, z), magnetometer (x, y, z), and force signals.Tablet signal files contain milliseconds, position coordinates (x, y, z), and pressure force signals.The transcription (labels) file contains labels and the start and stop time-stamps for every sample.Additional files concerning the sensor calibration and recording meta data are provided. The dataset website describes the format and meta data.

6. 3

 3 TCN architecture A TCN (Temporal Convolutional Network) like layer consists of dilated, residual non-causal 1D convolutional layers with the same input and output lengths. The input tensor of our TCN implementation has the shape (batch size=None, input length=None, channels num=13) and the output tensor has the shape (batch size=None, output length=None, channels num=2). Since each TCN layer has the same input and output length, only the third dimension of the input and output tensors vary ("None" here denote the learning with batches of different size). The TCN layer consists of 4 blocks in TCN-49 and 3 blocks in TCN-373, with each block containing 100 filters followed by a RELU activation function and padded to the "same" input shape. A batch normalization layer was introduced between each pair of consecutive blocks. The performance of a TCN architecture based model depends on the size of the receptive field (RF) of the network. Since a TCN's receptive field depends on the network depth N as well as filter size k and dilation factor d, stabilization of deeper and larger TCNs becomes important [2]. The receptive field of our TCN based model is computed as :

For a regular convolution filter the dilation d is equal to 1 .

 1 Using larger dilation enables an output at the top level to represent a wider range of inputs. In the equation, there is a multiplication by 2 because there are two 1D CNN layers in a single residual block of the TCN architecture. For the purposes of the trajectory reconstruction task, we set the stride equal to 1. For the TCN-49 we choose to set k=3, N=4 and d=1, 2 and for the TCN-373 k=3, N=3 and d=1, 2, 4, 8, 16.

Fig. 11

 11 Fig. 11 Metrics comparison. On the top, the ground truth and on the bottom two predictions coming from different models.

A

  curve A in S is a continuous map from the unit interval into S, i.e. A : [0, 1] → S. A reparameterization α of [0, 1] is a continuous, non-decreasing, surjection α : [0, 1] → [0, 1] Let A and B be two given curves in S. Then, the Fréchet distance between A and B is defined as the infimum over all reparameterizations α and β of [0, 1] of the maximum over all t ∈ [0, 1] of the distance in S betweenA(α(t)) and B(β(t)). The Fréchet distance F is defined by the following equation:F (A, B) = inf α,β max t∈[0,1] {d(A(α(t)), B(β(t))} (3)Appropriate assessment Metrics are needed to evaluate trajectory reconstruction to give students useful feedback. Fréchet distance doesn't disagree with visual evaluation like DTW and RMSE.

Figure 11 indicates

 11 Figure 11 indicates that the left upper loop of the f is better reconstructed. Fréchet distance is needed to observe this.

Table 1

 1 Fréchet distance with standard deviation from TCN model with varied receptive fields on FAU-EINNS dataset

	illustrates the size of TCN models
	in terms of number of learnable parameters with
	regard to the receptive fields.
	By comparing TCN-49 and TCN-373, during
	hovering strokes, the network may see the last
	and next touching points of two successive touch-
	ing strokes, as illustrated in Table 2 with better

Table 2

 2 Fréchet distance with standard deviation from TCN model with varied receptive fields on IRISA-KIHT dataset

	Type	TCN-49	TCN-85	TCN-169	TCN-373
	Global	0.2039 ± 0.1427	0.2415 ± 0.1676	0.2042 ± 0.1419	0.1807 ± 0.1338
	Characters	0.2758 ± 0.1709	0.3048 ± 0.2087	0.236 ± 0.1901	0.2402 ± 0.1642
	Words	0.1383 ± 0.0727	0.1716 ± 0.0886	0.1605 ± 0.0627	0.1207 ± 0.0702
	Equation	0.1438 ± 0.0568	0.213 ± 0.0998	0.1964 ± 0.0639	0.1462 ± 0.0584
	Shapes	0.281 ± 0.1071	0.3069 ± 0.1289	0.2758 ± 0.12	0.2505 ± 0.1195
	Word groups	0.1552 ± 0.0665	0.1971 ± 0.0775	0.2028 ± 0.0843	0.1269 ± 0.0522

Table 4 .

 4 Results show that our pipeline (TCN-49 model and DTW based alignment) outperforms

Table 3

 3 TCN models' size with various receptive fields

	# Params	TCN-49	TCN-85	TCN-169	TCN-373
	Total	467,452	528,452	870,452	894,452
	Trainable	464,152	524,752	866,752	888,352

Table 4

 4 Average Fréchet distance with standard deviation on IRISA-KIHT-S over 3-fold compared to[START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF]

		CNN model [29]	Our approach
	Fold 1	0.4055 ± 0.2699	0.2099 ± 0.1750
	Fold 2	0.5188 ± 0.3926	0.3122 ± 0.1731
	Fold 3	0.4383 ± 0.3701	0.1766 ± 0.1056
	Mean	0.4542 ± 0.3442	0.2329 ± 0.1512

Table 5

 5 Average Fréchet distance with standard deviation of our pipeline compared to[START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF]

		CNN model [29]	Our approach
	FAU-EINNS	0.2628 ± 0.1256	0.077 ± 0.0356
	IRISA-KIHT	0.4691 ± 0.2465	0.2039 ± 0.1427
	IRISA-KIHT-S	0.4542 ± 0.3442	0.2329 ± 0.1512

Table 6

 6 Model and alignment method comparison between[START_REF] Wehbi | Surface-free multi-stroke trajectory reconstruction and word recognition using an imuenhanced digital pen[END_REF] and ours on both datasets.

		CNN model [29]	Our TCN model-49
		Linear interpolation alignment [29]	Our DTW alignment	Linear interpolation alignment [29]	Our DTW alignment
	FAU-EINNS 100Hz	0.2628 ± 0.1256	0.0801 ± 0.0352	0.1211 ± 0.0639	0.077 ± 0.0356
	IRISA-KIHT 400Hz	0.4691 ± 0.1427	0.2095 ± 0.1255	0.3859 ± 0.2003	0.2039 ± 0.1427
	FAU-EINNS 100 Hz		IRISA-KIHT 400 Hz

Table 7

 7 Training on touching strokes versus learning on touching & hovering strokes results

		Our TCN Model-49
	Type	Trained on touching & hovering strokes	Trained on touching strokes only
	Global	0.2471 ± 0.1711	0.2039 ± 0.1427
	Characters	0.3587 ± 0.185	0.2758 ± 0.1709
	Words	0.1497 ± 0.0671	0.1383 ± 0.0727
	Equations	0.1483 ± 0.0666	0.1438 ± 0.0568
	Shapes	0.429 ± 0.2094	0.281 ± 0.1071
	Word groups	0.1497 ± 0.0696	0.1552 ± 0.0665

Table 8

 8 IRISA-KIHT and IRISA-KIHT-S datasets descriptions.

	Datasets	Sets	# writers	# Recordings	# Samples	Characters	Words	Equations	Shapes	Word groups
				61 (mixed)
		Training	25	+	3,770	915	2,306	305	122	122
	IRISA-KIHT			53 (words)
		Test	10	10 (mixed)	340	150	100	50	20	20
		Total	35	124	4,110	1065	2,406	355	142	142
	IRISA-KIHT-S	Training	20	20 (mixed)	680	300	200	100	40	40
	(3-Folds
	Cross-validation)	Test	10	10 (mixed)	340	150	100	50	20	20
		Total	30	30	1020	450	300	150	60	60
	To evaluate the generalization capacity of
	our proposed approach, we decided to train and
	test our model on different writers. The IRISA-
	KIHT dataset consists of 25 writers' recordings in
	the training set (3,770 samples) which represents
	around 71.5% of the writers' total number. The
	rest 10 writers' recordings never seen in training
	refer to the test set, representing 28.5% of writers
	(8.3% of samples). Figure 10 shows some samples
	of the IRISA-KIHT datasets.
	The IRISA-KIHT-S dataset is a subset of
	the IRISA-KIHT dataset which is available on
	request 3 . This dataset is composed of 30 record-
	ings and it is writer balanced as there is one
	recording per writer. Table 8 presents the IRISA-
	KIHT and IRISA-KIHT-S statistical description.
	Every 34-sample recording session generates
	files from the data acquisition mobile app. The
	sensor signals file has 15 columns and N rows,
	where N is the number of IMU signals, times-
	tamps, and sensor values. The table has 13
	columns: milliseconds, accelerometer front (x, y,

3

Available free of charge for research community upon demand for research purposes only

Acknowledgments

This project is financed by the KIHT French-German bilateral ANR-21-FAI2-0007-01 project and these four partners, IRISA, KIT, Learn & Go and Stabilo. This work was performed using HPC resources from GENCI-IDRIS (Grant 2021-AD011013148)

Appendix 6.1 DTW alignment

The DTW algorithm [START_REF] Sakoe | Dynamic programming algorithm optimization for spoken word recognition[END_REF] is based on dynamic programming to assess the similarity between time series. For two multivariate time series x ∈ R Tx×z , and y ∈ R Ty×z of equal feature dimensionality z and respective lengths T x and T y , the DTW is written as follows:

where E(x, y) represents the set of all admissible alignments between x and y and d is a distance metric in R z . Commonly, the squared Euclidean

An alignment is a sequence of pairs of timestamps that is admissible if (i) it matches the first (and respectively the last) indices of time series

x and y together, (ii) it is monotonically increasing, and (iii) it connects the two time series by matching at least one index of each series. Using dynamic programming, the admissible alignment paths can be computed according to the following recurrence formula:

where x →i denotes time series x observed up to timestamp i.

The DTW method on timestamps links multiple sensor signal points to at least one tablet signal data point. In our case, we do not allow the association of multiple points of the tablet signal to one point of the sensor signal. In other words, in the previous recurrence formula the predecessor DT W (x →i , y →j-1 ) is removed. According to the resulting alignment path, the timestamps from the sensor signal are aligned with their corresponding timestamps from the tablet signal.

Datasets description


Thibault Le Gentil
email: thibault.le-gentil@ens-rennes.fr

Daniel Therriault
email: daniel.therriault@polymtl.ca

Olivier Kerbrat
email: olivier.kerbrat@ens-rennes.fr

A comprehensive methodology to support decision-making for additive manufacturing of short carbon-fiber reinforced polyamide 12 from energy, cost and mechanical perspectives

Keywords: Fused filament fabrication, Manufacturing cost, Energy consumption, Energy model, Multiple-criteria decision analysis, Polymer composites

come

A comprehensive methodology to support decision-making for additive manufacturing of short carbon-fiber reinforced polyamide 12 from energy, cost and mechanical perspectives

Introduction

Additive manufacturing (AM) has attracted interest from both academia and industry, on account of its numerous applications in a variety of fields [START_REF] Wang | 3D printing of polymer matrix composites: A review and prospective[END_REF]. Contrary to a traditional manufacturing process, AM refers to a family of manufacturing techniques in which three-dimensional components are fabricated by adding materials layerby-layer. The increased demand on more complex products and designs in a more efficient and sustainable way reflects some of the advantages of AM. The trend in both industry and research is to take advantage of the geometry freedom to produce lighter parts while keeping similar mechanical properties for composite and metallic parts [START_REF] Mabkhot | Mapping industry 4.0 enabling technologies into united nations sustainability development goals[END_REF][START_REF] Ramalho | Effect of contaminations on the acoustic emissions during wire and arc additive manufacturing of 316L stainless steel[END_REF][START_REF] Zuo | Wire-based directed energy deposition of NiTiTa shape memory alloys: Microstructure, phase transformation, electrochemistry, X-ray visibility and mechanical properties[END_REF]. These advantages includes a higher flexibility in both design and supply chain, and allows parts to be made that are impossible to produce in conventional manufacturing [START_REF] Suárez | Sustainability and environmental impact of fused deposition modelling (FDM) technologies[END_REF]. Additionally, for extrusion-based AM processes, this flexibility helps reduce raw material consumption and material waste [START_REF] Paris | Comparative environmental impacts of additive and subtractive manufacturing technologies[END_REF]. Therefore, in the context of Industry 4.0 and sustainability, AM is seen as one of the many means of bringing about this transition [START_REF] Mabkhot | Mapping industry 4.0 enabling technologies into united nations sustainability development goals[END_REF]. Among the AM processes, fused filament fabrication (FFF) the most widespread AM technique, has found applications within various sectors including aerospace, automotive, sport, and construction [START_REF] Tofail | Additive manufacturing : scientific and technological challenges, market uptake and opportunities[END_REF]. In particular, the combination of AM and carbon-fiber-reinforced polymer opens up the possibility of producing parts with a high stiffness-to-weight ratio [START_REF] Van De Werken | Additively manufactured carbon fiber-reinforced composites: State of the art and perspective[END_REF], whilst reducing total mass through complex designs [START_REF] Herrmann | Life cycle engineering of lightweight structures[END_REF].

Thanks to recent technological progress, many materials are now applied to AM. Polyamide 12 (PA12) found many applications due to its high ductility (elongation at break up to 10% for FFF printed part [START_REF] Rahim | The improvement of mechanical and thermal properties of polyamide 12 3D printed parts by fused deposition modelling[END_REF]) and high toughness (about 500 MJ/m 3 [START_REF] Vidakis | Parameter effects and process modelling of Polyamide 12 3D-printed parts strength and toughness[END_REF][START_REF] Griehl | Nylon-12-Preparation, Properties, and Applications[END_REF]), with relative low cost. Additionally for the case of aerospace or automobile, PA12 respect flame resistance standards [START_REF] Kausar | Advances in Carbon Fiber Reinforced Polyamide-Based Composite Materials[END_REF]. While, PA12 might be limited to relative low strength resistance, the addition of carbon fiber greatly enhance mechanical properties of the material, mainly stiffness-to-weight ratio [START_REF] Abderrafai | Additive manufacturing of short carbon fiber-reinforced polyamide composites by fused filament fabrication: Formulation, manufacturing and characterization[END_REF]. Short carbon-fiber reinforced polyamide 12 (PA12-CF) have been used to form lightweight aerospace non-structural applications and automotive parts [START_REF] Kausar | Advances in Carbon Fiber Reinforced Polyamide-Based Composite Materials[END_REF]. Moreover, the mechanical properties and printability of the PA12-CF filament varies with printing parameters or AM technology [START_REF] Ferreira | Machinability of PA12 and short fibre-reinforced PA12 materials produced by fused filament fabrication[END_REF]. Table 1 compiles mechanical properties of PA12 and PA12-CF for virgin material and the two main AM process FFF and Selective Laser Sintering (SLS), with FFF PA12-CF showing higher mechanical resistance with a large variation depending on the carbon weight percentages used.

Regarding sustainability, AM has several potential drawbacks, for examples material's high embodied impacts and recyclability [START_REF] Saade | How has LCA been applied to 3D printing? A systematic literature review and recommendations for future studies[END_REF]. Specifically, related to the printing process, there are two major issues, the first being the high energy demand during the manufacturing phase, and the second, the reduced productivity, compared to conventional processes [START_REF] Lunetto | On the correlation between process parameters and specific energy consumption in fused deposition modelling[END_REF][START_REF] Elkaseer | Experiment-Based Process Modeling and Optimization for High-Quality and Resource-Efficient FFF 3D Printing[END_REF][START_REF] Baumers | Charting the Environmental Dimensions of Additive Manufacturing and 3D Printing[END_REF]. Given the rapid growth of AM in industry, and the importance of ensuring sustainable development, there is a real need to investigate the relationship between energy consumption, cost, and the quality of the produced parts. In the present study, we therefore sought to develop a methodology for connecting these three aspects in the case of FFF of PA12-CF. After reviewing the relevant literature on energy and cost assessment in AM (Section II), we describe the methodology we used to model the process and acquire the relevant data (Section III). Then we present the design of experiments based on the fabrication of tensile specimens (Section IV). Two printing parameters are varied (printhead temperature and heatbed strategy) on three levels each in order to obtain energy consumption, mass and tensile test. Results are then compiled in economic (total cost), energetic (Specific Energy Consumption and Specific Printing Energy) and mechanical (tensile resistance and Young's modulus) indicators (Section V), followed by a discussion (Section VI) and conclusion (Section VII).

Relevant literature on energy and cost assessment 2.1 Energy assessment and specific energy consumption of FFF

Initially designed for subtractive processes by Kara and Li [START_REF] Kara | Unit process energy consumption models for material removal processes[END_REF], the specific energy consumption (SEC) indicator has since been applied to AM as a way of comparing and quantifying the energy efficiency of different processes or machines. SEC is defined as the ratio of total energy consumption to the mass deposited, and can be used to quantify energy efficiency in material deposition. Liu & al. [33] examine SEC values for different AM processes and their relation to global warming potential hence, creating a connection from energy consumption to a more global Life Cycle Assessment (LCA). This study also shows the opportunity to increase energy efficiency without compromising the print quality. Dunaway & al. [START_REF] Dunaway | A Preliminary Experimental Study of Additive Manufacturing Energy Consumption[END_REF] studied experimentally the relation between energy consumption and part geometry characteristics. When Lunetto and al. compiled all the available SEC data in the literature [START_REF] Lunetto | On the correlation between process parameters and specific energy consumption in fused deposition modelling[END_REF], they found significant variability in energy efficiency, ranging from 19 MJ/kg to 1247 MJ/kg for polymers. This variability can be explained by the differences in SEC, depending on the printer's architecture (e.g., type, enclosure), the materials, the process parameters, and the geometric complexity. The warming up of the heating elements has been identified as the main demand of power [START_REF] Ma | Energy consumption distribution and optimization of additive manufacturing[END_REF]. The warm-up phase greatly varies depending on the printer architecture or the working environment (i.e., room temperature, humidity). Hence a second indicator, specific printing energy (SPE; i.e. SEC during the deposition phase), can be determined to focus on the printing phase. Total energy consumption can be divided into two categories: energy for the preparation of the printing (E p ), and energy directly needed for the fabrication phase (E f ).

Cost assessment

The modeling of cost with environmental consideration helps framing decision-making during manufacturing [START_REF] Ruckstuhl | Design and responsible research innovation in the additive manufacturing industry[END_REF]. Many types of cost models exists regarding printing a single part, sometimes specific to a process or a material [START_REF] Freeman | Cost estimation of CNC vs composite reinforced FDM3D printing for low-volume production of motorcycle components[END_REF][START_REF] Lunetto | A comparative LCA method for environmentally friendly manufacturing: Additive manufacturing versus Machining case[END_REF][START_REF] Urbanic | A manufacturing cost analysis framework to evaluate machining and fused filament fabrication additive manufacturing approaches[END_REF][START_REF] Ingarao | A comparative assessment of energy demand and life cycle costs for additive-and subtractivebased manufacturing approaches[END_REF]. Still, six distinct cost categories can be identified: 1) materials (including supporting material); 2) electricity (e.g., warm-up, build-up, calibration); 3) workforce (software and hardware); 4) pre-and postprocess (including quality control); 5) indirect cost (e.g., investment, maintenance); and 6) consumables. The total cost C tot of manufacturing is estimated as follows:

C tot = C materials + C electricity + C workforce + C pre/post process + C indirect + C consumables (1)
One of the main advantages of AM compared to conventional manufacturing is complexity-forfree. AM does not rely on specific moulds or tools to manufacture part, which implies that the production cost does not rely on the complexity of the design, easing the production of lightweight parts [START_REF] Niaki | Why manufacturers adopt additive manufacturing technologies: The role of sustainability[END_REF].

Coupling aspects

Approach combining economic, energy and technical aspects to support decision-making are necessary to evaluate printing efficiency. Yosofi & al. [START_REF] Yosofi | Additive manufacturing processes from an environmental point of view-a new methodologuy for combining technical, economic, and environmental predictive models[END_REF] proposed a holistic approach for evaluating a part produced by an AM process regarding surface roughness, material, electrical and fluid consumption. Gutierrez-Osorio & al. [START_REF] Gutierrez-Osorio | Energy consumption analysis for additive manufacturing processes[END_REF] compared SEC, Young's modulus and tensile strength as a function of the layer thickness for different AM process. However based on literature review, a gap in comprehensive investigations on process energy consumption and print quality still exists.

Motivation for the present approach

In the literature, energy and cost assessments have mainly focused on the AM of polymer. In the case of composite materials, especially fiberreinforced polymers, the demand for high performance and high mechanical properties requires the produced parts to be assessed from a mechanical perspective. Hence, there is a need to analyze the technical, economic and environmental impact of printing parameters for the AM of fiberreinforced composites. The method we devised is based on a flow analysis of the manufacturing phase. This analysis yields an overview of the different parameters involved in the process, allowing optimization solutions to be explored. First, establishing the total manufacturing cost gives an indication of the parameters that can be modified and optimized from an economic pointof-view. Those modifications have impacts that need to be assessed from an environmental pointof-view. Finally, to ensure the industrial viability of the process, the impact on mechanical performance needs to be measured from a mechanical point-of-view.

3 Modelling approach

Flows

A cradle-to-gate representation of the material and energy flows involved in FFF of composite material is provided in Figure 1. Cradle-to-gate correspond to the lifecycle of the product, from the raw materials to the final product. This schematic shows that the process encompasses a wide range of subprocesses that are all interconnected and influence each other. There is still a lack of knowledge regarding the influence and the interconnection between each subprocesses, in terms of economic, environmental, and mechanical aspects. For a gate-to-gate approach, where only the pre-manufacturing and manufacturing steps are considered (in green on Figure 1), the storage of the material and file creations are the first steps. Regarding the FFF machine, manufacturing a part can be divided in two subprocesses : first, warming-up of the different heating elements (i.e., preparation subprocess related to E p ), and second, the deposition of the part (i.e., fabrication subprocess related to E f ).

Cost model

Coupling the flow model of Figure 1 with the previously identified cost categories enable to obtain a generic formula for the total cost (see variables definitions in Table 2):

C tot = C part × M part + C sup × M sup + C elec × (P p × t p + P f × t f ) + C op,hard × (t f + t post ) + C op,soft × t soft + C soft t soft + Cm α + M t hr × β × 1 t f + t p + N 1 C cons tcons Tcons (2)
The Eq. 2 consider the six categories established in Eq. 1, and the entirety of the process.

C man = C materials + C electricity + C consumables (3)
C man = C part × M part + C sup × M sup + C elec × (P p × t p + P f × t f ) + C printhead × t printhead T printhead + C heatbead × t heatbead T heatbead (4)
The Eq. 4 presents the different factors an operator can modify during the fabrication phase, namely the material, electricity, and consumables (here, printhead and heatbed). The material cost remains constant for the desired part design, and the consumables cost is only related to printing duration. Consequently, the electricity cost is the only variables that can be indirectly controlled by the operator through printing parameters. Divided into preparation (p) and fabrication (f) consumptions, those two phases depend on the power demand (P p , P f ) and the duration (t p , t f ). Reducing these variables through printing parameters (e.g., temperature) can reduce the total cost.

Energy model

In the context of transition into a more sustainable industry, it is particularly crucial to consider the impact of decisions on energy consumption. SEC is an interesting quantitative indicator of total electricity consumption during the manufacturing phase. Additionally, the power profile can give qualitative information about the distribution of the power demand. Figure 2 provides an example of power profiles plotted as a function of process time for PA12-CF. Six different process phases can be identified: 1) idle and file launching; 2) heatbed warm-up; 3) printhead warm-up; 4) calibration and printing; 5) end and return to 0; and 6) idle. Phases 2) and 3) are temperature (hence material) dependent while Phase 4) is design dependent, finally, Phases 1), 4) and 6) are operator dependent. The preparation energy corresponds to the sum of the switch-on, file launch, heating (heatbed and printhead), calibration, return to 0 and idle phases. While fabrication energy corresponds to the energy used to keep the elements warm and displace them. As shown in Figure 2, the terms for fabrication energy E f are directly given by the power demand in Phase 4), without the need to individually measure each displacement or heating energy term. Additionally, the energy consumption for the warm-up and E f are related to the temperatures of the printhead and the heatbed.

Mechanical performance

Carbon-fiber-reinforced polymer can be used in AM to improve material properties, thus, even with energetic consideration it is essential to control the quality of the part. In particular, if replacing a metallic part with a composite one means a drastic loss in mechanical performances, any reductions in cost and energy are irrelevant. Hence to ensure industrial viability, mechanical property needs to be assessed.

Experimental methods

Materials and equipment

We used a Raise 3D Pro2 FFF printer, an enclosed printer. To assess the mechanical properties of the printed part, we selected a tensile specimen Type A1, from standard ISO 527-2:2012. The filament material was CarbonX™ Nylon-CF Gen 3 from 3DXTech, with 240-270 °C and 80-110 °C as recommended temperatures from the manufacturer for the nozzle and the heatbed, respectively. Filament was stored in a vacuum oven to control humidity. We used a Fluke 289 multimeter to measure the electrical current during manufacturing. Mass of printed part are directly measured at the end of manufacturing. The voltage was measured upstream and assumed to remain constant throughout the experiment. Tensile tests were performed on Instron 1362 testing system with a 5kN load cell. Speed of testing is 5 mm/min, ultimate tensile resistance and Young's modulus are calculated following ISO 527-1:2012.

Design of experiment

To illustrate the impact of printing parameters on energy consumption and cost, as seen with Eq. 3, we selected two factors: printhead temperature, and heatbed heating strategy. Deposition orientation and infill were also considered, as they are common variables in FFF manufacturing. However, on one hand, the best tensile resistance for CRFP will always be delivered throughout the filament orientation without affecting energy consumption. On the other hand, for tensile specimen, an infill of 100 % must be chosen for optimal performance.

From the recommended temperature, we selected 245 °C, 250 °C, and 260 °C as our values for the nozzle temperature and the lower limit for the heatbed temperature 80 °C. To minimize energy consumption, one could select a temperature below the recommended temperature, however, capability of the print is not ensured anymore. For the second parameter, we chose three levels depending on the number of heated layers relatively to the total number of layers printed (in percentage): full heating of the heatbed (100%), heating stopped at half the total layers (50%), and no heating (0%). In total, we performed nine different tests, with four repetitions for each condition. Tables 3 and4 lists both fixed and variables printing parameters. In the case of Experiments #1 and #4, the adhesion between the part and the heatbed does not allow a proper deposition of the materials. This indicates threshold temperature conditions below which proper adhesion and deposition are difficult.

Conditions #1, #4 and #6 have the same printing parameters, except printhead temperature. Condition #1 can not be successfully printed, meaning a 245 °C printhead temperature coupled with 0 % heatbed are insufficient. For condition #4, T P = 250 °C and 0 % heatbed strategy are at the limit of the capability for the printer. Other conditions are within the range of capability of the printer and the variability is in terms of performance.

Results

Cost data

Total cost and cost distribution between materials, electricity and consumables are shown in Figure 3. Associated constant parameters are defined in Table 5. Materials represented the main cost, ranging from 74% to 94% of the total cost, depending on the conditions. As material consumption was roughly the same across all experiments, the absolute value of the material cost remained constant. It was therefore the differences in energy consumption that were responsible for the changes in total cost. From the conditions without heatbed heating to conditions with full heating, differences in power demand meant that the electricity cost represented between 5% and 20% of the total cost. As printing duration was equal for each experiment, the cost of consumables was not affected.

Energy data

Figure 4 shows the different results of SEC (all six phases described in Figure 2), and SPE (Phase 4 only). As expected, energy consumption increased as the temperature increased. The difference in energy was, however, smaller between 50% and 100% than between 0% and 50%.

Tensile test data

Each of the 36 tensile specimens was mechanically tested, without any post-process. Figure 5 show results of each experiment regarding Young's modulus and ultimate tensile strength (UTS).

Results revealed a substantial drop-off for experiments without heatbed heating (#4 and #7) on both Young's modulus and UTS. Young's modulus is relatively constant for experiments when heatbed strategy is 50 % or 100 %. Highest ultimate tensile strength is achieved for Condition #9, while Conditions #6 and #8 are within 5% drop-off. Those results can be explained, on one hand as, Conditions #6 and #9 corresponded to full heating of the heatbed. On the other hand, Condition #8 also corresponds to the highest printhead temperature and the drop-off in UTS from 100 % to 50 % heatbed strategy is limited. Similarly, the conditions with 50 % heatbed strategy exhibit a limited drop-off in UTS compared to their counterpart with 100 %. A lower ultimate tensile strength for other conditions (#2, #3, #4, #5 and #7) indicate that both printing parameters need to be optimized in order to achieved the highest resistance.

Discussion

The printhead temperature and the heatbed strategy were compared in three cost categories: materials, electricity, and consumables. The materials cost is the main source of cost and remained constant. Nonetheless, electricity cost is significant and was the most fluctuating of all three categories, ranging from 5 to 20% of the total cost. Moreover, electricity cost would increase for a longer duration of printing. The change in total consumption according to the different parameters was as expected reflecting a total energy that was more than four times higher when the heatbed was heated, increasing linearly across half and total heating. SEC and SPE underwent similar changes, as the temperature of the printhead and the heating strategy of the heatbed influenced both preparation power and fabrication power. Between the two parameters, the heatbed has the greater impact on energy consumption. Additionally, the unsuccessful printing in Condition #1 indicated a trade-off value or combination of the two parameters, determining the printability or otherwise of a part.

The results of tensile tests on the printed specimens also changed as expected: heating positively increased ultimate tensile strength. However, the change in strength between the different heating strategies was not linear. On one side a printing strategy of 0% for the heatbed considerably reduced tensile strength, 68% and 55% reduction compare to full heatbed strategy for printhead temperatures of 250 °C and 260 °C, respectively. On the other side, the difference between a 50% strategy and full heating was relatively small, with 5% for 245 °C, 9% for 250 °C and 4% for 260 °C. Similarly for Young modulus, there are a 69% and 52% decreases between full and no heatbed heating for T p = 250 °C and T p = 260 °C. Between full and half heating, the drop-off are 11%, 10% and 14%, respectively. Additionally, conditions with same heatbed strategy (#4 and #7; #2, #5 and #8; #3, #6 and #9) present values of Young modulus within 0.1 GPa differences.

Figure 6 present each conditions regarding three main indicators : manufacturing cost, SPE and ultimate tensile strength. Such radar charts ease comparison between different conditions. Condition #9 is the appropriate solution for a case where the highest mechanical performance is searched, as it exhibits the maximum UTS and Young's modulus. In the opposite case, conditions #4 and #7 are the cheapest and less energy consumingand are thus appropriate for cheap and energy scenario. Interestingly, the combination of 260 °C and half heating of the heatbed (condition #8) present a trade-off, being 4% less efficient on tensile resistance but offering a reduction of 28% in energy consumption and 3% in manufacturing cost. Depending on the targeted application, Condition #8 can thus be an appropriate compromise.

Conclusion

This work integrated emerging specific printing energy indicator to traditional cost and mechanical indicators in order to support decision-making in favor of more sustainable additive manufacturing. The aim of the present study was to develop an experimental method for gauging the impact of FFF printing parameters on three key indicators: manufacturing cost, specific printing energy, and ultimate tensile strength. Analysis of the flows involved in the process highlighted several economic variables as potential mean for optimization. To optimize the process in terms of cost and energy, both duration and power demand were identified as important variables. As duration is dependent on part design and desired mechanical strength, an optimization study would involve assessing both printing parameters and redesign. However, in the case of redesign the study would be case dependent thus harder to generalize.

By contrast, power demand is directly linked to printing parameters, making it much simpler to adjust for the operator. The present study therefore focused on a quick impact analysis of two parameters that influence the power demand for both the preparation phase and the printing phase: printhead temperature and heatbed heating strategy. With a relatively small design of experiments, an overview of all cost, energy and mechanical impacts can be estimated. Hence, the proposed methodology can be used to present solutions and strategies for optimizing the process, depending on the operator's requirements.

Future work would require to also investigate the duration variable, by evaluating parameters such as deposition speed and layer thickness. Moreover measuring the interactions between all the parameters could also give a more precise idea of the ideal combination of parameters, depending on the situation. Regarding cost evaluation, taking full account of Eq. 1 with machine cost, especially in the case of production series would provide a more exhaustive view. Finally, the tensile strength and Young's modulus are not the only mechanical or technical indicators needed, as printing precision and surface roughness are key indicators in AM and need to be investigated further. Other mechanical tests, such as fatigue or flexural tests, could also be implemented and would be compatible with the proposed methodology. would also like to thank Sarah Bensrhir for her help on uncertainty quantification and tensile tests.

Statements and Declarations

Fig. 1

 1 Fig. 1 Production line of FFF printing with flow representations (energy flows in orange, resources flows in blue, and consumables flows in purple). Green background represents the focus of the article on manufacturing; blue background is upstream process and red background is potential downstream.

Fig. 2

 2 Fig.2Power profile and identification of main printing phases. This profile corresponds to the printing of a specimen A1, PA12-CF at 260 °C printhead and 100% heating at 80 °C for the heatbed. Steps 1-3, 5 and 6 are preparation steps (green periods). Step 4 is fabrication (red period).

Fig. 3

 3 Fig.3The estimated manufacturing cost and cost distribution for each condition. Note : Condition #1 is not represented.

Fig. 4

 4 Fig. 4 Mean data for SEC and SPE for each experiment. Error bars correspond to 95% confidence interval. Note : Condition #1 is not represented.

Fig. 5

 5 Fig. 5 Comparison of tensile modulus and UTS for each conditions. Error bars correspond to 95% confidence interval. Note : Condition #1 is not represented.

Fig. 6

 6 Fig. 6 Radar charts for Conditions 2-9. Values are normalized, 100% being the optimum value (maximum for ultimate tensile strength and minimum for total cost and SEC).

Table 1

 1 Compilation of available data on mechanical properties of virgin PA12, PA12 by SLS and FFF, and PA12-CF by SLS and FFF. Note : NA for Non Applicable.

	Material

Process Filler Mechanical properties Source Tensile strength [MPa] Young modulus [GPa] Flexural strength [MPa] Flexural modulus [GPa] Tensile Elongation at break [%]

		Virgin	NA	53	1.8	68	1.7	200	Ensinger Plastics [16]
				43	1.4	44	1.2	350	Evonik [17]
	PA12	SLS	NA	-42.5	-0.86	53.9 68	1.54 -	-1.4	[18] [19]
				35	0.9	-	1.5	-	[20]
		FFF	NA	61	0.53	42	1.06	439	[21]
				54	0.94	32	0.84	-	[22]
		SLS	30% wt 33% wt	-66	-6.3	74 -	3 -	-5	[23] [24]
	PA12-CF		10% wt	94	3.6	125	5.26	8	[25]
		FFF	6% wt 35% wt	33.5 89	1.85 5.2	55.3 -	0.31 -	--	[26] [14]
			<20% wt	63	3.8	84	3.75	3	3DXTech [27]

Table 2

 2 Nomenclature of the different variables ant their associated definitions and unit.

	Variables	Definitions [Unit]	Variables	Definitions [Unit]
	C tot	Total cost [€]	C man	Cost for manufacturing scope [€]
	C part	Base material part cost [€/kg]	C sup	Support cost [€/kg]
	M part	Base material part mass [kg]	M sup	Support mass [kg]
	C elec	Base electricity cost [€/Whr]	t p	Preparation time [hr]
	P p	Preparation power [W]	t f	Fabrication time [hr]
	P f	Fabrication power [W]	C op,hard	Hardware operator cost [€/hr]
	C op,soft	Software operator cost [€/hr]	t post	Post-process time [hr]
	t soft	Software (slicing and CAD) time [hr]	C m	Machine cost [€]
	M	Maintenance cost [€]	α	Machine lifetime [hr]
	t hr	Annual working hours [hr]	β	Percentage of use time [%]
	N	Total number of consumables	C printhead	Printhead cost [€]
	t printhead	Use time of the printhead [hr]	T printhead	Printhead lifetime [hr]
	t heatbed	Use time of heatbed [hr]	T heatbed	Heatbed lifetime [hr]
	C heatbed	Heatbed cost [€]

However in most cases, the parameters cannot all be modified separately. With regards only on manufacturing scope, certain parameters are out of scope. In this case, the global equation (Eq. 2) is replaced by the gate-to-gate scope such as :

Table 3

 3 Printing parameters held constant throughout the experiment.

	Constant parameter	Value	Unit
	Deposition speed	80	mm/s
	Heatbed temperature (T H )	80	°C
	Layer thickness	0.2	mm
	Outline shell	1	-
	Infill percentage	100	%
	Nozzle diameter	0.4	mm
	Deposition orientation	+/-45	°Table
	4 Variables and design of experiment.
	Condition #	Printhead temperature T P [°C]	Heatbed strategy [%]
	1	245		0
	2	245		50
	3	245		100
	4	250		0
	5	250		50
	6	250		100
	7	260		0
	8	260		50
	9	260		100

Table 5

 5 Cost parameters held constant throughout the experiment. Note : prices were converted into Euros according to the exchange rate onMarch 9 2022.

	Category	Parameters	Value	Source
	Material	Base material part cost	52.78 EUR/500g Supplier
	Electricity	Base electricity cost	0.32 EUR/kWhr Mean from public data [44]
		Printhead cost	227.49 EUR	Supplier
	Consumables	Expected lifetime of printhead 1 year Heatbed cost 109.19 EUR	Estimation from experience Supplier
		Expected lifetime of heatbed	1 year	Estimation from experience

  • Funding : The authors gratefully acknowledge the financial support from the Natural Sciences and Engineering Research Council of Canada through Discovery Grants program -Individual (NSERC) (RGPIN-4566-2018 NSERC) and financial support provided by Brittany Regional Council for the mobility of the PhD student. • Conflict of interest/Competing interests : The authors have no relevant financial or nonfinancial interests to disclose. • Availability of data and materials : The public data used for "Base electricity cost" in Table 5, were derived from the following resources available in the public domain : https://mern.gouv.qc.ca/energie/statistiquesenergetiques/prix-electricite/ [44]. Any other data that support the findings of this study are available on request from the corresponding author. • Code availability : Not applicable • Ethics approval : Not applicable • Consent to participate : Not applicable • Consent for publication : Not applicable • Authors' contributions : All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Thibault Le Gentil, Daniel Therriault and Olivier Kerbrat. The first draft of the manuscript was written by Thibault Le Gentil and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript. The authors