Selene Leya

Héber Cerna Nahuis

Hwang Arcolezi

Christophe Guyeux

Guillaume Royer-Fey

Céline Chevallier

Selene Cerna

Héber H Arcolezi

Machine learning-based forecasting of firemen ambulances' turnaround time in hospitals, considering the COVID-19 impact

Keywords: Firemen, Turnaround time, Emergency medical services, Forecasting, COVID-19, Service breakdown

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

It is well known that the Emergency Department (ED) is one of the most crowded departments in hospitals [START_REF] Cone | Analysis and impact of delays in ambulance to emergency department handovers[END_REF][START_REF] Lee | Emergency department overcrowding and ambulance turnaround time[END_REF][START_REF] Segal | The in-hospital interval: A description of EMT time spent in the emergency department[END_REF][START_REF] Eckstein | Facilitating EMS turnaround intervals at hospitals 36 in the face of receiving facility overcrowding[END_REF][START_REF] Carter | Can emergency medical services use turnaround time as a proxy for measuring ambulance offload time?[END_REF], where people seek immediate attention given the severity of their problem and long lines are generated contributing to patient dissatisfaction. What is more, EDs' overcrowding can have a knock-on effect on other external services, such as ambulances. Theoretically, when an ambulance arrives at the hospital with patient(s), the crew transfers the patient's care to the ED staff, completes the necessary reports, and cleans and restocks the ambulance (e.g., replaces stretcher linens) to return on service. This total time an ambulance spends at the hospital for handing over patient(s) is referred to as turnaround interval [START_REF] Cone | A time-motion study of the emergency medical services turnaround interval[END_REF], which we interchangeably refer to as the turnaround time (TT) of ambulances throughout this paper.

However, with EDs' overcrowding, there is a problem referred to as ambulance offload delay (AOD), which occurs when ambulances' patients cannot be transferred for immediate care to hospitals' ED [START_REF] Cone | Analysis and impact of delays in ambulance to emergency department handovers[END_REF][START_REF] Li | A review on ambulance offload delay literature[END_REF]. On the one hand, AOD risks patients' life due to delays to receive adequate treatment and/or diagnosis, for example. Besides, AOD affects the emergency medical system (EMS) as their ambulance and staff will be in use for more time. That is, AOD directly increases the ambulances' TT, which poses a population at risk if other major incidents occur and they cannot attend to them [START_REF] Eckstein | The effect of emergency department crowding on paramedic ambulance availability[END_REF][START_REF] Lee | Emergency department overcrowding and ambulance turnaround time[END_REF][START_REF] Eckstein | Facilitating EMS turnaround intervals at hospitals 36 in the face of receiving facility overcrowding[END_REF][START_REF] Perry | The ethics of ambulance ramping[END_REF].

Fire departments are a key component of civil security that ensures the well-being of the population. In some countries such as France, they are also responsible for emergency medical services and have been facing a continuous increase in the number of interventions over the years, which represents a need for the acquisition of more resources and the optimal reorganization of them [START_REF] Arcolezi | Forecasting the number of firefighter interventions per region with local-differential-privacy-based data[END_REF].

In this study, we analyzed the Departmental Fire and Rescue Service of the Doubs (SDIS 25) that currently has 71 centers deployed throughout the Doubs region. In total there are 573 cities and 440 districts. Its main cities are: Besançon, its capital, where the nearest hospital is Besançon Regional University Hospital Center (CHRU), Montbeliard in the north, near the North Franche-Comté Hospital (HNFC), and Pontarlier in the south, near the Hospital Center Intercommunal de Haute-Comté (CHIH).

The SDIS 25 is equipped with specialized engines for each type of intervention and the firefighters have various skills in operating these engines. If there are not enough resources available for a long period to attend to an intervention, either because of the lack of firefighters, the lack of engines or both, a service breakdown occurs, i.e., the inability to assist within the time limits, which puts the safety of a certain area or population at risk. For instance, over the years, it has been observed that most of the breakdowns have occurred in July, since more people go on vacation during this time, including firefighters, i.e., there could be an increase in the number of interventions and with reduced personnel more breakdowns would be generated [START_REF] Cerna | Predicting fire brigades operational breakdowns: A real case study[END_REF]. If we add to this the impact of a natural phenomenon, epidemic or pandemic, centers would be more affected, as well as their resilience if resources are scarce.

On 12th January 2020, the World Health Organization (WHO) announced [START_REF]Novel coronavirus -china[END_REF] a novel Coronavirus [START_REF] Wang | A novel coronavirus outbreak of global health concern[END_REF][START_REF] Sohrabi | World health organization declares global emergency: A review of the 2019 novel coronavirus (COVID-19)[END_REF][START_REF] Li | Early transmission dynamics in wuhan, china, of novel coronavirus-infected pneumonia[END_REF][START_REF] Huang | Clinical features of patients infected with 2019 novel coronavirus in wuhan, china[END_REF][START_REF] Dairi | Comparative study of machine learning methods for COVID-19 transmission forecasting[END_REF][START_REF] Rahimi | A review on COVID-19 forecasting models[END_REF], which was officially named as 'COVID-19' (coronavirus disease 2019) [START_REF]World-Health-Organization, Naming the coronavirus disease (COVID-19) and the virus that causes it[END_REF] on the 11th of February.

Further, with the spread of COVID-19 across the world, it was declared as a pandemic [START_REF]WHO announces covid-19 outbreak a pandemic[END_REF] on 11th March 2020. In general, multiple measures have been reported for dealing with this novel Coronavirus across the world [START_REF] Chinazzi | The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak[END_REF][START_REF] Wu | Nowcasting and forecasting the potential domestic and international spread of the 2019-nCoV outbreak originating in wuhan, china: a modelling study[END_REF][START_REF] Anastassopoulou | Data-based analysis, modelling and forecasting of the COVID-19 outbreak[END_REF][START_REF] Grasselli | Critical care utilization for the COVID-19 outbreak in lombardy, italy[END_REF].

In France, as stated in [START_REF] Stoecklin | First cases of coronavirus disease 2019 (COVID-19) in france: surveillance, investigations and control measures[END_REF], strengthened surveillance of COVID-19 cases was implemented on 10th January 2020. In their paper, the authors carefully described the real-time surveillance system for the first three cases, which were detected on 24th January 2020. For the scope of this paper, the first official cases in the Doubs region have been reported on 18th March 2020 as shown in daily statistics published by Santé Publique France in [START_REF] Fr | Données hospitalières relatives à l'épidémie de covid-19[END_REF]. Nevertheless, the SDIS 25 early started to attend interventions in which patients had symptoms of the disease by 29th February 2020.

Although numerous measures were taken to secure peoples' health and wellbeing, the COVID-19 pandemic has proven to be quite a challenge, particularly to the public health sector [START_REF] Carroll | European and united kingdom COVID-19 pandemic experience: The same but different[END_REF][START_REF] Goniewicz | Current response and management decisions of the european union to the COVID-19 outbreak: A review[END_REF][START_REF] Grasselli | Critical care utilization for the COVID-19 outbreak in lombardy, italy[END_REF]. For instance, in several parts of the world, numerous news reported an increment to the ambulances' waiting time in hospitals. In the Texas county US, there are reports of ambulances waiting about 4-12 hours as hospitals are crowded with coronavirus cases [START_REF] Hendricks | With hospitals full of coronavirus patients, ambulances wait hours for beds to become available[END_REF]. There is also a report in France, in which ambulances had to wait about 3 hours in a hospital [START_REF] Homery | Quimper. l'hôpital, le calvaire des ambulanciers[END_REF]. This increment could lead sometimes to the worst-case scenario, i.e, patients' death, as every minute count in such a situation. This is a case, which was reported in Australia, where two patients have died for long waiting time in ambulances outside hospitals [START_REF] Dow | Patient deaths after wait in ambulances lead to hospital probe[END_REF].

Description of the intervention process

The process of dealing with an intervention by SDIS 25 is briefly described as follows:

(a) First, an emergency call is received and the required armament is gathered to go to the scene.

(b) Once arrived at the scene, if necessary, the victim is taken to the hospital.

(c) At the hospital, the firefighters wait to transfer the victim to the hospital and to do the corresponding administrative process.

(d) Finally, they return to their center and are available again to attend other interventions.

Each performed step requires a record of the firefighter's status, which is done manually by pressing a button. The firefighter crew chief makes a record to indicate that they are going to the hospital, another record to indicate when they arrive at the hospital, and another one to indicate that they are leaving it. However, there is the possibility of human error, where the firefighter may have forgotten to record the status and registered it a long time later, or on the contrary, where the firefighter may have accidentally recorded too soon.

In addition, an intervention can have several ambulances departing at different times from the scene, depending on the number of victims, or either, sometimes in the same hour we can have several simultaneous interventions but of different types. If firefighters spend more time in hospitals, i.e., high TT, there will be fewer resources available at the centers to respond immediately to an incident. Notice that the terms ambulance and firefighter's ambulance are used interchangeably throughout this paper.

Purpose and contributions

With these elements in mind, the present work proposes a novel methodology based on machine learning (ML) to make predictions for the TT of each ambulance in a given time and hospital, aiming to provide a decision support tool for SDIS 25 and EMSs, in general. This way, EMSs can activate various proactive mediations according to the time their personal and resources will be in use, aiming to mitigate service breakdown and, consequently, being able to save more lives. In other words, for the short-term, such predictions could allow better allocation of the remaining and available resources if known a priori the time each ambulance will spend in hospitals. For the medium-to long-term analysis, such a system can be improved to re-calculate a better hospital option considering the predicted TT. To summarize, this article proposes 3 main contributions described in the following. LGBM model consistently and considerably outperformed the other ML models, for this task of predicting each ambulance's TT (not average TT per hour), we performed experiments with only LGBM.

To the authors' knowledge, this is the first work that investigates the impact of COVID-19 on ambulances' TT. With our findings, we noticed that SDIS 25 ambulances' TT increased during the outbreak, which led to more breakdowns in service due to the shortage of ambulances that would normally return to service in less time. This way, aiming at providing a decision support tool for SDIS 25 and EMSs, in general, this paper also proposes a novel ML-based methodology to forecast the TT of each ambulance in a given time and hospital.

More precisely, considering that ambulances can arrive at hospitals anytime, our problem (c) refers to irregularly spaced time series [START_REF] Hyndman | Forecasting: principles and practice[END_REF], i.e., the spacing of observation times is not constant (e.g., per day). Thus, we first propose to reconstruct a regularly spaced time series (i.e., AvTT per hour), which gives us a reference on the average time that several ambulances could have been waiting, and thus, recognize a daily or weekly seasonality. Since not every hour nor every day SDIS 25 ambulances went to hospitals, a linear interpolation was applied to complete the dataset. This is because SDIS 25 is not the only institution to transport victims to hospitals. Finally, besides the prediction of AvTT per hour/hospital (b), with the second model (c), we can refine the predictions for each ambulance, i.e., an irregularly spaced time series problem.

The present paper is organized as follows. Section 1.3 reviews contributions from related works. Section 2 details the construction of datasets (Subsection 2.1); the data analysis performed to understand the normal and abnormal behaviors in the collected data (Subsection 2.2); the breakdowns generated due to long TT periods (Subsection 2.3); the proposed methodology composed of two forecasting models and their baselines (Subsection 2.4); finally, we describe the metrics used to evaluate our models (Subsection 2.6). Section 3 presents the results of the predictive models. Section 4 discusses this research and its impact in fire brigade services. Finally, Section 5 shows our conclusions and future work.

Literature Review

ED overcrowding has been reported as one of the main causes of AOD [START_REF] Eckstein | Facilitating EMS turnaround intervals at hospitals 36 in the face of receiving facility overcrowding[END_REF][START_REF] Cone | Analysis and impact of delays in ambulance to emergency department handovers[END_REF][START_REF] Segal | The in-hospital interval: A description of EMT time spent in the emergency department[END_REF][START_REF] Carter | Can emergency medical services use turnaround time as a proxy for measuring ambulance offload time?[END_REF] since ED staff may not prioritize ambulances' patients. AOD may lead to several risks to patients' care and directly increases the ambulances' TT in hospitals, which puts a population at risk if other major accidents occur and ambulances are still in use. In [START_REF] Eckstein | The effect of emergency department crowding on paramedic ambulance availability[END_REF], the authors conducted a prospective longitudinal study to analyze the impact of ED overcrowding on firefighters' AOD. The study identified 21,240 cases where their ambulances were out of service due to AOD, which may have a significant effect on their ability to respond to other calls. As noticed in the survey work on AOD from Li et al. [START_REF] Li | A review on ambulance offload delay literature[END_REF], the impact of AOD on EMS resource availability has received less attention from the research community.

In [START_REF] Cone | A time-motion study of the emergency medical services turnaround interval[END_REF], the authors coined the term turnaround interval for the total time an ambulance spends at the hospital, i.e., transferring patient(s)' care, completing paperwork, and reestablishing the equipment for the next call. The authors conducted a prospective study by analyzing firefighters' ambulances' activity on transferring 122 patients to a hospital. In [START_REF] Clarey | Ambulance handovers: can a dedicated ED nurse solve the delay in ambulance turnaround times?[END_REF], the authors designed a discrete event simulation model to evaluate the change on AOD by having dedicated ED nurses for ambulances' handover. Although the authors identified that this practice may reduce the ambulances' TT, this would also lead to low staff utilization. In [START_REF] Vandeventer | The association between ambulance hospital turnaround times and patient acuity, destination hospital, and time of day[END_REF], the authors investigated the relationship between ambulances' TT with patients' acuity, destination hospital, and time of the day, using oneyear data from 61,094 patients. In [START_REF] Lee | Emergency department overcrowding and ambulance turnaround time[END_REF], the authors have analyzed the impact of ED overcrowding and ambulances' turnaround interval.

Notice that most aforementioned works have been performed to highlight the importance of the topic and to identify the relationship between ED overcrowding and ambulances' TT, for example. To the authors' knowledge, there is no research trying to forecast the TT of ambulances in hospitals, as we present in this paper, but rather for the prediction of patients' waiting time in ED [START_REF] Ding | Characterizing waiting room time, treatment time, and boarding time in the emergency department using quantile regression[END_REF][START_REF] Sun | Real-time prediction of waiting time in the emergency department, using quantile regression[END_REF][START_REF] Kuo | An integrated approach of machine learning and systems thinking for waiting time prediction in an emergency department[END_REF].

On the other hand, within the context of COVID-19, multiple works have investigated machine-learning-based solutions to help to fight the outbreak, e.g., forecasting of ED volume [START_REF] Lo | Prediction of emergency department volume and severity during a novel virus pandemic: Experience from the COVID-19 pandemic[END_REF] and forecasting the number of confirmed COVID-19 cases [START_REF] Arunkumar | Forecasting the dynamics of cumulative COVID-19 cases (confirmed, recovered and deaths) for top-16 countries using statistical machine learning models: Auto-regressive integrated moving average (ARIMA) and seasonal auto-regressive integrated moving average (SARIMA)[END_REF][START_REF] Ribeiro | Short-term forecasting COVID-19 cumulative confirmed cases: Perspectives for brazil[END_REF][START_REF] Devaraj | Forecasting of COVID-19 cases using deep learning models: Is it reliable and practically significant?[END_REF][START_REF] Da Silva | Forecasting brazilian and american COVID-19 cases based on artificial intelligence coupled with climatic exogenous variables[END_REF]; we refer the readers to recent survey works on ML forecasting models and COVID-19 in [START_REF] Dairi | Comparative study of machine learning methods for COVID-19 transmission forecasting[END_REF][START_REF] Rahimi | A review on COVID-19 forecasting models[END_REF].

In research directly related to firefighters and their processes, Pirklbauer and Dieter [START_REF] Pirklbauer | Predicting the category of fire department operations[END_REF] proposed a data-driven forecasting model for the type of firefighter interventions. The models were built using two-years data, which achieved a preliminary accuracy of about 61%. On the other hand, our group has proposed a multi-forecasting model to the number of firefighter interventions per region in [START_REF] Arcolezi | Forecasting the number of firefighter interventions per region with local-differential-privacy-based data[END_REF]. In that work, it was noticed that while the number of interventions is increasing each year, fire departments' resources are reduced. For this reason, there is a need to optimize the use of firefighters' personnel and resources, which could relieve the pressure on this emergency medical transport system. The complete methodology also allows a privacy-preserving publication/sharing of interventions' location data, where the Extreme Gradient Boosting technique was used to forecast the number of interventions per region each day with both original and privatized data. Also, it is necessary to know the current status of the fire department's resources and services in order to develop long and shortterm reorganization strategies. Thus, in [START_REF] Cerna | Predicting fire brigades operational breakdowns: A real case study[END_REF], a methodology was created for the calculation of service ruptures, which allows identifying the types and quantity of ruptures, as well as the unavailability of resources at a given time. In addition, with breakage data calculated in the last 3 years, predictive models were built for the number of daily breakdowns, considering the number of firefighters and available engines as the main variables.

Material and Methods

In this section, we describe the collection process of internal and external variables. Besides, we analyze the recorded ambulances' TT through the years and the impact of COVID-19 on ambulances' TT; and, consequently, the impact of COVID-19 and high TTs on the fire service. Further, we present our proposed methodology for forecasting the TT of each ambulance and AvTT per hour, and the metrics used to evaluate our models.

Data Collection

The main source of data comes from the Departmental Fire and Rescue Service of the Doubs department, France. It contains a history of ambulances arrivals collected from January 2015 to June 2020 for CHRU and CHIH hospitals, and from February 2017 to June 2020 for HNFC. For each hospital, two types of datasets were created:

• Arrivals dataset (Arr-DS). The samples represent the TT of each ambulance in a hospital at any time of the day. The features are: year, month, day, day of the week, day of the year, and hour. Also, an indicator to recognize if it is the beginning or end of the week, month, and year. In addition, it was added three variables that jointly describe the type of intervention, where each variable represents a set of subcategories. The first variable represents the 5 main activities that firefighters cover, for example: rescue people, fires, among others. For each activity, there are subcategories that describe the level of urgency (second variable), for example, in rescue people, we found 6 subcategories, two of them are "Emergency situation" and "Special Circumstances of the emergency (public road)". The third variable reports the state of the victim, for example, in "Emergency situation", there are 10 subcategories, two of which are "Severe exteriorized or external hemorrhage" and "Respiratory distress". Finally, the TT in minutes (min) recorded was included. In some records, we found TTs less than 5 minutes, which is very rare, and it is possible that they were caused by a bad manual recording. Therefore, if the TT is less than 5 minutes, we clipped it at 5. Similarly, there were five unusual samples with TT between 4-7 hours, which were clipped at 90 minutes, since most TTs (99.7%) had less than or equal to 90 minutes.

• Average turnaround time dataset (AvTT-DS). This dataset has been organized by hour, where each hour represents the average TT of all ambulances that arrived at the hospital in that hour. For example, if between 15h and 16h there were 3 ambulances with TTs: 20 min, 15 min, and 30 min, the mean was 21.67 min for 15h. In addition, each sample includes the following features: year, month, day, day of the week, day of the year, hour, an indicator to recognize if it is the beginning or end of the week, month, and year. And, since 90% of all hospital arrivals have had an AvTT between 10 and 60 minutes, we set the average minimum and maximum with these values. The hours in which there were no arrivals and, therefore, there was no recorded TT and AvTT, were completed by linear interpolation since the TT over the years has been constantly increasing.

In order to discover possible influential external variables, we built another dataset (External-DS) by hour from "01/01/2015 00:00:00" to "30/06/2020 23:00:00", where we incorporated variables from the following sources:

• Google Trends. We used the Pytrends1 library to get the hourly trends in a scale from 0 to 100 for the keywords in french: 'H1N1', 'coronavirus', 'SARS', 'Influenza', 'COVID-19', 'diarrhee' (diarrhea), 'grippe' (flu), 'varicelle' (chickenpox), 'incendie' (fire), 'inondation' (flood), 'greve' (strike), 'samu', and 'suicide'.

• Bison-Futé [START_REF] Bison-Futé | Les prévisions de trafic[END_REF]. This source gives us the prediction of the traffic level for the Doubs region, as indicators from 1 to 4, where 1 means a regular circulation and 4 means an extremely difficult circulation.

• Météo-France [START_REF]Données publiques[END_REF]. It provides us with historical weather information such as: precipitation, temperature, barometric trend, pressure, humidity, dew point, wind direction, wind speed and gust speed.

• Data Gouv [START_REF] Fr | Données hospitalières relatives à l'épidémie de covid-19[END_REF]. It is a platform for the diffusion of public data from the French government, from which we extracted information on the situation of the COVID-19 pandemic from March 2020 to June 2020 to the region of Doubs (Department 25). Data prior to this date were completed with zeros, as there were no statistics reported.

Other variables included from the primary source were the total number of interventions recorded in a given hour over the 6 years, and the number of cases attended with a suspect of COVID-19 per hour from February 2020 to June 2020. This last variable was completed with zeros before February 2020 too.

Data Analysis

The dataset at our disposal has 78, 777 interventions where firefighters transported victims to one of the three aforementioned hospitals. The frequency on the number of times each hospital received a victim is 55.44% (CHRU), 18.27% (CHIH), and 26.29% (HNFC), respectively. In order to understand how the ambulances' TT are distributed in our dataset, Figure 1 illustrates, for each hospital, a histogram with bins of 1 minute and the cumulative number of TT in hours (y-axis) for each day of the week and hour in the day (x-axis).

In the first column of Figure 1, CHRU, CHIH, and HNFC have right-skewed distributions with mean and standard deviation (std) values as 18.43 ± 10.68, 14.91 ± 8.66, and 22.61 ± 11.26, respectively. In the second column of Figure 1, one can notice a similar pattern for the cumulative sum of TT in hours that firefighters' ambulances spend in the three hospitals with different peak values, which depends on the frequency of times each hospital received victims. This pattern is also noticed in the works [START_REF] Vandeventer | The association between ambulance hospital turnaround times and patient acuity, destination hospital, and time of day[END_REF][START_REF] Lee | Emergency department overcrowding and ambulance turnaround time[END_REF]. In our case, from 8h in the morning on, the TT starts to increase and remains high up to 19h when it starts to decrease. Also, between 0-6h, the highest cumulative TT is during the weekend, On the left side, the frequencies for the three hospitals follow a positively skewed distribution, where more than 80% of cases were less than one hour. On the right side, we can deduce that between 8h and 22h the accumulated time follows the intense human activity while awake, which is translated as more workload and longer TTs.

i.e., Friday, Saturday, and Sunday. This is because at weekends people tend to go out more and until the wee hours of the morning, which might lead to more accidents, more patient visits to hospitals during on-call hours with reduced staff, and results in a slight increase in ambulances' TT.

In addition, as stated in the introduction (Subsection 1.2), this work aims to study the COVID-19 impact on ambulances' TT in the three aforementioned hospitals. First, Table 1 exhibits for each semester (Sem.) of the analyzed years (2015-2020) and hospital, the following statistics: the total number of arrivals (Arr.) and the mean±std TT values in minutes.

As one can notice in Table 1, CHIH has fewer arrivals and a shorter AvTT per semester whereas CHRU presents more arrivals than the other two hospitals since it is located in the capital of the territory, and a lower AvTT than HNFC.

Besides, when looking at the number of arrivals for the first semester, it can be detected that there was a higher workload in the firemen department for the years 2018 and 2019 compared to the other years. In the case of 2020, the reduced workload was due to a lockdown period2 , which decreased the movement of people and reduced the number of firefighters' interventions, for example, fewer traffic accidents. The second semester of each year normally presents a higher AvTT, which could be due to two holiday periods (Jul-Aug and Dec),

where people travel more due to vacations. Although the AvTT normally increased for each hospital throughout the years according to its workload, the AvTT for 2020 is higher than all years and for the 3 hospitals, even with a reduced workload compared to 2018 and 2019. For instance, hospital CHRU presented about 9.13% more AvTT in the first semester of 2020 than in the first semester of 2019, which is 9 times higher comparing the increment for the same period for the years 2019 and 2018 (1%). Similarly, hospital HNFC presented about 6.54% more AvTT in the first semester of 2020 than in the first semester of 2019, while the years 2017-2019 presented similar AvTTs. This proves the impact of the pandemic on the increase in ambulances' TT.

Furthermore, we now try to comprehend how the on-going pandemic has made an impact on the ambulances' AvTT per day on hospitals for 2020 only.

For this purpose, the plot on the top of Figure 2 In Figure 2, while the first day with suspicious cases of COVID-19 is 29/02/2020 reported by the SDIS 25, the date with official cases is almost three weeks later, on 18/03/2020. However, one can notice that as soon as the SDIS 25 starts to take a significant number of people with the symptoms of the disease to hospitals in March (after 10/03/2020), the AvTT per day starts to increase for CHRU and HNFC. This increment is more remarkable, for the three hospitals, after the first official cases have been reported. Furthermore, the peak period of hospitalized cases occurred approximately from the middle to the end of April.

In such conditions, there were more hospitalized patients in which some of them started to need more and even intensive care as the number of cases in reanimation indicates. When comparing the AvTT distributions of the 3 hospitals and the number of discharged cases (Ret. home), we find an inverse correlation, given that as hospital resources are freed up, AvTTs decrease. The opposite case can be found when comparing the AvTT of the 3 hospitals and the number of hospitalized cases (Hosp.), where the 4 distributions follow a similar pattern, indicating that there is a positive correlation between the variables, i.e., as the number of hospitalized patients increased, the hospitals became saturated and their response time to receive new patients was affected. This generated larger TTs for the ambulances, leaving them unavailable to attend other interventions and limiting the resources of the fire brigade. In short, a chain effect on the availability of resources.

To complement Figure 2, Table 2 exhibits the AvTT per hospital and month for 2020 highlighting in italic font the highest value per hospital. Comparing the highest AvTT and the lowest one per hospital, considerable increments were noticed. For instance, hospital CHRU presented about 24.96% more AvTT in April than in January, hospital CHIH present about 42.68% more AvTT in April than in February, and hospital HNFC presented about 29.13% more AvTT in May than in February. In general, for each hospital, the period pre-pandemic (January and February) presented low AvTT while months March, April, and May (during-peak-pandemic) had peak-values, which is in accordance with the number of hospitalized cases and reanimation ones in Figure 2. Subsequently, in June (post-peak-pandemic) and for each hospital, the AvTT had an intermediate value between the two aforementioned periods. This can be due to the increasing number of patients that returned home and did not need special care as the 'ret. home' curve in Figure 2 indicates. Proportionally, the number of hospitalized and reanimation cases started to decrease, which alleviates the healthcare system. With these elements in mind, it is now evident that the COVID-19 pandemic has affected the ambulances' TT in hospitals of Doubs-France. This increment might affect the SDIS 25 service to the population as their staff and resources would be in use for more time. That is, as a chain-like effect, this would increment the number of breakdowns for the fire brigade when needing to prepare for future interventions, as presented in the work [START_REF] Cerna | Predicting fire brigades operational breakdowns: A real case study[END_REF]. In the next Subsection 2.3, we present and discuss an analysis of the service breakdown during 2020 to validate the negative impact of the COVID-19 pandemic to SDIS 25.

Service Breakdown Analysis

Besides an in-depth analysis of the COVID-19 impact on ambulances' TT in hospitals of Doubs France, we also present an study on the chain-like effect that high TT had on the fire service. Following the work in [START_REF] Cerna | Predicting fire brigades operational breakdowns: A real case study[END_REF], we applied the methodology for the breakage calculation for the months of March to June 2020, considering only the public service breakdowns of the type "Rescue People".

Public service breakdowns occur when there are no adapted engines, firemen, or both in the centers. This means that no adapted armament is available for a certain period of time to respond to an incident. In more detail, Figures 4 and5 illustrate the causes of the breakdowns during the first semester of the years 2019 and 2020, respectively. One can notice that for January and February of both years, the results are close. However, in March 2020 there is notable increment in the number of breakdowns due to lack of engines, which is the starting month with increasing COVID-19 cases. On the other hand, April and May show a lower number of breakdowns, which is because there were fewer interventions during the lockdown period 2 as people spent more time in their homes. However, according to Figure 3, the breakdown time was longer, as the available adapted engine spent more time in hospitals. Also, from March to June 2020, it can be seen that there were few or no breakdowns due to the lack of firefighters. The reason behind this is that firefighters and especially volunteers were at home and showed more availability, and there were not as many simultaneous interventions as in 2019.

For this reason, this study aims at developing ML models for forecasting the TT of ambulances. Such knowledge is paramount for EMSs, as they could service as previously analyzed. Such data-driven systems should be of high confidence in order to adequately assist in decision-making solutions in real-life.

Moreover, these systems should also be robust enough to abnormal situations such as natural disasters or even the current COVID-19 pandemic.

Proposed Methodology

Overview

The present research proposes a new methodology based on ML techniques for predicting the TT of each ambulance related to a hospital. The approach is composed of two time series models (i.e., regular and irregular). Figure 6 describes the interaction of both models that are created in two stages as explained in the following: Stage 1: Predicting the average turnaround time per hour at each hospital. To predict the TT for an ambulance, a valuable input would be the approximate waiting time that patients may be having in a hospital. However, we do not have internal hospital data to make this prediction. Also, we may consider as entry the AvTT in a given hour for ambulances from different public and private organizations. However, we do not have a record of all of them or their flow.

But, what we do have is the history of ambulances arrivals over the years of the fire department of Doubs. So, from this history we generated a regularly spaced time series per hour, which is the AvTT-DS. In this way, it is possible to capture trends and seasonality over time, for example: over the years the AvTT per hour has been increasing, during the day the AvTT is higher than at night, etc. Then, we developed a predictive model of AvTT for the next hour, considering external variables from External-DS, that can influence long waiting periods.

Stage 2: Predicting the turnaround time for an ambulance. This stage involves the construction of an irregularly spaced time series model, using the Arr-DS generated from the arrival history, it contains temporal characteristics and the type of intervention. Likewise, we included the External-DS and the AvTT predicted in the first stage. The objective is to forecast the TT of a specific ambulance, that is, at the moment an ambulance warns that it will go to the hospital, we make the prediction to know its approximate TT in a certain hospital. In this way, firefighters will be able to establish better strategies with their resources.

To evaluate the proposed methodology, the months April, May, and June 2020 were selected as the testing set for each hospital, since these are the months with a higher number of COVID-19 cases (peak-pandemic period), as previously analyzed in Subsection 2. was a rolling-origin evaluation [START_REF] Hyndman | Forecasting: principles and practice[END_REF][START_REF] Tashman | Out-of-sample tests of forecasting accuracy: an analysis and review[END_REF] (a.k.a. forward chaining). This is a more realistic approach to re-train time series models with data that becomes 480 available each time (e.g., hour or day) for further predictions. More specifically, our models are first fitted with the training set (with data until March 2020) for each hospital, making predictions one-step-ahead for all the hours of a day with the AvTT model, and per ambulance departure with the TT model for a day. This way, for all days in the testing set (April, May, and June 2020),

485

the training set is expanded to include all the known values of each day and the process is repeated, i.e., the training set is updated day by day. Besides, all the predictions of each model (i.e., the three months selected as the testing set) in an iteration are stored, and in the end, we compute the models' metrics.

For the hyperparameter tuning process, we used the root mean square error as the guiding metric. Thus, a new hyperparameter configuration is generated and the search process is repeated during a given number of iterations. The pre-processing and modeling of each model, AvTT and TT, are described in the following two subsections.

AvTT model: pre-processing and modeling

The dataset for the regularly spaced time series model, that will forecast the AvTT per hour, was organized as follows:

• We took the AvTT-DS and for each sample we added 3 moving averages of AvTT as features, using a window size of 2 hours back.

• Also, the AvTT of the last 24 hours were added as features.

• Finally, we complete the dataset with the features of the External-DS, according to the previous hour, since in real life these are data that we can obtain previously.

The structure can be seen in Figure 7, where each line represents a sample by hour with its identifier (Hour ID), and the columns illustrate the predictors and the target, in yellow and orange, respectively. Next, all features were standardized using Scikit-Learn's [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF] StandardScaler function, except by the target, which is the AvTT per hour in minutes.

In the search for the best model that gives us more accurate results, we compared the performance of various intelligent approaches such as traditional neural networks (MLP and LSTM), decision trees (LGBM), and a framework specifically oriented to time series (Prophet). Each technique is briefly described as follows:

• Light Gradient Boosted Machine (LGBM) [START_REF] Ke | Lightgbm: A highly efficient gradient boosting decision tree[END_REF]. It is a novel gradient boosting decision tree algorithm, which uses the leaf-wise tree growth strategy to significantly reduce calculation speed and memory consumption. • Multilayer Perceptron (MLP). It is an artificial neural network of the feedforward type, since the information flows through several neurons organized in interconnected layers classified as: input, intermediate (which can be more than one) and output [START_REF] Goodfellow | Deep Learning[END_REF]. In this study, we used the implementation developed by Scikit-learn library, where the activation function considered was Rectified Linear Unit (ReLu), and the optimization algorithm for learning information was Adaptive Moment Estimation (Adam) [START_REF] Kingma | A method for stochastic optimization[END_REF].

• Long Short-Term Memory (LSTM) [START_REF] Hochreiter | Long short-term memory[END_REF]. It is a type of recurring neural network that overcomes the vanishing gradient problem. Inside its cell memory unit, the learning process is controlled by three gates: input, forget and output, which give it the ability to forget part of the previous memory and add new information. In this study, we used the Keras library [52] to build our own architecture.

• Prophet [START_REF] Taylor | Forecasting at scale[END_REF]. It is a forecasting tool for time series data, where trends are fit with a certain seasonality, depending on the additive or multiplicative model selected. Furthermore, it allows the addition of changepoints such as holidays, and is robust enough to missing data.

To navigate the hyperparameter space and pick the most optimal set, it was used the HYPEROPT library [54] with 50 iterations for each technique described previously. This library is based on Bayesian optimization and the selected logic was the algorithm Tree Parzen Estimator suggest (tpe.suggest), which instead of modeling the probability of an observation given a new configuration, models two density functions according to a defined percentile, to finally estimate the expected improvement of a given run. The root mean square error, described in Subsection 2.6, was used as a metric to guide the search for the best configuration per technique.

TT model: pre-processing and modeling

The dataset for the irregularly spaced time series model, that will predict the TT for each ambulance, was constructed as follows:

• We took the Arr-DS and added the features of the External-DS, according to a previous hour, since these features come from sources that export their data online and by hour.

• Since the AvTT model runs earlier, it will give us the average prediction for the current hour, in which an ambulance is on the scene and reports that it will go to the hospital. In this way, the predicted AvTT will be used as a predictor when we forecast the real turnaround time of a specific ambulance. In addition, the AvTT per hour of the last 48 hours were added in order to improve the model's ability to recognize daily seasonality.

The structure can be seen in Figure 8, where each line represents an ambulance departure to the hospital with its identifier (Departure ID), and the columns in yellow and orange exemplify the predictors and the target, respectively. Next, all features were standardized using Scikit-Learn's StandardScaler function, except for the three variables that describe the type of intervention, which are categorical and were encoded using One-Hot-Encoding (OHE) function from Scikit-Learn library too. The target, which is the TT of an ambulance, was kept in its original format (minutes). To build the time series models, it was chosen the LGBM technique, because of its robustness and execution speed. To find the best set of hyperparameters, we performed 100 iterations using tpe.suggest from HYPEROPT technique. The loss function used was the root mean square error described in Subsection 2.6.

Baseline models

Two straightforward baseline models are used to compare the effectiveness of the proposed methodology. The first baseline is to compare with the multivariate regularly spaced time series model from Subsection 2.4.2. The second baseline is to compare with the multivariate irregularly spaced time series model from Subsection 2.4.3.

Baseline: AvTT per hour

The baseline model to predict the AvTT per hour (BSAvTT) uses the Arr-DS, where the TT of ambulances until the previous day are averaged by hour, and these values are used as a prediction for each hour during the current day.

At the end of each day, if there were ambulances transporting victims to the hospital, the average for each hour is updated.

Baseline: TT for an ambulance

The baseline model for predicting the TT of an ambulance at a given hour and hospital (BSTT) considers adding the External-DS features, according to a previous hour, to each ambulance in the Arr-DS. In this solution, we do not consider as features any AvTT per hour of previous or predicted hours.

Similar to the standardization described in Subsection 2.4.3, predictors were transformed into features using the StandardScaler and One-Hot-Encoder from Scikit-Learn's library. The target kept its original value in minutes. The technique used for the modeling was LGBM and for the optimization of hyperparameters was HYPEROPT with its tpe.suggest algorithm, 100 iterations, and the root mean square error as the loss function described in Subection 2.6

Metrics

Let y i be the real output, ŷi be the predicted output, and n be the total number of samples, for i ∈ [1, n]. In this paper, the models were evaluated using the metrics described in following.

• Mean absolute error (MAE): MAE measures the averaged absolute difference between real and predicted values and is calculated as:

M AE = 1 n n i=1 |y i -ŷi |.
• Root mean squared error (RMSE): RMSE measures the square root average of the squares of the errors and is calculated as

: RM SE = 1 n n i=1 (y i -ŷi ) 2 .
• Accuracy with margin of error ±10 (ACC10): ACC10 is the ratio of number of correct predictions, with a maximum margin of error ±10, to the total number of input samples. Let f (y i , ŷi ) be a function that f (y i , ŷi ) = 1 if ŷi ∈ {y i ± 1, y i ± 2, ..., y i ± 10} and 0 otherwise. ACC10 is calculated as:

ACC10 = 1 n n i=1 f (y i , ŷi )
• 100(%). For example, if y i = 35 and ŷi = 28 or ŷi = 41, f (y i , ŷi ) = 1 in both cases.

Results

In this section, it is first analyzed the results for the time series model to predict AvTT per hour and hospital (Subsection 3.1). Further, we analyze results for the main goal of this work, i.e., to predict the TT for each ambulance (Subsection 3.2) transporting victims to hospitals.

Forecasting the average turnaround time of ambulances per hour

To predict the AvTT of ambulances per hour and hospital during April, May, and June 2020, respectively, the baseline BSAvTT and four ML-based models were evaluated, namely LGBM, MLP, LSTM, and Prophet. Table 3 presents the metrics, discussed in Subsection 2.6, for each model and hospital, where the best results are highlighted in italic font. Both MAE and RMSE metrics express model prediction error in units of the variable of interest AvTT (minutes).

In Appendix A, Table A.5 indicates the range of each hyperparameter we considered in the Bayesian optimization, as well as the best configurations used to train and evaluate the models. The search space for each technique was established with previous empirical experiments that helped to limit the search and define the most influential hyperparameters. The latter are shown in the table for each technique while those that do not appear keep their default values.

In the case of LGBM, the maximum number of boosted trees was 1000 with a maximum depth of 12, and subsets of samples and features were greater than 50% of the complete dataset. In the case of neural networks such as MLP, 2 dense layers were established, where the number of neurons varied for each layer.

Unlike LSTM, where the number of layers and neurons were initially defined, and the variations occurred at the batch size and learning rate level. In the case of Prophet, the default number of n changepoints is 25, and it was modified to vary up to 100 in order to recognize the changes in the trends of the TTs over hours, days, weeks, months and years, Similarly, varying the seasonality prior scale allows us to experiment with different sizes of fluctuations over time.

Moreover, to highlight the effectiveness of the techniques in some peak values, Figure 9 illustrates for each hospital and two days (48h), the comparison of each ML model to forecasting the AvTT of ambulances per hour in different days. Lastly, Figure 10 illustrates for each hospital, the time series results for the period with more breakdowns in service (last weeks of April and the first weeks of May 2020), according to Figure 3, during the COVID-19 peak-period.

This figure considers only the LGBM model, according to the best results of Table 3, and the baseline BSAvTT.

One can notice in One can notice in Figure 9 that ML-based models follow the AvTT hourly trend, as well as some peak values. In Figure 10, while LGBM presents accurate prediction results for the AvTT of ambulances for all three hospitals, BSAvTT presents poor predictions following a similar pattern for them through the days. Due to COVID-19, the TTs of ambulances increased during some periods (cf.

Figure 2), and therefore, using only the average per hour resulted in poor performance.

Forecasting the turnaround time for each ambulance

To predict the TT of each ambulance in a given time and hospital during 665 April, May, and June 2020, respectively, the baseline BSTT and our proposed methodology were evaluated. In both cases, LGBM is the modeling technique. In our proposal, there are additional predictors such as the AvTT predicted by the first (and best) regularly spaced multivariate time series model, i.e., also using LGBM, as well as past AvTTs for recognizing average daily trends. These results demonstrate that it is possible to forecast the TT of ambulances in a given time and hospital with good accuracy for practical purposes, i.e., 86% of the time, the error is less than or equal to 10 minutes for CHIH, for example.

These results are also reflected in Figure 11, where our proposed methodology

Discussion

Extended waiting times to transfer patient(s) from ambulances to EDs (i.e., AOD) and eventually, high TTs for ambulances, may lead to numerous con- The study in [START_REF] Vandeventer | The association between ambulance hospital turnaround times and patient acuity, destination hospital, and time of day[END_REF] identified a high correlation between patient acuity and TT.

In our case, there are three variables related to the type of intervention, in which the latter is the severity level of the victim. Taking this into consideration and for the first time, this research proposed a ML-based methodology (cf. Figure 6)

to forecast the TT for each ambulance in a given time and hospital, using as a reference the forecasting of AvTT per hour for ambulances in hospitals, its type of intervention, and external variables. Indeed, forecasting the TT of an ambulance in a given time and hospital could provide valuable information for SDIS 25, and in general, for EMSs. For instance, EMSs could activate proactive decision-making with the available resources and personnel in order to be able to save more lives. Further, if there are policies on ambulance diversion, EMSs may consider diverging their ambulances to other EDs whose TT is smaller to provide adequate care to the ambulances' patient(s), as well as, returning on service in less time. Besides, the predicted AvTT for ambulances in the next hour(s) could provide approximate information for ambulances' TT, which hospitals and EMSs services may consider as a reference value, and use as a

predictor in a second model as we propose.

These data-driven systems should be of high confidence in order to assist adequately as a decision-support tool in real-life. Moreover, these systems should also be robust enough to abnormal situations such as natural disasters and pandemics. For instance, in adverse cases such as the current COVID-19 pandemic, the need for such a kind of information (ambulances' TT and AvTT) becomes vital since healthcare systems may saturate. While our work was motivated to include the impact of the novel COVID-19 pandemic, our solutions are not limited to it. Indeed, these forecasting models could help other private or public EMSs to forecast the TT of their ambulances and AvTT of hospitals, as well as to future abnormal situations.

To evaluate our proposed methodology, experiments were performed for the period of April, May, and June of 2020, which considers the peak-period of the COVID-19 first wave in the Doubs Region (cf. Figure 2) and a high number of breakdowns in the SDIS 25 service (cf. Figures 3 and5). As shown in the results, it is possible to accurately forecast the TT of each ambulance in a given time and hospital, as well as, to forecast the AvTT of ambulances per hour and hospital. On the one hand, our proposed methodology achieves ACC10 metrics ranging from ∼ 75% (CHRU) to ∼ 87% (CHIH), considering a margin of error of ±10 minutes, which are promising accuracies for practical purposes to forecasting the TT of an ambulance. In addition, the multivariate regularly spaced time series model trained with LGBM achieves ACC10 higher than 90% for all three hospitals for predicting AvTT.

In our experiments, it is remarkable the improvement of the results with the proposed methodology, for such complex and important tasks, comparing to straightforward prediction models such as the baselines BSAvTT and BSTT.

On the one hand, BSAvTT is a straightforward solution that considers only the historical data of ambulances' TT and averages the TT per hour for prediction.

Even though this is an intuitive solution that hospital staff and/or EMS could think of applying, we demonstrate that training multivariate time series models based on ML result in much higher performance. For instance, external variables such as meteorological (bad weather may results in more traffic accidents, floods, etc), temporal (day of the week/year, holiday or not), trends (google search for disease-related keywords), are variables that may lead to reasons of overcrowding in hospitals' EDs. On the other hand, BSTT is a straightforward model that one might consider applying by using only the historical data of ambulances' TT, external variables (traffic, weather, etc.), and variables related to the incident (type, hour, etc.). However, we demonstrate that adding the predicted AvTT of ambulances for the hour an ambulance is going to the hospital and past AvTTs, leads to much higher performance for this complex task.

Finally, the present work has some key limitations that are described in the following. We analyzed and build our models using the data of SDIS 25.

Although it may represent a sufficient amount of samples, other emergency medical services in France also transport victims to hospitals' EDs. In addition, there was no data from hospitals such as a history of all ambulances' TT, patient flow and the number of doctors on duty, for example. These variables could be of high importance as they are internal to hospitals, which would help ML models to better forecast the TT of each ambulance and AvTT for ambulances.

Conclusion and Future Work

This paper presents a first study on the impact of the current COVID-19 pandemic on the firefighter ambulances' TT on hospitals in the region of Doubs-France. A significant increase in the AvTT per day has been identified in 2020 as soon as the number of official and suspected cases of the disease began to rise. Additionally, in comparison with previous years, this increment is not normal, which strengthens the claim that the AvTT per day increased due to the COVID-19 pandemic. A direct and negative chain-like effect is the increment of breakdowns on the firefighters' service in 2020 due to the lack of ambulances and personnel in the centers of the SDIS 25. Therefore, it is vital to have some data-driven system to forecast the TT that will have an ambulance in a certain hospital, as we propose in this paper. This prediction could be made at the moment when the personnel report that they will go to the hospital. This would help fire brigades, and in a global context, EMSs to activate proactive decision-making with the available resources in order to allow saving more lives.

For future work, we will extend our analyses and forecasting to the second semester of 2020, which includes the current second wave of COVID-19 in

France. Also, we aim to improve the proposed methodology by adding new features and analyzing their influence (i.e., feature selection), and to test with other ML techniques. Finally, there is also another direction for further research, and that is about finding the nearest hospital with a shorter TT, considering internal hospital data.

( a )

 a Analysis of the COVID-19 impact on ambulances' TT and the breakdowns generated in the fire service. We provide an in-depth analysis of the COVID-19 impact on the average turnaround time (AvTT) per day in each hospital during the first semester of 2020 in the Doubs region. Further, we describe the already existing breakdown problems in the fire service due to long TTs in hospitals. Finally, we show how this problem worsens with the arrival of COVID-19, demonstrating the need for a system to prevent breakdowns in the fire service in the face of a pandemic. (b) Creation of a regularly spaced time series model that represents the average hourly turnaround time at each hospital. The model is based on the history of TTs reported by SDIS 25 ambulances since 2015 for CHRU and CHIH hospitals, and since 2017 for HNFC, in order to recognize hourly, daily and weekly trends. To this was added internal explanatory variables such as the number of interventions by firefighters (a greater increase in interventions in a certain period of time may indicate a greater number of victims attending the hospital) and the suspected cases of COVID-19 registered by the SDIS 25. External variables such as the number of COVID-19 cases officially reported in the region; keywords most searched in Google to retrieve trends; traffic predictions, since a very congested day can generate accidents and victims to transport; and meteorological data were also considered, since many floods usually occur in the region, which generates material damage and victims. Four state-of-the-art ML models were considered in this task, i.e., a decision-tree based model (LGBM -Light Gradient Boosted Machine), a feedforward type neural network (MLP -Multilayer Perceptron), a recurrent neural network (LSTM -Long Short-Term Memory), and a decomposable time series model (Prophet). (c) Creation of a irregularly spaced time series model to predict the turnaround time for each intervention and its ambulances. One or more ambulances belong to an intervention, if the intervention has a long duration, the ambulances could go to the hospital at different times of the day. It is here where the first model described in (b) will provide us with predicted information on the AvTT in a given hour as an explanatory variable. Along with the type of intervention (to recognize the degree of severity of the incident) and other variables mentioned previously we propose to forecast the TT for each ambulance. Because in the previous task (b) the

Figure 1 :

 1 Figure 1: Statistical analysis for the TT of ambulances: each row illustrates a histogram with bins of 1 minute to the TT frequency distribution (left side) and the cumulative sum of TT in hours for each day of the week and hour in the day (right side), considering each hospital.

Figure 2 :

 2 Figure 2: Statistical analysis for the AvTT per day considering the COVID-19 impact. The plot on the top illustrates statistics on the average turnaround time in minutes per day for 2020 and each hospital. The plot on the bottom illustrates the COVID-19 related statistics provided by Data Gouv and the SDIS 25.

Figure 3

 3 Figure3shows the average breakage time in seconds per day for the months mentioned above comparing the years 2019 and 2020. One can notice small peaks during the months of March and June, which are periods before and after the peak of the pandemic, respectively. However, in the first week of April, there

Figure 3 :

 3 Figure 3: Average breakage time in seconds per day for 2020, considering service public breakdowns of the type "rescue people".

  better prepare themselves for future interventions and avoid breakdowns on the

Figure 4 :

 4 Figure 4: Causes of public service breakdowns of the type "rescue people" by month in 2019.

Figure 5 :

 5 Figure 5: Causes of public service breakdowns of the type "rescue people" by month in 2020.

Figure 6 :

 6 Figure 6: The proposed methodology is based on ML techniques to predict the TT of an ambulance in a given hour and hospital. The first stage (dashed line flow) is based on a multivariate model of regularly spaced time series to predict the AvTT of the next hour in each hospital. The second stage (solid line flow) is a multivariate irregularly spaced time series model, which considers temporal variables of each ambulance departure, type of intervention, external variables and the previously predicted AvTT as inputs. For example, to predict the TT of ambulances departing to a hospital at 2:25h, 13:47h and 18:15h, the predicted AvTTs for 2h, 13h and 18h (highlighted) are used, respectively.

Figure 7 :

 7 Figure 7: Illustrated example of the input data structure used for the AvTT model. In gray the hour identifier (Hour ID), in yellow the explanatory variables such as time, traffic, weather, statistics of COVID-19 cases from Data Gouv, trends in Google, number of assisted interventions, AvTT of the previous 24 hours, etc. And in orange, the AvTT to predict.

Figure 8 :

 8 Figure 8: Illustrated example of the input data structure used for the TT model. In gray the depart identifier, in yellow the explanatory variables such as the type of intervention, the predicted AvTT for the present hour and the AvTT of the last 48h, time variables, Google trends, traffic, weather, etc. And in orange, the TT of the ambulance to predict.

Figure 9 :

 9 Figure 9: Comparison between the real and the predicted AvTT for ambulances per hour in each hospital, namely CHRU, CHIH, and HNFC, respectively, by each ML technique and the baseline BSAvTT.

Figure 10 :

 10 Figure 10: Comparison between the real hourly AvTT, the predictions of the best model obtained with the LGBM technique and the predictions of the baseline model BSAvTT, for each hospital (CHRU, CHIH and HNFC). It is observed that the AvTT predicted by LGBM better recognizes the patterns of the mean times and their variation through time, while the BSAvTT maintains a more constant pattern.

Figure 11 :

 11 Figure 11: Comparison between the real TT of each ambulance for each hospital (CHRU, CHIH and HNFC), the predictions of the proposed methodology and the predictions of the baseline model BSTT.

700

  sequences for patients' care, financial losses, and unavailability of providing adequate emergency medical services[START_REF] Li | A review on ambulance offload delay literature[END_REF][START_REF] Segal | The in-hospital interval: A description of EMT time spent in the emergency department[END_REF][START_REF] Perry | The ethics of ambulance ramping[END_REF][START_REF] Eckstein | The effect of emergency department crowding on paramedic ambulance availability[END_REF][START_REF] Eckstein | Facilitating EMS turnaround intervals at hospitals 36 in the face of receiving facility overcrowding[END_REF], for example. In this paper, we analyzed ambulances' TT from January 2015 to June 2020 on two hospitals (CHRU and CHIH), and from February 2017 to June 2020 on one hospital (HNFC), for the SDIS 25 fire department in Doubs, France. Although one can find works in the literature investigating the TTs of ambulances[START_REF] Cone | A time-motion study of the emergency medical services turnaround interval[END_REF][START_REF] Vandeventer | The association between ambulance hospital turnaround times and patient acuity, destination hospital, and time of day[END_REF][START_REF] Lee | Emergency department overcrowding and ambulance turnaround time[END_REF][START_REF] Clarey | Ambulance handovers: can a dedicated ED nurse solve the delay in ambulance turnaround times?[END_REF], we included in our analysis (Subsection 2.2) the impact of COVID-19 in the TTs of ambulances in a larger longitudinal study. Further, we analyzed the negative impact due to COVID-19, which increased the TTs of firefighters' ambulances and, consequently, generated more breakdowns in services (Subsection 2.3).

[

  54] J. Bergstra, D. Yamins, D. D. Cox, Making a science of model search: Hy-1015 perparameter optimization in hundreds of dimensions for vision architectures, in: Proceedings of the 30th International Conference on International Conference on Machine Learning, ICML'13, JMLR, 2013, p. I-115-I-123. Appendix A. Settings used during modeling

Table 1 :

 1 Data analysis, for each hospital, on the number of arrivals (Arr.) and Mean±std TT values in minutes per semester during 2015-2020.

	Sem.	Year	Arr.	CHRU Mean±std TT	Arr.	CHIH Mean±std TT	Arr.	HNFC Mean±std TT
		2015	3,114	17.72±10.24	1,230	14.50±7.84	-	-
		2016	3,125	17.68±10.36	1,176	14.33±7.64	-	-
	1st	2017 2018	3,676 5,233	17.55±10.31 18.75±11.37	1,288 1,548	13.99±7.43 15.05±9.41	2,272 3,824	22.97±11.12 22.91±12.18
		2019	4,672	18.94±11.66	1,368	15.63±9.80	2,994	22.92±11.90
		2020	3,899	20.67±12.57	1,184	15.96±10.51	2,136	24.42±13.19
		2015	2,893	17.59±10.28	1,174	14.69±8.91	-	-
		2016	3,320	17.77±10.32	1,222	13.88±7.32	-	-
	2nd	2017 2018	4,829 5,182	18.93±11.76 18.56±11.35	1,556 1,440	15.04±9.41 15.50±9.32	3,454 3,600	22.07±10.90 22.04±11.55
		2019	3,730	20.08±13.03	1,208	16.52±11.23	2,430	23.25±12.37

  illustrates for each day (x-axis) the AvTT in minutes (y-axis) for each hospital. Additionally, at the bottom of this figure, the plot illustrates the official COVID-19 statistics from two data sources, i.e., Data Gouv and the ones reported by the SDIS 25, to search for evident patterns. More specifically, this plot illustrates the current number of hospitalized individuals (hosp.), the number of patients in reanimation or critical care (rean.), the cumulative number of individuals who returned home (ret.

	10	Jan	Feb	Mar	Apr	May	Jun	Jul

home), and the cumulative number of individuals who died (dead) regarding the Data Gouv source, and the number of cases per day attended by the SDIS 25 with a suspicion of COVID-19 (Susp. COVID-19). For both data sources, there is an indication of the first day with COVID-19 cases.

Table 2 :

 2 Data analysis, for each hospital, on the mean±std TT values in minutes per month during 2020.

		January	February	March	April	May	June
	CHRU	18.34±7.75	18.93±8.39	21.88±10.62	22.90±9.12	21.55±8.89	20.38±9.45
	CHIH	14.92±6.81	14.15±4.95	16.38±6.64	20.19±10.55	17.44±6.20	16.32±7.86
	HNFC	23.29±10.63	21.11±7.22	25.15±11.27	24.24±9.02	27.26±9.92	24.38±9.77

Table 3 :

 3 Prediction results for the AvTT of ambulances per hour and hospital by ML technique.

	Hospital	Metric	BSAvTT	LGBM	MLP	LSTM	Prophet
		RMSE	9.70	7.36	7.78	8.77	7.70
	CHRU	MAE	6.82	4.94	5.49	6.05	5.34
		ACC10	83.29%	90.16%	88.05%	84.02%	88.46%
		RMSE	9.08	3.79	4.27	4.85	4.00
	CHIH	MAE	5.60	1.76	2.60	2.75	2.15
		ACC10	86.81%	97.02%	96.79%	95.28%	96.98%
		RMSE	10.08	5.97	6.77	7.88	6.66
	HNFC	MAE	7.58	3.52	4.45	5.53	4.55
		ACC10	77.24%	93.09%	91.30%	86.22%	91.48%

  Table3that all ML-based methods consistently and considerably outperform the baseline BSAvTT model. Among the four ML-based models, LGBM presented the best performance for all metrics and hospitals.

	Similar metric results were achieved by MLP and Prophet. Additionally, LGBM
	provided faster execution time than neural network-based methods (MLP and
	LSTM) and Prophet. For hospital CHIH, LGBM provided lower estimation
	error with RM SE = 3.79 and M AE = 1.76 minutes, which naturally led to
	more correct predictions, considering a margin of error of ±10 minutes, and
	ACC10 = 97.02%. These results demonstrate that it is possible to forecast
	the AvTT of ambulances in hospitals per hour with good accuracy for practical
	purposes, i.e., 97% of the time, the error is less than or equal to 10 minutes. Sim-
	ilarly, although not as good as for hospital CHIH, hospitals CHRU and HNFC
	present ACC10 metric higher than 90% with RMSE about 6 to 7 minutes.

Table 3

 3 670presents the metrics, discussed in Subsection 2.6, for each model and hospital, where the best results are highlighted in italic font. Both MAE and RMSE metrics express model prediction error in units of the variable of interest TT (minutes). In Appendix A, TableA.6 indicates the range of each hyperparameter we considered in the Bayesian optimization, as well as the best configurations used to train and evaluate the models. The description of the LGBM search space follows the one made in section 3.1

Table 4 :

 4 Prediction results for the TT using the proposed methodology and the BSTT model.

	Hospital	Metric	BSTT	Proposed methodology
		RMSE	13.42	12.92
	CHRU	MAE	8.74	8.31
		ACC10	73.96	74.42
		RMSE	13.99	10.53
	CHIH	MAE	7.70	5.37
		ACC10	82.97	86.63
		RMSE	13.31	11.70
	HNFC	MAE	9.58	7.74
		ACC10	68.02	76.67
	Moreover, Figure 11 illustrates the comparison of each model to forecasting
	the ambulance's TT in a given time and hospital, in different periods during the
	COVID-19 outbreak. Notice that there might be several or no ambulance in a
	given hour in the hospital. Also, this figure plots the ambulances' TT according
	to the moment in which they reported that they were going to the hospital.
	As one can notice in Table 4, the proposed methodology consistently out-
	performs the straightforward BSTT model. While for hospital CHRU the dif-
	ference is small and favoring our methodology, for hospital CHIH and HNFC,
	our proposal considerably outperforms the baseline model. For instance, our
	proposed methodology provided a lower estimation error with RM SE = 10.53
	and M AE = 5.37 minutes for hospital CHIH, which naturally led to more cor-
	rect predictions by achieving ACC10 = 86.63%. Similarly, hospitals CHRU and
	HNFC present ACC10 metric around 75% with RMSE about 12 to 13 minutes.

Table A .

 A 5: Search space for hyperparameters by technique and the best configuration obtained for predicting AvTT per hour for each hospital. Table A.6: Search space for hyperparameters in LGBM and the best configuration obtained according to the method applied for predicting the TTs of each ambulance.

	Technique	Search space		CHRU	Best configuration CHIH	HNFC
		max depth: [1-12]		6			12	5
	LGBM	learning rate: [0.001-1] n estimators: [50-1000] num leaves: [31-100] Method LGBM search space		0.8498 143 40 CHRU Best configuration 0.0103 808 61 CHIH HNFC 0.0094 768 45
		subsample: [0.5-1] max depth: [1-12]		0.99	2		0.92 1	1	0.5
		colsample bytree: [0.5-1] n estimators: [50-1000]	0.94	978	0.6 820	678	0.63
	MLP	Dense layers: 2 nb neurons: [100-500] num leaves: [31-100] (100, 302) 2 98 learning rate: [0.001-1] 0.0034 alpha: [0.00001-0.01] 0.000029 subsample: [0.5-1] 0.78 learning rate init: [0.0001-0.1] BSTT 0.0355 colsample bytree: [0.5-1] 0.86	2 (125, 325) 59 0.0099 0.003674 0.68 0.003611 0.89	2 (362, 562) 99 0.0157 0.55 0.000034 0.000543 0.92
		max iter: [50-200] max depth: [1-12]		154	6		86 2	9	177
		tol: [0.00001-0.01] n estimators: [50-1000] 0.003731 150	0.002888 129	0.000661 401
	momentum: [0.00001-0.01] Early stopping: 20 num leaves: [31-100] Proposed methodology learning rate: [0.001-1]	0.0073 20 94 0.1888	0.006057 20 46 0.0956	0.00479 20 75 0.0299
	LSTM	LSTM layers and neurons: 1, (110) subsample: [0.5-1] Dense layers and neurons: 2, (128, 1) colsample bytree: [0.5-1] 1, (110) 0.72 2, (128, 1) 0.73	1, (110) 0.65 2, (128, 1) 0.76	0.79 1, (110) 2, (128, 1) 0.5
		Activation function: ReLU		ReLU		ReLU	ReLU
		Dropout: 0.5		0.5			0.5	0.5
		Loss function: 'mse'		'mse'		'mse'	'mse'
		Optimizer: Adam		Adam		Adam	Adam
		Early stopping: 15		15			15	15
		Max. epochs: 100		100			100	100
		Batch size: [40-250]		93			142	112
		Learning rate: [0.005-0.01]	0.00841		0.00595	0.00894
		n changepoints: [20-100]		45			75	35
	Prophet	seasonality prior scale: [0-50]		36.82		18.17	12.13
		holidays prior scale: [0-50]		23.47		22.42	34.77

https://github.com/GeneralMills/pytrends

https://en.wikipedia.org/wiki/COVID-19 pandemic in France#Lockdown measures

Acknowledgment This work was funded by SDIS du Doubs, with the support of the French Ministry of Higher Education and Research (managed by the National Association of Research and Technology (ANRT) for the CIFRE thesis (N 2019/0372), by the EIPHI-BFC Graduate School (contract "ANR-17-EURE-0002"), and by the Region of Bourgogne Franche-Comté CADRAN Project. Computations have been performed on the supercomputer facilities of "Mésocentre de Calcul de Franche-Comté".

Highlights

• Analysis of the ambulances' turnaround time for three hospitals from 2015-2020.

• Breakage calculation in the fire department ambulance service during COVID-19.

• Predicting the ambulances' average turnaround time in a given hour and hospital.

• Predicting the turnaround time for each intervention and its ambulances.

Declaration of interests

☐ The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

☐The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: