Dr Philippe Gorphe
email: philippe.gorphe@gustaveroussy.fr

Christian Simon

A systematic review and meta-analysis of margins in transoral surgery for oropharyngeal carcinoma

Keywords: Transoral surgery, transoral robotic surgery, transoral laser microsurgery, oropharyngeal carcinoma, margins Oropharyngeal carcinoma, transoral surgery, transoral laser microsurgery, transoral robotic surgery, margin status, positive margins, meta-analysis

Background: The objective of this study was to conduct a systematic review and meta-analysis of the incidence of positive surgical margins after transoral surgery for oropharyngeal carcinoma, as well as the factors associated with positive margins and their impact on local tumor control.

Method: An electronic search of English-language literature databases was conducted, and a systematic review was performed in accordance with the PRISMA guidelines.

Results: A total of 42 articles were included in the analysis. The overall rate of positive margins using transoral conventional surgery (CTS), transoral laser microsurgery (TLM), or transoral robotic surgery (TORS) was 7.8% in a cumulative total of 3,619 patients. A positive margin status was associated with a reduction in local control.

Assessment of intraoperative frozen sections was associated with a reduced risk of definitive positive margins, whereas a T4 classification was associated with an increased risk of definitive positive margins. Neither the primary site (the tonsillar fossa versus the base of the tongue), nor the HPV status, were associated with the margin status. The level of heterogeneity between the various studies was very high.

Conclusion:

The currently used transoral procedures are safe in regard to proper tumor resection and they should continue to be part of the armamentarium of surgical techniques used in head and neck surgery. The very high level of heterogeneity between studies calls for a definition consensus for margin status assessments in transoral surgery.

Introduction

Head and neck cancers are the 6 th most common cancer worldwide, with 630,000 new cases annually [START_REF] Parkin | Global cancer statistics[END_REF]. More than half of all head and neck cancer patients require primary surgical treatment, and typically more than twothirds will have to undergo a surgery at some point during their treatment [START_REF] Bozec | Current role of primary surgical treatment in patients with head and neck squamous cell carcinoma[END_REF]. Due to the biological properties of epithelial cancers, namely their invasive growth, comprehensive and radical resection of the primary tumor with a minimal risk of residual disease often requires a large cuff of macroscopically healthy tissue around the tumor.

In more than one-third of all surgical cases, the patients ultimately have positive or cut-through margins as a consequence of the infiltrative growth patterns of the tumor, or due to restrictions during the resection owing to confinements of the anatomy as a result of nearby vital structures [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF]. As a result, there is often the persistence of residual disease in the surgical wound bed. These typically give rise to loco-regional recurrences that can ultimately lead to death of the patient. The decision-making for potential intensification of the postoperative treatment with concurrent chemoradiation, therefore, relies, amongst other factors, on assessment of the surgical margin in order to assess the risk for microscopically residual disease [START_REF] Bernier | Defining risk levels in locally advanced head and neck cancers: a comparative analysis of concurrent postoperative radiation plus chemotherapy trials of the EORTC (#22931) and RTOG (# 9501)[END_REF]. Traditional open surgical resection in head and neck cancer mandated free microscopic margins of at least five millimeters to be considered negative, like used in the EORTC 22931 trial [START_REF] Bernier | Defining risk levels in locally advanced head and neck cancers: a comparative analysis of concurrent postoperative radiation plus chemotherapy trials of the EORTC (#22931) and RTOG (# 9501)[END_REF]. However, the development of transoral techniques for minimal-invasive surgery in narrow spaces has challenged this concept of 5-mm free margins due to anatomical limitations, without decrease in disease control reported. For example, in endoscopic laser microsurgery for early laryngeal glottic tumors, negative margins down to one millimeter have been shown to be oncologically safe for some authors [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF][START_REF] Ansarin | Laser surgery for early glottic cancer: impact of margin status on local control and organ preservation[END_REF]. With recent developments in minimal-invasive transoral surgery for pharyngeal cancer, this has led to considerable heterogeneity amongst institutions and authors with how to define adequate margins [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF].

Actually, transoral surgery is a surgical treatment strategy for selected head and neck cancers that are amenable to transoral exposure and resection [START_REF] Tirelli | Transoral surgery (TOS) in oropharyngeal cancer: Different tools, a single mini-invasive philosophy[END_REF]. This is nowadays performed either with conventional instruments (transoral conventional surgery, TCS), a CO2-laser (transoral laser microsurgery, TLM), or a robotic surgical system (transoral robotic surgery, TORS). While the head and neck surgeon during open trans-cervical procedures operates from the outside to the inside and needs to be fully acquainted with the specific anatomy, the transoral surgeon needs to master an "inside-out" approach and the changing anatomy and landmarks inherent with this approach [START_REF] Tirelli | Transoral surgery (TOS) in oropharyngeal cancer: Different tools, a single mini-invasive philosophy[END_REF][START_REF] Holsinger | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: I. Technique, complications, and functional results[END_REF]. In selected patients, transoral surgery may offer significant advantages based on the fact that it helps avoid transcervical access trauma, it reduces the healing time, and it allows for better and faster functional recovery. While all of these advantages are certainly of benefit to the patient, transoral approaches can result in intraoperative situations whereby visibility of critical structures and tissue handling is hampered as a consequence of the confined workspace and thus can impact on the adequacy of the surgical resection. It is, therefore, of critical importance to analyze the available data regarding the surgical margin status after this type of intervention.

In this systematic review, we investigated the incidence of positive surgical margins after transoral surgery for oropharyngeal carcinoma, as well as the factors associated with positive margin and their impact on local tumor control. ). The results were restricted to English-language studies. The records were screened for exclusion criteria, which comprised: articles not dedicated to transoral surgery in oropharyngeal cancer; articles other than case-series; case-series with fewer than ten patients; histologies other than squamous-cell carcinoma; case-series with multiple head and neck cancer localizations such as the hypopharynx and the supraglottic larynx; and reports in which the margins were not indicated. The available full-text articles were assessed for eligibility before they were included in the final qualitative analysis. Potential additional records were identified by screening of the references in the full-text articles included in the final qualitative synthesis They were then checked for duplicates, screened for the exclusion criteria, and assessed for eligibility. The initial search was performed on the 1 st of November 2018, with a follow-up search to identify newly published articles on the 1 st of May 2019. The data were reviewed by two reviewers (PG and CS). The extracted data comprised: the number of patients operated; an oropharyngeal primary tumor subsite (the tonsillar fossa, the base of tongue, the soft palate, or the pharyngeal wall); the primary T classification; the surgical approach (conventional transoral surgery, transoral laser microsurgery, or transoral robotic surgery); the type of margins reported for positive/close/negative or positive/negative; the definition of the perioperative macroscopic surgical margins; assessment of intraoperative frozen sections; the outcomes of the definitive microscopic analyses; the threshold for close margins; the number of definitive positive/close/negative margins; and the number of patients with local recurrence. The statistical analysis was based on the cumulative number of patients, using IBM SPSS Statistics for Windows version 23 software (IBM Corp., Armonk, NY, USA) and Comprehensive Meta-Analysis version 3.3.070 software (Biostat Inc., Englewood, NJ, USA). Dichotomous outcomes were compared using risk ratios (RRs) and 95% confidence intervals. Heterogeneity across the studies was evaluated by the chi-squared statistic and the Fisher's exact test. The I 2 test was used to measure the degree of inconsistency between the results. Meta-analysis was carried out using the random effects model.

Material and Methods

This

Results

Study selection and margins

A total of 42 articles were included in the analysis (Figure 1): 3 series of conventional transoral surgery (CTS) [START_REF] Laccourreye | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: II. An analysis of the incidence, related variables, and consequences of local recurrence[END_REF][START_REF] Moore | Transoral resection of tonsillar squamous cell carcinoma[END_REF][START_REF] Henstrom | Transoral resection for squamous cell carcinoma of the base of the tongue[END_REF], 4 series of CTS or TLM [START_REF] Karatzanis | Surgical management of T1 oropharyngeal carcinoma[END_REF][START_REF] Psychogios | Prognostic factors in limited (T1-2, N0-1) oropharyngeal carcinoma treated with surgery +/-adjuvant therapy[END_REF][START_REF] Chauhan | Oncological and functional outcomes of transoral surgery for the treatment of oropharyngeal cancer[END_REF][START_REF] Molony | Impact of positive margins on outcomes of oropharyngeal squamous cell carcinoma according to p16 status[END_REF], 10 series of TLM [START_REF] Steiner | Transoral laser microsurgery for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Grant | Carcinoma of the tongue base treated by transoral laser microsurgery, part two: Persistent, recurrent and second primary tumors[END_REF][START_REF] Grant | Oropharyngeal cancer: a case for single modality treatment with transoral laser microsurgery[END_REF][START_REF] Rich | Transoral laser microsurgery (TLM) +/-adjuvant therapy for advanced stage oropharyngeal cancer: outcomes and prognostic factors[END_REF][START_REF] Hinni | Margin mapping in transoral surgery for head and neck cancer[END_REF][START_REF] Patel | Transoral laser microsurgery followed by radiation therapy for oropharyngeal tumors: the Mayo Clinic Arizona experience[END_REF][START_REF] Haughey | Transoral laser microsurgery as primary treatment for advanced-stage oropharyngeal cancer: a United States multicenter study[END_REF][START_REF] Melong | Transoral laser microsurgery for the treatment of oropharyngeal cancer: the Dalhousie University experience[END_REF][START_REF] Wilkie | Transoral laser microsurgery for oropharyngeal squamous cell carcinoma: A paradigm shift in therapeutic approach[END_REF][START_REF] Sinha | Long-term analysis of transorally resected p16 + Oropharynx cancer: Outcomes and prognostic factors[END_REF], 1 series of CTS or TORS [START_REF] Lee | Comparison of oncologic and functional outcomes after transoral robotic lateral oropharyngectomy versus conventional surgery for T1 to T3 tonsillar cancer[END_REF], and 24 series of TORS . One series with CTS and TLM procedures also comprised of a small number of patients treated by an open approach [START_REF] Molony | Impact of positive margins on outcomes of oropharyngeal squamous cell carcinoma according to p16 status[END_REF]. Some authors published several articles, resulting in a risk of cross-matching data [9, 10, 16, 17, 19-21, 24, 26, 31, 40, 41, 44, 48, 49]. The cumulative number of patients was 3,725 (Table 1). Of the 2,299 patients evaluated for HPV status in 27/42 series, 1,810 patients (78.7%) were considered HPVpositive due to either positive p16 immunohistochemistry or positive genotyping.

Four articles out of 42 provided the intraoperative macroscopic margins of the healthy tissue around the primary tumor. This was 10 mm in three series [START_REF] Rich | Transoral laser microsurgery (TLM) +/-adjuvant therapy for advanced stage oropharyngeal cancer: outcomes and prognostic factors[END_REF][START_REF] Park | Transoral robotic surgery-based therapy in patients with stage III-IV oropharyngeal squamous cell carcinoma[END_REF][START_REF] Morisod | Minimizing adjuvant treatment after transoral robotic surgery through surgical margin revision and exclusion of radiographic extracapsular extension: A Prospective observational cohort study[END_REF] and 10-15 mm in one series [START_REF] Haughey | Transoral laser microsurgery as primary treatment for advanced-stage oropharyngeal cancer: a United States multicenter study[END_REF]. In 14/42 articles (33.3%), definitive margins were categorized as positive, close, or negative. The definition of close margins was 1 mm in one series [START_REF] Patel | Transoral laser microsurgery followed by radiation therapy for oropharyngeal tumors: the Mayo Clinic Arizona experience[END_REF], 2 mm in 5 series [START_REF] Chauhan | Oncological and functional outcomes of transoral surgery for the treatment of oropharyngeal cancer[END_REF][START_REF] Cohen | Transoral robotic surgery and human papillomavirus status: Oncologic results[END_REF][START_REF] Dabas | Transoral robotic surgery in management of oropharyngeal cancers: a preliminary experience at a tertiary cancer centre in India[END_REF][START_REF] Kaczmar | HPV-related oropharyngeal cancer: Risk factors for treatment failure in patients managed with primary transoral robotic surgery[END_REF][START_REF] Rubek | Primary transoral robotic surgery with concurrent neck dissection for early stage oropharyngeal squamous cell carcinoma implemented at a Danish head and neck cancer center: a phase II trial on feasibility and tumour margin status[END_REF], 3 mm in two series [START_REF] Ling | Oncologic outcomes and patient-reported quality of life in patients with oropharyngeal squamous cell carcinoma treated with definitive transoral robotic surgery versus definitive chemoradiation[END_REF][START_REF] Morisod | Minimizing adjuvant treatment after transoral robotic surgery through surgical margin revision and exclusion of radiographic extracapsular extension: A Prospective observational cohort study[END_REF], and 5 mm in four series [START_REF] Molony | Impact of positive margins on outcomes of oropharyngeal squamous cell carcinoma according to p16 status[END_REF][START_REF] Melong | Transoral laser microsurgery for the treatment of oropharyngeal cancer: the Dalhousie University experience[END_REF][START_REF] Ford | Transoral robotic versus open surgical approaches to oropharyngeal squamous cell carcinoma by human papillomavirus status[END_REF][START_REF] Lorincz | Functional outcomes, feasibility, and safety of resection of transoral robotic surgery: single-institution series of 35 consecutive cases of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF]. In one series, the definition was not provided [START_REF] Laccourreye | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: II. An analysis of the incidence, related variables, and consequences of local recurrence[END_REF], and in one series close margins were defined as 5 mm for infiltrative carcinomas or 3 mm for tumors with "pushing" borders [START_REF] Van Loon | Outcome of transoral robotic surgery for stage I-II oropharyngeal cancer[END_REF]. Two articles [START_REF] Kaczmar | HPV-related oropharyngeal cancer: Risk factors for treatment failure in patients managed with primary transoral robotic surgery[END_REF][START_REF] Rubek | Primary transoral robotic surgery with concurrent neck dissection for early stage oropharyngeal squamous cell carcinoma implemented at a Danish head and neck cancer center: a phase II trial on feasibility and tumour margin status[END_REF] did not report the rate of close margins separately from positive margins. For patients from these series, the margins were positive in 84 patients (12%) and negative in 614 patients (88%), of whom 96 had close margins (13.8%) and 518 were reported to have clear margins (74.2%). In the remaining 28 articles, definitive margins were reported as being either positive or negative. The total cumulative number of patients analyzed for margin status was 3,619. The margins were positive in 283 patients (7.8%) and negative in 3,336 patients (92.2%). Factors analyzed in univariate analysis for a risk of positive margins were the primary tumor site, the primary tumor T-classification, the HPV status, the transoral surgical procedure, and assessment of intraoperative frozen sections (Figure 2).

The primary tumor site and positive margins

The primary tumor subsites were the tonsillar fossa in 58.3% of the patients, the base of the tongue in 32.2%, the soft palate in 7.4%, and the pharyngeal wall in 2.2%. The margin status was reported in 677 patients with tonsillar carcinoma and 254 patients with tongue base carcinoma. The rates of positive margins were 4% and 5.9%, respectively, and they were not significantly different (two-tailed Fisher's exact test, p=0.217; RR, 1.48; 95% CI, 0.80 to 2.74).

The primary tumor T-classification and positive margins

The T classification of the primary tumor, when reported, was T1 in 44.4% of cases, T2 in 41.5%, T3 in 9.4%, and T4 in 4.6%. The rate of positive margins was higher in the patients with T4 OPSCC (13.2%) than in the patients with T1-3 carcinoma (two-tailed Fisher's exact test, p=0.0248; RR, 2.54; 95% CI, 1.21 to 5.33), whereas there was no difference between T3 OPSCC and T1-2 tumors (two-tailed Fisher's exact test, p=1.000; RR, 1.31; 95% CI, 0.42 to 4.11).

HPV status and positive margins

The margin status according to the HPV status was available in five series [START_REF] Molony | Impact of positive margins on outcomes of oropharyngeal squamous cell carcinoma according to p16 status[END_REF][START_REF] Rich | Transoral laser microsurgery (TLM) +/-adjuvant therapy for advanced stage oropharyngeal cancer: outcomes and prognostic factors[END_REF][START_REF] Wilkie | Transoral laser microsurgery for oropharyngeal squamous cell carcinoma: A paradigm shift in therapeutic approach[END_REF][START_REF] Cohen | Transoral robotic surgery and human papillomavirus status: Oncologic results[END_REF][START_REF] Quon | Transoral robotic surgery and adjuvant therapy for oropharyngeal carcinomas and the influence of p16 INK4a on treatment outcomes[END_REF]. The margins were positive in 13 patients out of 283 HPV-positive cases (4.6%), and they were positive in 10 out of 118 HPVnegative cases (8.5%). The HPV-positive and the HPV-negative tumors exhibited similar rates of positive margins (two-tailed Fisher's exact test, p=0.1565; RR, 1.78; 95% CI, 0.80 to 3.95).

Frozen sections and positive definitive margins

The series were categorized as series reporting systematic intraoperative frozen sections analysis, series reporting on demand intraoperative frozen sections analysis, and series that did not report frozen sections.

Among the 24 studies which reported frozen section, ten reported specimen-based frozen section [18-21, 26-28, 30, 48, 49], four reported patient-based frozen section [START_REF] Steiner | Transoral laser microsurgery for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Melong | Transoral laser microsurgery for the treatment of oropharyngeal cancer: the Dalhousie University experience[END_REF][START_REF] Rubek | Primary transoral robotic surgery with concurrent neck dissection for early stage oropharyngeal squamous cell carcinoma implemented at a Danish head and neck cancer center: a phase II trial on feasibility and tumour margin status[END_REF][START_REF] Morisod | Minimizing adjuvant treatment after transoral robotic surgery through surgical margin revision and exclusion of radiographic extracapsular extension: A Prospective observational cohort study[END_REF], and ten did not specifically described the method used [START_REF] Moore | Transoral resection of tonsillar squamous cell carcinoma[END_REF][START_REF] Henstrom | Transoral resection for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Grant | Carcinoma of the tongue base treated by transoral laser microsurgery, part two: Persistent, recurrent and second primary tumors[END_REF][START_REF] Grant | Oropharyngeal cancer: a case for single modality treatment with transoral laser microsurgery[END_REF][START_REF] Wilkie | Transoral laser microsurgery for oropharyngeal squamous cell carcinoma: A paradigm shift in therapeutic approach[END_REF][START_REF] Sinha | Long-term analysis of transorally resected p16 + Oropharynx cancer: Outcomes and prognostic factors[END_REF][START_REF] Moore | Long-term functional and oncologic results of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF][START_REF] Dziegielewski | Transoral robotic surgery for oropharyngeal cancer: long-term quality of life and functional outcomes[END_REF][START_REF] Dabas | Transoral robotic surgery in management of oropharyngeal cancers: a preliminary experience at a tertiary cancer centre in India[END_REF][START_REF] Lorincz | Functional outcomes, feasibility, and safety of resection of transoral robotic surgery: single-institution series of 35 consecutive cases of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF]. Eight series reported systematic frozen section analysis [START_REF] Moore | Transoral resection of tonsillar squamous cell carcinoma[END_REF][START_REF] Henstrom | Transoral resection for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Steiner | Transoral laser microsurgery for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Grant | Oropharyngeal cancer: a case for single modality treatment with transoral laser microsurgery[END_REF][START_REF] Patel | Transoral laser microsurgery followed by radiation therapy for oropharyngeal tumors: the Mayo Clinic Arizona experience[END_REF][START_REF] Moore | Long-term functional and oncologic results of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF][START_REF] Dziegielewski | Transoral robotic surgery for oropharyngeal cancer: long-term quality of life and functional outcomes[END_REF][START_REF] Lorincz | Functional outcomes, feasibility, and safety of resection of transoral robotic surgery: single-institution series of 35 consecutive cases of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF]. The cumulative number of patients was 501, of whom 25 (5%) had positive final margins. Sixteen series reported on demand frozen section analysis, depending on the intraoperative assessment of the quality of the resection [16, 18, 19, 21-24, 26-28, 30, 36, 45, 46, 48, 49]. The cumulative number of patients was 2,046, of whom 69 (4.6%) had positive final margins. Thirteen series did not report frozen section analysis, with a cumulative number of patients of 1,367, of whom 169 (12.3%) had positive final margins. The chi-squared comparison test was significant (p < 0.0001). The rates of final positive margins were significantly higher in the patients in series where frozen section analyses were not reported (Chi-squared with Yates' correction, p < 0.0001; RR, 2.64; 95% CI, 2.07 to 3.37).

The transoral surgical procedure and positive margins

When the patients were combined, the cumulative number of patients operated via CTS with an available margin status was 301, of whom 20 had positive margins (6.6%) and 281 had negative margins (93.4%). The cumulative number of analyzed patients who underwent TLM with an available margin status was 1,012, of whom 48 had positive margins (4.7%) and 964 had negative margins (95.3%). The cumulative number of analyzed patients who underwent TORS with an available margin status was 1,676, of whom 135 had positive margins (8.1%) and 1,541 had negative margins (91.9%). The chi-squared comparison test was significant (p=0.0042). The reported rate of positive margins was lower in the TLM patients than in the CTS or the TORS patients (Chi-squared with Yates' correction, p=0.0019; RR, 1.65; 95% CI, 1.21 to 2.27).

Multivariate analysis

Out of the 12 TLM series, three series reported systematic intraoperative use of frozen sections, nine series reported intraoperative use of frozen sections when requested, and none of the TLM series were categorized as not reporting intraoperative use of frozen sections. Meanwhile, out of 24 TORS series, three reported systematic intraoperative use of frozen sections, nine reported intraoperative use of frozen sections when requested, and 12 did not report intraoperative use of frozen sections. As expected, frozen section analysis was reported more frequently in the TLM series than in the TORS series (p=0.0025). A meta-regression was not feasible on the cumulative number of patients with positive and negative margins depending on the T-classification, the intraoperative use of frozen sections, and on the surgical approach (Figure 3). We performed a two-way independent analysis of variance with a Bonferroni post-hoc tests, using the rates of positive margins per series as a dependent variable (Figure 4). There was no significant difference in the rates of positive margins (p=1.000) according to the surgical approach.

The influence of definitive margins on the oncological outcomes

Five series provided data on local control, with 63 patients in the positive margins arm and 664 patients in the negative margins arm, respectively [START_REF] Laccourreye | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: II. An analysis of the incidence, related variables, and consequences of local recurrence[END_REF][START_REF] Psychogios | Prognostic factors in limited (T1-2, N0-1) oropharyngeal carcinoma treated with surgery +/-adjuvant therapy[END_REF][START_REF] Molony | Impact of positive margins on outcomes of oropharyngeal squamous cell carcinoma according to p16 status[END_REF][START_REF] Haughey | Transoral laser microsurgery as primary treatment for advanced-stage oropharyngeal cancer: a United States multicenter study[END_REF][START_REF] Melong | Transoral laser microsurgery for the treatment of oropharyngeal cancer: the Dalhousie University experience[END_REF]. Four series reported no definitive positive margins [START_REF] Steiner | Transoral laser microsurgery for squamous cell carcinoma of the base of the tongue[END_REF][START_REF] Grant | Oropharyngeal cancer: a case for single modality treatment with transoral laser microsurgery[END_REF][START_REF] Wilkie | Transoral laser microsurgery for oropharyngeal squamous cell carcinoma: A paradigm shift in therapeutic approach[END_REF][START_REF] Moore | Transoral robotic surgery for oropharyngeal squamous cell carcinoma: a prospective study of feasibility and functional outcomes[END_REF]. Three series reported no local recurrences [START_REF] Patel | Transoral laser microsurgery followed by radiation therapy for oropharyngeal tumors: the Mayo Clinic Arizona experience[END_REF][START_REF] Lee | Comparison of oncologic and functional outcomes after transoral robotic lateral oropharyngectomy versus conventional surgery for T1 to T3 tonsillar cancer[END_REF][START_REF] Van Loon | Outcome of transoral robotic surgery for stage I-II oropharyngeal cancer[END_REF]. No data on recurrence-free survival, diseasespecific survival, or overall survival were available for a pooled analysis. When the data of the five series reporting data on local control were pooled, there was a very small level of heterogeneity among the studies (Qvalue; 0.606; I 2 = 93.4%). The overall pooled effect with an odds ratio of 3.651 [1.691, 7.882] showed that there was a significant difference between the patients with positive margins and the patients with negative margins in terms of local control (Figure 5 and Figure 6).

Three series reported outcomes of patients with close margins, with 48 patients in the close margins arm and 249 patients in the clear margins arm, respectively [START_REF] Laccourreye | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: II. An analysis of the incidence, related variables, and consequences of local recurrence[END_REF][START_REF] Lorincz | Functional outcomes, feasibility, and safety of resection of transoral robotic surgery: single-institution series of 35 consecutive cases of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF][START_REF] Kaczmar | HPV-related oropharyngeal cancer: Risk factors for treatment failure in patients managed with primary transoral robotic surgery[END_REF]. Definition of close margins was not provided in one series [START_REF] Laccourreye | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: II. An analysis of the incidence, related variables, and consequences of local recurrence[END_REF], was two millimeters in one series [START_REF] Kaczmar | HPV-related oropharyngeal cancer: Risk factors for treatment failure in patients managed with primary transoral robotic surgery[END_REF], and five millimeters in one series [START_REF] Lorincz | Functional outcomes, feasibility, and safety of resection of transoral robotic surgery: single-institution series of 35 consecutive cases of transoral robotic surgery for oropharyngeal squamous cell carcinoma[END_REF]. When the data of the three series reporting data on local control were pooled, there was a very small level of heterogeneity among the studies (Q-value, 0.646; I 2 =96.9%), The overall pooled effect with an odds ratio of 1.768 [6.671, 4.660] showed no difference between the patients with close margins and the patients with clear margins in terms of local control, given the limits of heterogeneity bias, and the influence of postoperative treatment (Figure 7 and Figure 8). The heterogeneity statistic I² must be interpreted very cautiously when a meta-analysis has few studies such as this one, given the risk of bias and underestimation [START_REF] Hippel | The heterogeneity statistic I(2) can be biased in small meta-analyses[END_REF].

Discussion

Transoral surgery in the oropharynx is a relatively new surgical strategy to access and resect tumors of the oropharynx directly through the mouth. Compared to transcervical approaches, direct access to the tumor helps avoid the morbidity of the open approaches and thus allows for faster recovery of function, of which swallowing is the most important. Although many series have been published on the efficacy of transoral surgery, either as series on TORS, TLM, or CTS, there has been no large-scale analysis to date of the margin status after this type of surgery in the oropharynx to assess its safety. In this systematic review, the overall rate of positive margins using various types of transoral procedures was 7.8% in a population of 3,619 patients. This relatively low rate suggests that transoral procedures are safe in terms of proper tumor resection and should continue to be part of the armamentarium of surgical techniques used in head and neck surgery. We found that positive margins were associated with a reduction in local tumor control while close margins were not. The assessment of frozen sections taken during the transoral procedure was associated with a reduced risk of definitive positive margins, whereas the T4 classification of the primary tumor was associated with an increased risk of definitive positive margins. Neither the primary site (the tonsillar fossa, the base of the tongue), nor the HPV status, were associated with the margin status. Our review also highlights that there is a high level of heterogeneity regarding how margins are defined in the literature. Two-thirds of the included series reported the margins as being either positive or negative. One-third of the included articles reported an analysis of close versus clear margins, and their definition of a close margin was highly variable.

Our finding that a T4 classification was associated with a 2.54 relative risk of positive margins compared to a T1-3 classification is consistent with previous publications that have emphasized the higher risk of inadequate resection in deep connective tissues than in mucosal tissues [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF][START_REF] Woolgar | A histopathological appraisal of surgical margins in oral and oropharyngeal cancer resection specimens[END_REF]. In oropharyngeal carcinoma specifically, few studies to date have investigated this issue. Woolgar and Triantafyllou reviewed 301 carcinoma specimens, of which 48 were oropharyngeal cancer (OPSCC), using the guidelines of the UK Royal College of Pathologists [START_REF] Woolgar | A histopathological appraisal of surgical margins in oral and oropharyngeal cancer resection specimens[END_REF]. The margins were involved in 18/48 (37.5%) of the OPSCCs, of which 15 (83.3%) were pT3-T4 lesions.

The involved margin was mucosal in just a single patient, while they were mucosal and deep soft tissues in three patients, bone and deep soft tissues in one patient, and deep soft tissues only in 13 patients. For 14/48 of the OPSCCs, the margins were clear (29.2%), of which 10 (71.4%) were pT1-T2 lesions. The association of the pT stage with the rate of positive margins was significant. Invasive local extension of oropharyngeal carcinoma into the medial pterygoid muscle or the extrinsic muscles of the tongue can represent a challenge with transoral approaches due to the absence of reliable haptic feedback. The association of TLM with a lower rate of positive margins than for TCS or TORS in this study may then be explained by a high rate of systematic intraoperative use of frozen sections in TLM, due to the piecemeal resection described in the Steiner's landmark TLM procedure with analysis of successive frozen sections until a clear resection is achieved [START_REF] Steiner | Transoral laser microsurgery for squamous cell carcinoma of the base of the tongue[END_REF]. On the other hand, "en-bloc" resection of the primary tumor with a cuff of healthy surrounding tissues, as performed in open surgery, is the standard procedure in TCS as well as in TORS, and the final quality of the resection depends on the intraoperative evaluation of the local extensions [START_REF] Holsinger | Transoral lateral oropharyngectomy for squamous cell carcinoma of the tonsillar region: I. Technique, complications, and functional results[END_REF][START_REF] Weinstein | Transoral robotic surgery: radical tonsillectomy[END_REF][START_REF] Malley | Transoral robotic surgery (TORS) for base of tongue neoplasms[END_REF]. However, when comparing margins in series that reported the use of frozen sections irrespective of the procedure, there was no longer a difference in our findings between TLM, TCS, and TORS. The adjusted results confirm that the main methodological factor associated with a reduced risk of definitive positive margins is the intraoperative assessment of the quality of the resection with frozen sections, and not the surgical procedure itself. Close collaboration between the surgeon and the pathologist is hence paramount to ensure that the patient will benefit from the best standard-of-care [START_REF] Tirelli | Transoral surgery (TOS) in oropharyngeal cancer: Different tools, a single mini-invasive philosophy[END_REF].

Moreover, our review found discrepancies between authors with regards to methods used for margin assessment when using frozen section analysis, notably between specimen-driven or patient-driven margins. This issue has been thoroughly investigated in oral cancer surgery, showing intraoperative frozen margins from the specimen to better predict the risk of local recurrence than frozen margins from the tumor bed [START_REF] Buchakjian | Association of Main Specimen and Tumor Bed Margin Status With Local Recurrence and Survival in Oral Cancer Surgery[END_REF][START_REF] Kain | Surgical margins in oral cavity squamous cell carcinoma: Current practices and future directions[END_REF]. Therefore, specimenbased frozen sections are probably the best method for intraoperative assessment of comprehensiveness of the disease resection, and should be systematically recommended.

It is difficult to adequately evaluate the association of margins with oncological endpoints in transoral surgery for oropharyngeal carcinoma in single-institution series given the small number of events reported as a result of the high rate of HPV-related tumors, which exhibit a lower risk of recurrence at any site [START_REF] Rasmussen | Risk profiling based on p16 and HPV DNA more accurately predicts location of disease relapse in patients with oropharyngeal squamous cell carcinoma[END_REF]. Our meta-analysis findings that the margin status was associated with local control is consistent with results in other head and neck carcinoma localizations [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF]. In oropharyngeal carcinoma specifically, few series to date have reported assessment of margins in open surgery. Sessions et al. analyzed 262 patient with carcinoma of the base of the tongue, of whom 177 had a T1-T2 lesion [START_REF] Sessions | Analysis of treatment results for base of tongue cancer[END_REF]. There were 202 cases that were treated with surgery. The margins were defined as clear (≥ 5 mm) in 54.6% of the patients, close (< 5 mm) in 13%, or positive (microscopically involved) in 32.4%. Close and/or involved margins were associated with an increased risk of local failure. Kwok et al. reviewed data from 417 patients with oral (n=83), oropharyngeal (n=180), or hypopharyngeal (n=154) carcinoma [START_REF] Kwok | Prognostic importance of "clear versus revised margins" in oral and pharyngeal cancer[END_REF]. Failure to achieve clear margins, despite the use of frozen sections, was associated with reduced survival, in the global cohort as well as specifically in the T1-T2 and in the T3-T4 cancer patients. Iyer et al.

analyzed 201 OPSCC patients treated surgically with or without adjuvant radiotherapy [START_REF] Iyer | Detailed Analysis of Clinicopathologic Factors Demonstrate Distinct Difference in Outcome and Prognostic Factors Between Surgically Treated HPV-Positive and Negative Oropharyngeal Cancer[END_REF]. The subsites were the base of the tongue, the tonsillar fossa, and the soft palate, in 44%, 33%, and 23% of the patients. Seventy percent of the patients had a T1-T2 lesion. A close/positive versus a negative margin status was associated with overall survival, disease-specific survival, and recurrence-free survival in p16-negative OPSCC patients. Roux et al. reviewed data from 44 OPSCC patients treated surgically with or without postoperative radiotherapy [START_REF] Roux | Transoral-transcervical oropharyngectomy without mandibulotomy, associated to fasciocutaneous radial forearm free-flap reconstruction, for oropharyngeal cancer: Postoperative course, oncologic and functional results, and prognostic factors[END_REF].

Positive margins (27%) were associated with reduced disease-specific survival. To the best of our knowledge, no publication to date has addressed whether there is an association of the margin status with the oncological endpoints in HPV-related OPSCC treated by open surgery. Given the very high rate of HPV-related tumors included in our analysis (78.7%), we can assume that the association of the margin status with local control still holds true in HPV-related OPSCC. An adequate resection of the primary tumor in head and neck cancer is a challenging balance between the physician's attempts at a complete resection and the patient's expectations that local sequelae that can dramatically reduce the quality of life are minimized. The estimated risk of residual tumor associated with adverse features on a definitive pathological examination such as microscopic margins directly influences the indication for adjuvant therapy [START_REF] Bernier | Defining risk levels in locally advanced head and neck cancers: a comparative analysis of concurrent postoperative radiation plus chemotherapy trials of the EORTC (#22931) and RTOG (# 9501)[END_REF][START_REF] Colevas | NCCN Guidelines Insights: Head and Neck Cancers, Version 1[END_REF]. However, an association of this risk with close margins in head and neck cancer is difficult to assess in a meta-analysis given the retrospective and heterogeneous data as well as the limited number of publications reporting margins sizes. A high level of caution is warranted since the calculated I 2 meant that more than 90% of the variance was deemed attributable to study heterogeneity. As summarized in a clinical review by Hinni et al., the margin distance should always be measured in millimeters and recorded on the surgical pathology report [START_REF] Hinni | Surgical margins in head and neck cancer: a contemporary review[END_REF]. We believe that close margins should also be systematically reported in publications along with the threshold used. In an international American Head and Neck Society member survey in 2005, Meier et al. investigated practices in surgical margin determination in head and neck oncology [START_REF] Meier | Surgical margin determination in head and neck oncology: current clinical practice. The results of an International American Head and Neck Society Member Survey[END_REF]. The most frequent definition of a clear margin was > 5 mm on microscopic examination. This is consistent with the National Comprehensive Cancer Network (NCCN) Guidelines, for which a clear margin is defined as a distance from the invasive tumor front that is 5 mm or more from the resected margin [START_REF] Colevas | NCCN Guidelines Insights: Head and Neck Cancers, Version 1[END_REF]. However, the value of this empirical threshold in oropharynx has yet to be validated given its specific anatomical boundaries and the paucity of publications in this regard. Future results of cooperative trials such as EORTC-1420, PATHOS, or ECOG 3311, will probably help clarify the significance of a "close" margin.

Our study has several limitations that should be kept in mind. First of all, our results were based mainly on retrospective case-series, with a subsequent risk of multiple biases. This issue should be addressed in large and multicentric prospective clinical trials, with published protocols that allow for precise evaluation of the influence of margins, and with both available per-protocol results and intention-to-treat results. However, the expected heterogeneity of the surgical methods and procedures as well as of the pathological margin assessments between expert centers makes it very difficult to generate a reliable pooled analysis of the influence of close margins on outcomes. Secondly, the validity of our conclusions relies on the quality and the exhaustivity of the reported outcomes. The very high rate of negative margins in our pooled analysis (92.2%) was unexpectedly better than the reported rates of positive margins in comparable situations such as oral carcinoma [START_REF] Smits | Resection margins in oral cancer surgery: Room for improvement[END_REF]. This could conceivably be due to the high rate of intraoperative use of frozen sections. However, the most probable reason for this low rate of positive margins is that most of the publications were from expert teams in high-volume facilities such as academic centers. Hanna et al. recently published the results of a national quality study investigating NCDB data from 2,661 patients who underwent TORS in the USA. Thus, this study involved nearly one thousand patients more than the cumulative number of TORS patients from the studies included in our analysis [START_REF] Hanna | Positive margin rates and predictors in transoral robotic surgery after federal approval: A national quality study[END_REF]. The overall rate of positive margins was 16.9%, which is considerably higher than the 8.1% cumulative rate that we encountered. They, in fact, confirmed that the risk of a positive margin status was lower in high-volume facilities compared to medium-and low-volume facilities and that it was also lower in academic centers compared to nonacademic ones, leading to a strong publication bias in our study. Finally, masked confounding factors may interact with the performance of an analysis of frozen sections, such as the difference between patient-directed and specimen-directed sampling. Nevertheless, the processing of frozen sections during the surgical procedure to intraoperatively determine the quality of the resection was significantly associated with the final margin status in our study. It is indeed to be expected that definitive margins should be better in a given patient when frozen sections are generated and assessed intraoperatively so that further resection can be performed if needed, irrespective of the methodology used.

Conclusion

In a cumulative total of 3,619 patients who underwent transoral surgery for oropharyngeal carcinoma in published studies, the rate of positive margins was 7.8%. A positive margin status was associated with reduced local control. A close margin was not associated with decreased local control as compared to clear margins. The main factor associated with a lower risk of a positive margin was the use of frozen sections to intraoperatively determine the margin status. The degree of heterogeneity was very high among the various studies, thus calling for a definition of standards for margin status assessment in transoral surgery. Future results of cooperative trials will help clarify the significance of close margins.

  systematic review was carried out in accordance with the Cochrane handbook for systematic reviews of interventions, and the results are reported according to the Preferred Reporting of Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Records were identified by searching the MEDLINE database, using the keywords (TRANSORAL [All Fields] OR LASERS [MeSH Terms] OR LASERS [All Fields] OR LASER [All Fields] OR ROBOTICS [MeSH Terms] OR ROBOTICS [All Fields] OR ROBOTIC [All Field] OR TORS [All Fields]) AND (OROPHARYNX [MeSH Terms] OR OROPHARYNX [All Field] OR OROPHARYNGEAL [All Fields]

Table 1 .

 1 Characteristics of studies included in the systematic review.

Figure 1 .

 1 Figure 1. PRISMA 2009 Flow diagram.

Figure 2 .

 2 Figure 2. The criteria included in the univariate analysis for a reported risk of positive microscopic margins in transoral surgery for oropharyngeal carcinoma.

Figure 3 .

 3 Figure 3. The rates of positive margins among the cumulative total number of patients who underwent transoral surgery for oropharyngeal carcinoma, depending on the surgical approach used and on the reported intraoperative use of frozen sections. (TCS, transoral conventional surgery; TLM, transoral laser microsurgery; TORS, transoral robotic surgery).

Figure 4 .

 4 Figure 4. The rates of reported positive margins according to the surgical approach, in series of transoral surgery for oropharyngeal carcinoma.

Figure 5 .

 5 Figure 5. Forest plot of the comparison between positive margins and negative margins in series of transoral surgery for oropharyngeal carcinoma with respect to local control.

Figure 6 .

 6 Figure 6. Publication bias in comparison between positive and negative margins in series of transoral surgery for oropharyngeal carcinoma with respect to local control. Funnel plot of standard error by log odds ratio.

Figure 7 .

 7 Figure 7. Forest plot of the comparison between close margins and clear margins in series of transoral surgery for oropharyngeal carcinoma with respect to local control.

Figure 8 .

 8 Figure 8. Publication bias in comparison between close and clear margins in series of transoral surgery for oropharyngeal carcinoma with respect to local control. Funnel plot of standard error by log odds ratio.

  BOT, base of the tongue; SP, soft palate; PW, pharyngeal wall; CTS, conventional transoral surgery; TLM, transoral laser microsurgery; TORS, transoral robotic surgery. • LORINCZ 2015: five cases of tonsillolingual sulcus included as base of the tongue • DABAS 2014: ten cases of greater tubercle sulcus and two cases of vallecula included as base of the tongue • RICH 2009: "tonsil and soft palate" • WEINSTEIN 2012: three cases of glossotonsillar sulcus included as base of the tongue

	WEINSTEIN 2007	27	27				5 16 6		OD	TORS	0		Questiona 7.4% ble in	25
	MOORE 2009	45	19 26			15 18 3 9	OD	TORS	0		-	-	45
	WEINSTEIN 2010	47	23 23 1		13 23 9 2	OD	TORS	1		-	-	46
	COHEN 2011	50	23 24 2	1 15 24 8 3	NR	TORS	1		2	47	49
	WEINSTEIN 2012	30	14	12 *	3	1	9 16 4 1	OD	TORS	1		-	-	29
	MOORE 2012	66	40 26			30 26 3 3 SYS TORS	2		-	-	64
	LACCOURR EYE 2005 DZIEGIELE WSKI 2013 QUON 2013 MOORE 2009 LEE 2014 HENSTROM 2009 KARATZANI S 2011 FORD 2014 VANLOON 2014 PSYCHOGIO S2013 DABAS 2014 LORINCZ 2015 CHOBY 2015 CHAUHAN 2014 KACZMAR 2015 LING 2016 GILDENER 2016	166 81 48 102 43 20 223 87 34 16 6 65 16 10 2 43 20 65 38 51 18 9 6 266 10 0 45 60 24 30 * 35 13 19 * 34 16 15 2 8 3 2 9 4 6 3 12 9 1 1 114 6 43 92 38 47 1 76 35 36	60 87 34 39 6 2 SYS TORS 1 NR TCS 14 23 8 3 NR TORS 9 50 36 TCS 1 6 SYS TCS 16 21 6 NR (16) or TORS 8 8 1 3 SYS TCS (27) 22 3 -6 21 36 6 2 19 NR TORS TCS (66) or 1 8 10 NR TORS TLM (21) 27 13 9 12 7 NR 21 31 4 OD TORS TCS (200) or 19 15 1 SYS TORS 1 20 13 1 NR TORS TLM (66) 1 8 4 NR 6 4 NR TORS TCS 2 53 35 NR TORS or TLM (not 34 25 2 1 NR TORS	8 13 1 3 3 2 10 18 4 2 11 2 1 14 1 7	26	-8 ----8 -2 2 -14 -15 3 -	-142 ----47 -12 56 -19 -88 63 8 -	73 150 47 99 40 18 55 205 14 58 33 255 33 78 11 63
	ACHIM 2017	74	42 31			43 30			NR	specifi TORS	17		-	-	57
	MOLONY 2017 PARK 2017 ALBERGOT TI 2017 STEINER 2003 RUBEK 2017 GRANT 2006 MORISOD 2017 SETHIA 2018 111 79 19 55 40 8 80 66 13 1 7 160 54 58 48 48 30 21 7 59 36 29 29 15 1 2	36 16 3 21 30 1 5 63 77 1 7 1 12 7 2 13 17 16 23 2 29 1 2 43 60 5 2 1 4 1 2 8 8	NR NR NR OD SYS OD OD NR	ed) TCS TORS (51) or TLM TORS (4) TORS TLM TORS TLM TORS	14 21 22 0 4 2 3	1	-28 --7 --	-6 -29 -18 --	66 34 138 48 25 57 108
	GRANT 2009 RICH 2009 PERSKY 2018 HINNI 2013 MOORE 2018	69 84 140 89 51 28 29 * 38 * 46 128 8 12 314 20 4 0	4	8 25 30 29 33 64 76 13 0 14 3	1 2 1 5 2 3	2 SYS OD 7 OD OD 1 4 OD	TLM TORS TLM TLM TORS	0 14 5 2 7		-----	-----	69 126 79 126 306
	PATEL 2014	80	35 45			42 31			SYS	TLM	8		6	66	72
	HAUGHEY 2011 TF, tonsillar fossa; 204 98	10 6			61 74	4 5	4 2	OD	TLM	14		-	-	190
	MELONG 2015	39	19 20			9 23	2 7		OD	TLM	4		1	34	35
	WILKIE2016	153 96 38	2 1		42 74	3 7		OD	TLM	0		-	-	153
	SINHA 2018	171					62 53	3 2	2 4	OD	TLM	14		-	-	157

Acknowledgements

The authors are very grateful to Dr. Pierre Blanchard for his advice regarding the statistical analyses, and to Mrs.

Sophie Domingues for English editing.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-forprofit sectors.

Conflict of interest statement

PG is consultant (Proctor) for Intuitive Surgical Inc.


Dr Brice Leclère
email: brice.leclere@chu-nantes.fr

David L Buckeridge

Didier Lepelletier

Evaluation of a Web-Based Tool for Labelling Potential Hospital Outbreaks, a Mixed Methods Study

Keywords: Healthcare associated infection, hospital outbreak, outbreak detection, medical informatics, hospital surveillance

Background: Labelling outbreaks in surveillance data is necessary to train advanced analytical methods for outbreak detection, but there is a lack of software tools dedicated to this task.

Aim: The goal of this study was to evaluate the usability of a web-based tool by infection control practitioners for labelling potential outbreaks.

Methods: A mixed-methods design was used to evaluate how 25 experts from France and Canada interacted with a web-based application to identify potential outbreaks. Each expert used the application to retrospectively review eleven to twelve one-year incidence time series from 23 different types of microorganisms. The interactions between the users and the application were recorded and analysed using mixed-effect models. The users' comments were analysed via qualitative methods.

Findings: Over the 240 reviews completed, 439 potential outbreaks were labelled, about half of with a high probability. A significant heterogeneity was observed between users regarding their answers and behaviours (evaluation time, usage of the different options). A significant learning effect was also observed on the experts' interactions with the tool, but it did not seem to impact their answers. The content analysis of the comments highlighted the difficulty of early outbreak identification for practitioners, but also the potential utility of web applications such as the one evaluated for routine surveillance.

INTRODUCTION

In 1976, Lawrence Kunz wrote with enthusiasm about the coming of age of computerized microbiology [START_REF] Kunz | Computerization in microbiology[END_REF], already envisioning what we now call syndromic surveillance and predictive medicine. The 1970s were indeed an era of rapid development for both informatics and infection control, and the first computerized surveillance systems within hospitals were built during this period [START_REF] Gooch | Computerized system analyzes epidemiological data[END_REF][START_REF] Phillips | Application of Clinical Laboratory Database for Monitoring Infection Control and Antibiotic Control[END_REF].

Since then, infection control professionals have continued to adapt computer science innovations to address problems in their field. Technologies in routine use now include complex computer algorithms [START_REF] Trick | Computer Algorithms To Detect Bloodstream Infections[END_REF], dedicated data warehouses [START_REF] Wisniewski | Development of a Clinical Data Warehouse for Hospital Infection Control[END_REF], and cloud-based reporting systems [START_REF] Wang | Cloud Computing for Infectious Disease Surveillance and Control: Development and Evaluation of a Hospital Automated Laboratory Reporting System[END_REF]. This computational infrastructure is used for a variety of activities, such as monitoring nosocomial infections [START_REF] Atreja | Opportunities and challenges in utilizing electronic health records for infection surveillance, prevention, and control[END_REF], identifying high-risk patients [START_REF] Escobar | The neonatal "sepsis work-up": personal reflections on the development of an evidence-based approach toward newborn infections in a managed care organization[END_REF], and guiding antibiotic prescriptions [START_REF] Lee | Digital Decision Making: Computer Models and Antibiotic Prescribing in the Twenty-First Century[END_REF].

Yet, outbreak detection -another area of computerized microbiology envisioned by Kunz-has not reached the same level of development. If several attempts have been made in the last decades to implement automated detection systems within hospitals, their characteristics and evaluations varied widely, and there is still a lack of evidence about their practical utility [START_REF] Leclère | Automated detection of hospital outbreaks: A systematic review of methods[END_REF]. A major barrier is the absence of data sets in which outbreak periods are clearly labelled that could serve as reference standards.

In order to support the creation of such data sets for the evaluation of early outbreak detection methods, we developed an interactive web tool that experts can use to retrospectively examine available data and indicate when outbreaks likely have occurred.

Our goal with this study was to evaluate the tool's usability, analyse how users interacted with it, and try to understand what influenced their behaviours.

MATERIALS AND METHODS

1.Initial Design of the Web Application

To serve its purpose, the tool had to meet several requirements. First, it had to display the necessary data. As we wanted to focus on early outbreak detection, the data available prospectively are rather minimal: most clinical and microbiological data are usually gathered after an alert is issued and would therefore not be relevant in this context. However, as outbreaks can spread more easily across wards that shared a lot of patients, additional data was necessary to help users understand how wards related to one another. Second, the tool had to be interactive, to allow the users to explore the data and label potential outbreaks.

Third, in addition to keeping track of these outbreaks, the tool also needed to record how users interacted with the data. Finally, it had to be accessible simultaneously to several users from different institutions in France and Canada.

To meet the requirements of accessibility, data visualization and interactivity, we developed a web application based on the D3 JavaScript library for interactive data visualization [START_REF] Bostock | Data-Driven Documents[END_REF]. A screenshot of the application is available in Figure 1, as well as a video demonstration. Data visualization and interactivity were allowed via three panels (incidence, network and event list). The incidence panel mainly consisted of a bar chart presenting a year's worth of weekly incidence data for a given type of microorganism, along with a moving average curve to help reviewing time series that exhibited a lot of incidence variability. A red circle over a bar indicated that a notification about a potential outbreak was recorded in the hospital's notification system that week. A description of the notification was available by hovering the mouse over the circle.

The network panel was created to help users understand how the different wards were connected. The wards were represented as connected circles in a graph, laid out in such a way that the distances reflected the number of patients transferred between wards in the previous year: the more patients transferred, the closer the wards were to each other.

The incidence and network panels interacted with each other. When a user selected a period within the bar chart, the wards with new cases during that period were highlighted in the network view. The user could then hover the mouse over the nodes for these wards to obtain more information (type of ward, number of cases) and could click on the nodes to see the incidence data for these wards appear as a new layer on the bar chart (Figure 1).

To label a selected period as a potential outbreak, the user had to click on a button to add the period to the third panel of the application, the event list. The user would then be asked to describe their certainty (low, medium or high) that the event corresponded to an actual outbreak.

At the end of a review, the user clicked on a validation button to save the created events to a PostgreSQL database. Using Google Analytics, we also collected data about the duration of the review, how many selections were made in the incidence view, and how many clicks and mouse hovering events were made in the network view.

Evaluation of the Web Application

Surveillance data

We chose to focus the surveillance on twenty-three types of microorganism, based on their frequency and potential harm for the patients (Table I). A patient infected or colonized with one of these microorganisms (i.e. presenting a positive bacteriological culture) for the first time within a year was considered as a case. We used data from the bacteriological laboratory's information system to identify every case that occurred from 2013 to 2016 at Nantes University Hospital. The number of cases was aggregated by week for each type of microorganism and each year, resulting in 92 time series.

To build the network view, we used admission-transfer-discharge data that allowed us to tally the number of transfers for every pair of wards. For each year, the network was based on the data from the preceding year. The distances on the graph were calculated based on these counts using the Kamada-Kawaii layout algorithm [START_REF] Kamada | An algorithm for drawing general undirected graphs[END_REF]. Potential outbreak notifications were gathered from the hospital's adverse events and incident reporting system and a short summary was written for each of them.

Application trial

A convenience sample of twenty-five French and Canadian experts in infection control

surveillance was asked to test the application. Each expert had to review eleven to twelve randomly assigned time series between October and December 2017. After receiving their consent to participate, we sent the experts an email with detailed instructions, login credentials and links to the application and to a video tutorial. Two follow-up emails were sent one and two months after the beginning of the study.

After their initial login, each user was asked about their position, specialty and years of experience in that field. They also self-rated their Internet, computer, and surveillance skills and reported how many hours they spend on average in front of a computer each day. After completing all of their evaluations, they were asked to assess the application's usability using the System Usability Scale (SUS), a ten-item scale with a score ranging from 0 to 100 [START_REF] Brooke | SUS-A quick and dirty usability scale[END_REF]. They also had the opportunity to give their opinion about the application in an open text field.

Analysis

In order to account for the repeated nature of the measures, we estimated potential variations between users and time series with random effect models, adjusting for characteristics of users and time series when necessary. To enable comparisons between the models, the sizes of the random effects are displayed in this article using intra-cluster correlations (ICCs), which represent the percentages of the overall model variance explained by the effects.

The number of events by time series was modelled using a Poisson model, after verifying the absence of overdispersion. The length of the events and data from the Google Analytics trackers were modelled using log-linear models. The probability of the event was modelled as a quantitative variable (0 for no outbreak to 4 for a high probability) and was also modelled using a log-linear model. All statistical analyses were performed with R, version 3.5.1 [START_REF] Core | R: A Language and Environment for Statistical Computing[END_REF].

The users' comments were examined via a qualitative content analysis using the RQDA package [START_REF] Huang | RQDA: R-based Qualitative Data Analysis[END_REF] version 0.3.1 within R. Each comment was read thoroughly to identify the units of meaning, which were then coded and gathered into categories. In the article, verbatim citations of the comments are displayed within quotation marks, and modifications made for clarity are indicated with brackets.

RESULTS

Users & Participation

Out of the 25 experts recruited, 23 evaluated at least one time series. Their characteristics are displayed in Table II. Most were infection control physicians or pharmacists (78.3%), had 20 or more years of experience in their field (52.2%) and considered themselves to have high or very high skills in disease surveillance (82.6%). The majority of the experts also reported having medium to high Internet and computer skills (91.2% and 78.2%, respectively) and working on average six or more hours on a computer every day (52.2%). Most of the experts reviewed all of the time series they were assigned, and a total of 240 reviews were completed, out of the 276 initially planned (87.0%). Five time series were reviewed once (5.4%), 26 were reviewed twice (28.3%) and 61 were reviewed three times (66.3%).

Application usage

It took the users between 16 minutes and 9 seconds to evaluate a time series, with a median of 2.7 minutes. During these evaluations, they made between 1 and 194 selections in the bar plot (median: 24), hovered the mouse over the network nodes between 1 and 290 times (median: 29), and clicked on the nodes to see ward-specific incidences between 1 and 146 times (median: 14).

Part of these variations could be explained by significant differences in users' behaviour (p-values<0.001). Indeed, according to the ICC, the user effect accounted for 37% of the evaluation time variance. Similarly, differences among users explained a quarter to a third of the variances of the number of clicks (34%), the number of selections (28%) and the number of hover events (26%).

When a random effect was included for users, none of the recorded baseline user characteristics influenced the way in which the experts interacted with the application, except for the average daily computer time and the reported Internet skills. The average daily computer time was positively correlated with the number of clicks on the network nodes (p=0.02), whereas the reported Internet skills were correlated with the number of selections in the bar plot (p=0.047).

The way users interacted with the application was also partly determined by the time series they had to review. Indeed, the nature of the time series significantly influenced how many selections the users made in the bar plot (ICC=21%, p=0.047) and how much time it took to complete the evaluation (ICC=22%, p=0.02). However, they did not significantly influence how the users interacted with the network. These significant variations between time series were almost completely due to differences in the type of microorganisms, with corresponding ICCs equal to 17% (p=0.003) for the number of selections, and 18% (p=0.008) for the evaluation time.

Taking account of all of these significant effects, we also observed that the way users interacted with the application changed as they gained experience. The time to complete an evaluation decreased log-linearly with the number of time series evaluated, as did the number of selections, clicks and hover events (Figure 2).

The results of the final models are presented in the appendix.

Created events

Over the 240 evaluations completed, the users created 439 events, out of which 196 were flagged with a high outbreak probability (44.6%), 167 with a medium probability (38.0%) and 76 with a low probability (17.3%). The number of events by time series varied from 0 to 15 (median=2) and the length of the events varied from 1 to 53 weeks (median=4). As shown in the examples in Figure 3, the number, length and estimated probability of the events varied significantly between users (p-values<0.001), with ICCs equal to 46%, 35%, and 35% respectively. These measures also varied significantly between the types of microorganisms, with ICCs of 20% (p<0.001), 7% (p=0.003) and 11% (p<0.001) respectively. With the random effects taken into account, these characteristics did not vary significantly as the users gained experience with the application, nor did they change with their baseline characteristics.

Application evaluation

All of the experts who completed their set of evaluations also completed the final questionnaire (n=21). They gave on average a SUS score of 67.33, with a median at 69. Out of ten, two items of the SUS score were less favourably graded by the users: items 4 ("I think that I would need the support of a technical person to be able to use this app") and 9 ("I felt very confident using the app"). In their comments, several experts reported a "very good general opinion" and highlighted how enjoyable their experience with the application was, even if "understanding all of [its] subtleties [was] necessary to use it at its full potential".

Many experts expressed their desire to see the future developments of this work.

Besides the application itself, some experts felt that the task they were assigned was too complex, either because of the difficulty of envisioning what a potential outbreak could be ("I sometimes had trouble understanding what you meant by 'event'") or because of the difficulty of estimating the outbreak probability ("the main challenge for me was to understand the level of likelihood of each scenario"). The main difficulty, however, seemed to have been identifying the underlying "endemic trend", because "the background noises [were] too important". To help them in this task, several experts suggested adding some features, such as ward-specific statistics. For some experts, the task would have been easier

with additional microbiological and clinical data, such as the antibiotic resistance profiles, the type of infection and the case-mix of each ward. Because these data were not available, one expert noted that he had "little confidence in the answers [he had] given", and another had doubts about the reproducibility of the evaluations.

Ultimately, many experts noted the potential usefulness of the application in the context of routine surveillance. As one expert stated, "if this application could work in real time, it would be very interesting for infection control practitioners to identity wards with potential outbreaks". In this context, the lack of clinical and microbiological data was considered less critical, because "in real life [practitioners] would already have this information". Indeed, as stated by another expert, "this tool is surely very useful when used in one's own hospital, with knowledge of the different wards and of the background microbial epidemiology".

DISCUSSION

Building surveillance data in which outbreaks are correctly labelled is necessary to evaluate algorithms for early outbreak detection. In this study, we showed that the application we developed for this task was both usable and useful: it allowed experts to review incidence charts in under a couple of minutes and to identify several dozens of potential outbreaks.

The comments were for the most positive and encouraging, with several experts expressing their wish to see the application developed further. However, the application's usability was only average on the SUS score [START_REF] Brooke | SUS: a retrospective[END_REF]: users did not feel extremely confident using it and felt that they needed technical support. Even if about half of the events were categorized with a high probability, several users expressed a lack of confidence in the answers they had given.

The qualitative analysis of the comments was particularly useful to explore these apparent contradictions: it seemed indeed that the difficulties lied in the task the users were assigned, rather than in the application usage.

Identifying outbreaks is indeed complex, notably because there is no clear definition for them. Textbooks usually describe outbreaks as localized increases in the incidence of a disease [17,[START_REF] Horsburgh Cr | Infectious Disease Epidemiology[END_REF], but more practical definitions acknowledge that they cannot be reduced to mere statistical events. In practice, as Buehler et al. wrote, "the confirmation of an outbreak is a judgment that depends on past experience […], the severity [and] communicability of the condition, confidence in the diagnosis […], public health concern about outbreaks at the time, having options for effective prevention or control, and the resources required and available to respond." [START_REF] Buehler | Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC Working Group[END_REF] To summarize, identifying outbreaks requires much information and the definition is susceptible to change depending on the context and the persons involved. This probably explains the significant amount of heterogeneity that we observed among users. Providing more data, as some experts suggested, might have compensated for the lack of a formal definition, and improved the overall homogeneity of the answers. But it would have been incompatible with our goal of building data for outbreak identification at an early stage when, precisely, specific data are not yet available. This lack of specificity is common in early detection systems which usually identify situations that require human scrutiny or further investigation. Sensitivity and timeliness are priorities for such systems.

With regards to timeliness, clinical and biological signs are the first data to be available, and they constitute the theoretical basis of what is called syndromic surveillance. As data about clinical symptoms are not yet routinely computerized, our tool relied primarily on bacteriological test results. In the future, additional data could be incorporated in the tool, but for this study, the reliance on bacteriological results is a potential limitation. However, infections are sometimes treated empirically without bacteriological documentation, and even if bacteriological tests are performed, they do not have perfect sensibility and specificity.

Knowledge can also come from experience, rather than data. In our study, we only included experts without any prior knowledge of the local context to ensure blinded evaluations and to achieve a reasonable sample size. It is possible that local users would have identified different, possibly more relevant events. Experience with the tool itself could also influence the answers given by the experts: the events identified by novices might be different from those identified by more advanced users. In our study, we did observe a change in users' behaviours as they gained experience, but this learning effect did not significantly influence their answers. Unfortunately, we were not able to compare our results with those of other studies, as few tools are available to support outbreak labelling. The only one that we know of was proposed by Debin et al. for determining influenza outbreak periods at a national level [START_REF] Debin | Determination of French influenza outbreaks periods between 1985 and 2011 through a web-based Delphi method[END_REF].

However, this tool is not suitable for labelling hospital-acquired infection outbreaks, because they are less predictable than seasonal influenza outbreaks, which have one incidence peak per winter season.

One particularly encouraging result from the qualitative analysis was that many experts emphasized the great potential of the application for routine surveillance. Although it was not its initial purpose, the application could indeed easily be integrated with a hospital information system and used in daily infection control practice, notably by letting the user decide the germ and antibiotic resistance profile for which they want to explore the incidence trends. With the addition of other readily available data (e.g., antibiograms, patient transfer history) it could help practitioners investigate outbreaks. This evolution of our tool from ad-hoc outbreak labelling to routine surveillance could in fact be very useful for automated outbreak detection: infection control practitioners could record outbreaks prospectively, generating true data with a good face validity [START_REF] Buckeridge | Algorithms for rapid outbreak detection: a research synthesis[END_REF] that could be used to continuously refine the algorithms. This strategy of reinforcement learning [START_REF] Sutton | Reinforcement learning: An introduction[END_REF] could be a great example of a learning healthcare system [START_REF] Olsen | Institute of Medicine (US). Roundtable on Evidence-Based Medicine. The learning healthcare system: workshop summary[END_REF], where data on performance are routinely analysed to produce knowledge that can guide future decisions.

Of course, all these efforts to computerize infection control surveillance are not simply about integrating new technologies. They are part of a continuous process of automating some of the most demanding tasks to allow practitioners to reinvest their time in prevention activities [START_REF] Russo | Impact of electronic healthcare-associated infection surveillance software on infection prevention resources: a systematic review of the literature[END_REF]. With hindsight, the relative delay in the automation of hospital outbreak detection is understandable: the task is difficult, and the data hard to gather. For Kunz, more than forty years ago, the main problem with outbreak surveillance was that "the significant data […] are not readily available; they are buried in masses of routine background data not immediately applicable to the problem." Today, informatics can help with the complex preprocessing [START_REF] Shmueli | Statistical Challenges Facing Early Outbreak Detection in Biosurveillance[END_REF]. However, effort is still needed to improve the way in which relevant data are presented to practitioners and to facilitate outbreak detection overall, a complex but highly beneficial task for both practitioners and patients.

Figure 3. Examples of time series evaluations

The horizontal bars represent the events created by the experts. The events at the same height are from the same user.

Table I . • 8 Figure 1 .

 .81 Figure 1. Screenshot of the web application (training version)

Figure 2 .

 2 Figure 2. Users' behaviour over time

Acknowledgment

We would like to thank the panel of experts (Raoul Baron, Anne Berger-Carbonne, Philippe Berthelot, Hervé Blanchard, Alex Carignan, Stéphanie Deboscker, Arnaud Florentin, Élise Fortin, Bruno Grandbastien, Olivia Keita-Perse, Caroline Landelle, Yves Longtin, Jean-Christophe Lucet, Véronique Merle, Pierre Parneix, Caroline Quach, Christian Rabaud, Patrice Savard, Anne Savey, Nisha Thampi, Philippe Vanhems, Anne-Gaëlle Venier, and Jean-Ralph Zahar) for their essential participation to this project.

We would also like to thank the infection control practitioners of Nantes university hospital, the members of McGill Clinical & Health Informatics Research group, and the public health residents of the University of Nantes for their priceless contribution during alpha testing.

Funding Sources

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conclusion:

The interactive web application was both usable and useful for infection control practitioners. Its implementation in routine practice could help professionals identify potential outbreaks while creating data to train automated detection algorithms.

Conflict of interest statement

None declared.

Appendix. Results of the mixed models

Estimates and p-values are provided for the fixed effects, as well as the estimated variances for the random effects. The evaluation rank refers to the position of the review in the expert's list (1 = first review, 2 = second review, etc.).

Estimate