Elena Jelisic
email: elena.jelisic@fon.bg.ac.rs

Nenad Ivezic
email: nivezic@nist.gov

Boonserm Kulvatunyou

Scott Nieman
email: stnieman@landolakes.com

Hakju Oh
email: hakju.oh@nist.gov

Sladjan Babarogic
email: sladjan.babarogic@fon.bg.ac.rs

Zoran Marjanovic
email: zoran.marjanovic@fon.bg.ac.rs

Towards inter-operable enterprise systemsgraph-based validation of a context-driven approach for message profiling

Keywords: Business context, Services integration, Enterprise interoperability

Providing the business context has a potential to become a powerful mechanism for the interoperable usage and efficient maintenance of message standards. In the literature, there are multiple techniques for its representation and application. Industry use cases have identified multiple issues that come with currently used techniques, which can represent that context. Initial assessment of a logic-based technique for doing so has been conducted and showed that some of the identified issues can be resolved. This paper presents plan for validation of the logic-based technique where proposed algorithms will be assessed in realistic integration scenarios. The paper gives details of the validation steps, goals that need to be met, and indicates issues that guide our future research plans.

Introduction

Distributed and autonomous services are increasingly becoming components of the manufacturing and logistics enterprises. To achieve efficient integration of these services, message standards are needed. Messages (i.e., payload or business document specifications) define types of transaction-related information that need to be exchanged among different services. Messages are necessarily standardized because of the wide variety of inputs to those services. Inputs come from numerous business sectors, business processes, business contexts, and business representatives. However, traditional message standards have been only partially successful. We have identified four major issues inhibiting the adoption of messaging standards. First, "outof-the-box" message standards are necessarily generic, making them difficult to adapt to specific use cases. Second, the standards are typically documented in text form to facilitate human interpretation and do not have a computational form that can be processed by validation tools to assure integration and interoperability. Third, over time the standards become large supersets of duplicative data elements contributed by various industries making the standards difficult to implement. And, fourth, as more elements are added, detailed refinement or profiling of a message standard is necessary to recapture the original business intent and business context.

On-going work by the Open Application Group [START_REF]The Open Applications Group Inc[END_REF], industry, and National Institute of Standards and Technology (NIST) [2] continues to explore the concept of using business context to improve 1) the enablement of new syntax-neutral message standards and 2) the resolution of the identified issues [START_REF] Ivezic | Business process context for message standards[END_REF][START_REF] Jelisic | A Business-Context-Based Approach for Message Standards Use -A Validation Study[END_REF][START_REF] Jelisic | A Two-Tiered Database Design Based on Core Components Methodology[END_REF]. Some of the currently implemented context schemas and management approaches, however, have proved to be inadequate [START_REF] Jelisic | Knowledge representation for hierarchical and interconnected business contexts[END_REF]. On the other hand, some newly proposed, promising approaches are introduced [START_REF] Novakovic | Business Context Aware Core Components Modeling[END_REF] but must be tested with realistic industry situations to establish their capability and feasibility. Yet, such testing is resource-intensive and requires commitment and engagement of industry and standardization communities.

We have started to validate a newly proposed approach using existing scenarios already contributed to OAG-i by industry stakeholders [START_REF]The Open Applications Group Inc[END_REF][START_REF] Jelisic | Knowledge representation for hierarchical and interconnected business contexts[END_REF]. Further industry adoption of the proposed context management approach depends on its capability to address all real-world use cases. Accordingly, the theme for this paper is our proposed approach to validate a promising, context-management technique for cross-industry use cases. Since the outcome of our approach depends on the validation process, this paper provides a detailed description of that process. That description identifies the goals and steps needed to achieve those goals. To the best of our knowledge, there are no research papers that deal with validation of a proposed, business-context-management approach.

In the following, Section 2 introduces background information about important concepts that are used in the paper, Section 3 describes the validation process through steps needed to achieve identified validation goals, Section 4 describes validation domain, and, finally, and Section 5 discusses the expected results and future research steps.

Background

Message standards are expressed in terms of message components. Message components are developed and offered as parts of a message standard suite where they are used to form multiple types of messages. An important, international standard, Core Component Technical Specification (CCTS), has been proposed to enable uniform and consistent development of message components and message standards. The goal of CCTS is to advance interoperability among applications and services [START_REF]UN/CEFACT Core Components Technical Specification CCTS, version 3.0[END_REF].

A key concept in CCTS is business context, which is used to describe the business intent of using a message component or message profile, which is a subset of a corresponding message standard. In our view, capturing that intent is essential to finding and reusing the right message components and profiles, which will enable interoperability of corresponding data and services. Following CCTS, a specific business context is represented as a set of context values that are associated with their corresponding context categories (e.g., business process party profiles, geo-political location, tasks with a business process, resources available, and industry).

While the business context can indeed identify situations in which some components will be reused, we found this to be insufficient for interoperability. This is because semantic interoperability must also consider what happens in the background (e.g., transformation of the data into the database). Hence, in this paper, we assume that business context must also provide semantic definitions and validation rules that may be triggered when choosing the components in the business context. Providing those definitions and rules requires business-context knowledge.

Business-context knowledge can be represented using multiple, modeling techniques including graphical, object-oriented, logic-based, and ontology-based. For the purpose of this paper, we will employ a logic-based modeling technique -Enhanced UN/CEFACT's Context Model (E-UCM) [START_REF] Novakovic | Business Context Aware Core Components Modeling[END_REF]. E-UCM represents business-context knowledge using decentralized, directed, acyclic graphs. Each business-context category has at least one such graph that represents a list of possible values (or nodes) in the corresponding category. Then, a specific business context for a specific, information-exchange situation corresponds to a collection of nodes belonging to specific categories in the E-UCM graph.

E-UCM methodology for creating these graphs has two main parts. First, there is an infrastructure for (a) business-context presentation and expression and (b) contextualization of existing message profiles. Contextualization is supported by algorithms that can detect message components that are either relevant or irrelevant for a Requested business context. Second, there is a set of algorithms that can be used to generate the structure of new message profiles using existing message components that are relevant for Requested business context of a new message profile.

To determine a business context for a specific, information-exchange situation, we identify a collection of graph nodes that make up that business context. A list of specific nodes can be identified from E-UCM graph using E-UCM-provided expressions that are built using available operators and predicates. Every message component and profile has an associated E-UCM expression that captures its usage intent. This expression is called Assigned business context.

Along with the Assigned business context, E-UCM introduces terms Overall and Effective business context and provides directives for calculation of these terms for all situations and components, ultimately resulting in Effective business context for each component. The Overall business context can be understood as cumulative business context of some compound message component, while the Effective business context should reveal message component's relevancy in a Requested business context.

The process of assigning contexts to a message profile and its components, and calculation of Overall and Effective business contexts is called contextualization.

E-UCM Approach -On-going Validation

Our initial validation [START_REF] Jelisic | Knowledge representation for hierarchical and interconnected business contexts[END_REF] concluded that E-UCM methodology provides a powerful mechanism for business context presentation and expression. Our goal in this paper is to validate the algorithms that E-UCM proposed for both contextualization of existing message profiles and profile generation. Our approach to achieve that goal involves two steps. The first step is to assesses Effective business context as sufficient evidence of message component's relevancy or irrelevancy in the defined business context. The second step assesses the algorithms for message profile generation as capable of constructing a valid, message profile for a Requested business context. The key difference between those two goals is whether message profile structure is available or not. If it is available, profile refinement occurs for new integration scenarios. Otherwise, message profile structure should be generated using E-UCM algorithms on the basis of existing message profiles. We made two important assumptions in completing the validation study. First, we assumed contextualization would be realized by a specialized service provider. Such a service provider would need domain knowledge needed for the development (or selection from an existing standard library) and use of message components. Such domain knowledge would typically be obtained through 1) business analysis of existing business processes and 2) information exchanges that rely on documentary standards and standard operating procedures.

Second, we assumed that there is one contextualized Initial profile upon which all others are based. To obtain the Initial profile, a service provider would construct a special kind of domain knowledge that contains an initial collection of document components and a set of business rules that govern the usage of those components. Using this domain knowledge, the service provider would generate an output in the form of a contextualized Initial profile with its components assigned intended business context, corresponding to the business rules.

Our validation process is shown in Figure 1. First three steps are prerequisite and the same for both validation goals, fourth and fifth steps reveal our validation goals, while the sixth step is an assessment of validation results. The third step gathers validation cases that will be used to accomplish our, two, identified, validation goals. Each validation case is described using Requested business context.

Obtain validation cases

Figure 1 Validation process

Validation of Effective business context

The input for this validation step is semi-contextualized Initial profile with defined Assigned business context and calculated Overall business context and a set of validation cases. The output is a set of identified message profiles obtained through refinement of an Initial profile. The first step in our validation approach is to put the Initial Profile to use in a variety of validation cases, both boundary and normal, where business context is adequately and precisely expressed. For each validation case, Effective business contexts for each component will be calculated in order to identify irrelevant components.

Effective business context is calculated as intersection between component's Overall business context and business context in which corresponding component is supposed to be used (Requested business context). Intersection is conducted for each business context category separately.

In general, there are three possible outcomes: Effective business context is equal to null, Effective business context is narrower than component's Overall business context and Effective business context is equal to component's Overall business context. Effective business context can be equal to null if intersection of the Overall business context and Requested business context expressions across any of their business context categories gives an empty set. This outcome informs us that the corresponding component or field is not relevant for the Requested business context. Other two outcomes inform us that the corresponding component is relevant, but it might undergo additional considerations (because Effective business context of the component is narrower).

The refinement of an Initial profile results in the identification of new message profiles for a specific set of integration scenarios, where only components that are relevant for the Requested business contexts are included. Message profiles' refinement results in recalculation of Overall business contexts. New message profiles will have Assigned business context equal to Requested business context. Figure 2 presents a reusable Contextualization and profile refinement subprocess that can be applied to any domain. Since the calculation of Overall business context depends on message component's type it is represented currently as a subprocess and details are neglected.

Figure 2 Contextualization and profile refinement process

Validation of algorithms for message profile generation

A set of refined message profiles, obtained from the previous step, and a set of validation cases provide the input for this validation step. An output is a set of generated new message profiles constructed using E-UCM proposed algorithms. In this validation step, we will take one by one validation case, and compare Requested business context with business contexts assigned to existing profiles that we got as the output from the previous validation step. If there is a match, we will reuse it, otherwise, we will employ E-UCM algorithms to obtain components, from the repository of contextualized ones, that are relevant for a new validation case. E-UCM proposes an algorithm with five processing steps. The first step gathers all components from an existing standard component library, while each of the next steps has the goal to filter message components applicable to the Requested business context regarding different matching indexes, such as complete or partial matching. Details of this algorithm are out of scope of this paper and will be discussed in our future work.

Results assessment

To achieve the identified goals, the structure of resulting profiles, both from the first and the second step, will be compared with expected outcomes by consulting human expert or developer. Important remark is that the fifth validation step is conducted under assumption that the output of the fourth step is valid.

Validation domain

We demonstrate our validation process using a case study from a Visa application domain. Afterwards, all conclusions will be verified in other domains, such as a Smalland-Medium-Enterprise (SME) and its procure-to-pay scenarios. We have identified three variables in Visa application domain: Issuance country, Visa type and Applicant's country. Moreover, we use these three variables as our business-context categories. Associated message profiles can include different combinations of these three. To simplify the complexity, we have fixed the Issuance country=New Zealand and Visa type =Visitor, respectively.

Since each country has its own, specific visa policies, we used a two-step process to construct a set of validation cases. First, for New Zealand, for Visitor visa type we identified a list of eligible countries whose citizens can apply for a visa. Second, for the same issuance country, for the same visa type we have identified a list of visa waiver countries. After these two steps are conducted, we can identify possible visa application submissions.

In the real life, New Zealand has one Visa Application Form (VAF) for Visitor visa type and this VAF, with the same structure, is offered to all applicants, no matter their country. According to our validation methodology setup, this initial collection of document components will be analyzed by the service provider, along with the set of New Zealand's specific business rules. As the output, we will get the contextualized VAF Initial profile. By studying the structure of visa applications for different visa types and different countries, we have concluded that there is a great chance that irrelevant fields will be proposed to a certain applicant. Korean application form has the field Kanji Name that is applicable for a few applicants' countries only. New Zealand's application form has the field SWITCH card which is the United Kingdom-specific credit card type, to name a few. These examples clarify the purpose of our first goal -to approve Effective business context as enough to identify relevant fields from the VAF Initial profile. The output of this validation step would be a set of VAF profiles that would contain only components relevant to the Requested business contexts. Consequently, each applicant would get a customized VAF that contains only relevant set of fields, which will ultimately lead to less error-prone application process.

When a new visa application is recorded, first, we will compare Requested business context with business context assigned to existing profiles using the E-UCM businesscontext, matching algorithms. If there is a match, we will reuse it; otherwise, we will employ the E-UCM algorithm to construct a new message profile. That construction is based on relevant, obtained components from the repository of contextualized ones. The structure of resulting VAF profiles will be compared with expected outcomes by consulting human expert or developer. These conclusions contribute to the identified validation goals.

Discussion and Future Work

So far, first three validation steps have been conducted. Analysis of Initial profile and validation cases show that, in most situations, Effective business context can be used to identify irrelevant components from existing message profiles. However, there are some boundary scenarios where Effective business context is insufficient to make conclusions. We have identified the following types of unsuccessful validation cases. They justify the purpose of our validation process. From those example scenarios, E-UCM validation must be conducted prior to its industry adoption. Time-dependent -Values assigned to business-context categories denote the current state of an object. We identified scenarios where it was necessary to remember the history, not just the current state. For example, keeping history of applicant's country transitions might be important to decide whether some field is applicable to the integration situation at hand. In this case, Applicant's country business-context category would be used to denote transition of the applicant's country, not just his current citizenship. Example transitions include changes in citizenship, holidays, temporary work, and student-exchange programs. In practice, this means that the result of Effective business context might vary through time. Hence, considering only one state, without considering history of change, may not be sufficient to communicate needed information for the relevant integration situation.

Unpredictable -There are scenarios that lack the logic needed to determine whether some field is relevant to the applicant or not. For example, field Passport book number can often be found in VAFs. The Passport Book Number may appear in a passport in addition to the Passport Number. Some countries may have this detail in some versions of their passports, while others may not. There is no traceable guideline that will inform us which passport versions contain this detail, so defining Assigned business context for such component would be a challenge. Further, it means that calculation of Effective business context might be impossible.

Insufficient -There are scenarios where business context does not give us enough information. For example, Applicant's country may not be enough because applicant's current residence or applicant's nationality, to name a few, may also affect appearance of some field. This means that the business-context categories that are chosen to describe business context are not enough; and, other relevant categories should be introduced. In practice, Effective business context can be calculated, but it can lead to conclusions that are inadequate for the specific integration situation.

If our validation process proves Effective business context as insufficient, which we believe is the case, future research will be needed. We plan to consider using new context categories and employing data mining techniques for business context definition to resolve those failing scenarios and enable adequate calculation of Effective business context. These approaches may help identify combinations of context categories expected to adequately define a specific business context [START_REF] Jelisic | A Business-Context-Based Approach for Message Standards Use -A Validation Study[END_REF]. Also, there is a possibility for extension of E-UCM methodology to capture history data through business context categories. A possible solution would be introducing state-machine diagrams for business context definition that would enable capturing multiple states (values for specific business context category) through time.

The described on-going validation covers algorithms that can be employed for profiles' generation when there is only one document type considered. Also, E-UCM offers other algorithms that can be applied for profiles' generation when there are two, or more, paired document types. Future research will consider validation of this aspect of E-UCM methodology as well.

Disclaimer

Any mention of commercial products is for information only; it does not imply recommendation or endorsement by NIST.


Séverine Guillaume
email: severine.guillaume@cnrs.fr

Guillaume Wisniewski
email: guillaume.wisniewski@u-paris.fr

Cécile Macaire
email: cecile.macaire@univ-grenoble-alpes.fr

Guillaume Jacques

Alexis Michaud
email: alexis.michaud@cnrs.fr

Benjamin Galliot

Maximin Coavoux
email: maximin.coavoux@univ-grenoble-alpes.fr

Solange Rossato
email: solange.rossato@univ-grenoble-alpes.fr

Minh-Châu Nguyễn
email: minhchau.ntm@gmail.com

Maxime Fily
email: maxime.fily@gmail.com

Minh-Châu Nguyên

Fine-tuning pre-trained models for Automatic Speech Recognition: experiments on a fieldwork corpus of Japhug (Trans-Himalayan family)

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

The use of Transformer-type neural architectures to learn multilingual models of text and speech, coupled with methods for fine-tuning these generic representations, has opened up the possibility of developing tools for the many languages for which there is only a small amount of annotated data available. This approach has special appeal for linguistic documentation tasks: the development of semi-automatic or even automatic transcription and annotation methods based on a small amount of annotated data would reduce the annotation effort of field linguists and language workers, who could then focus their attention on linguistically and relationally meaningful tasks during fieldwork [START_REF] Thieberger | LD&C possibilities for the next decade[END_REF][START_REF] Michaud | Integrating automatic transcription into the language documentation workflow: experiments with Na data and the Persephone toolkit[END_REF][START_REF] Partanen | Speech recognition for endangered and extinct Samoyedic languages[END_REF][START_REF] Prud'hommeaux | Automatic speech recognition for supporting endangered language documentation[END_REF]. In this multidisciplinary endeavour, it is clear that "linguists and Natural Language Processing (NLP) scientists may want to adjust their expectations and workflows so that both can achieve optimal results with endangered data" [START_REF] Moeller | Integrating machine learning into language documentation and description[END_REF].

The present work reports on our experiments using a pre-trained model of speech, XLS-R [START_REF] Conneau | Unsupervised cross-lingual representation learning for speech recognition[END_REF], to develop a phonemic recognition system for a minority language of China: Japhug (Ethnologue language code: jya, Glottolog code: japh1234; see [START_REF] Jacques | Japhug[END_REF][START_REF] Jacques | A grammar of Japhug[END_REF]. The transcription of recordings in a newly documented language is a key task for fieldworkers (linguists and language workers). It is also an interesting topic for the speech processing community, as it raises several challenges, epistemological as well as practical.

First of all, the amount of data available for such languages is very small: for instance, of the 197 languages in the Pangloss Collection [START_REF] Michailovsky | Documenting and researching endangered languages: the Pangloss Collection[END_REF], which hosts audio recordings in various languages of the world (most of them endangered), only 44 corpora contain more than one hour of recordings. There is therefore a need for speech recognition methods that require as little training data as possible. In this respect, Japhug can be considered as an outlier, since there is a 32-hour transcribed corpus, freely available in the Pangloss Collection1 as well as from Zenodo [START_REF] Galliot | Deux corpus audio transcrits de langues rares (japhug et na) normalisés en vue d'expériences en traitement du signal[END_REF] 2 and as a Huggingface dataset. 3 The size of this corpus is one of the reasons for choosing Japhug as the test case for the present investigations: we wanted to be able to evaluate the amount of data that is necessary to obtain an automatic transcription of good quality -an important criterion here being the linguist's evaluation of the usefulness of the automatically generated transcript, as will be discussed again below.

Research in the field of resource-constrained Automatic Speech Recognition (ASR) has brought out "the importance of considering language-specific and corpus-specific factors and experimenting with multiple approaches when developing ASR systems for languages with limited training resources" (Morris et al., 2021, 4354). To mention two such factors:

• Endangered/little-described languages have structural features of their own, which may be widely different from those of the languages routinely taken into account in the work of the speech processing community. (It has even been argued that highly elaborate linguistic structures and typological oddities are more likely to be found in minority languages, for sociolinguistic reasons [START_REF] Haudricourt | Number of phonemes and number of speakers [translation of: Richesse en phonèmes et richesse en locuteurs][END_REF]: Haudricourt [original publication: 1961]]; Trudgill 2011.) For example, Japhug has a degree of morphosyntactic complexity that is particularly impressive, especially in view of its areal context (Jacques, 2021, passim).

• Speakers of minority languages frequently use words (or multi-word expressions, or even entire sentences) from other languages -typically the majority language of the country, or of the area [START_REF] Moore | Re-valuing code-switching: lessons from Kaska narrative performances[END_REF][START_REF] Aikhenvald | Language contact and endangered languages. The Oxford handbook of language contact[END_REF].

The presence of various loanwords, as well as cases of code-switching in the recordings, are a challenge for the automatic transcription of linguistic fieldwork data.

Conversely, there is one aspect in which automatic transcription tends to be easier for fieldwork data than for widely studied languages: namely, their high degree of orthographic transparency. Most endangered languages are languages transmitted through oral tradition, without a widely used writing system, and the transcriptions are usually made by linguists and language workers either in the International Phonetic Alphabet or in an orthography that is very close to the pronunciation. Thanks to this last characteristic one may realistically hope to achieve good quality transcriptions, as the system does not have to learn a complex spelling -unlike in the case of orthographies which have less straightforward correspondences between graphemes and phonemes (e.g. Uralic languages in Cyrillic orthography have a high degree of grapho-phonematic complexity, raising some technical difficulties: [START_REF] Gerstenberger | Utilizing language technology in the documentation of endangered Uralic languages[END_REF].

The sections below are organized as follows: we start out, in section 2, by briefly describing the model we have used. Then we move on to presenting, in section 3, the results of a first set of experiments on phonemic transcription, which show that XLS-R does indeed allow us to produce very good quality transcriptions from a small corpus of annotated data, and that these transcriptions meet a need from the linguists conducting language documentation and conservation work. However, a second set of experiments described in section 4 shows that this result is difficult to reproduce, which leads us to qualify our initial optimistic conclusion concerning the technological dimension of the work.4

2 Fine-tuning pre-trained models Principle The approach implemented in this work is based on the fine-tuning of a multilingual signal representation model, a method introduced in the field of speech recognition by [START_REF] Conneau | Unsupervised cross-lingual representation learning for speech recognition[END_REF] to build speech recognition models from little data. This approach is today at the core of many NLP models and is considered by many to be the most promising way to develop NLP and speech systems beyond the thirty or so languages (representing only 0.5 % of the world's linguistic diversity) for which there are large amounts of annotated data [START_REF] Pires | How multilingual is multilingual BERT?[END_REF][START_REF] Muller | When being unseen from mBERT is just the beginning: Handling new languages with multilingual language models[END_REF].

The proposed approach is composed of two steps. In the first step, XLS-R, 5 a multilingual model trained in an unsupervised way on a corpus of 56,000 hours of recordings in 53 languages, is used to automatically build a language-independent, 'generic' representation of the signal. In a second step, this representation is used as input to a phonemic recognition system, trained on audio data that are time-aligned with a manual transcription provided by the linguist. This second step allows to learn how to match the signal representations with labels: in this case, it is essentially the labels corresponding to the phonemes.

In our experiments, we used the XLS-R multilingual model6 and the HuggingFace API [START_REF] Wolf | Transformers: State-of-the-art natural language processing[END_REF] to use and fine-tune it. We ran the fine-tuning for 60 epochs (i.e. 60 iterations over the training data) to be assured that the fine-tuning had converged, and we kept the last model.

Using the model for phoneme prediction In order to apply the method described in the previous paragraph to the task of phoneme recognition, we simply defined a set of labels corresponding to the set of characters composing the phonemes. More precisely, the set of labels used for fine-tuning is made of the 44 characters that appear in at least one Japhug phoneme. 7 This technical choice is based on the experiments reported by [START_REF] Wisniewski | Phonemic transcription of lowresource languages: To what extent can preprocessing be automated?[END_REF] showing that the prediction of the characters composing the phonemes (instead of the phonemes as units) allows to obtain good predictions, sidestepping the task of explicitly listing the phonemes of the language (for example to specify that /ʈʂʰ/ constitutes a single phoneme, noted by a trigraph: ʈ+ʂ+ʰ). For the sake of simplicity at an initial exploratory stage, we also removed from the manual transcriptions all the punctuation marks and the other miscellaneous symbols used by linguists in their transcriptions (symbols to note linguistic phenomena of emphasis or focus, for example).

To this set of grapho-phonemic labels is added the space, to delimit words, thereby coming a step closer to the development of a true speech recognition system for endangered languages. The addition of a special character marking the word boundaries is a novelty in our work;8 it aims at allowlarge-100k-voxpopuli.

ing the system to recognize words directly. This avoids the need for post-processing or for a second system to segment the lattice of phonemes into words, such as the ones developed by [START_REF] Godard | Unsupervised word segmentation from speech with attention[END_REF] and [START_REF] Okabe | Segmentation en mots faiblement supervisée pour la documentation automatique des langues[END_REF]. To arrive at bona fide word recognition (and thus at fullfledged Automatic Speech Recognition), use of a language model is clearly the most efficient way to go, and this method has been successfully applied in the context of some minority/endangered languages [START_REF] Partanen | Speech recognition for endangered and extinct Samoyedic languages[END_REF][START_REF] Prud'hommeaux | Automatic speech recognition for supporting endangered language documentation[END_REF], but it should be remembered that there is huge diversity among the data sets available for endangered/low-resource languages, so that, surprising as it may seem, "no single ASR architecture outperforms all others" (Morris et al. 2021, 4354; see also [START_REF] Macaire | Automatic Speech Recognition and query by example for Creole languages documentation[END_REF] on two Creole languages). The use case addressed here is one in which the amount of text available is no greater than a few tens of thousands of words, i.e. an insufficient amount to train a language model according to standard workflows.

Evaluation on the Japhug language

In order to facilitate the reproduction of the experiments, the Japhug corpus is made available as a Huggingface dataset 9 which can be used off-theshelf with the tools described here.

Experimental results

The quality of our system is evaluated using two classical metrics: the character error rate (CER), i.e. the edit distance between the reference and the prediction computed at the character level, 10 and the word error rate (WER), a similar metric computed at the word level. Note that what makes the use of the latter metric possible is that the systems we trained are capable of predicting word boundaries (which was not the case in previous work such as [START_REF] Adams | Evaluating phonemic transcription of low-resource tonal languages for language documentation[END_REF].

Using a ten-hour corpus for fine-tuning XLS-R, the system obtains a CER of 7.4 % and a WER of 18.5 %. Figure 1 shows how the performances of a guage documentation setting: it constitutes common practice in character-level ASR. 9 https://huggingface.co/datasets/ BenjaminGalliot/pangloss 10 Our system is predicting a stream of characters and not of phonemes (as stated in §2, the label set is made of the characters used to write the phonemes) and the edit operations, at the heart of the CER computation, are defined directly on the characters. Computing the phoneme error rate in which each phonemes would be considered as an indivisible unit would weigh errors differently. fine-tuned model evolve for training sets whose size is close to the corpora usually collected in fieldwork on endangered/minority languages. It turns out that the CER is already very low (12.5 %) for a training corpus containing two hours of annotated data.

These two results show that the proposed approach allows to obtain transcriptions of good quality, which reach the threshold at which the framework provided by the computer tool constitutes a useful starting point (preferable to the traditional method: a completely manual input). In particular, the performance is improved by 4 points compared to the results of [START_REF] Wisniewski | Phonemic transcription of lowresource languages: To what extent can preprocessing be automated?[END_REF], which were also based on a neural method of phonemic transcription, but which learned a signal representation only from the training data, without using a pre-trained model. The word-level error is much higher than the character-level error, but the difference is primarily due to the way in which the two evaluation metrics are defined. There are significantly fewer words than characters, so that an error at the character level (which naturally translates into an error at the word level containing it) will have a stronger impact on the WER than on the CER. A closer analysis of the results shows that our system makes few errors on word boundaries: nearly 90 % of spaces are correctly predicted.

Quality assessment of transcriptions by the linguist

To evaluate the usefulness of the system described in the previous section, a specialist of the Japhug language (Guillaume Jacques) corrected the automatic transcription of a recording that he had not yet transcribed. This pilot experiment is not systematized like that of [START_REF] Sperber | Transcribing against time[END_REF] or other studies of post-editing processes in machine translation [START_REF] Nitzke | A short guide to post-editing[END_REF], and moreover concerns only 236 words, corresponding to a 2-minute recording of the Japhug language. The evaluation could therefore be dismissed as impressionistic and unreliable from the point of view of NLP tool evaluation. But it cannot be overemphasized that there is a "need for developers to directly engage with stakeholders when designing and deploying technologies for supporting language documentation" (Prud 'hommeaux et al., 2021, 491). The point of view of end users is clearly significant and relevant to guide multidisciplinary team work of the type reported here.

The evaluation experiment, even though it is conducted in a way that is not standard in NLP evaluation, leads to a clear observation: the number of corrections to be made to obtain a quality transcription is much lower than the CER suggests. The linguist only had to correct 1.9 % of the characters. The figure becomes 4.2 % if punctuation is taken into account: punctuation marks are not predicted by the system -remember that they were removed from the training corpus at the preprocessing stage -and must therefore be added manually by the person taking up the automatic transcription for further processing. The corresponding WER is at 5.9 %. The difference between the estimated CER (computed on data that have been annotated beforehand) and the number of actual corrections is largely explained by the ambiguity inherent in the task of phonemic transcription: the linguist transcribing the data does not work at an exclusively phonetic-phonological level, but makes many decisions based on high-level information (in short: word identification based on context). Table 1 shows a sample of manual corrections made by the linguist to the output of our system.

The observation of a gap between the metrics and the evaluation by the user is reminiscent of similar findings obtained in the evaluation of machine translation [START_REF] Wisniewski | Design and analysis of a large corpus of post-edited translations: Quality estimation, failure analysis and the variability of post-edition[END_REF]. Such observations are of great importance in the perspective of integrating the tools into workflows for linguistic documentation. It would seem that the actual degree of usefulness (the "real" quality) of the systems is higher than the evaluation metrics used so far would suggest. At least in the case of Japhug, the effort required to correct automatic transcrip-x tɕe kɯɕɯŋgɯ tɕe iɕqha @mingchao(u→␣ )ɯraŋg nɯ tɕu pjɤŋu tɕendɤre iɕqha nɤki @yanguo kɤti rɟɤlkhɤβ ɣɯ nɯrɟɤlpu nɯ kɯ, iɕqha nɯ, iɕqha nɯ ɯftsa nɯnɯ rɟɤlpu lusɯndɤm pjɤsɯso. tɕe nɯ rɟɤlpu lusɯndɤm pjɤsɯso tɕe, tɕendɤre nɤkinɯ, sɤtɕha ra tosɤtʂoʁloʁnɯ ʑo ɕti tɕe, tɕendɤre iɕqha nɯ, @shandong nɯtɕu ɯrmi @zhangxiaobing kɯrmi ci tɯtsɣe ɯkɯβzu ci pjɤtu, tɤtɕɯ. tɕendɤre ɯrʑaβ nɯ ɯskhrɯ mɯɲɤβdi χsɯsla ma mɯtoβzu ri tɕendɤre ɯpɕi joɬoʁndʑi ɲɤphɣondʑi pjɤra matɕi sɤtɕha ra pjɤkɤtʂoʁloʁci qhe tɕe nɯra tɕetha kɯsɤɣʑi ra pɯme ma ɲɤsɯsondʑi qhe tɕe nɯ jophɣondʑi.

y tɕe kɯɕɯŋgɯ tɕe iɕqha, @mingchao ɯraŋ nɯtɕu pjɤŋu␣ tɕendɤre iɕqha, nɤki, @yanguo kɤti rɟɤlkhɤβ ɣɯ, nɯrɟɤlpu nɯ kɯ, iɕqha nɯ(.→,) iɕqha nɯ, ɯftsa nɯnɯ rɟɤlpu lusɯndɤm pjɤsɯso. tɕe nɯ rɟɤlpu lusɯndɤm pjɤsɯso tɕe, tɕendɤre, nɤkinɯ, sɤtɕha ra tosɤtʂoʁloʁnɯ ʑo ɕti tɕe, tɕendɤre iɕqha nɯ, @shandong nɯtɕu, ɯrmi @zhangxiaobing kɯrmi ci, tɯtsɣe ɯkɯβzu ci pjɤtu, tɤtɕɯ. tɕendɤre ɯrʑaβ nɯ ɯskhrɯ␣ mɯɲɤβd(er,→i) χsɯsla␣ ma mɯtoβzu ri, tɕendɤre ɯpɕi joɬo(n→ʁ)ndʑi ɲɤphɣondʑi pjɤra matɕi, sɤtɕha ra pjɤkɤtʂoʁloʁci qhe tɕe nɯra tɕetha kɯsɤɣʑ(ɯ→i␣ )ra pɯme ma ɲɤsɯson(ɯ→dʑi) qhe tɕe nɯ jophɣondʑi.

Table 1: An excerpt from the manual corrections made to automatic transcriptions. System x, corresponding to the setup described in §3, does not predict punctuation, nor does it predict the symbol @ (which indicates Chinese loanwords), whereas system y predicts these two elements.

tions is considered "very low" by our expert on Japhug. A linguist's assessment of the amount of effort depends of course on many factors, including the degree of command of the target language. This makes the comparison from one case to another problematic; this is one of the difficulties encountered in interdisciplinary work between computer scientists and linguists. This point will be briefly taken up in the following paragraph.

Taking a critical look at the process of training statistical models

The results presented in the previous section are, to say the least, highly encouraging. They show that it is possible to achieve very good quality automatic phonemic transcriptions, even for languages for which relatively little annotated data is available (about 2 hours). Not only is the quality of the transcriptions sufficient to serve as a basis for further linguistic documentation work, but approaches based on pre-learning of representations open up the possibility of recognition at the word level, a major advance for the intended use cases (documentation of endangered languages in fieldwork). In practice, a phoneme lattice is not the best basis for further work by a field linguist. For a phoneme transcription to be complete, each individual phoneme would have to be recognizable from the audio signal, which would be contrary to all expectations, given the well-documented variability in the phonetic realization of phonemes [START_REF] Niebuhr | Perception of phonetic detail in the identification of highly reduced words[END_REF]. This variability, which carries a non-negligible part of the information contained in the signal, is particularly extensive in spontaneous speech, the object of study privileged by field linguists [START_REF] Bouquiaux | Enquête et description des langues à tradition orale[END_REF][START_REF] Newman | Linguistic fieldwork[END_REF]. Thus, the basic unit for the constitution of corpora of rare languages is clearly not the phoneme, but the morpheme (and the higherlevel units: word, sentence...).

Our initial results led us to consider more complex transcription tasks in which the system must also predict punctuation, as well as Chinese loanwords (cases of code-switching with the national language) found in Japhug documents (where they are transcribed according to the romanization conventions of standard Mandarin). The goal is, as before, to reduce the annotation effort of field linguists. Taking punctuation and loanwords into account essentially involves changing the preprocessing performed on the transcriptions before training.

The difficulties which we encountered during the development of this new system led us to study in a systematic way the degree of stability of the learning process. Neural network training is a difficult task in that it involves a very large number of parameters and relies on the optimization of a non-convex objective function. In practice, the optimization methods at the heart of deep learning rely on a very large number of hyper-parameters, 11 the choice of which has a direct impact on the performance of the resulting system. Thus, for the task of fine-tuning the XLS-R model (used in the work reported here), it is possible to change the value of more than twenty parameters that include the initial value of the learning step, its scheduling, the optimization method, the size of the batches, as well as various parameters for dropout.

We have represented in Figure 2 the performances (evaluated by the CER) obtained on the validation set during the different trainings we have performed during the development of these systems. Note that the systems were fine-tuned on a three-hour corpus (10% of which, making up 18 minutes, were used as a validation set) in order to keep the training times to a reasonable duration. The experiments we conducted with a larger corpus did not lead to improvements in the results obtained. These learning curves were obtained by varying the various parameters for optimization (training step, values for dropout, choice of the training set), but also by trying various experimental conditions: in particular, by taking into account the punctuation or not.

Among the 91 training curves shown in Figure 2, the CERs obtained on the validation set vary between 8.8 % and 28.8 % (M = 14.8, S = 2.2). Most of the learned systems perform significantly worse than the system described in our first experiments: only 6 systems have a CER at validation that is below 12.0 %, and none of them reaches the performance of the system described in section 3. Although not all of these error rates are directly comparable, these results show not only that performance on the validation set is highly sensitive to the choice of hyper-parameters (as expected), but more importantly, that the optimal value of these parameters varies across corpora, train-test splits and configurations.

However, as the results in Table 2 show, if we apply the different models obtained to the corrected text of section 3.2, the quality of the transcriptions is such that it requires only a small number of corrections. This result is all the more remarkable since these systems were only learned on 3 hours of annotated data, a reasonable amount of data to expect in scenarios of language documentation. Above all, it appears that the performance of the models on the validation set does not seem to be a value of which can only be found by trial-and-error and training a system completely. Tuning hyper-parameters tends to be highly time-consuming and resource-intensive. reliable indicator of their quality in practice. This makes their selection and more generally their development very difficult.

x y z CER validation 8.8 % 13.9 % WER 5.9 % 19.5 % 21.6 % CER 4.2 % 9.1 % 6.7 % punctuation 1.9 % 6.8 % 4.5 % Pinyin 0.7 % 2.9 % 4.0 % Table 2: Detailed evaluation of the various systems for phonemic transcription: x is the system described in section 3, y and z are two of the systems from our second series of experiments (described in §4): y is the systeme with lowest CER on the validation set, and z that with lowest CER on the test set. These last two systems predict punctuation and the @ symbol for loanwords.

In a more qualitative way, we have reported in Table 1 an extract of the transcription of this text by the system described in section 3 and by a system predicting the punctuation. It appears that, while the first system is able to achieve a perfect transcription except for Chinese words (romanized into Pinyin) and punctuation marks, the second system presents properties that may be quite interesting for innovative workflows for computational documentation of languages. First of all, it places the utterance boundaries (materialized by the dot) without errors. The division into sentences constitutes a fundamental dimension of the structure of linguistic documents, and an important dimension of the work curating transcriptions for electronic publication in language archives. Moreover, the model recognizes Chinese borrowings remarkably well, paving the way for their automatic identification. Such additional treatments down the line are key to a workflow that makes the most of a range of NLP tools. The ultimate aim is to arrive at Interlinear Glossed Texts (IGT), with annotation down to the level of the morpheme; in turn, IGT corpora have considerable usefulness in research, including possibilities for automatically inferring linguistic patterns from the glossed corpora [START_REF] Zamaraeva | Handling cross-cutting properties in automatic inference of lexical classes: A case study of Chintang[END_REF].

Conclusion

In this work, we have described how the fine-tuning of a multilingual model could be used to learn an automatic phonemic transcription system for an endangered language, and thus reduce the annotation effort of field linguists. Despite the large variability of the scores obtained on a validation set, we succeeded in developing systems whose predictions required only a small number of manual corrections by the linguist: a number that is much smaller than that estimated by the Character Error Rate (CER). This work shows the interest of this type of approach, and opens many perspectives. In particular, the approach seems to us to call for an extension of the experiments to other endangered languages (e.g. from other corpora hosted in archives of endangered languages, about which see Berez-Kroeker and Henke 2018), in order to evaluate more widely its usefulness for language documentation. We also wish, in our future work, to improve the quality of predictions at the word level, for example by integrating a language model.

Figure 1 :

 1 Figure 1: Evolution of performances as a function of the size of the training corpus.

Figure 2 :

 2 Figure 2: CER over the validation set in the course of various optimizations. The curve in red corresponds to the median value for CER at each stage.

https://pangloss.cnrs.fr/corpus/ Japhug

https://doi.org/10.5281/zenodo. 5521111

https://huggingface.co/datasets/ BenjaminGalliot/pangloss

The models and all the scripts used in our experiments are freely available https://github.com/ CNRS-LACITO/xlsr_for_pangloss.

 5 Note that many other pre-trained models are available, such as hubert-large-ls960-ft and wav2vec2-

This model is named wav2vec2-large-xlsr-53 in Hugging Face API.

This list is constructed simply by enumerating all the characters in the transcriptions and is not based on a phoneme inventory or a grapheme-to-phoneme mapping.

Note that the use of a special character directly predicted by our model is only novel in the context of a low-resource/lan-

Acknowledgments

We wish to express our deepest gratitude to the main Japhug language consultant, Tshendzin.

Financial support was given by Agence Nationale de la Recherche as part of grants ANR-

An important part of the linguistic resources used in the present work was collected in the course of the project "Himalayan Corpora: Parallel corpora in languages of the Greater Himalayan area" (ANR-12-CORP-0006).


Pierre Gagnieur

Maxime Fieux
email: maxime.fieux@chu-lyon.fr

Bruno Louis

Emilie Béquignon

Sophie Bartier

Delphine Vertu-Ciolino

Dr Maxime Fieux -

Objective diagnosis of internal nasal valve collapse

Keywords: rhinomanometry, diagnosis, nasal obstruction Level of evidence : level 4

come

Introduction

Chronic nasal obstruction is the fourth most common reason for ENT consultations in France [START_REF] Papon | Obstruction nasale chronique[END_REF] . Its prevalence is hard to estimate but the condition may affect up to 30% of the population [START_REF] Åkerlund | Prevalence of upper and lower airway symptoms: the Skövde population-based study[END_REF] . Nasal obstruction is defined as insufficient nasal airflow resulting in respiratory discomfort and is considered chronic when symptoms persist for more than 3 months [START_REF] Papon | Obstruction nasale chronique[END_REF] . The main etiologies are constitutional or acquired deformities of the various cartilages or nasal bones [START_REF] Constantian | The relative importance of septal and nasal valvular surgery in correcting airway obstruction in primary and secondary rhinoplasty[END_REF][START_REF] Egmond Mmht Van, Rovers | Septoplasty for nasal obstruction due to a deviated nasal septum in adults: a systematic review[END_REF] . Collapse of the internal valve (internal valve collapse) is a rarer cause of nasal obstruction and is more rarely considered by nasal surgeons [START_REF] Milicic | Middle nasal valve collapse: a way to resolve[END_REF] .

Missing this diagnosis can have serious consequences however: long-term nasal corticosteroid treatment may be incorrectly initiated, and septoplasty or turbinoplasty surgery may be offered to patients but prove ineffective or even deleterious [START_REF] Constantian | The relative importance of septal and nasal valvular surgery in correcting airway obstruction in primary and secondary rhinoplasty[END_REF][START_REF] Howard | Understanding the nasal airway: principles and practice[END_REF] .

The internal valve region is the flow-limiting segment of the nasal cavity [START_REF] Vogt | 4-Phase-Rhinomanometry (4PR)--basics and practice 2010[END_REF] . It is bounded by the septum, the caudal edge of the upper lateral cartilage, and the cephalic edge of the lower lateral cartilage. On entering this constricted segment, the airflow accelerates because of the Venturi effect and the intraluminal pressure drops in accordance with Bernoulli's principle [START_REF] Bloching | Disorders of the nasal valve area[END_REF] . Depending on the rigidity of the structures [START_REF] Daniel | The Nasal Ligaments and Tip Support in Rhinoplasty: An Anatomical Study[END_REF] , this pressure drop can lead to the collapse of the internal valve, obstructing the nasal passage [START_REF] Ishii | Are diagnostic tests useful for nasal valve compromise?: Diagnostic Tests for Nasal Valve Compromise[END_REF] . Internal valve collapse can be caused by trauma, aging, paralysis, but commonly occurs as an adverse effect of rhinoplasty surgery [START_REF] Wittkopf | The diagnosis and treatment of nasal valve collapse: Current Opinion in Otolaryngology & Head and Neck Surgery[END_REF] .

No imaging or functional test has ever been validated and diagnosis rests on clinical findings [START_REF] Ishii | Are diagnostic tests useful for nasal valve compromise?: Diagnostic Tests for Nasal Valve Compromise[END_REF] : collapse of the upper lateral cartilages on moderate inspiration and/or breathing facilitated by the modified Cottle maneuver (supporting the upper lateral cartilages using a cotton swab) [START_REF] Apaydin | Nasal Valve Surgery[END_REF] . The development of rhinomanometry has significantly improved the diagnosis of chronic nasal obstruction [START_REF] Schumacher | Nasal Dyspnea: The Place of Rhinomanometry in its Objective Assessment[END_REF] , by limiting the reliance on subjective clinical examinations. For internal valve collapse, while the diagnostic performance of the FRIED test, based on posterior rhinomanometry, is reportedly limited (a sensitivity of 82% and specificity of 59%) [START_REF] Maalouf | A functional tool to differentiate nasal valve collapse from other causes of nasal obstruction: the FRIED test[END_REF] , 4-phase rhinomanometry, which correlates better with patients' sensations [START_REF] Vogt | Four-phase rhinomanometry: a multicentric retrospective analysis of 36,563 clinical measurements[END_REF] , should reveal a hysteresis loop in the inspiratory curve in patients with internal valve collapse [START_REF] Vogt | Airway assessment by four-phase rhinomanometry in septal surgery: Current Opinion in[END_REF] . However, this notion of a looped curve has not been precisely defined and to our knowledge, no data have been published on the use of 4-phase rhinomanometry to diagnose internal valve collapse. The objective of this work was therefore to evaluate the potential of 4-phase rhinomanometry as a diagnostic test for internal valve collapse.

Methods

Study design and population

This was a case-control diagnostic accuracy study of all adult patients (> 18 years old) who attended the otolaryngology clinic of Lyon University Hospital for chronic nasal obstruction between January 2019 and January 2021. Patients were excluded if the obstruction was due to a tumor, chronic sinusitis or sinonasal polyposis, as determined by endoscopic examination and/or imaging.

Ethics

The study was conducted in accordance with the Declaration of Helsinki and was approved by the local ethics committee (approval n°21-137) but written consent was waived (retrospective study). All data were anonymized.

Data collection and clinical evaluation

The data collected were the patients' age, sex, body mass index (BMI), atopic status, smoking status, cardiovascular history (to eliminate a contraindication for oxymetazoline), and history of nasal trauma, paralysis or rhino-sinus surgery. Each patient was examined by two senior ENT physicians in a blinded fashion in daily clinical routine. During the first consultation, patients were examined and rhinomanometry was scheduled. Patients were then reexamined by the second physician when they returned for the rhinomanometry measurements. Disagreements in the results of the physical examinations were resolved at the last medical visit by a third senior ENT physician. The physical examination involved: i) static examination of the nasal pyramid: nasal shape, presence of deviation, frontal width of the middle third (thin, normal, wide), appearance of the dorsum on profile view (straight, hump, kyphosis), presence of anterior septal dislocation and/or inferior turbinate hypertrophy on anterior rhinoscopy, presence and type of septum deviation on nasal endoscopy (anterior dislocation, caudal dislocation, C-shaped deviation, or various combinations thereof); and ii) functional examination: behavior of the lateral cartilage on weak or moderate inspiration, effect of passive abduction of the triangular cartilages with a cotton swab (modified Cottle maneuver), and effect of lateral traction of the nostril and cheek (Cottle maneuver). Internal valve collapse was defined by the observation of lateral wall collapse on low to moderate inspiration and/or a reduced sense of obstruction during the modified Cottle maneuver. Patients who had lateral wall collapse on inspiration but whose breathing was not improved by the modified Cottle maneuver were not considered to have internal valve collapse.

4-phase rhinomanometry

The 4-phase anterior rhinomanometry measurements were performed by a third ENT physician blinded to the patients' diagnosis ("collapsed" or "non-collapsed") using a Rhinolab 4-Rhino device (Rhinolab GmbH, Freiburg, Germany), and the 4-Rhino software (v. 6.1.1). All patients were examined in sitting position after 30 minutes' rest. The contralateral nostril was occluded with medical tape, to avoid modifying the structure of the nasal wing and nasal valve area. The data collected for each nasal cavity were the vertex resistance (VR), the effective resistance (REff) and the flow/pressure curve [START_REF] Vogt | 4-Phase-Rhinomanometry (4PR)--basics and practice 2010[END_REF] . The area of the hysteresis loop in the flow/pressure curve, the "inspiratory loop area" was obtained (Figure 1) by subtracting the area under the curve during phase 2 of inspiration from the area under the curve during phase 1. A nasal decongestant was then administered (oxymetazoline spray, 50 micrograms in each nostril if not contraindicated) and measurements were repeated under identical conditions 15 min later [START_REF] Clement | Consensus report on acoustic rhinometry and rhinomanometry[END_REF] .

Statistical analysis

Univariate comparisons between collapsed and non-collapsed patients were performed using t tests for quantitative variables (Mann-Whitney tests if the assumptions of the t test were not met) and chi square tests for categorical variables (Fisher exact tests for small sample sizes). To evaluate the diagnostic accuracy of the inspiratory loop area for internal valve collapse (independent of prevalence, the intrinsic diagnostic accuracy), the values obtained for the right and left nasal cavity were separately compared before and after nasal decongestion with the clinical diagnosis of each patient (collapsed or non-collapsed), used here as the reference test. Contingency tables were built by dichotomizing the loop areas at different thresholds. The corresponding receiver operating characteristic (ROC) curve was plotted and the threshold with the highest Youden index (combined sensitivity and specificity) was selected [START_REF] Metz | Basic principles of ROC analysis[END_REF] . After confirming that the results obtained for left and right nostrils were equivalent, the data were combined to increase the size of the dataset and thus the power of the statistical analyses. ROC curves were also constructed for the effective resistance and vertex resistance as well as for the FRIED test applied to the same data. The threshold for statistical significance was set at 0.05. All analyses were performed with the software R (v. 4.1.2, www.rproject.org).

Results

Study population

Sixty-six patients with chronic nasal obstruction were included, 36 of which were diagnosed as having internal valve collapse (26 bilaterally, 10 unilaterally), while the remaining 30 were classified as noncollapsed. Patient characteristics are reported in Table 1 and physical examination results in Table 2.

There were no significant differences between the two groups in terms of history of nasal trauma and nasal surgery, age, sex, height, weight, smoking habit or atopic status. In terms of physical examination results, the middle third of the nose was significantly thinner in the collapsed group (16/36, 44%) than in the non-collapsed group (3/30, 10%; p = 0.03) and there was more turbinate hypertrophy in the non-collapsed group (15/30, 50%) than in the collapsed group (6/36, 17%; p = 0.005).

4-phase rhinomanometry

Effective resistance and vertex resistance, and inspiratory loop area before nasal decongestion all differed significantly between the two groups (Figure 2a and2b). After nasal decongestion on the other hand, the only significant difference between the two groups was for the loop area, which was larger in the collapsed group than in the non-collapsed group (mean ± standard deviation, 87.0 ± 104.9 versus 8.2 ± 8.8 Pa•L•s -1 ; p < 0.001). Rhinomanometry results are presented in full in Table 3.

ROC curves

The data for right and left nostrils were analyzed together, as explained in the methods section. The threshold value of the inspiratory loop area (hysteresis area) before nasal decongestion with the highest Youden index was 17.3 Pa•L•s -1 , with a corresponding sensitivity and specificity of 88.3% [95% confidence interval, 80.0-95.0%] and 89.9% [82.6-95.7%] respectively (Figure 3a). In the data obtained after nasal decongestion, the optimal threshold was 15.1 Pa•L•s -1 and the corresponding sensitivity and specificity were 91.8% [83.7-98.0%] and 87.3% [78.2-94.5%] respectively. The ROC curve constructed from the difference in resistances (VR -REff) indicated an optimal threshold of 0, for which the sensitivity and specificity were 56.7% [45.0-70.0%] and 72.5% [62.3-82.6%] respectively (Figure 3b). The ROC curve constructed from the FRIED test results revealed an optimal threshold of 82.6 mL•s -1 , with a corresponding sensitivity and specificity of 80.0% [70.0-90.0%] and 76.8% [66.7-87.0%] respectively (Figure 3c).

Discussion

In this group of patients, the inspiratory loop area in 4-phase rhinomanometry curves was significantly larger in patients with internal valve collapse than in those with a different cause of nasal obstruction.

A cutoff inspiratory hysteresis area accurately reproduced the results of physical examination as the reference standard. These results suggest that 4-phase rhinomanometry can be used in this way to diagnose internal valve collapse with a high sensitivity and specificity (up to 90%).

During the first phase of inspiration, the intranasal pressure decreases on lowering of the diaphragm) and the airflow increases accordingly. Internal valve collapse can occur at a certain flow rate because of the Venturi effect, leading to a drop in the flow rate at the same differential pressure. This leads to hysteresis in the flow/pressure curve in the second phase of inspiration, when the differential pressure decreases once more. This phenomenon has never previously been studied or quantified as a means of discriminating patients with internal valve collapse from those with other causes of nasal obstruction. The results of this study show that hysteresis can also be observed, albeit to a lesser extent, in patients without internal valve collapse, with a mean loop area of 11.0 Pa•L•s -1 (± 15.9) in the non-collapsed group. This can be explained by the inertia of the nasal walls, which are not perfectly elastic, and reflects the dynamic narrowing of the internal nasal valve area during inspiration, which in these patients has no clinical impact. The optimal threshold value of the inspiratory loop area to diagnose or eliminate internal valve collapse in this group of patients was 17.3 Pa•L•s -1 . We chose to use the measurements before nasal decongestion (with oxymetazoline) because we believe these more reliably reflect patients' respiratory physiology. Nasal decongestion is useful to diagnose nasal hyperreactivity [START_REF] Vogt | Airway assessment by four-phase rhinomanometry in septal surgery: Current Opinion in[END_REF] but does not reflect the pathophysiology of internal valve collapse. The mean inspiratory loop areas were similar before and after nasal decongestion in both groups (75.5 Pa•L•s -1 vs 87.0 Pa•L•s -1 in the collapsed group and 11.0 Pa•L•s -1 vs 8.2 Pa•L•s -1 in the non-collapsed group, respectively). This suggests that the potential decrease in resistance (and thus increase in flow) induced by vasoconstrictors has little or no effect on valve collapse. This may be because vasoconstrictors affect the entire nasal mucosa, increasing the diameter at the entrance and exit of the valve by a similar amount. Since the Venturi effect, which depends on the difference in inlet/outlet diameter, is not strengthened [START_REF] Kelly | Detailed flow patterns in the nasal cavity[END_REF] nasal decongestion has no aggravating effect on internal valve collapse. Another possible explanation for this small effect of vasoconstriction is the low vascular density of the valve area.

The optimal diagnostic accuracy with the inspiratory loop area (sensitivity, 88.3%; specificity, 89.9%) was much higher than obtained from the difference between VR and REff (sensitivity, 56.7%, specificity, 72.5%), a measure described by Vogt et al. (2007), and higher also than obtained with the FRIED test based on posterior rhinomanometry, both as measured here (sensitivity, 80.0%, specificity, 76.8%) and as reported by Maalouf et al. (2016) (sensitivity, 82%, specificity, 59%). The lower diagnostic accuracy of the FRIED-test is probably due to the greater susceptibility of posterior rhinomanometry to measurement bias. In posterior rhinometry indeed, the pressure sensor is placed inside the mouth and measurements are therefore affected by the position of the soft palate [START_REF] Vogt | 4-Phase-Rhinomanometry (4PR)--basics and practice 2010[END_REF][START_REF] Vogt | The new agreement of the international RIGA consensus conference on nasal airway function tests[END_REF] . In 4phase rhinomanometry on the other hand, the sensor is placed at the entrance of the nasal cavity, a more reproducible and reliable position for measuring the pressure of the nasal cavity with reduced inter-individual variability.

To our knowledge, no functional or imaging test has ever been shown to have such a high level of diagnostic accuracy for internal valve collapse. The reference standard is still physical examination, which is known to be limited by inter-practitioner and inter-patient variability [START_REF] Maalouf | A functional tool to differentiate nasal valve collapse from other causes of nasal obstruction: the FRIED test[END_REF] . Our results suggest the inspiratory loop area in 4-phase rhinomanometry offers a similar level of specificity and sensitivity as physical examination, while being an objective test. This could have immediate benefits in clinical practice. Indeed, since the treatment options for internal valve collapse vary in difficulty and associated risks [START_REF] Apaydin | Nasal Valve Surgery[END_REF] (rhinoplasty surgery with autologous cartilage grafting [START_REF] Gruber | The Spreader Flap in Primary Rhinoplasty[END_REF] , titanium [START_REF] Van Den Broek | The effect of the titanium butterfly implant on nasal patency and quality of life[END_REF] or bio-absorbable implants [START_REF] Nicoló | Absorbable Implant to Treat Nasal Valve Collapse[END_REF][START_REF] Stolovitzky | A prospective study for treatment of nasal valve collapse due to lateral wall insufficiency: Outcomes using a bioabsorbable implant: Nasal Valve Treatment Using an Absorbable Implant[END_REF] , injection of fillers [START_REF] Nyte | Hyaluronic acid spreader-graft injection for internal nasal valve collapse[END_REF][START_REF] Nyte | Spreader Graft Injection with Calcium Hydroxylapatite[END_REF] , insertion of valve dilators), diagnostic objectivity is essential to validate surgical indications. This should improve patient adherence to the proposed treatment, while the presence of an objective measure in patients' medical files should help to resolve medicolegal problems that may arise from adverse outcomes [START_REF] Hosemann | Danger points, complications and medico-legal aspects in endoscopic sinus surgery[END_REF][START_REF] Razmpa | Litigation After Nasal Plastic Surgery[END_REF][START_REF] Thomassin | Rhinoplasty: medicolegal issues[END_REF] .

The studied cohort was representative of the population of patients typically seen in otorhinolaryngology clinics for chronic nasal obstruction. Indeed, both groups of patients had REff values greater that 1 before decongestion (1.34 and 1.15 respectively in the collapsed and noncollapsed groups), corresponding to a high level of nasal obstruction [START_REF] Vogt | 4-Phase-Rhinomanometry (4PR)--basics and practice 2010[END_REF] . Our results are therefore relevant to clinical practice. The inspiratory loop area is calculated directly from the flow-pressure curve produced by the 4-phase rhinomanometry software and requires no specific training for practitioners, and no additional measurements for patients. The study avoided various biases often associated with diagnostic test accuracy studies [START_REF] Delacour | Critères d'évaluation de la validité d'un test biologique[END_REF][START_REF] Doyon | Evaluation of diagnostic methods[END_REF][START_REF] Knottnerus | Evaluation of diagnostic procedures[END_REF] . Additional strengths were the blinding of the two physical examinations and of the 4-phase rhinomanometry measurements with respect to the clinical diagnosis. On the other hand, the limitations of this study include its small size (n=66) and the fact that the 4-phase rhinomanometry measurements were performed on each nostril separately, even if the contralateral nostril was occluded with medical tape rather than a nasal plug to avoid altering the structure and biomechanical properties of the studied nostril [START_REF] Vogt | 4-Phase-Rhinomanometry (4PR)--basics and practice 2010[END_REF] . Note however that the analysis by nostril rather than by patient increased the statistical power of the analysis and was further justified by the significant proportion of patients (10/36) with unilateral internal valve collapse.

Conclusion

The results of this study support the use of the inspiratory loop area in 4-phase rhinometry as a reliable and objective alternative to physical examination for the diagnosis of internal valve collapse.

Larger studies with a predefined threshold loop area are nevertheless required to confirm these promising results.

Tables and Figures

Figure 1 . 4 -Figure 2 . 4 -

 1424 Figure 1. 4-phase rhinomanometry flow/pressure curve

Table 1 : Study population Note:

 1 The values correspond to the numbers (proportions) for the categorical variables and the means (standard deviation) for the quantitative variables. * indicates statistical significance (p< 0.05).

		Collapsed (n = 36)	Non-collapsed (n = 30)	p-value
	Age (years)	40 (±15)	29 (±11)	0.001*
	Female Gender	14 (39%)	12 (40%)	1
	Height (cm)	175 (±9)	174 (±10)	0.27
	Weight (kg)	75 (±14)	75 (±18)	0.93
	Tobacco use	31 (86%)	23 (77%)	0.50
	Atopy	32 (89%)	23 (77%)	0.32
	History of nasal trauma	24 (67%)	17 (57%)	0.56
	History of rhino-sinus surgery	2 (6%)	1 (3%)	1

Table 2 : Nasal clinical evaluation 344

 2

			Collapsed (n = 36)	Non-collapsed (n = 30)	p-value
	Nasal ethnic type
		Caucasian	36 (100%)	30 (100%)	0.46
		African	0 (0%)	0 (0%)
		Asian	0 (0%)	0 (0%)
	Deviated Nose	17 (47%)	19 (63%)	0.29
	Kyphosis	16 (44%)	17 (46%)	0.46
	Width of the middle third
		Narrow	16 (44%)	3 (10%)	0.03*
		Normal	19 (53%)	23 (77%)
		Wide	1 (3%)	4 (13%)
	Septal deviation	26 (87%)	32 (89%)	1
		Anterior dislocation	6 (17%)	13 (43%)	0.05*
		Inferior dislocation	11 (31%)	18 (60%)	0.01*
		C-shaped	23 (64%)	15 (50%)	0.32
		Vomerian spur	21 (58%)	17 (57%)	0.58
	Turbinate hypertrophy	6 (16%)	15 (50%)	0.005*
	Upper lateral cartilage collapse	36 (100%)	0 (0%)	0.001*
	Modified Cottle maneuver	36 (100%)	0 (0%)	0.001*
	Cottle maneuver	10 (28%)	2 (6%)	0.003*
	345	Note: The values correspond to the numbers (proportions) for the categorical variables and
	346	the means (standard deviation) for the quantitative variables. * indicates statistical
	347	significance (p< 0.05).

Table 3 : 4-phases rhinomanometry results (data by nostril)

 3 Note: Values are means (standard deviation) for quantitative variables. * indicates statistical significance (p< 0.05).

		Collapsed (n = 60)	Non-collapsed (n = 72)	p-value
	VR before ND (Pa.L.s -1 )	2.41 (±2.76)	1.95 (±1.91)	0.037*
	REff before ND (Pa.L.s -1 )	3.47 (±4.10)	2.14 (±2.36)	0.005*
	VR after ND (Pa.L.s -1 )	2.14 (± 2.64)	1.67 (±1.14)	0.88
	REff after ND (Pa.L.s -1 )	2.51 (±3.10)	2.51 (±3.65)	0.21
	VR-REff before ND (Pa.L.s -1 )	-1.07 (±2.60)	-0.21 (±1.10)	0.001*
	VR-REff after ND (Pa.L.s -1 )	-0.80 (±3.05)	-0.35 (±2.26)	0.016*
	Area before ND (Pa.L.s -1 )	67.5 (±81.8)	7.9 (±17.6)	< 0.001*
	Area after ND (Pa.L.s -1 )	60.3 (±122.5)	7.1 (±9.6)	< 0.001*

Abbreviations: VR, Vertex Resistance ; REff, Effective Resistance ; VR-REff, difference between VR and REff ; ND : Nasal Decongestion (with oxymetazoline).

Acknowledgments:

The proofreading of this article was supported by the Bibliothèque Scientifique de l'Internat de Lyon and the Hospices Civils de Lyon.

This study was not sponsored by any external financial support.

Data Availability Statement : The datasets used during the current study are available from the corresponding author on reasonable request.

Author contributions: PG, MF, EB and DVC contributed to the conception and design of the study.

All authors contributed to the drafting of the article. MF and BL performed the statistical analysis.

Figure 3. ROC curves constructed from the hysteresis loop area (3a), difference in resistances (3b) and FRIED test (3c)

Legend: The ROC curve constructed from the hysteresis loop area (3a), from the difference in resistances (VR -REff) (3b) and form the FRIED test results (3c) are given with the highest Youden index in Pa.L.s -1 , a corresponding sensitivity and specificity respectively in %.