Jean-Sylvain Liénard

Claude Barras
email: claude.barras@limsi.fr

François Signol
email: signol_francois@yahoo.fr

TIME STRUCTURE AND DETECTION OF THE MULTIVOICED SEGMENTS IN MIXED SPEECH

Keywords: voicing, pitch, multipitch, speech separation

When two speech signals are mixed in a single channel the voiced parts of any of them remain mostly unaltered during the voicing interruptions of the other, i.e. pauses and voiceless consonants. The mixture is made of 3 types of multivoiced segments noted 0V (unvoiced), 1V (one voicing) and 2V (two voicings). A statistical study of readaloud texts reveals that total time spent in the 1V state is twice as long as the time spent in any of the other states. The HSC multipitch algorithm, based on a specific mechanism that eliminates the f 0 halving and doubling errors, is used to locate the 3 segments types in the signal. This feature is illustrated by the task of spotting a short utterance repeatedly mixed with a long text.

INTRODUCTION

Today the Pitch Estimation Algorithms for a single speech signal (sPEA) produce more voicing errors than f 0 errors. The most part of them occur at the beginning and end of the voiced segments, where the periodicity is uncertain.

The problem remains and even augments in the f 0 estimation of mixed speech signals (co-channel speech), which is important in the speech separation perspective [START_REF]Speech separation by humans and machines[END_REF] [START_REF] De Cheveigné | Multiple F0 estimation[END_REF]. In addition to difficulties of the single voice case, spectrotemporal interferences between the voice sources produce unpredictable errors. Determining whether a given frame or segment of the mixed signal comes from one or several periodic sources is a challenging issue, with applications in several domains: speech separation, automatic recognition, diarization, transcription alignment, database annotation.

The first part of the present study deals with the time structure of the voiced and multivoiced segments in the Keele database [START_REF] Plante | A pitch extraction reference database[END_REF]. The second part deals with their automatic detection in the same data, using a Multiple Pitch Estimation Algorithm (mPEA) of which only the Voiced-Unvoiced decision is taken into account.

VOICED AND MULTIVOICED SEGMENTS

In a single-speaker signal voicing may be defined either acoustically as the presence of a "voice bar", or phonologically as a distinctive feature. The terms Voiced and Unvoiced (V and U) used in this paper refer to the acoustical meaning. In a 2speaker mixture this binary categorization does not suffice because three types of frames or segments may appear, noted 0V for no voicing, 1V for a single voicing, and 2V for two voicings.

Voicing in single speaker signals

Any part of a single speaker signal may be described acoustically as an alternate series of V and U segments, with some uncertainty on their boundaries. Table 1 gives for each Keele speaker (5 females, 5 males) the total reading duration of the text "The Northwind and the Sun" (pauses included), as well as the cumulative fraction of time spent in the V and U states. Those figures were computed from the f 0 labels provided by the authors, who used an autocorrelation sPEA working on the associated EGG data (window 25.6 ms, manual corrections). spkr > fl f2 f3 f4 f5 m1 m2 m3 m4 m5 dur (s) 32.2 33.7 30.5 31.6 38.7 37.4 31.9 27.2 33.7 40.3 %V 47.5 56.4 49.5 57.1 48.0 48.7 43.4 53.8 48.2 51.4 %U 52.5 43.6 50.5 42.9 52.0 51.3 56.6 46.2 51.8 48. 6 The above results suggest that the V and U states have about the same 50% probability. They contrast with those reported in [START_REF] Hu | Segregation of unvoiced speech from nonspeech interference[END_REF], i.e. approximately 75% V and 25% U. This apparent discrepancy reflects the fact that the mentioned International Congress of Phonetic Sciences (ICPhS XVII), Hong Kong, August 17-21, 2011 study was built on a phonological basis (counting the voiced versus unvoiced phonemes from a phonetic transcription), while the present study deals with acoustical measures and takes the pauses into account.

Figure 1 shows the duration distribution of both types, all speakers pooled. The mean duration is the same: 156 ms for the V segments, 158 ms for the U segments. But the distributions differ in some respects. The frequency of occurrence of the V segments decreases regularly as their duration increases, up to a maximum of 800 ms. On the other hand, the distribution of the U segments exhibits a clear mode around 120 ms and falls rapidly until a value of some 250 ms is reached. Then one observes pauses, less and less frequent as their duration increases up to 1.7 s.

The 120 ms mode of the U segments may reflect the voicing interruption of the voiceless consonants (phonemic value), while the long interruptions (pauses, above 250 ms), contribute to the prosodic/semantic structuration of the discourse.

Voicing in mixed signals

When two single-speaker sequences are mixed, the multivoiced segments 0V, 1V and 2V exhibit a different distribution. Table 2 gives their cumulative and mean durations, for all the database files concatenated and superimposed to themselves in a different order (counts from the given f 0 labels). The observed cumulative durations could be predicted by simple calculus, knowing that the V and U states of the constituents were equally probable. The main point is that the 2V segments occupy only 25% of the total duration. If the cumulative durations of the V and U singlespeaker segments were those reported in [START_REF] Hu | Segregation of unvoiced speech from nonspeech interference[END_REF], the cumulative durations of the 0V, 1V and 2V segments in the 2-speaker mixture would be 6%, 38% and 56%, respectively, i.e. the 2V segments would by far outnumber the other segments.

A consequence of this finding is that a separation strategy, based in the first place on the detection and identification of the sole 1V segments, may be envisioned. This view relates closely to the notion of "glimpsing" [START_REF] Cooke | Glimpsing speech[END_REF].

Figure 2 shows that the distributions of the 3 types of segments are very much alike: similar mean and max values (about 80 ms and 300 ms), and similar smoothly decreasing shape. The 25/50/25% proportion for the 0V/1V/2V segments keeps approximately true for all duration classes.

MULTIPITCH ESTIMATION

Few mPEAs are available today. The one used here works in the frequency domain and is called HSC (Harmonic Suppression Comb) [START_REF] Lienard | Using sets of combs to control pitch estimation errors, Acoustics'08[END_REF] [7].

Principle of the HSC multipitch algorithm

HSC strictly performs a frame-to-frame analysis, without any top-down knowledge or postprocessing. It makes use of spectral combs, i.e. sets of discrete unit values (teeth), multiples of the base frequency f c that covers the frequency interval given for f 0 . The crosscorrelation with the modulus of the short-term spectrum yields a "pitch function" which has a series of peaks noted (p,q) where f c /f 0 =p/q (p and q positive integers). The peak corresponding to the fundamental is the (1,1) peak. There may be several (1,1) peaks if several periodicities are present.

In a single-pitch estimator the main peak (1,1) is usually larger than the parasitic peaks, so that a simple maximum detection performs correctly. But in a multipitch situation the main peak of the second sound may be smaller than a parasitic peak of the dominant sound, which causes a harmonic or sub-harmonic error.

The specific feature of HSC is a mechanism that suppresses most of the parasitic peaks by jointly using two families of irregular combs: the missing teeth combs eliminate the sub-harmonic errors and the negative teeth combs eliminate the harmonic errors. This feature provides a crucial advantage in detecting the voicings in mixed speech.

Performance of single-pitch HSC on the Keele single-speaker data

In order to test the mPEA in the usual singlespeaker situation HSC was allowed to deliver only one pitch/voicing hypothesis per frame. The results were compared to the pitch labels provided with the database (table 3). The voicing errors were counted on the set of valid frames, i.e. the frames where the pitch could be correctly defined according to the pitch labels and the chosen f 0 interval (75-600 Hz). The pitch errors were counted on the set of frames jointly declared voiced by HSC and the pitch labels.

The above usual pitch/voicing measurements do not easily extend to the multipitch case. In the rest of the paper the Recall/Precision measurement is preferred. The Recall indicates to what extent all of the correct information has been recovered, while the Precision tells what proportion of the recovered information is correct. Those quantities are given as percentages, as well as the F-measure which is a tradeoff (harmonic mean) between recall and precision. Table 4 shows those figures for the above voicing evaluation. Those measurements indicate that the voicing detection by HSC is in good agreement with the database labels, despite the differences in the analysis methods used. In the next section the HSC voicing estimates on single signals are considered as the reference, in order to fairly evaluate the multipitch behavior of the algorithm.

MULTIPLE VOICING DETECTION

2-voicing estimation of a speech mixture

The speech material consisted of a concatenation of the 10 Keele speech files, after level equalization. This operation was performed twice, in different permutation orders, yielding two signals having the same total duration. The voicing references were given by HSC in monopitch on both signals. A "reference multivoicing" function was established by adding the number of voicings found in both, thus taking the value 0, 1 or 2.

Then the two signals were mixed and the mixture was analyzed by the same HSC, now in bipitch mode (2 voicing hypotheses), yielding a "detected multivoicing" function varying in the same range. Both functions were compared using the recall/precision measurements applied to 4 types of entities: voicing hypotheses, 0V, 1V and 2V frames (table 5). The first line, compared to table 4 (single-pitch estimation on single-speaker signals), demonstrates the good behavior of HSC in this bipitch estimation of a 2-speaker mixture: the degradation, in terms of individual voicings, is limited to a small loss in recall (1% more misses) and a moderate loss in precision (12.4% more false alarms). The next 3 lines, in terms of multivoiced frames, indicate that:

• the 0V frames are well detected • the 1V frames are poorly recovered (many misses) but with a good precision (few false alarms). • the 2V frames are well recovered but with a poor precision (many false alarms).

LOCATING A REPEATED SHORT UTTERANCE IN A SPEECH MIXTURE

We are now interested in locating a short utterance repeatedly mixed with a long speech signal. It may happen that two mixed signals do not locally produce many 2V frames, because some voiced frames of the first speaker appear at instants where the second is in the unvoiced state. If the sequences are long enough the distributions of figure 2 reappear, but there is a possibility that short sequences cannot be detected on the sole basis of the 2V voicing.

In order to illustrate this point a Keele sequence (speaker f5) has been repeatedly mixed with the first 0.8 s of another sequence (speaker m3, 5 syllables) at 1.5 s intervals. The beginning of the mixture (8 s) is illustrated in figure 3: spectrograms of the constituents, the frame to frame multivoicing functions and their smoothed version (1 s averaging and low-pass filtering). In this example the first 3 occurrences of the short sentence could be detected with a threshold value of 1 or 0.9; the 4th would be missed because there is no real overlap of the signals in that region.

However the reference and test smoothed functions are very similar. This means that despite the false alarms observed in the 2V detection, the information provided by the multivoiced segments remains and could help to locate the superimposed sequences.

CONCLUSION

The mixture of two speech signals in the reading style can be acoustically analyzed as a series of segments pertaining to 3 categories according to the number of simultaneous voicings: 0V, 1V and 2V. The 1V segments occupy about half of the total duration. Using a multipitch estimation algorithm as a frame-to-frame multivoicing detector gave satisfactory results. Time integration of the number of detected voicings looks promising for spotting short speech sequences superimposed to a long speech flow.

Table 2 :

 2 % duration for the 3 multivoicing states in a random mixing of the Keele database with itself.

Figure 1 :

 1 Figure 1: histogram of the V and U segments durations from the whole Keele database.

Figure 2 :

 2 Figure 2: distribution of the 0V, 1V et 2V segments durations in the mixed signal formed by addition of two permutations of the Keele database.

  multivoicing fct (from single-spkr data) and smoothed version multivoicing fct (from multipitch analysis) and smoothed version

Figure 3 :

 3 Figure 3: from top to bottom: spectrograms (8s) of the 1st sound, the second (repeated), the mixture, the reference multivoicing function and its smoothed version (thick line), the detected multivoicing function and its smoothed version (thick line).

Table 1 :

 1 duration (s) and % cumulative duration of the V and U states for the 10 speakers of the Keele database. Global means are 50.3% V and 49.7% U.

Table 3 :

 3 HSC

	monopitch error rates (in %) on the
	Keele single-speaker data: V>U (undervoicing); U>V
	(overvoicing); total Voicing Error Rate (VER); pitch
	Gross Error Rate (GER) at 20% max deviation.
	Error rates %	V>U	U>V	VER GER
	ref= Keele data	2.4	2.3	4.7	0.9

Table 4 :

 4 Voicing measurements of table 3 expressed as Recall/Precision/F-measure

	Voicing detection %	Recall Precision F-measure
	ref= Keele data	95.3	95.5	95.4

Table 5 :

 5 Multivoicing results of HSC Multipitch on a mixture of 2 different permutations of the 10 Keele sequences.

	Voicing detection %	Recall Precision F-measure
	cumulative Voiced hypotheses	94.3	83.1	88.3
	cumulative 0V frames	97.1	98.0	97.6
	cumulative 1V frames	62.0	84.2	71.4
	cumulative 2V frames	79.1	52.0	62.8


Delphine Charlet
email: delphine.charlet@orange.com

Claude Barras
email: barras@limsi.fr

Jean-Sylvain Liénard
email: jslienard@gmail.com

IMPACT OF OVERLAPPING SPEECH DETECTION ON SPEAKER DIARIZATION FOR BROADCAST NEWS AND DEBATES

Keywords:

The overlapping speech detection systems developped by Orange and LIMSI for the ETAPE evaluation campaign on French broadcast news and debates are described. Using either cepstral features or a multi-pitch analysis, a F1-measure for overlapping speech detection up to 59.2% is reported on the TV data of the ETAPE evaluation set, where 6.7% of the speech was measured as overlapping, ranging from 1.2% in the news to 10.7% in the debates. Overlapping speech segments were excluded during the speaker diarization stage, and these segments were further labelled with the two nearest speaker labels, taking into account the temporal distance. We describe the effects of this strategy for various overlapping speech systems and we show that it improves the diarization error rate in all situations and up to 26.1% relative in our best configuration.

INTRODUCTION

Automatic speech recognition and speaker diarization on broadcast data long focused on contents where speech overlaps were rare, or excluded speech overlap segments from their evaluation. On the other hand, studies on more spontaneous speech from multi-party conversations, especially telephone conversations and meetings report that 6 to 14% of words are overlapped [START_REF] Shriberg | Observations on Overlap: Findings and Implications for Automatic Processing of Multi-Party Conversation[END_REF][START_REF] Cetin | Errors in meetings: Effects before, during, and after the overlap[END_REF], and overlapping speech was identified as a major cause of error for speaker diarization [START_REF] Huijbregts | The blame game: Performance analysis of speaker diarization system components[END_REF]. On broadcast data, the assumption that speech overlap is negligible is no longer valid, when it comes to deal with political interviews [START_REF] Adda | Speech Overlap and Interplay with Disfluencies in Political Interviews[END_REF] or talk-shows [START_REF] Gravier | The ETAPE corpus for the evaluation of speech-based TV content processing in the french language[END_REF].

To the best of our knowledge, all the published studies about overlapped speech in speaker diarization focused on meeting data or on telephone conversations. This work focuses on overlapping speech detection in French TV broadcasts and its impact on speaker diarization in the context of the ETAPE evaluation campaign 1 . The next sections describes the ETAPE evaluation and its data, the proposed overlapping speech detection approaches and their performance, their integration for speaker diarization and concludes with a comparison with existing work.

Experiments were performed in the context of the French ETAPE evaluation campaign. LIMSI work was partly realized as part of the Quaero Program and the QCompere project, respectively funded by OSEO (French State agency for innovation) and ANR (French national research agency). Orange work was partly realized as part of the PERCOL project funded by ANR. 1 http://www.afcp-parole.org/etape.html

ETAPE CAMPAIGN AND DATA

The ETAPE evaluation campaign took place in Spring 2012 [START_REF] Gravier | The ETAPE corpus for the evaluation of speech-based TV content processing in the french language[END_REF]. It evaluated segmentation, transcription and information extraction in the audio channel of French TV and radio broadcasts. This work addresses the segmentation tasks, i.e. the multiple speaker detection and speaker turn segmentation. The multiple speaker detection task (SES-2) aims at detecting the start and end times of segments containing speech from more than one speaker. Manual annotation of overlaps along with the reference transcription was provided by ELDA, and the temporal extent of the overlaps was refined through a forced alignment between the reference and an automatic transcription 2 . Due to the exploratory nature of the task, several metrics were proposed but no official metric was chosen. We report recall and precision of multiple speech detection expressed in duration, with non-speech regions excluded from the scoring, and the resulting F1-measure. Moreover, we restrict to the TV data since the forced alignment was not performed for the radio training data subset.

Performance in the speaker diarization task (SRL) is measured by the Diarization Error Rate (DER) as the sum of Miss Detection Rate, False Alarm Rate and Speaker Error Rate, where Speaker Error Rate is obtained after optimal mapping between automatic clusters and reference speakers 3 . Usually, evaluation of speaker diarization is performed after excluding regions of overlapped speech. Here, we evaluate the system including overlapped speech regions. If an overlapped speech region is assigned to only one speaker, this region is considered as Missed speech for the second speaker. If an nonoverlapped speech region is falsely detected as overlapped speech region and assigned to 2 speakers, this region will be counted as false alarm speech for the second speaker.

The ETAPE TV subset consists in 29 hours of data (18 hours training, 5.5 hours development and 5.5 hours test) from three French TV channels (LCP, BFM and TV8) with news (BFM Story, LCP Top Questions), debates (LCP Pile et Face, LCP C ¸a vous regarde, LCP Entre les lignes) and reportages from a local TV with unprofesionnal speakers (TV8 La place du village). For the whole dataset, the ratio of overlaps amounts to 5.9%, ranging from 1.8% in the news to 3% in the reportage and 8.7% in the debates (cf. Table 1). The mean duration of overlap segments in the training set is 1.07 sec. and their median duration is 0.72 sec (see Figure 1 for the normalized histogram of overlap durations). The cumulated duration of overlaps on the same Figure shows that overlaps longer than 1 sec. cover about 70% of the cumulated duration of overlapping speech, even if they represent only 35% of the occurences. The corpus is biased towards male speakers, with a lot of journalists and politicians: there are 160 male vs. 43 female speakers in the training set, accounting for about 90% of the total duration, and overlaps almost exclusively involve male speakers.

OVERLAPPING SPEECH DETECTION

This section describes the overlapping speech detection systems developed by LIMSI and Orange for the ETAPE evaluation campaign and their performance in the SES2 task.

LIMSI system combining cepstral and multi-pich features

For the LIMSI system, three GMM {λi}i=0...2 with 256 Gaussians, respectively for non-speech, non-overlapping speech and overlapping speech, are trained using forced alignement between automatic and reference transcriptions of the ETAPE training data provided by LNE. Cepstral features (12 PLP and log-energy along with their firstand second-order derivatives) are computed over a windows of 30 ms with a 10 ms step. The frame-level likelihood-ratio (LLR) between the multiple-speaker model f (x|λ2) and the two other hypothesis is smoothed over a Hamming window H and the resulting value lt compared to a decision threshold optimized on the development set:

lt = d/2-1 j=-d/2 H(j). log f (xt+j|λ2) f (xt+j|λ0) + f (xt+j|λ1)
In our developments, this approach was found to perform better than a Viterbi decoding with the {λi} models as proposed in [START_REF] Boakye | Overlapped speech detection for improved speaker diarization in multiparty meetings[END_REF] (either with a minimal duration or a transition penalty).

Given the harmonic nature of voiced speech, it can be expected that approaches and features which are relevant for speech separation and multi-pitch detection [START_REF] Cheveigné | Multiple F0 estimation[END_REF] are also of interest for overlapping speech detection. We performed our experiments with the PSH algorithm designed by Liénard et al. [START_REF] Liénard | Using sets of combs to control pitch estimation errors[END_REF]. It is based on a frequential approach and uses several spectral combs. Spectral combs are used as pattern matching tools for detecting the harmonic structures of voiced segments of speech. The dot product between a comb and the amplitude spectrum produces a pitch function exhibiting local maxima at frequencies where F0 is most probable. But in the multipitch cases, numerous spurious peaks appear. To strongly attenuate them, two families of combs are used: negative teeth comb and missing teeth comb which treat selectively harmonics errors and subharmonics errors. The algorithm performs a frame-to-frame analysis over a 50 ms window with 10 ms step without any post-processing, thus remaining computationnaly light compared to other multi-pitch estimators. Let pt ∈ {0, 1, 2} be the number of hypothesized F0 output by the multi-pitch detector for the frame xt. This value is then smoothed through a Hamming window H of size d, resulting in the frame-level harmonic feature ht:

ht = d/2-1 j=-d/2 H(j).pt+j
Finally, a frame-level linear combination of the lt and ht values was submitted as LIMSI primary system to the ETAPE SES2 task, with combination weights and decision thresholds optimized on the development set.

Cepstral Features by Orange

Three GMMs {λi}i=0...2 with 256 Gaussians, resp. for male non-overlapped speech, female non-overlapped speech and overlapped speech, were trained using forced-alignement between automatic and reference transcriptions of the ETAPE training data provided by LNE. Cepstral features (12 MFCC and log-energy with their first and second order derivatives) are computed over a windows of 32 ms with a 16 ms frame rate. A 2-class HMM (overlapped/non-overlapped (with male and female GMM models) is then built. Viterbi decoding, only applied after a first external speech/non-speech segmentation step, is associated with a minimal state duration (2s in non-overlapped speech and 0.5s in overlapped speech). Finally, a post-processing filtering discards all the detected overlapped-speech segments whose length is less than 1s. This was the system integrated to in the diarization process submitted to the ETAPE evaluation campaign.

Since the campaign, the post-processing filtering based on the length of the segments has been replaced with a filtering based on log-likelihood ratio value at the segment level. For a detected overlapped speech segment X of N frames (x1, .., xN ), whith the same definition as above, the confidence measure is:

S(X) = 1 log(N ) N t=1 log f (xt|λ2) f (xt|λ1)
The confidence measure S(X) is compared to a threshold to validate the detection or not. The length normalisation by log(N ) instead of the usual N is meant to favor the long detections. Indeed, we have observed that the long detections of overlapped speech were more likely to be correct than the short ones.

Development and evaluation results

For LIMSI system, the performance of the multi-pitch system was significantly worse than the cepstral system on the development set (F1 measure of 45.3% vs. 54.5%), however it only relies on a framelevel ternary feature compared to the 13 real-valued features used in the cepstral system. The optimal size for the Hamming smoothing window was found to be slightly different (2.5 sec. for lt vs. 2 sec. for ht). The combination of both systems further improved the F1 performance to 55.8% on the development set and was chosen for the LIMSI primary submission to the ETAPE SES2 task.

Table 2 presents the results on the evaluation TV subset. The segment length filtering dramatically improves F1 value of Orange cepstral system from 43.3% to 55.2% mainly due to an increase in precision, and the alternative LLR filtering further improves it to 59.8%. LIMSI system presents a F1-measure slightly lower at 58.2%

DIARIZATION EXPERIMENTS

Principle

In previous studies about speaker diarization in meetings [START_REF] Huijbregts | The blame game: Performance analysis of speaker diarization system components[END_REF], detailed error analysis showed that overlapped speech was a major cause of error, and [START_REF] Otterson | Efficient use of overlap information in speaker diarization[END_REF] proposed two methods for handling overlap in speaker diarization:

• overlap exclusion: exclude overlapped speech segments from the diarization process;

• overlap labeling: label the overlapped speech segments with the speaker labels of the 2 nearest speakers (in time).

On oracle experiments, with perfect overlap detection, the overlap labeling strategies with the 2 nearest speakers proved to be very effective, and closed to a perfect (oracle) labeling strategies. This overlap handling scheme has been commonly adopted in the studies about the impact of overlapped speech in speaker diarization (e.g. [START_REF] Yella | Speaker diarization of overlapping speech based on silence distribution in meetings recordings[END_REF]). Here, we perform overlap exclusion and propose a slightly modified labeling strategy, to cope with the errors of overlap detection:

• overlap labeling: always label the segment with the nearest speaker (in time), and label with the second nearest speaker only if its temporal distance to the segment is below a given threshold Ts. This variant is meant to cope with error of false detection of overlapped speech in the middle of a speaker turn.

The diarization system used in these experiments is the one developped by Orange based on the principles of [START_REF] Barras | Multi-stage speaker diarization of broadcast news[END_REF]: the first step consists in building an agglomerative clustering of speech segments based on Bayesian Information Criterion (where each cluster is modeled by a single Gaussian with a full covariance matrix). When each cluster contains enough data to model the voice more precisely, the clusters are modeled with Gaussians mixture, and the agglomerative clustering is pursued with a distance between clusters based on a cross-likelihood criterion. At each iteration of the clustering based on cross-likelihood, a Viterbi decoding is also performed to resegment the speech data into speaker turns, given the new clusters.

Evaluations

In the Figure 3, we plot the diarization error rate (DER) obtained with different overlapping speech detection systems, when including overlapping speech in the evaluation, and as a function of the threshold Ts. For comparison, the baseline system processes the documents without detection of overlapped speech. The other systems apply the overlap exclusion step, and the proposed new labeling strategy, using one of the proposed overlapping speech detection systems O1, O2 or L1. The system with O1 was the one submitted by Orange to the SRL task of the ETAPE challenge. Finally, oracle experiments (i.e. automatic speaker diarization with a perfect overlapping speech detection) are also reported. Ts = 0 corresponds to the performances obtained when a second speaker label is never attributed to the detected overlapping speech segments. On the contrary, for Ts = ∞ a second speaker label is always attributed to these segments.

First, we can observe that, whichever overlapping speech detection system is used and for any Ts, it always outperforms the baseline system without overlapping speech detection. The performances obtained with Ts = 0 (only one speaker is assigned to the overlapping speech segment) are always far better than the baseline system. This improvement is due to the purification of the clusters, which are only fed with detected non-overlapping speech. It can be seen that the best automatic system reaches the same performance as the oracle system, when only purification is performed. Thus, even though the performance of the overlapping speech detector is average (F1=59.8%), it is good enough for the exclusion step, to get all the benefits of the purified clusters. Then, the impact of the overlapping speech detections on the labeling strategies can be seen on the rest of the curves. For the system with high recall and low precision on overlapped speech detection, it is better not to always assign a second speaker: indeed, the increase of errors due to false alarm of overlapped speech is bigger than the reduction of errors due to the attribution of segment of overlapped speech. For the other systems with higher precision on overlapped speech detection, always attributing a second speaker appears to be a valid strategy. Table 3 shows the detailed components of the DER for the "always 2-nearest speakers" strategy, with the different overlapping speech detectors. The speaker error obtained with O2 is the same as the one obtained with the oracle detector, and the main differences lies in the false alarm rate on speaker, which is bigger because of the false alarm of overlapping speech detection. On the other hand, the LIMSI approach has a higher precision than the other ones and leads to a smaller fale alarm rate, but does not fully benefits from the clusters purification, thus leading to a higher speaker error rate.

The DER per type of shows (news, debates or reportage) are presented in Table 4, for the baseline system, and for the best system O2 (along with the relative improvement rate), and the overlapped speech ratio per type of shows. The more overlapping speech there is in the data, the better the improvement due to overlapping speech handling is. For news shows with very little amount of overlapped speech, the imprecision due to the overlapped speech detector degrades the overall results, while for debates shows, the decrease of DER reaches 33.2%. 4. Relative DER improvement per type of shows for the best diarization system integrating O2 vs. the baseline system.

5. RELATION TO PRIOR WORK Many studies have been published on overlapping speech detection for speaker diarization of meetings. Some of them perform source separation or source localization relying on multiple channel recordings [START_REF] Pfau | Multispeaker Speech Activity Detection for the ICSI Meeting Recorder[END_REF][START_REF] Wrigley | Speech and crosstalk detection in multi-channel audio[END_REF][START_REF] Laskowski | Unsupervised learning of overlapped speech model parameters for multichannel speech activity detection in meetings[END_REF][START_REF] Pardo | Speaker diarization for multiple distant microphone meetings: Mixing acoustic features and inter-channel time differences[END_REF] which are not available for broadcast data. [START_REF] Boakye | Two'sa crowd: Improving speaker diarization by automatically identifying and excluding overlapped speech[END_REF] tested various features for overlapping speech detection in a HMMbased segmenter. On far-field recordings of the AMI meeting corpus with 18% of overlapped speech, they get 38% F-score in overlaps detection. Features such as silence distribution [START_REF] Yella | Speaker diarization of overlapping speech based on silence distribution in meetings recordings[END_REF] or prosodic features [START_REF] Zelenak | The detection of overlapping speech with prosodic features for speaker diarization[END_REF] also gives a F-score on overlap detection around 40% in meetings. [START_REF] Vipperla | Speech overlap detection and attribution using convolutive non-negative sparse coding[END_REF] proposed a convolutive non-negative sparse coding approach to speech overlap detection ; they get a 16.1% recall and 28.6% precision of overlapping speech detection on NIST RT meetings. On telephone converstations, [START_REF] Ben-Harush | Frame level entropy based overlapped speech detection as a pre-processing stage for speaker diarization[END_REF] used entropy features estimated in the time domain for detecting overlapping speech, but their approach is only suitable in a two-speakers situation. Finally, relevant research for overlapping speech detection is also developed in the context of single-channel speech separation [START_REF] Mowlaee | A MAP criterion for detecting the number of speakers at frame level in model-based single-channel speech separation[END_REF][START_REF] Saeidi | Improving monaural speaker identification by double-talk detection[END_REF].

When it comes to the integration of overlapping speech detection in speaker diarization system, the classical approach consists in applying exclusion and labeling, when labeling is either performed with speaker posterior probabilities [START_REF] Boakye | Overlapped speech detection for improved speaker diarization in multiparty meetings[END_REF][START_REF] Boakye | Two'sa crowd: Improving speaker diarization by automatically identifying and excluding overlapped speech[END_REF][START_REF] Boakye | Improved overlapped speech handling for speaker diarization[END_REF] or 2-nearest speaker labeling [START_REF] Otterson | Efficient use of overlap information in speaker diarization[END_REF]. Relative improvement of DER such as 4.2% [START_REF] Huijbregts | Speech overlap detection in a two-pass speaker diarization system[END_REF], 6.5% [START_REF] Vipperla | Speech overlap detection and attribution using convolutive non-negative sparse coding[END_REF], 7.2% [START_REF] Zelenak | The detection of overlapping speech with prosodic features for speaker diarization[END_REF], 12.4% [START_REF] Boakye | Improved overlapped speech handling for speaker diarization[END_REF] and 18.7% [START_REF] Yella | Speaker diarization of overlapping speech based on silence distribution in meetings recordings[END_REF] have been reported for meetings data, when the major part of the improvement is due to the exclusion step.

Thus, our work on broadcast data give consistent results with prior studies on meetings data. But the results obtained on broadcast are significantly better than those reported on meetings, either for F-mesure on overlap detection or for relative DER improvement.

CONCLUSIONS

In this paper, we have studied the impact of overlapping speech detection in speaker diarization for broadcast news and debates. Whereas many studies have been done in the context of meetings diarization, this is the first time that this question is treated and evaluated in the broadcast context. The basic strategy of overlap handling proposed in [START_REF] Otterson | Efficient use of overlap information in speaker diarization[END_REF] has been applied, with different overlapped speech detectors. The influence of the second-speaker labeling step has been studied with a modified labeling strategy. The experiments were conducted on a corpus of 5.5 hours of 7 different TV shows with a varying level of overlapped speech, from the ETAPE evaluation campaign. Two overlapping speech detection systems were developed by Orange and LIMSI, relying on standard cepstral features or on a multi-pitch analysis. The best configuration presents a F1-measure of about 60%. The diarization experiments show that this level of performance is sufficient to provide all the benefits of the exclusion step due to the purification of the clusters, and enable also improvement at the labeling step. The DER decreases from 23.8% with no overlap handling to 17.6% with automatic overlap detection.

Fig. 1 .

 1 Fig. 1. Distribution of overlap durations in the training set

Fig. 2 .

 2 Fig.2. Performance of overlap detection as a function of minimal segment length in the reference for O2 system.

Fig. 3 .

 3 Fig. 3. Diarization error rate (DER) as a function of the threshold Ts controlling the attribution of the second speaker.

Table 1 .

 1 Duration (in minutes) and ratio of overlapping speech relative to total speech in train, development and evaluation subsets, depending on the genre of the show

	Show type		Train		Development	Evaluation	All
	News	5.6 / 297.9 (1.9 %)		1.1 / 67.6 (1.7 %)	0.8 / 66.4 (1.2 %)	7.6 / 431.9 (1.8 %)
	Debates	41.5 / 486.3 (8.5 %)	9.6 / 128.7 (7.5 %) 13.6 / 130.1 (10.4 %)	64.7 / 745.1 (8.7 %)
	Reportage		-		0.7 / 37.6 (1.8 %)	1.7 / 41.9 (4.0 %)	2.4 / 79.4 (3.0 %)
	Total	47.1 / 784.2 (6.0 %) 11.4 / 233.9 (4.9 %)	16.0 / 238.3 (6.7 %) 74.6 / 1256.4 (5.9 %)
	with a lower recall (52.7% vs. 64.3%) but a better precision than
	the best Orange system (64.9% vs. 56.0%). As could be expected,
	longer overlaps are easier to detect, and this behaviour is illustrated
	for the best performing O2 system on Figure 2 where the overlap
	segments in the reference which are shorter than a minimal duration
	are ignored for the scoring.
	System		P	R	F1
	Orange cepstral		29.9 78.5 43.3
	Orange cepstral+length filtering (O1) 45.5 70.2 55.2
	Orange cepstral+LLR filtering (O2)	56.0 64.3 59.8
	LIMSI system (L1)		64.9 52.7 58.2

Table 2 .

 2 Precision

(P), recall (R) and F1-measure of overlap detection on the ETAPE TV evaluation subset for the different systems (official submissions to the ETAPE evaluation in bold).

Table 3 .

 3 Decomposition of the Diarization Error Rate into false alarm (F.A), missed detection (M.D.) and speaker error rate (SER).

	Overlaps detector	M.D. F.A. SER DER
	None (baseline)	6.1	0.7	17.0	23.8
	O1 (best recall)	1.6	4.9	14.6	21.2
	L1 (best precision)	3.1	2.0	14.2	19.3
	O2 (best F1)	2.0	3.0	12.6	17.6
	Oracle	2.1	1.3	12.6	16.0

Thanks to Olivier Galibert from LNE for providing these alignments to the ETAPE participants.

DER reported in this paper were computed using the conventionnal NIST evaluation tools with the default value of collar of 250ms


Philippe Martin
email: philippe.martin@univ-reunion.fr

Jérémy Benard
email: jeremy.benard@logicells.com

An Ontology for Specifying and Parsing Knowledge Representations Structures and Notations

Keywords: Language ontology, Meta-modelling, Syntactic translation, Knowledge representation languages

In its introduction, this article gives a short state of the art about ontologies of knowledge representation languages (KRLs) and the problems caused by i) the lack of relations between these ontologies, and ii) the lack of ontologies about notations (concrete syntaxes). For programmers, these are the difficulties of importing, exporting or translating between KRLs; for end-users, the difficulties of adapting, extending or mixing notations. To show how these problems can be solved, this article first shows how concepts of the main KRL standards can be aligned and organized. Then, it shows how this KRL model ontology can be reused and completed by a notation ontology. Based on these two ontologies, KRLs models and notations -and thereby parsing and generation -can be specified in a concise way that even KRL end-users can adapt. The article gives representative examples. For these ontologies or specifications, a concise KRL notation is introduced and used. However, the presented approach is independent of any notation and model that has at least OWL-2 expressiveness. Thus, the results can easily be replicated. A Web address for the full specification of the two ontologies, and for a knowledge server to test or use them, is also given.

INTRODUCTION

Various language models are used for knowledge representation, retrieval and exploitation. For each model (abstract syntax) there are also many possible notations (concrete syntaxes). Creating a parser or an export procedure for each knowledge representation language (KRL; one model and/or one notation) is time-consuming. Specifying the automatic translation of knowledge (representations) from one KRL to another is difficult, especially without some shared ontology of these KRLs, hence without formal semantic relations between their components. Learning and understanding a KRL is also difficult for a person. These are therefore also difficulties for knowledge sharing. For a knowledge provider, not being able to adapt a KRL notation, is also limiting.

There have been many works for partially addressing these problems, except for the last one which requires the use of a KRL notation ontology to enable any significant adaptation.

An early major work was KIF (Knowledge Interchange Format) [START_REF] Genesereth | Knowledge Interchange Format, Version 3.0, Reference Manual[END_REF], a 1st-order logic based KRL -with a 2nd-order notationto which most KRLs could be translated to and formalized with. Many were. To ease this, the Ontolingua "ontology server or shared repository" [START_REF] Farquhar | The Ontolingua Server: a tool for collaborative ontology construction[END_REF] provided a well formalized KRL model ontology. E.g., it included a formalization of frame-based language concepts in KIF (concepts similar to those of OWL).

Later, with the popularization of MOF (the Meta-Object Facility of the OMG: Object Management Group), XML and then RDF, many language models or ontologies were created in these three languages. These were often simple lists of KRL components and their structural relations. Indeed, MOF, XML and RDF do not permit to fully define KRL components and hence relate all of them as in Ontolingua. They still permit to declare and use a set of KRL components that corresponds to a certain logics with well studied properties. Thus, the W3C provided the different language ontologies of the OWL family (OWL 2, 2009). With RIF-FLD, it also provided an expressive and extensible "Framework for Logic Dialects" (RIF-FLD, 2013). ANSI provided CL [START_REF] Logic | Information technology -Common Logic (CL): a framework for a family of logic-based languages[END_REF], a "framework for a family of logic-based languages" restricted to 1st-order logic. The OMG created a "Conceptual package" along with an ontology for the "Semantics of Business Vocabulary and Business Rules" [START_REF] Sbvr ; Zamazal | Semantics of Business Vocabulary and Business Rules (SBVR), Version 1.0[END_REF].

These standards (RDF+OWL+RIF-FLD, CL and MOF+SBVR) have similar or complementary components. They are declared in their respective XML schemas but, to our knowledge, no ontology semantically relates them nor to concepts in Ontolingua. However, within the scope of each of these standards, there are works on translating between models or ontologies. E.g., the W3C specifies ways to re-use RDF and OWL knowledge in RIF.

Model translation is often only a part of knowledge translation or (re-)presentation. Indeed, there are many existing or potential notations for KRL models and, so far, unlike some KRL models, no notation was represented by an ontology. Thus, no notation could be adapted or extended by their users, except very partially via a system of macros such as the one usable with the C programming language. A different parser and generator also had to be built for each notation, except for XML-based notations (e.g, RDF/XML: RDF in XML). The W3C proposes XSLT for specifying syntactic translations between XML based notations. It also proposes GRDDL for specifying where a software agent can find "algorithms (typically represented in XSLT)" to convert a structure or notation to RDF/XML. Conversely, there are some style-sheet based transformation languages and ontologies for specifying how RDF abstract structures can be presented, e.g., in a certain order, in bold, in a pop-up window, etc. : Xenon [START_REF] Quan | Xenon: An RDF Stylesheet Ontology[END_REF], Fresnel [START_REF] Pietriga | Fresnel: A Browser-Independent Presentation Vocabulary for RDF[END_REF], OWL-PL [START_REF] Brophy | OWL-PL: A Presentation Language for Displaying Semantic Data on the Web[END_REF] and SPARQL Template [START_REF] Corby | SPARQL Template: A Transformation Language for RDF[END_REF]. With these tools or the approach behind these tools, each modification to a notation requires a new template or style-sheet, and parsing is not addressed.

Supporting knowledge import/export/translation in a generic way requires specifying KRLs with respect to a KRL model ontology and a KRL notation ontology. This article presents such ontologies and gives examples of their use. To do so in a sufficiently concise and readable way, Section 2 first introduces FL, a concise and "visually structured" notation. Then, using FL, it shows how the main concepts of RIF-FLD, CL and SBVR can be related, defined and generalized to create the above cited two ontologies. This work required many readings of the specifications and grammars of RIF-FLD since they leave their underlying ontology implicit. Section 3 shows how the models and grammar of KRLs -and thereby their parsing, presentation and translation -can be specified based on these two ontologies. CSSlike presentation based on syntactic or semantic features could also be similarly specified but this is outside the scope of this article.

The generic approach we propose to solve the initially listed problems is independent of any notation and any model that has at least OWL 2 expressiveness. This article focuses on presenting the main ideas of the approach. The whole ontologies and model+notation specifications of various KRLs, as well as a Web server interface to test or use them, are available at http://www.webkb.org/KRLs/. This interface is similar to Google Translate except that the input and output languages are KRLs and, instead of KRL names, KRL specifications can also be given.

LANGUAGE ELEMENTS

To allow the display and understanding of its numerous required illustrations, this article needs a concise and intuitive notation for KRLs of OWL-like expressiveness. Unfortunately, graphical notations are not concise enough and common notations such as those of the W3C are not sufficiently concise and "structured" enough. Here, "structured" means that all direct or indirect relations from an object can be (re-)presented into a unique tree-like statement so that the various inter-relations can readily be seen. Table 1 illustrates this by representing the same statement in five notations: FL then UML, Turtle (or Notation3), OWL Manchester notation and OWL Functional-style. The way to read the content for FL is explained and given in italics within a paragraph following the table.

The last notation is "positional relation" based. The first four are graph-based notations: they are composed of concept nodes and relation nodes. These textual graph-based notations are frame-based. A frame is a statement composed of a first "object" (alias "node": individual or type, quantified or not) and several links associated to it (links from/to other objects). In this article, "link" refers to an instance of a "binary relation type". In OWL, such a type is instance of "owl:Property" (in FL: owl#Property). What is not an individual is a type: relation type or concept type (an instance of owl#Class in OWL).

In this article, the default namespace is for the types we introduce. The names of a concept type or individual that we introduce is a nominal expression beginning by an uppercase letter. The name of a relation type we introduce begins by "r_" (or "rc_" if this is a type of link with destination a concrete term). Thus, names not following these conventions and not prefixed by a namespace are KRL keywords. //Notes. ">" is an abbreviation for the "subtype" link (as in // some other notations). "<" is its inverse. "exclusion{...}" // specifies a union of disjoint types. If "T = exclusion{...}" // this is one subtype partition of T. If "T > exclusion{...}" // this is not a partition (or it is an "incomplete" one). A "," // separates 2 links of different types. For consecutive links // of the same type, this type needs not be repeated and the // destinations are only separated by one or several spaces.

Language_or_Language-element

Language

Language-element Within nominal expressions, '_' and '-" are used for separating words. When both are used, '-' connects words that are more closely associated.

Since nouns are used for the introduced types, the common convention for reading links in graph-based KRLs can be used: links of the form "X R: Y" can be read "X has for R Y". If "of" is used for reversing the direction of a link, the form "X R of: Y" can be read "X is the R of Y". In FL, if a link is not a subtype link (or another "link from a type"), the first node is quantified and its default quantifier is "any", the "forall" quantifier for definitions (in other words, the type in the first node is defined by this link). Links with the same first node may quantify it differently. Indeed, in FL, the quantifiers of the source node and destination node of each link may also be specified in its relation node or in its destination node. This permits FL to gather any number of statements into a unique visually connected graph. However, in this article, the quantifier for the first node is always "any" and left implicit. A destination node can also be source of links if they are encapsulated within parenthesis. Thus, given all this and the notes at the end of the FL content in Table 1, its first six lines can be read: "The type Language_or_Language-element is equivalent to its subtype partition composed of Language_element and Language, and any instance of Language has for (r_)part at least 1 instance of Language_element. This last type has for subtypes (at least) KRL and Grammar". The other tables of this article can now be read (any new keyword will be explained, most often via a comment next to it). In these tables, bold characters are only for highlighting important types and for readability purposes.

Table 2 shows how types for KRL models and notations can be organized and inter-related. E.g., RIF-FLD includes RIF-BLD, both are part of the RIF family of models, and both have a Presentation Syntax ("PS") and an XML linearization.

Table 3 relates Language_element and some of its direct subtypes to important top-level types, thus adding precisions to these subtypes. Such a specification is missing in RIF-FLD but is well detailed in SBVR. This is why Table 3 includes many top-level SBVR types, although indirectly: the types with names in italics are still types that we introduce but they have the same names as types in SBVR and are equal to them or slight generalizations of them. This approach is for readability reasons and flexibility: if the SBVR authors disagree with our interpretation of their types, only some links to SBVR types will have to be changed, not our ontology. As illustrated by Table 3, to complement and organize types from other ontologies, ours had to include many new types. In RIF-FLD, depending on the context, the word "term" has different meanings. In our ontologies, Gterm generalizes all these meanings of "term": it is identical to Language_element and sbvr#Expression. In RIF-FLD, an "individual term" is an abstract term that is not a Phrase (see Table 3), although it may refer to one. Individual_gTerm -or, simply "Iterm"generalizes this notion to concrete terms too. This distinction was very useful to organize types of language elements, especially those from the implicit ontology of RIF-FLD (this framework uses different vocabulary lists, including one for signatures; in our ontology, all these terms are interrelated). In this context, "individual" does not refer to "something that is not a type". Since an Iterm may refer to a Phrase, an Iterm identifier may be a Phrase identifier. Thus, Table 3 uses the construct "near_exclusion" instead of "exclusion".

(Verb_concept = Fact_type) ) ) (Description_container > (File > RDF_file)) (Description_instrument > (Language_or_Language-element = exclusion { (Language > KRL Grammar, r_part: 1..* Language_element ) Language_element //see below } ) ) ) } ) }; Language_element = Gterm Expression, r_representation of: 1 Meaning, > near_exclusion //String is both abstract and concrete { (Representation > Statement, rc_type: Concrete_term ) (Concrete_term > (Expression > Text), > (Concrete_iTerm < Iterm) ) //see Table 7 }
near_exclusion //a reference to a phrase is an Iterm { (Phrase > Statement Definition Frame) //Tables 56(Individual_gTerm = Iterm, //see This construct has no formal meaning (it does not set exclusion links). It is only useful for readability purposes. Table 3 also uses it to group and distinguish types for abstract and concrete terms. Indeed, a (character) string may be seen by some persons as being both abstract and concrete. Our ontology must be compatible with such visions when they come at no cost.

RIF-FLD distinguishes three types of generic structures for a Gterm that is a function or a phrase. We dropped their RIF-related restrictions and named them Positional_gTerm, Gterm_with_named_arguments and Frame. Table 1 gave examples for positional and frame terms. A term with named arguments is similar to a frame except that, as in object-oriented languages, local attribute names are used instead of link types (types are global). It could be argued that a same term could be presented in any of these three forms and hence that these three distinctions should rather be syntactic. However, the authors of RIF-FLD have not formalized the equivalence/correspondence between i) "classes and properties" ("interpreted as sets and binary relations") and ii) "unary and binary predicates", in order to have a "uniform syntax for the RIF component of both RIF-OWL 2 DL and RIF-RDF/OWL 2 Full combinations" (RIF-FLD-OWL, 2013). According to this vision, each person re-using ontologies must decide if, for its applications, stating such an equivalence is interesting or not. RIF rules or a macro language such as OPPL can certainly be used for such structural translations (Šváb-Zamazal et al., 2012). However, to avoid imposing this exercise to most users of our KRL model ontology, and to avoid limiting its use for specifying KRLs, it formalizes relations between a frame and a Conjunction_of_links_from_a_same_source (this is done in the last 15 lines of Table 6 plus the 3 lines related to Half-link in Table 7; reminder: a link is -or can also be seen as -a binary relation).

We found that a small number of link types are sufficient for defining a structure for abstract terms and specifying their related concrete terms. Table 4 lists and explains the main link types. They can be seen as a representation and extension of the signature system of RIF-FLD. The ideas are that 1) every composite term can be decomposed into a (possibly implicit) operator (e.g., a predicate, a quantifier, a connective, a collection type) and a list of parameters (alias, "parts"), and 2) many non-binary relations can be specified as links to a collection of terms. Table 5 and the subsequent tables use the link types of Table 4 directly or via functions which are shortcuts for specifying such links. This is highlighted via bold characters in those tables. The end of Table 4 specified one of these functions. In the tables 5 to 7, which illustrate the organization of subtypes of Phrase and Iterm, this function is used to define certain abstract terms as links and hence enable to store them or present them as such when necessary. Some of such links are used for both abstract and concrete terms. E.g., rc_operator_name is often also associated to an abstract term for specifying a default name for its operator. If no such link is specified or if "" is given as destination, the operator type name (without its namespace identifier) is used as default operator name. :<= [any ^(Thing r_part of: ?e) r_member of: ?list];

/* Notes: in FL, ":=' permits to give a full definition, ":=>" gives only "necessary conditions", ":<=" gives only "sufficient conditions", "^(" and ")" delimit a lambda-abstraction (a construct defining and returning a type; in OWL related KRLs, owl#Restriction can be used), "_(" and ")" delimit the parameters of a function call, "_[" and "]" delimit the parameters of a definition, ".[" and "]" delimit the elements of a list, ".{" and "}" delimit the elements of a set. ) ) (Non-modularizing_phrase //this may include non-monotonic phrases: assertions, queries, removals > (Formula > Positional_formula Formula_with_named_arguments Phrase_of_a_grammar cl#Sentence, = exclusion //the 3 following distinctions come from KIF { (Definition = exclusion { Non_conservative_definition Conservative_definition } ) (Sentence //fact in a world: formula assigned a truth-value in an interpretation > Belief //the fact that someone believes in a certain thing Axiom ) //sentence assumed to be true, from/by which others are derived (Inference_rule> Production_rule) //like an implication but the conclusion is "true" only if/when the rule is fired } near_exclusion { Composite_formula Atomic_formula_or_reference_to_formula } ) ), //see Table 7 Termula_phrase ) //a termula is a RIF function/atomic_formula parameter; its subtypes are not listed in this article }; //with the next subtype of r_part, the source ?x has some parts of type ?pt but no other parts with type the genus of ?pt r_only_such_part_of_that_type _[?x ?pt] < r_part _(?x ?pt), //this definition requires that relations of type := [?x r_part: 1..* ?pt 0 ^(?t != ?pt, < (?gpt r_genus_supertype of: ?pt))]; // r_genus_supertype are set by definitions //Thanks to this link type, our general model for KRLs and the default presentation associated to its abstract terms, // KRLs can be defined in a very concise way. Below are examples for some abstract terms of some KRLs. // The next section gives examples for some concrete terms of some KRLs. For the Triplet_notation, nothing else is required.

RIF r_only_such_part_of_that_type: //any model of the RIF family has for part terms defined by the following lambdas: ^(Gterm_that_can_be_annotated_without_link > Phrase) ^(Grouped_phrases r_part: 0..* Document) ^(Quantification > Classic_quantification) ^(Frame > Minimal_frame) ^(Collection > List) ^(Delimited_string > Delimited_Unicode_string); //in RIF, the only "delimited strings" are "delimited Unicode strings" RIF-BLD r_only_such_part_of_that_type: ^(Rule_conclusion > rif-bld#Formula) //these are just two examples, ^(Rule_premise > Connective_phrase_on_atomic_formulas Conjunction_phrase); // RIF-BLD has other restrictions Triplet_notation = ^(KRL r_only_such_part_of_that_type: ^(Phrase > Link) ^(Individual_gTerm > Constant_or_variable) ); Table 6: Important types of formulas and connections between frames, links and positional formulas (note: names in italics come from RIF-FLD, names in bold italics are used in RIF-FLD signatures, bold is for highlighting). Composite_formula = f_relation_type_("",r_relation,.[1..* Formula]), // => r_part: 1..* Formula > exclusion { (Formula_connective r_operator_type: 1 connective_operator, > cl#Boolean_sentence, > exclusion { (Connective_phrase_with_1_argument = f_relation_type_("",r_unary_relation,.[1..* Formula]), > (Negating_formula=exclusion{(Symmetric_negating_formula = f_relation_type_("Not",r_not,.[1..* Formula])) (Negation-as-failure_formula = f_relation_type_("Naf",r_naf,.[1..* Formula])) })) (Connective_phrase_with_2_arguments = f_relation_type_("",r_binary_relation,.[1..* Formula]), > (Rule = f_relation_type_(":-",r_rule_implication,.[1..* Formula]), = exclusion{ (Inference_rule > Production_rule) (Logical_rule < Sentence, > Logical_implication) } exclusion{ (Implication_only > Production_rule) (Logical_equivalence r_operator: r_equivalence) }, r_part: 1 (Rule_premise < Formula) 1 (Rule_conclusion < Formula) ) ) (Variable-n-ary_connective_phrase = f_relation_type_("",r_variable-ary_relation,.[1..* Formula]), > exclusion { (Disjunction_phrase = f_relation_type_("Or",r_or,.[1..* Formula])) (Conjunction_phrase = f_relation_type_("And",r_and,.[1..* Formula]), > (Conjunction_of_links = f_relation_type_("And",r_and,Link), > Frame_as_conjunction_of_links_from_a_same_source ) ) } ) } ) (Quantification = f_quantification_type_("",Quantifier,.[1 Type],Constant-or-variable,Formula), > (Classic_quantification = f_quantification_type_("",Quantifier,.[],Variable,Formula) ) //no guard, no constant exclusion { (Universal_quantification = f_quantification_type_("Forall",q_forall,.[1 Type],Constant_or_variable,Formula), > (Classic_universal_quantification = f_quantification_type_("Forall",q_forall,.[],Variable,Formula) ) ) (Existential_quantification = f_quantification_type_("Exists",q_exists,.[1 Type],Constant_or_variable,Formula), > (Classic_existential_quantification = f_quantification_type_("Exists",q_exists,.[],Variable,Formula) ) ) } ) }; < f_link_type_("",r_binary_relation_type,Termula,Termula), r_part of: (1 Frame ?f r_frame_head: 1 Termula ?fh), r_link_source: ?fh ) (Link_as_frame_part r_part of: (1 Frame ?f r_frame_head: 1 Termula ?fh), r_operator: ?fh, r_link_source: ?fh, r_link_destination: 1 Termula ?ld, r_part: (1 Half_link r_link_source: ?fh, r_operator: ?fh, r_parts: ?ld ) ) ) ) } ) }; Table 7: Important types of "individual terms" (terms that are not phrases except for those referring to phrases) (note: names in italics come from RIF-FLD, names in bold italics are used in RIF-FLD signatures, bold is for highlighting). (Negative_integer < ^(f_string_type_("-","","") r_part: 1..* Digit)) ) ) ) );

Atomic_formula_or_reference_to_formula

PRESENTING AND PARSING

Table 8 lists major kinds of structured concrete terms and thus also the main presentation possibilities for structured abstract terms (see the 14 names in italics).

Based on the five main categories for these concrete terms (see the names in bold and not in italics), it is easy to find the five categories of abstract terms they correspond to, even though such links are not shown in Table 8. We found that each of these concrete term types can be defined with only a few types of links, those that begin by "rc_" and that were listed in Table 4. We defined some functions to provide shortcuts for setting those links when defining a particular concrete term, e.g., fc_prefix-fct-like_type.

In our ontologies, links from a type do not specify that the given destination is the only one possible (to do so in FL, "=>" must be used instead of '":" after the link type name; in OWL-based models, owl#allValuesFrom can be used). Thus, such links represent "default" relationships: if a link from a type T specializes a link from a supertype of T, it overrides this inherited link. This is also true when the link type is functional (i.e., can have only one destination) and its destination for T does not specialize the destination for a supertype of T. The links beginning by "rc_" looks functional but are not: in FL, multiple destinations can be stated to indicate different presentation possibilities. However, by convention, such links override inherited links of the same types. (Positional_cTerm //e.g., with operator "f" and parts/parameters A, B and C rc_operator-name: "", rc_operator_begin_mark: "", rc_operator_end_mark: "", //link types listed in Table 4 rc_parts_begin-mark: "(", rc_parts_separator: "", rc_parts_end-mark: ")", rc_infix-operator_position: 0, //when different from 0, this indicates the operator position within the parts > exclusion { (Fct-like-cTerm = exclusion { (Prefix_fct-like-cTerm rc_parts_begin-mark: "_(") //e.g.: f _(A B C) (Postfix_fct-like-cTerm rc_parts_begin-mark: "(_") } )

//e.g.: (_ A B C)f (List-like_fct_cTerm = exclusion { (List-like_prefix-fct_cTerm rc_parts_begin-mark: ".(") //e.g.: .(f A B C) (List-like_infix-fct_cTerm rc_parts_begin-mark: "(.", rc_operator_begin_mark: ".") //e.g.: (. A B .f C) (List-like_postfix-fct_cTerm rc_parts_begin-mark: "(..") } ) //e.g.: (.. A B C f) } ) (Frame_cTerm //e.g., for the example below, with operator the type "f" and with parts two half-links of type r1 and r2 rc_operator-name: "", rc_operator_begin_mark: "", rc_operator_end_mark: "", rc_parts_begin-mark: "{", rc_parts_separator: ",", rc_parts_end-mark: "}", //as in JSON-LD rc_parts: 1..* Half-link_cTerm, > exclusion { (Prefix_frame_cTerm rc_parts_begin-mark: "_{") //e.g.: f_{ r1: A, r2: B } (List-like_frame_cTerm rc_parts_begin-mark: "{.", > List-like_prefix-frame_cTerm //e.g.: {. f r1: A, r2: B} List-like_infix-frame_cTerm ) //e.g.: {. r_id: f, r1: A, r2: B} (Postfix_frame_cTerm rc_parts_begin-mark: "{_") //e.g.: {_ r1: A, r2: B } f Alternating-XML_cTerm //Frame in the Alternating-XML style where concept nodes alternate } )

// with link nodes, as in RDF/XML Cterm_with_named_arguments //quite rare in KRLs, hence not detailed in this article }; fc_prefix-fct-like_type _[?notationSet, ?operator_name, ?begin_mark, ?separator, ?end_mark] //call examples are in Table 9 := ^(Prefix_fct-like-cTerm r_direct-or-indirect_part of: ?notationSet, rc_operator-name: ?operator_name, rc_parts_begin_mark: ?begin_mark, rc_parts_separator: ?separator, rc_parts_end_mark: ?end_mark )

Phrase //any phrase has at least these presentations in these 2 kinds of notations (see Table 2), e.g., in RIF-PS and RIF-XML: rc_type: ^(fc_prefix-fct-like_type _(.{Function-like_based_notation},"","(","",")") rc_annotation-position: -1) ^(fc_alternating-XML_type_(.{XML_based_notation},"") rc_annotation-position: 0 ); List rc_type: fc_list_type _( .{Notation}, "[", "," ,"]" ); //by default, in any notation, a list has for representation a // comma separated list of element delimited by square brackets; note that fc_list_type has no argument for an operator name Table 8 shows how different kinds of "default presentations" can be represented in concise ways.

In a KRL that is perfectly regular with respect to a particular kind of abstract/concrete term -e.g, the concrete "operator based terms" (those that have an operator in our approach) -allows the terms of this kind to be (re-)presented in the same way. A perfectly regular KRL is then one which is perfectly regular for all the kinds of terms it allows. The "Triplet notation" is perfectly regular. To be so, a more expressive KRL would have to be fully based on an ontology and be Nth-order logic based. Since KIF re-uses the LISP notation, it is perfectly regular with respect to "operator based concrete terms" and "concrete terms for collections". Most KRLs have some ad hoc abstract and concrete terms. E.g., in RIF-XML the directives of a document are presented in different ways: some via links, some via XML attributes. In RIF-PS, they are presented as positional terms but not links. Thanks to the fact that our general model represents the directives both as parts and links (see Table 5), these RIF predefined directives can be represented within/via frames as well as via positional terms. The first part of Table 9 shows how ad hoc concrete terms of particular types of KRLs can be specified in a concise way. The approach used to do Table 9: Ways to specify concrete terms for particular kinds of terms in particular notations, via our ontology.

//Thanks to the default values in our specifications for abstract and concrete terms, only the following lines are needed for // defining the presentation in RIF-PS of the abstract terms shared by the KRLs of the RIF family. For instance, the order and // operator names of the directives of a document can be found in Table 5. Since these directives follow the default presentation // for phrases in RIF-PS, nothing needs to be specified about them here. The abstract term restrictions can be specified here (as // illustrated below for "Frame" or separately, as illustrated by the second part of Table 5. RIF r_only_such_part_of_that_type: //because of the default values, there is no need for more than the next lines ^(Phrase rc_type: fc_prefix-fct-like_type_(.{RIF-PS},"","(","",")") ) //by default, a phrase in RIF_PS follows this style ^(RIF_annotation rc_type: fc_list_type_(.{RIF-PS},"(*","","*)")) //this is overridden by some subtypes of Phrase, e.g., this one ^(Quantification_bound_list rc_type: fc_list_type_(.{RIF-PS},"","","")) ^(Rule rc_type: fc-like_infix-fct_type_(.{RIF-PS},":-","","","")) ^(Externally_defined_term rc_type: fc_prefix-fct-like_type_(.{RIF-PS},"External","(","",")")) ^(Equality_formula rc_type: fc_list-like_infix-fct_type_(.{RIF-PS},"=","","","")) ^(Subclass_formula rc_type: fc_list-like_infix-fct_type_(.{RIF-PS},"##","","","")) //e.g., "?t1 ## ?t2"; in FL: "?t1 < ?t2" ^(Class-membership_formula rc_type: fc_list-like_infix-fct_type_(.{RIF-PS},"#","","","")) ^(Frame > Minimal_frame, rc_type: fc_infix_list-like_frame_type_(.{RIF-PS},"","[","","]") ) //abstract+concrete specification ^(Half_link rc_type: fc_half-link_type_(.{RIF-PS},"","","->","","")) ^(Name-Termula_pair rc_type: fc_list_type_(.{RIF-PS},"","->","")) ^(Open_list rc_type: fc_prefix-fct-like_type_(.{RIF-PS},"List","(","|",")")) ^(Open-list_rest rc_type: fc_list_type_(.{RIF-PS},"","","","")) ^(Aggregate_function rc_type: fc_prefix-fct-like_type_(.{RIF-PS},"","{","|","}")) ^(Aggregate_function_bound_list rc_type: fc_fct-like_list_cTerm_(.{RIF-PS},"[","",]"")); RIF-FLD r_only_such_part_of_that_type: //only 1 example for RIF-XML: the concrete term for Document in RIF-FLD ^(Document rc_type: (1 fc_alternating-XML-cTerm_type_(.{RIF-XML},"Document") rc_annotation-position: 0, rc_XML-attribute_type: r_dialect xml#base xml#prefix, //the last two are predefined in XML rc_XML-link_types: .[rif#directive rif#payload] ) );

JSON-LD_model r_only_such_part_of_that_type: //the specifications of both the JSON-LD_model and the JSON-LD notation ^(Phrase rc_type: fc_list-like_infix-frame_type_(.{JSON-LD},"","{",",","}")) // except for the concrete terms ^(Half_link rc_type: fc_half-link_type_(.{JSON-LD},"",":","","")) ^(Module_header rc_type: fc_list-like_infix-frame_type_(.{JSON-LD},'"@context:"',"{",",","}")) ^(Module_body rc_type: fc_list_type_(.{JSON-LD},"","",","")) ^(Formula > ^(Minimal_frame r_operator: 1 Constant_gTerm)) //only 1 destination per link ^(Fterm_or_variable > Constant_or_set_or_closed_list) ^(Set rc_type: fc_list_type_(.{JSON-LD},"[",",","]")) //by default in JSON-LD (whereas in JSON, this would be for a list) ^(Closed_list > ^(Frame r_part: 1 .[r_container, Closed_list], //1st way to represent a list in JSON-LD rc_type: fc_half-link_type_(.{JSON-LD},"","@container",":","@list","") ) ^(Frame r_part: .[r_list, 1 Set], rc_type: fc_half-link_type_(.{JSON-LD},"","@list",":","","") ) ); //2nd way ^(Thing ?t rc_: (a Enclosed_list_cTerm ?c r_KRL-set: ^?notationSet)) //"^?" prefixes variables that are implicitly rc_parts: f_remove_empty_elements_in_list _( .[ (^?cb rc_begin_mark of: ?c), // universally quantified fc_r_parts_(?notationSet,(^?tp r_parts of: ?t),(^?cs rc_parts_separator of: ?c)) (^?cb rc_end_mark of: ?c) ] ); so for abstract terms (see the second part of Table 5) is here re-used. Thus, the abstract and concrete terms of a KRL -or a family of KRLs -can be specified at the same time. This enables organized specifications and thus eases the comparison of KRLs.

The second part of Table 9 shows how an ordered list of concrete terms can be specified for a type of abstract term, given a type of presentation and a list of notation types. Since the function fc_r_parts is recursive and, in turn, uses such specifications (links of type rc_parts or, for non-structured terms, links of type rc_), the specified ordered list only contains strings. Finally, given the value of rc_separator between tokens in the considered notation (i.e., the kinds of space characters separating them), the kinds of strings that can be associated to this collected list are specified. Thus, the whole specification is fully declarative. However, for concrete term generation purposes, choices have to be made, e.g., about space indentation. In our system, this is implemented via generation functions (also included in our ontologies) which recursively navigate the abstract and concrete specifications to find the most precise relevant specifications. Since our system rejects the entering of ambiguous knowledge (e.g., different concrete term specifications for a same type of abstract term and the same type of notation), finding the most precise relevant specifications was easy to implement.

Specifying parsing rules and generating them -for a given abstract term and grammar notation -can be represented using the same techniques. The first part of Table 10 shows the beginning of an ontology for grammars. The second part shows an example of grammar rule (and its connection to a grammar but this part actually needs not be generated). Once the grammar rules are generated -in a way similar to presentation generation -the generation of their presentation is then done exactly as for any other statement, according to the given grammar notation.

Our ontologies can be represented with OWL-2 based KRLs. E.g., r_parts links with "lists with cardinalities" (e.g., .[0..1 Y, 1..* Z]) as destinations can be replaced by lists without cardinalities (e.g., .[Y, Z]) as long as r_part links are also used for specifying the cardinalities (e.g., X r_part: 0..1 Y, 1..* Z ). Functions are not mandatory since their definitions can be expanded whenever they are called.

Replicating our work does not require details on the implementation of our system: our ontologies are the required declarative code. The used inference engine is irrelevant as long as it can handle the specifications. However, some readers might be interested to know that our translation server exploits the parser available at http://goldparser.org while its inference engine was implemented in Pascal Object (for portability purposes) and exploits tableaux decision procedures [START_REF] Horrocks | Optimising Tableaux Decision Procedures for Description Logics[END_REF]. This server and its inference engine have recently been designed by Logicells/GTH (http://www.mitechnologies.net/). This work on a generic approach for handling KRLs comes from the many problems encountered to handle various versions of FL and other KRLs in the knowledge sharing servers WebKB-1 (Martin and Eklund, 1999) and WebKB-2 (Martin, 2002[START_REF] Ph | Collaborative knowledge sharing and editing[END_REF].

Table 10: Important links from Grammar_element, followed by an example of grammar head rule. Non-lexical-grammar_rule = NLG_rule, //this is a beginning but the representation of the whole grammar is similar r_part: 1 NLG_rule_left-hand-side 1 NLG_expression 0..1 (Parsing_action_phrase < Phrase), rc_: (1 fc_list_type_(.{W3C-EBNF,XBNF,Grammar},"","","") //like fc_prefix-fct-like-cTerm_type but without operator rc_parts: .[NLG_rule_left-hand-side "::=" NLG_expression] ) //→"A::=B" ("Grammar "→default presentation) (1 fc_list_type_(.{ISO-EBNF},"","","") rc_parts: .[NLG_rule_left-hand-side "=" NLG_expression] ) //→ "A = B" in ISO-EBNF (1 fc_list_type_(.{Yacc, Bison},"","","") rc_parts: .

CONCLUSIONS

One contribution of this article is a generic model for structured abstract or concrete terms. It is simple: only a few types of links and a few distinctions (Tables 4 and8). This operator+parameters based model permits to define terms in a concise and flexible way, and thus also their presentation and parsing.

A second contribution is the design of a KRL model ontology by representing, aligning and extending various KRL models, and defining their elements via the above cited few links, as illustrated by Tables 3 and5-7. Thus, the merged models are also easier to re-use.

A third one is the design of a KRL notation ontology -to our knowledge, the first one -based on the above two cited contributions, as illustrated by Tables 8910.

These three contributions permit to solve or reduce the problems listed in the introduction: KRL syntactic translations, KRL parser implementation, dynamic extension of notations, etc. Thus, they provide an ontology-based concise alternative to the use of XML as a meta-language for easily creating KRLs following KRL ontologies. Therefore, this also complements GRDDL and can be seen as a new research avenue. This avenue is important given the frequent need for applications to i) integrate or easily import and export from/to an ever growing number of models and syntaxes (XML-based or not), and ii) let the users parameter these processes.

Previous attempts (by the first author of this article) based on directly extending EBNF -or directly representing or generating concrete terms in a KRL or transformation language -required much lengthier specifications that were also more difficult to re-use.

Besides its translation server, the Logicells/GTH company will use this work in its applications for them to i) collect and aggregate KRs from the knowledge bases they exploit, and ii) enable endusers to adapt the input and output formats they wish to use or see. The goal behind these two points is to make these applications -and the ones they relatemore (re-)usable, flexible, robust and inter-operable.

One theme of our future work on this approach will be the generation of parsing actions in parsing rules, given an implementation "data model". A second theme will be the representation and integration of more models and notations for KRLs as well as query languages and programming languages. A third theme will be the extension of our notation ontology into a presentation ontology with concepts from stylesheets and, more generally, user interfaces.

  KRL_model> (First-order-logic_with_sets_and_meta-statements > (KIF_model r_model_type of: LISP-based_KIF), r_part: 1..* First-order-logic ) (First-order-logic > (CL r_model_type of: CLIF) ) (RIF > (RIF-FLD r_model_type of: RIF-FLD_PS, r_part: RIF-BLD ) (RIF-BLD r_model_type of: RIF-FLD_PS) ) (Graph-based_model > (JSON-LD_model r_model_type of: JSON-LD) (RDF r_part: 1..* JSON-LD_model, r_model_type of: JSON-LD RDF/XML)> OWL-Lite OWL-DL OWL1-Full) (OWL2_model > OWL2_EL OWL2 -RL OWL2-Full), r_part: 1..* (OWL1-Full r_part: 1..* RDF 1..* OWL-DL ) 1..* (OWL2-Full r_part: 1..* OWL2_EL 1..* OW-2_RL ) );

  Phrase < ^(Gterm r_operator: 1 (Operator_type > (owl#Property r_instance: r_binary_relation ) ), r_result: 1 (Truth_value r_instance: True False Indeterminate_truth-value/*for example*/ ) ), > (Phrase_not_referable_in_RIF-FLD //hence, phrase that cannot have an annotation in RIF-FLD > (Annotation > cl#Comment , //"cl#' prefixes terms from Common Logics > (Formal_annotation > (RIF_annotation r_parts: .[0..1 Constant, 0..1 Frame_or_Frame-conjuction]= Grouped_phrases, r_part: 0..* Phrase, > cl#Text, //Phrases is the head element of a KRL grammar > (Module > cl#Module cl#NamedText, > (Document r_part: 0..1 Document_Directive 0..1 Phrases), r_part: 0..1 (Module_parts_that_are_directives < Module, > (Module_header = f_link_type_("",r_header,Module, .[0..* Module_directive]) ) ) 0..1 (Module_parts_that_are_not_directives = f_link_type_("Group",r_group,Module,.[0..* Phrases], < Module, > Module_body Group_of_phrases ), r_parts: .[0..1 Module_header, 0..1 Module_body] ) ) (Module_directive = f_link_type_("",r_relation,Module,Thing), > (Module_name_directive = f_link_type_("Name",r_name,Module,Name) ) (Excluded_Gterm-reference_directive = f_link_type_("",r_excluded_gTerm,Module,.[1..* Gterm_reference]) ) (Document_directive > (Dialect_directive = f_link_type_("Dialect",r_dialect,Module,Name)) (Base_directive = f_link_type_("Base",r_base,Module,Document_locator)) (Prefix_directive = f_link_type_("Prefix",r_prefix,Module,NamespaceShortcut-DocumentLocator_pair)) (Import-or-module_directive > cl#Importation, > (Import_directive = f_link_type_("Import",r_imported-doc,Document,Imported_document_reference) ) (Remote_module = f_link_type_("Module",r_imported-module,Module,Remote_module_reference) ) ) )

  > exclusion { (Formula_reference //this is also an Individual_gTerm > exclusion { Variable_for_a_formula Reference_to_formula_in_remote_module //with the same KRL Reference_to_externally_defined_formula } ) //not in a module and not with the same KRL (Atomic_formula > { Constant_for_a_formula (Atomic_formula_that_is_not_a_constant > near_exclusion //possible shared subtypes: subclass_or_equal, link { (Positional-or-name-based_formula r_operator: 1 Termula, > cl#Atomic_sentence, > exclusion { (Positional_formula r_part: 1..* Termula) (Name-based_formula r_part: 1..* Name-Termula_pair) } ) (Equality_formula = f_link_type_("=",r_equal,Termula,Termula), > cl#Equation) (Class-membership = f_link_type_("#",r_type,Termula,Termula) ) (Subclass_formula = f_link_type_("##",r_supertype,Termula,Termula) ) (Frame = (Frame_as_conjunction_of_links_from_a_same_source ?f r_frame_head: 1 Termula ?fh, r_part: (1..* Link r_link_source: ?fh) ) (Frame_as_head_and_half-links_from_head ?f r_operator: (1 Termula ?fh r_frame_head of: ?f), r_part: (1..* Half_link r_link_source: ?fh), > (Minimal_frame r_part: 1..* Minimal_half-link)

  Individual_gTerm //the expression "Individual term" comes from RIF--FLD //predefined in RIF-FLD which does not rely on an ontology > exclusion { Quantifier Connective_operator Aggregation-function_or_list_operator } ) Symbol_space_identifier //e.g., xs:decimal, rif:iri (Name-Termula_pair r_parts: .[1 Name, 1 Termula]) (Half_link r_link_source: 1 Termula, r_operator: 1 Link_type, r_part: 1..* Link_destination, > (Minimal_half-link r_operator: 1 Link_type, r_part: 1 Minimal_Link_destination) ), (Link_destination r_parts: .[0..1 Cardinality, 1 Termula], > (Minimal_link_destination r_part: 1 Termula)) } ) Fterm_or_variable Individual_abstract_term_of_a_grammar (Operator r_type: Operator_type, > r_relation f_function Operator_not_referable_in_RIF-FLD) (Symbol_space > rif#iri rif#local xs#string xs#integer xs#decimal xs#double) ) }; Fterm_or_variable //cl#Term_or_sequence_marquer, = exclusion { (Variable > Variable_for_a_formula) //cl#Sequence_marquer (Fterm //cl#Term > exclusion { (Gterm_reference > (Constant_gTerm = ^(Gterm r_operator: 0 Operator_type), = exclusion { Individual //in the classic sense of "category that is not a type" (Predicate = Type, //cl#predicate > rdfs#Class (Literal_or_datatype > rdfs#Literal rdfs#Datatype) ) } ) (Reference_to_external_gTerm > Gterm_locator Imported_document_reference /* ... */) ) (Functional_term r_operator: 1 (Function_type < Type), = exclusion { (Non-aggregate_functional_term = Expression) (Aggregate_function_or_collection > (Aggregate_function r_operator: 1 Aggregation-function_operator , r_parts: .[1 Aggregate_function_bound_list, 1 Formula] ) (Collection r_operator: 1 Collection_type, //e.g., rdfs#Container = exclusion { (Unordered_collection > Set) (Ordered_collection > (List = f_function_type_("List",fd_list,.[1..* Ftermula]), = exclusion { Closed_list Open-list } ) ) } ) ) } ) } ) }; Concrete-term_for_constant_or_name //just some examples to show that the same approach applies for concrete terms > (Symbol-space_name r_identifier of: 1..* Symbol_space, > exclusion { (Symbol-space_name_via_bracketed_IRI r_part: 1 IRI_reference ) (Symbol-space_name_via_compact_URI r_part: 1 Compact_URI) } ) //xs:decimal, rif:iri, ... (Variable_name r_identifier of: 1..* Variable, < ^(f_string_type_("?","","") r_part: 1 Undelimited_variable-name) Positive_integer < ^(f_string_type_("+","","") r_part: 1..* Digit))

  Grammar_element //currently, the specifications are mainly only for EBNF-like grammars and Lex&Yacc-like grammars r_part of: 1..* Grammar, //and conversely: Grammar r_part: 1..* Grammar_element; > exclusion { (Phrase_of_a_grammar = exclusion{Non-lexical-grammar_rule Lexical-grammar_rule}, > Head_grammar-rule ) (Individual_gTerm_of_a_grammar = exclusion{ Lexical-grammar_individual-gTerm //what Lex grammars handle Non-lexical-grammar_individual-gTerm } ) };

  [NLG_rule_left-hand-side ":" NLG_expression] ); //→ "A : B" in Yacc or Bison (without parsing actions) Grammar_for_RIF_FLD_in_RIF-PS < Grammar, r_description of: 1..* (RIF-FLD < (KRL_model r_part of: 1..* KRL)), r_part: 1 (fc_NLG_rule_type_( .{RIF-PS}, "RIF-FLD_document", .[ 0..1 Annotation "Document" "(" 0..1 Dialect_directive 0..1 Base_directive 0..* Prefix_directive 0..* Import_directive 0..* Remote_module_directive 0..1 Group ")" ] ) < Head_grammar-rule );

Table 1 :

 1 The same statement -or set of statements (here, a set of relations about Language_or_Language-element) -in five different notations: FL, UML, Turtle, OWL Manchester, OWL Functional-Style. In all other tables, FL will be used.

	Language_or_Language-element //below: links defining it
	= exclusion
	{ (Language r_part: 1..* Language_element,
	> KRL Grammar )
	Language_element
	};

Table 2 :

 2 Examples of relations between KRLs.

	KRL r_part: 1..* Language_element,
	> exclusion { KRL_notation KRL_model },
	r_grammar_head_element_type: Grouped_phrases;
	KRL_notation
	> (S-expression_based_notation > LISP_based_KIF)
	(Function-like_based_notation
	> (RIF_PS > RIF-FLD_PS RIF-BLD_PS) )
	(Graph-based_notation
	> (Markup_language_based_notation
	> (XML_based_notation
	> (RIF_XML > RIF-FLD_XML) )
	(Frame_based_notation > FL JSON-LD Turtle) ) );

Table 3 :

 3 Situating Language_element w.r.t. other types (note: names in italics come from SBVR).

	Thing = owl#Thing, r_identifier: 0..* Individual_gTerm,
	= exclusion
	{ (Situation = exclusion{State Process},
	r_description: 0..* Phrase )
	(Entity //thing that can be involved in a situation
	> exclusion
	{ Spatial_entity //e.g., Square, Physical_Entity
	(Non-spatial_entity //e.g., Justice, Attribute, ...
	> (Description_content = Meaning,
	> Proposition Question
	(Concept > Noun_concept //e.g., types

Table 7 >

 7 Place_holder

, r_identifier of: 1 Thing ) } near_exclusion { Positional_gTerm Frame Gterm_with_named_arguments } near_exclusion //subtyping these types is KRL dependent { Non-referable_gTerm //e.g., a predefined term (Referable_gTerm //via constant/variable/function/phrase r_variable: 0..* Variable, r_result of: 1..* Function, r_annotation: 0..* Annotation, //referable→linkable > (Gterm_that_cannot_be_annotated_without_link r_annotation: 0 Annotation ) Termula ) };

Table 4 :

 4 Main links for defining a structure for abstract terms and specifying concrete terms.

	Gterm r_identifier_or_description of: 1 Thing,
	r_operator: 0..1 Operator , //Table 7
	r_part: 0..* Gterm, //object parts or fct/relation arguments
	r_parts: 1 List, //r_part destinations, sequentially ordered
	r_result: 1 Gterm, //e.g., a phrase has for r_result a boolean
	rc_type: Concrete_term, //rc_type is defined below
	r_variable: 0..* Variable, r_result of: 1...* Function;
	rc_link_to_concrete-term //also often from a Concrete_term
	_[Gterm,Concrete_term] //signature of this relation type
	> rc_begin-mark rc_separator rc_end-mark
	rc_operator_begin_mark rc_operator_end_mark
	rc_operator_name rc_infix-operator_position
	rc_parts_begin-mark rc_parts_separator
	rc_parts_end-mark rc_annotation_position; //-1: before
	r_gTerm_part r_type: Transitive_relation_type,
	> (r_operator > r_frame_source rdf#predicate)
	(r_parameter = r_part, //"r_part" used for concision
	> (r_link_parameter
	> (r_link_source	> rdfs#domain)
	(r_link_destination > rdfs#range) ) )
	(r_phrase_part > rdf#subject rdf#object);
	//r_parts permits to order the parts, this is sometimes
	// needed for abstract terms and this also permits to give
	// a default order for presentation purposes.
	r_parts _[?e,?list]

:=> [any ^(Thing r_member of: ?list) r_part of: ?e]; r_parts _

[?e,?list]

Table 5 :

 5 Important top-level types of phrases (first row); a way to restrict this general model for KRLs (2nd row) (note: names in italics come from RIF-FLD, names in bold italics are used in RIF-FLD signatures, bold is for highlighting).

	*/

Table 8 :

 8 Important types of structured concrete terms (except for strings) and definition of their default presentation. Structured_concrete_term_that_is_not_a_string //the examples in the comments below are in FL; with their delimiters a KRL > exclusion // may have all these structures and still only requires an LALR(1) parser { (List_cTerm > Enclosed_list_cTerm /* e.g., .[A B C] */ Fct-like_list_cTerm /* e.g., A ..[B C] */ ) (Set_cTerm > Enclosed_set_cTerm /* e.g., .{A B C} */ Fct-like_set_cTerm /* e.g., A ..{B C} */ )