Solène Tarride
email: solene.tarride@irisa.fr

Aurélie Lemaitre
email: aurelie.lemaitre@irisa.fr

Signature detection as a way to recognise historical parish register structure

Keywords: Applied computing → Document analysis parish registers, neural networks, signature detection, structure analysis

This article deals with the analysis of pages of French parish registers from 16th to 18th century. These documents are structured in paragraphs called acts. Each act contains valuable demographic information that can be useful to genealogists willing to find information about their ancestors. The first step toward parish register analysis consists of delimiting each act. But these documents are so poorly-structured that the visual separation between the acts is not always clearly visible. One of the main visual indication of separation is the signature of the priest at the end of each act. In this work, we propose to train and compare several u-shaped neural networks for signature detection. We also propose a rule-based system for segmentation into acts and evaluate the impact of signature detection at act level.

INTRODUCTION

Parish registers -or church books -are documents in which were recorded the acts of baptisms, marriages and burials. From 1539, priests were required by law to keep these records written in French in order to save nominative and dated information about the population. This type of document is especially useful to people willing to find their ancestors. In France, parish registers are all the more important before 1792 since the French administration did not register the population before this date. Consequently, parish registers are a major source of demographic data.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. HIP '19, September 20-21, 2019, Sydney, NSW, Australia © 2019 Association for Computing Machinery. ACM ISBN 978-1-4503-7668-6/19/09. . . $15.00 https://doi.org/10.1145/3352631.3352636 Recently, there has been growing interest in transforming these records into digital documents to automatically extract information. However this task is laborious : unlike civil records, parish registers are quite poorly-structured documents, as shown in Figure 1, and present many challenges relative to old documents such as bleed-through degradations, faded ink and contrast variations. But they also present artefacts related to the writer, like cross-outs, ink stains, small spacing and overlapping texts and signatures. All these characteristics make them hard to understand and process. Besides being challenging to read, their structure is also hard to recognise. Each page can contain up to ten acts. At first glance, one could believe that several elements allow for visualising the separation between each act: margin annotations, punctuation, vertical or horizontal spacing... However, these visual elements are not consistent among documents since they depend on the writer. Yet, the end of each act is very frequently signed by the priest, as shown in Figure 2a and Figure 2b. Sometimes, it is also signed by the witnesses, especially for marriages, like in Figure 2c. Consequently, signatures are the main graphical elements that could help to delimit the acts.

Yet, signature detection is not easy in such documents. First of all, signature appearances are variable as shown in Figure 2. Each writer has a unique signature making the signature detection task hard for an automatic system. Moreover, it can be hard to detect some signatures without any information on the meaning of each word. For instance in Figure 2b, the two last words are actually a (a) Act followed by a stylized signature (b) Act followed by a non-stylized signature (c) Act followed by a group of signatures Figure 2: Illustration of different kinds of signatures found in parish registers signature but could easily be interpreted as words by non-Frenchspeaking readers. Second, their localisation is not regular. If most signatures are likely found at the end of a text line (e.g in the right side of the page), some can be found under the act in the middle of the page or in the left side of the page.

In this paper, we aim to delimit each act in these records using signatures as separators. Isolating an act is the first step towards church record analysis and would allow to extract more meaningful information at this level, such as dates and names.

RELATED WORKS

To the best of our knowledge, the challenge of finding signatures in historical books has not been addressed yet. This can be easily explained since this task is very specific to our problematic. However, many similar issues have been successively tackled in historical documents: layout analysis, text line segmentation, baseline detection and ornament/decoration detection.

In the last literature, many neural-network based methods managed to significantly improve state-of-the-art performances [START_REF] Diem | cBAD: ICDAR2017 Competition on Baseline Detection[END_REF] [START_REF] Simistira | ICDAR2017 Competition on Layout Analysis for Challenging Medieval Manuscripts[END_REF], especially u-shaped networks. U-Nets were originally introduced by Ronneberger et al. [START_REF] Ronneberger | U-Net: Convolutional Networks for Biomedical Image Segmentation[END_REF] for biomedical segmentation. It is now widely used for semantic segmentation in many fields of research and industry, including document image segmentation. The principle of U-Net is the following: first, an encoder network takes the image to be processed as an input and outputs a feature map, i.e. a representation of the input in the feature space. Second, a decoder network that is symmetrical to the encoder takes that feature map and outputs the closest match to the actual input.

Recently, Oliveira et al. [START_REF] Oliveira | dhSegment: A generic deep-learning approach for document segmentation[END_REF] introduced dhSegment, a u-shaped neural network that was successively used to solve five different tasks relative to document image segmentation, including ornament detection which is very similar to our task. Promising results were obtained using few training data. One of the advantages of dhSegment is that it uses a pre-trained model (e.g ResNet-50 [START_REF] He | Deep Residual Learning for Image Recognition[END_REF], VGG-16 [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF]) as an encoder so that it takes advantage of the high level features learned on the ImageNet dataset [START_REF] Deng | ImageNet: A largescale hierarchical image database[END_REF]. Such a strategy also significantly reduces the training time.

Grüning et al. [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] propose several improvements to the U-Net for baseline detection. They designed a U-Net with residual blocks, called RU-Net. Residual blocks increase the representative power of the network. The authors also introduced ARU-Net, which is RU-Net designed with a spatial attention mechanism. Finally, they proposed LARU-Net, which is ARU-Net with a MDLSTM layer at the lowest resolution. Taking advantage of data augmentation strategies, they managed to train these models from scratch with few training data. Results show that the ARU-Net architecture performs significantly better than the others for text-line segmentation.

In the following, we refer to all these improved U-Net architectures to {LARU}-Net.

We choose to train both dhSegment [START_REF] Oliveira | dhSegment: A generic deep-learning approach for document segmentation[END_REF] and {LARU}-Net [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] for the signature detection task. This decision is motivated by three main reasons:

• both networks perform well when trained on similar tasks; • they do not require a lot of training data;

• both frameworks are freely available on Github.

Another interesting approach is presented in [START_REF] Lemaitre | A combined strategy of analysis for the localization of heterogeneous form fields in ancient pre-printed records[END_REF] where a rulebased grammar is used to find the location of handwritten fields in pre-printed records. Since the documents share a similar layout, the authors design a logical description of the contents of the collection of documents in order to find a region of interest based on other visual indicators (line segments, keywords...). In the end, they were able to successively detect 97.2% of the fields.

This paper aims to take advantage of both approaches. It is organised as follows: first we describe the objective of the article as well as the methods used, then we describe our database and the experiments performed, finally we evaluate our strategies and comment on the obtained results.

METHOD FOR ACT DETECTION 3.1 Objective

Since signatures are key elements regarding the document structure, our first goal is to achieve pixel-wise segmentation of the signature class. We also want our model to be able to recognise signatures using few training data. Finally, as our goal is to segment pages into acts, we have designed a rule-based description grammar to delimit the acts from the location of the signatures. We evaluate our system at the pixel level and at the act level by comparing the acts obtained when using ground truth and predicted signatures as an input.

Presentation of the networks used

We choose to train dhSegment and {LARU}-Net to detect signatures. We use the code provided by the authors. Both networks were introduced to extract text baselines in old documents. The two architectures are both u-shaped networks -meaning that they consist of a contracting path followed by a symmetrical expensive path. Figure 3 shows the structure of U-Net as it was originally proposed by Ronneberger et al. [START_REF] Ronneberger | U-Net: Convolutional Networks for Biomedical Image Segmentation[END_REF]. The authors also observed that using pre-trained weights was also beneficial to the learning process since the model appeared less sensitive to outliers. For our task, we use a ResNet as a contracting path since it gives better results. Weights are initialised using Xavier initialisation [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF] and optimised using Adam optimiser [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF]. dhSegment can take as input images of any size as long as they fit into memory. To this end, images are resized so that the total number of pixels lies between 6 × 10 5 and 10 6 . Images can also be cropped into patches (we used patches of size 300 × 300) in order to allow batch training. A margin was added to avoid any border effects. Data augmentation is applied to all images: rotation and scaling. The authors claim that these parameters can be applied to most tasks. Just note that the input resolution of the images must be chosen carefully to fit the task. We refer the reader to [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] for a detailed description of the different architectures. We choose to compare U-Net, RU-Net, ARU-Net and LARU-Net to build our method. Each model is trained from scratch with the following parameters. Input images can be of any size. They are normalized and downscaled by a factor of 2, 3 or 4 depending of their original size. Xavier initialisation [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF] is used and the weights are optimised using Adam [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF]. Due to memory limitations, batch size is fixed to 1.

Rule based-grammar

We design a logical description of the documents at the collection level. Our main goal would be to delimit each act in parish registers. Signatures are essential since they mark the end of each act. Another key visual aspect of the acts lies in the text lines. Information about the localisation of signatures and text lines in every documents allow us to design a logical description of parish registers to delimit the acts. This idea has been presented in [START_REF] Lemaitre | A combined strategy of analysis for the localization of heterogeneous form fields in ancient pre-printed records[END_REF] where the authors combine the results of an automatic analysis with a logical description of the contents of the documents.

3.

3.1 Description. The designed system takes as an input the localisation of text lines and signatures, and outputs the bounding box of each act. Text lines were extracted using the baseline detector proposed in [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] which is freely available on Github.

We use the DMOS-PI method to implement logical rules for this collection. The main outline of the algorithm is the following:

• Detect the pages in the image, • In each page, look for a group of signatures. Look for text lines upwards. This combination of text lines and signatures forms an act. • Repeat until finding all the acts in the page.

Result for one act is presented in Figure 4c. Applying this logical description algorithm using baselines and ground truth signatures outputs a pseudo ground truth of the acts.

Limit of the description.

Logical rules were designed to produce a correct delimitation of the acts that are signed. However, while most of the acts are signed, some of them are not. For instance, in Figure 5a, there are actually 3 acts, however two of them are not signed. These acts are ignored to be eventually incorporated in the next signed act, as shown in Figure 5b. Obtaining training data is time-consuming since each signature must be delineated. We have arbitrarily selected 200 images from this database such that each city is represented, and that it spans the whole period. We used VGG Image Annotator [START_REF] Dutta | VGG Image Annotator (VIA)[END_REF] to draw polygons around each signature (Figure 9a). Two ground truth images are obtained :

• Figure 6c: pixels belonging to polygonal zones are set to 255, others are set to 0. • Figure 6d: pixels belonging to polygonal zones are set to their value in the corresponding binary image (Figure 9b), other pixels are set to 0.

For each image, we tried to capture the signatures without including surrounding elements (text, ink stain...) in order to limit the noise induced by the binarisation algorithm. Ground truth images could be improved by precisely delineating the signature outlines, however this would require a significant amount of time.

Signature detection

We choose to compare two Neural Network architectures: dhSegment and {LARU}-Net where {LARU}-Net refers to all the models presented by Grüning et al. [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] (e.g U-Net, RU-Net...). [START_REF] Oliveira | dhSegment: A generic deep-learning approach for document segmentation[END_REF] and [START_REF] Grüning | A Two-Stage Method for Text Line Detection in Historical Documents[END_REF] propose strategies of data augmentation. However, they were specifically designed for baseline detection. For signature detection, we chose to only apply transformations that do not alter the symmetry of the documents. All images are rotated with r ∈ [-0.2, 0.2] rad and scaled by a random factor s ∈ [0.2, 0.5] to speed-up computations. These transformations are applied on-the-fly, and double the training set size.

Data augmentation. Both

Experiments.

We train different architectures in order to compare them. To this end, we perform two main experiments.

The first main experiment consists in a binary classification problem where we use the ground truth images of the outline of signatures (Figure 6d). Background corresponds to class 0 and signature outlines to class 1. In this task, the two classes are exclusive so the softmax cross entropy loss function is used.

Our second experiment is based on the method presented in [5] that suggests that using multiple classes actually helps the training. In the same way, we use 3 classes for signature detection: signatures (Figure 6d), surrounding area (e.g polygonal zones in Figure 6c) and background. For this task, a pixel can belong to more than one class (e.g signature and polygonal zone) so the sigmoid cross entropy loss function is used.

We use the Adam optimizer [START_REF] Diederik | Adam: A Method for Stochastic Optimization[END_REF] and Xavier initialisation [START_REF] Glorot | Understanding the difficulty of training deep feedforward neural networks[END_REF] for all experiments. {LARU} architectures are trained for 250 epochs. dhSegment is trained for 100 epochs since a pre-trained Resnet is used as a contracting path.

4.2.3

Training. We use 5-cross-validation which means that our dataset is divided into 5 subsets. We run the training 5 times, using 3 subsets for training (120 images), 1 for validation (40 images) and 1 for testing (40 images). The main advantages is that every image is at one point in the testing subset so we get an estimation of the test error on all 200 images. It should be noted that we followed the recommendations of the original articles for the number of epochs. {LARU}-Net is trained from scratch, which explain why more epochs are needed.

All experiments run on Nvidia RTX 2080 Ti GPU. Each fold takes between 2 and 10 hours depending on the model.

Post-processing.

The output probability map is threshold with a threshold of 0.5. We do not apply further post-processing technique.

We propose to evaluate the models on their ability to build the acts from their signature predictions. We also evaluate the models at the pixel level in order to compare our results to other tasks. Results at the pixel level show how well the image pixels are correctly classified while results at the act level show how well the acts are detected using the signature predictions. All scores are summarised in Table 1.

At the pixel level

We compute several metrics to compare the performance of the tested models. Note that we only compute scores for pixels belonging to signatures, e.g for the class pictured in Figure 6d.

We use 5 cross validations which means that every image has been tested exactly one time. All scores presented in Table 1 are averaged on all test images except for 6 images that contain no signature. Three scores are presented :

• Recall: a high recall means most of the signature pixels are detected. • Precision: a high precision means that more relevant signature pixels than irrelevant ones are detected. • F1-score: the harmonic mean of recall and precision.

In Figure 7, we depict some results obtained for the best network on 3 images: 2 containing signatures with high and low F1-score and one containing no signature. For each image, we zoom on one specific predicted signature or group of signatures to give an illustration of the segmentation quality. We observe that the model sometimes struggles to differentiate text and signatures.

At the act level

We obtain a pseudo-ground-truth of the acts by applying the grammatical description to the ground truth signatures. Now, we will apply these same rules using the signature predictions, and compare the acts obtained with the pseudo-ground truth. We use the criteria introduced in the PASCAL VOC challenge 2012: a prediction is considered positive if IoU > t. In some images there are more than one detected act overlapping a ground truth act. For those cases the detection with the highest IoU is taken while the others are counted as false positives.

For most object detection tasks, the threshold t is set to 0.5. However, this threshold is too permissive for our case since we mainly observe vertical variations: a predicted act could overlap two ground truth acts and still be accepted as true positive. We choose to set a stricter threshold t = 0.65 in order to avoid this issue. Figure 8 compares two predicted acts with different thresholds, showing that t = 0.5 is too permissive while t = 0.65 appears reasonable. Once the predicted acts have been associated with ground truth acts, several metrics can be computed. We have chosen to compute the precision, recall and average precision -which are frequently used metrics for object detection. The average precision (AP) is the area under the precision/recall curve with precision decreasing. All results are presented in Table 1.

One example of ground truth versus detection is depicted in Figure 9. On the detection image, a false positive signature leads to the creation of a false positive act. In this case, none of the two detected acts sufficiently overlap the ground truth so they are counted as 2 false positives. The ground truth is missed and will be counted as false negative.

Discussion

The aim of our method is to segment registers into acts by using signatures as separators. Our approach allows us to recover 80% of the acts using the LARU-Net architecture with 3 classes. Not only this architecture performs better than the others, but the training is also more stable over the 5 folds. We also show that data augmentation significantly improves the predictions for this model. More generally, we observe that using three classes instead of two definitely improves the performance on the signature class.

As expected, we observe a correlation between the scores at the pixel and act levels. Scores at the act level are higher than at the pixel level because our rule-based grammar is able to construct the acts from a few detected signature pixels, regardless of their precise delineation. This result is encouraging considering the difficulty of the task. However, the obtained scores for signature segmentation are low as compared to performance obtained by dhSegment and • Image quality: contrast variation, blur, uneven lightning... • Ground truth quality: the signature ground truth is dependant on the binarisation and can therefore be noisy, especially on poorly contrasted images.

• Training settings: only 120 images are used for training, which corresponds to approximately 900 signatures. • Task complexity: signatures are by nature very similar to surrounding text since they come from the same writer. A signature can look like a word (Figure 7c) and a word can look like a signature (Figure 7e).

CONCLUSION AND OUTLOOK

In this article, we have tried to recognise the structure of parish registers (e.g the acts) based on signature detection. To this end, we designed a logical description of the collection that takes the localisation of signatures as an input and outputs the acts. We also trained and compared two state-of-the-art neural networks to detect signatures in parish registers. Using this approach, we were able to correctly detect 80% of the acts. We believe that our results could be improved by taking into account other visual indicators (spacing, margin annotations, recurrent keywords...) that could help delimiting the acts in the logical description. We plan to train a neural network to recognise all these visual indicators. This would allow us to build more reliable acts. The logical rules have allowed us to build a pseudo-ground truth of the acts. We plan to take advantages of this pseudo-ground-truth by training a neural network to directly recognise the structure of the acts in order to compare the results with our current method. Finally, we plan to use a metric based on the text lines contained in the acts rather than the overlap in order to associate a predicted act with a ground truth.

Being able to automatically delimit the acts of parish registers would be a first step towards further record analysis. This would allow to extract useful information such as names, places and dates and, more importantly, to index the acts in order to ease the search for ancestors.

Figure 1 :

 1 Figure 1: Pages from a French parish register from 1727 containing 5 acts on the left page and 3 acts on the right page

Figure 3 :

 3 Figure 3: U-Net architecture [10], ©Springer

3. 2

 2 .2 {LARU}-Net. {LARU}-Net refers to 3 different improvements to the U-Net architectures proposed by Grüning et al. in [5]: • RU-Net: U-Net with residual blocks • ARU-Net: RU-Net with attention mechanism • LARU-Net: ARU-Net with a MDLSTM layer

Figure 4 :

 4 Figure 4: Act construction using baselines and signatures

Figure 5 :

 5 Figure 5: Act construction when some acts are not signed

Figure 6 :

 6 Figure 6: Polygonal zones are drawn around each signatures. Ground truth is obtained by combining the binary image and the polygonal zones.

3 Figure 7 :

 37 Figure 7: Error visualisation for signatures taken from: an image with a high F1-score (F 1 = 0.73) (1), an image with a low F1-score (F 1 = 0.35) (2), an image with no signature (3). True positives appear in green, true negatives in black, false positives in blue and false negatives in red.

Figure 8 :

 8 Figure 8: Comparison between different thresholds. The green box represents a ground-truth act, the blue one is a prediction with IoU = 0.65, the pink one is a prediction with IoU = 0.5 that overlaps two ground truth acts.

  (a) Ground truth image (b) Detection image

Figure 9 :

 9 Figure 9: Comparison between ground truth acts and corresponding detection

Table 1 :

 1 Performance of various architectures averaged on test images. On the 200 test images, 6 containing no signatures have been discarded to compute the pixel level results. Standard deviation over the 5 folds is shown for pixel level performance. All 200 images are used to compute the act level results, which corresponds to more than 1500 acts.{LARU}-Net for baseline detection. Several factors can explain this poor performance:

	Pixel level	Act level

ACKNOWLEDGMENTS

The database used for this work was provided by Les Archives Départementales d'Ille-et-Vilaine.


David R Bickel
email: dbickel@uncg.edu

A small-sample Bayesian information criterion that does not overstate the evidence, with an application to calibrating p-values from

Keywords: corrected BIC, corrected Bayesian information criterion, calibrated p-value, calibration of p-values, exaggeration of evidence, overstatement of evidence, strength of statistical evidence

likelihood-ratio tests

Introduction

"In statistical practice, perhaps the single biggest problem with p-values is that they are often misinterpreted in ways that lead to overstating the evidence against the null hypothesis" [START_REF] Benjamin | Three recommendations for improving the use of p-values[END_REF]. The argument that p-values exaggerate the evidence against the null hypothesis has gained ground over decades [START_REF] Berger | Testing a point null hypothesis: The irreconcilability of p values and evidence[END_REF][START_REF] Goodman | Values, Hypothesis Tests, and Likelihood: Implications for Epidemiology of a Neglected Historical Debate[END_REF][START_REF] Stang | The ongoing tyranny of statistical significance testing in biomedical research[END_REF], culminating in an initiative to reduce the level of statistical significance from 0.05 to 0.005 in certain fields of social science [START_REF] Benjamin | Redefine statistical significance[END_REF][START_REF] Machery | The alpha war[END_REF].

According to that Bayesian school of thought, as opposed to a classical frequentist school [START_REF] Mayo | Statistical significance and its critics: practicing damaging science, or damaging scientific practice?[END_REF], the ideal measure of evidence against a null hypothesis is the constant B (x ) of proportionality between the posterior odds and the prior odds of the null hypothesis:

Pr (H 0 | X = x ) 1 -Pr (H 0 | X = x ) = B (x ) Pr (H 0 ) 1 -Pr (H 0 )
, where H 0 is the null hypothesis, X is the random sample of data, and x is the observed sample of data. B (x ), called the Bayes factor, quantifies the strength of the evidence in x against H 0 [START_REF] Jeffreys | Theory of Probability[END_REF][START_REF] Kass | Bayes factors[END_REF] in the sense that lower values of B (x ) mean there is more evidence that H 0 is false, for lower values mean the posterior odds is smaller relative to the prior odds:

B (x ) = Pr(H 0 | X =x ) 1-Pr(H 0 | X =x )/
Pr(H 0 ) 1-Pr(H 0 ) .

(1)

An advantage of the Bayes factor is that it does not depend on Pr (H 0 ), the prior probability of the null hypothesis; by Bayes's theorem,

B (x ) = f 0 (x ) f 1 (x ) ,
where f 0 (x ) is the probability density of the sample conditional on H 0 , and f 1 (x ) the probability density of the sample conditional on H 1 , the alternative hypothesis that H 0 is false.

If H 0 and H 1 correspond to single values of the parameter of interest, possibly using a pseudolikelihood to eliminate any nuisance parameters, then B (x ) is called the likelihood ratio, seen as a measure of statistical evidence within a third school of thought [START_REF] Edwards | Likelihood[END_REF][START_REF] Royall | Statistical Evidence: A Likelihood Paradigm[END_REF][START_REF] Blume | Likelihood methods for measuring statistical evidence[END_REF][START_REF] Strug | A pure likelihood approach to the analysis of genetic association data: An alternative to Bayesian and frequentist analysis[END_REF], which can trace its roots to the likelihood intervals of Fisher (1973, pp. 75-76). More generally, f 0 (x ) and f 1 (x ) are called integrated likelihoods (e.g., [START_REF] Severini | Integrated likelihood functions for non-Bayesian inference[END_REF] or marginal likelihoods (e.g., [START_REF] Oaks | Marginal likelihoods in phylogenetics: a review of methods and applications[END_REF] since they integrate any parameters over prior distributions conditional on H 0 and H 1 , respectively. For example, if f 1 (x |θ 1 ) is the probability density of the sample conditional on the event that the vector parameter is equal to θ 1 , and π 1 (θ 1 )

is the prior probability density (with respect to the Lebesgue measure) of θ 1 conditional on H 1 , then

f 1 (x ) = f 1 (x |θ 1 ) π 1 (θ 1 ) dθ 1 .
The same can be written for H 0 by replacing each "1" by "0."

A way to approximate the Bayes factor without specifying those prior distributions is to use the Bayesian information criterion [START_REF] Schwarz | Estimating the Dimension of a Model[END_REF],

BIC i (x ) = -2 ln f i x | θ i + D i ln n, (2)
as an approximation of -2 ln f i (x ) plus an irrelevant constant term, where

θ i = arg sup θi f i (x |θ i ),
called the maximum likelihood estimate of θ i , again with i = 0 for H 0 and i = 1 for H 1 . The D i , called the dimension of θ i , is how many scalars (real numbers) over which the likelihood

function f i (x |θ i ), as a function of θ i with x fixed, is maximized. Then f i (x ) is approximated by exp (-BIC i (x ) /2
) times an irrelevant constant factor, with the result that B (x ) is approximated by

B BIC n (x ) = exp (-BIC 0 (x ) /2) exp (-BIC 1 (x ) /2) = f 0 x | θ 0 / n D0 /2 f 1 x | θ 1 / n D1 /2 = n (D1 -D0)/2 f 0 x | θ 0 f 1 x | θ 1 . (3)
That approximate Bayes factor has been recommended as an alternative to the p-value [START_REF] Glover | Likelihood ratios: A simple and flexible statistic for empirical psychologists[END_REF][START_REF] Wagenmakers | A practical solution to the pervasive problems of p values[END_REF]. Those papers show how to compute the BIC from sums of squared errors given in the output of standard statistical software. Another readily available quantity from which B BIC n (x ) may be computed is the p-value from a likelihood-ratio test, for its test statistic is a simple function of the f 0 x | θ 0 / f 1 x | θ 1 factor in equation (3). While prior-free and widely applicable, this approach requires samples to be large enough for the approximations to be close.

This paper proposes a simple way to calibrate a p-value from the likelihood-ratio test by transforming it into a Bayes factor that is based on a new correction of the BIC for small samples.

When the unknown parameter of interest is a scalar (as opposed to a vector of two or more scalars), the calibration typically proceeds as follows. Let n denote the size of a sample of more than one observation (n ≥ 2). The calibration transforms the p-value into the Bayes factor

B ⋆ n (p) = √ n exp (1 -1/n) F -1 1 (1 -p) /2 , (4)
where F -1 1 is the quantile function of the χ 2 distribution with 1 degree of freedom (qchisq in R or

CHISQ.INV in Excel

). As the sample size increases, 1 -1/n approaches 1, and B ⋆ n (p) approaches B BIC n (x ), the Bayes factor approximation corresponding to the BIC.

Ideally, a p-value calibration would satisfy these properties:

1. The calibration neither overstates nor understates the evidence against the null hypothesis.

What that means is defined in Section 2.

2. The Bayes factor resulting from the calibration becomes compliant with the BIC as the sample size increases. That has the advantage of sharing in the BIC's eventual selection of the correct hypothesis under broad conditions [START_REF] Neath | The Bayesian information criterion: Background, derivation, and applications[END_REF]. A corrected BIC is proposed in Section 3 to satisfy asymptotic BIC equivalence.

3. The Bayes factor resulting from the calibration can be written as a simple function of the p-value and the sample size-simple enough to easily compute on a phone (cf. [START_REF] Matthews | The p-value statement, five years on[END_REF].

The desired properties are met by equation (4), the general form of which is derived from the proposed BIC correction in Section 4. Calibrations achieving property 3 but lacking either property 1 or property 2 differ substantially from the proposed calibration, as illustrated in Section 5.

2 What does it mean to overstate the evidence against a tested model?

With the Bayes factor in mind as the measure of statistical evidence suggested by equation ( 1), the following definitions specify exactly what is meant by overstating the evidence against the null hypothesis or other tested model and, going further, quantify the extent of that overstatement or exaggeration. With X as the sample space, a measurable function B : X → [0, ∞] is called a Bayes factor estimator, where [0, ∞] is the union of the nonnegative real numbers and ∞.

H 0 and H 1 are labeled in such a way that is that H 0 is the tested hypothesis, or, in terms of Bayesian model selection and Bayesian model averaging, the "model" that is tested. The probability density functions f 0 and f 1 are Radon-Nikodym derivatives with respect to a dominating measure ν. In the case of the Lebesgue measure, the differential element dν (x ) may be written as d x .

Definition 1. The evidential bias of B is bias

B = E 1 B (X ) - 1 B (X ) | H 0 = 1 B (x ) - 1 B (x ) f 0 (x ) dν (x ) ,
Three cases are possible:

1. If bias B = 0, then B is evidence-unbiased.

2. If bias B > 0, then B is evidence-overstating to degree bias B .

3. If bias B < 0, then B is evidence-understating to degree bias B .

Technically, bias B is the prediction bias of 1/ B (X ) as a predictor of 1/ B (X ). Its particular form is suggested by properties of 1/ B (X ) as a martingale (e.g., Feller, 1968, vol. 2, §VI.12), as will be discussed in Remark 1.

Lemma 1. For any Bayes factor estimator B ,

bias B = E 1 B (X ) | H 0 -1 = f 0 (x ) dν (x ) B (x ) -1. Proof. By substitution, bias B = E 1 B (X ) - f 1 (X ) f 0 (X ) | H 0 = E 1 B (X ) | H 0 -E f 1 (X ) f 0 (X ) | H 0 = E 1 B (X ) | H 0 - f 1 (x ) f 0 (x ) f 0 (x ) dν (x ) = E 1 B (X ) | H 0 -f 1 (x ) dν (x ) = 1 B (x ) f 0 (x ) dν (x ) -f 1 (x ) dν (x ) .
The claim follows since f 1 , being a probability density function with respect to ν, satisfies f 1 (x ) dν (x ) = 1.

The next results are immediate consequences.

Corollary 1. For any Bayes factor estimator B ,

1. B is evidence-unbiased if and only if f 0 (x ) dν (x ) / B (x ) = 1. 2. B is evidence-overstating to degree f 0 (x ) dν (x ) / B (x )-1 if and only if f 0 (x ) dν (x ) / B (x ) > 1.
3. B is evidence-understating to degree 1-f 0 (x ) dν (x ) / B (x ) if and only if f 0 (x ) dν (x ) / B (x ) < 1.

All three cases will be illustrated by examples in Sections 3 and 4.

3 Correcting the Bayesian information criterion for smaller samples 3.1 Small-sample corrections of the BIC While the BIC performs well for sufficiently large samples, for smaller samples, it tends to be biased toward selecting more complex models, those of higher parameter dimensions. To compensate for that bias, [START_REF] Mcquarrie | A small-sample correction for the Schwarz SIC model selection criterion[END_REF] proposed this corrected Bayesian information criterion, conventionally abbreviated by "BICc" [START_REF] Ventura | Log-symmetric regression models: information criteria and application to movie business and industry data with economic implications[END_REF]:

BICc i (x ) = -2 ln f i x | θ i + n n -D i -2 D i ln n, (5)
where n ≥ D i +3 to ensure that the second term is positive. The corresponding Bayes factor is

B BIC n (x ) = exp (-BICc 0 (x ) /2) exp (-BICc 1 (x ) /2) = f 0 x | θ 0 / n n n -D 0 -2 D0 /2 f 1 x | θ 1 / n n n -D 1 -2 D1 /2 = n D 1 n -D 1 -2 - D 0 n -D 0 -2 n 2 f 0 x | θ 0 f 1 x | θ 1 , which, in the case of D 0 = 0, is B BICc n (x ) = f 0 x | θ 0 / n 0 f 1 x | θ 1 / n n n -D 1 -2 D1 /2 = n n D 1 2(n -D 1 -2) f 0 x | θ 0 f 1 x | θ 1 .
The second term of equation ( 5) amplifies the likelihood-penalizing term of the BIC (2) by a factor that is larger for more complex models.

Another way to correct for smaller samples is to instead multiply the first term of the BIC (2) by a factor of 1 -1/ n. The evidential Bayesian information criterion (EvBIC) and the its corresponding Bayes factor estimate are, respectively,

EvBIC i (x ) = -2 (1 -1/ n) ln f i x | θ i + D i ln n; B EvBIC n (x ) = exp (-EvBIC 0 (x ) /2) exp (-EvBIC 1 (x ) /2) = f 1-1/ n 0 x | θ 0 / n D0 /2 f 1-1/ n 1 x | θ 1 / n D1 /2 = n (D1 -D0)/2   f 0 x | θ 0 f 1 x | θ 1   1-1/ n . (6)
In short, the BIC is corrected by raising the likelihood ratio in B BIC n (x ) to the power of the exponent

1 -1/ n.
3.2 Evidential biases of the BIC, the BICc, and the EvBIC

The corrections of the BIC are designed to retain the performance advantages the BIC has for large enough samples. Asymptotic equivalence is defined here is terms of the Bayes factor estimators that correspond to the BIC and its variants.

lim n→∞ B ′ n (X ) B ′′ n (X ) = 1.
A sequence of Bayes factor estimators that is asymptotically equivalent to B BIC 2 (X ) , B BIC 3 (X ) , . . . is called, after [START_REF] Schwarz | Estimating the Dimension of a Model[END_REF], a Schwarz sequence. A sequence of Bayes factor estimators B 2 , B 3 , . . . such that B n is evidence-unbiased at every sample size n is called evidence-unbiased. A Schwarz sequence that is evidence-unbiased is called an evidence-unbiased Schwarz sequence.

To avoid overly complicated notation, those definitions are stated assuming the estimators are defined for sample sizes as small as 2. The definitions extend to more general estimators by replacing each "2" with the smallest legal sample size and each "3" with that sample size plus 1.

The likelihood-ratio statistic,

τ (X ) = 2 ln f 1 X | θ 0 f 0 X | θ 1 , (7)
will be used to determine the evidential bias of the BIC, the BICc, and the EvBIC. For that purpose, it is assumed that D 1 > D 0 and that the conditional distribution of τ (X ), conditional on H 0 , is Almost surely,

Pr D1 -D0 ,
= dν (x ) B BICc n (x ) -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 f 1 x | θ 0 f 0 x | θ 1 f 0 (x ) dν (x ) -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 e τ (x )/2 f 0 (x ) dν (x ) -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 ∞ 0 exp 1 2 u d Pr D1 -D0 (u) -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 lim ε↓0 ∞ 0 exp 1 -ε 2 u d Pr D1 -D0 (u) -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 lim ε↓0 ε -(D1 -D0)/2 -1 = n D 0 n -D 0 -2 - D 1 n -D 1 -2 n 2 ∞ -1 = ∞, establishing that B BICc
lim n→∞ B BICc n (X ) B BIC n (X ) = lim n→∞ n D 1 n -D 1 -2 - D 0 n -D 0 -2 n 2 n (D1 -D0)/2 = lim n→∞ n ( D 1 n - D 0 n ) n 2 n (D1 -D0)/2 = 1, establishing that the sequence B BICc D1 +3 , B BICc D1 +4 , . . . is also a Schwarz sequence.
A more positive result is found for the new BIC correction.

= dν (x ) B EvBIC n (x ) -1 = n (D0 -D1)/2   f 1 x | θ 0 f 0 x | θ 1   1-1/ n f 0 (x ) dν (x ) -1 = n (D0 -D1)/2 e τ (x )/2 1-1/ n f 0 (x ) dν (x ) -1 = n (D0 -D1)/2 ∞ 0 exp 1 -1/ n 2 u d Pr D1 -D0 (u) -1 = n (D0 -D1)/2 1 n (D0 -D1)/2 -1 = 1 (D0 -D1)/2 -1 = 0,
establishing that the sequence is evidence-unbiased. Almost surely,

lim n→∞ B EvBIC n (X ) B BIC n (X ) = lim n→∞ n (D1 -D0)/2 n (D1 -D0)/2   f 0 X | θ 0 f 1 X | θ 1   1-1/ n -1 = lim n→∞   f 0 X | θ 0 f 1 X | θ 1   -1/ n = 1,
establishing that the sequence is a Schwarz sequence.

In other words, EvBIC achieves desired properties 1 and 2 of Section 1. Achieving property 3 requires a simple transformation from a p-value to B EvBIC [START_REF] Held | On p-values and Bayes factors[END_REF]. A special case is the exact-p Bayes factor, the constant B ⋆ (p) of proportionality between the posterior odds, given the p-value, and the prior odds of the null hypothesis:

Pr (H 0 | P = p) 1 -Pr (H 0 | P = p) = B ⋆ (p) Pr (H 0 ) 1 -Pr (H 0 )
,

where P is a random variable with values in [0, 1] such that the conditional distribution of P is uniform between 0 and 1 conditional on H 0 . Let f ⋆ 0 and f ⋆ 1 denote the probability density functions of P conditional on H 0 and H 1 , respectively, each with respect to the Lebesgue measure on [0, 1].

Then f ⋆ 0 (p) = 1 for any p between 0 and 1, and the exact-p Bayes factor is

B ⋆ (p) = f ⋆ 0 (p) f ⋆ 1 (p) = 1 f ⋆ 1 (p)
.

Regarding the p-value as a reduced sample of data, the mathematical framework of Section 2 applies without modification to exact-p Bayes factors since they are special cases of Bayes factors.

Specifically, the evidential bias of B ⋆ , any estimator of an exact-p Bayes factor B ⋆ , is bias

B ⋆ = E 1 B ⋆ (P ) - 1 B ⋆ (P ) | H 0 = 1 0 1 B ⋆ (p) - 1 B ⋆ (p) d p = E 1 B ⋆ (P ) | H 0 -1 = 1 0 d p B ⋆ (p) -1.
by Definition 1 and Lemma 1. According to Corollary 1, 1. B ⋆ is evidence-unbiased if and only if

1 0 d p / B ⋆ (p) = 1.
2. B ⋆ is evidence-overstating to degree

1 0 d p / B ⋆ (p) -1 if and only if 1 0 d p / B ⋆ (p) > 1.
3. B ⋆ is evidence-understating to degree 1 -

1 0 d p / B ⋆ (p) if and only if 1 0 d p / B ⋆ (p) < 1.
Example 1. Setting B ⋆ (p) = p means the p-value is used to estimate the exact-p Bayes factor.

Since

1 0 d p p = lim ε↓0 1 ε d p p = lim ε↓0 ln 1 ε = ∞,
that choice of B ⋆ is evidence-overstating to an infinite degree. ▲ Example 2. To argue that p-values overstate the evidence against the null hypothesis, comparisons are often made with a lower bound on the Bayes factor (e.g., [START_REF] Goodman | A dirty dozen: Twelve p-value misconceptions[END_REF], Table 3). The lower bound B ⋆ (p) = -exp (1) p ln p, for p < 1/ exp (1) ≈ 0.37 [START_REF] Vovk | A logic of probability, with application to the foundations of statistics[END_REF][START_REF] Sellke | Calibration of p values for testing precise null hypotheses[END_REF][START_REF] Benjamin | Three recommendations for improving the use of p-values[END_REF], is commonly used due to its convenience. If B ⋆ (p) ≥ 0 for any p ≥ 1/ exp (1),

then 1 0 d p B ⋆ (p) ≥ 1/ exp(1) 0 d p -exp (1) p ln p = lim ε↓0 1/ exp(1) ε d p -exp (1) p ln p = lim ε↓0 ln ln (1/ε) exp (1) = ∞.
It follows that any such choice of B ⋆ is evidence-overstating to an infinite degree; see [START_REF] Vovk | E-values: Calibration, combination and applications[END_REF]. ▲ Remark 1. Reciprocals of evidence-understating and evidence-unbiased estimators of the p-exact

Bayes factor largely overlap with martingales used in game-theoretic probability [START_REF] Shafer | Author's reply to the discussion of ẗesting by betting: A strategy for statistical and scientific communication by glenn shafer[END_REF]Vovk, 2001, 2019). The approaches intersect in the special case that H 0 is simple as opposed to composite, the case in which the null hypothesis corresponds to a single, non-mixture probability distribution as opposed to a mixture of distributions over a prior distribution. Then the reciprocal 1/ B ⋆ of a Bayes factor estimator of a p-exact Bayes factor, as a function on

[0, 1] with values in [0, ∞], is a p-to-e calibrator if B ⋆ (p)
is increasing as a function of p and if 2023) call an "e-value") is a data-based Bayes factor estimator B that is not evidence-overstating, and the reciprocals of e-values that satisfy the property of "a unit bet against" the simple null hypothesis [START_REF] Ramdas | Game-Theoretic Statistics and Safe Anytime-Valid Inference[END_REF] 2. B ⋆ is evidence-overstating to degree 1/ ln (b) -1 if b < exp (1). For instance, -1/ log 2 is evidence-overstating to degree 0.44. [START_REF] Greenland | Divergence versus decision p-values: A distinction worth making in theory and keeping in practice: Or, how divergence p-values measure evidence even when decision p-values do not[END_REF] interprets -log 2 p as the number of bits of information against the null hypothesis. Shafer (2021) mentions that -log 2 p is not a betting score.

1 0 d p / B ⋆ (p) ≤ 1; if, in addition, 1 0 d p / B ⋆ (p) = 1, if B ⋆ is lower semicontinuous, and if B ⋆ (0) = 0, then 1/ B ⋆
3. B ⋆ is evidence-understating to degree 1 -1/ ln (b) if b > exp (1). For instance, -1/ log 10 is evidence-understating to degree 0.57. [START_REF] Gibson | The role of p-values in judging the strength of evidence and realistic replication expectations[END_REF] argues that -log 10 p measures the strength of the evidence against the null hypothesis.

▲

Remark 2. As the p-to-e calibrators suggested in the literature do not depend on the sample size, they cannot form Schwarz sequences. Nor can they satisfy the generally weaker condition of model selection consistency [START_REF] Neath | The Bayesian information criterion: Background, derivation, and applications[END_REF]. For, when the null hypothesis is true, their reciprocals (the corresponding Bayes factor estimates) do not increase with the sample size. As a result, the corresponding posterior probabilities of the null hypothesis do not approach 1 as the sample size increases. Those difficulties reflect fundamental differences in how p-values and Bayes factors depend on the sample size, as explained in [START_REF] Efron | Scales of evidence for model selection: Fisher versus Jeffreys[END_REF] and epitomized as the Lindley paradox (cf. [START_REF] Naaman | Almost sure hypothesis testing and a resolution of the jeffreys-lindley paradox[END_REF][START_REF] Cousins | The Jeffreys-Lindley paradox and discovery criteria in high energy physics[END_REF].

Calibrating p-values using the evidential Bayesian information criterion

Recall that the p-value of a likelihood-ratio test given x is

p = 1 -F D1 -D0 (τ (x )) , (9)
where τ is the function defined by equation ( 7

⋆ n (p) = n (D1 -D0)/2 exp (1 -1/n) F -1 D1 -D0 (1 -p) /2 , ( 10
)
and 1/ B ⋆ n is an admissible p-to-e calibrator.

Proof. From equations ( 7) and (9), Substitution into equation ( 6) gives

f 1 x | θ 1 f 0 x | θ 0 = e τ (x )/2 = e F -1 D 1 -D 0 (1-p)/2 .
B ⋆ n (p) = n (D1 -D0)/2 1 exp F -1 D1 -D0 (1 -p) /2 1-1/ n .
Since B ⋆ n is evidence-unbiased (Theorem 1), continuous, and increasing with p, and since B ⋆ n (0) = 0, it follows that 1/ B ⋆ n meets the sufficient conditions for an admissible p-to-e calibrator given in Remark 1.

Comparisons of simple Bayes factor estimators

The Bayes factor estimator based on the EvBIC is related to two classes of other simple estimators:

1. Schwarz sequences that are evidence-overstating (compared in Section 5.1) 2. Evidence-unbiased Bayes factor estimators that do not form Schwarz sequences (compared in Section 5.2)

Figure 1 lists some cases of each class.

Comparisons of simple Schwarz sequences

Using

Comparisons of simple evidence-unbiased calibrations

Definition 2 .

 2 The sequence of Bayes factor estimators B ′ 2 , B ′ 3 , . . . is asymptotically equivalent to the sequence of Bayes factor estimators B ′′ 2 , B ′′ 3 , . . . if, almost surely,

  the χ 2 distribution with D 1 -D 0 degrees of freedom. . and B BICc D1 +3 , B BICc D1 +4 , . . . are Schwarz sequences, but they are not evidence-unbiased. Rather, for every allowed n, B BIC n For each n ≥ D 1 +3, the evidential bias of B BICc n is, by Lemma 1, bias B BICc n

n

  is evidence-overstating to an infinite degree. Analogous steps prove bias B BIC n = ∞ for every n ≥ 2. Since the BIC is asymptotically equivalent to itself, B BIC 2 , B BIC 3 , . . . is a Schwarz sequence.

  . is an evidence-unbiased Schwarz sequence.Proof. For each n = 2, 3, . . . , the evidential bias of B EvBIC n is, by Lemma 1, bias B EvBIC n

4

  Calibrating p-values by transforming them into Bayes factors 4.1 Calibrating p-values using exact-p Bayes factors The Bayes factor B (x ) is what Held and Ott (2018) call a "data-based Bayes factor" since it depends on the distribution of X . A Bayes factor that instead depends on the distribution of the p-value is called a "p-based Bayes factor"

  is an admissible p-to-e calibrator (Vovk and Wang, 2021, Proposition 2.1). It follows that the reciprocals of such calibrators are not evidence-overstating, and that the reciprocals that are admissible are evidenceunbiased. Moving from functions of P to functions of X , the reciprocal of what Shafer (2021) calls a "betting score" (what Grünwald et al. (

0 -

 0 are evidence-unbiased Bayes factor estimators. Example 3. Consider B ⋆ (p) = -1/ log b p for some b > 1. Greenland (2023) recommends 1/ B ⋆ (p) = -log b p as the surprisal (cf. Bickel, 2023). Since bias (-1/ log b ) = 1 log b p d p -1 = 1 ln (b) -1, there are three cases: 1. B ⋆ is evidence-unbiased if b = exp (1). Shafer (2021) notes that -ln p is a betting score, albeit not one that brings traditional thresholds of frequentist and Bayesian inference into as close agreement as does p -1/2 -1. (Achieving that agreement remains a challenging problem, for reasons summarized in Remark 2.)

  ), and F D1 -D0 is the cumulative distribution function of the χ 2 distribution with D 1 -D 0 degrees of freedom. Access to such a p-value from software or the literature enables recovering the likelihood ratio, thereby facilitating calculation of the Bayes factor estimated from the BIC, BICc, or the EvBIC. The EvBIC case is shown next. Theorem 2. If B ⋆ n (p) = B EvBIC n (x ) for a p-value of a likelihood-ratio test given x and for any n ≥ 2, then B

Figure 1 :

 1 Figure 1: Simple Bayes factor estimators from left to right: B BIC ⋆ n

  D 1 -D 0 = 1, Figures 2-3 compare the proposed B ⋆ n (p) to B BICc ⋆ n (p) and B BIC ⋆ n (p), the two quantities defined in the caption of Figure 1. While the three Bayes factor estimates are asymptotically equivalent, B ⋆ n (p) is evidence-unbiased (bias (B ⋆ n ) = 0), whereas the other two are

Figure 2 :

 2 Figure 2: Bayes factor estimates B ⋆ n (p), B BICc ⋆ n

Figure 3 :

 3 Figure 3: Bayes factor estimates B ⋆ n (p), B BICc ⋆ n

Figures 4 -

 4 Figures 4-5 compare the proposed B ⋆ n (p), in the D 1 -D 0 = 1 case, to the two evidence-unbiased Bayes factor estimates discussed in Example 3, item 1. Since -1/ ln p and 1/ p -1/2 -1 do not depend on n, they do not have B ⋆ n (p)'s Schwarz-sequence property of asymptotic equivalence tothe Bayes factor estimate that corresponds to the BIC (Section 1, property 2; see Remark 2). They do satisfy properties 1 and 3 of Section 1.

Figure 4 :

 4 Figure 4: Bayes factor estimates B ⋆ n (p), -1/ ln p, and 1/ p -1/2 -1 , as functions of n, for p = 0.05, 0.005.

Figure 5 :

 5 Figure 5: Bayes factor estimates B ⋆ n (p), -1/ ln p, and 1/ p -1/2 -1 , as functions of p, for n = 2, 4, 8, 16.

Acknowledgments

This research was supported by the University of North Carolina at Greensboro.


E L Hamzaoui

Omar Kharbouch

Hamzaoui El

ORCID Omar

ORCID, Kharbouch Omar

Keywords: Open data, portal, implementation, use, KPIs

Déclaration de divulgation : L'auteur n'a pas connaissance de quelconque financement qui pourrait affecter l'objectivité de cette étude.

Introduction

Open data as a concept was born in 2009 in the United States and the United Kingdom following a new approach to the management and publication of public data in favor of a resurgence of the commons approach in the digital ecosystem, some apprehend these open data shared as potential unprecedented forms of common goods as stated by [START_REF] Larroche | Loi n° 09-08 relative à la protection des personnes physiques à l'égard du traitement des données à caractère personnel[END_REF]. This movement, is more specifically part of the specific movement of Open Data from the public sector, supported by governmental and territorial authorities supporting policies aimed at creating dedicated platforms. More precisely, In Morocco, the study of the issue of Open Data, as a project of application of the right of access of citizens to public information was raised on March 31, 2012 by the general assembly of the Economic, Social and Environmental Council (ESEC) which had entrusted, within the framework of the self-referral procedure, the Commission for Cultural Affairs and New Technologies to study this project, as it showed on their report (ESEC, 2013).

The scope of this project is twofold: it aims to give concrete form to the advances made by the Constitution of July 2011 in terms of the right of access to public information while enabling Morocco to comply with a number of its international commitments, and to meet the standards in force in all democratic countries.

With the advent of the digital era, a new movement, called Open Government Data (or Open Data), emerged under a new dimension of the right of access to information, emphasizing the possibility of exploiting and reusing public information through free access to raw data collected or produced by administrations for their valorization.

Beyond the progress it implies in terms of democracy and transparency, Open Data aims through the innovative approach to provide citizens and businesses with access to public data; to boost economic development by making information more widely available and circulating it more efficiently; to develop information technologies and establish a digital economy. Open Data, like many of the vectors of digital transformation, is based on a whole range of movements: societal, legal, digital, cultural, organizational and citizen-based. This requires the implementation of appropriate mechanisms, particularly legal and operational, to drive the paradigm shift in the role of public administration.

As demonstrated by [START_REF] Carmes | L'Open Data du secteur public au prisme de la pensée des communs : une techno-politique empêchée ?[END_REF], Open Data is a crucial tool for improving the quality of public services. In fact, the main beneficiary of the provision/feeding/supply of raw data by the administration is the administration itself. In a public administrations and establishments, thus constituting an important lever for reinforcing this transformation and promoting the opening up of public data, mainly from public administrations, establishments and companies, and local authorities, and maximizing the impact of their reuse and exploitation by the various socio-economic players.

The subject of this paper addresses the efforts made by Moroccan public administrations, establishments and companies to ensure the proper implementation of datasets that meet the various evolving needs of socio-economic players, and thus make the most of Open Data applications by maximizing the use of the national Open Data portal http://www.data.gov.ma.

As a result, we will be looking more closely at the link between the implementation of the portal and its use, and at the prospects for developing an ecosystem conducive to the use and promotion of public data. The aim of this research, is to study the potential linked to the use and promotion of the open data portal in Morocco as a function of its supply and implementation by public data, and to draw out its development prospects as an important lever for the promotion of the opening of public data.

To address this issue, the research study will be structured in two main parts. This paper will first present the key concepts of Open Data, the data life cycle, detailing the steps involved in publishing open data, as well as the legal framework of Open Data. Secondly, we will explore, through our empirical study, the relationship between the implementation of the Open Data portal and its use, presenting the methodology used and the study's findings. Following the example of the world's developed countries, Open Data Morocco aims beyond the right of access to information, Open Data focuses on two main dimensions: access to raw data and the ability to reuse these data from the portal that hosts data sets from certain organizations and institutions, and references data sets hosted on sectoral open data portals.

Concept of Open data

Open data is a concept that emerged in 2009 in the United States and the United Kingdom and inspired by a new approach to public services and the management and publication of public data, based mainly on intra-and extra-governmental collaboration that aims to promote transparency of governance, by providing citizens with the means to evaluate the policies implemented by the government and their impacts.

At the global level, 153 governments have adopted the Open data portal according to the latest e-government survey conducted by the Department of Economic and Social Affairs in 2020.

As shown in Figure 1, Europe is at the forefront of continents in Open data provision in all sectors of activity, while Africa is at the bottom of the ranking compared to other regions, particularly in the sectors of environment, justice and social security. of a mold, meteorological records from a ship that crossed the Atlantic in the 18th century, or the recording of a rare dialect. All these elements have in common that they are (now) digital data, produced during a research process, as demonstrated by [START_REF] Gaillard | De l'Open data à l'Open research data : quelle(s) politique(s) pour les données de recherche[END_REF]. This concept originates from the conviction that government data of any type according to table 1, should be freely available to be published, used and reused without limitations or control.

According to the definition adopted by (the World Bank, 2019

Statistical Data

Population, economy, environment.

Political Data

Election results, assembly interventions, presence, etc.

Research Data

Data generated by public research institutes and organizations.

Sectoral Data

Packed agricultural areas, annual number of tourists, passenger air traffic.

Financial Data

Exchange rates, citizen budget, finance law.

Source: developed by the authors, based on examples taken from a benchmarking

Data lifecycle

The main aim of implementing Open Data is to transform public structures so that their open data is at the heart of public action. The goal is to put in place several elements within public structures. Among these elements is the implementation of a robust data management process in which the publication of open data becomes an integrated output́ in the production and exploitation of data from its collection to its archiving. In the data lifecycle, as illustrated in

The power of data

According to [START_REF] Monino | L'information au coeur de l'intelligence économiquestratégique[END_REF] This concept embodies the foundations of a governance culture based on innovative and sustainable public policies, inspired by the principles of transparency, accountability, and participation, which strengthen democracy and contribute to inclusive growth.

In Morocco, the adoption of an open government approach is a key component in the success of establishing a new system of public governance. It is grounded in the Moroccan constitution, encompassing articles 1, 12, 13, 14, 15, 25, 26, 27, 28, 29, 36, 154, 155, and 156, as

e-Government

The term e-Government (e-Gov) or online public administration emerged in the late 1990s, although the history of government digitization dates back to the early days of computing.

According to the World Bank ( 2015), e-Government is officially defined as "the use by public agencies of information technologies (such as wide-area networks, the Internet, and mobile

e-Participation

E-participation is generally considered a part of e-government. According to the United Nations' survey on electronic administration, e-participation is defined as the mechanism of engaging citizens through ICT in the design of policies, decisions, and services to make them inclusive and deliberative.

In the 2020 survey, questions about electronic participation were carefully examined and developed to reflect current trends and methods in which governments involve their populations in the development, implementation, and evaluation of public policies. Updates included information on citizens' rights to access government information and citizens' feedback regarding the improvement of online public services.

As part of the same survey, the United Nations introduced the E-Participation Index (EPI).

Mathematically, this index is calculated by subtracting the lowest score from the total score of a given country and then dividing the result by the subtraction between the highest score and the lowest score, UNDESA (2020).

Development of hypotheses

Through the literature review and in order to carry out our empirical investigation and answer our research question, we formulated the following hypotheses:

H1: The implementation and supply of Open Data foster its use by citizens and users;

H2: Users have a preference for raw data, data sets in the form of "dumps" rather than eparticipation interfaces that allow querying the database to extract subsets of data;

H3: The evolving needs of future users of Open Data Morocco are met as the portal is used.

Research methodology

In the study of our research question, we adopted a hypothetico-inductive approach based on observational results through both quantitative and qualitative methods.

The choice of this methodological approach is motivated by the epistemological positioning linked to the inductive approach, which involves formulating conjectures based on observations that should be tested subsequently [START_REF] Koenig | Production de la connaissance et constitution de pratiques organisationnelles[END_REF]. This mode of reasoning is based on considering observations derived from the reality of open data implementation as true.

Therefore, any proposition resulting from a logical analysis related to its use and feeding can be considered as true. This research methodology was implemented in a two-step research process:

The first step involved conducting semi-structured interviews with operational and functional experts from the DDA responsible for the data.gov.ma portal. These interviews focused on key study areas, including implementation, use, and future development prospects of Open Data.

The insights gathered from these interviews allowed us to identify the main Key Performance Indicators (KPIs) of the data portal, categorized into data provisioning and usage. We then statistically analyzed the results obtained from an exploratory documentary analysis of Open Data KPI reports covering the period from the portal's launch in December 2021 to June 2023.

The second step involved the qualitative analysis of the content from the semi-structured interviews with DDA officials. The aim was to analyze the data collected during the interviews using content analysis methodology to uncover the determinants influencing the perspectives of various stakeholders and citizens regarding the use of the Open Data portal.

Considering that the progressive deployment of the functionalities, domains of the portal and its provision have allowed to increase its use and to broaden the domains and sectors of activity related to public data, and access to public data in a flexible and efficient way while improving the quality of services provided to users.

Study field and description

To carry out our study, we decided to collect data following both a quantitative and qualitative approach to be able to identify the research variables that are closely related. Therefore, our adopted research methodology is a mixed, exploratory descriptive methodology based on the technique of documentary analysis and semi-structured interviews through:

1. Quantitative analysis through the examination of the evolution of the indicators "Open Data KPIs" extracted on the basis of a framework designed by ourselves and communicated and filled in by the operational managers of the DDA (table 3). The two main research variables are: the implementation and use of the Open Data portal: www.data.gov.ma;

2. Qualitative analysis of the results of findings and determinants related to the perspectives of evolution of Open Data Morocco, through semi-structured interviews with the functional and operational managers of the Portal classified by interview themes (table 4).

Research Model and Data Processing

Our initial problem and objective are to explain and understand the reality and to draw out findings related to the implementation of Open Data in Morocco, as well as to test hypotheses derived from existing theories and the literature review. For our adopted research methodology, we have chosen a descriptive mixed exploratory approach based on documentary analysis and semi-structured interviews.

This approach appeared most suitable for addressing our research questions. Additionally, (Wacheux, 1993)

Results & Discussion

The results of this analysis and exploration of the designed KPIs have allowed us to understand the implementation strategy carried out by the National Open Data Steering Committee of Morocco and to explain the evolution and performance in terms of data usage, supply, and the quality of service rendered to users. From this perspective, we have observed that the DDA has undertaken numerous actions in line with the National Open Data Action Plan. Notably, the flagship indicator for data provision has shown a significant increase of over 100% in the number of published datasets (Figure 5), as well as in the number of website visits and

. Evolution of published datasets on the OD Portal

The number of datasets published on the portal has experienced a significant increase since its launch, starting with 136 datasets. By the 1st semester of 2023, this indicator reached 412 published datasets (figure 6). This substantial increase can be attributed to the existing potential of open data that can be published by various public institutions and organizations that voluntarily embraced the concept of open data through the National Open Data Portal.

Furthermore, this progression can also be attributed to the efforts made by the DDA in terms of communication, training, and awareness campaigns. These endeavors aimed at encouraging both existing and new data producers to publish their public datasets.

It is worth noting that the implementation of the new "e-participation" feature in 2023, at processing user requests on the platform, has facilitated feeding of new datasets to the portal.

Evolution of the number of themes published on the OD portal: 2021-2023

The number of data themes published on the portal has experienced a parallel increase to that of datasets during the same period. It is worth noting that the portal was initially launched with 9 data themes and has now expanded to include 19 themes, representing an increase of over 100% in less than two years (figure 7). This significant growth can be attributed to two main factors:

-The first factor is related to the increase and addition of new data producers operating across various socio-economic sectors in Morocco.

-The second factor is linked to the introduction of the "e-participation" section, which is designed to address user requests and cater to the evolving needs of users for specific themes.

Evolution of the number of data producers on the OD portal

Data producers refer to organizations and state institutions that contribute to the Open Data

portal by providing open datasets. A dedicated section for consultation has been set up on the portal to allow quick access to datasets grouped by their respective producers.

This indicator originally encompasses the two previous indicators: datasets and data themes, as it is intuitively strongly related to their evolution. This indicator has shown significant growth, increasing from an initial number of 16 data producers to 36 producers during the first half of 2023. This represents a remarkable increase of over 100%, similar to the pace of evolution observed for data themes and datasets (figure 8).

Number of data publication requests processed via the e-participation interface

This indicator reflects the level of interactivity with users on the portal. The examination of the report extracted from the portal www.data.gov.ma revealed a total of 14 requests for the addition of public data (figure 9), distributed across the following themes:

• Health theme: 2 requests significant growth, this particular indicator does not show a remarkable evolution. This observation can be explained by two hypotheses:

•
-The first hypothesis confirms that users have a preference for raw data published directly rather than processed data that caters to specific needs.

-The second hypothesis suggests that the increased provision of diverse datasets on the portal allows for anticipating potential user needs. The number of visitors to the Morocco Open Data portal surpassed 100.000, reaching 128.000 visitors in the first half of 2023, indicating a growth of over 47% compared to the year 2022.

However, this indicator alone is insufficient to fully analyze the use of the portal's data, as some visits may not involve actual use of the published data (Figure 10).

Evolution of the number of downloads on the OD portal

The number of downloads from the Morocco Open Data portal exhibits a clear upward trend, as seen in Figure 12, reflecting an increasing frequency of access to the published data by users.

This indicator correlates with the number of uploads to the portal's hosting server. In addition to the analysis of the previous indicator regarding visitor traffic around the portal, the evolution of this key indicator highlights the growing demand for utilizing the data published under various themes, experiencing a 53% increase between 2022 and the first half of 2023.

Correlation Coefficients

The correlation coefficient is the specific measure adopted to quantify the strength of the linear relationship between the key indicators of the two variables: supply (X) vs. utilization (Y). The coefficient is denoted as 'r' in the correlation matrix calculated using the formula: According to Table 5, the results obtained from the correlation calculation table analyzed between the items of the two study variables have revealed a strong positive correlation (𝑟) approaching 1, indicating the intensity of a linear association between the sub-variables of supply and the utilization of OD. This is evident in the correlations among the following: -The number of data producers and the number of downloads on the OD portal;

-The number of data producers and the number of page views on the OD portal;

-The number of supplied datasets and the number of downloads on the OD portal;

-The number of supplied datasets and the number of visitor countries of origin. The data was collected through four semi-structured interviews with experts from the OD portal at the DDA (Table 6). Specifically, the Head of Digital Transformation Department of the company, the Head of Digital Transformation Department of the Economy, and others. The aim is to analyze the collected interview data using the content analysis method to determine potential areas of development for the Open Data Morocco project and the perspectives that influence development in both aspects of the study: implementation and perspectives. 2.3. Discussion and Hypotheses Testing Hypothesis 1 (H1): "The implementation and feeding of Open Data further promote its utilization by citizens and users." According to Table 5, the correlation coefficient (r) demonstrates positive and satisfactory values of 0.947, 0.966, and 0.999, which are close to the value of 1. This confirms a positive relationship between the studied constructs, namely the implementation and feeding of Open Data, and the utilization of the Open Data portal. This is reflected in quite significant correlations between the supply items and the utilization items:

Hypotheses

1. The number of fed datasets and the number of downloads on the OD portal;

2. The number of data producers and the number of downloads on the OD portal;

3. The number of added themes and the number of portal visitors.

These three validated correlations confirm a positive relationship between the constructs. Thus, Hypothesis (H1) is validated. This result aligns with the findings of the United Nations Department of Economic and Social Affairs' 2020 e-government survey, which affirms the evolving trends in the use of public data across countries worldwide, with the number of countries with Open Data portals increasing from 43 in 2018 to 153 (an 80% increase). Making open data available enhances public access to information. This observation is corroborated by the results of the 2020 survey on access to individual data, indicating that while there has been improvement since 2018, regional percentages of access to individual data vary.

Hypothesis 2 (H2): "Users have a preference for raw data, dataset 'dumps,' over e-participation interfaces allowing queries for subsets of data." The analysis of the correlation between the number of requests for publication of processed data through the e-participation interface and the number of fed datasets on the OD portal reveals a coefficient (r) with a value of 0.0193. This demonstrates a positive yet insignificant correlation between these two variables, indicating a weak relationship between the number of e-participation queries and the evolving number of themes fed and published on the OD portal. The historical evolution of requests for specific data through the e-participation section does not mirror the pace of feeding and utilizing the OD portal. This is explained by users' preference for raw data available on the portal over processed data via the interactive e-participation feature. Therefore, Hypothesis (H2) is validated. This observation aligns with the findings of the United Nations' 2020 e-government survey regarding e-participation, which demonstrates its progress faces various existing and new challenges and risks, such as cybersecurity and data privacy. These correlations reflect a positive relationship between three items of the "OD supply" variable and three items of the "OD usage" variable:

1. The number of fed datasets and the number of downloads on the OD portal;

2. The number of data producers and the number of downloads on the OD portal;

3. The number of fed datasets and the number of visitor countries on the OD portal.

Given the highly significant and positive correlation values between variables related to OD portal utilization and variables related to user needs being fed into the portal, Hypothesis (H3) is therefore validated. Open Data also faces a number of obstacles that may hinder its development and implementation. Some of these obstacles are related to data producers or data users. In Morocco, these challenges can be attributed to both parties and are primarily related to: Based on our analysis of the results, we can conclude that this study provides evidence of the impact of feeding Open Data with quality public data that meets the evolving needs users on the utilization and promotion of Open Data in Morocco. By working towards establishing a new approach to utilizing public data that favors Morocco's move towards open government, focusing on openness, transparency, and accessibility of digital services, this study aims to strengthen trust in the government. This research offers several recommendations. Future researchers could empirically examine the effect of the public data reuse function on citizen engagement and involvement in sectoral policies in Morocco. Studies could also compare two or more regions within the same culture to assess the significance of public data openness in entrepreneurship and local development.

Finally, considering the importance of Open Data as a digital transformation tool, this study regards the quality of the information system of public data producers as an accelerator for implementing an open public data polic.

  government Data Camp (2010) "Studies conducted in certain developed countries have shown that 70% of public data that has been subject to transactions was acquired and used by other public entities". Morocco through its Maroc Numeric 2013 e-gov strategy had set up a first version of data.gov.ma in March 2011, which made Morocco a pioneer country in terms of Open Data platform. the Open big Data Maroc project was relaunched by the Digital Development Agency (DDA) and financed by the world bank through the new version of the national open data portal www.data.gov. ma, by the end of 2021, with over 250 datasets covering 14 themes and 24

  In recent years, Morocco has committed itself to its economic and social transformation by developing entrepreneurship and digital innovation in the private sector, through improving transparency and modernizing public administration. In this context, the Open Big Data Morocco project was launched by the Digital Development Agency (DDA) through a new version of the national open data portal www.data.gov.ma, constituting an important lever to reinforce this transformation and promote the opening of public data that mainly come from administrations, public institutions and enterprises, and local authorities and maximize the impact of their reuse and exploitation by various socio-economic actors.

  Explicitly, "Open Data consists of exposing, in free and open access, data produced by the concerned organizations and institutions at the level of the national open data portal and/or the portals of these organizations and institutions using open, standard and documented formats to facilitate the exploitation and reuse of these data" (Definition taken from the Portal: www.data.gov.ma). The diversity of typologies related to the concept of data makes it difficult to define this concept precisely. The main challenge in defining the notion of "research data" in the field of scientific research is to identify what can bring together such diverse elements as a photograph of a beetle collected in Madagascar, a spectroheliogram produced in Meudon, information about the genes

Figure N°1 :

 N°1 Figure N°1: Provision of sectoral Open Data through International Open Data Initiatives

Figure 2 ,

 2 Figure 2, Open Data is an integral part of this cycle in the preservation & reuse phase.

Figure

  Figure N°2: Data lifecycle

Tableau N°2 :

 N°2 List of Creative Commons Licenses Source: as listed by Creative Commons organizationIn Morocco, there is a clear legal framework that governs the publication of data, distinguishing between data that can be freely published, data that cannot be disclosed, and data that require . The development of Open Data relies on two essential legislative pillars: the laws of access to information and the laws relating to the protection of personal data. The Access to Information Law, Law No. 31-13 of February 22, 2018: This law precisely defines the public information that can be disclosed upon request or proactively, Morocco (2018).The Law on the Protection of Individuals with regard to the Processing of Personal Data, Law No. 09-08 of February 18, 2009: This law provides a regulatory framework for safeguarding privacy and identifies specific data falling under its scope, along with the required processing before publication, Morocco (2009). Article 27 of the 2011 Constitution states: "Citizens have the right to access information held by the public administration, elected institutions and bodies entrusted with a public service mission. The right to information can only be limited by law, in order to ensure the protection of everything related to national defense, internal and external security of the State, as well as the privacy of individuals, to prevent any infringement on the rights and freedoms stated in this Constitution, and to protect sources and areas expressly determined by law." Morocco (2011).

  , the use of data generates billions of dollars according to various reports. The McKinsey Institute recently indicated that the public release of Open Data would enable the United States to save $230 billion by 2020. This would be achieved by allowing startups to develop innovative services aimed at reducing unnecessary energy expenses. The Hierarchical model mentioned in figure 3, can be applied to open data by using structured analysis to gather, process, and interpret data from various sources. This model can aid in understanding trends, patterns, and potential future developments within the realm of open data, helping organizations make informed decisions and optimize their data utilization strategies.

Figure

  Figure N°3 : Data lifecycle

  well as royal orientations articulated in various speeches, such as those on July 29, 2018, during the Throne Day celebration, and on October 14, 2016, at the beginning of the legislative year. Based on the OECD (2020)'s recommendations in its review report on public governance in Morocco also emphasize the need to "continue efforts to promote openness, transparency, and accessibility of digital services to enhance trust in the government and create a more transparent and accountable public sector." To materialize these directions, a National Open Government Action Plan (OGAP) for 2021-2023 has been formulated, with the establishment of a multi-party entity to oversee the OGAP process, comprising three bodies: the Steering Committee, the Implementation Committee, and the Civil Society Space. In continuity, a portal, www.gouvernement-ouvert.ma, has been launched to consolidate all news related to the open government initiative in Morocco. This platform facilitates the transparent publication of the state of implementation of the national open government action plan by public administrations, provides a means to monitor its implementation, and offers digital spaces for citizens to engage and participate in co-creating open government action plans by submitting ideas, proposals, and contributions.

  Numéro 20, Octobre 2023 computing) that have the ability to transform relations with citizens, businesses, and other arms of government. These technologies can serve a variety of different purposes: better delivery of government services to citizens, improved interactions with businesses and industry, empowering citizens through access to information, or more efficient government management. The resulting benefits can include reduced corruption, increased transparency, greater convenience, increased revenue, and/or cost reductions." The rapid development of e-Government in the 1990s and 2000s generated a wave of optimism regarding citizen participation through information and communication technologies (ICTs), but so far, these techno-deterministic dreams have proven unfounded, (Tayazime & Moutahaddib, 2021). The terms e-Gov and Digital Gov are often confused by the majority of researchers, necessitating clarification. The OECD (2014) recognized a new stage of maturity in the use of digital technologies by governments, transitioning from electronic administration to digital administration, aiming to open, innovate, and modernize the public sectors. It distinguishes between the terms by providing two clear definitions: (1) e-Government refers to the use of ICTs, including the Internet, by governments as a tool for better administration; (2) digital government, on the other hand, refers to the use of digital technologies as an integral part of government modernization strategies to create public value, as demonstrated by (Tayazime & Moutahaddib 2021).

  Numéro 20, Octobre 2023 In Morocco, the Open Data portal (www.data.gov.ma) includes a dedicated section for eparticipation. Users can submit or access requests for datasets. Additionally, they can interact with available requests by commenting on or supporting them, which increases their visibility. parking zone coordinates, and lists of textbooks approved by the Ministry of National Education (MNE).

Figure N°4 :

 N°4 Figure N°4: The relationship between e-participation and other dimensions of governance

  the first year of the portal's deployment. This clearly illustrates the dynamic environment created around the portal.

Figure 1 :

 1 Figure 1: Distribution of Datasets on the Open Data Portal by theme as indicated by Open data Maroc portal

  Directory theme: 5 requests • Economy and finance theme: 3 requests • Agriculture theme: 1 request • Research and development theme: 2 requests • Justice theme: 1 request As indicated in the analysis of the evolution of the number of data producers, the e-participation interface has played a key role in enriching the pool of data producers on the portal.Upon analyzing the graph representing the evolution of the number of data publication requests processed via the e-participation interface, it can be observed that the requests have remained relatively stable over the three reporting periods, with a potential slight increase predicted towards the end of the year 2023. In comparison to other supply KPIs that have experienced

Figure 2 :Figure 4 :Figure 5 :

 245 Figure 2: Evolution of the number of datasets fed on the OD Portal

Figure 6 :

 6 Figure 6: Evolution of the number of visitors to the Data Portal

Figure 7 :

 7 Figure 7: Evolution of number of page views on the OD

Figure 8 :

 8 Figure 8: Evolution of the number of downloads on the OD portal

Figure 9 :

 9 Figure 9: Evolution of number of countries of visitor origin

  the Pearson correlation coefficient is primarily based on the accuracy criterion associated with the KPI values, which are of a technical nature, obtained from the reporting extracted from the hosting platform of the OD portal, having undergone no degree of estimation or aberration. In light of this observation, the selection of the Pearson correlation coefficient (𝑟) remains a logical choice for determining the intensity of the linear correlation between two indicators arising from the studied variables: provision and use of OD.

-

  The number of added themes and the number of visitors to the OD portal;

  Hypothesis 3 (H3): "The evolving needs of future users of Open Data Maroc are gradually fed through portal utilization." According to Table 5, the correlation coefficient (r) indicates positive and satisfactory values of 0.966, 0.999, and 0.947, which are close to the value of 1.

  , we aimed to analyze the implementation, feeding, and usage of Open Data Morocco, focusing on its underlying factors and outcomes. We observed that the feeding function of the Open Data Morocco portal (linked to its implementation) had a direct positive impact on its utilization by users.Considered a lever for establishing a new system of public governance, free and open access to data produced by public institutions and organizations via a National Portal remains a tool recently discovered by Moroccan users from various categories: citizens, civil society, businesses, professionals, researchers, etc. Over a period spanning three fiscal years: 2021, 2022, and 2023 (up to the first semester), the implementation and feeding of the Open Data portal had a very positive impact on the demand and utilization of public open data. The evolution of these two parameters demonstrates a growing trend in user needs that are potentially evolving, quantitatively expressed in previous KPI analyses, and qualitatively expressed through the expansion of published data themes on the portal and through prospective axes for the development of the Open Data portal and its national and international perspectives.However, the challenges to be addressed are of various dimensions. Institutionally, they include the avoidance culture of certain government institutions hesitant to open public data, the cumbersome procedure for accessing and reusing data, and the lack of consideration of user ideas in governmental administration. On the user level, challenges encompass the lack of advanced research facilities, user support facilities, and user expertise to analyze data.

----

  The predominance of a closed government culture and the lack of an open public data policy;Insufficient and inaccurate data availability, including obsolete and invalid data; -Difficulty in searching and navigating data due to lack of metadata or an index, complex data formats, available datasets' complexity, information overload, and a lack of open data usage manuals; -Absence of a national policy standardizing open data management; -Lack of metadata standards and absence of standardized software for open data processing. Other obstacles are faced by both data publishers and open data users, including: Technical unawareness of metadata quality among portal owners, leading to inappropriate metadata publication that hinders users from finding relevant data for their needs; Geospatial data has its specific barriers due to the use of different standards compared to other types of open data, requiring technical knowledge and expertise.

African Scientific Journal ISSN : 2658-9311 Vol : 3, Numéro 20, Octobre 2023 Tableau N°1: Typology of Governmental Data Administrative Data Administrative

  divisions, addresses, public buildings, agendas, etc.

	), data are considered "open" if
	anyone can use, reuse and redistribute them freely, free of charge, for any purpose and without
	restriction. Many data are published on the websites of public administrations, but most of them
	are only intended to be consulted in isolation, and are not reusable for other purposes. Open
	data must be reusable, that is, downloadable in an open and software-readable format, and users
	must be legally authorized to reuse it.(Hassan, 2021) summarized in his study the principles of
	Open Data. These data must be complete, primary, timely, accessible and machine-readable.
	They must also be non-discriminatory, non-proprietary and license-free. In addition, public
	institutions must publish all the data they have as long as they do not violate security,
	confidentiality or other legitimate restrictions. According to Larroche, V., Peyrelong, M. F., &
	Beaune, P. (2018) the Open data is considered a good shared by users shared by users, but only
	as a commodity made available to all by local authorities.

1.1.3 Legal Framework of Data By

  its basic definition, data is considered "open" only if it meets two major criteria:1. Technically open: available in a format standardized and exploitable by a machine,

	accessible online, and not provided in a modifiable electronic format or through a
	programming interface (API);
	2. Legally open: possessing a licensing agreement that permits both commercial and non-
	commercial use and unrestricted reuse. This implies that anyone can utilize, reuse, and
	redistribute these data freely.
	The non-profit organization "Creative Commons" has established several internationally
	recognized usage licenses, totaling seven, known as the Creative Commons Licenses as shown
	in

table 2 .

 2 These licenses formalize the release of data at the time of distribution, defining various levels of freedom for reuse, which are in effect today.

originate African Scientific Journal ISSN : 2658-9311 Vol : 3, Numéro 20, Octobre 2023 complexity

	Research Construct	Variable /item	Description	Current value
		Item 1	Number of datasets uploaded on the OD portal	412
	Item 2 approach allows us to highlight the relationships between the supply/provision and usage of Number of added themes 19
	Open Data through tables and graphical representations. Moreover, it enables us to analyze the
	Item 3 interconnections forming our research problem, leading to precise scientific conclusions and Number of added resources 464
	OD Implementation & Provision insights into the prospects of Open Data development in Morocco. Item 4 Number of data producers added	36
		Item 5	Number of data publication requests processed and fulfilled via the e-participation interface	14
		Item 6	Portal availability rate of www.data.gov.ma	99.98%
		Item 1	Number of visitors on the www.data.gov.ma portal	128.000
	OD Use	Item 2	Number of downloads from the www.data.gov.ma portal	52.477
		Item 4	Number of pages viewed	690.000
		Item 5	Number of countries from which visitors

highlights that "the use of a case study methodology is justified by the of the studied problem. It involves a spatial and temporal analysis of a complex phenomenon, considering conditions, events, actors, and implications," necessitating a comprehensive documentary research. In this context, our research work involves an exploratory analysis of quantitative data, leveraging Key Performance Indicators (KPIs) extracted from activity reports on the platform and interviews with experts from the DDA. This

Table 1 :Table of correlation calculations between KPIs for data supply and usage.

 1

Source: Authors' calculations, from the evolution data of Key KPIs on Open Data Morocco portal (2023) over the study period

Scientific Journal ISSN : 2658-9311 Vol : 3, Numéro 20, Octobre 2023

					African
					Agriculture
	4% 1% 1% 2%	3% 1% 1% 4% 8% 13% 0% 1%	18%	Insurance Mapping Culture Economy and Finance Education Employment Equipement Housing and Urbanism IT Telecom Justice
		13%	4%	Media Standardization
		5%	6% 0% 15%	Maritime Fisheries Research & Development
					Health
					Society
					Tourism
					Transparency

Table 2 : Examination of results for formulated hypotheses Source: compiled

 2 by the authors, based on reported KPIs for Open Data provided by DDA 2.

2. Results of Qualitative Studies

Table 3 : Interview results by theme compiled by the authors Open data aspect Interview Theme Content of results

 3 Expansion of the national Open Data Steering Committee to involve any other organization, institution, or expert in the field that can enrich the portal with content or successful experience; -Development of common data repositories and their periodic update: geographical repositories, addresses, activity nomenclature, etc.; -Implementation of communication and awareness actions tailored to different Open Data targets (public, private actors, civil society, etc.).

	Governance & Steering Aspects -Functional aspects 1. Open Data Steering Committee; 2. Data Governance; 3. Communication and awareness around the use of the Open Data portal 1. Development of new features on the portal: -Creation of a dedicated interface for reuse, allowing users to query the exploitation of www.Data.gov.ma; downloaded data from the portal and analyze their
		2. Enhancement of budget	reuse;
		transparency	-Development and adoption in consultation with
			the Moroccan open government portal, of a
			permanent consultation mechanism to involve
			citizens in the preparation of the citizen's budget.
		1. Allocation of necessary	-Creation of a Data Manager position within each
	Organizational	resources for the proper	organization;
	aspects	functioning of the portal;	-Development of necessary procedure manuals:
		2. Formalization of	Open Data Manager, Data Manager, data
		operational data	inventory.
		management proceedings
		1. Technical assistance to	-Implementation of technical support for relevant
	Aspects Techniques	public institutions and	public institutions and bodies;
		bodies;	-Identification of technical standards (format,
		2. Deployment of new	metadata, etc.) to be established for the publication
		formats for published data;	of open data, in accordance with international
		3. Technical standardization	standards.
	Legal & Regulatory	1. Expansion of the legal	-Strengthening and development of the legal
	Aspects	framework;	framework related to the opening and classification
		2. Right of Access to	of public data;
		Information at the level of	-Implementation of the network of persons
		ministries and public	responsible for access to information, as a space
		institutions	for sharing experiences

Source: compiled by the authors

African Scientific Journal ISSN : 2658-9311 Vol : 3, Numéro 20, Octobre 2023

Wacheux,F. (1993). L'utilisation de l'étude de cas dans l'analyse dans les situations de travail : Propositions méthodologique constructiviste, (p.128).

World Bank. ( 2015