Pengfei Liu
email: pengfei.liu@eric.univ-lyon2.fr

Sabine Loudcher
email: sabine.loudcher@univ-lyon2.fr

Jérôme Darmont
email: jerome.darmont@univ-lyon2.fr

Camille Noûs
email: camille.nous@cogitamus.fr

ArchaeoDAL: A Data Lake for Archaeological Data Management and Analytics

Keywords: Computer systems organization → Data flow architectures;, Information systems → Data warehouses Data lake architecture, Data lake implementation, Metadata management, Archaeological data, Thesaurus

With new emerging technologies, such as satellites and drones, archaeologists collect data over large areas. However, it becomes difficult to process such data in time. Archaeological data also have many different formats (images, texts, sensor data) and can be structured, semi-structured and unstructured. Such variety makes data difficult to collect, store, manage, search and analyze effectively. A few approaches have been proposed, but none of them covers the full data lifecycle nor provides an efficient data management system. Hence, we propose the use of a data lake to provide centralized data stores to host heterogeneous data, as well as tools for data quality checking, cleaning, transformation and analysis. In this paper, we propose a generic, flexible and complete data lake architecture. Our metadata management system exploits goldMEDAL, which is the most generic metadata model currently available. Finally, we detail the concrete implementation of this architecture dedicated to an archaeological project.

INTRODUCTION

Over the past decade, new forms of data such as geospatial data and aerial photography have been included in archaeology research [START_REF] Mccoy | Geospatial Big Data and archaeology: Prospects and problems too great to ignore[END_REF], leading to new challenges such as storing massive, heterogeneous data, high-performance data processing and data governance [START_REF] Gattiglia | Think big about data: Archaeology and the Big Data challenge[END_REF]. As a result, archaeologists need a platform that can host, process, analyze and share such data.

In this context, a multidisciplinary consortium of archaeologists and computer scientists proposed the HyperThesau project 1 , which aims at designing a data management and analysis platform. Hyper-Thesau has two main objectives: 1) the design and implementation of an integrated platform to host, search, analyze and share archaeological data; 2) the design of an archaeological thesaurus taking the whole data lifecycle into account, from data creation to publication. Classical data management solutions, i.e., databases or data warehouses, only manage previously modeled structured data (schemaon-write approach). However, archaeologists need to store data of all formats and they may discover the use of data over time. Hence, we propose the use of a data lake [START_REF] Dixon | Pentaho, Hadoop, and Data Lakes[END_REF], i.e., a scalable, fully integrated platform that can collect, store, clean, transform and analyze data of all types, while retaining their original formats, with no predefined structure (schema-on-read approach). Our data lake, named ArchaeoDAL, provides centralized storage for heterogeneous data and data quality checking, cleaning, transformation and analysis tools. Moreover, by including machine learning frameworks into ArchaeoDAL, we can achieve descriptive and predictive analyses.

Many existing data lake solutions provide architecture and/or implementation, but few include a metadata management system, which is nevertheless essential to avoid building a so-called data swamp, i.e., an unexploitable data lake [START_REF] Inmon | Data Lake Architecture: Designing the Data Lake and Avoiding the Garbage Dump[END_REF][START_REF] Pegdwendé | Metadata Systems for Data Lakes: Models and Features[END_REF]. Moreover, none of the existing metadata management systems can provide all the needed metadata features we need. For example, in archaeology, thesauri are often used for organizing and searching data. Therefore, the metadata system must allow users to define one or more thesauri, associate data with specific terms and create relations between terms, e.g., synonyms and antonyms. Thus, we conclude that existing data lake architectures, including metadata systems, are not generic, flexible and complete enough for our purpose.

To address these problems, we propose in this paper a generic, flexible and complete data lake architecture. Moreover, our metadata model exploits and enriches goldMEDAL, which is the most generic metadata model currently available [START_REF] Scholly | Coining goldMEDAL: A New Contribution to Data Lake Generic Metadata Modeling[END_REF]. To illustrate the flexibility and completeness of ArchaeoDAL's architecture, we provide a concrete implementation dedicated to the HyperThesau project. With a fully integrated metadata management and security system, we can not only ensure data security, but also track all data transformations.

The remainder of this paper is organized as follows. In Section 2, we review and discuss existing data lake architectures. In Section 3, we present ArchaeoDAL's abstract architecture, implementation and deployment. In Section 4, we present two archaeological application examples. In Section 5, we finally conclude this paper and present future works.

DATA LAKE ARCHITECTURES

The concept of data lake was first introduced by Dixon [START_REF] Dixon | Pentaho, Hadoop, and Data Lakes[END_REF] in association with the Hadoop file system, which can host large heterogeneous data sets without any predefined schema. Soon after, the data lake concept was quickly adopted [START_REF] Fang | Managing data lakes in big data era: What's a data lake and why has it became popular in data management ecosystem[END_REF][START_REF] Daniel | Embedding AI and Crowdsourcing in the Big Data Lake[END_REF]. With the growing popularity of data lakes, many solutions were proposed. After studying them, we divide data lake architectures into two categories: 1) data storage-centric architecture; 2) data storage and processing-centric architecture.

Data Storage-Centric Architectures

In the early days, a data lake was viewed as a central, physical storage repository for any type of raw data, aiming for future insight. In this line, Inmon proposes an architecture that organizes data by formats, in so-called data ponds [START_REF] Inmon | Data Lake Architecture: Designing the Data Lake and Avoiding the Garbage Dump[END_REF]. The raw data pond is the place where data first enters the lake. The analog data pond stores data generated by sensors or machines. The application data pond stores data generated by applications. Finally, the textual data pond stores unstructured, textual data.

Based on such zone solutions, Gorelik proposes that a common data lake architecture includes four zones [START_REF] Gorelik | Architecting the Data Lake[END_REF]: a landing zone that hosts raw ingested data; a gold zone that hosts cleansed and enriched data; a work zone that hosts transformed, structured data for analysis; and a sensitive zone that hosts confidential data. Bird also proposes a similar architecture [START_REF] Bird | Architecture and prototype of a WLCG data lake for HL-LHC[END_REF]. Such architectures organize data with respect to how deeply data are processed and security levels.

The advantage of storage-centric architectures is that they provide a way to organize data inside a data lake by default. However, the predefined data organization may not satisfy the requirements of all projects. For example, HyperThesau needs to store data from different research entities. Thus, one requirement is to organize data by research entities first. Yet, the bigger problem of storagecentric architectures is that they omit important parts of a data lake, e.g., data processing, metadata management, etc.

Data Storage and Processing-Centric Architectures

With the evolution of data lakes, they have been viewed as platforms, resulting in more complete architectures. Alrehamy and Walker propose a "Personal Data Lake" architecture that consists of five components [START_REF] Walker | Personal Data Lake with Data Gravity Pull[END_REF], which addresses data ingestion and metadata management issues. However, it transforms data into a special JSON object that stores both data and metadata. By changing the original data format, this solution contradicts the data lake philosophy of conserving original data formats. Pankaj and Tomcy propose a data lake architecture based on the Lambda architecture (Figure 1) that covers almost all key stages of the data life cycle [START_REF] Tomcy | Lambda Architecture-driven Data Lake[END_REF]. However, it omits metadata management and security issues. Moreover, not all data lakes need near real-time data processing capacities. Mehmood et al. propose an interesting architecture consisting of four layers: a data ingestion layer that acquires data for storage; a data storage layer; a data exploration and analysis layer; a data visualization layer [START_REF] Mehmood | Implementing Big Data Lake for Heterogeneous Data Sources[END_REF]. This architecture is close to our definition of a data lake, i.e., a fully integrated platform to collect, store, transform and analyze data for knowledge extraction. Moreover, Mehmood et al. provide an implementation of their architecture. However, although they mention the importance of metadata management, they do not include a metadata system in their architecture. Eventually, data security is not addressed and data visualization is the only proposed analysis method. Raju et al. propose an architecture that is similar to Mehmood et al. 's [START_REF] Raju | Data Lake Architecture for Air Traffic Management[END_REF]. They essentially use a different tool-set to implement their approach and also omit to take metadata management and data security into account.

Discussion

In our opinion, a data lake architecture must be generic, flexible and complete. Genericity implies that the architecture must not rely on any specific tools nor frameworks. Flexibility means that users must be able to define their own ways of organizing data. Completeness means that not only functional features (e.g., ingestion, storage, analysis, etc.) must be handled, but also non-functional features (e.g., data governance and data security). Table 1 provides an evaluation of seven data lake architectures with respect to these three properties.

The solutions by Mehmood et al. and Raju et al. are not generic because their architecture heavily relies on certain tools. The zone architectures by Inmon, Gorelik and Bird are not flexible, because they force a specific data organization. Finally, Alrehamy and Walker's platform is the only complete architecture that addresses data governance and security, but is not a canonical data lake.

ARCHAEODAL'S ARCHITECTURE, IMPLEMENTATION AND DEPLOYMENT

In this section, we propose a generic, flexible abstract data lake architecture that covers the full data lifecycle (Figure 2) and contains eleven layers. The orange layers (from layer 1 to layer 6) cover the full data lifecycle. After data processing in these layers, data become clearer and easier to use for end-users. The yellow layers cover non-functional requirements. After the definition of each layer, we present how each layer is implemented in our current ArchaeoDAL instance. As this instance is dedicated to the project HyperThesau, it does not cover all the features of the abstract architecture. For example, real-time data ingestion and processing are not implemented. However,real-time or near real-time data ingestion and processing feature can be achieved by adding tools such as Apache Storm2 or Apache Kafka3 in the data ingestion and data insights layers.

Data Source Layer

In the data source layer, we gather the basic properties of data sources, e.g., volume, format, velocity, connectivity, etc. Based on these properties, data engineers can determine how to import data into the lake. If metadata are required, data engineers must also find the best fitting metadata model to govern input data.

In our instance of ArchaeoDAL, we have data sources such as relational databases and various files stored in the archaeologists' personal computers.

Data Ingestion Layer

The data ingestion layer provides a set of tools that allow users to perform batch or real-time data ingestion. Based on the data source properties that are gathered in the data source layer, data engineers can choose the right tools and plans to ingest data into the lake. They must also consider the capacity of the data lake to avoid data loss, especially for real-time data ingestion. During ingestion, metadata provided by the data sources, e.g., the name of excavation sites or instruments, must be gathered as much as possible. After data are loaded into the lake, we may lose track of data sources. It is indeed more difficult to gather this kind of metadata without knowledge about data sources.

ArchaeoDAL's implementation exploits Apache Sqoop4 to ingest structured data and Apache Flume5 to ingest semi-structured and unstructured data. Apache Sqoop can efficiently transfer bulk data from structured data stores such as relational databases. Apache Flume is a distributed service for efficiently collecting, aggregating and moving large amounts of data. For one-time data ingestion, our instance provides Sqoop scripts and a web interface to ingest bulk data. For repeated data loading, we developed Flume agents to achieve automated data ingestion.

Data Storage Layer

The data storage layer is the core layer of a data lake. It must have the capacity to store all data, e.g., structured, semi-structured and unstructured data, in any format.

ArchaeoDAL's implementation uses the Hadoop Distributed File System6 (HDFS) to store ingested data, because HDFS stores data on commodity machines and provides horizontal scalability and fault tolerance. As a result, we do not need to build large clusters. We just add nodes when data volume grows. To better support the storage of structured and semi-structured data, we add two tools: Apache Hive7 to store data with explicit data structures and Apache HBase8 that is a distributed, versioned, column-oriented database that provides better semi-structured data retrieval speed.

Data Distillation Layer

The data distillation layer provides a set of tools for data cleaning and encoding formalization. Data cleaning refers to eliminating errors such as duplicates and type violations, e.g., a numeric column contains non-numeric values. Data encoding formalization refers to converting various data and character encoding, e.g., ASCII, ISO-8859-15, or Latin-1, into a unified encoding, e.g., UTF-8, which covers all language symbols and graphic characters.

ArchaeoDAL's implementation uses Apache Spark9 to clean and transform data. We developed a set of Spark programs that can detect duplicates, NULL values and type violations. Based on the percentage of detected errors, we can hint at data quality.

Data Insights Layer

The data insights layer provides a set of tools for data transformation and exploratory analysis. Data transformation refers to transforming data from one or diverse sources into specific models that are ready to use for data application, e.g., reporting and visualization. Exploratory data analysis refers to the process of performing initial investigations on data to discover patterns, test hypotheses, eliminate meaningless columns, etc. Transformed data may also be persisted in the data storage layer for later reuse.

ArchaeoDAL's implementation resorts to Apache Spark to perform data transformation and exploratory data analysis. Spark also provides machine learning libraries that allow developers to perform more sophisticated exploratory data analyses.

Data Application Layer

The data application layer provides applications that allow users to extract value from data. For example, a data lake may provide an interactive query system to do descriptive and predictive analytics. It may also provide tools to produce reports and visualize data.

In ArchaeoDAL's implementation, we use a Web-based notebook, Apache Zeppelin10 , as the front end. Zeppelin connects to the data analytics engine Apache Spark that can run a complex directed acyclic graph of tasks for processing data. Our notebook interface supports various languages and their associated Application Programming Interfaces (APIs), e.g., R, Python, Java, Scala and SQL. It provides a default data visualization system that can be enriched by Python or R libraries.

ArchaeoDAL also provides a web interface that helps users download, upload or delete data.

Data Governance Layer

The data governance layer provides a set of tools to establish and execute plans and programs for data quality control [START_REF] Khatri | Designing data governance[END_REF]. This layer is closely linked to the data storage, ingestion, distillation, insights and application layers to capture all relevant metadata. A key component of the data governance layer is a metadata model [START_REF] Pegdwendé | Metadata Systems for Data Lakes: Models and Features[END_REF].

Metadata Model.

In ArchaeoDAL, we adopt goldMEDAL [START_REF] Scholly | Coining goldMEDAL: A New Contribution to Data Lake Generic Metadata Modeling[END_REF], which is modeled at the conceptual (formal description), logical (graph model) and physical (various implementations) levels. goldMEDAL features four main metadata concepts (Figure 3): 1) data entities, i.e., basic data units such as spreadsheet tables or textual documents; 2) groupings that bring together data entities w.r.t. common properties in groups; 3) links that associate either data entities or groups with each other; and 4) processes, i.e., transformations applied to data entities that produce new data entities. All concepts bear metadata, which make goldMEDAL the most generic metadata model in the literature, to the best of our knowledge. However, we encountered a problem of terminology variation when creating metadata. goldMEDAL does indeed not provide explicit guidance for metadata creation. This can lead to consistency and efficiency problems. For example, when users create data entities, they have their own way of defining attributes, i.e., key/value pairs that describe the basic properties of data. Without a universal guideline or template, every user can invent his own way. The number, name and type of attributes can be different. As a result, without explicit guidelines or templates, it may become quite difficult to retrieve or search metadata.

Thus, we enrich goldMEDAL with a new concept, data entity type, which explicitly defines the number, names and types of the attributes in a data entity.

All data entity types form a type system that specifies how metadata describe data inside the lake. Since each and every data lake has specific requirements on how to represent data to fulfill domainspecific requirements, metadata must contain adequate attributes. Thus, we need to design a domain-specific type system for each domain. For example, in the HyperThesau project, users need not only semantic metadata to understand the content of data, but also geographical metadata to know where archaeological objects are discovered. As a result, the type system is quite different from other domains.

In sum, the benefits of having a data entity type system include: 1) consistency, a universal definition of metadata can avoid terminology variations that may cause data retrieval problems; 2) flexibility, a domain-specific type system helps define specific metadata for requirements in each use case; 3) efficiency, with a given metadata type system, it is easy to write and implement search queries. Because we know in advance the names and types of all metadata attributes, we can filter data with metadata predicates such as 𝑢𝑝𝑙𝑜𝑎𝑑_𝑑𝑎𝑡𝑒 > 10/02/2016.

Thesaurus Modeling.

Although thesauri, ontologies and taxonomies can definitely be modeled with goldMEDAL, its specifications do not provide details on how to conceptually and logically model such semantic resources, while we especially need to manage thesauri in the HyperThesau project.

A thesaurus consists of a set of categories and terms that help regroup data. A category may have one and only one parent. A category without a parent is called the root category. A category may have one or more children. The child of a category is called subcategory or term. A term is a special type of category that has no child but must have a parent category. A term may have relationships with other terms (related words, synonyms, antonyms, etc.).

Fortunately, categories and terms can easily be modeled as data entities and structured with labeled links, with labels defined as link metadata. It is also easy to extend this thesaurus model to represent ontologies or taxonomies.

3.7.

3 Data Governance Implementation. The HyperThesau project does not require sophisticated data governance tools to fix decision domains and decide who takes decisions to ensure effective data management [START_REF] Khatri | Designing data governance[END_REF]. Thus, ArchaeoDAL's data governance layer implementation only focuses on how to use metadata to govern data inside the lake. We use Apache Atlas 11 , which is a data governance and metadata management framework, to implement our extended version of goldMEDAL (Section 3.7.1). With these metadata, we build a data catalog that allows searching and filtering data through different metadata attributes, organize data with user-defined classifications and the thesaurus and trace data lineage.

Data Security Layer

The data security layer provides tools to ensure data security. It should ensure the user's authenticity, data confidentiality and integrity.

ArchaeoDAL's implementation orchestrates more than twenty tools and frameworks, most of which have their own authentication system. If we used their default authentication systems, a given user could have twenty login and password pairs. But even if a user decided to use the same login and password for all services, s/he would have to change it twenty times in case of need. To avoid this, we have deployed an OpenLDAP server as a centralized authentication server. All services connect to the centralized authentication server to check user login and password. The access control system consists of two parts. First, we need to control the access to each service. For example, when a user wants to create a new thesaurus, Atlas needs to check whether this user has the right to. Second, we need to control the access to data in the lake. A user may access data via different tools and the authorization answer of these tools must be uniform. We use Apache Ranger 12 to set security policy rules for each service. For data access control, we implement a role-based access control (RBAC) system by using the Hadoop group mapping service.

Data security is enforced by combining the three systems. For example, a user wants to view data from a table stored in Hive. S/he uses his/her login and password to connect to the Zeppelin notebook. This login and password are checked by the OpenLDAP server. After login, s/he runs a SQL query that is then submitted to Hive. Before query execution, Hive sends an authorization request to Ranger. If Ranger allows the user to run the query, it starts and retrieves data with the associated user credentials. Then, HDFS checks whether the associated user credentials have the right to access data. If a user does not have the right to read data, an access denied exception is produced.

Workflow Manager Layer

The workflow manager layer provides tools to automate the flow of data processes. This layer is optional.

For now, we have not identified any repeated data processing tasks in the HyperThesau project. As a result, we have not implemented this layer. However, it can easily be implemented by integrating tools such as Apache Airflow 13 or Apache NiFi14 .

Resource Manager Layer

As data lakes may involve many servers working together, the resource manager layer is responsible for negotiating resources and scheduling tasks on each server. This layer is optional, because a reasonably small data lake may be implemented on one server only.

As ArchaeoDAL rests on a distributed storage and computation framework, a resource manager layer is mandatory. We use YARN 15 , since this resource manager can not only manage the resources in a cluster, but also schedule jobs.

Communication Layer

The communication layer provides tools that allow other layers, e.g., data application, data security and data governance, to communicate with each other. It must provide both synchronous and asynchronous communication capability. For example, a data transformation generates new metadata that are registered by the data governance layer. Metadata registration should not block the data transformation process. Therefore, the data insights and governance layers require asynchronous communication. However, when a user wants to visualize data, the data application and security layers require synchronous communication, since it is not desirable to have a user read data before the security layer authorizes the access.

ArchaeoDAL's implementation uses Apache Kafka, which provides a unified, high-throughput, low-latency platform for handling real-time data feeds. Kafka can connect by default many frameworks and tools, e.g., Sqoop, Flume, Hive and Spark. It also provides both synchronous and asynchronous communication.

ArchaeoDAL's Deployment

We have deployed ArchaeoDAL's implementation in a self-hosted cloud. The current platform is a cluster containing 6 virtual machines, each having 4 virtual cores, 16 GB of RAM and 1 TB of disk space. We use Ambari16 to manage and monitor the virtual machines and installed tools. The current platform allows users to ingest, store, clean, transform, analyze, visualize and share data by using different tools and frameworks.

ArchaeodAL already hosts the data of two archaeological research facilities, i.e., Bibracte 17 and Artefacts 18 . Artefacts currently amounts to 20,475 records and 180,478 inventoried archaeological objects. Bibracte currently contains 114 excavation site reports that contain 30,106 excavation units and 83,328 inventoried objects. We have imported a thesaurus developed by our partner researchers from the Maison de l'Orient et de la Méditerranée19 , who study ancient societies in all their aspects, from prehistory to the medieval world, in the Mediterranean countries, the Near and Middle-East territories. This thesaurus implements the ISO-25964 norm. A dedicated thesaurus by the linguistic expert of the HyperThesau project is also in the pipe. With the help of our metadata management system, users can associate data with the imported thesaurus, and our search engine allows users to search and filter data based on the thesaurus.

APPLICATION EXAMPLES

In this section, we illustrate how ArchaeoDAL supports users throughout the archaeological data lifecycle, via metadata. We also demonstrate the flexibility and completeness of ArchaeoDAL (Section 2.3).

Heterogeneous Archaeological Data Analysis

In this first example, we show how to analyze heterogeneous archaeological data, how to generate relevant metadata during data ingestion and transformation, and how to organize data flexibly via the metadata management system. The Artefacts dataset consists of a SQL database of 32 tables and a set of files that stores detailed object descriptions as semi-structured data. This dataset inventories 180,478 objects. The data management system is implemented with Apache Atlas (Section 3.7.3) and provides three ways to ingest metadata: 1) pre-coded atlas hook (script), 2) self-developed atlas hook and 3) REpresentational State Transfer (REST) API.

Structured Data Ingestion and Metadata Generation.

To import data from a SQL database, we use a hook dedicated to Sqoop that can generate the metadata of the imported data and ingest them automatically. A new database is created in ArchaeoDAL and its metadata is generated and inserted into the Atlas instance (Figure 4). Figure 5 shows an example of table metadata inside Artefacts.

4.1.2 Semi-structured Data Ingestion and Metadata Generation. We provide three ways to ingest semi-structured data. The simplest way is to use Ambari's Web interface (Figure 6). The second way is to use the HDFS command-line client.

The third way is to use a data ingestion tool. ArchaeoDAL provides a tool called Flume. The Flume agent can monitor any file system of any computer. When a file is created on the monitored file system, the Flume agent will upload it to ArchaeoDAL automatically.

Now that we have uploaded the required files into ArchaeoDAL, we need to generate and insert the metadata into Atlas. Since Atlas does not provide a hook for HDFS, we develop our own20 . This hook is triggered by the HDFS create, update and delete events. For example, the upload action generates a file creation event in HDFS, which in turn triggers our metadata generation hook (Listing 1).

After the hook has uploaded metadata into Atlas, it can be visualized (Figure 7).

1 { " entities " : [ { 2 " typeName " : " hdfs_path " , 3 " createdBy " : " pliu " , 4 " attributes " : { 5 " qualifiedName " : " hdfs : // lin 0 2 . udl . org : 9 0 0 0 / HyperThesau / Artefacts / object -1 6 8 . txt " ,

6

" name " : " object -1 6 8 . txt " , 7 " path " : " hdfs : // lin 0 2 . udl . org : 9 0 0 0 / HyperThesau / Artefacts " , 8 " user " : " pliu " , 9 " group " : " artefacts " , 10 " creation_time " : " 2 0 2 0 -1 2 -2 9 " , 11 " owner " : " pliu " , 12 " numberOfReplicas " : 0 , We also developed an Atlas API in Python21 to allow data engineers to generate and insert metadata into Atlas more easily. As Amazon S3 is the most popular cloud storage, we also developed an Atlas S3 hook22 .

Data Transformation and Metadata

Generation for Data Lineage Tracing. Once Artefacts data are ingested into ArchaeoDAL, let us imagine that an archaeologist wants to link the detailed description of objects (stored in table objects) with their discovery and storage locations. The first step is to convert the semi-structured object descriptions into structured data. We developed a simple Spark Extract, Transform and Load (ETL) script for this sake. Then, we save the output structured data into Hive tables location (discovery location) and musee (the museum where objects are stored).

Eventually, we join the three tables into a new table called ob-jects_origin that contains the objects' descriptions and their discovery and storage locations.

Thereafter, objects_origin's metadata can be gathered into Atlas with the help of the default Hive hook23 and a Spark hook developed by Hortonworks24 . All Hive and Spark data transformations are tracked and all relevant metadata are pushed automatically into Atlas. Figures 8 and9 show table objects_origin's metadata and lineage, respectively. 4.1.4 Flexible Data Organization. As we mentioned in Section 2.3, existing data lake solutions do not allow users to define their own ways of organizing data, while ArchaeoDAL users should. Moreover, ArchaeoDAL must allow multiple data organizations to coexist. For example, let us define four different ways to organize data: 1) by maturity, e.g., raw data vs. enriched data; 2) by provenance, e.g., Artefacts and Bibracte; 3) by confidentiality level, e.g., strictly confidential, restricted or public; and 4) by year of creation. This is achieved through Atlas' classifications, which can group data of the same nature. Moreover, data can be associated with multiple classifications, i.e., be in different data groups at the same time. Figure 10 shows the implementation of the above data organization in Atlas. We associate table object_origin with four classifications (i.e. enriched, Artefacts, confidential, 2020). With table object_origin belonging to four classifications, if we want to filter data by maturity, we click on the enriched classification to find the table. In sum, classifications allow users to organize data easily and flexibly.

Data Indexing and Search through thesauri

As mentioned in Section 3.7.2, thesauri are important metadata for project HyperThesau. As an example, we import a thesaurus This thesaurus is mainly used to index the inventoried archaeological objects of Artefacts. Its basic building blocks are terms that can be grouped by categories (Figure 11).

We can associate any data entity with any term. A data entity can be associated with multiple terms. After we index a data entity with a term, we can search data by using the terms of a thesaurus. For example, we have a database table called bibliographie and a file called 204docannexe.csv that contains information about shields. Suppose we need to associate these two data entities with the term bouclier (shield in French). After indexing, we can click on the term bouclier to find all data associated with this term (Figure 12). 4.2.1 Data Indexing with Multiple thesauri. One of the biggest challenges of project HyperThesau is that each research facility uses its own thesaurus. Moreover, there is no standard thesaurus. Thus, if we index data with one given thesaurus, archaeologists using another one cannot use ArchaeoDAL. To overcome this challenge, ArchaeoDAL supports multiple thesauri. Moreover, we can define relations, e.g., synonyms, antonyms, or related terms, between terms of different thesauri. For example, Figure 13 shows a term from a Chinese thesaurus that we set as the synonym of the term bouclier. As a result, even though the Chinese term does not relate to any data directly, by using the relations, we can find terms that are linked to actual data. A full video demo of this example can be found online 25 .

CONCLUSION AND FUTURE WORKS

In this article, we first introduce the need of archaeologists for software platforms that can host, process and share new, voluminous and heterogeneous archaeological data. Data lakes looking like a viable solution, we examine different existing data lake solutions and conclude that they are not generic, flexible nor complete enough to fulfill project HyperThesau's requirements. layer. Yet, one of our collaborators could easily replace HDFS by Amazon S3. In Section 4.1.4, we demonstrate how to organize data flexibly, which many existing solutions [START_REF] Bird | Architecture and prototype of a WLCG data lake for HL-LHC[END_REF][START_REF] Gorelik | Architecting the Data Lake[END_REF][START_REF] Inmon | Data Lake Architecture: Designing the Data Lake and Avoiding the Garbage Dump[END_REF][START_REF] Walker | Personal Data Lake with Data Gravity Pull[END_REF] do not allow. In Section 4.1.4, we also demonstrate that ArchaeoDAL can gather metadata automatically during the full data lifecycle. Eventually, many features of ArchaeoDAL are very hard to demonstrate in a paper. Thus, we recorded demo videos that are available online26 .

Archaeologists encounter two major problems while working with ArchaeoDAL. First, to associate data and terms in a thesaurus, domain experts are needed. Moreover, this data-terms matching is a very expensive and time-consuming operation. Thus, we plan to use natural language processing techniques to associate data with a thesaurus automatically, calling domain experts only for a posteriori verification.

Figure 1 :

 1 Figure1: Lambda data lake architecture[START_REF] Tomcy | Lambda Architecture-driven Data Lake[END_REF]

Figure 2 :

 2 Figure 2: ArchaeoDAL's architecture

Figure 3 :

 3 Figure 3: goldMEDAL's concepts[START_REF] Scholly | Coining goldMEDAL: A New Contribution to Data Lake Generic Metadata Modeling[END_REF]

13 "Listing 1 :

 131 Sample generated metadata from a HDFS file

Figure 4 :

 4 Figure 4: Database metadata

Figure 5 :

 5 Figure 5: Table metadata

Figure 6 :Figure 7 :

 67 Figure 6: Data upload interface

Figure 8 :

 8 Figure 8: Metadata per se

Figure 9 :

 9 Figure 9: Lineage

Figure 10 :

 10 Figure 10: Sample Atlas classification

Figure 12 :Figure 13 :

 1213 Figure 12: Sample data search through Artefacts' thesaurus

Table 1 :

 1 Comparison of data lake architectures

	Architecture	Generic Flexible Complete
	Alrehamy and Walker (2015)	✓	✓
	Inmon (2016)	✓
	Pankaj and Tomcy (2017)	✓	✓
	Raju et al. (2018)		✓
	Bird (2019)	✓
	Gorelik (2019)	✓
	Mehmood et al. (2019)		✓

https://imu.universite-lyon.fr/projet/hypertheseau-hyper-thesaurus-et-lacs-dedonnees-fouiller-la-ville-et-ses-archives-archeologiques-2018/

https://storm.apache.org/

https://kafka.apache.org/

https://sqoop.apache.org/

https://flume.apache.org/

https://hadoop.apache.org/

https://hive.apache.org/

https://hbase.apache.org/

https://spark.apache.org/

https://zeppelin.apache.org/

https://atlas.apache.org/

https://ranger.apache.org/

https://airflow.apache.org/

https://nifi.apache.org/

https://yarnpkg.com/

https://ambari.apache.org/

http://www.bibracte.fr/

https://artefacts.mom.fr/

https://www.mom.fr/

https://github.com/pengfei99/AtlasHDFSHook

https://pypi.org/project/atlaspyapi/

https://pypi.org/project/atlass3hook/

https://atlas.apache.org/1.2.0/Hook-Hive.html

https://github.com/hortonworks-spark/spark-atlas-connector

https://youtube.com/playlist?list=PLrj4IMV47FypKK5WyEd4Oj3-JnfSuU_H1 Second, we handle a lot of images, e.g., aerial photographs and satellite images. It is also very time consuming to detect useful objects in such images. Although some machine learning tasks can already be performed from ArchaeoDAL via Spark-ML, we would like to use deep learning techniques to assist archaeologists in processing images more efficiently.


Lieven Clarisse
email: lieven.clarisse@ulb.be

Martin Van Damme

Dani Hurtmans

Bruno Franco

Cathy Clerbaux

Pierre-François Coheur

The Diel Cycle of NH 3 Observed From the FY-4A Geostationary Interferometric Infrared Sounder (GIIRS)

We report the first observations of atmospheric NH 3 from the geostationary satellite FY-4A GIIRS over Asia • The potential of GIIRS to reveal the diurnal and nocturnal variations of NH 3 total columns is demonstrated over two sites • To better exploit geostationary data, improved temperature profiles and knowledge on the vertical NH 3 distribution is needed

Introduction

As the atmosphere's primary base and one of the main reactive nitrogen compounds, ammonia (NH 3 ) plays a key role in our atmosphere and the global nitrogen cycle [START_REF] Fowler | The global nitrogen cycle in the twenty-first century[END_REF]. Emitted in excess, NH 3 can lead to a dramatic loss of biodiversity and deterioration of water, soil and air quality [START_REF] Bauer | Significant atmospheric aerosol pollution caused by world food cultivation[END_REF][START_REF] Bobbink | Global assessment of nitrogen deposition effects on terrestrial plant diversity: A synthesis[END_REF][START_REF] Sutton | Our nutrient world: The challenge to produce more food and energy with less pollution[END_REF]. Unlike other primary gaseous pollutants, such as the acidifying NO x and SO 2 [START_REF] Aas | Global and regional trends of atmospheric sulfur[END_REF][START_REF] Georgoulias | Trends and trend reversal detection in 2 decades of tropospheric NO 2 satellite observations[END_REF], concentrations of NH 3 are steadily increasing in large parts of the world [START_REF] Sutton | Alkaline air: Changing perspectives on nitrogen and air pollution in an ammonia-rich world[END_REF][START_REF] Van Damme | Global, regional and national trends of atmospheric ammonia derived from a decadal (2008-2018) satellite record[END_REF]. NH 3 has stayed under the radar for long, partially as a result of the historic dearth of measurements. Fortunately, thanks to the availability of new and cheaper techniques, large measurement networks of ground-based stations are gradually put in place [START_REF] Nair | Quantification of atmospheric ammonia concentrations: A review of its measurement and modeling[END_REF][START_REF] Viatte | Ammonia and PM2.5 air pollution in Paris during the 2020 COVID lockdown[END_REF]. In addition, since over a decade, satellites equipped with high-resolution infrared spectrometers offer an unprecedented temporal and spatial sampling of the global NH 3 distributions, which has led to important progress in our understanding of its local to global emission and deposition fluxes and how these are evolving over time (e.g., [START_REF] Cao | Inverse modeling of NH 3 sources using CrIS remote sensing measurements[END_REF][START_REF] Kharol | Dry deposition of reactive nitrogen from satellite observations of ammonia and nitrogen dioxide over North America[END_REF][START_REF] Van Damme | Industrial and agricultural ammonia point sources exposed[END_REF][START_REF] Warner | Increased atmospheric ammonia over the world's major agricultural areas detected from space[END_REF]. There are currently several polar-orbiting satellites that provide global coverage up to twice a day at spatial resolutions above 10 km. These include IASI at 9.30 a.m./p.m. [START_REF] Whitburn | A flexible and robust neural network IASI-NH 3 retrieval algorithm[END_REF], AIRS [START_REF] Warner | The global tropospheric ammonia distribution as seen in the 13-year AIRS measurement record[END_REF] and CrIS at 1.30 a.m./p.m. [START_REF] Shephard | Ammonia measurements from space with the Cross-track Infrared Sounder: Characteristics and applications[END_REF], and TANSO-FTS [START_REF] Someya | Atmospheric ammonia retrieval from the TANSO-FTS/GOSAT thermal infrared sounder[END_REF] at 1.00 a.m./p.m.

Abstract Limiting excess atmospheric ammonia (NH 3 ) is one of the great challenges for humanity in the 21st century but can only be achieved with adequate monitoring means in place. Here, we report the first NH 3 measurements from the Geostationary Interferometric Infrared Sounder (GIIRS) onboard the Chinese FY-4A satellite. The instrument scans almost all of Asia 10 times per day. We show that GIIRS′ unprecedented temporal sampling can be exploited to measure diurnal and nocturnal variations of NH 3 and demonstrate this on two case studies over Punjab and the North China Plain. Day-night variations are found to be almost absent in winter, but can reach a factor 2-3 in the warmer months. These case studies are very promising given the future landscape of geostationary sounders, but also show that improved knowledge on lower tropospheric air temperatures and the vertical profile of NH 3 is key to better exploit their measurements.

Plain Language Summary

To reduce the worldwide concentrations of ammonia (NH 3 ) in the air is one of the great challenges for humanity in the 21st century, but this can only be achieved with stringent policies in place, and with the support of accurate and frequent measurements. In this paper, we report the first ammonia measurements from the Geostationary Interferometric Infrared Sounder (GIIRS) onboard the Chinese FY-4A satellite. The satellite is positioned above Indonesia and measures almost all of Asia 10 times per day. We show that GIIRS' measurements can be used to measure ammonia both during day and night. We look into more detail at two locations, over Punjab (Pakistan) and the North China Plain. Day-night variations are found to be almost absent in the colder months, but can reach a factor 2-3 in the warmer months. These case studies are very promising given the future landscape of geostationary sounders, but demonstrate also that improved knowledge on the air temperature at the surface and the behavior of ammonia at higher altitudes will result in more accurate ammonia measurements.

The diel (day and night) cycle of NH 3 can provide valuable information on its sources, surface exchanges, deposition, gas-particle partitioning and transport, and how each of these are affected by meteorological conditions [START_REF] Kuang | Explosive morning growth phenomena of NH 3 on the North China Plain: Causes and potential impacts on aerosol formation[END_REF][START_REF] Schulte | Unraveling the diurnal atmospheric ammonia budget of a prototypical convective boundary layer[END_REF]. Its understanding is also critically needed to improve atmospheric models [START_REF] Cao | Inverse modeling of NH 3 sources using CrIS remote sensing measurements[END_REF][START_REF] Lonsdale | Modeling the diurnal variability of agricultural ammonia in Bakersfield, California, during the CalNex campaign[END_REF][START_REF] Zhu | Global evaluation of ammonia bidirectional exchange and livestock diurnal variation schemes[END_REF]. When talking about daily variations, it is important to differentiate between emissions, surface concentrations and total columns. (i) Emissions of NH 3 typically exhibit a maximum during the day, due to increased temperature-driven volatilization [START_REF] Sutton | Toward a climate-dependent paradigm of ammonia emission and deposition[END_REF] and the presence of diurnal sources [START_REF] Chang | The importance of vehicle emissions as a source of atmospheric ammonia in the megacity of Shanghai[END_REF]. (ii) By contrast, there are large variations in the time when the minimum and maximum surface concentrations are reached (see [START_REF] Tevlin | Tall tower vertical profiles and diurnal trends of ammonia in the Colorado Front Range[END_REF] for an overview). A key element that drives their cycle is the vertical mixing that occurs within the planetary boundary layer (PBL) which determines to what extent NH 3 remains trapped at the surface or undergoes transport to higher altitudes (e.g., [START_REF] Saylor | Continuous gaseous and total ammonia measurements from the southeastern aerosol research and characterization (SEARCH) study[END_REF][START_REF] Walker | Measurement of bi-directional ammonia fluxes over soybean using the modified bowen-ratio technique[END_REF]. (iii) Some diurnal measurements of NH 3 total columns are available [START_REF] Dammers | Measuring atmospheric ammonia with remote sensing campaign: Part 1 -Characterisation of vertical ammonia concentration profile in the centre of the Netherlands[END_REF][START_REF] Kutzner | Diurnal evolution of total column and surface atmospheric ammonia in the megacity of Paris, France, during an intense springtime pollution episode[END_REF]W. Wang et al., 2020) from ground-based spectroscopy. These show that the daily evolution of the column can also significantly differ (and even anti-correlate) from the variations at the surface. No measurements of NH 3 total column variations during nighttime have been reported. Furthermore, until now, it is almost uniquely daytime satellite measurements that have been exploited, as nighttime measurements typically come with larger retrieval uncertainties [START_REF] Van Damme | Global distributions, time series and error characterization of atmospheric ammonia (NH 3 ) from IASI satellite observations[END_REF].

The geostationary meteorological satellite FengYun-4 was brought into orbit on 11 December 2016 around the Earth at an altitude of 35,786 km above the equator at 104.7° E. On board it carries several instruments including the Geostationary Interferometric Infrared Sounder (GIIRS) [START_REF] Hua | Review of Geostationary Interferometric Infrared Sounder[END_REF][START_REF] Shang | Instrument observation strategy for a new generation of three-axis-stabilized geostationary meteorological satellites from China[END_REF][START_REF] Yang | Introducing the new generation of Chinese geostationary weather satellites, Fengyun-4[END_REF]. Its coverage includes almost all of Asia, which is scanned 10 times per day (starting at 0, 2, 4, 6, 8, 10, 12, 14, 20, and 22 h UTC) (Q. [START_REF] Zhang | Cloud detection from FY-4A's Geostationary Interferometric Infrared Sounder using machine learning approaches[END_REF]. GIIRS measures the Earth's infrared radiation in two bands at 700-1130 cm -1 and 1650-2250 cm -1 at a sampling rate of 0.625 cm -1 . The noise varies between 0.25 and 0.35 K at 280 K between 900 and 1000 cm -1 . The spatial resolution is 16 km. GIIRS is the first interferometer on board a geostationary satellite and its main goal is the provision of temperature and humidity profiles for improved weather forecasts [START_REF] Cai | Temperature and humidity profile retrieval from FY4-GIIRS hyperspectral data using artificial neural networks[END_REF][START_REF] Smith | Improved severe weather forecasts using LEO and GEO satellite soundings[END_REF]. However, with a spectral resolution and range similar to that of CrIS, GIIRS is in principle capable of measuring trace gases as well, and with its 10 overpasses per day could have the potential of capturing the full diel cycle of NH 3 total columns.

In this paper, we report the first NH 3 observations from GIIRS (Section 2) and subsequently introduce a dataset of GIIRS-derived NH 3 columns (Section 3). In Section 4, we provide monthly climatologies of the daily variations of GIIRS-derived columns, in particular focusing on two distinct locations with year-round high NH 3 columns. We use these case studies also to illustrate dependencies on auxiliary data and sensitivity to retrieval assumptions. We conclude with an outlook (Section 5).

Absorption and Emission

Two example spectra observed over the North China Plain (NCP) are shown in Figure 1. The top panel presents a spectrum (blue) that was measured by GIIRS in an afternoon overpass on a summer's day. This spectrum was fitted with an inverse model (red), to enable visualizing the NH 3 spectral contribution (orange and black), similarly as in [START_REF] Clarisse | Satellite monitoring of ammonia: A case study of the San Joaquin Valley[END_REF]. The two largest absorption features of NH 3 in the longwave infrared, around 930 and 965 cm -1 show up with a contribution between 2 and 4 K, in addition to several other smaller enhancements, which altogether amply demonstrate the capability of GIIRS to measure NH 3 . This fit also illustrates the fact that no large spectral miscalibrations exist in the L1 data (this is not the case for spectra recorded prior to November 2019, which have a noticeable wavenumber shift; see also [START_REF] Smith | Improved severe weather forecasts using LEO and GEO satellite soundings[END_REF]). A large NH 3 column combined with a large thermal contrast (TC), as is the case here, represent ideal conditions for retrieving NH 3 columns from satellite measurements.

We briefly recall the concept of TC as it is important for what follows and refer to [START_REF] Clarisse | Satellite monitoring of ammonia: A case study of the San Joaquin Valley[END_REF] and [START_REF] Bauduin | IASI's sensitivity to near-surface carbon monoxide (CO): Theoretical analyses and retrievals on test cases[END_REF] for in-depth discussions. TC is the temperature difference between the surface and the lowermost atmospheric layer, where most NH 3 is located. As a consequence of the fact that Earth's surface heats up and cools down more quickly than the atmosphere, it typically becomes positive after sunrise and negative in the evening and at night. For a given NH 3 column, large TCs result in larger spectral signatures, and thus smaller retrieval uncertainties. On the other hand, when the TC is negative, NH 3 can be observed in emission instead of absorption. This case is illustrated in the bottom panel of Figure 1, which shows a spectrum observed in the early morning in winter. As the air (and thus the NH 3 gas) is in this case warmer than the surface, the NH 3 layer emits more radiation than it absorbs, providing favorable conditions for its measurement. TC thus determines the sign and the dynamic range over which the NH 3 spectral signature can vary, and therefore the retrieval uncertainty.

Retrieval

For this study, we have adapted the fast NH 3 retrieval approach originally developed for IASI, which has been detailed in a series of papers [START_REF] Franco | A general framework for global retrievals of trace gases from IASI: Application to methanol, formic acid, and PAN[END_REF][START_REF] Van Damme | Version 2 of the IASI NH 3 neural network retrieval algorithm: Near-real-time and reanalysed datasets[END_REF], 2021;[START_REF] Whitburn | A flexible and robust neural network IASI-NH 3 retrieval algorithm[END_REF]. Apart from instrument-specific aspects (such as the spectral range), the retrieval procedure is identical to version 3 of the IASI retrieval outlined in the last two papers.

At the heart of the retrieval is the hyperspectral radiance index (HRI), which quantifies the NH 3 spectral signature. This index is normalized to have a mean of zero and a standard deviation of one in the absence of observable NH 3 , and its sign can be both positive or negative depending on whether NH 3 is observed in absorption or emission. The HRI depends on the NH 3 abundance, but also on the temperature profile and other parameters (like viewing angle). The algorithm converts HRIs to columns with a neural network that takes into account all these dependencies. The neural network was trained with a large library of synthetic spectra that were simulated with random, but representative NH 3 columns and auxiliary variables. The state of the atmosphere at the overpass time and place of GIIRS were obtained from the ERA5 reanalysis [START_REF] Hersbach | The ERA5 global reanalysis[END_REF]. An important aspect of any NH 3 retrieval is the assumption that is made on the shape of the NH 3 vertical profile. NH 3 is emitted mainly at the surface and has an atmospheric lifetime of a couple of hours [START_REF] Van Damme | Industrial and agricultural ammonia point sources exposed[END_REF]; it is therefore largely confined to the PBL. To model the resulting vertical profile, the retrieval assumes a Gaussian function with a maximum at the surface and a spread σ equal to the ERA5 modeled PBL height. This profile is assumed over all land surfaces; over ocean a profile that peaks at 1.4 km is assumed. Fortunately, small uncertainties in the assumed profile generally do not majorly affect the retrieved column. In daytime conditions for instance, the difference in the retrieved column assuming a perfectly mixed PBL of 1 km compared to one of 0.1 or 2 km is about 25% [START_REF] Whitburn | A flexible and robust neural network IASI-NH 3 retrieval algorithm[END_REF]. CLARISSE ET AL. A postfilter was applied which removes cloudy observations and those measurements with limited sensitivity to NH 3 due to low TC. These were selected based on having a conversion factor between the HRI and the column in excess of 10 ⋅ 10 15 molec.cm -2 . In addition, as in previous work on IASI, a retrieval uncertainty was estimated for each individual pixel, by propagation of the uncertainty of the individual input parameters. The uncertainty on σ was set to 25%. For the detection of clouds, a dedicated neural network was trained here from collocated IASI-GIIRS observations and the IASI L2 cloud product. This network, which takes as input a handful of GIIRS channels, was set up to select scenes without clouds.

The retrieval has been performed on a complete year of GIIRS observations, from November 2019 to October 2020, and the resulting dataset is with this publication made available. The average of all observations measured in this period is shown in Figure 2 and qualitatively exhibits a good agreement with the IASI distributions [START_REF] Van Damme | Industrial and agricultural ammonia point sources exposed[END_REF]. Note that the edges of the seven latitudinal bands in which GIIRS scans the entire area can be discerned in the NH 3 distribution, hinting toward a residual artifact in the L1 data. The resulting biases are however very small in comparison to the actual variability of the columns.

We intercompared GIIRS-derived columns with v3 of the IASI NH 3 dataset [START_REF] Van Damme | Global, regional and national trends of atmospheric ammonia derived from a decadal (2008-2018) satellite record[END_REF], collocated within 2 km and 15 min, and as a whole found a regression slope of 1.14, negligible bias and a correlation coefficient of R 2 = 0.67 (see Supporting information S1). Possible reasons for the non-unit slope include instrument calibration, the HRI calculation, treatment of the surface temperature, systematic biases in the neural networks and the different viewing geometries.

CLARISSE ET AL.

Diel Cycle

Figure 3 shows the diel cycle observed from GIIRS during January, April, and July, representative for the distributions respectively seen in October-February, March-May, and June-September. A clear seasonal cycle is visible, with a maximum in April-July, consistent with IASI observations [START_REF] Van Damme | Worldwide spatiotemporal atmospheric ammonia (NH 3 ) columns variability revealed by satellite[END_REF].

We also observe a more pronounced diel cycle in April/July than in January. Due to the postfilter, the coverage of the NH 3 measurements varies greatly as a function of the time of the day, time of the year and the location. Clouds are unfortunately year-round prevalent in south(east) China and Southeastern Asia. The situation is especially unfavorable in the monsoon season (June-September). In addition, a lot of observations at night, early morning and evening are removed by the sensitivity test.

In what follows we look more into detail into two 1° by 1° areas located in Punjab (Pakistan) and the NCP. These are agricultural intensive regions, with intensively irrigated crops and a high density of livestock (Perrone, 2020; [START_REF] Robinson | Mapping the global distribution of livestock[END_REF][START_REF] Zhan | Improved estimates of ammonia emissions from global croplands[END_REF], where some of the largest NH 3 columns in the world are measured [START_REF] Kuttippurath | Record high levels of atmospheric ammonia over India: Spatial and temporal analyses[END_REF][START_REF] Meng | Role of ambient ammonia in particulate ammonium formation at a rural site in the North China Plain[END_REF][START_REF] Wang | Why is the Indo-Gangetic Plain the region with the largest NH 3 column in the globe during pre-monsoon and monsoon seasons?[END_REF][START_REF] Zhang | Agricultural ammonia emissions inventory and spatial distribution in the North China Plain[END_REF]. Figure 4 shows their monthly averaged diel cycles of the NH 3 column, ERA5 PBL height, HRI, and TC. The TC was calculated as the difference between the temperature measured from GIIRS spectra around 970 cm -1 and the temperature corresponding to middle of the PBL. This avoids relying on a dedicated surface temperature retrieval but underestimates its actual value by up to 1-2 K in the dry seasons, and up to 4 K in the wet seasons, because of the water vapor continuum absorption. When fewer than 30 observations are available for a given overpass time, the NH 3 column was obtained by interpolation from the other points of the time series (indicated with hollow squares). This helps to remove potential outliers in the time series. The median uncertainty over all averaged pixels for a given overpass time is shown as a shaded area around the mean value. While this uncertainty includes a random component, it also includes potential sources of bias, related to for example, the vertical profile of NH 3 or the ERA5 temperature profile. For that reason we recommend to exercise care when interpreting the data.

January

For both sites, the diel cycle is weak in January, consistent with the absence of pronounced diurnal or nocturnal sources at these rural sites in the winter. A slight increase of the columns is observed during nighttime. Such increases have been reported for surface measurements, which are easily explained by the lower nocturnal mixing height, trapping emissions near the surface [START_REF] Saraswati | Five-year measurements of ambient ammonia and its relationships with other trace gases at an urban site of Delhi, India[END_REF][START_REF] Tevlin | Tall tower vertical profiles and diurnal trends of ammonia in the Colorado Front Range[END_REF]Q. Wang et al., 2019). However, as we consider the entire column, the small decreases seen during daytime are more likely due to small biases in the assumed vertical NH 3 profile, the PBL height or the vertical temperature profiles. In January, the diel cycle is captured almost completely, thanks to favorable measurement conditions both during night and day. Ranging between -6 and 7 K, the TCs are sufficiently large to provide sensitivity to the PBL at all times of the day except around dawn and dusk. This is when the HRI and TC cross the zero mark and the time of the day when all sensitivity to PBL NH 3 is lost, irrespective of the magnitude of the column.

April

With increasing temperatures, and with the growing season well underway, much larger NH 3 columns are observed during April. Columns over Punjab are double those over the NCP, even though emissions are estimated to be lower. This apparent contradiction was attributed to the higher concentrations of the neutralizing NO x and SO 2 compounds over the NCP, leading to a shorter atmospheric lifetime of NH 3 [START_REF] Liu | Rapid SO 2 emission reductions significantly increase tropospheric ammonia concentrations over the North China Plain[END_REF][START_REF] Wang | Why is the Indo-Gangetic Plain the region with the largest NH 3 column in the globe during pre-monsoon and monsoon seasons?[END_REF]. Just as in January, the HRI and TC undergo a synchronous cycle in April, alternating between negative and positive values, and providing favorable measurement conditions at all times of the day except during twilight. The outlying NH 3 column (much below the average) in April near 18:00 in the NCP results likely from a retrieval artifact. As can be seen, the HRI in this case is very close to zero. Assuming no abrupt changes in the NH 3 column, this either points to very low measurement sensitivity or to a situation where emission and absorption cancel each other out in different parts of the atmosphere. In either case, this renders the retrieval from a theoretical standpoint ill-defined. In the majority of cases the postfilter takes care of removing such measurements. However, the thermal conditions as provided within ERA5, coupled with the assumed NH 3 vertical profile, are apparently such that a number of these measurements slipped through the net of the postfilter.

A much larger diel cycle is observed in April as compared to January, especially in Punjab with a column more than three times larger during day than at night. Similarly high ratios, with daytime maximums, have been observed in the diel variations of total columns over Paris [START_REF] Kutzner | Diurnal evolution of total column and surface atmospheric ammonia in the megacity of Paris, France, during an intense springtime pollution episode[END_REF] and in a rural area in the Netherlands [START_REF] Dammers | Measuring atmospheric ammonia with remote sensing campaign: Part 1 -Characterisation of vertical ammonia concentration profile in the centre of the Netherlands[END_REF]. There are a number of factors which explain such large day-night variations. (i) During day, agricultural practices and animal activity [START_REF] Aarnink | Effect of slatted floor area on ammonia emission and on the excretory and lying behaviour of growing pigs[END_REF] et al., 2020). Emissions exhibit in particular an up to exponential increase with temperature [START_REF] Ernst | The effects of several factors on volatilization of ammonia formed from urea in the soil[END_REF][START_REF] Schjoerring | Physiological parameters controlling plant-atmosphere ammonia exchange[END_REF][START_REF] Sutton | Toward a climate-dependent paradigm of ammonia emission and deposition[END_REF][START_REF] Tevlin | Atmospheric ammonia: Measurements, modeling, and chemistry-Climate interactions[END_REF]L. Zhang et al., 2010), making it an important driver both for the seasonal and the diel emission cycle. While day-night temperature differences (between 15 and 22 K) are not much larger in April than in January, surface temperatures are some 10-15 K larger in April, and via the exponential dependence have a much larger relative effect [START_REF] Hempel | Non-linear temperature dependency of ammonia and methane emissions from a naturally ventilated dairy barn[END_REF]. In addition, during daytime, relative humidity is on average lower and wind speed on average higher than at night, two factors which again favor daytime emissions [START_REF] Meng | Role of ambient ammonia in particulate ammonium formation at a rural site in the North China Plain[END_REF][START_REF] Olesen | Modelling effects of wind speed and surface cover on ammonia volatilization from stored pig slurry. Atmospheric Environment Part A[END_REF][START_REF] Tevlin | Tall tower vertical profiles and diurnal trends of ammonia in the Colorado Front Range[END_REF][START_REF] Wentworth | The role of dew as a night-time reservoir and morning source for atmospheric ammonia[END_REF]. (iii) At night, temperatures are lower, humidity levels higher, thereby favoring the conversion of NH 3 gas to particulate. The reverse process may contribute to daytime enhancements [START_REF] Kutzner | Diurnal evolution of total column and surface atmospheric ammonia in the megacity of Paris, France, during an intense springtime pollution episode[END_REF][START_REF] Meng | Role of ambient ammonia in particulate ammonium formation at a rural site in the North China Plain[END_REF][START_REF] Saylor | Continuous gaseous and total ammonia measurements from the southeastern aerosol research and characterization (SEARCH) study[END_REF][START_REF] Tevlin | Tall tower vertical profiles and diurnal trends of ammonia in the Colorado Front Range[END_REF].

(iv) Finally, nocturnal columns could be underestimated in the retrieval as a result of the vertical profile assumption. The observed negative HRIs are compatible with the sign of the TC, demonstrating that most of the NH 3 column is indeed located near the surface as assumed by the retrieval. However, it cannot be excluded (see below) that compared to the assumed profile, a relative larger portion of the NH 3 column is situated higher up. In this case, the higher and colder parts of the column absorb radiation and therefore cancel out some of the spectral emission at night.

The variations in the diel cycles found here are larger than those typically found for surface concentrations.

While NH 3 concentrations are expected to increase during daytime due to increased emissions and also evaporation of ammonium-containing aerosols, the larger boundary layer prevents a large buildup of NH 3 near the surface. For example, in a rural site in the NCP, surface concentrations vary only between 25 and 40 ppb in April, with the maximum reached in the morning due to the evaporation of dew and guttation droplets [START_REF] Kuang | Explosive morning growth phenomena of NH 3 on the North China Plain: Causes and potential impacts on aerosol formation[END_REF][START_REF] Wentworth | The role of dew as a night-time reservoir and morning source for atmospheric ammonia[END_REF]. For other sites in Asia, even smaller differences are reported [START_REF] Datta | Ammonia emission from subtropical crop land area in India[END_REF][START_REF] Saraswati | Five-year measurements of ambient ammonia and its relationships with other trace gases at an urban site of Delhi, India[END_REF][START_REF] Saylor | Continuous gaseous and total ammonia measurements from the southeastern aerosol research and characterization (SEARCH) study[END_REF][START_REF] Wang | Atmospheric ammonia and its impacts on regional air quality over the megacity of Shanghai, China[END_REF]. In addition, surface concentrations reported in the winter are often only just below those in April (e.g., [START_REF] Kuang | Explosive morning growth phenomena of NH 3 on the North China Plain: Causes and potential impacts on aerosol formation[END_REF][START_REF] Wang | Atmospheric ammonia and its impacts on regional air quality over the megacity of Shanghai, China[END_REF]. At night, mixing layers in the two months are also of comparable magnitude, and assuming that most NH 3 is located in the PBL, this implies that columns should be of similar magnitude too at night, what is consistent with what is observed here.

July

July is the month when some of the highest columns are measured for both sites, but unfortunately only 4 or 5 of the 10 overpasses have enough measurements for obtaining meaningful averages, so that little can be said on the diel cycle. In what follows we explore the reasons for the poor measurement conditions.

The first thing to note is that the temperatures of both the surface and the air (not shown) vary little (between 7 and 10 K) throughout the day and night, much smaller than in the January/April months. Smaller day-night variations entail a smaller dynamic range of the TC, as observed clearly in Figure 4, and inherently worsen the conditions for NH 3 detection by GIIRS.

For Punjab, TCs are mostly negative, only just reaching positive numbers around noon. At night, a TC is observed around -6 K. Allowing for the underestimation of the TC by an average of 4 K in this humid month, this puts the actual TC close to zero. This explains why the corresponding measurements are removed by the retrieval postfilter. However, we observe a mean HRI of about five during the night, which suggests a positive TC. A likely explanation is that the NH 3 vertical profile for this site, at this time of the year, is significantly different from the assumed profile at night. Recall that the retrieval assumes that the majority (about 68%) of the NH 3 column is located within the PBL, as supported by several studies [START_REF] Dammers | Measuring atmospheric ammonia with remote sensing campaign: Part 1 -Characterisation of vertical ammonia concentration profile in the centre of the Netherlands[END_REF][START_REF] Guo | Validation of IASI satellite ammonia observations at the pixel scale using in-situ vertical profiles[END_REF][START_REF] Sun | Validation of TES ammonia observations at the single pixel scale in the San Joaquin Valley during DISCOVER-AQ[END_REF][START_REF] Tevlin | Tall tower vertical profiles and diurnal trends of ammonia in the Colorado Front Range[END_REF]. However, in the evening just after the collapse of the PBL, NH 3 is likely to remain present in the residual layer for several hours (see also [START_REF] Kuang | Explosive morning growth phenomena of NH 3 on the North China Plain: Causes and potential impacts on aerosol formation[END_REF][START_REF] Lonsdale | Modeling the diurnal variability of agricultural ammonia in Bakersfield, California, during the CalNex campaign[END_REF]. Given the relative longer lifetime of NH 3 in the Indo-Gangetic Plain, and the abundance of NH 3 in the daytime summer column, this could explain the positive HRIs observed for Punjab during the night. The measurements with a positive HRI that were removed by the postfilter could in principle be exploited by using a more appropriate vertical profile in the retrieval. Also note that during daytime, elevated free-tropospheric NH 3 layers can occur, as illustrated in a recent aircraft campaign over Baoding (NCP) [START_REF] Pu | Aircraft measurements reveal vertical distribution of atmospheric ammonia over the North China Plain in early autumn[END_REF].

Geophysical Research Letters

The diel cycle over the NCP follows a similar pattern as for Punjab in July, but here the absence of sensitivity during the night is obvious with HRI values very close to zero.

Outlook

GIIRS is the first geostationary hyperspectral infrared sounder in orbit. As we have shown in this letter, its 10 overpasses provide an unprecedented opportunity to study atmospheric NH 3 and its evolution throughout the day. The availability of so many overpasses per day could also greatly benefit NH 3 modeling and model assimilation [START_REF] Chen | High-resolution hybrid inversion of IASI ammonia columns to constrain US ammonia emissions using the CMAQ adjoint model[END_REF]Z. Wang et al., 2020). Finally, compared to polar orbits, GIIRS has the advantage of revisiting the same footprint every day, which will be most useful for tracking seasonal variations over particular locations, and for validation and intercomparison.

While the presented NH 3 dataset was prepared with great care, improvements are certainly possible. An important improvement could come from the use of temperature profiles directly retrieved from the GIIRS radiances, rather than ERA5 profiles. In addition, a better parametrization of the NH 3 vertical profile as a function of time and place would be highly desirable. This could be achieved by coupling the retrieval with the output of a state of the art tropospheric chemistry model. It is our hope that the presented results will stimulate further research in these directions.

The future landscape of geostationary sounders is looking bright. With a lifetime of 5-7 years, FY-4A is nearing its end, but fortunately, its successor FY-4B is to be launched this year. In addition, Europe is soon to be served with its own geostationary infrared sounder (IRS) onboard the Meteosat Third Generation (MTG) satellite (https://www.eumetsat.int/meteosat-third-generation). IRS is similar to GIIRS but will offer more frequent overpasses and a smaller footprint on ground. Given the results obtained with GIIRS, the arrival of IRS is extremely promising and will no doubt provide fantastic monitoring means of NH 3 over Africa and Europe.

Acknowledgments

Figure 1 .

 1 Figure1. GIIRS spectra (blue) observed over the North China Plain in summer (top) and winter (bottom). The red spectra were calculated with a forward model and were adjusted to best match the observed spectra. The difference between observed and fitted is shown in green. When the NH 3 -contribution (black) is removed from the simulations, the orange residuals are obtained.

  -2020, 06:37 UTC at 34.21°N, 119.66°E; TC = 8.3 K; NH 3

Figure 2 .

 2 Figure 2. November 2019 to October 2020 average of GIIRS-derived NH 3 columns on a 0.5° × 0.5° grid. Due to the cloud or sensitivity filter (see text), certain periods of the day or year might locally be under/overrepresented in this average. The red stars indicate the locations of the two sites analyzed in Section 4 and Figure 4 (30° N, 73° E for Pakistan and 38° N, 115.5° E for China).

Figure 3 .

 3 Figure 3. Monthly NH 3 averages for each of the 10 GIIRS overpass times for the months of January, April, and July.

  are enhanced. (ii) NH 3 emissions from plants and soils increase during the day due to meteorological factors (

Figure 4 .

 4 Figure 4. Diurnal cycle of NH 3 (blue) over 1° by 1° areas in Punjab, Pakistan (left) and the North China Plain (right). Data represented with open squares were obtained through interpolation (see text). Shown are also the ERA5 PBL height (black), the TC (dark red), and the HRI (green). The shaded area represent the median uncertainty for a given overpass time. The time axis is in local time (calculated from the longitude) and includes the time sunrise and sunset on the 15th of each month.

  We are very grateful to the entire scientific and technical team of GIIRS and FY-4A, the National Satellite Meteorological Center (NSMC, http:// www.nsmc.org.cn/NSMC/Home/ Index.html) and the China Meteorological Administration (CMA) for making GIIRS possible. We gratefully acknowledge the NSMC for providing open access to the GIIRS data. This research was funded by the F.R.S.-FNRS and the Belgian State Federal Office for Scientific, Technical and Cultural Affairs (Prodex arrangement HIRS). L. Clarisse is Research Associate and M. Van Damme is Postdoctoral Researcher, both supported by the Belgian F.R.S.-FNRS. C. Clerbaux is grateful to CNES for scientific collaboration and financial support.

Data Availability Statement

The NH 3 dataset presented in this study is available at https://dx.doi.org/10.5281/zenodo.4540024.