Roger Marí

Gabriele Facciolo

Thibaud Ehret

Multi-Date Earth Observation NeRF: The Detail Is in the Shadows

We introduce Earth Observation NeRF (EO-NeRF), a new method for digital surface modeling and novel view synthesis from collections of multi-date remote sensing images. In contrast to previous variants of NeRF proposed in the literature for satellite images, EO-NeRF outperforms the altitude accuracy of advanced pipelines for 3D reconstruction from multiple satellite images, including classic and learned stereovision methods. This is largely due to a rendering of building shadows that is strictly consistent with the scene geometry and independent from other transient phenomena. In addition to that, a number of strategies are also proposed with the aim to exploit raw satellite images. We add model parameters to circumvent usual pre-processing steps, such as the relative radiometric normalization of the input images and the bundle adjustment for refining the camera models. We evaluate our method on different areas of interest using sets of 10-20 pre-processed and raw pansharpened WorldView-3 images.

Introduction

Today, hundreds of satellite Earth observation missions are in operation and the number continues to grow [54]. Many Earth observation satellites acquire optical images periodically over the same areas, contributing to large multidate collections of satellite images. These collections offer the opportunity to observe the evolution of a site over time and to identify permanent and transient structures. However, when it comes to digital surface modeling from remote sensing, multi-date imagery is usually disregarded in favor of a few stereo or tri-stereo image products. The latter are more expensive, as images are acquired almost simultaneously to facilitate the use of classic photogrammetry.

The state of the art in digital surface modeling from satellite images relies on the availability of stereo image products. The reason of this is that the Multi-View Stereo (MVS) logic is predominantly adopted to address the task as a multi-pair problem instead of a real multi-view problem [START_REF] Furukawa | Multi-view stereo: A tutorial[END_REF]. This is the case of the NASA Ames stereo pipeline [START_REF] Beyer | The Ames Stereo Pipeline: NASA's open source software for deriving and processing terrain data[END_REF], MicMac [START_REF] Rupnik | MicMac-a free, open-source solution for photogrammetry[END_REF] by the the French National Ge-Pansharpened image EO-NeRF altitude Sat-NeRF altitude Figure 1. EO-NeRF can be used for novel view synthesis and 3D reconstruction from an input set of multi-date satellite images. Elevation models obtained with EO-NeRF reveal scene geometry with an unprecedented level of detail that surpasses previous NeRF variants [32] and is comparable to that of intrusive aerial acquisitions. This includes narrow and irregular structures such as the arches and roller coasters observed in the pansharpened image.

ographic Institute (IGN), CARS [START_REF] Michel | A new satellite imagery stereo pipeline designed for scalability, robustness and performance[END_REF] by the French Space Agency (CNES), S2P [11] by the Centre Borelli research center, or the CATENA [23] multi-stereo chain by the German Aerospace Center (DLR). In MVS, each stereo product is used to obtain an independent digital surface model (DSM). When several pairs are available, the resulting DSMs are merged into a single model of higher accuracy. Parallel to this scenario, MVS approaches for 3D reconstruction have lost hegemony for common close-range imagery. Since the release of NeRF [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF] in 2020, neural rendering methods have become extremely popular for conventional image collections, as they represent a highly accurate, unsupervised, and truly multi-view solution to the 3D modeling problem. Satellite imagery is a niche where neural rendering still has a long way to go, but some pioneering work has already started to bridge the gap between the two worlds. In particular, S-NeRF [13] and Sat-NeRF [32] explored NeRF for multi-date satellite images, but their performance remained inferior to state-of-the-art MVS using a set of manually selected pairs.

We introduce EO-NeRF, the Earth Observation NeRF, for digital surface modeling and novel view synthesis using sets of multi-date remote sensing images. The altitude accuracy of EO-NeRF outperforms state-of-the-art stereo- vision methods and previous NeRF approaches. Figure 1 shows the level of detail that EO-NeRF achieves in narrow and irregular structures that are normally lost using concurrent methods. This is made possible thanks to the following contributions:

-A NeRF approach for multi-date satellite images that does not predict shadows, but renders them according to the geometry and the position of the sun. The geometrically consistent shadows provide hints that permit to refine the geometry, which in turn refines the shadows. This approach also ensures that the model generalizes for novel view synthesis using solar directions completely different from those of the input images.

-A variety of strategies to use unprocessed satellite images. These include the use of UTM coordinates to handle georeferenced data in a more appropriate way and the addition of network parameters to address inaccuracies in the camera models and color biases in the images during the optimization process.

We test EO-NeRF on 7 areas of interest covering 256 × 256 m each, using ∼10-20 crops from multi-date WorldView-3 images with a resolution of 30 cm/pixel. Differently from previous NeRF variants, our evaluation is not limited to pre-processed RGB images, as EO-NeRF can directly handle raw pansharpened products. The input data and EO-NeRF results are released to contribute to the creation of a NeRF benchmark for multi-date satellite images.

Related work

Stereovision methods and NeRF can be used for multiview 3D reconstruction under the assumption that the scene geometry and radiance are invariant or nearly invariant. On the one hand, the strengths of NeRF are its simplicity (no need to select or merge pairs and no geometry supervision), its impressive level of detail and the ability to solve the colorization problem simultaneously with 3D estimation. On the other hand, the weaknesses of NeRF are the need for larger sets of input images and a long optimization time (usually hours), whereas stereo methods can quickly produce coarse geometry estimates with as few as two images.

In this section we review the fundamentals of NeRF, stereovision satellite 3D reconstruction, and NeRF variants for multi-date images that are most relevant to our work.

NeRF in a nutshell

A NeRF, or Neural Radiance Field [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF], is a continuous function F that represents the geometry and appearance of a 3D scene. F is encoded using a fully-connected neural network or multi layer perceptron (MLP) that is specific to each scene and does not generalize to others. In its simplest form, a NeRF MLP takes as input some 3D coordinates x and, optionally, a viewing direction d, which are used to predict the observed RGB color c and a non-negative scalar volume density σ at the input point.

F : (x, d) → (σ, c). (1)
Using a collection of input views and their camera poses, the MLP encoding (1) is optimized by casting individual rays that project onto the known pixels. Each ray r is defined by a point of origin o and a direction vector d. The color c(r) of a ray r(t) = o + td is computed as

c(r) = N i=1 T i α i c i .
(2)

The rendered color c(r) results from integrating the colors c i predicted at different points of the ray r. Each ray r is therefore discretized into N 3D points x i between the near and far bounds of the scene, t n and t f . Each point x i in r is obtained as

x i = o + t i d, where t i ∈ [t n , t f ].
The contribution of each point x i in r to the rendered color (2) depends on the opacity α i and the transmittance T i :

α i = 1 -exp(-σ i δ i ) and T i = i-1 j=1 (1 -α j ) . ( 3
)
Both factors take values in the interval [0, 1] and depend only on the volume density σ that defines the geometry. The opacity increases with σ and is the probability that x i belongs to a non-transparent surface. The transmittance is the probability that light reaches x i without hitting previous opaque points in the ray r and is used to handle occlusions.

Using (3), the depth d(r) observed by casting a ray r can be determined in a similar manner to the color (2) as

d(r) = N i=1 T i α i t i . ( 4
)
The MLP is optimized by minimizing the mean squared error (MSE) between the rendered color c(r) and the real color c GT (r) of the image pixel intersected by each ray r:

r∈R ∥c(r) -c GT (r)∥ 2 2 , ( 5
)
where R is a batch of arbitrarily selected rays.

Classic and learned stereovision methods

Stereovision 3D reconstruction pipelines usually share some fundamental steps illustrated in Figure 2. The most critical task is dense matching or disparity estimation.

Among the classic methods for disparity estimation, Semi-Global Matching (SGM) [21] is the preferred choice for satellite imagery due to its efficiency to exploit spatial regularity along different cardinal directions. A good number of DSM production pipelines use SGM with the census transform [51] as matching cost for its robustness to illumination changes [START_REF] Angelo | Digital elevation models from stereo, video and multi-view imagery captured by small satellites[END_REF][START_REF] Michel | A new satellite imagery stereo pipeline designed for scalability, robustness and performance[END_REF][START_REF] Bosch | Semantic stereo for incidental satellite images[END_REF]. Other pipelines employ SGM variants: MGM [15,[START_REF] Facciolo | Automatic 3D reconstruction from multi-date satellite images[END_REF] increases the directions taken into account for regularization without loss of efficiency, tSGM [27,[START_REF] Chibane | Stereo radiance fields (SRF): Learning view synthesis for sparse views of novel scenes[END_REF] adopts a coarse-to-fine strategy to limit the disparity search range, Semi-Global Block Matching [25] uses windows instead of individual pixels, etc.

Learned methods for disparity estimation use deep neural networks. These networks comprise different modules dedicated to feature extraction, cost volume construction, cost volume regularization and disparity regression (based on minimum cost). Architectures such as PSM [5], HSM [START_REF] Yang | Hierarchical deep stereo matching on highresolution images[END_REF] or GA-Net [52] have been tested on remote sensing images with encouraging results [19,[START_REF] Marí | Disparity estimation networks for aerial and high-resolution satellite images: A review[END_REF]44]. Unlike classic algorithms, deep neural networks are prone to fail or lose accuracy in unseen scenarios (e.g., different shapes, disparity range, viewing angles or rectification criteria). In addition to this generalization issue, learned methods (including multi-view ones, such as MVSNet [47]) are subject to long training times (in the order of days or weeks) and require large disparity benchmarks for supervision [44].

NeRF for multi-date image collections

NeRF [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF] opened the door to a large number of variants that address the limitations of the original method. This includes speeding up the optimization process [START_REF] Chen | TensoRF: Tensorial radiance fields[END_REF][START_REF] Fridovich-Keil | Plenoxels: Radiance fields without neural networks[END_REF][START_REF] Müller | Instant neural graphics primitives with a multiresolution hash encoding[END_REF]42], reducing the number of input views [START_REF] Ehret | Nerf, meet differential geometry![END_REF]22,[START_REF] Chen | MVSNeRF: Fast generalizable radiance field reconstruction from multi-view stereo[END_REF] or generalizing to unseen scenes [7, 9, 49]. Certain variants have also addressed the use of in-the-wild photo collections, such as multi-date images, presenting changes in appearance and geometry across the input views [13,32,[START_REF] Martin-Brualla | NeRF in the wild: Neural radiance fields for unconstrained photo collections[END_REF].

The trend in NeRF variants for unconstrained photo collections is to add auxiliary networks or heads to the single MLP architecture originally used in NeRF. The color rendering operation (2) and/or the loss function (5) are modified to account for image-dependent reflectance models that can accommodate the complex inconsistencies between the input views while recovering the underlying shared geometry [24,53]. Image-dependent features are normally extracted using embedding modules learned during the NeRF optimization [START_REF] Martin-Brualla | NeRF in the wild: Neural radiance fields for unconstrained photo collections[END_REF] or independently pre-trained convolutional neural networks [8].

NeRF in the Wild (or NeRF-W) [START_REF] Martin-Brualla | NeRF in the wild: Neural radiance fields for unconstrained photo collections[END_REF] was created for collections of hundreds or thousands of multi-date street view images. It added auxiliary heads to predict transient colors and densities in parallel to the original multi-view consistent outputs [START_REF] Beyer | The Ames Stereo Pipeline: NASA's open source software for deriving and processing terrain data[END_REF]. Shadow NeRF (or S-NeRF) [13] worked with tens of satellite images modeled locally as pinhole cameras. An auxiliary head was used to render building shadows in the input images as a function of the direction of solar rays. Satellite NeRF (or Sat-NeRF) [32] improved the results of S-NeRF by directly employing the actual RPC (Rational Polynomial Coefficients) camera models associated with the satellite images and by adding an auxiliary head to handle transient objects similarly to NeRF-W.

Method

EO-NeRF is optimized using the ray casting strategy proposed in NeRF [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF] (Section 2.1). The original NeRF function ( 1) is transformed into:

F : (x, d sun , t j ) → (σ, c a , a, β, τ, A j , b j ). (6)
The inputs of ( 6) are the same as in Sat-NeRF [32]: the 3D point coordinates x, the solar direction vector d sun and an image-dependent embedding vector t j (where j is the image index). The outputs of ( 6) are the following: -σ: Volume density scalar at location x.

c a : Albedo RGB, related only to the geometry, i.e. the spatial coordinates x. -a: Ambient RGB color that defines a global hue according to the solar direction d sun . -β: Uncertainty scalar related to the probability that the color of x corresponds to a transient object. -τ : Transient scalar, learned as a function of x and t j . -A j , b j : RGB vectors encoding a color affine transformation between the albedo and the current image j.

EO-NeRF uses the outputs of (6) to render the color c(r) of each ray r as follows:

c(r) = A j ℓ(r) • N i=1 T i α i c a + b j (7)
where

ℓ(r) = s(r) + (1 -s(r))a with s(r) defined in (9). (8) Input EO-NeRF outputs (a) (b) (c) (d) (e) (f) (g) (h) (i)

Geometrically consistent shadow model

Both S-NeRF and Sat-NeRF predict shadows as a color property depending on the solar direction [13,32]. This leads the model to overfit the input views, taking advantage of that freedom to render other complex changes such as vegetation color or transient objects. As a result, it does not generalize well to solar directions different from those in the input images (Figure 4). S-NeRF and Sat-NeRF attempted to minimize this misbehavior by adding a solar correction term to the loss that seeks to bring shadow renderings closer to binary masks strictly related to object shadows. At the same time, this choice forces the network to explain differently the other transient phenomena, which may have side effects, so it is necessary to set an appropriate weight for the auxiliary solar correction term.

In EO-NeRF, we propose a model that does not predict shadows but instead renders them based on the geometry at each optimization step. We denote the rendering of geometrically consistent shadows as s geo . The value of s geo is obtained by casting rays from each surface point to the source of light, similarly to [2, 41, 46]. For each camera ray r, we find the point on the surface using (4), i.e., x S in Figure 5. This point is taken as the origin to cast a solar ray, r sun , towards the position of the sun. The transmittance T (3) of the last point in r sun is equal to the amount of geometric shadow, s geo = T (r sun (t N )), which reveals whether x S is visible in the direction of the sun or not. The value of s geo renders shadows based on the geometry of the scene and ignores the other transient phenomena in each image. To represent the latter, we use the transient scalar τ predicted by the transient head of EO-NeRF (MLP 3 in Figure 6). τ is an arbitrary scalar between 0 and 1 that is not bound by geometry. The product of s geo with τ defines the s(r) of the irradiance model (8):

s(r) = s geo (r)τ (r) = T (r sun (t N )) N i=1 T i α i τ (x i ), (9)
where the dependency of τ on t j is omitted for simplicity and x i ∈ r. The use of a product in (9) guarantees that s(r) always contains geometrically consistent shadows in addition to the rest of transient phenomena. τ (r) = 1 can be set at test time for transient object removal, as in Figure 3(h).

Compared to previous work, the strength of (9) is that it decouples shadows from other transient phenomena. Note that casting solar rays is also necessary to compute the solar correction term used in S-NeRF and Sat-NeRF [13,32]. We use solar rays in a more direct way, which does not require modifying the loss function or tuning any hyperparameters to balance the optimization process.

Network architecture and loss function

The EO-NeRF network architecture is shown in Figure 6. The main MLP 1 takes as input the 3D point coordinates x and predicts the volume density σ, as in the original NeRF [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF]. The features extracted by MLP 1 are then plugged into the auxiliary heads MLP 2 and MLP 3. MLP 2 uses them to predict the albedo, while MLP 3 merges them with t j to predict the transient-related magnitudes, i.e., β and τ . MLP 4 is disconnected from the rest as it predicts the ambient color a, which is constant for all points x and only depends on the solar direction d sun . All image-dependent vectors t j , A j , b j are learned using embedding modules that take as input the image index j. We use positional encoding [START_REF] Mildenhall | NeRF: Representing scenes as neural radiance fields for view synthesis[END_REF] for inputs x and d sun , as well as ReLU nonlinearity between MLP layers.

EO-NeRF uses a loss function similar to NeRF-W [START_REF] Martin-Brualla | NeRF in the wild: Neural radiance fields for unconstrained photo collections[END_REF] and Sat-NeRF [32], where the uncertainty scalar β reduces the contribution of camera rays emitted from pixels with a high probability of representing transient phenomena:

r∈R ∥c(r) -c GT (r)∥ 2 2 2β ′ (r) 2 + log β ′ (r)+η 2 , ( 10
)
where β ′ (r) = β(r) + β min . As in [32], β min = 0.05 and η = 3 are used in [START_REF] Angelo | Digital elevation models from stereo, video and multi-view imagery captured by small satellites[END_REF] to avoid negative values in the logarithm. The logarithm prevents β from converging to infinity to minimize [START_REF] Angelo | Digital elevation models from stereo, video and multi-view imagery captured by small satellites[END_REF] and encourages the network to correctly use β to identify which points are worthy of being regarded as transient or not depending on each image.

UTM-based point coordinates

S-NeRF [13] approximated the RPC camera models of the input images as pinhole camera models. Sat-NeRF [32] demonstrated the benefits in accuracy of directly using the RPC models to sample points in the 3D space. For that purpose, Sat-NeRF uses Earth-centered Earth-fixed (ECEF) coordinates to represent points in the 3D space. We follow the RPC-based sampling but using UTM coordinates and altitude to represent 3D points, which locally preserves the properties of a Cartesian system and offers the advantage that the altitude of the scene is aligned with the z-axis. As shown in Figure 7, this ensures a better use of the space occupied by the scene with respect to the 3D volume that contains it.

Internal bundle adjustment

Sat-NeRF stressed the importance of bundle adjustment to ensure consistency between the input RPC camera models associated with the satellite images. However, it proposed to perform the bundle adjustment correction before the MLP optimization, using an independent software [29]. Recent work with NeRF shows that it is possible to refine the camera models simultaneously during the MLP optimization [28,48]. Especially if a good initialization is available, as in the case of satellite RPC camera models [START_REF] Marí | To bundle adjust or not: A comparison of relative geolocation correction strategies for satellite multiview stereo[END_REF]. We propose a simple way to integrate the refinement of RPC camera models into the EO-NeRF optimization, based on the assumption that RPC cameras can be locally approximated as affine projection models [11]. Under such assumption it is common to refine each RPC by composition of the projection function with an offset correction (i.e., a translation on the image plane) [START_REF] Marí | To bundle adjust or not: A comparison of relative geolocation correction strategies for satellite multiview stereo[END_REF][START_REF] Chen | Hallucinated neural radiance fields in the wild[END_REF]. In our UTM-based coordinate system the ray origin related to each pixel lies on the upper plane of the 3D volume (see Figure 7(b)), which is coincident with the maximum altitude of the scene [32]. Thus, the offset correction can be approximated as a displacement in the x and y axis. If r is a ray intersecting the j-th camera, with origin o and direction d, the bundle adjusted version of r can be approximated as r(t) = (o + q j ) + td such that q j = (q 1 , q 2 , 0) j . (11) Expression (11) replaces the usual r(t) = o + td and is constrained to points inside the boundaries of the cube containing the scene to prevent large displacements. The offset coefficients q 1 and q 2 of each camera are learned using a two-dimensional embedding module in the same fashion as t j , A j and b j . In Section 4, the results show that using (11) significantly narrows the gap compared to correcting the camera models separately before optimizing the MLP.

Experiments

We evaluate EO-NeRF on 7 areas of interest (AOI) covering 256 × 256 m each, using ∼10-20 crops from multidate WorldView-3 images with a resolution of 30 The images are taken from the public data of the 2019 IEEE GRSS Data Fusion Contest (DFC2019) [3, 26] and 2016 IARPA Multi-View Stereo 3D Mapping Challenge (IARPA2016) [4]. The specific number of images and other details of each AOI are shown in Table 1. In both cases, we used the panchromatic and multispectral products to create raw pansharpened images without any color correction. The DFC2019 data additionally provides true color RGB images over the AOIs with reduced dynamic range in 8-bit unsigned integer format. We assess the EO-NeRF elevation models using lidar DSMs with a resolution of 30-50 cm/pixel. The PSNR of the image renderings and the altitude mean absolute error (MAE) with respect to the lidar data are used as evaluation metrics. Geometrically corrected RPC camera models obtained with the bundle adjustment package [29] were used in all experiments unless raw RPCs is mentioned.

Implementation details

We use a batch size of 1024 rays, Adam optimizer and an initial learning rate of 5e -4 . The first optimization steps are performed using the standard MSE loss (5). The complexity of the optimization problem is gradually increased: first by plugging β and adopting the full loss [START_REF] Angelo | Digital elevation models from stereo, video and multi-view imagery captured by small satellites[END_REF] and then by adding the internal bundle adjustment offsets (11). Both camera rays r and solar rays r sun are discretized into N = 128 evenly distributed samples to encourage detailed shadows. For novel view synthesis from unseen viewpoints, we use affine projection camera models to locally approximate unknown RPC functions [START_REF] Marí | To bundle adjust or not: A comparison of relative geolocation correction strategies for satellite multiview stereo[END_REF]. Convergence takes about 250-300k steps (∼10-20 h, similarly to Sat-NeRF, depending on the number of images using a 12 GB GPU).

DFC2019 areas -Results and discussion

Figure 8 shows the DSMs of the DFC2019 areas obtained with EO-NeRF and concurrent methods. We propose three categories of experiments, corresponding to the rows of Table 2 discussed below. EO-NeRF achieves the lowest MAE on average, as indicated in bold in Table 2. Due to the difficulty of computing PSNR for unseen views (e.g. change of ambient color or transients), it is computed using the input views and thus might not improve over Sat-NeRF. The latter has more freedom to overfit the input views since shadows are not forced to be consistent with geometry.

Category 1. We test the state-of-the-art Sat-NeRF model using N = 64 points per ray (row 1), as in [32]. Using UTM-based point coordinates (Section 3.3) we obtain similar or better altitude MAE with respect to the results originally reported in [32], while using half of units in the MLP layers (h = 256 instead of h = 512). Increasing the number of points per ray to N = 128 (row 2) does not necessarily improve the performance. We note that Sat-NeRF does not adapt to non-radiometrically normalized inputs, as the MAE increases significantly using raw pansharpened data.

Visual inspection of the DSMs in Figure 8 shows that Sat-NeRF is more prone to surface irregularities than EO-NeRF.

In fact, Sat-NeRF explored an auxiliary depth supervision term to prevent geometry irregularities [12]. The geometrically consistent shadow model of EO-NeRF is a natural solution to prevent undesired holes or blobs in the geometry that would result in unrealistic shadows. Category 2. We test EO-NeRF (row 3) using the same externally bundle adjusted RPC cameras as in the Sat-NeRF experiments. The altitude MAE improves dramatically by an average of more than half a meter. The improvement is especially noticeable for raw pansharpened images. Using the unrefined camera models from the image metadata significantly reduces performance in both PSNR and MAE (row 4), unless the EO-NeRF internal bundle adjustment is active (row 5). Externally corrected RPCs provide the most consistent geometry by a small margin, which we attribute to the fact that the proposed offset correction is highly dependent on the affine approximation of the RPC functions.

Category 3. We run the open-source stereo pipeline S2P [11] to reconstruct 10 manually selected image pairs. The pairs are selected following the heuristic criterion that won the 2016 IARPA multi-view stereo challenge using S2P [START_REF] Facciolo | Automatic 3D reconstruction from multi-date satellite images[END_REF]. All pairs are reconstructed using bundle adjusted RPCs and they are merged into a single surface model by taking the median altitude along the z-axis. It should be noted that the MVS surface models in Figures 8 and 9 present incomplete regions (in black) beyond the water zones (also in black), which we ignore to compute the altitude MAE. As explained in Figure 9, for a fair comparison, we remove surface points non-observed by the majority of cameras before computing the altitude error. 4. Ours, raw RPCs+BA 31.76 / 1.37 31.72 / 1.55 33.28 / 1.30 32.25 / 1.41 5. Classic MVS (MGM) -/ 1.40 -/ 2.48 -/ 1.34 -/ 1.74 6. Learned MVS (PSM) -/ 1.15 -/ 1.44 -/ 1.27 -/ 1.29 Table 3. Numerical results, IARPA2016 areas.

IARPA2016 areas -Results and discussion

Figure 9 shows the DSMs of some IARPA2016 challenge areas obtained with EO-NeRF. We repeat the same categories of experiments of Section 4.2. Only raw pansharpened images are available in this case. The numerical results in the different rows of Table 3 are discussed below.

Category 1. We test Sat-NeRF using UTM-based point coordinates and N = 128 points per ray (row 1). Consistent with the DFC2019 results, the model does not fit nonradiometrically normalized inputs and the altitude MAE is above 2 m in all areas. A detailed view of the altitude provided by Sat-NeRF in area 003 is shown in Figure 1.

Category 2. We test EO-NeRF using externally bundle adjusted and internally bundle adjusted RPC camera models (row 2 and 4 respectively). When the input RPCs are consistent, because of the external or the internal bundle adjustment, EO-NeRF outperforms Sat-NeRF by about one meter of altitude in average. This advantage is lost if the unrefined camera models are used instead (row 3).

Category 3. As in Section 4.2 (Category 3), we run the stereo pipeline S2P using 10 manually selected image pairs and MGM and PSM for dense matching. Again, EO-NeRF achieves better overall altitude MAE with respect to MGM and PSM. The DSMs in Figure 9 show the superior level of detail in the shapes reconstructed by EO-NeRF. Note that the geometry of narrow and irregular structures, such as the arches or roller coasters in the area 003 (top row) is only visible in the lidar and EO-NeRF DSMs.

Conclusion

We presented EO-NeRF, a variant of NeRF for Earth observation adapted to multi-date satellite images. Our method achieves state-of-the-art novel view synthesis and digital surface modeling from this kind of data. Using input views with a resolution of 30 cm/pixel, EO-NeRF elevation models reveal scene geometry with a level of detail comparable to intrusive aerial acquisitions.

EO-NeRF improves on previous concurrent NeRF variants mainly thanks to a geometrically consistent rendering of shadows that does not modify the loss function and the addition of network parameters to handle raw satellite images. The fine-scale details recovered by EO-NeRF are lost in modern multi-view stereo pipelines for satellite 3D reconstruction, which are not adapted for multi-date inputs. Classic matching algorithms are usually subject to strong regularization, whereas deep neural networks may have generalization problems. While there is much room for improvement, EO-NeRF demonstrates that neural rendering has the potential to take Earth observation to another level.

Figure 2 .

 2 Figure 2. Stereovision methods used in modern production chains for 3D reconstruction from multiple satellite images require control of a multitude of stages, each of which can induce fatal errors. EO-NeRF replaces the entire chain with a single, more practical stage.

Figure 3 .

 3 Figure 3. Left to right: (a) Input image, (b) albedo, (c) geometric shadows, (d) transient scalar, (e) uncertainty scalar, (f) altitude, (g) albedo with geometric shadows irradiance, (h) albedo with geometric shadows irradiance after the affine transformation defined by [Aj, bj], (i) albedo with geometric shadows and transient objects irradiance after the affine transformation, i.e., the rendered color c(r) from (7).

Figure 3

 3 Figure 3 illustrates the different outputs of EO-NeRF. According to (7), the output color c(r) results from the product of the albedo and an irradiance model ℓ(r), subject to an image-specific affine transformation defined by A j and b j . Affine correction models are a common practice in the literature for relative radiometric normalization of multi-date satellite imagery [20, 50]. The key element in the irradiance model (8) is s(r), a scalar in the interval [0, 1]. Low values of s(r) correspond to points in the shade or involving transient objects, where the ambient color a is allowed to be stronger. High values of s(r) correspond to points of the scene that can be fully explained by the albedo color. S-NeRF and Sat-NeRF used irradiance models similar to (8), but the way s(r) is defined in EO-NeRF is critically different as explained in Section 3.1.

Figure 4 .Figure 5 .

 45 Figure 4. Left to right (in each set): output RGB, albedo and irradiance ℓ(r) (8). Predicted shadows, as in Sat-NeRF or S-NeRF, can correctly interpolate within the range of solar directions observed in the input images. However, they produce unrealistic results outside this range. Top row: Solar direction taken from the input data. Bottom row: Solar direction perpendicular to the sun path, which could never be observed in a real satellite image of the area. The geometrically consistent shadow model used in EO-NeRF can realistically render arbitrary solar directions as that of the bottom row and simultaneously improve the geometry estimation.

  -NeRF network architecture, with the parameters to be optimized delimited by boxes. Multi layer perceptron components are shown in blue and embedding vectors are shown in red.

Figure 7 .

 7 Figure 7. ECEF vs. UTM-based representation of geographic 3D point coordinates. The scene has an altitude range of [hmin, hmax]. Using the same RPC camera, the rays in blue originate at different pixels, localized at hmax, and end at the same pixel, localized at hmin. All coordinates are normalized in the interval [-1, 1].

Figure 8 .

 8 Figure 8. DFC2019 areas. Top to bottom: 004, 068, 214, 260.DSM resolution: 50 cm/pixel. For each method, the DSM that achieved the best altitude MAE is chosen between true color RGB and raw pansharpened inputs.

Figure 9 .

 9 Figure 9. IARPA2016 areas. Top to bottom: 003, 002, 001. DSM resolution: 30 cm/pixel. Close-up views at the bottom: (a) full EO-NeRF prediction, (b) EO-NeRF prediction at points observed by more than 3/4 of all input cameras, (c) PSM prediction. Similarly to the left-right consistency check in stereo methods such as PSM, (b) rejects frequently occluded points near object borders. The altitude MAE is computed using the surface points in (b).

Table 1 .

 1 Number of input images used for each area, altitude bounds of the scene and approximate latitude and longitude.

	DFC2019 data -Location: Jacksonville (United States)
	Area index	004	068	214	260
	Input images	9	17	21	15
	Alt. bounds [m]	[-24, 1]	[-27, 30]	[-29, 73]	[-30, 13]
	Latitude	30.357	30.348	30.316	30.311
	Longitude	-81.706	-81.663	-81.663	-81.663
	IARPA2016 data -Location: Buenos Aires area (Argentina)
	Area index		001	002	003
	Input images		25	21	21
	Alt. bounds [m]		[12, 59]	[12, 80]	[12, 80]
	Latitude		-34.490	-34.447	-34.417
	Longitude		-58.584	-58.575	-58.575

Table 2 .

 2 Numerical results, DFC2019 areas. Sat-NeRF fails to handle the raw pansharpened images, while MVS methods struggle with the less textured true color RGB images. EO-NeRF achieves the best overall MAE with no major differences between the two input types.

			True color RGB (uint8)			Raw pansharpened (float32)
			PSNR ↑ / Alt. MAE [m] ↓			PSNR ↑ / Alt. MAE [m] ↓		Mean
	Area index	004	068	214		260	004	068	214	260
	1. Sat-NeRF (N = 64, h = 256, UTM)	29.94 / 1.34	27.72 / 0.94	26.86 / 1.90	26.91 / 1.70	38.67 / 3.02	33.74 / 1.24	31.84 / 2.53	35.97 / 2.43	31.46 / 1.89
	2. Sat-NeRF (N = 128, h = 256, UTM)	28.89 / 1.48	27.18 / 0.98	26.08 / 2.29	27.75 / 1.76	38.59 / 2.64	32.80 / 1.39	30.89 / 2.38	34.66 / 2.37	30.86 / 1.91
	3. Ours (N = 128, h = 256, UTM)	28.56 / 1.25	27.25 / 0.91	26.59 / 1.52	26.09 / 1.43	38.13 / 1.33	33.33 / 0.89	31.48 / 1.41	34.77 / 1.34	30.78 / 1.26
	4. Ours, raw RPCs	27.78 / 1.39	26.11 / 1.42	25.62 / 1.74	25.07 / 1.95	37.55 / 1.72	31.54 / 1.08	30.21 / 1.58	32.08 / 1.53	29.49 / 1.55
	5. Ours, raw RPCs+BA 27.93 / 1.30 27.27 / 0.90	26.36 / 1.38	25.75 / 1.48	37.56 / 1.68	32.88 / 0.95	31.43 / 1.44	33.89 / 1.36	30.38 / 1.31
	6. Classic MVS (MGM) -/ 1.97	-/ 1.71	-/ 2.49	-/ 2.16	-/ 1.37	-/ 1.17	-/ 1.81	-/ 1.64	-/ 1.79
	7. Learned MVS (PSM)	-/ 2.28	-/ 0.88	-/ 1.77	-/ 1.67	-/ 1.16	-/ 0.83	-/ 1.48	-/ 1.20	-/ 1.41
		Raw pansharpened (float32)
		PSNR ↑ / Alt. MAE [m] ↓	Mean
	Area index	001	002	003
	1. Sat-NeRF (N = 128, h = 256, UTM)	30.55 / 2.12 31.78 / 2.27 32.79 / 2.75 31.71 / 2.38
	2. Ours (N = 128, h = 256, UTM)	31.70 / 1.23 31.71 / 1.43 33.07 / 1.19 32.16 / 1.28
	3. Ours, raw RPCs	29.68 / 2.07 29.78 / 2.39 30.97 / 1.99 30.14 / 2.15

Acknowledgements. Work partly financed by Office of Naval research grant N00014-17-1-2552, MENRT, and Kayrros. It was also performed using HPC resources from GENCI-IDRIS (grants 2023-AD011014015 and AD011011801R3) and from the "Mésocentre" computing center of CentraleSupélec and ENS Paris-Saclay supported by CNRS and Région Île-de-France (http://mesocentre.centralesupelec.fr). Centre Borelli is also with Université Paris Cité, SSA and INSERM.


Cloud GA Grant Agreement of FAIR-IMPACT KPI Key Performance Indicator PID Persistent Identifier SF Synchronisation Force

Introduction

Building on the successful Synchronisation Force approach from the FAIRsFAIR project 1 (2019-2021), FAIR-IMPACT continues a dialogue for collaboration and harmonisation with various projects, initiatives, and actors in both EOSC and FAIR ecosystems. We do this to reduce redundancy and to ensure that solutions are more widely promoted, sustainable and can be transferred to the relevant EOSC Partnership. This supports current and future EOSC stakeholders to take the next step in implementing FAIR-enabling practices.

To address these challenges FAIR-IMPACT set up the Synchronisation Force with representatives from all of the project's work packages. The main instrument of the Synchronisation Force is a series of three annual workshops to be delivered in the period of 2022-2024. Key representatives of projects and initiatives in the FAIR and EOSC ecosystem are selected and invited (see Image 1).

Image 1: FAIR-IMPACT's landscape of key stakeholders

1 FAIRsFAIR https://fair-impact.eu/fairsfair-legacy This landscape for synchronising consists of the Board of Directors of the EOSC Association and a selection of Task Forces under the EOSC Association that are most relevant for the FAIR-IMPACT focus areas (top-left). FAIR is also in the remit of European projects, especially those in the HORIZON-INFRA-EOSC funding scheme, but likewise in ESFRI Cluster projects, the EOSC Technical Core, as well as in discipline-independent providers (right hand side). Finally, representatives of Open Science initiatives (bottom-left) were invited to the workshop 2022.

To set the stage, four topics were defined, which fit the FAIR-IMPACT core areas. Each topic focused on selected recommendations and ambitions from the FAIRsFAIR White Paper 2 (2021), the SRIA 3 (version June 2021) and EOSC Multi-Annual Roadmap 4 (2023-2024). Whereas the FAIRsFAIR Synchronisation Force workshops and White Paper were inspired specifically by the Turning FAIR into Reality Report 5 (2018), we now added more recent strategic documents.

Based on the workshop input and discussions, this report provides supporting recommendations for each topic.

Description of the Milestone

The Synchronisation Force workshops 2022-2024 bring together the various projects and actors to periodically assess the work undergoing around the FAIR-IMPACT focus areas. This Milestone concerns the 2022 workshop.

Role of the milestone

The Milestone indicates that FAIR-IMPACT, in particular Task 1.3 Synchronisation with EOSC Partnership and relevant projects and initiatives, organises an annual workshop to keep track of FAIR-related developments in a wide range of EOSC initiatives, and also to inform representatives of these initiatives about such developments.

Means of verification

Information about the Synchronisation Force workshop 2022 is available -at the project website 6 : programme and slide decks; The Synchronisation Force workshop 2022 consisted of six online sessions, between 8 November and 12 December 2022:

-an introduction to FAIR-IMPACT, the goal of the workshop and the request to all workshop participants to provide information about their FAIR activities ahead of the four thematic sessions;

-session on Metrics and assessing FAIRness -session on Persistent Identifiers -session on Trustworthy and FAIR-enabling repositories -session on Metadata, semantics and interoperability -a final session in which highlights and recommendations from the thematic sessions were presented and discussed with the participants.

More than 120 people registered for the series; attendance in individual sessions ranged from 40 to over 60 people.

Proof of Milestone fulfilment as per the respective GA table

The verification method for the fulfilment of the milestone is this report.

Highlights and recommendations from the Milestone

Metrics and assessing FAIRness

Underlying recommendation/ambition: "Provide the metrics and tools to measure the adoption of the FAIR principles for research outputs." (Operational Objective 6 from EOSC Multi-Annual Roadmap7 (2023-2024), p.15)

Recommendations based on the Synchronisation Force workshop 2022 session

The session on Metrics and assessing FAIRness was well attended with around 40 participants in the virtual room. These participants represented a large variety of projects, initiatives, as well as the EOSC Association Task Forces and the European Commission. This resulted in a lively conversation and useful information exchange.

Already in 2018, the Turning FAIR into Reality report recommended the development of metrics for FAIR digital objects and suggested a mix of automated and manual assessments.

The session showed that in 2022 the importance of making research outputs FAIR is now widely on the radar. This is an area in which quite some progress was made over the last years, resulting in an abundance of different outputs.

During the session several issues were raised. We currently witness a plethora of different FAIR assessment tools. These tools are based on different metrics. They use different methods and weighing factors and run different tests to produce an outcome. The result of this is a very complex landscape that is difficult to navigate for the end user and other interested stakeholders.

The current assessment tools are mainly generic. If domain specificity is missed, the results from the tools are less meaningful. Although the FAIR assessment of all research outputs is recommended, the current assessment tools mainly focus on (meta)data. Finally, the session showed that due to the issues mentioned above, a lot of caution is needed not to use assessment results as absolute numbers to judge upon. Numbers need a narrative to assist organisations in taking next steps on the journey towards FAIR.

The following recommendations were suggested in the session to address the issues mentioned above:

• We need to work on a further convergence of metrics and tools, which requires further discussion, synchronisation and alignment;

• We need more domain-sensitive assessment methods, in order to incorporate domain maturity as well as specific good practices and requirements.

• We need assessment tools for other research outputs, like software and semantic artefacts.

• The instrument of FAIR assessment and scoring should be seen and used as the starting point for assistance and improvement.

Shortly after the workshop the EOSC-A Task Force FAIR Metrics and Data Quality 8 produced the report Community-driven Governance of FAIRness Assessment: An Open Issue, an Open Discussion 9 . The Task Force states that "FAIRness is "stuck" between an increasingly common research and publishing requirement yet still an unmeasurable set of ideals." This statement aligns with the findings of the Synchronisation Force session. Their recommendations also show considerable overlap with the recommendations from the session. This provides a solid basis to work together in the coming years to improve the current set of tools to be able to assess FAIRness of research outputs in a transparent and consistent way.

Persistent Identifiers

The EOSC Persistent Identifier (PID) policy10 defines a set of expectations about what persistent identifiers will be used to support a functioning environment of FAIR research in EOSC. In the Synchronisation Force workshop session the key concepts of the EOSC PID policy were discussed, as well as how PID policies and implementations currently look in different contexts. Across different communities there is a wide range of identifiers in use, not all of which necessarily qualify as PIDs according to the EOSC PID Policy definition. Analysis and discussion is recommended to find which extant identifiers are or should be considered emerging PIDs.

Underlying recommendation/ambition

In this context communities should make recommendations on PID use and describe their use cases, as this helps to create a shared understanding and can contribute to a shared language. Additionally, participants in the concluding workshop session strongly recommended that PID systems themselves should be sustainable.

More than half of the participants indicated that their organisation has no PID policy or that they did not know if their organisation has one. It is recommended that all stakeholders develop explicit PID policies, either as separate documents or as part of e.g. a data policy. FAIR-IMPACT will support this by providing PID policy templates.

A few recommendations emerged from the session to address the gaps and challenges identified.

• We need to discuss and illustrate the roles identified in the EOSC PID Policy.

• Contracts and documentation should be aligned with the EOSC definitions of the roles.

• We should analyse which of the various identifiers in use should or could be considered PIDs in the EOSC context. The sustainability of PID systems should be taken into account.

Trustworthy and FAIR-enabling repositories

According to the Turning FAIR into Reality report, depositing research data with Trustworthy Digital Repositories (TDRs) and, where possible, certified repositories is crucial for realising a FAIR ecosystem.

Underlying recommendations/ambitions:

"Percentage of the repositories in EOSC that will have a certification such as CoreTrustSeal is 30% by 2025." (KPI for SRIA12 objective "Establish a sustainable and federated infrastructure enabling open sharing of scientific results", p.148)

"Provide continuous guidance and assistance to small repositories to engage with certification processes. (...) If the federated data layer is to include small repositories, which are important in a substantial range of domains and geographies, guidance, support and capacity building for these repositories is also required." (Recommendation 5 from FAIRsFAIR White Paper13 , p.12)

Recommendations based on the Synchronisation Force workshop 2022 session

The TDR session had more than sixty participants including representatives from the EOSC Association and Task Forces, COAR, INFRA-EOSC projects, repositories, and research performing organisations.

The session showed that there is a good deal of current activity related to developing and supporting networks of TDRs to share experiences, including work being carried out by the EOSC Task Force on long term digital preservation, ENVRI-FAIR Task Force Triple Stores and data storage certification, and CESSDA (Consortium of European Social Science Data Archives)14 community support. There is also activity at the national level such as the work being done by Research Data Alliance (RDA) in France15 to provide training and guidance as well as financial support for self-assessment and review of CoreTrustSeal (CTS) 16 . Research Infrastructures are also active in undertaking work relating to trustworthiness. For example, ELIXIR17 provides badges to core resources that are of key importance to the life-science community, which could be considered in the broader context of trustworthiness.

During the session, some key issues were raised. The first is that the process of preparing for certification can be more valuable than achieving certified status. Participants also stressed that there are different types of certification available and also other ways to become more trustworthy. Rather than pushing for a single certification route, it is better to allow repositories and their user communities to co-determine the best route for their needs. Either way, transparency is crucial: service users' trust is based on the clarity of information that the repository or data service provider presents. Services like re3data.org18 and FAIRsharing19 help to make such information about the repositories and their FAIR-enabling capabilities more visible. In a similar vein, CoreTrustSeal certification requires explicit, public information from the repository.

Several gaps and challenges were identified during the session. We need to make clear what is meant by the terms 'large' and 'small' repositories, which are often used. These terms were not seen as helpful by the session participants who suggested that it may be more useful to focus on resource levels and scope of the repository. The ongoing work being carried out by the EOSC Task Force on long term preservation work can be useful in this respect. Financial and skills support is crucial for carrying out self-assessments and the availability of resources to help guide the process. In particular, there is a need for beginner level support and help for those repositories with fewer resources to understand the potential benefits that may be realised through certification, the process(es), and how to become more trustworthy and FAIR-enabling. As noted above, we should avoid a single certification route but rather embrace multiple routes to demonstrating trustworthiness. To this end, mappings between CoreTustSeal and domain-specific certification processes could be valuable. The EOSC Task Force on Long Term Data Preservation 20 recommendations (in progress) of a European network of FAIR-enabling, trustworthy repositories could also help address this. Participants felt there may be potential to reuse the RDA Data Repository Attributes Working Group 21 set of minimal, common attributes/criteria for data repositories for such mappings. The perceived lack of legitimacy of certification bodies was also considered a challenge that must be addressed. Finally, participants stressed that we must aim to collaborate globally rather than just across Europe. Bodies such as COAR and their members must be part of these discussions.

A few recommendations emerged from the session to address the gaps and challenges identified.

• We must focus on making a wider range of aspects relating to trust transparent rather than just focusing on achieving certified status.

• There must be cooperation across the current initiatives to build and sustain a network of TDRs -not just in Europe but globally.

• An incremental approach to adoption of good practices is what we should be striving for and we should build on previous work to support this such as COAR's Community Framework for Good Practices in Repositories 22 .

• More sustainable support is needed for repositories to become trustworthy and/or certified and there is potential to replicate the national approach being implemented in France through RDA.

Metadata, semantics and interoperability

The session on semantic artefacts -a broader term to include ontologies, terminologies, taxonomies, thesauri, vocabularies, metadata schemas and standards -had nearly sixty participants. Semantic artefacts are essential for supporting semantic interoperability, which in turn is essential for a functioning EOSC. Multiple scientific communities were surveyed: "Further develop and implement semantic technologies, particularly in domains where their use is less advanced." (Recommendation 2 from FAIRsFAIR White Paper24 , p.9)

Recommendations based on the Synchronisation Force workshop 2022 session

The session started by introducing a few definitions, based on previous analysis and common legacy of the FAIRsFAIR project:

• Semantic artefacts: a broader term to include ontologies, terminologies, taxonomies, thesauri, vocabularies, metadata schemas and standards (Legacy of FAIRsFAIR and adopted in the EOSC Interoperability Framework)

• Semantic artefact catalogues: encompass any existing ontology repositories, registries, vocabulary/terminology services and metadata schemas catalogues.

• (Semantic) Crosswalks and mappings: formal links between the content of these semantic artefacts.

The main outcomes of the discussion revolve around the following aspects. For what concerns development, use and governance of semantic artefacts, every disciplinary community has its own semantic artefacts -thesauri, ontologies -which usually look very discipline oriented. Some domains lack semantic artefacts whereas other domains, such as Social Science and Humanities, are so large that they apply different semantic artefacts. There are also overlapping semantic artefacts across domains, thus the need for crosswalks and mappings.

In general there are a lot of differences in data types, data collection, theories, and methods per domain. In some cases semantic artefacts are also managed by international alliances but there is no global governance or coordination although different scientific domains clearly demonstrated different levels of maturity.

Limitations in the usage of semantic artefacts are quite common across disciplines and cases. They range from findability issues, quality/FAIRness and curation of the artefacts, to lack of governance and strategy, a lack of guidance on which ones to use, and long-term availability and maintenance. Sometimes multilingualism is also an issue.

Several semantic artefact catalogues exist and can help address some of the challenges related to semantic artefacts (governance, findability, FAIRness, mappings, etc.) BioPortal When dealing with crosswalks and mappings, the general feeling is that here is where work still needs to be done. Some tools are emerging, some mappings are available, for instance in the Astronomy community. There are already quite a few best practices and use cases to look at to build reference crosswalks, like the SSSOM initiative testes by the Biology/biomedical community, but there is still a lack of shared strategies about how to deal with crosswalk and mappings between semantic artefacts in different domains.

A few recommendations emerged from the session to address the gaps and challenges identified.

• More cross-disciplinary work is needed to align semantic artefacts with the same terms or concepts.

• Maintenance, sustainability, and governance of semantic artefacts deserve attention and agreement across disciplinary communities.

• The FAIR-at-large community should intensify the work on crosswalks and mappings to produce more best practices.

• Recommended practices should be shared and collected.

Conclusions and next steps

The workshop was delivered according to plan and successfully brought together many participants from different EOSC and FAIR initiatives. Several of them had participated in the earlier FAIRsFAIR Synchronisation Force workshops (2019-2021), and likewise, several participants indicated they look forward to the FAIR-IMPACT Synchronisation Force workshop 2023. The collected information is available from the project website 32 as well as published on Zenodo (see Appendices). It will help the various work packages and project partners to identify the current state of FAIR developments more broadly.

Appendices

Underlying materials

Available in the FAIR-IMPACT community in Zenodo 33 :

• Data provided by workshop participants in the collaborative spreadsheet 34 (separate spreadsheets per session) The collaborative notes from three thematic sessions are available from the project drive, but without long-term commitment. Because the PID session consisted mainly of polling and breakout activity, there are no session notes.

2

  FAIRsFAIR White Paper https://doi.org/10.5281/zenodo.5744786 3 SRIA https://www.eosc.eu/sites/default/files/SRIA_2022_01.pdf 4 EOSC Multi-Annual Roadmap https://eosc.eu/sites/default/files/2022-05/20220523_MAR_02_GL.pdf 5 Turning FAIR into Reality https://doi.org/10.2777/1524 6 FAIR-IMPACT https://fair-impact.eu/events/synchronisation-force-events/synchronisation-force-1stworkshop-november-2022 -in Zenodo (see Appendices for the links): a.o. the spreadsheet with input provided by workshop participants.

  Session featured panellists: 1. Biomedicine: Nicolas Matenzoglu & Pier Luigi Buttigieg 2. Ecology/biodiversity: Naouel Karam & Ilaria Rosati 3. Agri-food: Clement Jonquet 4. Social sciences & humanities: Arnaud Gingold 21 RDA Data Repository Attributes Working Group https://www.rd-alliance.org/groups/data-repositoryattributes-wg 22 COAR Framework https://www.coar-repositories.org/files/COAR-best-practices-framework-for-repositories-Version-2-July-19-2022.pdf 5. Industry: Hedi Karray 6. Astronomy: Baptiste Cecconi 7. Earth Sciences: Jean-Christophe Desconnets, V. Agazzi, Christelle Pierkot Underlying recommendations/ambitions "Develop domain and cross-domain interoperability frameworks at the level of vocabularies, ontologies, and metadata schema." (Recommendation 1 from FAIRsFAIR White Paper 23 , p.8)

  32 FAIR-IMPACT Synchronisation Force https://fair-impact.eu/synchronisation-force 33 FAIR -IMPACT Zenodo community https://zenodo.org/communities/fair-impact/ 34 Workshop spreadsheet https://doi.org/10.5281/zenodo.7457122 35 Opening session https://doi.org/10.5281/zenodo.7318689 36 Session on Metrics and assessing FAIRness https://doi.org/10.5281/zenodo.7446410 37 Session on PIDs https://doi.org/10.5281/zenodo.7457304 38 Session on Trustworthy and FAIR-enabling repositories https://doi.org/10.5281/zenodo.7446757 39 Session on Metadata, semantics and interoperability https://doi.org/10.5281/zenodo.7446806 40 Concluding session https://doi.org/10.5281/zenodo.7446827

  25 , AgroPortal 26 , OBO Foundry 27 , GFbio Terminology Service 28 , Research Vocabularies Australia 29 , NERC Vocabulary Server 30 , and FAIRSharing 31 were mentioned as examples. They feature different levels of services: from simple metadata description libraries to complete repositories supporting the content of the semantic artefacts in addition to their metadata. Agri-food and ecology/biodiversity, both identified as use cases in FAIR-IMPACT, seem to have a stronger awareness about the catalogues of reference. Other communities, such as Astronomy, Earth Science, and Social Sciences and Humanities cope with different levels of maturity of catalogues.

  • Slides from opening session 35 • Slides from 'Metrics and assessing FAIRness' 36 • Slides and polling results from 'PIDs' 37 • Slides from 'Trustworthy and FAIR-enabling repositories' 38 • Slides from 'Metadata, semantics and interoperability' 39 • Slides from concluding session 40

EOSC Multi-Annual Roadmap https://eosc.eu/sites/default/files/2022-05/20220523_MAR_02_GL.pdf

Taskforce FAIR Metrics and Data Quality https://www.eosc.eu/advisory-groups/fair-metrics-and-data-quality

Community-driven Governance of FAIRness Assessment https://doi.org/10.5281/zenodo.7390482

EOSC PID policy https://data.europa.eu/doi/10.2777/926037

EOSC Multi-Annual Roadmap https://eosc.eu/sites/default/files/2022-05/20220523_MAR_02_GL.pdf

SRIA https://www.eosc.eu/sites/default/files/SRIA_2022_01.pdf

FAIRsFAIR White Paper https://doi.org/10.5281/zenodo.5744786

CESSDA https://www.cessda.eu/

RDA in France https://grants.rd-alliance.org/national-nodes/rda-france

CoreTrustSeal https://www.coretrustseal.org/

ELIXIR https://elixir-europe.org/

Re3data https://www.re3data.org/

FAIRsharing https://fairsharing.org/

EOSC Task Fore on Long-Term Data Preservation https://www.eosc.eu/advisory-groups/long-term-datapreservation

FAIRsFAIR White Paper https://doi.org/10.5281/zenodo.5744786

FAIRsFAIR White Paper https://doi.org/10.5281/zenodo.5744786

BioPortal https://bioportal.bioontology.org

AgroPortal https://agroportal.lirmm.fr

OBO Foundry https://obofoundry.org

GFbio https://terminologies.gfbio.org

Research Vocabularies Australia https://vocabs.ardc.edu.au

Nerc http://vocab.nerc.ac.uk

FAIRSharing https://fairsharing.org

• Notes from 'Metrics and assessing FAIRness' 41 • Notes from 'Trustworthy and FAIR-enabling repositories' 42 • Notes from 'Metadata, semantics and interoperability' 43

Participant list

The 120 workshop participants represent the following organisations: