Guillaume Colas
email: guillaume.colas@femto-st.fr

Romain Petre-Bordenave
email: romain.petrebordenave@cnes.fr

Michel Yann
email: yann.michel@cnes.fr

Pierre-Henri Cornuault
email: pierre-henri.cornuault@ens2m.fr

FRETTING WEAR ANOMALY OF FINAL CUBESAT EJECTION TESTS: EXPERTISE, SOLUTION TESTING, AND LESSONS LEARNT

ABSTRACT

During the qualification campaign of the satellite (PFM), vibration testing of the ANGELS cubesat in its deployer was performed at PFM levels. A deployement test was then carried on and showed that the satellite was stuck inside the deployer. Investigations quickly focused on a tribological issue because wear marks and particles were found after extraction of the cubesat. Equivalent friction coefficient calculations from sat/deployer ejection tests led to µ>0.9 which suggested sticking or gripping phenomena.

A fretting test campaign carried out at tribometer scale demonstrated that wear resulted in accumulation of debris inside the contact. That resulted in the creation of a very adhesive 3 rd body. Gentle cleaning and manual deposition of MAPLub PF100b grease revealed to efficiently lubricate the damaged contact. The PTFE particles embedded in the grease accumulated in the damaged areas to created, in situ, a thick lubricious layer made of PTFE based materials. The solution was successfully applied on the cubstat/deployer. Tests in representative conditions with representative materials/coatings shall be made as soon as possible during development in order to avoid late and critical anomalies.

Context

ANGELS (Argos-Neo on a Generic Economical and

Light Satellite) is a 12U satellite designed to demonstrate a new and more economical solution for ARGOS systems. ANGELS was launch on the 18 th of December 2019 using a soyouz launcher and a cubesat deployer.

During the qualification campaign of the satellite (PFM), vibration testing of the cubesat in its deployer was performed at PFM levels. A deployement test was then carried on and showed that the satellite was stuck inside the deployer. 9 different possible root causes were identified: tribological issue, residual deformations of satellite/deployer, geometric non-conformance of satellite/deployer, etc. Investigations quickly focused on a tribological issue because wear marks and particles were found after extraction of the cubesat. These marks were found on both satellite rails and deployer rails. Both CNES and FEMTO-ST started to investigate right after these observations in July 2019. First of all, one should note that all the tests and investigations were constrained by the time available to find a solution and by the fact that the deployer and the cubesat had to be maintained in a clean room.

Visual inspection and particle analysis

CNES teams were send to perform visual inspections and particles sampling (see Figure 2).

Figure 2: Sampling areas

During visual inspections, we noticed that at least 3 areas were subjected to abnormal wear and a lot of particles were present. Two of these areas are located near switches of the deployer rails. The corresponding areas on the satellites rails exhibits also a lot of wear.

Figure 3: Wear and particles near a switch

Particles were sampled using carbon adhesive on SEM stubs and were analysed using SEM and EDX. The results showed that the particles were mainly composed of alumina. Traces of Ziconium, Titanium and fluor were also found. On satellite side the rails are made of Al7075 treated by hard anodic oxidation and surtec 650. On the deployer side the rails are made of Al7021 treated by Hard anodic oxidation and interflon. The composition of the particles is thus consistent with the materials in contact.

Figure 4: Example of an EDX analysis

At this stage, the hypothesis of an external contamination acting like a third body in the contact and increasing the wear was excluded. All the particles came from the contact itself and the third body is not an external contamination but the product of the wear itself.

Topological analysis of the wear zones

After the particle sampling, the interest areas were cleaned using isopropyl alcohol. The wear areas were replicated using a silicone resin. In the following pictures, be aware that the analyses were performed on a negative of the surface. A positive step on the measurement corresponds to a negative step on the real surface.

Figure 5: Example of a silicone replica on the deployer rail

The wear area on the Figure 5 shows almost no wear. The mean surface in the area is 800nm above the original mean surface. We also notice that the surface in the wear area is smoother than the original surface. The same analysis performed on the satellite rails exhibited more than 7µm of wear in average. Wear areas are well defined and it is not hard at all to define a frontier between the wear and the original surface. Moreover the bearing surface at 7µm of the satellite tracks if 100%.

Meaning that the bearing surface can no longer increase with wear (locally at least) and that the wear phenomenon is not behaving like a run-in but will continue without any counter-measures.

It is worth mentioning also that the waviness and roughness of the deployer were higher than the ones of the satellite. To understand the origin of the anomaly, a fretting test campaign has been carried out on a TE77 Cameron-Plint tribometer. The objective was to reproduce the wear phenomena (particles and surface morphologies) on elementary scale tests. The tribometer scale study is summarized here. The full study is presented in [START_REF] Colas | Fretting wear_Lubrication of initially worn parts through tribo-induced anchorage of PTFE in wear scars, and tribochemical creation of graphene oxide[END_REF].

Tribo elementary tests

Contact conditions and post-tests analysis

Samples emulating both the satellite and deployers are comprised of the exact same materials than the satellite and deployers themselves. The sample emulating the satellite is a cylinder of 8mm in diameter and 20mm in length. Chamfer at both ends reduces the end diameter to 7mm. It is the end disc that is put in contact with the cubic sample emulating the deployer. The cubic sample is 14mm x 14mm x 12mm. Sliding occurs on the 14x14 surface. Contact condition are presented in Table 1. They have been determined after following a first test campaign during which different set of parameters were screened. Note that the difference in humidity compared to the clean room may not affect our results considering literature is not demonstrating any evidence of it [START_REF] Cai | Torsional fretting wear behaviour of 7075 aluminium alloy in various relative humidity environments[END_REF][START_REF] Zhang | Comparative study on the fretting wear property of 7075 aluminum alloys under lubricated and dry conditions[END_REF].

To reproduce at best the vibration test sequence and the anomaly, we started with dry contact configuration. Once the test is over, the contact is opened, samples are cleaned off particles and debris, lubricant is applied, the contact is closed, and the test sequence is run again. In order to have a detailed look at the wear mechanisms, 3 samples were submitted to the dry conditions only, 3 we submitted to both sequence (dry + lubricated). Two lubricants were tested:

-MAPLbub PF100b because it can be applied manually on parts. At contact closing the exess of grease is ejected at the periphery of the contact, which forms a bulged hallo around the cylinder on the surface of the cubic sample -Lubricants based on alkyl-phosphonic acids molecules because they are dispersed in ethanolbased solvent, and applied by spraying the solution on parts. Within few minutes the solvent evaporated, leaving the molecules organizing themselves on the surface. Such lubrication is considered as dry lubrication

After the tests, samples are analyzed using optical profilometry to analyzed the wear scares and determine wear depth and thicknesses of the 3 rd body materials accumulating inside the contact area. Then follows more detailed study of the morphology and the elemental composition of the worn surfaces (including debris) with SEM/EDS analysis. Regarding grease lubricated samples, grease is removed for the analysis and the remaining 3 rd body is analyzed. Infrared analysis is also performed to determine the nature of 3 rd body created during lubricated tests.

Results

Figure 9 allows to compared the aspects of the worn surfaces on the satellite and on the tribometer scale samples, after tests in dry conditions. Features, morphology (surfaces and debris) are successfully reproduced. Contact effectively occurs over few large spots. Over the 3 unlubricated test samples, the damaged area remains quite constant and varies little from one test to another. The optical analysis of the unlubricated samples shows that the worn samples is mainly the samples emulating the satellite. It also and that the 3 rd body accumulate to form thick 3 rd body patches. Figure 11 indeed shows that the cubic sample is not demonstrating significant wear, the surface at the centre of the contact is slightly smoother than the pristine surface. At the mostly loaded areas, 3 rd body accumulates to create patches reaching a thickness of 14 µm above the overall sample surface. The cylinder surface (not shown) demonstrate mostly few microns deep worn region. Areas with accumulated 3 rd body demonstrate 3 rd body whose thickness may locally reach 2 µm. EDS analysis shows that debris and 3 rd body are only coming from the cylinder (satellite-based material) [START_REF] Colas | Fretting wear_Lubrication of initially worn parts through tribo-induced anchorage of PTFE in wear scars, and tribochemical creation of graphene oxide[END_REF]. Note that no fluorine is detected on the cubic samples (layer too thin?) but it on the cylinder material. Surtec650 is indeed containing fluorine [START_REF] Qi | Chromium valence change in trivalent chromium conversion coatings on aluminium deposited under applied potentials[END_REF]. The oxygen content is however significantly higher than what it is detected on the pristine materials, which is typical of tribo-oxidation seen during fretting tests in air [START_REF] Cai | Torsional fretting wear behaviour of 7075 aluminium alloy in various relative humidity environments[END_REF][START_REF] Merstallinger | Cold welding in hold down points of space mechanisms due to fretting when omitting grease[END_REF].

Figure 11 -Optical topography image and height distribution of the 3rd body.

The alkyl-phosphonic acid lubricated case shows that lubrication is sustained over half the test duration, which removed the lubricant from the list. This lubricant may however remain interesting to investigate as it showed to create lubricious graphene oxide base 3 rd body [START_REF] Colas | Fretting wear_Lubrication of initially worn parts through tribo-induced anchorage of PTFE in wear scars, and tribochemical creation of graphene oxide[END_REF].

Figure 12 -Optical topography image of the cylinder sample demonstrating large accumulation of PTFE based lubricious material in previously worn areas

The grease lubricated test sequence demonstrates very stable pure sliding friction with a friction force of 15N in average (0.1 friction coefficient). The optical topography shows accumulation of 3 rd body from coming from the grease and remaining rigidly bounded to the initially worn areas of the surfaces. This demonstrated on the mostly worn samples, the cylinder (Figure 12). Such PTFE layer formation behaviour has been shown to happen with grease containing PTFE particles [START_REF] Busquet | Space Grease Tribological Behavior for Reformulation : Numerical[END_REF]. While PTFE appears to bind only on the initially worn areas on the cylinder (i.e. on the satellite material), it is found everywhere in the contact on the cubic samples (i.e. deployer material), with slightly thickest patches in initially worn areas.

Pros / Cons between "Use-as-is" and grease lubrication solutions

Due to several ejection anomalies on different tests "dry lubricated" models (PFM and dummy) and to elementary tribological testing showing that fretting on satellite/deployer rails can lead to an adhesive third body, it was decided to apply grease on deployer rails. Indeed, equivalent friction coefficient calculations from sat/deployer ejection tests led to µ>0.9 (compared to µ<0.35 before vibration). Such a high value of friction coefficient is consistent with the high friction measured during the tribometer scale test campaign. That consequently confirm the sticking or gripping phenomena suggested on the cubsat/deployer system. MAPLub PF100b grease was selected due to its excellent thermal stability and the presence of PTFE particles within the grease (in case of oil segregation / migration).

Use

Grease lubrication validation at

satellite/deployer level

MAPLub PF100B validation on satellite dummy

In order to qualify this configuration on a dedicated and representative model of both satellite and deployer, a structural model of the satellite was manufactured and tested in the deployer spare model (flight configuration).

Both satellite dummy and deployer rails were cleaned meticulously with isopropyl alcohol. MAPLub PF100b grease was applied on deployer side as shown in Figure 13 with a particular attention on the amount of grease: groove of the rail was filled of grease and a dedicated tool was used to spread the rest of the grease (0.2mm film thickness) while removing lubricant excess (cf. Figure 13 C).

Satellite dummy was then inserted into the deployer with a particular care on Sat/deployer alignment in order to avoid removing greases from the lubricated surfaces (cf. Figure 14). Both satellite dummy and deployer QM were submitted to a full mechanical vibrations qualification sequence (@qual level for 2 minutes, cf. Figure 15). Grease behaviour was observed during vibrations on bottom rails through an opening in the deployer. A grease bulge was observed and confirmed a satisfying grease circulation in the contact (cf. Figure 16).

Figure 15 Deployer QM with satellite dummy on MID shaker

An ejection test was then performed at 75° (cf. Figure 17) in order to limit gravity effect on friction (with 1g compensation). A particular care has been paid in transporting the deployer on the ejection test GSE to avoid "unsticking" the satellite from the rails. Satellite dummy was successfully ejected from the deployer with satisfying margin. In addition, no wear or fretting marks were observed on both the satellite dummy and the deployer rails (cf. Figure 18). Grease was indeed observed on all sliding surfaces as expected. This successful test confirmed the choice of the corrective actions to solve ejection anomaly.

Satellite PFM ejection

Exactly the same procedure was applied on satellite PFM and deployer FM. ANGELS satellite was successfully launched and ejected from it deployer in December 2019.

Lessons learnt

A particular care shall be paid to any system function that depends on tribology because it is often a single point of failure. This is particularly important when motorisation margins are low because they are directly related to critical performances (in this case, ejection velocity shall be controlled and it was not possible to tune/increase ejection springs stiffness). Moreover, tribology issue is not only a matter of friction coefficient and some degradations of coating in contact might lead to adhesive third body.

Tribometer scale test demonstrated that the MAPLub PF100b demonstrated to be an efficient solution to solve the ejection anomaly. The PTFE particles embedded in the grease accumulated in the damaged areas of the surfaces to created, in situ, a lubricious layer made of PTFE based materials. The layer's thickness is sufficient to separate the bodies in contact and avoid further contact between the satellite and deployer material.

Tests in representative conditions with representative materials/coatings shall be made as soon as possible during development in order to avoid late and critical anomalies. In Sat/deployer ejection case, early tests were performed but surfaces and satellite dummy were not fully representative.

Acknowledgement

This benefited from support from the EUR EIPHI Graduate School (contact ANR-17-EURE-0002), and from the French RENATECH network through its FEMTO-ST technological facility MIMENTO.

Figure 1 :

 1 Figure 1: ANGELS and its deployer

Figure 6 :

 6 Figure 6: Profilometry on the deployer railWith a closer look on this surface we were able to see that the weared surface could be locally 5 to 6µm above the original surface.

Figure 7 :

 7 Figure 7: Material trapped in the roughness of the deployer rails

Figure 9 -

 9 Figure 9 -Comparison of damaged areas from satellite and from tribometer tests samples. The arrow v displays the sliding direction during the friction tests

Figure 13

 13 Figure 13 Lubrication procedure: rail cleaning (A), grease application (B), grease spreading and grease excess removing (C)

Figure 16

 16 Figure 16 Grease bulge showing grease circulation on bottom rails

Table 1 -

 1

	Fretting tests	Vibration test data
	Normal	140 N
	Load	(Contact	Contact pressure:
		pressure: 3.6	1.8 MPa minimum
		MPa)
	Amplitude	±300 µm	±100 µm max
	Frequency	30 Hz	Eigenmode @60Hz
		(sliding speed:	to 100Hz
		36 mm/s)	(24 to 40 mm/s)
	Duration	3min	2min per axis
	Environment	Air	Clean room
		(43 ± 4 %HR,	(55 ± 10 %HR, 22
		22 ± 1 •C)	± 3 •C)

Contact conditions of the fretting tests and comparisons with estimated contact condition from the analysis of the satellite and the deployer.


Marie-Adèle Magnaldo

Quentin Libois
email: quentin.libois@meteo.fr

Sébastien Riette

Christine Lac

Methods for assessment of models Evaluation of surface shortwave downward radiation forecasts by the numerical weather prediction model AROME

With the worldwide development of the solar energy sector, the need for reliable surface shortwave downward radiation (SWD) forecasts has significantly increased in recent years. SWD forecasts of a few hours to a few days based on numerical weather prediction (NWP) models are essential to facilitate the incorporation of solar energy into the electric grid and ensure network stability. However, SWD errors in NWP models can be substantial. In order to characterize the performances of AROME in detail, the operational NWP model of the French weather service Météo-France, a full year of hourly AROME forecasts is compared to corresponding in situ SWD measurements from 168 highquality pyranometers covering France. In addition, to classify cloud scenes at high temporal frequency and over the whole territory, cloud products derived from the Satellite Application Facility for Nowcasting and Very Short Range Forecasting (SAF NWC) from geostationary satellites are also used. The 2020 mean bias is positive, with a value of 18 W m -2 , meaning that AROME on average overestimates the SWD. The root-mean-square error is 98 W m -2 . The situations that contribute the most to the bias correspond to cloudy skies in the model and in the observations, situations that are very frequent (66 %) and characterized by an annual bias of 24 W m -2 . Part of this positive bias probably comes from an underestimation of cloud fraction in AROME, although this is not fully addressed in this study due to the lack of consistent observations at kilometer resolution. The other situations have less impact on SWD errors. Missed cloudy situations and erroneously predicted clouds, which generally correspond to clouds with a low impact on the SWD, also have low occurrence (4 % and 11 %). Likewise, wellpredicted clear-sky conditions are characterized by a low bias (3 W m -2 ). When limited to overcast situations in the model, the bias in cloudy skies is small (1 W m -2 ) but results from large compensating errors. Indeed, further investigation shows that high clouds are systematically associated with a SWD positive bias, while low clouds are associated with a negative bias. This detailed analysis shows that the errors result from a combination of incorrect cloud optical properties and cloud fraction errors, highlighting the need for a more detailed evaluation of cloud properties. This study also provides valuable insights into the potential improvement of AROME physical parametrizations.

Introduction

In the context of global warming, the European Union's Green Deal calls for at least 45 % of energy to come from renewable energy sources (RESs) by 2030. This is expected to reduce greenhouse gas emissions by 55 % compared to 1990 levels (https://energy.ec.europa.eu/topics/renewable-energy/ renewable-energy-directive-targets-and-rules/ renewable-energy-targets_en, last access: 25 April 2023). The share of RESs has already increased significantly in Europe and worldwide over the last decade (International Energy Agency, 2019). In France, in 2021, 13 067 MWp (MW peak) of solar photovoltaic (PV) generation capacity was installed (which corresponds to the potential production under standard test conditions), and 2687 MWp was added in 2021 (Réseau de transport d'électricité, 2021). Solar energy covers 3.1 % of the French annual electricity consumption (https: //bilan-electrique-2021.rte-france.com/production_solaire/, last access: 25 April 2023). This highlights that solar energy is a key element in moving to a more sustainable energy system.

Published by Copernicus Publications on behalf of the European Geosciences Union.

However, solar energy production is highly dependent on weather conditions, especially clouds, which are responsible for its very high spatiotemporal variability [START_REF] Widén | Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources[END_REF][START_REF] Antonanzas | Review of photovoltaic power forecasting[END_REF]. This variability is an issue for the planning of solar energy production and for the overall stability of the electric grid, which requires a balance between supply and demand [START_REF] Antonanzas | Review of photovoltaic power forecasting[END_REF][START_REF] Betti | Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications: Third Edition[END_REF]. To deal with that, accurate forecasts, mainly for surface shortwave downward radiation (SWD), are essential. Different forecasting horizons are relevant for the solar energy sector, ranging from minutes to months ahead [START_REF] Das | Forecasting of photovoltaic power generation and model optimization: A review[END_REF][START_REF] Antonanzas | Review of photovoltaic power forecasting[END_REF][START_REF] Raza | On recent advances in PV output power forecast[END_REF][START_REF] Betti | Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications: Third Edition[END_REF]. Very short term forecasts, covering horizons from seconds to 1 h, are important for power smoothing, real-time electricity dispatch and the optimization storage. Short-term forecasts, from 1 h to several days, are necessary for unit commitment, scheduling and dispatch of electrical power, and also the enhancement of the security of grid operation. Medium-term forecasts, from 1 week to 1 month, are useful, for example, for scheduling maintenance. Long-term forecasts, from 1 month to 1 year, are useful for planning electricity generation, transmission and distribution aside from energy bidding and securing operations [START_REF] Das | Forecasting of photovoltaic power generation and model optimization: A review[END_REF]. In the much longer term, in particular in the context of global warming, estimating the future potential of variable renewable resources, including solar, and the impact of temperature on electricity demand is essential to provide an assessment of future energy systems [START_REF] Dubus | Towards a futureproof climate database for European energy system studies[END_REF].

Depending on the time horizon, different forecasting techniques are used [START_REF] Widén | Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources[END_REF][START_REF] Raza | On recent advances in PV output power forecast[END_REF][START_REF] Das | Forecasting of photovoltaic power generation and model optimization: A review[END_REF][START_REF] Betti | Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications: Third Edition[END_REF]. For instance, for intra-hour forecasts, the use of two or more all-sky imagers (ASIs) organized in a network has proven successful (Nouri et al., 2019b, a;[START_REF] Logothetis | Benchmarking of solar irradiance nowcast performance derived from all-sky imagers[END_REF][START_REF] Chu | A network of sky imagers for spatial solar irradiance assessment[END_REF]. Forecasting techniques based on historical data, including machine learning approaches, can also be effective for the very short term [START_REF] Das | Forecasting of photovoltaic power generation and model optimization: A review[END_REF]. For short-term forecast up to 6 h horizon, satellite-based methods that extrapolate cloud location using cloud-motion vectors can be effective (e.g., [START_REF] Cros | Reliability Predictors for Solar Irradiance Satellite-Based Forecast[END_REF]. From hours to a few days, physical methods based on weather forecasts produced by numerical weather prediction (NWP) models are the most common [START_REF] Widén | Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources[END_REF][START_REF] Antonanzas | Review of photovoltaic power forecasting[END_REF][START_REF] Betti | Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications: Third Edition[END_REF]. As a consequence, these NWP models, which solve the mesoscale atmospheric dynamics and account for small-scale physical processes, are an essential element for the management of power systems involving a significant amount of solar energy.

However, the performance of NWP models in predicting SWD remains limited. As an illustration, we compared the 1 d SWD forecasts of AROME [START_REF] Seity | The AROME-France Convective-Scale Operational Model[END_REF], the operational NWP model of the French weather service Météo-France, to hourly averages from the national pyranometer network comprising 168 stations. For 2020, the mean annual bias and root-mean-square error (RMSE) are respectively 18 and 97 W m -2 , for a mean SWD of 340 W m -2 . This shows that the errors can be significant, with correspondingly high uncertainties in the PV production forecasts.

Nevertheless, until now, SWD errors have not been a priority for weather forecasts produced by NWP models, since they did not have radiative scores, whereas they have been particularly studied in climate models. Growing interest from various end users, including the PV community, is now highlighting these substantial SWD errors and the need for better forecasts.

For example, [START_REF] Nielsen | Using shortwave radiation to evaluate the HARMONIE-AROME weather model[END_REF] evaluated the SWD forecasts of the NWP model HARMONIE-AROME in Denmark with a pyranometer network and found a negative bias for days with optically thick clouds, which they attributed to an excess of cloud water in the model thick clouds. This approach is interesting because it allows a model to be evaluated over a large domain and a long period, but it does not distinguish between cloud regimes such as cloud altitude or phase, which could help better understand SWD errors. [START_REF] Köhler | Critical weather situations for renewable energies -Part B: Low stratus risk for solar power[END_REF] had a different approach, allowing for a distinction by cloud regime. They analyzed the PV power forecast errors of the NWP model COSMO-DE in Germany for 2013 and 2014 and highlighted that nearly one-third of the 100 d with the largest SWD errors was associated with fog and low stratus events. However, while this study demonstrated the need for more reliable forecasts of low cloud cover, it was limited to 100 d and the cloud regimes were analyzed manually, which did not allow a systematic distinction of cloud situations. Other more systematic studies relied on highly instrumented sites with lidars and radars to detect the presence of clouds (Tuononen, 2019), or to automatically classify clouds based on their base and thickness [START_REF] Ahlgrimm | The Impact of Low Clouds on Surface Shortwave Radiation in the ECMWF Model[END_REF]. These approaches are useful for determining how cloud regimes and cloud physical parameters contribute to SWD errors in NWP models. More specifically, Tuononen (2019) showed that low and mid-level clouds in the Integrated Forecast System (IFS) model in Helsinki are associated with a positive bias of SWD when the liquid water path (LWP) is low. They showed that an overestimation of SWD correlates with an underestimation of cloud fraction in the IFS model. Similarly, [START_REF] Ahlgrimm | The Impact of Low Clouds on Surface Shortwave Radiation in the ECMWF Model[END_REF] estimated that at the ARM Southern Great Plains (SGP) site, in overcast low cloud conditions, the frequency of low-LWP clouds was overestimated, and the frequency of high-LWP clouds was underestimated in the IFS model.

These highly instrumented sites also allow other parameters to be evaluated in the absence of clouds. For instance, at the ARM SGP site, Weverberg et al. (2018) analyzed the contributions of surface albedo, surface longwave emission, integrated water vapor, aerosols and cloud properties in nine global circulation models and found that cloud errors generally dominate SWD errors. Likewise, [START_REF] Morcrette | Assessment of the ECMWF model cloudiness and surface radiation fields at the ARM SGP site[END_REF] found an underestimation of water vapor absorption and errors in humidity and in aerosol concentrations in the IFS model. Rieger et al. (2017) also reported large SWD errors during a Saharan dust outbreak over Germany in the ICON NWP model.

While most studies investigating the causes of SWD errors in NWP forecasts have used highly instrumented sites where detailed, high-frequency observations are available, this approach remains limited to a single site. In our study, we aim to develop a general method to evaluate SWD forecasts from NWP models and identify the situations that contribute most to errors. The method should allow a large domain to be dealt with, with a sufficient number of reference observations, as in [START_REF] Nielsen | Using shortwave radiation to evaluate the HARMONIE-AROME weather model[END_REF]. Hence, it should not be limited to a single supersite because such a site could be unrepresentative of the whole domain. The method should also allow us to go further in identifying cloud errors. Our strategy is also to explore a wide variety of meteorological situations, which is possible by evaluating the model throughout the year and not only during a short period where specific seasonal errors could dominate the overall behavior. Finally, our method is intended to be general and systematic enough to be applied to any NWP model, relying only on SWD observations and making extensive use of cloud satellite products.

In this paper, our evaluation methodology is applied to AROME at 1.3 km horizontal resolution. For this purpose, a full year of 1 d hourly forecasts is compared with in situ SWD measurements from the pyranometer network operated by Météo-France. The cloud products derived from the Satellite Application Facility for Nowcasting and Very Short Range Forecasting (SAF NWC) from geostationary satellites are used to classify cloud scenes at high temporal frequency and over the whole territory.

The paper is organized as follows. Section 2.1 presents the SWD measurements and the satellite products, as well as the AROME forecasts used in this study. The evaluation methodology is then detailed in Sect. 2.2. Section 3 presents the results, first in terms of cloud occurrence and then in terms of cloud situations. The results are then put into perspective in Sect. 4, where the limitations of the observations and potential sources of error in AROME are discussed.

2 Data and methods 2.1 Observation and modeling data 2.1.1 AROME forecasts AROME (Applications de la Recherche à l'Opérationnel à Méso-Echelle) is a limited-area non-hydrostatic model developed by the French weather service, Météo-France [START_REF] Seity | The AROME-France Convective-Scale Operational Model[END_REF]Brousseau et al., 2016). The French operational configuration of AROME covers a large part of western Europe and has a horizontal resolution of 1.3 km. The number of model pixels is 1525 × 1429, and the number of vertical levels is 90. The time step is 50 s. The lateral boundary conditions are provided by ARPEGE (Action de Recherche Petite Echelle Grande Echelle), the French operational global NWP model.

The AROME model physical package is derived from the Meso-NH model (Lafore et al., 1998;[START_REF] Taufour | Evaluation of the two-moment scheme LIMA based on microphysical observations from the HyMeX campaign[END_REF]. The shallow convection is parametrized using the EDMF (Eddy Diffusivity Mass Flux) approach (Pergaud et al., 2009). The microphysical scheme is the one-moment mixed ICE3 scheme, completed with a subgrid condensation scheme presented in [START_REF] Riette | A New Framework to Compare Mass-Flux Schemes Within the AROME Numerical Weather Prediction Model[END_REF]. The turbulence parametrization considers a prognostic turbulent kinetic energy (TKE) equation from the [START_REF] Cuxart | A turbulence scheme allowing for mesoscale and largeeddy simulations[END_REF] scheme in a 1D mode and is closed with the [START_REF] Bougeault | Parameterization of orographyinduced turbulence in a mesobeta-scale model[END_REF] mixing length. The radiation parametrization comes from the IFS model and comprises the six spectral bands' shortwave radiation scheme of [START_REF] Fouquart | Computations of solar heating of the Earth's atmosphere: A new parameterization[END_REF] and the Rapid Radiative Transfer Model for longwave radiation [START_REF] Mlawer | Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave[END_REF]. AROME is coupled online with the SURFEX (externalized land and ocean surface platform) model, which describes the surface fluxes and the evolution of four types of surfaces: natural, town, inland water and ocean (Masson et al., 2013).

In this study, 24 h operational hourly forecasts starting at 00:00 UTC are evaluated, considering the average hourly SWD and average hourly total cloud fraction forecasts. We also use the average hourly cloud fraction in slices across the tropospheric column, located at pressure levels 1013-785 hPa (∼ 2 km), 785-450 hPa (∼ 6 km) and 450 hPa-top of the model. This study focuses on the year 2020 (corresponding to the AROME cycle 43t1). This year was chosen because it corresponds to a recent and continuous period without major changes in the AROME operational model.

Observations of shortwave downward radiation

Within the operational observation network of Météo-France, 221 pyranometers in mainland France provide hourly means of SWD. Among them, only 168 were used during the whole year 2020 and considered of sufficient quality. Their locations are shown in Fig. 1. Practically, measurements taken under critical environmental conditions (e.g., with significant local masks or in mountainous regions above 1000 m) or with time series showing anomalies have been discarded to avoid introducing observational errors in the reference measurements.

Cloud satellite products

In order to identify the presence and type of clouds at high frequency and over the whole AROME domain, cloud satellite products developed by the NWC EUMETSAT SAF are used [START_REF] Legleau | User Manual for the Cloud Product Processors of the NWC/GEO: Science Part[END_REF][START_REF] Legléau | Scientific and Validation report for the Cloud Product Processors of the NWC/GEO[END_REF]. Over France, the horizontal resolution is around 5 km, and the temporal frequency is 15 min, so there are four values for 1 h. The cloud mask product gives discrete values: 0 for no cloud and 1 for clouds. The cloud type is a classification among 15 https://doi.org/10.5194/gmd-17-1091-2024

Geosci. Model Dev., 17, 1091-1109, 2024 different classes: 4 corresponding to cloud-free scenes and 11 corresponding to cloud scenes (see Sect. 2.2.3).

Methods

Model-observation comparison of shortwave downward radiation

We compare hourly means of SWD from the model and from observations. To avoid grazing angles under which pyranometer measurements can be inaccurate, only the diurnal values corresponding to a cosine of the solar zenith angle (SZA) greater than 0.1 (equivalent to a SZA less than 84.3 • ) are considered. To avoid double-penalty issues, usual for high-resolution models like AROME that are unlikely to form clouds at exactly the right place and time [START_REF] Amodei | Deterministic and fuzzy verification methods for a hierarchy of numerical models[END_REF][START_REF] Stein | Neighborhood-Based Contingency Tables Including Errors Compensation[END_REF], we do not necessarily select the closest point, unlike most SWD evaluation studies, but instead extract it in the vicinity of the observation point. In practice, a neighborhood strategy is set up that allows the selection of a model point with a forecast SWD value close to the observed value in a small square domain centered on the observation point. While such a neighborhood strategy may seem unsatisfactory to PV producers who are concerned about SWD at the exact location of the PV plant, these users should keep in mind that NWP forecasts are not expected to be accurate at individual grid points. We sort the absolute model-observation errors in this neighborhood in ascending order and select the 10th percentile value. This strategy avoids the selection of a point very close to the observation by chance and ensures that the method is not too sensitive to the size of the neighborhood. Here, the size of the neighborhood is set to 5 × 5 model pixels, which means that the point with the second smallest difference (among 25 values) is retained for each observation. The neighborhood size is chosen to be comparable to the satellite pixel size (see Sect. 2.2.2).

The error metrics commonly used to evaluate the SWD forecast errors are the mean error or bias (ME) and the rootmean-square error (RMSE). The ME describes systematic deviations and provides information on the sign of error. However, the ME is not sufficient to assess errors because of compensation effects. The RMSE is more sensitive to large errors and thus particularly suited to the electricity market, where large errors are much more critical than small errors [START_REF] Perez | Comparison of numerical weather prediction solar irradiance forecasts in the US, Canada and Europe[END_REF][START_REF] Betti | Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications: Third Edition[END_REF]. The RMSE can be divided into systematic errors (MEs) and unsystematic errors or standard deviation of errors (SDE), as described in [START_REF] Widén | Variability assessment and forecasting of renewables: A review for solar, wind, wave and tidal resources[END_REF]:

RMSE 2 = SDE 2 + ME 2 . ( 1
)
In practice, while it can be relatively easy to correct the bias in a model, with a simple tuning or a statistical adjustment, reducing SDE is generally more challenging.

Cloud occurrence classification

To identify cloud presence in the observations, we use the satellite cloud mask. As AROME forecasts are only available as hourly means, comparable satellite products at hourly resolution are computed. Clear-sky conditions are considered for a pyranometer location (i.e., the satellite pixel including the location of the pyranometer) at hour H when the last four consecutive observations report no clouds. To identify clear-sky conditions in AROME, the hourly mean total cloud fraction is computed as the average over the neighborhood defined in the previous section. A threshold value of 2 % is fixed: if the hourly mean total cloud fraction is larger than this value, the hour is considered cloudy and clear sky otherwise. Once cloudy skies are identified in both observations and the model, we set up a contingency table, following Tuononen (2019): "hit" when clouds are present in the model and in the observation, "false alarm" when clouds are present in the model only, "miss" when clouds are present in the observation only, and "correct negative" when both observations and the model agree on clear-sky conditions.

Cloud regimes and cloud types

In the model, to further discriminate between different cloudy situations, we use a modeled cloud regime classification similar to Weverberg et al. (2018). In AROME forecasts, the cloud fraction for each of the three distinct vertical slices is used. For each region, a 2 % threshold, similar to the threshold value for cloud occurrence, is set to distinguish cloudy and clear-sky conditions. The value of this threshold is the same as the threshold value for cloud occurrence classification (see Sect. 2.2.2). Seven cloud regimes (CR1 to 7) are defined from the combination of these cloudy layers. When the liquid water content is larger than 10 -5 kg kg -1 at the first vertical level of the model, the cloud regime is set to CR0, which corresponds to fog. These eight cloud regimes are depicted in Fig. 2. Table 1 summarizes the different naming used to describe groups of cloud regimes used in this article.

In the observations, to further distinguish cloudy skies, we use a cloud type classification based on the cloud type satellite product developed by SAF NWC and described in Sect. 2.1.3. As the values are instantaneous and discrete, we select the values at H -30 min. The 11 cloud types of the product are merged into 6 (Table 2), allowing an easier comparison with model outputs.

Figure 3 shows the monthly relative frequency for each cloud regime (respectively type) and for the clear skies in the model (respectively in the satellite product) over the pyranometer locations and when cos(SZA) > 0.1. In the model, the frequency of clear skies is 22 % against 30 % in the satellite product, suggesting that the model predicts too many clouds, and/or some optically thin clouds are not detected by the NWC SAF product, which is a known caveat of passive sensors (Sun et al., 2011). AROME predicts fewer low clouds (23 % of CR0+CR1+CR4) than satellite observations (29 % of LCs). In addition, the relative frequency of observed fractional clouds is higher in summer than in winter, while low clouds are overall less frequent in spring and summer, highlighting seasonal cycles in the observations that do not have obvious equivalents in the model. We can also note that the model predicts too many thick clouds with a high cloud top (29 % of CR5+CR7) compared to the observations (9 % of HCs). Note that the sum of the relative frequencies of clear skies and all cloud regimes is not exactly equal to 1 in Fig. 3a. This is because there are situations with very low relative frequencies with a total cloud fraction larger than 2 % (implying that the scene is considered cloudy) but with a cloud fraction for each of the three vertical slices of the troposphere lower than 2 % (implying that it is not associated with any CR).

Statistical analysis

The method presented in the previous section is now applied to evaluate the AROME SWD forecasts, starting with an overall evaluation before going into detail.

Overall evaluation: clear-sky index histograms

To begin with, we use the clear-sky index (CSI), which is defined as Only values for which the SZA is less than 70 • are considered to ensure that the CSI distribution is not affected by measurement errors and model limitations at grazing angles.

CSI mod = SWD mod SWD clear,mod ; CSI obs = SWD obs SWD clear,mod , (2
In the observations the CSI can greatly exceed 1, as reported by [START_REF] Nielsen | Using shortwave radiation to evaluate the HARMONIE-AROME weather model[END_REF]. This can be explained by an underestimation of the model clear-sky SWD, which can be due to a reduction in aerosol emissions over the last decades [START_REF] Wild | Global dimming and brightening: A review[END_REF], while the aerosol climatology used in AROME is older [START_REF] Tegen | Contribution of different aerosol species to the global aerosol extinction optical thickness: Estimates from model results[END_REF] and thus overestimates present-day aerosol loadings. These values exceeding 1 can also be due to cloud enhancement effects (e.g., Gueymard, 2017), which occur under broken cloud conditions. Such effects cannot be simulated with standard plane-parallel radiative codes [START_REF] Wissmeier | paNTICA: A Fast 3D Radiative Transfer Scheme to Calculate Surface Solar Irradiance for NWP and LES Models[END_REF]. The few CSI values in the model exceeding 1 are unrealistic and are due to the fact that the theoretical SWD under clear-sky conditions from the model is slightly delayed compared to the SWD. The CSI in the model is more frequently between 0.8 and 1 compared to observations (which may partly be due to the lack of values exceeding 1 in the model), or less than 0.1, and in contrast less frequently between 0.1 and 0.75. It suggests that optically thick clouds are too thick, which agrees with the excess of CR5 and CR7 pointed in Fig. 3. Interestingly the CSI distributions are similar to those reported by [START_REF] Nielsen | Using shortwave radiation to evaluate the HARMONIE-AROME weather model[END_REF] with HARMONIE-AROME [START_REF] Bengtsson | The HARMONIE-AROME model configuration in the ALADIN-HIRLAM NWP system[END_REF]. Although these models share the same code, the operational configurations rely on different sets of parametrizations. It suggests that NWP model errors are to some extent systematic over a wide range of situations. The opposite behaviors in different ranges of CSI show that errors in AROME are not limited to a systematic bias, which could be easily corrected with a rough tuning. Instead, the behaviors seem to depend on the cloud situation, which will be investigated further in the following.

Attributing model errors to cloud occurrence

To go beyond Fig. 4, we now investigate how the different cases of cloud presence defined in Sect. 2.2.2 contribute to the overall errors. To this end we compute the relative frequency, SWD bias and SDE for each of the four occurrence cases. To identify the situations that contribute most to errors, another metric is used: the contribution to the total bias, which is the frequency-weighted bias. The sum of these contributions equals the total bias. If a situation is associated with a high bias but rarely occurs, it barely contributes to the total bias. In contrast, a situation associated with a low bias can significantly contribute to the total bias if it frequently occurs. Figure 5a shows the monthly bias, frequency and contribution to the total bias. As expected, the bias is negative for the false alarm cases (ranging from -5 to -25 W m -2 ), and positive for the miss cases (with monthly biases up to 80 W m -2 ). The false alarm cases are almost 3 times more frequent than miss cases (11 % of false alarm cases and 4 % of miss cases during the year 2020), which is consistent with the higher clear skies frequency in the observations pointed in Sect. 2.2.3 and suggests that AROME predicts too many clouds or that some clouds are not detected by the satellite. With a cloud fraction threshold value of 10 %, the gap between the two frequencies is reduced, but false alarm cases remain more frequent (10 % of false alarm cases and 6 % of miss cases). The bias is stronger for the miss cases than for the false alarm cases. This suggests that clouds missed by AROME have more impact on the SWD than those simulated in false alarm cases or that some false alarm cases are actually hit cases with undetected clouds. When both the model and observations predict clear skies, the bias is much smaller, ranging from -10 to 10 W m -2 , and is on average slightly positive during the year. The hit cases represent 66 % of the situations and are associated with significant biases up to 50 W m -2 . Their contribution to the total bias is the most important (16 W m -2 , for a total bias of 18 W m -2 ). Interestingly, the bias associated with hit cases is invariably positive throughout the year, showing a marked tendency of AROME to underestimate the impact of clouds on the SWD. Figure 5b shows that the SDE is also the strongest for the hit cases, with annual values of 110 W m -2 , compared to 50 W m -2 for the false alarm cases, 95 W m -2 for the miss cases and 22 W m -2 for the correct negative cases. Note that several tests were performed by changing the neighborhood size (1 pixel and 10 × 10 pixels), the cloud detection threshold (1 %, 5 %, 10 %) and the SZA threshold (70 • ), which did not qualitatively change these results. Hence we consider these results are robust and independent of the mentioned thresholds. To summarize, errors mostly occur when clouds are present in the model and in the observations, which is consistent with Weverberg et al. (2018) and Tuononen (2019).

Attributing model errors to AROME cloud regimes and satellite cloud types

In what follows, we provide a more physical overview of correct negative cases, false alarm cases and miss cases results.

Then we focus more extensively on the hit cases that con- tribute most to SWD errors. To this end, we distinguish between situations in terms of cloud regimes in AROME or in terms of the satellite cloud types, so that bias and SDE can be attributed to these specific cloud situations.

Correct negative cases

Various factors can explain SWD errors for the correct negative cases. For example, [START_REF] Morcrette | Assessment of the ECMWF model cloudiness and surface radiation fields at the ARM SGP site[END_REF] found a persistent positive bias in clear sky that was attributed to an underestimation of gaseous absorption in the IFS water vapor spectroscopy, which is still the one used in AROME. Another factor which can contribute to SWD errors in clear-sky condition is relative to aerosols. In AROME, the aerosols are prescribed by a monthly climatology [START_REF] Tegen | Contribution of different aerosol species to the global aerosol extinction optical thickness: Estimates from model results[END_REF], meaning that only the average seasonal cycle of aerosols is captured, but the individual aerosol events are not accounted for. Such events, for instance dust outbreaks, can result in localized SWD attenuation of 40 %-50 %, as pointed out by Kosmopoulos et al. (2017), so not taking them into account can lead to significant SWD errors (Rieger et al., 2017).

To estimate the extent to which such a misrepresentation of aerosols contributes to the errors of the correct nega- tive cases, we examine the correlation between SWD errors and aerosol optical depth (AOD) errors, the latter being defined as the difference between the climatological AOD from AROME and the AOD estimated from Copernicus Atmohttps://doi.org/10.5194/gmd-17-1091-2024

Geosci. Model Dev., 17, 1091-1109, 2024

sphere Monitoring Service (CAMS), a chemistry-transport model that provides near-real time AOD forecasts and fully captures aerosol-related events [START_REF] Benedetti | Aerosol analysis and forecast in the European Centre for Medium-Range Weather Forecasts Integrated Forecast System: 2. Data assimilation[END_REF][START_REF] Morcrette | Aerosol analysis and forecast in the European Centre for Medium-Range Weather Forecasts Integrated Forecast System: Forward modeling[END_REF], interpolated to the AROME grid. Figure 6 shows the natural logarithm of the transmittance (defined as T = SWD/SWD TOA with SWD TOA the SWD at the top of the atmosphere) errors as a function of AOD errors divided by cos(SZA) in the correct negative cases, as well as the linear regression line. Again, only values for which the SZA is less than 70 • are considered in order to avoid values of SWD TOA that are too low.

As expected, a negative slope is obtained during the year 2020 with a value of -0.07 and also for each individual month (not shown). The correlation coefficient is low, showing a large variability in the data, with a value of -0.19 during the year and less negative values in winter and higher values in summer. However, throughout the year the p value is very low, indicating a statistically significant relationship between SWD errors and AOD errors. The low correlation coefficient could be explained by the different factors mentioned above that explain SWD errors but mainly by clouds which are not detected by cloud products but alter the SWD (see Sect. 4.1.2). Using passive measurements for cloud detection limits our study of aerosol errors because cloud detection errors influence clear-sky SWD errors at the first order. A better cloud detection is required to identify real clear skies in reality, as discussed in Sect. 4.1.2.

We do not extend further on these clear-sky cases which have been treated elsewhere, notably Rieger et al. (2017), who show an overall improvement of the PV-power forecast during a Saharan dust event over Germany in the ICON model when extended with modules accounting for trace gases and aerosols and related feedback processes. Moreover, the Tegen climatology used in AROME is outdated, and a more recent climatology such as the CAMS climatology could reduce SWD errors, as implemented in the IFS (Bozzo et al., 2020).

False alarm cases

False alarm cases correspond to clouds predicted by AROME but not observed by the satellite. Here we study the type of clouds present in AROME in such cases in more detail. Figure 7a shows that the relative frequency for geometrically thin clouds in AROME, designating cloud regimes with clouds in only one vertical slice of the troposphere, is higher when no clouds are detected than in the AROME total cloud climatology (for instance 50 % vs 18 % for CR3; see Fig. 3a). This suggests that some thin clouds (mostly high clouds) predicted by AROME are not physically realistic or are not detected by the satellite. Regarding the biases, they are mostly negative for most of the cloud regimes, except for CR3 for a few months, CR3 having the lowest absolute bias throughout the year. This also shows that some geometrically thin high clouds may not be detected. The major contribution to the total bias in false alarm cases is for geometrically thin low clouds (CR1) which have a lower frequency than CR3 but a much larger bias throughout the year. Given the SWD errors associated with CR1, it is likely that some are erroneously predicted by the model. In contrast, almost no geometrically thick clouds (CR4, CR5 and CR7) are simulated when no clouds are observed (5 % for CR4, 6 % for CR5 and 2.7 % for CR7), while they are predominant in the total climatology (13 % for CR4, 11 % for CR5 and 28 % for CR7). This is consistent with the fact that simulating thick clouds in the absence of clouds is unlikely and that thick clouds are less likely to be undetected. Note that the annual mean SWD in the model for the false alarm cases (436 W m -2 ) is much higher than the annual mean SWD in the model for the hit cases (296 W m -2 ), suggesting that undetected simulated clouds have on average less impact on the SWD than actually observed clouds.

Around 4 % of the false alarm cases are fog, compared to about 3 % of simulated clouds in total. In winter, there can be up to 3 times more fog in the false alarm cases than in the total AROME cloud climatology, suggesting that during the day in winter, there is a lot of simulated fog, while no cloud is detected. As fogs are largely present in the morning (7.6 % in the morning and 0.5 % in the afternoon during the year), this may be due to the delay in dissipation of AROME fog highlighted by [START_REF] Antoine | Evaluation of an improved AROME configuration for fog forecasts during the SOFOG3D campaign[END_REF]. In winter, the highest negative bias is for fog (minimum of -83 W m -2 ) with a relatively high frequency, showing that the contribution of fog to the bias is particularly important in winter and that improving AROME fog forecasts is an important issue for SWD errors.

Miss cases

As for the false alarm cases, the objective is to identify which cloud types have been missed by AROME. Figure 7b shows that the seasonal cycle is more pronounced than for the false alarm cases. In winter, the relative frequency of low clouds (LCs) in the miss cases is much higher (maximum of 74 %) than for the observed climatology as a whole (41 %; see Fig. 3b), while the opposite is true in summer with a minimum of 21 %. Figure 7b also shows that, as expected, the bias for all cloud types is positive and that the main contribution to the bias for the year 2020 comes from LC, especially in winter and autumn.

The annual relative frequency of fractional clouds (FCs) in miss cases is the highest (47 %), especially in summer when they are more frequent with a maximum of 67 %. Regardless of the season, the relative frequency of FC in miss cases is higher than in the total observed climatology (maximum of 27 %).

Few high clouds (HCs) are missed, with an annual relative frequency of 1 % (13 % in the total observed climatology), unlike semi-transparent clouds (SCs), which are often missed with an annual relative frequency of 14 % (20 % of the total observed climatology). The highest biases come from midlevel clouds (MCs) (211 W m -2 ) and HCs (203 W m -2 ), but due to their very low occurrence, their contribution to the bias in miss cases is very small, in contrast to SC. Note that the annual mean SWD in the observations for miss cases (409 W m -2 ) is much higher than for hit cases (272 W m -2 ), suggesting that the clouds missed by the model have, on average, a small impact on the observed SWD.

Hit cases: in all situations

We now focus on the hit cases, which are the most frequent and contribute the most to the overall errors. We investihttps://doi.org/10.5194/gmd-17-1091-2024

Geosci. Model Dev., 17, 1091-1109, 2024 gate how in these cases the errors depend on the AROME cloud regimes and satellite cloud types. Figure 8a shows the monthly bias and relative frequency as well as the contribution to the total bias for the hit cases over 2020. It appears that AROME overestimates the SWD for almost all CRs, except for CR0 in all the months and CR4 and CR7 in some months. The largest positive bias is for CR3, with values up to 93 W m -2 . Figure 8b shows the same behavior for almost all cloud types, except for FC. Such a positive bias could be explained both by a systematic underestimation of the cloud fraction and/or by an underestimation of the cloud optical thickness. Since we do not have an estimate of the cloud fraction, it is difficult to conclude. Indeed, the evaluation of the cloud fraction at such high spatial and temporal resolutions over a large domain remains a challenge.

To go further, Fig. 9 shows the bias for each cloud regime and four ranges of forecasted cloud fraction. At low cloud fraction (i.e., less than 10 %), the bias, in addition to being the highest for almost all cloud regimes, is positive for all cloud regimes. This is not surprising since when the cloud fraction is low in the model, if the cloud fraction is not correct, it is most likely to be underestimated, resulting in a positive bias. For a high cloud fraction (more than 95 %), the bias remains positive for CR2, CR3, CR5 and CR6, despite the fact that in this case, if the cloud fraction is not correct, it is most likely to be overestimated, which would lead to a negative bias. This suggests that for these cloud regimes, errors are not only governed by cloud fraction errors.

This preliminary analysis highlights errors that can be attributed to cloud fraction errors. Since we do not have access to a reliable cloud fraction observation for the AROME resolution and domain, we do not pursue the evaluation under all cloudy conditions. Instead, from now on we focus on situations where the AROME cloud fraction is close to unity, which means that positive biases at least cannot be attributed to an underestimation of the cloud fraction.

Hit cases: in overcast situations

As explained above, we now focus on overcast situations. In practice, we consider the hit cases for which the AROME cloud fraction is larger than 95 %, which corresponds to 61 % of the hit cases and 40 % of all the cases. The frequency of overcast situations for each cloud regime is shown in Fig. 9 (black numbers in %). In practice, these proportions are largely the same when the overcast threshold is set between 95 % and 99 %. The first effect of this overcast sub-selection is to change the annual bias, leading to a nearly zero bias (1.1 compared to 24 W m -2 before). This suggests that part of the overall positive bias may be due to errors in cloud fractions or optical thickness that is too low under partial cloudiness. In contrast, the SDE is barely reduced (101 vs 110 W m -2 ), showing that the overcast cases still deserve attention and that the nearly zero bias results from error compensations. In this section, we investigate how the errors vary with the AROME cloud regimes and satellite cloud types.

We start by distinguishing these overcast hit cases in terms of modeled cloud regime or observed cloud type in the satellite classification. Figure 10a shows the 2020 monthly bias, relative frequency and contribution to total bias. When the situations are sorted according to the AROME cloud regime, the high clouds (CR3 and CR5) are systematically associated with a positive bias throughout the year, with values up to 100 W m -2 . Since these two cloud regimes correspond to 21 % of the cases, this results in a significant positive contribution to the bias. In contrast, the bias for the low clouds (CR1 and CR4) is negative with a more pronounced seasonal cycle, characterized by a stronger bias in summer up to -80 W m -2 . Fog cases are associated with negative biases with an annual bias of -25 W m -2 and a higher frequency in January. The annual bias is consistent with the already mentioned delay in fog dissipation and the optical depth that is too thick, related to the excess of water content and droplet concentration in the fogs simulated by AROME, as pointed out by [START_REF] Antoine | Evaluation of an improved AROME configuration for fog forecasts during the SOFOG3D campaign[END_REF].

When satellite cloud types are used (Fig. 10b), again we obtain a positive bias for HC, with values up to 70 W m -2 ; a bias of variable sign for SC; and a persistent negative bias for LC, with values up to -40 W m -2 . The bias is always neg-ative for FC. This is consistent with the fact that this cloud type probably corresponds to situations with a cloud fraction lower than 100 %, meaning that the cloud fraction, if not correct, is overestimated in AROME. Note that satellites with passive instruments cannot detect low clouds when a high cloud layer is present, and therefore the modeled cloud regimes and observed cloud types are not directly comparable.

The results are essentially the same when the cloud fraction threshold for defining the cloud regimes, applied independently over each vertical slice of the troposphere as explained in Sect. 2.2.3, is set to 1 %, 2 %, 5 % or 10 %.

These results, consistent between the AROME and satellite classifications, highlight that although on average the SWD bias is small under overcast conditions, it results from the compensation of large and systematic errors, positive for high clouds and negative for low clouds. This also explains why the SDE remains large compared to the bias.

To further investigate the error distribution for each cloud type, it is useful to examine the CSI for the AROME cloud regimes in overcast situations. Figure 11 shows the CSI distribution for all cloud regimes in the model and in the observations. For CR0, CR1, CR4, CR6 and CR7, i.e., for the cloud regimes which have a non-zero cloud fraction in the lower atmosphere, the model has too many low CSI values below 0.1 compared to the observation. For CR1, and CR4, the model does not have enough values greater than 0.6. These two findings suggest that in overcast situations, the optically thick low clouds are too thick in AROME and that the model does not have enough optically thin low clouds. This explains the overall negative bias for these cloud regimes. For CR3, CR5 and CR6, the model has too many high values (between 0.8 and 1 for CR3, between 0.5 and 0.9 for CR5, and between 0.6 and 1 for CR6) and for CR3 and CR5 not enough low values. This suggests that high clouds are overall optically too thin in AROME. Even though the model does not have CSI values greater than 1, while the observations do, the bias is positive for these cloud regimes, as seen in Fig. 10a. This positive bias is not due to a potentially too small cloud fraction in the model, as only overcast situations in the model are selected.

To further investigate the SDE associated with each cloud regime, Fig. 12 shows the distribution of SWD errors for all AROME cloud regimes. For each CR, the SDE is high compared to the mean flux, in particular for CR2, CR4, CR5 and CR6. Interestingly, it shows that for all cloud regimes the mean biases result from both positive and negative contributions, indicating that multiple sources of errors are involved. The same is obtained for the distribution of SWD errors for observed cloud type in the satellite classification (not shown). This suggests that improving the mean bias of individual cloud regimes would not necessarily imply much better forecasts. It also implies that more detailed observations are needed to better understand these errors and their sources.

To summarize, the errors in overcast conditions depend on the cloud regimes. Low clouds seem on average optically too thick, although the negative bias may be due to an overestimation of the cloud fraction in overcast situations in the model. In contrast, simulated high clouds are often optically too thin. Although it would be interesting to evaluate the model over a longer period, the fact that our results are consistent for each month of 2020 suggest that they are not specific to this particular year.

Discussion

In this section, we first address the cloud fraction issue and provide a critical analysis of the satellite cloud mask and cloud type. We then investigate potential sources of error in AROME that could explain the observed SWD errors.

Limitation of the observations

Cloud fraction

In this study we did not evaluate cloud fraction, so we could not attribute SWD errors to potential cloud fraction errors. Indeed, evaluating the cloud fraction at AROME resolution over a large domain is challenging because spatially and temporally resolved cloud fraction observations at such spatial resolution are not common.

The cloud fraction of AROME at 2.5 km horizontal resolution has already been evaluated at a larger timescale between 2000 and 2018 with the 0.05 • COMET and 0.25 • CLARA satellite products by [START_REF] Lucas-Picher | Evaluation of the convection-permitting regional climate model CNRM-AROME41t1 over northwestern Europe[END_REF]. They show that in summer, AROME underestimates the cloud fraction over land, resulting in an overestimation of SWD, while it overestimates the cloud fraction in winter and spring. SWD is also overestimated in spring and fall, although less than in summer. Like [START_REF] Lucas-Picher | Evaluation of the convection-permitting regional climate model CNRM-AROME41t1 over northwestern Europe[END_REF], we found a SWD positive bias for every month of the year (Fig. 5a), even in winter and spring when the cloud fraction is possibly overestimated. This suggests that sources of error other than the cloud fraction are responsible for the positive SWD bias and that the underestimation of the cloud fraction in summer further accentuates the bias.

Satellite products at finer resolution should be used to analyze cloud fraction errors in AROME at 1.3 km resolution over the whole year of 2020, such as the daily MODIS product [START_REF] Ackerman | Cloud detection with MODIS. Part II: validation[END_REF], although this product cannot capture the targeted hourly resolution. An alternative to satellite observations would be to use ground observations from instrumented sites, allowing evaluation at a finer scale. Cloud fraction can be estimated from global and diffuse shortwave broadband measurements [START_REF] Long | Estimation of fractional sky cover from broadband shortwave radiometer measurements[END_REF], all-sky imaging systems [START_REF] Pfister | Cloud coverage based on all-sky imaging and its impact on surface solar irradiance[END_REF], or remote sensing instruments such as radar, lidar and microwave radiometers (e.g., [START_REF] Illingworth | Cloudnet[END_REF] However, the cloud fractions derived from different observations generally differ [START_REF] Long | Estimation of fractional sky cover from broadband shortwave radiometer measurements[END_REF][START_REF] Wagner | Error characteristics of ceilometerbased observations of cloud amount[END_REF], and more critically, the cloud fraction is somehow loosely defined, so that the observed cloud fraction does not necessarily match the model definition [START_REF] Brooks | Parameterizing the difference in cloud fraction defined by area and by volume as observed with radar and lidar[END_REF]. Above all, assessing the performance based solely on a few sites would not be consistent with the evaluation strategy adopted in this study.

Cloud detection by satellite

In this paper, several hints suggest that some clouds are not detected by the satellite product, such as the asymmetry between the frequency of clear skies in the model and in the observations (Sect. 2.2.3) or between the frequency of false alarm and miss cases (Sect. 3.2). In addition, the temporal variability of SWD under clear sky is much larger in the ob-servations than in the model (not shown). This questions the reliability of the satellite cloud mask used.

The likely non-detection of clouds may have an impact on our results. For the correct negative cases, this could explain a SWD positive bias in AROME, with clouds present in reality but not detected, and an overestimation of the frequency of correct negative cases. Likewise, it may lead to an underestimation of the hit cases' frequency, in addition to not taking into account some optically thin real clouds in the error calculations. However, the impact on the hit cases is probably limited due to their high relative frequency compared to the probably low relative occurrence of nondetection. In the future, a more reliable cloud detection could be based on ground observations from remote sensing instruments or high-frequency global and diffuse SWD measurements [START_REF] Long | Identification of clear skies from broadband pyranometer measurements and calculation of downwelling shortwave cloud effects[END_REF]. It should be noted that there may also be cases of clouds detected but not present in reality (LeGléau and Kerdraon, 2019) but that the impact is probably negligible in our study.

Cross-comparison between the cloud regimes and the cloud types

To assess the consistency between simulated cloud regimes (CRs) and observed cloud types (CTs) and analyze the matches between the model and the observations, a crosscomparison is shown in Fig. 13. It displays the distribution of cloud types for each cloud regime and, conversely, for the hit cases in overcast situations. Recall that the parallel between cloud regime and cloud type is not direct since satellite products do not see what is under an optically thick cloud. For low clouds, Fig. 13a shows that for CR0, CR4 and mainly for CR1, LC is most often observed. In contrast, Fig. 13b shows that when LC is observed, CR7 and CR6 are most often simulated, more than CR0, CR1 and CR4. CR2, CR3 and CR5 remain very rare in this case. The relative high frequencies of CR6 when LCs are observed (18.8 %) and of LCs when CR6 is simulated (50.9 %) can be explained by the non-detection of optically thin high clouds. Note that in Sect. 2.2.3, we have shown that CR7 clouds are far too frequent in the model compared to observations (HC) and that it is therefore not surprising to find an excessive occurrence of https://doi.org/10.5194/gmd-17-1091-2024

Geosci. Model Dev., 17, 1091-1109, 2024 CR7 in some cloud types. Even so, this suggests that for low clouds, the cloud type is in reasonable agreement with the model, and thus the negative bias of SWD found in Sect. 3.3 is actually related to poorly simulated low clouds. The study of fog remains limited by satellite because the satellite product includes fog in low clouds (LeGleau, 2019), as fog and low stratus are difficult to distinguish [START_REF] Bendix | Ground Fog Detection from Space Based on MODIS Daytime Data -A Feasibility Study[END_REF].

For optically thin high clouds, Fig. 13a shows that for CR3, SCs (48.4 %) and FCs (26.1 %) are mostly observed, which means a good match for optically thin high clouds since the distinction between SC and FC is questionable. Indeed, very thin cirrus are often classified as fractional clouds [START_REF] Legleau | User Manual for the Cloud Product Processors of the NWC/GEO: Science Part[END_REF]. Conversely, in Fig. 13b for SC, mostly CR7 (36.5 %), CR5 (22.0 %) and CR3 (19.9 %) are simulated, so the relative frequency of CR7, as for LC, seems too high. For FC, CR3 is mostly simulated (32.5 %).

For optically thick high clouds that most likely correspond to CR5, CR7 and HC, Fig. 13a shows that for CR5, SC and HC are mainly observed (63.8 % when added together), much more than LC, MC and FC. For HC, CR5 (20.3 %) and CR7 (68.0 %) are mainly simulated, while few CR0, CR1 and CR4 clouds are simulated. This means that we also have a relatively good match for high opaque clouds (HCs) and geometrically thick high clouds in the model (CR5, CR7). Thus, for high clouds, the model cloud regimes are in reasonable agreement with the observed cloud type, even though CR7 is too frequent in the model. This suggests that the positive SWD bias found in Sect. 3.3 is indeed related to high clouds.

In summary, there is generally a good correlation between the modeled cloud regime and the observed cloud type, although the match is not perfect. When the model predicts low clouds, mostly low clouds are observed, and when the model predicts high clouds, mostly semi-transparent and opaque clouds are observed. However, CR7 seems too frequent, with, in particular, too many occurrences when LCs are observed. Note, however, that some of the errors may also come from the fact that we compare instantaneous satellite products to hourly mean values of the model.

Investigating AROME errors

We now identify potential sources of error in AROME that could explain the biases highlighted in Sect. 3.3. Wurtz et al. (2021) pointed out that in AROME, the ice and snow contents of the anvils of mesoscale convective systems are too small. Their spatial extent is also too small, which they attributed to excessive snowfall velocities in AROME, resulting from a poor parametrization of the ice particle size distribution, an issue already reported by [START_REF] Taufour | Evaluation of the two-moment scheme LIMA based on microphysical observations from the HyMeX campaign[END_REF]. Wurtz et al. (2023) improved the treatment of snow in AROME, a correction that could help increase the snow mass and thus reduce the positive SWD bias for high clouds if snow is properly accounted for in the radiative code.

Another explanation for the positive SWD bias for high clouds could be that snow (which is one of the hydrometeors simulated by the microphysical scheme) is currently not taken into account in the AROME radiative code. Yet, on average over 2020, the total snow mass is 1.6 times that of cloud ice, meaning that a significant mass is neglected in the AROME radiative calculations. This is especially true for CR3, CR5 and CR7 where the snow mass dominates both the cloud liquid and cloud ice masses (not shown). In practice, it is recommended to include snow in radiative calculations, as snow has a significant radiative impact on the SWD as shown in the IFS model (Li et al., 2014a) and CMIP simulations (Li et al., 2014b[START_REF] Li | Exploring Radiation Biases Over the Tropical and Subtropical Oceans Based on Treatments of Frozen-Hydrometeor Radiative Properties in CMIP6 Models[END_REF], at least in regions with high precipitation and/or convective activity. Applying the correction developed by Wurtz et al. (2023) and taking into account the radiative effect of snow could reduce the positive SWD bias for high clouds found in this study. Note however that it could also deteriorate the bias for low clouds that are already too opaque (e.g., CR4 in winter).

Apart from the treatment of snow and ice, SWD errors have often attributed to liquid water path (LWP) errors. Evaluation of LWP in AROME could provide information on LWP errors and help understand the SWD negative bias in overcast situations. However, evaluating the LWP at AROME resolution over a large domain and a full year is challenging since geostationary satellite LWP products, such as the NWC SAF product, remain very limited because they rely on passive instruments. To investigate SWD errors due to LWP errors, a case study could be conducted, such as [START_REF] Ahlgrimm | The Impact of Low Clouds on Surface Shortwave Radiation in the ECMWF Model[END_REF], who identified LWP errors using microwave radiometer measurements that could explain the SWD positive bias in overcast low cloud situations in the IFS model. For instance, observations from the SIRTA supersite on the Saclay plateau (Chiriaco et al., 2018) could be used to better separate the contributions of variables such as cloud fraction, liquid water path and cloud droplet effective radius. Polar-orbiting satellites with active measurements onboard (e.g., CloudSat and CALIPSO, [START_REF] Stephens | CloudSat and CALIPSO within the A-Train: Ten years of actively observing the Earth system[END_REF] could also be used to analyze the contributions of LWP and IWP (ice water path) errors, but their spatial and temporal coverage is very limited for evaluating the performance of NWP models at hourly and kilometric resolutions.

Errors in the representation of the mixed phase were also highlighted by Forbes and Ahlgrimm (2014) in the IFS model and [START_REF] Barrett | Why are mixed-phase altocumulus clouds poorly predicted by largescale models? Part 2. Vertical resolution sensitivity and parameterization[END_REF] in five operational NWP models including the IFS model, as well as [START_REF] Engdahl | Improving the representation of supercooled liquid water in the HARMONIE-AROME weather forecast model[END_REF] in HARMONIE-AROME. They reported an underestimation of the supercooled liquid water content in boundary layer clouds. This would likely result in an overestimation of SWD [START_REF] Hogan | Characteristics of mixed-phase clouds. I: Lidar, radar and aircraft observations from CLARE'98[END_REF], which could explain the positive SWD bias for the clouds involved, namely CR2, CR4 and CR5. Although this error source is not consistent with the average negative SWD bias we found for CR4 in overcast conditions, it may account for some positive errors (see Fig. 12) for CR4 and participate in the positive bias for CR2 and CR5.

To conclude, several sources of error have been highlighted in the literature, some of which may contribute to the SWD errors reported in our study. Further investigations with more advanced observations should be used to overcome the limitations of the observations used in this study.

Conclusions

In this study, we performed a detailed evaluation of the 24 h SWD forecasts of the French NWP model AROME at 1.3 km horizontal resolution, comparing a full year of hourly forecasts with in situ SWD measurements from the pyranometer network operated by Météo-France. A preliminary analysis showed that errors mainly occur when clouds are present in the model and in the observations, while erroneously predicted and missed clouds contribute less to the overall errors, as these situations are less frequent. Missed clouds and erroneously predicted clouds mainly correspond to clouds with a low impact on the SWD. Errors in cloudy situations, which have a positive bias overall, can also result from errors in the simulated cloud fraction or in the cloud optical thickness. Since errors in the cloud fraction are difficult to evaluate over a large domain, we limited our study to overcast situations in the model, implying that the SWD overestimation cannot be attributed to an underestimation of the cloud fraction. In such overcast conditions, the overall bias is close to zero. We then quantified the SWD errors for different cloud regimes, corresponding to different cloud altitudes. In doing so, we found a systematic SWD negative bias for low clouds and a systematic SWD positive bias for high clouds, consistent throughout the year, which essentially compensate to the near zero bias. In addition to these systematic deviations, we found unsystematic errors with significant SDE for all cloud regimes, which is critical for the solar energy sector. Note that this study was based on only 1 year, and even if the results seem to be robust with similar SWD errors through the year, it might be relevant to extend this study to several years. A cross-comparison between cloud regimes and cloud types showed relatively good agreement between the model and observations, especially for low clouds, confirming that the negative bias of SWD is indeed related to low clouds and the positive bias of the SWD is indeed related to high clouds. We pointed out that the positive bias of ice clouds may be due to the failure to account for snow in the AROME radiation code or to the too low snow content of ice clouds, issues that could be addressed in the future. Other sources of SWD errors have been mentioned, such as LWP or mixed-phase representation errors.

Our results also suggest that some clouds are not detected by the satellite, highlighting the need for more detailed cloud observations to go further in error assessment. In addition, efforts should be made to evaluate the cloud fraction at AROME resolution over a large domain to fully characterize the performance of AROME for SWD forecasts. This may be addressed using large networks of ceilometers or more complete shortwave radiation measurements. To better understand which physical properties of clouds cause SWD errors, further evaluations that distinguish cloud fraction errors from errors in the vertical distributions of condensed water and cloud particle effective radius are needed and should be based on highly instrumented sites. Although the study focused on AROME, the presented methodology can be applied to any NWP model, and the detailed evaluation can provide valuable physical information about the model performance, paving the way for future model improvement.

https://doi.org/10.5194/gmd-17-1091-2024

Geosci. Model Dev., 17, 1091-1109, 2024

Figure 1 .

 1 Figure 1. Localization of the 168 pyranometers used from the network operated by Météo-France.

Figure 2 .

 2 Figure 2. Modeled cloud regime classification.

  ) with SWD mod the SWD in the model, SWD obs the SWD in the observations, and SWD clear,mod the theoretical SWD under clear-sky conditions from the model. The histograms of CSI are used to point differences in SWD between AROME and observations (Fig. 4), similarly to what Nielsen and Gleeson (2018) did to evaluate the NWP model HARMONIE-AROME against the Danish pyranometer network. The CSI roughly quantifies overall cloud transmittance, bearing the signature of both cloud cover and cloud optical thickness.

Figure 3 .

 3 Figure 3. Monthly relative frequency for each (a) cloud regime in AROME and clear skies (CS) and (b) cloud type in the satellite images and clear skies (CS), over 2020 for the pixels including the pyranometers.

Figure 4 .

 4 Figure 4. Clear-sky index distribution in the model and in the observations when SZA > 70 • over 2020.

Figure 5 .

 5 Figure 5. (a) Monthly mean SWD bias (bar height, in W m -2 ) and relative frequency (bar width) for each category during the year 2020: red for the hit cases, yellow for the false alarm cases, green for the miss cases and blue for the correct negative cases. The total contribution is the bar surface. (b) Monthly SWD standard deviation of errors (SDE) for each category during the year 2020.

Figure 6 .

 6 Figure 6. ln(SWD/SWD TOA ) errors as a function of AOD errors with the fit line during clear skies in the year 2020 when the SZA is less than 70 • .

Figure 7 .

 7 Figure 7. Monthly bias (bar height) and relative frequency (bar width) for (a) each cloud regime for the false alarm cases and (b) each cloud type for the miss cases. The vignette of the cloud regimes is inserted to facilitate the reading.

Figure 8 .

 8 Figure 8. Monthly mean SWD bias (bar height, in W m -2 ) and relative frequency (bar width) for each (a) cloud regime and (b) cloud type, during the year 2020 for the hit cases. The total contribution is the bar surface.

Figure 9 .

 9 Figure 9. Annual mean SWD bias for different AROME cloud fractions and for each cloud type during the year 2020. The number above each bar represents the frequency (in %) by cloud fraction range for each cloud regime.

Figure 10 .

 10 Figure 10. Monthly mean SWD bias (bar height, in W m -2 ) and relative frequency (bar width) for (a) all modeled cloud regimes and (b) all observed cloud types, in overcast situations, during the year 2020. The total contribution is the bar surface.

Figure 11 .

 11 Figure 11. Distribution of clear-sky index for CR0, CR1, CR2, CR3, CR4, CR5, CR6 and CR7 in AROME and in the observations when the SZA is less than 70 • .

Figure 12 .

 12 Figure 12. Distribution of SWD errors (bias and SDE indicated) for CR0, CR1, CR2, CR3, CR4, CR5, CR6 and CR7 cloud regimes in overcast situations.

Figure 13 .

 13 Figure 13. Cross-comparison of cloud regime/cloud type (in %) in hit cases. (a) For each CR, the relative frequency of each CT. The last column is the relative frequency of each CR for all hits in overcast situations. The sum of each row is equal to 1. (b) For each the relative frequency for each CR. The last column is the relative frequency of each CT over all hits in overcast situations. The sum of each row is equal to 1.

Table 1 .

 1 Table summarizing the different naming used to describe groupings of cloud regimes used in this article.

	Low clouds	Cloud regimes with clouds present in the lower vertical slices: CR0, CR1 and CR4
	High clouds	Cloud regimes with clouds present in the upper vertical slices (except CR6): CR3, CR5 and CR7
	Geometrically	Cloud regimes with clouds present in only one vertical slice: CR1, CR2 and CR3
	thin clouds
	Geometrically	Cloud regimes with clouds present in two or more vertical slices (except CR6): CR4, CR5 and CR7
	thick clouds

Table 2 .

 2 Observed cloud type classification.

	LC	Low clouds, which include very low and low clouds
	MC	Mid-level clouds, which include mid-level clouds
	SC	Semi-transparent clouds, which include high semi-transparent thin clouds, high semi-transparent moderately thick
		clouds, high semi-transparent thick clouds, high semi-transparent above low or medium clouds, and high semi-
		transparent above snow/ice
	HC	High clouds, which include high opaque clouds and very high opaque clouds
	FC	Fractional clouds, which include fractional clouds
	Others	Fill values or when the cloud type product indicates no clouds unlike the cloud mask product

  .

	https://doi.org/10.5194/gmd-17-1091-2024	Geosci. Model Dev., 17, 1091-1109, 2024

Geosci. Model Dev., 17, 1091-1109, 2024 https://doi.org/10.5194/gmd-17-1091-2024

https://doi.org/10.5194/gmd-17-1091-2024 Geosci. Model Dev., 17, 1091-1109, 2024

Acknowledgements. We thank Yves-Marie Saint-Drenan, Yves Bouteloup, Yann Seity, Marie Cassas and Emmanuel Fontaine for the fruitful discussions and/or the technical help.

Financial support. This research has been supported by the European Commission, Horizon 2020 Framework Programme (Smart4RES (grant no. 864337)) and the Région Occitanie Pyrénées-Méditerranée (grant no. 0296-217001-REGION SO-LAROME).

Review statement. This paper was edited by Axel Lauer and reviewed by two anonymous referees.

Data availability. The cloud satellite products developed by the NWC EUMETSAT SAF are available at https://www.icare.univ-lille.fr/asd-content/archive/ (() and the documentation at http://www.nwcsaf.org (NWC SAF Documentation, 2024). The aerosol product from Copernicus Atmosphere Monitoring Service ( 2024) is available at https://atmosphere.copernicus.eu/. The observations of shortwave downward radiation from the opobservation network of Météo-France (2024) are freely available for research purposes at https://donneespubliques. meteofrance.fr/?fond=produit&id_produit=298&id_rubrique=32. AROME forecasts of SWD used in this study are available at https://doi.org/10.5281/zenodo.7928622 [START_REF] Magnaldo | AROME forecasts of surface shortwave downward radiation for year 2020[END_REF]. As the data set used for this study is very large (800 GB), only the shortwave downward radiation data are available. The other parameters are available upon request from the corresponding author. Access to the AROME code can be requested via the ACCORD consortium web page (ACCORD, 2024): http://www.umr-cnrm.fr/accord/.

Author contributions. All authors designed the methodology. MAM performed the analysis and prepared the figures. MAM wrote the manuscript with contributions from all co-authors.

Competing interests. The contact author has declared that none of the authors has any competing interests.

Disclaimer. The sole responsibility of this publication lies with the authors. The European Union is not responsible for any use that may be made of the information contained therein.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.