Julia Linhart
email: julia.linhart@inria.fr

Alexandre Gramfort
email: alexandre.gramfort@inria.fr

Pedro L C Rodrigues
email: pedro.rodrigues@inria.fr

Validation Diagnostics for SBI algorithms based on Normalizing Flows

Building on the recent trend of new deep generative models known as Normalizing Flows (NF), simulation-based inference (SBI) algorithms can now efficiently accommodate arbitrary complex and high-dimensional data distributions. The development of appropriate validation methods however has fallen behind. Indeed, most of the existing metrics either require access to the true posterior distribution, or fail to provide theoretical guarantees on the consistency of the inferred approximation beyond the one-dimensional setting. This work proposes easy to interpret validation diagnostics for multi-dimensional conditional (posterior) density estimators based on NF. It also offers theoretical guarantees based on results of local consistency. The proposed workflow can be used to check, analyse and guarantee consistent behavior of the estimator. The method is illustrated with a challenging example that involves tightly coupled parameters in the context of computational neuroscience. This work should help the design of better specified models or drive the development of novel SBI-algorithms, hence allowing to build up trust on their ability to address important questions in experimental science.

Introduction

Recent advances in computing have led to a new generation of expressive simulators used to study complex systems in many scientific fields [START_REF] Cranmer | The frontier of simulation-based inference[END_REF]. They implicitly encode the intractable likelihood p(x | θ) of the underlying mechanistic model which relates observed data x ∈ R d to scientifically meaningful internal parameters θ ∈ R m . To perform statistical inference in this setting, one can recur to simulation based inference (SBI) [START_REF] Cranmer | The frontier of simulation-based inference[END_REF] to approximate the posterior distribution p(θ | x) using samples from the joint pdf p(x, θ). In this work, we consider SBI methods based on normalizing flows [START_REF] Papamakarios | Normalizing flows for probabilistic modeling and inference[END_REF], which are invertible neural networks that can be trained via maximum likelihood. Once the flow is trained, and for any new observation x, one can directly evaluate the estimated density over the parameter space (i.e. θ-space) [START_REF] Gonçalves | Training deep neural density estimators to identify mechanistic models of neural dynamics[END_REF], draw samples to construct confidence regions [START_REF] Masserano | Simulation-based inference with waldo: Perfectly calibrated confidence regions using any prediction or posterior estimation algorithm[END_REF], etc. Flow-based SBI has been used in many recent applied works [START_REF] Lueckmann | Benchmarking simulation-based inference[END_REF][START_REF] Bittner | Interrogating theoretical models of neural computation with emergent property inference[END_REF][START_REF] Gonçalves | Training deep neural density estimators to identify mechanistic models of neural dynamics[END_REF], but it lacks an important feature before becoming a technology for experimental science: validation. Ideally, one would like to have a method that provides finite-sample guarantees of nominal coverage (or calibration) of the estimated posterior regions, but also ensures that the approximation fits the true underlying posterior of the model when new data is observed. While solutions have been proposed to provide such finite-sample guarantees [START_REF] Dalmasso | Likelihood-free frequentist inference: Confidence sets with correct conditional coverage[END_REF][START_REF] Masserano | Simulation-based inference with waldo: Perfectly calibrated confidence regions using any prediction or posterior estimation algorithm[END_REF], assessing the convergence and consistency of the underlying Machine Learning and the Physical Sciences workshop, NeurIPS 2022. inference remains a challenging task [START_REF] Lueckmann | Benchmarking simulation-based inference[END_REF]. Simulation based calibration (SBC) [START_REF] Talts | Validating bayesian inference algorithms with simulation-based calibration[END_REF] is arguably the most popular metric for validating posterior approximations in the applied SBI community [START_REF] Boelts | Flexible and efficient simulation-based inference for models of decision-making[END_REF], but it only provides necessary conditions for consistency and fails to give any insight on the local behavior of the estimator. Moreover, it is a univariate procedure, thus ignoring any information about the coupling between parameters. Zhao et al. [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF] recently proposed local coverage tests (LCT). They leverage machine learning to evaluate their test quantities on different locations of the feature space (i.e. x-space). In 1D, the probability integral transform (PIT) provides necessary and sufficient conditions for consistency, which is not the case for the proposed multivariate extensions (e.g. HPD).

In this work, we present a multivariate version of LCT for density models based on normalizing flows. Our method comes with the same theoretical guarantees on local consistency and interpretable diagnostics as provided by [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF] in 1D. We also present a workflow that can be used as a practical user guide. Lastly, we provide numerical illustrations on a well known model from computational neuroscience [START_REF] Jansen | Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns[END_REF], and demonstrate the importance of trustworthy diagnostics for multivariate posterior distributions when correlations between parameter variables can play an important role.

Methods

Our conditional density estimator q ϕ is a normalizing flow defined for samples θ ∈ R m and is conditioned on observations x ∈ R d . It uses a Gaussian base distribution p(z) = N (0, I m ) and a bijective transformation defined for every x, T ϕ (.; x) := (T ϕ,1 (.; x), . . . , T ϕ,m (.; x)), with Jacobian J T ϕ (.; x), such that q ϕ (θ

| x) = p(T ϕ (z; x)) = p(z) det J T ϕ (z; x) -1 .
Our goal is to evaluate the local consistency [START_REF] Kim | Global and local two-sample tests via regression[END_REF] of q ϕ with respect to the true posterior density, i.e. whether for a given x the following null hypothesis holds:

H 0 (x) : q ϕ (θ | x) = p(θ | x), ∀θ ∈ R m . (1)
We define the multivariate probability integral transform (PIT) of θ at x and associated to q ϕ as the vector of m one-dimensional projections:

PIT m (θ, x, q ϕ ) = [P 1 (θ, x), . . . , P m (θ, x)], P i (.,

x) = F N (0,1) • T -1 ϕ,i (.; x), ∀i ∈ [1, m] (2
) where F N (0,1) is the cumulative distribution function (c.d.f.) of the univariate normal distribution.

Theorem 1: Local Consistency and multivariate PIT. For any x ∈ R d , the null hypothesis H 0 (x) holds if, and only if, the covariates of PIT m (θ, x, q ϕ ) conditioned on x are mutually independent and uniformly distributed over (0, 1). We refer to Appendix A for a detailed proof of this result.

p(θ | x) = q ϕ (θ | x) ⇐⇒ p(T -1 ϕ (θ, x) | x) = p(z) ⇐⇒ p(T -1 ϕ,1 (θ, x), . . . T -1 ϕ,m (θ, x) | x) = N (0, I m ) ⇐⇒ p(P i (θ, x) | x) = U(0, 1) ∀i ∈ [1, m] and {P i (θ, x) | x} i=1,..

.,m are mutually independent

We can now verify the null hypothesis of local consistency H 0 (x) (1) via m statistical tests for the uniformity of the 1D local PIT covariates and an additional test for their mutual independence.

Multivariate Local Coverage Tests (LCT).

Noting that for every i = 1, . . . , m we have

P i (θ, x) | x ∼ U(0, 1) ⇐⇒ ∀α ∈ [0, 1] r i,α (x) = P(P i (θ, x) ≤ α | x) = α (3)
and following the same approach as in [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF], we propose m test statistics

T i (x) := 1 |G| α∈G r i,α (x) -α 2 ∀i ∈ [1, m] , (4)
where G is a grid of α-values and the estimators r i,α are obtained by regressing I {Pi(θ,x)≤α} on x, which is optimal for E[I {Pi(θ,x)≤α} | x] = r i,α (x) (when using appropriate loss-functions [START_REF] Miller | On loss functions which minimii to conditional expected values and posterior probabilities[END_REF]). We refer to Algorithm 1 (resp. 2) in [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF] for computing the p-values (resp. confidence bands) associated to each test. Since there are m independent tests, we recur to a Bonferroni correction of the p-values (resp. confidence levels) [START_REF] Bland | Multiple significance tests: the bonferroni method[END_REF]. These tests with interpretable graphical diagnostics such as PP-plots or histograms that depict distributional deviations in 1D, as shown in Figure 1. If any of the uniformity tests is rejected, we reject H 0 (x). If not, we proceed to the mutual independence test.

We now assume that the covariates of PIT m conditioned on x are uniformly distributed over (0, 1), which also means that every coordinate of the flow-transformation T -1 ϕ (θ, x) follows a normal distribution given x (cf. Theorem 1). Their mutual independence is thus characterized by a covariance matrix equal to the identity I m . We are currently working on how to perform this check in practice.

The workflow in practice. Let D = {x n , θ n } N n=1 be a calibration dataset with (x n , θ n ) ∼ p(x, θ) which were not used to train q ϕ . We use D to calculate the PIT values PIT m (θ n , x n , q ϕ ) (2) and estimate our local test quantities as functions of x. We investigate the consistency of q ϕ in two parts:

(1) Global consistency check: first, we look at the global PIT-distribution, i.e. on average over the entire x-space. More specifically, we directly compute the empirical approximation of

r i,α = P(P i (θ, x) ≤ α) ≈ 1 N N n=1 I {Pi(θn,xn)≤α} , ∀i ∈ [1, m]
with samples from D to test the global uniformity of each PIT-covariate and check which one(s) might be responsible for making q ϕ deviate from the true posterior distribution. Note that passing such global test is only a neccessary condition for consistency of q ϕ , as it is insensitive to covariate transformations in x-space [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF] (and ignores the condition on mutual independence).

(2) Local consistency check: we construct m × |G| transformed datasets from

D D i,α = {(x n , W i,α n )} N n=1 , α ∈ G, ∀i ∈ [1, m] where W i,α n = I {Pi(θn,xn)≤α} .
We can then compute the test statistics defined in (4) for any new observation x and check whether H 0 (x) should be rejected or not. In the latter case we proceed to the mutual independence test. Only then can we conclude on the validity of H 0 (x) according to Theorem 1. If the check in (1) passes, this allows us to guarantee (or reject) consistency anywhere in x-space. Even if the check in (1) does not pass, analyzing local consistency allows to 'open the box' and better understand why and where in x-space the estimator fails. In such situations -as the goal is not to guarantee local consistency -it can be enough to test for the covariate-wise uniformity of PIT m , putting the test for mutual independence aside. This approach was adopted for our numerical illustrations in Section 3.

Numerical illustrations

We apply our method to check the validity of a posterior estimate q ϕ of the Jansen and Rit neural mass model (JRNMM) [START_REF] Jansen | Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns[END_REF]. This well known model from computational neuroscience takes parameters θ = (C, µ, σ, g) ∈ R 4 as input and generates time series x ∈ R 1024 with properties similar to brain signals obtained in neurophysiology. Parameter C influences the oscillatory behavior of the signals and (µ, σ) characterize their amplitude. The gain factor g rescales the signals and models the effects of the amplifier used for measuring them in practice. Note that the coupling-effect of g and (µ, σ) on the amplitude leads to intrinsic indeterminacies in the inversion of the model [START_REF] Pedro | Hnpe: Leveraging global parameters for neural posterior estimation[END_REF]. Our approximation q ϕ is a conditioned masked autoregressive flow (MAF) [START_REF] Papamakarios | Masked autoregressive flow for density estimation[END_REF] with 10 layers and implemented in the sbi package [START_REF] Tejero-Cantero | Sbi -a toolkit for simulation-based inference[END_REF]. Based on the setup in [START_REF] Pedro | Hnpe: Leveraging global parameters for neural posterior estimation[END_REF], we train q ϕ on 50 000 simulations from the JRNMM with a uniform prior defined on physiologically relevant values (cf. pair-plots in Figure 2).

Results and discussion. All test quantities are computed with a calibration dataset D containing 10 000 simulations and a grid of |G| = 100 α-values in [0, 1]. Following the workflow described in Section 2, we first check the global consistency of q ϕ . In the left part of Figure 1, our graphical diagnostics illustrate how the c.d.f. for every PIT-covariate deviates from the identity function (black dashed line), outside of the 95%-confidence region (in gray), thus rejecting the null hypothesis of global consistency. We compare our method to SBC (right part of Figure 1) as implemented in the sbi package. We observe that SBC is unable to detect any inconsistencies in q ϕ . We proceed to investigate the local consistency of q ϕ on different locations of x-space. Note that since the global test for consistency has not passed, we may focus on the uniformity of the local PIT-covariates (cf. (2) of the workflow in Section 2). We consider a 1D subspace in θ-space, where (C 0 , µ 0 , σ 0 ) are fixed and the gain g 0 varies in [-20, 20] to generate observations x 0 . The upper-left part of Figure 2 shows how the test statistics evolve with g 0 . We observe a strong deviation from Figure 1: Global Consistency of the JRNMM posterior estimator: PP-plots and corresponding histograms obtained for every global PIT covariate P i (θ) (right) and for SBC applied to every element θ i of θ (left). The gray zone indicates the 95%-confidence region of acceptance outside of which the uniformity test is rejected. SBC fails to reject the null hypothesis of global consistency.

Figure 2: Local Consistency Analysis. (Left) Diagnostics obtained for every PIT-covariate, when evaluated for different x 0 , simulated via the JRNMM with fixed parameters (C 0 , µ 0 , σ 0 ) and variable g 0 in [-20, 20]. The test-statistics are plotted as a function of g 0 and indicate inconsistent behavior (values are not constantly close to zero). Below, the PP-plots report the nature (bias/dispersion) of these inconsistencies for g 0 ∈ {-20, 0, 20}. The gray zone indicates the 95%-confidence region of acceptance outside of which the uniformity test is rejected. (Right) Pair-plots of q ϕ (θ | x 0 ) with ground-truth parameters θ 0 = (C 0 , µ 0 , σ 0 , g 0 ) (black dots and dashed lines) for g 0 ∈ {-20, 0, 20}. uniformity for covariates P 2 and P 3 : they vary smoothly in a 'U-shape', with higher values as g 0 deviates from zero. We also generate local PP-plots (lower-left part of Figure 2) and observe positive (resp. negative) bias for small (resp. high) values of g 0 . Our procedure shows that there are certain locations in x-space (here g 0 = 0) where q ϕ performs well (i.e. test statistics are close to zero and PP-plots show little deviations from the 95%-confidence region in gray), even though global consistency does not hold. These observations can be compared to the pair-plots (right part of Figure 2) which shows the estimated posterior density over the entire θ-space. We observe that gain values at the boundary of the prior support (g 0 = -20 and g 0 = 20) induce discrepancies (cf. marginals plotted in the diagonals of the pair-plots in Figure 2) that are detected by our PIT-based diagnostics.

The above results are obtained by directly applying the method from [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF] using their default regressor, MLPClassifier from scikit-learn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. Indeed, this model is well suited for the binary target variables W i,α and, based on neural networks, is able to scale to high-dimensional x-spaces. Furthermore, it has been shown [START_REF] Miller | On loss functions which minimii to conditional expected values and posterior probabilities[END_REF] that for a large enough dataset D (and number of training iterations), the cross-entropy loss converges to the optimal solution of our regression problem. Note that although such method allows to capture and visualize local discrepancies, it does not cope well with our high-dimensional, complex data-distributions (high variances represented by large confidence regions computed over 100 trials). Also, it requires the training of a large number (m × |G|) of regressors (one for each θ-dimension and α-value). Finally, this method is only applicable for the uniformity tests. We are currently investigating algorithms that are computationally more efficient and more accurate in estimating the full multivariate local PIT-distribution.

Conclusions. Numerical illustrations demonstrate that our diagnostics capture well the inconsistencies in q ϕ with respect to the true JRNMM posterior. To be precise, the proposed validation method is statistically more powerful and computationally more efficient than SBC and, importantly, it allows for a local analysis that reveals where in x-space the estimations should be improved. Our method exploits useful properties of modern normalizing flows. Indeed, contrarily to other SBI-strategies [START_REF] Hermans | Likelihood-free MCMC with amortized approximate ratio estimators[END_REF], they allow for efficient density evaluation and use bijective transformations involving the mixing between elements of θ. These transformations can be interpreted as a set of 1D-projections of θ, including information about its joint p.d.f. (i.e. interactions between its elements θ i ). The covariates of our multivariate PIT therefore define fast and easy to compute 1D test-quantities on which we can perform univariate LCTs that do not completely ignore correlations in the θ-space. Combined with a test to check their mutual independence, this would provide theoretical guarantees for consistency, which is not the case for HPD, the existing multivariate version of LCT proposed in [START_REF] Zhao | Diagnostics for conditional density models and bayesian inference algorithms[END_REF].

Ongoing work will investigate the link between PIT-covariates and their deviations from uniformity with the actual parameters of the model. This could reveal the true nature of their coupling which can be used for the development of better specified models (e.g. hierarchical posterior estimation [START_REF] Pedro | Hnpe: Leveraging global parameters for neural posterior estimation[END_REF][START_REF] Rouillard | ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models[END_REF]). Ultimately, the goal is to use our diagnostics as a tool for model selection.

Broader Impact

This work tackles an important open question in simulation based inference. We introduce theoretically valid and interpretable validation diagnostics that scale to both high-dimensional data and parameter spaces. Our contribution should help to further improve SBI methods and drive the design of better specified models, hence allowing to build up trust on their ability to address important questions in experimental science. Moreover, our work is in line with many other tentatives in the machine learning community of ensuring the quality and calibration of complex models based on neural networks.

A Appendix

Proof of Theorem 1. Let x ∈ R d . We consider the random variables defined on R m : Θ ∼ p(θ | x) and Θ ϕ ∼ q ϕ (θ | x). Our local null hypothesis (cf. Equation 1), can be rewritten as

H 0 (x) : Prob Θ ∈ Ω θ = Prob Θ ϕ ∈ Ω θ ∀Ω θ ⊂ R m (5)
From the definition of our normalizing flow q ϕ with bijective transformation T ϕ and normal base distribution, there exists Z ∼ p(z) = N (0, I m ), such that Θ ϕ = T ϕ (Z, x).

Replacing Θ ϕ by T ϕ (Z, x) in ( 5) and defining

Ω z = T -1 ϕ (Ω θ ; x), we have Prob Θ ∈ Ω θ = Prob T ϕ (Z; x) ∈ Ω θ ⇐⇒ Prob T -1 ϕ (Θ; x) ∈ Ω z = Prob Z ∈ Ω z
Considering T -1 ϕ,i , the i-th coordinate of T -1 ϕ , and Z 1 , . . . , Z m the mutually independent and normally distributed covariates of Z ∼ N (0, I m ), we can rewrite the right part of the above equivalence as:

Prob T -1 ϕ,1 (Θ; x) ∈ Ω z,1 ∩ • • • ∩ T -1 ϕ,m (Θ; x) ∈ Ω z,m = m i=1 Prob Z i ∈ Ω z,i (6)
where the Ω z,i are projections of the set Ω z on each of its dimensions.

Note that since the above equivalence is true for any choice of Ω θ , hence of Ω z , we can write what happens when we fix a given Ω z,1 and let Ω z,2 = • • • = Ω z,m = R:

Prob T -1 ϕ,1 (Θ; x) ∈ Ω z,1 = Prob Z 1 ∈ Ω z,1
Doing the same for all other coordinates and using these equalities in (6), we end up with Prob T -1 ϕ,i (Θ; x) ∈ Ω z,i [START_REF] Jansen | Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns[END_REF] where [START_REF] Jansen | Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns[END_REF] is the definition of mutual independence itself.

∀i ∈ [1, m], ∀Ω z,i ⊂ R, Prob T -1 ϕ,i (Θ; x) ∈ Ω z,i = Prob Z i ∈ Ω z,i (7)
We now apply the c.d.f. F N (0,1) to each random variable in [START_REF] Hermans | Likelihood-free MCMC with amortized approximate ratio estimators[END_REF] and [START_REF] Jansen | Electroencephalogram and visual evoked potential generation in a mathematical model of coupled cortical columns[END_REF]. Mutual independence stays true and the probability integral transform theorem (in 1D) states that F N (0,1) (Z i ) ∼ U(0, 1). We therefore get that the null hypothesis H 0 (x) holds if, and only if, P i (Θ, x) = F N (0,1) (T -1 ϕ,i (Θ; x)) ∼ U(0, 1), ∀i = [1, m] and {P i (Θ, x)} i=1,...,m are mutually independent [START_REF] Kim | Global and local two-sample tests via regression[END_REF] The result in Theorem 1 directly follows from rewriting ( 5) and ( 9) with initial notations from Section 2: remember that Θ ∼ p(θ | x), so P i (Θ, x) ∼ p(P i (θ, x) | x) and we get: where P i (θ, x) is the i th covariate of PIT m (θ, x, q ϕ ), the multivariate PIT of θ at x, associated to q ϕ .

p(θ | x) = q ϕ (θ | x

Conclusion:

The null hypothesis H 0 (x) holds if, and only if, the covariates of PIT m conditioned on x are mutually independent and uniformly distributed over (0, 1).

and Prob T - 1 ϕ, 1 (

 11 Θ; x) ∈ Ω z,1 ∩ • • • ∩ T -1 ϕ,m (Θ; x) ∈ Ω z,p = m i=1

  ) ⇐⇒ p(P i (θ, x) | x) = U(0, 1) ∀i ∈ [1, m] and {P i (θ, x) | x} i=1,...,m are mutually independent

Acknowledgments

Julia Linhart is recipient of the Pierre-Aguilar Scholarship and thankful for the funding of the Capital Fund Management (CFM). Alexandre Gramfort thanks the support of the ANR BrAIN (ANR-20-CHIA0016) grant.


Charlotte Z Bond

Carlos M Correia

Jean-François Sauvage

Kacem El Hadi

Yannick Abautret

Benoit Neichel

Thierry Fusco

Optimized calibration of the adaptive optics system on the LAM Pyramid bench

Keywords: Pyramid wave-front sensor, adaptive optics, optical gain

The Pyramid wave-front sensor (WFS) is currently the baseline for several future adaptive optics (AO) systems, including the first light systems planned for the era of Extremely Large Telescopes (ELTs). Extensive investigation into the Pyramid WFS aim to prepare for this new generation of AO systems, characterizing its behavior under realistic conditions and developing experimental procedures to optimise performance.

An AO bench at Laboratoire d'Astrophysique de Marseille has been developed to analyze the behavior of the Pyramid and develop the necessary operational and calibration routines to optimize performance. The test bench comprises a Pyramid WFS, an ALPAO 9 × 9 deformable mirror (DM), a rotating phase screen to simulate atmospheric turbulence and imaging camera. The Pyramid WFS utilizes the low noise OCAM 2 camera to image the four pupils and real time control is realized using the adaptive optics simulation software OOMAO (Object Oriented Matlab Adaptive Optics toolbox). 1

Here we present the latest experimental results from the Pyramid test bench, including comparison with current Pyramid models and AO simulations. We focus on the calibration of the AO system and testing the impact of non-linear effects on the performance of the Pyramid. The results demonstrate good agreement with our current models, in particular with the addition of more realistic elements: non-common path aberrations and the optical quality of the Pyramid prism.

INTRODUCTION

Research into new wave-front sensors has been motivated by the demands of the adaptive optics (AO) systems proposed for the next generation of ground based observatories, the Extremely Large Telescopes (ELTs). In particular the Pyramid wave-front sensor (WFS) has emerged as a prime candidate for the baseline for many of these future systems.

The Pyramid WFS, first proposed by Ragazzoni in, [START_REF] Ragazzoni | Pupil plane wavefront sensing with an oscillating prism[END_REF] primarily consists of a 4 sided prism onto which light from a telescope is focused. The four sides of the prism project four images of the telescope pupil, which are realized onto a detector via re-imaging optics. Any local tilt of the wave-front will alter the distribution of the light among the 4 pupils, providing a measurement of the incoming wave-front. The Pyramid WFS is an attractive prospect for current and future AO systems, with the potential to perform better than the commonly used Shack-Hartmann WFS. In particular the Pyramid can provide more sensitive wave-front measurements within the AO correction band, whilst exhibiting reduced susceptibility to aliasing. [START_REF] Vérinaud | On the nature of the measurements provided by a pyramid wave-front sensor[END_REF] Demonstration on the Large Binocular Telescope 4, 5 has provided a compelling case for its use on future large telescopes.

In order to fully exploit the potential of the Pyramid WFS, its behavior under realistic operating conditions must be fully understood. This is of paramount importance as we enter into the design phase for the AO systems of the ELTs, such as HARMONI. [START_REF] Thatte | Harmoni: the first light integral field spectrograph for the e-elt[END_REF] With this aim an experimental test bench has been set up at Laboratoire d'Astrophysique de Marseille (LAM). Here we present the experimental bench and summarize the Contact Charlotte Z. Bond: czbond@hawaii.edu latest experimental results. Previous work presented on the LAM bench includes: details of the design of the Pyramid WFS and initial bench assembly; [START_REF] Hadi | Toward an experimental validation of new ao concepts for future e-elt instrumentation[END_REF][START_REF] Hadi | Development of a pyramid wavefront sensor[END_REF] a study on different algorithms for processing the Pyramid signals; [START_REF] Bond | Experimental implementation of a pyramid wfs: Towards the first scao systems for e-elt[END_REF] and presentation of the final AO system and initial closed loop results. [START_REF] Bond | Experimental study of an optimised pyramid wave-front sensor for extremely large telescopes[END_REF]

PYRAMID EXPERIMENT AT LABORATOIRE D'ASTROPHYSIQUE DE MARSEILLE 2.1 Optical layout

Figure 1 illustrates the optical layout of the LAM Pyramid WFS experiment. The experiment includes:

• Monochromatic light in the form of a laser with λ = 660 nm.

• A rotating phase screen, to simulate the effect of a turbulent atmosphere, providing an effective turbulence of d r0 = 3.2, where d is the actuator spacing and r 0 is the Fried parameter. • AO correction provided by an ALPAO deformable mirror (DM), with 9 actuators across the pupil (67 actuators in total).

• A fast tip-tilt mirror which modulates the beam focused onto the Pyramid to adjust the linear range of the sensor. The modulation is provided at 500 Hz with amplitudes up to 10 λ D . • The Pyramid WFS, incorporating an OCAM 2 camera to realize the four images of the pupil, with 60 pixels across the pupil. The over-sampling, with respect to the DM, allows for future upgrades with a higher number of actuators across the pupil.

• An imaging camera to provide a measurement of the final PSF, providing a means of assessing the performance of the AO system.

• A Shack-Hartmann WFS with 32 lenlets across the pupil to provide comparative wave-front measurements.

• Flat mirrors for alignment and reference measurements.

Controlling the optical bench

Control of the bench in order to close the loop involves processing the Pyramid signals and sending commands to the deformable mirror. The Pyramid signals are computed as the normalized difference in x and y between the four pupils:

S x [n, m] = I 1 [n, m] + I 3 [n, m] -I 2 [n, m] -I 4 [n, m] I tot. S y [n, m] = I 1 [n, m] + I 2 [n, m] -I 3 [n, m] -I 4 [n, m] I tot.
(1) where I j is the intensity of the equivalent pixel [n, m] in the j-th pyramid pupil. I tot. is the average intensity contained within the valid regions of the 4 pupil images. 9

The AO system is controlled using cameralink to communicate with the OCAM 2 camera and the ALPAO Matlab interface to send commands to the DM. The real time control is performed in Matlab using the AO simulation code OOMAO, 1 allowing for easy comparison between experiment and simulation and ease of testing different algorithms. The code is used to calibrate the system, taking an interaction matrix and then inverting it to provide a command matrix, used to convert the Pyramid signals into DM commands. In closed loop these commands are sent to the deformable mirror to correct for the measured turbulence. The camera is operated at 500 Hz, matched by the modulator speed. The time taken for a complete set of DM commands to be applied to the DM is ≈ 1 ms, resulting in a top AO loop speed of ∼ 300 Hz.

System characterisation

To optimize the performance of the AO system, and to accurately predict and compare with models we require detailed knowledge of the AO system. In this section we summarize some of the characterization processes carried out.

Characterizing the phase screen

To access the final behavior of the AO loop we need to know the strength of the turbulence we are simulating on the bench. Using the Shack-Hartmann WFS measurements of the phase screen were taken at different rotational intervals. The measurements were decomposed into Zernike polynomials and their variance computed, see figure 2. From these measurements the Fried parameter, r 0 , with respect to the actuator spacing, d, was extracted, resulting in a value of d r0 = 3.2. This is representative of typical values expected for the ELTs (i.e. d = 50 cm r 0 = 15.5 cm).

Étalonnage du miroir modulant

Le miroir de modulation est guidé par deux canaux dont chacun correspond au Tip et Tilt. Le but de la modulation est d'introduire une déviation circulaire du faisceau focalisé au sommet de la pyramide. Le rayon de cette modulation, typiquement de quelques largeurs de di raction seulement, doit-être quantifier en terme de ⁄/D. Chaque canal est activé au moyen d'un générateur de tension qui imprime un mouvement sinusoïdale au miroir à une fréquence de 500 Hz. Il est important de connaître parfaitement le

Modulator calibration

The behavior of the Pyramid is strongly dependent on the level of modulation. This is often referred to in units of λ D . It is crucial that we accurately calibrate the modulation in order to compare the performance with our models. This was carried out by placing a camera at the focal plane of the Pyramid WFS and measuring the amplitude of the modulation, first individually calibrating the x and y axes and finally monitoring the quality of the modulation circle. This work was carried out by Yannick Abautret during an internship at LAM.

Characterisation of non-common path aberrations

Non-common path aberrations (NCPA) refer to spatial differences in the optical path between the WFS and imaging camera. To produce the best PSF on the imaging camera the NCPA should be quantified and used to offset the WFS signals to compensate.

On the LAM bench the NCPA was characterized using phase diversity, [START_REF] Lamb | Noncommon path aberration corrections for current and future ao systems[END_REF][START_REF] Lamb | Quantifying telescope phase discontinuities external to adaptive optics systems by use of phase diversity and focal plane sharpening[END_REF] in order to maximize the Strehl ratio on the imaging camera when the loop is closed. Three images were used, one image in focus, one with positive focus and one with negative focus. The result, using the first 20 Zernike modes, was an estimate of the NCPA with a wave-front error of 21.5 nm rms. The modal breakdown is summarized in figure 3, showing the majority of the NCPA is astigmatism (mode number 5).

INITIAL PYRAMID MEASUREMENTS AND AO CALIBRATION

Measurements with flat wave-fronts

Once the optical bench was assembled and aligned the first measurements using the Pyramid were taken. Firstly a flat wave-front was sent to the Pyramid WFS, by flattening the deformable mirror. These first measurements are shown in figure 4. The images shown here show the 4 pupil images for different modulation amplitudes. The pupil configuration has not been specifically optimized for this optical setup and could easily be improved. The crucial aspect is to have enough pixels across the pupil to measure all the modes provided by the DM and to provide sufficient separation of the pupils to avoid interference, both of which are achieved here. Comparing the different modulation amplitudes shown here, we can see that without modulation there is a considerable amount of diffracted light outside the pupils. We also observe an imbalance in the light falling in the different pupils, with more light directed along one diagonal than the other. This is indicative of a 'roof-top' at the tip of the Pyramid prism and causes a drop in sensitivity for spatial modes oriented along the faint diagonal. As the beam is modulated about the Pyramid tip the impact of the roof-top is reduced, resulting in a more uniform distribution of light into the 4 pupils. These first measurements illustrate the importance of taking a reference WFS measurement corresponding to a flat wave-front. 13 Without this measurement the AO loop will try to compensate such effects with a non-flat wave-front. Such a reference measurement can also include offsets in order to produce the best possible image on the science camera. We will examine this in more detail in section 4.2.

Calibrating the Pyramid AO system

To close the loop we first calibrate the system in terms of the relationship between the deformable mirror modes and the Pyramid WFS signals (measuring the interaction matrix). Figure 5 shows the WFS measurements when an interaction matrix is taken using Zernike modes for a modulation of 3 λ D . In this case each mode corresponds to the same wave-front rms (10 nm) and the WFS signals demonstrate the variable sensitivity of the Pyramid to different modes (specifically different spatial frequencies). The modulated Pyramid exhibits the response of a slope sensor at low spatial frequencies (below the spatial frequency equivalent to the modulation amplitude). [START_REF] Vérinaud | On the nature of the measurements provided by a pyramid wave-front sensor[END_REF] Above this the response flattens out. This is exhibited here, with some variation over the different modes within one radial order. The consequence of this is, when the interaction matrix is taken with a fixed modal rms the resulting Pyramid signals will vary in strength across the different modes, with the lower orders exhibiting weaker signals with greater susceptibility to measurement noise. For control of the LAM bench we implemented measurement of the interaction matrix using different modal amplitudes to achieve a roughly flat response. The WFS signals for such a measurement are shown in figure 6.

NON-LINEAR EFFECTS AND OPTICAL GAIN

Linear range

The Pyramid WFS is often compared favorably with the Shack-Hartmann WFS in terms of the sensitivity of the sensor to the incoming wave-front. Another aspect to consider is the linear response of the sensors. The Pyramid wave-front sensor has a relatively short linear range, particularly when working without modulation.

Modulating the Pyramid allows for increasing the linear range of the sensor, at the expense of sensitivity. Ideally the modulation can be tuned to optimize the sensitivity and linearity for given conditions, i.e. for better or worse atmospheric seeing.

Measurements of the response of the Pyramid WFS on the LAM bench were taken by applying a ramp of a given mode to the deformable mirror in open loop. The results for two different modes are shown in figure 7, using different modulation amplitudes. The low order mode (focus) demonstrates an increase in linear range as the modulation is increased from 1 -3 λ D . The higher order mode (n = 6, m = 2) requires a larger modulation amplitude to observe a noticeable difference in linear range. The higher the spatial frequency the greater the The response is measured for different modulations of the Pyramid and for different spatial modes: focus (left) and a higher order Zernike mode with n = 6 and m = 2 (right). In the higher order mode case a simulation result is plotted for comparison.

modulation amplitude required to affect the Pyramid WFS response. This is related to the point at which the modulation equivalent spatial frequency ( m λ ) is greater than or equal to the spatial frequency of the mode.

Pyramid operation off-null

So far we have considered the Pyramid operating around 0, where 0 represents a flat wave-front and corresponds to a diffraction limited PSF focused onto the tip of the Pyramid. Expected operating conditions for the ELTs will degrade the PSF. This can be due to offsets applied to the Pyramid WFS operating point, required to compensate for non-common path aberrations (NCPA) between the WFS and science camera; or to the residual phase errors left after AO correction. The result is a wider PSF which reduces the signal produced by a given wave-front. A sensor calibrated under nominal conditions (a diffraction limited PSF) can be described as having a reduced optical gain, or reduced sensitivity, when operating under realistic conditions.

A reduced optical gain can impact the performance of the Pyramid wave-front sensor as it underestimates the incoming wave-front. When working with an offset applied to the WFS this can result in the offset being incorrectly applied and, for example, an over-correction of any NCPA. This is illustrated in figure 8. Here a focus offset is applied to the WFS during closed loop operation, applying this offset directly to the reconstructed modes. For comparison the open loop Pyramid response is also plotted, along with a linear reference.

When the applied offset is small in amplitude (within the linear range) the system is able to correctly apply the offset in closed loop. However, a larger offset results in an underestimate of the measured wave-front, shown here by the deviation of the open loop response from linear. The consequence in closed loop is an over-application of the offset. The focus measurement saturates around 140 nm, as shown by the maximum of the open loop measurement. If an offset > 140 nm is attempted the system will always measure the applied offset as 140 nm, causing the control system to continuously apply a larger offset, in an attempt to get the measurement to the set value. The loop at this point becomes unstable. This result is illustrative of the potential problem when trying to compensate for a large NCPA. Attempts to mitigate this effect (within the stable region) have lead to the proposal of schemes for tracking and correcting the optical gain and correctly applying NCPA correction.

CLOSING THE LOOP

Here we present the performance of the LAM Pyramid bench in closed loop, with the phase screen providing dynamic turbulence ( d r0 = 3.2) and the loop controlled using Zernike modes. The loop is closed using different modulation amplitudes and assessed by the PSF on the imaging camera, specifically the Strehl ratio.

Figure 9 summarizes the performance in closed loop. The long exposure image demonstrates a Strehl of up to 30% (with high modulation). With low modulation the performance drops to ∼ 20%. Although a low modulation produces higher sensitivity measurements, the impact of any optical imperfections or non-linear effects is greater, leading to a drop in performance. This can be observed by comparing the experimental results with two simulations. A perfect simulation exhibits a flatter response, with a Strehl ratio between 30 and 35%. The realistic simulation includes:

• The roof-top effect on the tip of the Pyramid • Non-common path aberrations

• Zernike modes projected onto the DM Including these effects produces a result close to that observed on the bench, exhibiting a greater variation in Strehl ratio across the range of modulation and specifically sees the larger drop at low modulation. This result provides confidence in both our models of the Pyramid WFS and the experimental bench, as we are able to recreate the experimental results and explain our observations.

CONCLUSION

Here we have presented the latest experimental results achieved on the Pyramid WFS bench at Laboratoire d'Astrophysique de Marseille. Our system uses a Pyramid WFS to close the loop using a 9 × 9 ALPAO DM, a tip-tilt mirror to provide modulation and a phase screen to provide turbulence representative of that expected for ELTs. Real time control is provided by the simulation OOMAO 1 in Matlab. The AO system is calibrated, first taking reference signals for a flat wave-front and then measuring an interaction matrix to calibrate the response of the Pyramid WFS to the DM modes. On the LAM bench the amplitude of the modes used in the measurement of the interaction matrix are tuned to provide a flat modal response (in terms of the WFS signals). The loop is closed using a reference signal to compensate for NCPA between the imaging camera and WFS, as estimated using phase diversity. A maximum Strehl ratio of ∼ 30% is achieved at the WFS wavelength (λ = 660 nm), with a dip in performance at low modulation due to the impact of non-linear effects and the optical quality of the Pyramid. These results are in good agreement with simulations of the bench, especially when realistic effects such as NCPA are included.

In addition to closing the loop with the Pyramid, we have begun to investigate the non-linear behavior of the Pyramid WFS and in particular the impact when working with an offset applied to the AO loop. We have shown that the small linear range of the Pyramid can result in an underestimate of the incoming wave-front and, in the case of off-null operation, an over application of the offset modes. In order to exploit the full sensitivity of the Pyramid WFS we will need to further investigate these issues and devise methods of compensating for the reduced optical gain experienced under realistic operating conditions. Such work is currently underway.

Figure 1 .

 1 Figure 1. Diagram of the optical layout of the Pyramid experiment at Laboratoire d'Astrophysique de Marseille. Different optical paths are available, in order to test the system in non-turbulent conditions and provide calibration measurements by way of flat reference mirrors. Measurements of the final PSF and the wave-front are made using an imaging camera, Shack-Hartman WFS and the Pyramid WFS.

Fig. 19 .

 19 Fig. 19. Erreur relative à l'estimation de l'échelle externe.

Fig. 20 .

 20 Fig. 20. Fit sur les données issues des mesures sur le banc LOOPS pour un écran de phase.

Figure 2 .

 2 Figure 2. Plot showing the variance of the Zernike modes measured from the phase screen installed on the Pyramid bench. A fitted curve is shown, corresponding to an atmospheric turbulence of D r 0 = 25.5 where D is the pupil diameter. With the current AO system this gives d r 0 = 3.2, where d is the actuator spacing. This work was carried out by Yannick Abautret during an internship at LAM.

  front[nm]

Figure 3 .

 3 Figure 3. Non-common path aberrations (NCPA) on the LAM bench estimated using phase diversity. Left: Surface map of estimated NCPA, corresponding to an rms wave-front error of 21.5 nm. Right: Zernike coefficients of the first 20 modes estimated in the NCPA (up to radial order n = 5).

Figure 4 .

 4 Figure 4. The four pupil images produced by the Pyramid WFS on the LAM bench for a flat wave-front. The 3 different cases shown here correspond to different modulations, from left to right: no modulation; 2 λ D modulation; and 5 λ D

Figure 5 .

 5 Figure 5. Pyramid WFS signals measured for the first 65 Zernike modes during measurement of an interaction matrix on the LAM bench. Each modes corresponds to 10 nm rms wave-front error and a modulation of 3 λ D was used. Left: WFS signal Sx for the first 65 Zernike modes. Right: Average (over Sx and Sy) WFS signal rms for the first 65 Zernike modes.

Figure 6 .

 6 Figure 6. Pyramid WFS signals measured for the first 65 Zernike modes during measurement of an interaction matrix on the LAM bench. The amplitude of each mode is chosen to produce a WFS signal rms of ∼ 0.1 and a modulation of 3 λ D was used. Left: WFS signal Sx for the first 65 Zernike modes. Right: Average (over Sx and Sy) WFS signal rms for the first 65 Zernike modes.

Figure 7 .

 7 Figure 7. Response of the Pyramid WFS to an applied wave-front, where the amplitude of the applied wave-front is varied.The response is measured for different modulations of the Pyramid and for different spatial modes: focus (left) and a higher order Zernike mode with n = 6 and m = 2 (right). In the higher order mode case a simulation result is plotted for comparison.

Figure 8 .Figure 9 .

 89 Figure 8. Plot showing the effective closed loop wave-front error introduced when an offset is applied modes reconstructed from Pyramid WFS signals. The example shown here applies a focus offset and shows the rms of the closed loop wave-front vs. the applied offset. Also shown is the open loop response of the Pyramid wave-front sensor and a linear reference (grey line).

  [START_REF] Esposito | Non common path aberration correction with non linear wfss[END_REF][START_REF] Bond | Optimising the performance of a pyramid wfs: tracking the optical gain[END_REF]

ACKNOWLEDGMENTS

The research leading to these results received the support of the A*MIDEX project (no. ANR-11-IDEX-0001-02) funded by the Investissements d'Avenir French Government program and managed by the French National Research Agency (ANR). This research was co-funded by the People Programme (Marie Curie Actions) of the European Union's Seventh Framework Programme (FP7/2007-2013) under REA grant agreement n. PCOFUND-GA-2013-609102, through the PRESTIGE programme coordinated by Campus France.


F.-R Orthous- Daunay
email: francois-regis.orthous-daunay@univ-grenoble-alpes.fr

J Isa

C Wolters

V Vuitton

R Thissen

H Naraoka

MOLECULAR GROWTH PATTERN OF SOLUBLE CHN COMPOUNDS FROM RYUGU

Introduction: Primitive extraterrestrial materials including carbonaceous meteorites contain a variety of organic matter. It has been empirically split into phases depending on the extraction method from the bearing rock. So-called soluble organic matter (SOM) covers everything that can be mobilized in a solvent. The diversity of SOM can be investigated thoroughly down to molecular identification, revealing numerous compounds, some known to be used by terrestrial life and some not. The origin and role in the emergence of life of the SOM are difficult to assess because of multiple synthesis and contamination pathways.

The Ryugu regolith sample brought to Earth by the Hayabusa 2 mission is a unique instance of the material of a C-type asteroid and can be compared to carbonaceous chondrites. For this sample, contamination is under control. The travel from the parent body extraction to the surface of the Earth included collection in situ followed by ~15 months of storage in a very protective environment in the spacecraft sampling module. This mitigates against organic contamination or modification during atmospheric entry.

In this study, we focused on the description of the 'complexity' the molecular mixture, probed by high resolution mass spectrometry and chromatographic separation. By mixture complexity, we mean mass spectrometric features that exist only because of the presence of several compounds or types of compounds. The main descriptor of complexity we use is the distribution of masses observed by mass spectrometry and associated with molecular formulas that share molecular characteristics as much as possible (e.g., the mass distribution of all compounds bearing no oxygen, 1 nitrogen atom, with DBE=4 and retention time of 1200±100 seconds). These proxies have been selected because they involve tens of items and are unlikely to be found in two different sample by chance. This statement of 'complexity' has been shown to provide robust fingerprints of synthesis processes in applications to polymer science [1], [2].

Recently, the evolution of such distribution has been used to help tracking the effect of aqueous alteration on a carbonaceous chondrite [3] as well as the effect of UV irradiation [4]. These interpretations relied on the identification of a typical polymerization pattern thanks to a good match with a theoretical model and its alteration, and agreement with independent markers like clay abundance or experimentally received photon dose.

Experimental Methods: In order to access the organic compounds in the Ryugu regolith sample, liquid extractions were performed at Kyushu University under the supervision of H. Naraoka. A small aggregate sample of Ryugu grains from the first touchdown site (A0106) were used in this study. The aggregate sample of A0106 (17.15 mg) was extracted sequentially with hexane, dichloromethane (DCM), methanol and H2O using a sonicator in a 1.5 mL Teflon vial followed by centrifugation. All extraction procedures were performed on an ISO 5 (Class 100) clean bench inside an ISO 6 (Class 1000) clean room. All glassware used in this study were heated in an oven at 500°C for 3 h prior to analysis to remove possible organic contaminants. Baked serpentine powder (500°C for 3h) was analyzed as a procedural blank.

The chromatographic separation and highresolution mass spectral analysis was achieved by nano-liquid chromatography coupled with Orbitrap mass spectrometry using electrospray ionization (ESI). The positive mass spectra were acquired by full scan over a range of m/z 62 to 500 with a mass resolution m/m at m/z 200. For the rest of this study, we'll focus on the methanol extracts separated on an Amide HILIC column (75 m i.d. × 15 cm) with an acetonitrile to water elution gradient before electrospray ionization (ESI) and Q-Exactive-Orbitrap mass spectrometry.

The treatment of the resulting time-versus-mass data field took place at IPAG in Grenoble, France. It consisted of the isolation of each peak and automatic splitting of peaks into so-called plexuses. Those plexuses are used for the complexity description referred to in the introduction. Results from A0106 were compared to Murchison with the same extraction method.

Results and Discussion:

The organic compound complexity of the Ryugu methanol extract is, to first order, like that for extracts from Murchison samples. Specifically, the

2367.pdf 53rd Lunar and Planetary Science Conference (2022)

chromatographic separation resulted into one single major split between water soluble and water insoluble compounds, as described in [START_REF] Naraoka | The Hayabusa2-initial-analysis core: Shogo Tachibana[END_REF] for the homologous series of 1-N bearing molecules. This discrepancy affects the whole sample, and compounds as well as plexuses will be referred to as belonging to either the low retention time (low RT, less water soluble) and high RT (more water soluble).

The most remarkable feature of our measurements of the Ryugu sample extract is a ubiquitous, bellshaped distribution in the CH2-plexuses. This is the typical polymerization pattern found as evidence of pristine preservation of organic matter in Tagish Lake [3]. The pattern is detected in the extract regardless of the retention time (solubility in water) and the saturation degree of the molecules. Figure 1 and2 show couples of plexuses found in A0106 and Murchison for some 1-N bearing molecules. Comparable features are seen for other chemical compositions. For some degrees of saturation, a slight shift toward higher masses is observed for Ryugu's extract compared to Murchison.

Distributions on an asteroid regolith and a meteorite being similar is consistent with a common origin of their constituents. Furthermore, the fact that A0106 is in some cases shifted to higher masses is inconsistent with the prediction from solar wind irradiation or heavy ions bombardment that could have occurred on the asteroid surface.

The mass distribution of various classes of compounds in the Ryugu extract is among the broadest ever measured in laboratory experimental analogues simulating gas phase synthesis, liquid phase polymerization or UV irradiation of ices. This means that to-date, the complexity pattern found in Ryugu extracts is dissimilar from those in any prior terrestrial experiments. The synthesis process responsible for the measurements of Ryugu extracts remains to be determined.

Figure 1 :

 1 Figure 1: Three polymerization patterns found in Ryugu (square, top) and Murchison (circles, bottom) extracts with DBE=4. Yellow and blue indicate the water-soluble compounds (high RT) and waterinsoluble compounds (low RT) respectively.

Figure 2 :

 2 Figure 2: Two patterns found on Ryugu extract as well as one found in Murchison.