Matthieu Pizenberg
email: matthieu@pizenberg.fr

Axel Carlier
email: axel.carlier@irit.fr

Emmanuel Faure
email: emmanuel.faure@irit.fr

Vincent Charvillat
email: vincent.charvillat@irit.fr

Web-Based Configurable Image Annotations

Keywords: annotation, open source software, dataset

Figure 1: Screenshot of the interface of our image annotation Web application.

INTRODUCTION

Image annotations are required in a wide range of applications including image classification (which requires textual labels), object detection (bounding boxes), or image segmentation (pixel-wise classification). The rise and successes of deep learning lead to an increasing need for annotations, as training sets should be of a large size for these algorithms to be efficient. Yet, researchers still spend time and resources to create ad hoc tools to prepare those datasets. The application we present in this paper aims at providing a customizable tool to ful ll most image annotation needs.

Many image annotation applications already exist (Table 1). La-belMe [START_REF] Bryan C Russell | LabelMe: a database and web-based tool for image annotation[END_REF], one of the most popular, provides an interface for drawing bounding boxes and polygons around objects in an image. It has been used extensively to create datasets for image segmentation. Some more recent softwares share the same goals, with their own speci cities. For example, Labelbox [START_REF]Labelbox[END_REF] and Dataturks [START_REF]Dataturks[END_REF] provide annotation tasks management, particularly useful when crowdsourcing the annotations; these softwares are proprietary. The VGG Image Annotator (VIA [START_REF] Dutta | VGG Image Annotator (VIA)[END_REF]) is an open-source client application like ours, with the speci city of providing annotation attributes, editable in a spreadsheet format.

We release an open-source application [START_REF]Application source code[END_REF], entirely client side, meaning that no data is uploaded to any server. Images are loaded from les and annotated locally, in the browser. The simplest tool, from a user perspective, should be immediately available i.e. should not require any additional installation to be fully functional. Our image annotation software is thus a Web-based application, easily con gurable to t users needs, as well as embeddable in the Mechanical Turk platform to design crowdsourcing campaigns.

We rst present the features of our application, then describe its architecture. Finally, we explain how it can be used to start crowdsourcing experiments.

PRESENTATION OF THE APPLICATION

A screenshot of the application can be seen in Figure 1. The image to be annotated occupies the central part of the screen; a toolbar is located on top, object classes are available on the left and images to be annotated on the right.

Images. Multiple images can be loaded at the same time using the image icon on the top-right corner of the application. These images are not uploaded on the server, and can either be loaded locally from the client's machine, or from a distant server.

Tools. Our application includes several tools to annotate images. Icons for these tools are depicted in Figure 2. From left to right, the rst available annotation is the point, that can be useful to designate objects in the image. It can also be used as a seed in regiongrowing image segmentation methods. The second annotation we included is the bounding box, which provides the localization of objects in the image, and is used in object detection problems. The information we acquire are the left, right, top and bottom coordinates of the bounding box. The third annotation we chose to implement is the stroke, or scribble, which is a popular interaction in image segmentation. It consists in a sequence of points, interpreted as a continuous line. The outline, fourth type of annotation, is a closed shape, typically drawn around objects. It is comparable to a bounding box in essence, but provides a more precise location of objects. Finally, polygons can also be drawn (as in LabelMe, for instance), by successively clicking new points as vertices.

All these tools are available both with a mouse or a touch interaction. As a matter of fact, some tools are better suited to touch devices (for example, outlines) than others (polygons). Object classes. For most annotation tasks, we also need to differentiate objects in the images. Typically each annotated area is attributed a class, or label. The PASCAL VOC dataset [START_REF] Everingham | The pascal visual object classes (voc) challenge[END_REF], for example, is composed of 20 classes, grouped by categories:

• Person: person • Animal: bird, cat, cow, dog, horse, sheep • Vehicle: aeroplane, bicycle, boat, bus, car, motorbike, train • Indoor: bottle, chair, dining table, potted plant, sofa, tv/monitor In our application, classes are speci ed in a JSON con guration le. A strict corresponding con g for PASCAL VOC classes is presented in Listing 1.

To attribute a class to an annotation, a user should rst select the class in the left sidebar, then use a tool to create an annotation. Selecting a class in the left sidebar also highlights the annotations corresponding to this class.

Con guration le. The ve annotation tools are optionally made available by the con guration le. In Listing 1, the last line of the depicted con guration le contains an annotations eld, listing the tools that should be available. In this case, they all are.

In addition to the ve fundamental annotation types, each type can be derived in virtually any number of variations. For example, interactive segmentation algorithms often require foreground and background scribbles. In our application, this would mean the user would need to draw two types of strokes. This can be achieved using the con guration le, as in Listing 2. Such con guration would result in two stroke icons in the toolbar, of di erent colors, just as in Figure 1.

TECHNICAL CHOICES

The application code is organized in two parts:

• A minimalist Node.js server, located in the server/ directory.

It is statically serving the content of server/dist/ with compression.

• A complete Elm client application, located in the client/ directory. Elm [START_REF] Czaplicki | Elm[END_REF][START_REF] Czaplicki | Asynchronous functional reactive programming for GUIs[END_REF] isn't a JavaScript framework, it is a functional programming language, compiling to JavaScript to run in browsers. Its syntax is inherited from Haskell but far simpler. The compiled application is 150 KB gzipped, which is great for low bandwidth connections. The application architecture enforces a unidirectional data transformation ow, visualized in Figure 3. The central entity is the Model. It contains all and every information about our application state. The visual aspect of our application is called the View (basically an HTML rendered document) which is generated by the view function, from the Model. Finally, all events generate messages, of type Msg. The update function, updates the model by reacting to those messages, closing the loop.

All functions are pure, meaning there is no side e ect, outputs of functions entirely de ned by inputs. There cannot be global variables mutations, real world events, network interaction etc. Basically such a program would be running in a predestined way from its start to its end, preventing us from loading images and interacting with them. This is why the application is attached to the Elm runtime, provided by the language, transforming all real world events ("side e ects") into our de ned set of messages, of type Msg.

The main challenge with pure functions is to describe side e ects without performing them. Those are described in three locations:

(1) View attributes as DOM event listeners for pointer events.

(2) Commands (Cmd) generated by the update function, like loading of images. (3) Subscriptions (Sub) to outside world events like the window resizing. The Elm runtime takes those side e ect descriptions, perform them, and, whenever there is a result / an answer, transforms it into one of our de ned messages (Msg) and routes it to our update function.

The model states

The state is the main component of the Model. It contains the images and con guration loaded as well as the annotations performed. Its type is de ned as in Listing 3 and can be modeled as a nite state machine, visualized in Figure 4.

The application available online starts in state 0 (NothingProvided) and enables you to reach state 2 (AllProvided) with buttons to load images and con guration. Two messages called LoadImages and ConfigLoaded produce transitions in the state machine. Listing 4: Msg type de nition.

The messages

All modi cations of the model are understood by looking at the Msg type de nition (Listing 4). The update function then performs the modi cations described by those messages.

• The WindowResizes message is triggered when the application is resized. In the update function, it takes the new size and recomputes some view parameters.

• A PointerMsg message is triggered by pointer events (mouse, touch, etc.). In the update function, this is the message activating all the annotations logic code of our application.

• Files are handled by ve messages:

-When loading images from the le explorer, a LoadImages message is generated with a list of the images les and their names as identi ers. For each image correctly loaded an ImageLoaded message is generated, providing a local url, corresponding to the image in memory. -The messages LoadConfig and ConfigLoaded behave similarly.

-The Export message causes the application to serialize into JSON all the annotations, and asks the user to save the generated le. It is triggered by clicking on the export button of the top action bar.

• Whenever an event should change the zooming level of the drawing area, a ZoomMsg message is generated.

• Finally, the RemoveLatestAnnotation message also explicit.

The view

The view of this application is based on four components, each implemented in its own module, with potentially di erent versions depending on the current state of the application.

• The top action bar (src/View/ActionBar.elm).

• The center annotations viewer area (src/View/AnnotationsArea.elm).

• The right images sidebar (src/View/DatasetSideBar.elm).

• The left classes sidebar (src/View/ClassesSideBar.elm).

Library and application duality

In order to o er a turnkey solution to image annotations, we created a con gurable application solving most needs. But we also thought of cases where advanced modi cations are required. Consequently, the foundation of this application has been extracted in the independent package elm-image-annotation [START_REF]Image annotation package[END_REF] This package also contains the following important modules, under the Annotation namespace:

• Annotation.Style: de nes types describing appearance of points, lines and llings of annotations.

• Annotation.Svg: exposes functions rendering SVG elements for each annotation kind.

• Annotation.Viewer: manages the central visualization area, supporting zooming and translations, relative to an image frame. If you are interested in creating another rendering target than SVG, like canvas, WebGL, . . . , it would require alternative modules to Annotation.Svg and Annotation.Viewer. The rest of the code can stay unchanged.

CROWDSOURCING ANNOTATIONS

Image annotation interfaces are often used in the context of large datasets of images to annotate. As such, tasks management for crowdsourcing campaigns is an important feature. Labelbox and Dataturks are all-in-one services providing tasks management directly in their applications. Just like LabelMe, we choose instead to provide a con guration, ready to use with Amazon Mechanical Turk (Mturk).

Mturk comes in two sides. A "requester" is de ning a set of tasks while a "worker" is performing them. Workers are payed by requesters through Mturk service. The concept of a "HIT" (Human Intelligence Task) characterizes the task unit. In our case, one HIT means one image to be annotated. We describe in details how to setup a campaign with our template in the application documentation.

CONCLUSION

In this paper we have introduced our web-based image annotation application. More information is available in the online documentation [1]. The application is still actively developed, we welcome all feedback and contributions.

Figure 2 :

 2 Figure 2: Annotation tools icons

3 [ bbox 4 , 5 ] 6 }

 3456 { type : stroke , variations : [ fg , bg ] } Listing 2: A con guration le to include two types of strokes.

Figure 3 :

 3 Figure 3: The application architecture.

1 type State 2 = NothingProvided 3 | 4 | 5 |Listing 3 :Figure 4 :type Msg 2 = WindowResizes Device . Size 3 --pointer events 4 | PointerMsg Pointer . Msg 5 --select things 6 | SelectImage Int 7 | SelectTool Int 8 | SelectClass Int 9 --files 10 |

 2345342345678910 Figure 4: The application states.

Table 1 :

 1 Most relevant image annotation Web applications.

	Application Year Tools	Con gurable interface	Tasks management	Type License
	LabelMe	2008 bbox, polygon, iterative semi-automatic segmentation no	Mturk integration server	OSS
	VIA	2016 bbox, polygon, point, circle, ellipse	no	no	client	OSS
	Labelbox	2018 bbox, polygon, point, line	yes	yes	server private
	Dataturks	2018 bbox, polygon	no	yes	server private
	Ours	2018 bbox, polygon, point, stroke, outline	yes	Mturk integration client	OSS

  . It is designed as an API to create, modify and visualize shapes, useful in the context of image annotation. Modules for manipulation and serialization (in JSON) of annotations are under the Annotation.Geometry namespace. It already contains one module for each tool presented earlier. If you want to introduce a new tool, this is where you can create a new module.

ACKNOWLEDGMENTS

We would like to thank:

• @tforgione and @GarciaDelMolino for your wise feedbacks.

• @dncg for your Windows tests.

• The online Elm community for their help along the road: @evancz for the delightful Elm language, @ianmackenzie for your fantastic geometry library, @mdgri th for your very refreshing layout library, @luke for the amazing tool Ellie, @norpan, @jessta, @loganmac, @antew, for your invaluable help on slack.


domain: spi phys

text:
S Vievard

F Cassaing

L M Mugnier

A Bonnefois

J Montri

Real-time full alignment and phasing of multiple-aperture imagers using focal-plane sensors on unresolved objects

Keywords: Telescopes, Alignment, Co-phasing, Phased-array imaging systems, Active or adaptive optics, Wavefront sensing, Multi-frame image processing

The alignment of the sub-apertures is a major challenge for future segmented telescopes and telescope arrays. We show here that a focal plane wave-front sensor using only two images can fully and efficiently align a multiple aperture system, both for the alignment (large amplitude tip/tilt aberrations correction) and phasing (piston and small amplitude tip/tilt aberrations correction) modes. We derive a new algorithm for the alignment of the sub-apertures : ELASTICS. We quantify the novel algorithm performance by numerical simulations. We show that the residues are within the capture range of the fine algorithms. We also study the performance of LAPD, a recent real-time algorithm for the phasing of the sub-apertures. The closed-loop alignment of a 6 sub-aperture mirror provides experimental demonstration for both algorithms.

INTRODUCTION

In optics imaging for astronomy or Earth observation, the spatial resolution power of a telescope increases with its aperture diameter. The monolithic telescope mirror size is currently limited by technology to about 8 m for ground-based telescopes and to a few meters for the space-based telescopes because of volume and mass considerations. Interferometry is a solution to overcome those limitations. It consists in making several subapertures interfere. The resulting instrument is called an interferometer or a multi-aperture telescope. This concept was already used for several ground-based instruments (VLTI, Keck) and is adopted for most of the future telescope project in space (JWST, LUVOIR, OST, WFIRST) or on the ground (E-ELT, TMT, GMT).

The new challenge is to reach the diffraction limit of such an instrument by (co)phasing the sub-apertures to within a small fraction of the wavelength. This is achieved by correcting the specific low-order aberrations of the interferometer: the relative orientation of the sub-aperture (called the tip/tilt aberration) and the optical path difference between the sub-apertures (called the piston aberration).

Several steps are required to bring the instrument from a potentially very disturbed state where the images of each sub-aperture can be scattered in the focal plan to the optimal phased state (see Fig. 1). First, all subapertures must be co-aligned so that their images overlap in the focal plane. We call this first step the geometric alignment, and it aims at correcting the large tip/tilt errors. Then one can search for fringes and correct the large piston errors (phasing or "coherencing"). Finally, the fine phasing aims at the measurement and correction of the residual piston/tip/tilt errors. This last step usually requires a small amplitude of the aberrations for them to be accurately measured and corrected.

In this paper, we study and validate experimentally a complete alignment and co-phasing procedure of a multi-aperture telescope, starting from a strongly misaligned state in which the images through all sub-apertures span the whole field of view (typically after integration or launch) and ending with an instrument phased to a Further author information: (Send correspondence to S.V) S.V.: E-mail: vievard@naoj.org, Telephone: ), the large tip/tilt errors that scatter each sub-aperture PSF in the focal plane must be corrected so that the sub-aperture PSFs superimpose. The phasing step consists in make each sub-aperture PSF interfere with the others, by correcting the large piston errors. Finally, during the fine phasing the small remaining piston/tip/tilt errors are corrected. In this paper, we focus on the alignment and the fine phasing.

small fraction of the wavelength. Since the source on our testbed is monochromatic, we do not need to correct the large pistons to obtain interferences between sub-apertures here. The whole procedure requires little or no additional hardware as it can use images recorded with the scientific sensor, and it only uses algorithms with low computing costs. More precisely:

• Concerning the first step of the procedure, i.e., the geometric alignment of the sub-apertures, an efficient method called ELASTIC (Estimation of Large Amplitude Sub-aperture Tip-tilt from Image Correlation) was developed recently. [START_REF] Vievard | Large amplitude tip/tilt estimation by geometric diversity for multiple-aperture telescopes[END_REF] Similarly to earlier works, [START_REF] Thurman | Method of obtaining wavefront slope data from through-focus point spread function measurements[END_REF][START_REF] Jurling | Extended capture range for focus-diverse phase retrieval in segmented aperture systems using geometrical optics[END_REF][START_REF] Carlisle | Demonstration of extended capture range for james webb space telescope phase retrieval[END_REF] it only requires images from the scientific camera and is based on geometrical optics. Its advantages are that it only requires two images of a point-source, and that it is non-iterative and computationally inexpensive. ELASTIC worked in two steps and needed to put the sub-PSFs (a sub-PSF being the PSF of a sub-aperture) in a transitional position named "parking position" before superimposition.

In the following, we first give a preview of ELASTIC method improvement (presented in detail in a further paper) so that it is possible to bring the sub-PSFs all the way to superposition in closed loop and not only to the so-called "parking position". The new algorithm is named ELASTICS (Estimation of Large Amplitude Sub-Aperture Tip/tilt by Image Correlation and projection on orthogonal Subspace).

• The intermediate step (large piston correction) is not addressed here but will be in a further paper thanks to the SPRING (Sub-aperture Piston Reduction by Interferences in Non-redundant Groups) algorithm.

Concerning the last step of the procedure, i.e., the fine co-phasing, phase diversity [START_REF] Gonsalves | Phase retrieval and diversity in adaptive optics[END_REF][START_REF] Mugnier | Phase diversity: a technique for wave-front sensing and for diffraction-limited imaging[END_REF] is a very appealing and much studied cophasing sensor [START_REF] Paxman | Optical misalignment sensing and image reconstruction using phase diversity[END_REF][START_REF] Redding | Wavefront sensing and control for a Next Generation Space Telescope[END_REF][START_REF] Carrara | Aberration correction of segmented-aperture telescopes by using phase diversity[END_REF][START_REF] Lee | Point-by-point approach to phase-diverse phase retrieval[END_REF][START_REF] Cassaing | DARWIN Fringe Sensor (DWARF): Concept study[END_REF][START_REF] Mugnier | Multipleaperture optical telescopes: some key issues for Earth observation from a GEO orbit[END_REF][START_REF] Dean | Phase retrieval algorithm for JWST flight and testbed telescope[END_REF] (CS), for several reasons: firstly, this CS can use the scientific camera; this both simplifies the hardware and avoids differential aberrations between the scientific sensor and the CS. Secondly, it is appropriate for an instrument with a large number of sub-apertures, because the complexity of the hardware does not scale with the number of sub-apertures and remains essentially independent of it, contrarily to the case of common pupil-plane sensors. [START_REF] Cassaing | DARWIN Fringe Sensor (DWARF): Concept study[END_REF] Thirdly, it can be used on very extended objects, for which it is actually the only reasonable CS. [START_REF] Mugnier | Continuous high-resolution Earth observation with multiple aperture optical telescopes[END_REF] In the following, we study the performance of a fast non-iterative phase-diversity algorithm [START_REF] Mocoeur | Analytical solution to the phase-diversity problem for real-time wavefront sensing[END_REF] nicknamed LAPD (Linearized Analytic Phase Diversity).

• Finally, we perform the experimental alignment and phasing of a segmented mirror with six apertures on our BRISE testbed [START_REF] Cassaing | Brise: a multipurpose bench for cophasing sensors[END_REF] using only focal-plane images and the ELASTICS and LAPD algorithms. We start from a state in which the misalignment is tens of wavelengths λ and we experimentally obtain a λ/75 rms closed-loop residual at the end of the procedure. This experiment is also the first quantitative experimental validation of LAPD and its first application in closed loop.

LARGE AMPLITUDE ALIGNMENT

Motivations

The goal of the algorithm described in this first section is to estimate the tip-tilt coefficients a k,n , k ∈ (2; 3) for (tip;tilt), of each element n of an array of N a sub-apertures (fig. 2a) whose sub-PSFs are formed on a common focal camera (fig. 2b). Because they appear in the data model and must be well calibrated, we will also consider the estimation of the illumination ρ n of each sub-aperture n. Only N a = 6 sub-apertures are considered to simplify Fig. 2, but the presented algorithms can also be used in a much more crowded field with N a = 18, as confirmed by the simulations of section 2.4. The main issue is not to estimate these positions (a local center-ofgravity-like algorithm would be sufficient for this task, at least when they are sufficiently far apart) but to know to which sub-aperture each sub-PSF belongs. To solve for this ambiguity, the principle of the ELASTIC algorithm we introduced in Vievard et al. [START_REF] Vievard | Large amplitude tip/tilt estimation by geometric diversity for multiple-aperture telescopes[END_REF] is to use a pair of images, the second so-called diversity image (Fig. 2c) being recorded in presence of a known and unique differential shift simultaneously applied to each sub-PSF. This can be made with a classical defocus (detector shift, inserted lens) or with what we called a pseudo-defocus introduced by the segment actuators themselves. As explained in Vievard et al. [START_REF] Vievard | Large amplitude tip/tilt estimation by geometric diversity for multiple-aperture telescopes[END_REF] and detailed further in this paper, we study a slightly modified correlation between those two images (stored in two vectors s 0 and s 1 of measured data), in the Fourier domain for fast computation. The latest is named the Frequency Shifted Cross-Spectrum (FSCS) and is defined as the product of s 0 shifted (in one way) Fourier Transform by s 1 shifted (in the opposite way) conjugated Fourier Transform. Two FCSC vectors j m are computed, one for each direction m ∈ {2, 3}, to estimate the a m,n coefficients (see further, Eq (2)). The Fourier Transform (FT) of j m , j m , the Frequency-Shifted Cross-Correlation (FSCC) of s 0 and s 1 (Fig. 2d), is an equivalent representation in the image domain and will be used here to describe the principle of the algorithm. It is not necessary at this stage to detail these two operators, they will be explicitely defined in the next sub-section.

The most important point is that since each image is the sum of N a sub-PSFs, the FSCS j m is the sum of N 2 a terms:

• N a of them (the "diagonal" terms) correspond to the FSCS j m,n of the two (one in each diversity plane) sub-PSFs related to the same sub-aperture n: they will be called autopeaks and their sum for all n values noted j A m ; • The N a (N a -1) other terms correspond to all the FSCSs j m,n,n ′ between a focal-plane sub-PSF of index n and a diversity-plane sub-PSF of index n ′ = n: the sum for all n = n ′ of these interpeaks will be noted j I m .

All these N 2 a terms are well seen in the FSCC as spots (Fig. 2d), resulting from the FSCC of the N a sub-PSFs in s 0 by the N a sub-PSFs of s 1 ; the N a autospots (the autopeaks Fourier Transform) are near the origin, at a fixed position imposed by the diversity and which is proportional to their position in the pupil when the diversity is a defocus, whereas the interspots (the interpeaks Fourier Transform) can spread all across the field, at positions mostly given by the differential tip/tilts between the sub-apertures (and marginally by the diversity).

Under the assumption of large amplitude tip/tilt errors ELASTIC algorithm exploits the sole autopeaks of the FSCS. Indeed the introduced shift on the OTFs before computation leads to a constant term in the autopeaks phase 1 (detailed in Eq.3). This so called piston is then proportional to the tip/tilt error. The problem is then linear and can be solved easily by computing an inverse matrix. The aim of ELASTICS is to take into account the interpeaks in this problem. Because they are naturally not linear regarding the tip/tilt errors we will try to derive the FSCS so that the interpeaks become linear when their influence is significant meaning when they start overlapping the autopeaks. This phenomena occurs when two sub-PSFs are too close from each other therefore when the tip/tilt errors are small.

The direct model

The aim here is first to remind how to derive the FSCS from both diversity plane OTFs. We will then derive a simplified direct model for the FSCS, that will be inverted with the ELASTICS algorithm.

We first derive a closed-form model for the multiple aperture OTF. We assume for simplicity that the aperture is composed of N a circular sub-apertures with identical radius R, indexed with n ranging from 1 to N a . It was previously shown 1 that the sampled OTF s d of such an instrument, in each diversity plane of index d and computed from N p by N p images, can be written for each 2D index pixel detector q as:

s d (q; {a k,n }) = Na n=1 ρ 2 n Λ n,d n s q N p exp j 3 k=2 4πa kn Z k n s q 2N p . ( 1
)
where Λ n,d is the OTF of the single sub-aperture of index n in the diversity image of index d, Z k is the Zernike mode of index k, q is the 2D pixel index, ρ n is the flux amplitude on each sub-aperture, n s is the sub-aperture sampling factor and j 2 = -1.

The FSCS was defined as the multiplication of one diversity plane OTF by the shifted conjugated other diversity plane OTF. It was shown 1 that a 1 or 2 pixel shift are required for a full camera field coverage. Moreover, the tip/tilt estimation was shown to be more efficient when the pixel shift increases. Therefore, we choose here to shift both the diversity plane OTF in opposite ways hence a 1 pixel shift will correspond to a 2 pixel relative shift between the two diversity plane OTFs. Noted j m , with m = 2, 3 the shift direction, the FSCS is derived as:

j m = ∆ - m [s d ] × ∆ + m [s * d ] , (2)
where ∆ m is the operator that performs the δ m (m = 2, 3) shift, with a δ pixel(s) amplitude. It was shown 1 that shifting the tip or tilt Zernike polynomials (m = 2 or 3), leads to the same tip (or tilt) plus a piston (constant phase) whose value depends on the shift amplitude and the tip/tilt slope. We retrieve the fact that the FSCS is the sum of N a autopeaks and N a (N a -1) interpeaks:

j m (q; {a k,n }) = Na n=1 ρ 4 n exp -j8πn s δ N p a m,n Λ - m,n,1 Λ * + m,n,2 n s q N p + Na n=1 Na n ′ =n ρ 2 n ρ 2 n ′ exp -j4πn s δ N p (a m,n + a m,n ′ ) Λ - m,n,1 Λ * + m,n ′ ,2 n s q N p × exp j 3 k=2 4π (a k,n -a k,n ′ )Z k n s q 2N p , (3) with
Λ - m,n,1 = ∆ - m [Λ n,1 ] and Λ * + m,n,2 = ∆ + m Λ * n,2 .
Each term of the FSCS bilinearly scales with the amplitudes ρ n . As the intensity mismatches between the sub-apertures are expected to be small, we can introduce an average amplitude ρ and a local log-amplitude deviation χ n ∈ R on each sub-aperture so that ρ n = ρ exp χ n .

To put all the unknows (a k,n , ρ and χ n ) in a global vector γ m , we introduce:

γ m 0,n = ρ 4 exp 4χ n - 8jπn s δ N p a m,n , (4)
γ m 1,n = ρ 4 2χ n - 4jπn s δ N p a m,n , (5)
γ m k,n = ρ 4 4jπ a k,n , k ∈ {2, 3}. (6)
This definition makes γ m different from the one introduced in Vievard et al., [START_REF] Vievard | Large amplitude tip/tilt estimation by geometric diversity for multiple-aperture telescopes[END_REF] with more terms. Last, to the tip/tilt modes Z k,q in discrete form we add the piston mode (k = 1) :

Z k,q = Z k n s q 2N p with Z 1,q = 1 (7)
With these notations, the autopeaks (j A m ) and interpeaks (j I m ) in Eq. ( 3) strictly write:

j A m (q; {γ k,n }) = Na n=1 m,n,n,q γ m 0,n (8)
j I m (q; {γ k,n }) = ρ 4 Na n=1 Na n ′ =n m,n,n ′ ,q exp 3 k=1 ρ -4 (γ m k,n + ǫ k γ m k,n ′ )Z k,q , with m,n,n,q = Λ - m,n,1 Λ * + m,n,2 , ǫ 1 =1 and ǫ 2 = ǫ 3 = -1. (9)
We show in detail in a further paper that the N a (N a -1) non-linear interpeaks j I m of the FCSC can be reduced to 3N a linear pseudo-interpeaks by performing a Taylor Expansion on j I m with the assumptions of small errors in tip/tilt a k,n or flux mismatch χ n . Introducing the index c concatenating the indices n in [1, N a ] and k in [0, 3], and filling the C matrix with autopeaks (k = 0) and interpeaks (k = 1 to 3), the FSCS can then be derived as a simple matricial linear equation, where equality strictly holds for the autoterms:

j m ≃ c C m,c γ m c ⇒ j m ≃ C m .γ m . (10)
The dimension of C m matrix is [4N a ; N p ] and the dimension of γ m vector is [4N a ].

The ELASTICS solution for the inverse problem

We then can compute the generalized inverse C † m of C m by Singular Value Decomposition. Thanks to that computation we make sure that all autopeaks of C m are orthogonal to the other modes. Thus we can solve the inverse problem to obtain the solution γ m :

γ m ≃ C † m j m (11)
The seeked tip/tilts and pupil amplitudes are then simply computed as: .!

a m,n = -Np 4π nsδ Arg( γ m 0,n ), ρ m,n = | γ m 0,n | 1/4 . (12)
The tip/tilt coefficients could also be retrieved (when they are small) thanks to the γ m 1 , γ m 2 and γ m 3 vectors but we choose to keep the γ m 0 vector as the most relevant to be in the continuity of ELASTIC philosophy (extraction of the tip/tilt information on the autopeaks). It must be noted that the only real-time operations are the computation the two (tip and tilt) frequency shifted cross-spectra from the two diversity images since C † m can be pre-computed. Moreover, for a given sub-aperture tip/tilt, the sub-PSF shifts (and thus the OTF slopes) are the same for all wavelengths. The ELASTICS algorithm can thus operate with broadband illumination, as long as the sub-OTFs Λ n,d considered in the direct model are the polychromatic OTF obtained by averaging over the spectral band the monochromatic sub-OTFs weighted by the source amplitude and the detector efficiency.

Numerical simulations

The optimization and performance evaluation performed in this section by numerical simulations are based on a compact 18 sub-aperture pupil chosen to mimick the JWST pupil (Fig. 3 a), with circular sub-apertures for compatibility with our software.

Quasi-monochomatic images of size N 2 p = 1024 × 1024 pixels are simulated from an unresolved object. The focal-plane detector is assumed to be sampled at the Nyquist frequency in the fine phasing mode. Since each sub-aperture diameter is 1/5 of the full aperture, the sub-PSFs are oversampled and the sub-aperture sampling ratio, n s , is 10. An amplitude of A 4 = 0.9 wave 1 is chosen for the pseudo-defocus. It allows as we can see Fig. 3 d to make quasi orthogonal the auto-terms (or autopeaks) in the FSCC (or in the FSCS).

Images are simulated with a total of 3 × 10 5 photo-electrons, Poissonian photon noise and a 5 electrons per pixel read-out-noise. Algorithm performance is quantified with the estimation of the Root Mean Square Error (RMSE) of N o outcomes (with N o = 50) defined as:

RM SE = 1 N a m,n ( a m,n -a m,n ) 2 , ( 13
)
where a m,n and a m,n are respectively the estimated and introduced aberration coefficients in waves. Figures 3b/c show the simulated images. As we can see, the sub-PSFs are spanned in the field of the camera, traducing the large tip/tilt introduced in the pupil plane (here, corresponding to the "parking position"). The corresponding phase in the pupil plane in presented in Fig. 4a. We can see that strong phase slopes are applied on each and every sub-aperture. The estimation of the pupil phase error by ELASTICS is shown in Fig. 4b. We can see that the two phase maps are quite similar. Using the Eq. ( 13), we find that the error on the estimation is around λ/20 RMS, which is far below the fine phasing algorithm upper working limit (as a reminder, about λ/8 RMS). Simulations then comfort us in ELASTICS ability to estimate large tip/tilt errors as a first stage of the alignment procedure, before the fine phasing step.

THE FINE COPHASING

Linearized phase diversity

Phase Diversity (PD) is a powerful focal-plane wavefront analyzer, that is well suited for high precision measurement of multi-aperture imagers individual piston and tilt-tilts, and is currently routinely used for such purposes. However, since the link between the aberrations and the focal and extrafocal images that are analyzed is highly non-linear, PD usually requires time-consuming, high computational cost iterative minimization algorithms.

In our case though, after coherencing the subapertures with the ELASTICS algorithm, remaining residual aberrations are small -typically lower than a wavelength (around λ/20 RMS according to Section 2.4). In this case, Mocoeur [START_REF] Mocoeur | Analytical solution to the phase-diversity problem for real-time wavefront sensing[END_REF] proved through simulations that usual PD equations could be linearized, and the algorithm thus made much simpler, and incomparably quicker, making even real-time operation possible : less than 0.3 second, with IDL and an average laptop, were indeed needed in order to compute pistons in a 19 aperture configuration, for instance. [START_REF] Vievard | Real-time alignment and co-phasing of multi-aperture systems using phase diversity[END_REF] We therefore computed Mocoeur's algorithm, but with a small change : we used it with a small number of iterations (typically less than ten, depending on the aberration value), in order to increase its range and accuracy. The algorithm, called LAPD for Linearized Analytical Phase Diversity, as well as numerical results, are described in the sections below.

The LAPD Algorithm

Our algorithm is based on Mocoeur's [START_REF] Mocoeur | Analytical solution to the phase-diversity problem for real-time wavefront sensing[END_REF] , that supposes the perturbations are small enough to allow a 1 rst order Taylor expansion of the PSF h at the last known point versus the residual aberration vector a :

h(a) = h(a ′ ) + a. grad(h) + o(a) (14)
where a = (a 0,0 , a 0,1 , ..., a k,n , ...) is the vector of the Zernike coefficients of the residual perturbations we want to measure, a ′ the last known aberration vector, and grad(h) = ∂h ∂a 0,0 , ∂h ∂a 0,1 ..., ∂h ∂a k,n , ... the gradient of h in a = a ′ . Doing so, Mocoeur showed that the PD criterion of two images ĩ1 and ĩ2 whose PSFs are h 1 ans h 2 could be simply written:

J(a) = 1 2σ 2 ν |A(ν)a -B(ν)| 2 + Cste (15)
where

A(ν) = 1 C(ν) [ ĩ2 (ν)α 1 (ν) -ĩ1 (ν)α 2 (ν)] (16)
B(ν) = 1 C(ν) [ ĩ1 β 2 (ν) -ĩ2 (ν)β 1 (ν)] (17)
C(ν) = |β 1 (ν)| 2 + |β 1 (ν)| 2 + ǫ (18)
and, for i ∈ {1, 2}

α i = T F [grad(h i (a = a ′ ))] (19)
β i = T F [h i (a = a ′ )] (20)
A, B and C are matrices and can be easily computed from the image formation modeling, and ǫ, whose origin is detailed in, [START_REF] Mocoeur | Analytical solution to the phase-diversity problem for real-time wavefront sensing[END_REF] is set to 10 -6 allowing the criteria not to diverge for high frequencies.

Then a can be estimated analytically thanks to a typical maximum-likelihood formula:

â = ℜ(A H A) † . ℜ(A H B) (21)
Mocoeur's algorithm is not supposed to be used iteratively, but we found that range and accuracy could be notably increased by performing a few iterations in closed loop, depending on the amount of remaining residual aberrations to be measured : after each iteration, subapertures are moved in order to compensate the amount of pistons and tip-tilts estimated by the algorithm, â. Then, new images i 1 and i 2 are recorded -and the second iteration can begin. Note that only the matrix A and B change, while all the other parameters remain the same, so the iterations can be made in real-time.

LAPD : numerical validation

Numerical simulations are performed with the same setup than in Section 2.4. However, we use here 512 × 512 pixel images for our simulations. The focal plane detector is still assumed to be at the Nyquist frequency in the fine phasing mode. Oversampling is 10 for a single sub-aperture, and the amplitude of the defocus is set to 0.3λ. This defocus value was proved [START_REF] Meynadier | Noise propagation in wave-front sensing with phase diversity[END_REF][START_REF] Lee | Cramer-rao analysis of phase-diverse wave-front sensing[END_REF] to be in the range of the optimal defocus values for phase diversity algorithms (in the case of a monolithic telescope). Figures 5a/b show the simulated images. We can see in both images the PSF degradation due to small piston/tip/tilt errors. We show in Fig. 5c the associated pupil plane phase error with a λ/10 RMS distribution to match the possible residual error of large amplitude correction. Using LAPD to estimate those errors leads to the pupil phase error estimation Fig. 5d. The estimation is qualitatively very close to the actual introduced error. Using the Eq.13, we quantify that the error on the estimation is around λ/300 RMS. Simulations then comfort us in LAPD ability to estimate small piston/tip/tilt errors in a multi-aperture imager with a high precision.

EXPERIMENTAL VALIDATION: LOOP CLOSURE ON A MONOCHROMATIC UNRESOLVED OBJECT

Implementation of the algorithms

In order to test our algorithms, we use a dedicated bench built by Onera. [START_REF] Vievard | Large amplitude tip/tilt estimation by geometric diversity for multiple-aperture telescopes[END_REF] We conducted the tests imaging the collimated output of a fibered single-mode fiber with a 635 nm wavelength, acting like an unresolved source to six out of the nineteen apertures of the segmented mirror. Hardware issues forced us to use the six sub-apertures of the first corona. Each segment is supported by three piezoelectric actuators, allowing to introduce piston, tip or tilt perturbations. Although these actuators have not internal feedback and suffer from hysteresis. Downstream, a phase diversity module can be used to simultaneously form a focused and a defocused image of the object on a 1300 × 1000 pixels camera, from which we will extract 512 × 512 images.

We tested LAPD using this configuration, with a 0.3λ defocus between the two images. The implementation of ELASTICS only requires one channel since the diversity is temporal. Hence the first diversity image is acquired on the focal plane part of the detector, then the pseudo-defocus is applied on the segments in order to acquire the second diversity image on the same part of the detector. The chosen amplitude for the pseudo-defocus, as discussed Section 2.4, is 0.9λ.

A closed-loop sequence is performed to correct random perturbations since it is not possible to perform a single-step correction, due to not perfectly deterministic active mounts. No large piston correction is needed here since our source is a laser hence has a very large coherence length.

Loop closure on an unresolved object

The diversity images of the unresolved source during ELASTICS and LAPD loop closure are presented on Fig. 6 first row. At ELASTICS first iteration the sub-PSFs are scattered in the field, evidencing the random tip/tilt errors over each sub-aperture. We used a 0.5 control gain for the integrator to perform the ELASTICS loop closure. We can see that the sub-PSFs migrate in the center of the focal plane image as the iterations go on. Last iteration shows that they are all superimposed. The size of the sub-PSFs pack is comparable to the size of one sub-PSF, leading us to affirm that the tip/tilt error should be smaller than λ/8 since the size of one sub-PSF is λ/4. This last iteration is taken as departure point for the LAPD closed loop control. We change the acquisition mode (spatial small defocus) and we can see in the focal and diversity planes that the PSF is still affected by small piston/tip/tilt errors. We use a 0.2 control gain for the integrator to perform the loop closure. We can see over the iterations that LAPD manages to estimate the small errors to obtain at last iteration the phased instrument PSF.

Last row of Fig. 6 shows the evolution of ELASTICS and LAPD estimations during the loop closures. For ELASTICS, at first iteration the estimation is larger than 2λ, traducing the large tip/tilt error over the subapertures. As the loop closure operates, the estimation decreases, to a stable state as from iteration 10, where the estimation is lower than λ/10. At this state, the phasing mode can theoretically begin. We can then follow the evolution of LAPD estimation. The estimated errors are around 0.1λ at the beginning of the loop closure, and are brought to less than 0.02λ, with a λ/70 RMS dispersion.

CONCLUSION

We presented in this paper a complete alignment and co-phasing procedure of a multi-aperture telescope starting from a strongly misaligned state and ending with an instrument phased to a small fraction of wavelength. The presented algorithms have the advantages to be unsupervised, don't need additional hardware and only need two image acquisition.

We have first improved the ELASTIC algorithm which provides sub-apertures large tip/tilt estimation under a sub-PSF non-overlap hypothesis. The new algorithm ELASTICS provides a large amplitude tip/tilt estimation even if sub-PSFs are overlapping. This allows to superimpose all the sub-PSFs during a closed loop control and can be used while searching for large piston errors (SPRING algorithm). Simulations showed that ELASTICS was able to estimate the pupil plane phase map with a λ/20 RMS precision, which is theoretically enough to enter the fine phasing mode of the alignment procedure. We then presented LAPD, a linearized phase diversity algorithm for the fine phasing mode. We showed thanks to simulations that with a λ/10 RMS piston/tip/tilt over a 18 sub-aperture instrument the error of LAPD estimation was around λ/300 RMS.

Finally, we validated the two algorithms on a 6 sub-aperture imager. We showed that from a large amplitude error state ELASTICS was able to superimpose the sub-PSFs in the focal plane. Then this final ELASTICS state was taken as starting point for the fine phasing with LAPD. The λ/20 RMS residual errors after ELASTICS were in LAPD capture range, since the loop closure allowed to phase the instrument with a λ/70 RMS RMSE.

Figure 1 .

 1 Figure 1. The alignment and (co)phasing procedure is divided in 3 steps. During the geometric alignment (simply called alignment in the table), the large tip/tilt errors that scatter each sub-aperture PSF in the focal plane must be corrected so that the sub-aperture PSFs superimpose. The phasing step consists in make each sub-aperture PSF interfere with the others, by correcting the large piston errors. Finally, during the fine phasing the small remaining piston/tip/tilt errors are corrected. In this paper, we focus on the alignment and the fine phasing.

  a) Telescope pupil b) Focal image c) Extra-focal image d) FSCC image

Figure 2 .

 2 Figure 2. Ingredients of the algorithm: (a) the telescope pupil, made of Na = 6 sub-apertures; (b) the focal image, made of Na sub-PSFs; (c) the diversity image, with deterministic shifts (materialized by arrows) with respect to the focal image; (d) the intercorrelation image, made of N 2 a spots including Na autospots at fixed positions inside the disk.

  ,,.. :!: t

  • is the average over the N o outcomes. a) Telescope pupil b) Focal image c) Extra-focal image d) FSCC image

Figure 3 .

 3 Figure 3. Numerical simulations: (a) the telescope pupil, made of 18 sub-apertures; (b) the focal image, made of 18 sub-PSFs; (c) the diversity image, with deterministic shifts with respect to the focal image; (d) the intercorrelation image, made of 18 2 spots including 18 autospots at fixed positions inside the disk.

Figure 4 .

 4 Figure 4. Tip/Tilt errors are introduced in the phase of the pupil plane (a). Estimation of the pupil plane phase error (b) is then provided by ELASTICS.

  a) Focal image b) Extra-focal image c) Pupil phase error d) Pupil phase estimation

Figure 5 .

 5 Figure 5. Numerical simulations: (a) the focal image; (b) the diversity image; (c) Phase error in the pupil plane; (d) LAPD phase estimation in the pupil plane.

Figure 6 .

 6 Figure 6. Experimental validation of ELASTICS and LAPD; Top and second rows: Focal and diversity planes during the loop closures ; Last row: Evolution of the estimation during the loop closures.

Acknowledgements

The authors would like to thank N. Treps for the direction of S. Vievard PhD; B. Denolle, A. Grabowski and C. Perrot for their early work on the algorithm and the experiments and J.-P. Amans from GEPI (Galaxies, Etoiles, Physique et Instrumentation) of Observatoire de Paris-Meudon, for the design and manufacturing of the segmented mirror.

Fundings

This research was partly funded by ONERA's internal research project VASCO ; Thales Alenia Space co-funded S. Vievard's PhD thesis; the BRISE bench was funded by French DGA