N Cam

T Georgelin

M Jaber

J-F Lambert

Karim Benzerara
email: karim.benzerara@impmc.upmc.fr

In vitro synthesis of amorphous Mg-, Ca-, Sr-and Ba-carbonates: what do we learn about intracellular calcification by cyanobacteria?

Some cyanobacteria, including Candidatus Gloeomargarita lithophora, which was isolated from Lake Alchichica (Mexico), can form intracellular carbonates. This contradicts the common paradigm that cyanobacterial calcification is always extracellular and suggests that calcification might be controlled by these cyanobacterial species. Intracellular carbonates have several peculiar characteristics: they are relatively small (between 60 and 500 nm), they are poorly crystalline, and they have Sr/Ca and Ba/Ca ratios much higher than the solution in which the cells grow. It is therefore crucial to understand whether these unique features may indicate the involvement of specific biological processes. Here, in vitro abiotic syntheses were performed to synthesize Mg-, Ca-, Sr-and Ba-containing carbonates with compositions, crystallinities and sizes close to those observed in intracellularly calcifying cyanobacteria.

Precipitates were characterized by scanning and transmission electron microscopies coupled with energy dispersive x-ray spectroscopy, thermogravimetric analysis and x-ray diffraction.

The size and the poor crystallinity of cyanobacterial intracellular carbonates could be mimicked under these abiotic conditions. It was shown that similarly to Mg, elements such as Sr and Ba can favour stabilization of poorly crystalline carbonates. In contrast, the differential partitioning of Sr, Ba and Ca between the solution and the solids as observed in cyanobacteria could not be mimicked in vitro. This provides keys to a better understanding of biological processes involved in the formation of intracellular carbonates by some cyanobacteria, including the involvement of membrane transporters.

INTRODUCTION

Cyanobacteria are photosynthetic prokaryotes that occupy a wide array of terrestrial, marine, and freshwater habitats [START_REF] Badger | The environmental plasticity and ecological genomics of the cyanobacterial CO 2 concentrating mechanism[END_REF]. They contribute significantly to the global carbon cycle by assimilating CO 2 to organic carbon [START_REF] Fuhrman | Genome sequences from the sea[END_REF] and by triggering CaCO 3 precipitation (e.g., [START_REF] Verrecchia | Spherulites in Calcrete Laminar Crusts -Biogenic CaCO 3 Precipitation[END_REF][START_REF] Kamennaya | Cyanobacteria as Biocatalysts for Carbonate Mineralization[END_REF].

The general reaction of CaCO 3 precipitation can be written as:

Ca 2+ aq + CO 3 2- aq = CaCO 3, solid (1)
This is coupled with the acid-base equilibrium of carbonates, e.g. Cyanobacteria impact that reaction by fixing CO 2 through photosynthesis, hence displacing equilibrium (3) towards the formation of CaCO 3 . More precisely, cyanobacteria usually import carbon under the form of HCO 3 -especially in environments where dissolved CO 2 is limiting. HCO 3 -is then converted to CO 2 within cell compartments called carboxysomes by specific enzymes, i.e. carbonic anhydrases [START_REF] Kupriyanova | Carbonic anhydrase: Enzyme that has transformed the biosphere[END_REF]. CO 2 is then fixed by the RuBisCO enzyme and transformed into organic carbon while CO 3 2-and/or OH - produced by the carbonic anhydrase reaction are exported to the extracellular medium (e.g., [START_REF] Kamennaya | Cyanobacteria as Biocatalysts for Carbonate Mineralization[END_REF]). This raises the supersaturation of the surrounding solution with respect to various carbonate minerals. An additional source of oversaturation is provided by an active export of Ca 2+ from the cells coupled with an import of H + [START_REF] Thompson | Whiting events: biogenic origin due to the photosynthetic activity of cyanobacterial picoplankton[END_REF].

HCO

Until recently, calcification by cyanobacteria has been invariably considered as an extracellular process [START_REF] Merz | The biology of carbonate precipitation by cyanobacteria[END_REF]. As a result, the CaCO 3 precipitation process and the chemical composition of the mineral by-products were supposed to be influenced only by the chemical conditions prevailing in the aqueous environment at the cell/solution interface.

However, several species of cyanobacteria have been discovered recently that have the ability to form intracellular carbonate inclusions [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF][START_REF] Benzerara | Intracellular Ca-carbonate biomineralization is widespread in cyanobacteria[END_REF].

Despite the potential importance of such a process [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF][START_REF] Riding | A Hard Life for Cyanobacteria[END_REF][START_REF] Benzerara | Intracellular Ca-carbonate biomineralization is widespread in cyanobacteria[END_REF], the mechanisms involved in the formation of intracellular carbonates by cyanobacteria are as yet unknown. These intracellular carbonates are quite different from usual abiotic carbonates: (1) they appear as nanospheres with a diameter between 60 and 500 nm;

(2) they are poorly crystalline as inferred by electron diffraction; (3) in some species, e.g.,

Candidatus Gloeomargarita lithophora, they contain high concentrations of strontium and barium, much higher than in the extracellular solution. For example, the Sr/Ca and Ba/Ca ratios in the intracellular carbonates of Candidatus G. lithophora were respectively 86 and 1370 times higher than in the extracellular solution [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF]. Interestingly,

similarly to what has been observed in G. lithophora, an enrichment in Sr and Ba relative to Ca was observed recently in carbonates from the water column of Lake Geneva (Switzerland) by [START_REF] Jaquet | Preliminary investigations on picoplanktonrelated precipitation of alkaline-earth metal carbonates in meso-oligotrophic lake Geneva (Switzerland)[END_REF].

The origin of such a high enrichment in Sr and Ba relative to Ca in intracellular precipitates, as well as the mechanism by which such poorly crystalline carbonates are stabilized in cyanobacteria have yet to be determined. In particular, it is crucial to ascertain whether they are indicative of a biological control or the result of non-controlled processes occurring under specific chemical conditions within the cytoplasm of the cells. In turn, relating the chemical and crystallographic features of such inclusions with the chemical environment in which they formed may provide a new approach to constrain the chemical composition of the cytoplasm in microorganisms. In vitro experiments have been classically performed for more than 50 years to better understand biomineralization processes [START_REF] Kitano | The influence of organic material on the polymorphic crystallization of calcium carbonate[END_REF]. Such modelling experiments do not intend to mimic the processes occurring within the cells, which remain unknown, and generally do not provide an ultimate and direct answer as to which biological processes are involved in biomineralization. Yet, these experiments constrain to some extent the chemical conditions necessary to produce mineral phases in vitro similar to those observed in organisms.

In the present study, we have conducted abiotic precipitation experiments under various conditions and we have systematically analyzed the solid products by x-ray diffraction (XRD), scanning and transmission electron microscopies (SEM and TEM) and thermogravimetric analyses (TGA) with the objective to mimic some of the chemical and crystallographic properties observed in cyanobacterial intracellular carbonates. From the comparison between the in vitro abiotic precipitates and cyanobacterial inclusions, we discuss the implications for intracellular carbonate biomineralization. white precipitate appeared instantaneously. The suspension was then centrifuged for 10 minutes at 8200 g and the supernatant was discarded. The pellet was spread on a Petri dish and dried at room temperature.

MATERIAL AND METHODS

Saturation indices calculation

Saturation indices were calculated using the Visual Minteq 3.0 software. They correspond to the logarithm of the ion activity product divided by the solubility product of the mineral phase of interest. The solubility product of amorphous calcium carbonate used in the present study was obtained by [START_REF] Kellermeier | A Straightforward Treatment of Activity in Aqueous CaCO 3 Solutions and the Consequences for Nucleation Theory[END_REF].

Scanning electron microscopy coupled with energy dispersive x-ray spectrometry

Scanning electron microscopy (SEM) images were obtained on gold-coated samples using a

Hitachi SU-70 SEM-FEG equipped with a field emission gun. Images were acquired with the microscope operating at 10 kV, and a working distance of ~15 mm in the secondary electron mode using a SE (Lower) detector. Energy dispersive x-ray spectrometry (EDXS) analyses were performed using a Zeiss Ultra 55 SEM-FEG on carbon coated samples at 10 kV and a working distance of ~7.5 mm. The spot size was about 50 nm with a total acquisition time of 5 seconds. Around ten acquisition areas were used for each sample to determine the average composition. Semi-quantitative analysis of EDXS data was performed with the INCA software using a XPP matrix correction and reference spectra acquired on standard samples.

The Kα lines of Mg and Ca and Lα lines of Sr and Ba were used for quantification.

Transmission electron microscopy

Transmission electron microscopy (TEM) analyses were performed in the bright field mode using a JEOL 2100F operating at 200 kV and equipped with a field emission gun, a highresolution UHR pole piece, and a Gatan energy filter GIF 200. Samples were prepared as fine powders by grinding in absolute ethanol in a mortar. Three microliters of this suspension were deposited on a copper grid covered with a carbon membrane.

X-ray diffraction

Bulk samples were analyzed by x-ray diffraction (XRD). A few milligrams of the samples were ground in an agate mortar in pure acetone. Powders were deposited on monocrystalline silicon sample holders. XRD measurements were performed using the Cu Kα radiation on a Panalytical X'Pert Pro MPD diffractometer. The following settings were used: the Soller slit was open at 0.04 rad; a beam mask of 20 mm, a divergence slit of 1/4 ° and an anti-scatter slit of 1/2 ° were inserted. Data were recorded at 40 kV and 40 mA in the 2θ continuous-scan mode over a 4-120° 2θ range with a step of 0.017° and a total counting time of ~5 h. XRD data were analyzed using the Panalytical X'Pert Highscore software for background subtraction.

Thermogravimetric analysis-Differential scanning calorimetry

Thermogravimetric analysis-Differential scanning calorimetry (TGA-DSC) analyses were performed using a SDT Q 600 TA. Around 10 mg of powder were placed in an aluminum crucible and the sample was heated up to 900 °C under a 100 mL/min air flow. The temperature scanning rate was set at 5 °C per minute.

Mass spectrometry

The mass spectrometry analyses were coupled with the TGA-DSC analyses on a

ThermoStar TM GSD 301 T3 (Pfeiffer Vacuum). The signal corresponding to 44 amu (CO 2 + ) was recorded.

RESULTS

In a first experiment, called "stoichiometric conditions", MgCl 2 , CaCl 2 , SrCl 2 , BaCl 2 and Na 2 CO 3 were mixed in a solution with relative proportions similar to those measured by SEM-EDXS by [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF] in G. lithophora intracellular inclusions (i.e., Mg = 18%, Ca = 53%, Sr = 8%, Ba = 21%, in atomic ratios relative to the total cation content).

Moreover, concentrations were set so that [

Na 2 CO 3 ] = [CaCl 2 ] + [BaCl 2 ] + [SrCl 2 ] + [MgCl 2 ]
= 100 mM (Table 1).

A second experiment was performed also under "stoichiometric conditions" at lower concentrations, i.e. with [

CO 3 ] = [Ca] + [Ba] + [Sr] + [Mg] = 10 mM.
Finally, the relative proportions of Ca, Mg, Sr and Ba added to the solution were varied to test the impact of such compositional variations of the solution on the crystallinity and chemical composition of resulting precipitates.

Precipitation of G. lithophora-like carbonates under stoichiometric conditions at high carbonate concentration (100 mM)

The precipitate formed in the presence of a total carbonate ion concentration of 100 mM (provided as Na 2 CO 3 ) and dried at 20 °C without prior rinsing, was composed of particles measuring 48 ± 19 nm in diameter (measured on 146 particles). The precipitates contained Ca, Mg, Sr, Ba in the same relative proportion as in the initial solution (Fig. 1). The diffractogram revealed the presence of two phases: (1) a poorly crystalline phase with broad peaks centered at 30°, 45° and 65° (2θ) attributed to a (Ca-, Mg-, Sr-, Ba)-carbonate phase consistently with TEM observations and former XRD analyses of amorphous calcium carbonates (ACC), and (2) a crystalline phase with narrow peaks centered at 27. 4°, 31.7°, 45.5°, 56.5°, 66.3° and 75.3° (2θ), assigned to NaCl. When collected one month, instead of few minutes, after the mixing of the solutions, the precipitate showed a similar diffractogram, suggesting that the poorly crystalline (Ca, Mg, Sr, Ba)-carbonate phase was relatively stable over time. When the synthesis was performed at a lower initial pH of 8.77, with carbonates provided as NaHCO 3 , precipitation occurred slower and the precipitate was a crystalline (Ca, Sr, Ba)-carbonate containing low amounts of Mg (Fig. 2).

TGA-DSC analyses of the poorly crystalline precipitates formed in Na 2 CO 3 -containing solutions showed four weight loss events (Fig. 1). The first event with a maximum at 77 °C was attributed to the loss of adsorbed and/or structural water. The peak extended up to about 300 °C, indicating that part of this water was strongly retained. The total water content amounted to 20% of the weight of the solid remaining at 300 °C. Overall, this corresponded to a water content of 0.86 water molecule per carbonate. Therefore, by coupling the TGA and EDXS data (Fig. 1), the stoichiometric formula of the precipitate could be written as Mg 0.16 Ca 0.51 Sr 0.08 Ba 0.25 CO 3 •0.86H 2 O.

The second loss event showed a sharp maximum at 345 °C. It was associated with CO 2 release as followed by mass spectrometry and a strongly exothermic event as shown by DSC.

The total weight loss during this event amounted to 3.7 to 4.0% of the dry weight. Therefore, this event could be interpreted as a partial decomposition (11-12%) of the carbonates.

A third weight loss was observed between 500 and 700 °C. It also closely matched CO 2 release but it was associated with a clear endothermic effect on the heat flow curve. This loss amounted to 16.5 to 17.7% of the dry weight, i.e., 47 to 51% of total carbonate.

Overall, the amount of carbonate decomposed in the second event (T max = 345 °C) roughly corresponded to the Mg content of the solid phase, while the amount decomposed in the third event (T max = 637 °C) closely corresponded to its Ca content, suggesting a multi-step carbonate decomposition (MgCO 3 first; CaCO 3 second; SrCO 3 and BaCO 3 above 710 °C).

XRD analyses were performed on the solid phase before and after the second weight loss event, at 325 °C and 375 °C, respectively (Fig. 1). An increased amount of calcite was detected at 375 °C, suggesting that the initially amorphous mixed carbonate crystallized at 345 °C, causing the segregation of MgCO 3 which decomposed at this temperature. The start of a fourth weight loss above 710 °C and not completed at 800 °C, might be consistent with the decomposition of a residual (Sr,Ba)-carbonate phase.

Precipitation of G. lithophora-like carbonates under stoichiometric conditions at low carbonate concentration (10 mM)

Syntheses were carried out using lower total concentrations of carbonate anions and divalent cations (10 mM instead of 100 mM). The relative proportions of Mg, Ca, Sr and Ba were kept similar to those measured in G. lithophora intracellular inclusions (Table 1). The resulting precipitate was not rinsed and was dried at 20 °C. The precipitate showed a broad XRD diffraction peak around 30° (2θ), consistent with a poorly crystalline (Ca, Sr, Mg, Ba)carbonate phase (Fig. 3). No NaCl was detected, likely due to the lower reactant concentrations. TEM observations showed that it was composed of spheres with an average diameter of 126 ± 32 nm (measurements on 385 particles; Fig. 3). Based on TGA and EDXS analyses, the average chemical composition of the precipitate was determined to be Mg 0.04 Ca 0.61 Sr 0.06 Ba 0.29 CO 3 •0.84H 2 O. The precipitate contained much lower Mg than the solid formed at high carbonate and divalent cation concentrations. In contrast, the water content and the relative Sr/Ca, and Ba/Ca ratios were similar.

In TGA, water loss occurred in two steps at low temperatures, and the main weight loss event occurred at 670 °C. An exothermic event, not associated with weight loss was recorded at 354

°C by TGA-DSC (Fig. 3) and was attributed to crystallization of the carbonate phase without decarbonatation. This contrasted with what was observed for the precipitate formed at high carbonate and divalent cation concentrations, for which crystallization was associated with the decomposition of MgCO 3 .

Rinsing with milliQ water induced the transformation of the poorly crystalline phase into a substituted calcite phase as shown by XRD (Fig. 3). Moreover, when collected one month instead of few minutes after mixing the solution, the precipitate consisted mostly in crystalline calcite with a small amount of monohydrocalcite as shown by XRD (Fig. 3).

Precipitation of carbonates at low carbonate concentration and varying Sr/Ca/Mg and Ba relative proportions

To test whether a poorly crystalline Mg-rich carbonate phase could be formed at low carbonate concentration ([CO 3

2-] = 10 mM), an additional synthesis was performed in a solution containing a higher Mg 2+ concentration, with the following molar ratios: Mg: 55%, Ca: 29%, Sr: 4% Ba: 12% (Table 1). The precipitate appeared by TEM as spheres larger than in the previous syntheses, with an average diameter of 192 ± 44 nm as measured on 178 particles (Fig. 4). The elemental composition was Mg 0.16 Ca 0.51 Sr 0.05 Ba 0.28 CO 3 , i.e. close to the precipitate formed in the experiment at high carbonate and divalent cation concentrations (100 mM). The XRD pattern indicated that the precipitate was poorly crystalline. TGA analyses showed a relatively high water amount of 1.03 molecules per carbonate with two distinct water loss events at 80 and 220 °C. DSC indicated an exothermic crystallization event at 370 °C, which was immediately followed by a small CO 2 loss (corresponding to the decomposition of 0.02 carbonate per formula unit). The main decarbonatation event, corresponding to the decomposition of 0.61 carbonate per formula unit, showed a maximum at 670 °C.

Additionally, syntheses were conducted without adding (1) Mg, (2) Sr, (3) Ba or (4) Sr and Ba in the solution in order to assess the influence of these elements on the size and the crystallinity of the precipitates.

In the absence of Mg 2+ in the solution, particles were much larger (4-5 µm) than in previous syntheses (Fig. 5). Their elemental composition was close to that of the solution, with no preferential incorporation of Ca, Sr or Ba in the solid (Table 1). The precipitate consisted in well-crystallized calcite substituted by Sr and Ba (Fig. 5), with the following stoichiometry:

Ca 0.59 Sr 0.07 Ba 0.34 CO 3 . TGA analyses showed the presence of only minor amounts of physisorbed water (1.4% in weight) and decarbonatation occurred between 500 and 700 °C (corresponding to the decomposition of 0.60 carbonates per formula unit).

When no Sr or Ba was added to the solution, precipitates appeared as spheres with a diameter of 181 ± 46 nm and 268 ± 61 nm, respectively (Fig. 67). These spheres were poorly crystalline as shown by XRD. Similarly to all other experiments, no differential partitioning of Ca, Ba and Sr was evidenced between the solution and the precipitate. The Sr-free sample contained 0.93 water molecules per carbonate (Fig. 6) and showed a crystallization peak at 379 °C. The Ba-free sample contained 0.88 water molecules per carbonate and no crystallization was detected on the heat flow curve (Fig. 7). In both cases, the precipitates crystallized when left in solution for one month. The synthesis with Sr resulted in the formation of a monohydrocalcite phase (Fig. 6) and the synthesis with Ba resulted in a mix of monohydrocalcite and aragonite phases (Fig. 7). Finally, monohydrocalcite formed when neither Sr nor Ba was added to the initial solution (Fig. 8). The precipitate appeared as spherulites measuring between 4 and 5 µm in diameter. Monohydrocalcite contained around 25 at% of Mg.

DISCUSSION

Over the last twenty years, amorphous calcium carbonates (ACC) have been increasingly

detected in diverse samples, including biomineralizing systems (e.g., [START_REF] Gebauer | Stable Prenucleation Calcium Carbonate Clusters[END_REF][START_REF] Weiner | Crystallization Pathways in Biomineralization[END_REF][START_REF] Cartwright | Calcium Carbonate Polyamorphism and Its Role in Biomineralization: How Many Amorphous Calcium Carbonates Are There?[END_REF]. It has been shown that ACC are relatively unstable with an enthalpy of crystallization of -17.1 ± 0.7 kJ/mol [START_REF] Radha | Energetic and structural studies of amorphous Ca 1-x Mg x CO 3 •nH 2 O (0≤x≤1)[END_REF], and a high solubility product, i.e 2.32 ± 0.41 x10 -8 M 2 at pH 9 [START_REF] Kellermeier | A Straightforward Treatment of Activity in Aqueous CaCO 3 Solutions and the Consequences for Nucleation Theory[END_REF].

Magnesium, some phosphorylated amino acids [START_REF] Bentov | Stabilization of amorphous calcium carbonate by phosphate rich organic matrix proteins and by single phosphoamino acids[END_REF], serine-and glycine-rich proteins [START_REF] Aizenberg | Stabilization of amorphous calcium carbonate by specialized macromolecules in biological and synthetic precipitates[END_REF] as well as confinement [START_REF] Stephens | Amorphous Calcium Carbonate is Stabilized in Confinement[END_REF] have been shown to stabilize ACC. Therefore, these different parameters, i.e., high Mg contents, presence of biomolecules or confinement within intracellular vesicles, may play a role in the stabilization of amorphous carbonate inclusions within intracellularly calcifying cyanobacteria. Alternatively, the high Sr and Ba contents might be additional drivers but no

previous study has investigated that potential effect. In the following sections, we compare the results obtained on synthesized amorphous Ca-, Mg-, Sr-and Ba-containing carbonates with previous studies which focused mostly on ACC and amorphous magnesium carbonates (AMC). We then discuss the implications on our understanding of the mechanisms involved in intracellular carbonate formation by cyanobacteria.

Formation of amorphous Ca-, Mg-, Sr-and Ba-carbonates

In the present study, we reported the formation of an amorphous Ca-, Sr-, Ba-and Mgcontaining carbonate. This phase contained around one molecule of water per carbonate similarly to what has been often observed in abiotic [START_REF] Brečević | Solubility of amorphous calcium carbonate[END_REF][START_REF] Michel | Structural Characteristics of Synthetic Amorphous Calcium Carbonate[END_REF][START_REF] Radha | Transformation and crystallization energetics of synthetic and biogenic amorphous calcium carbonate[END_REF][START_REF] Radha | Energetic and structural studies of amorphous Ca 1-x Mg x CO 3 •nH 2 O (0≤x≤1)[END_REF] and biogenic [START_REF] Levi-Kalisman | X-Ray absorption spectroscopy studies on the structure of a biogenic "amorphous" calcium carbonate phase[END_REF] ACC. This water content corresponds to that of « type 1 » ACC, observed in the early stage of sea urchin spicule biomineralization [START_REF] Politi | Transformation mechanism of amorphous calcium carbonate into calcite in the sea urchin larval spicule[END_REF].

The formation and stabilization of ACC [START_REF] Gebauer | Proto-Calcite and Proto-Vaterite in Amorphous Calcium Carbonates[END_REF][START_REF] Fernandez-Martinez | Pressure-Induced Polyamorphism and Formation of "Aragonitic" Amorphous Calcium Carbonate[END_REF]Ihli et al., 2013a), AMC [START_REF] Jiang | Gram-scale, low-cost, rapid synthesis of highly stable Mg-ACC nanoparticles and their long-term preservation[END_REF][START_REF] Radha | Energetic and structural studies of amorphous Ca 1-x Mg x CO 3 •nH 2 O (0≤x≤1)[END_REF][START_REF] Liu | Phase Transformation of Magnesium Amorphous Calcium Carbonate (Mg-ACC) in a Binary Solution of Ethanol and Water[END_REF] and amorphous iron (II) carbonate (AFC, [START_REF] Sel | Amorphous iron (II) carbonate: Crystallization energetics and comparison to other carbonate minerals related to CO 2 sequestration[END_REF] have been widely studied. Similarly to what has been described for ACC and AMC, amorphous (Ca, Sr, Ba, Mg)-carbonates were fairly unstable as shown by the fact that quick milliQ water rinsing induced their crystallization into a Mg-, Sr-and Ba-substituted calcite.

The crystallization temperature of amorphous Ca-, Mg-, Sr-and Ba-carbonates synthesized in the present study ranged between 345 and 390 °C, which is compatible with the 320-350 °C temperature range measured at higher temperature scanning rates for various ACC and AMC by previous studies [START_REF] Koga | Crystallization of amorphous calcium carbonate[END_REF][START_REF] Radha | Transformation and crystallization energetics of synthetic and biogenic amorphous calcium carbonate[END_REF][START_REF] Radha | Energetic and structural studies of amorphous Ca 1-x Mg x CO 3 •nH 2 O (0≤x≤1)[END_REF]Ihli et al., 2013b). It should be noted that the variations of the crystallization temperature were not correlated with (1) the proportions of Mg, Ca, Sr, or Ba, (2) the amount of water molecules per carbonate or (3) the pH of the precipitating solutions. Alternatively, such variations may parallel the variations of the average size of the precipitates, with crystallization temperature increasing with the particle size, although uncertainties on particle sizes are high. Modelling of prenucleation of nanoparticles suggests that particle stability increases with growing diameter, which supports the present observations [START_REF] Navrotsky | Energetic clues to pathways to biomineralization: Precursors, clusters, and nanoparticles[END_REF][START_REF] Gebauer | Stable Prenucleation Calcium Carbonate Clusters[END_REF][START_REF] Meldrum | Now You See Them[END_REF][START_REF] Raiteri | Water Is the Key to Nonclassical Nucleation of Amorphous Calcium Carbonate[END_REF]. However, [START_REF] Koga | Crystallization of amorphous calcium carbonate[END_REF] observed the opposite effect: the crystallization temperature actually increased with the decrease of the particle size (and it increased with the pH of the precipitating solution). The crystallization temperature may thus depend on related but different parameters such as the supersaturation of the precipitating solution. A possible explanation for the discrepancies between the cited studies might be the effect of Mg adsorbed on the particles surface, since Ihli et al. (2013b) showed that addition of Mg in the precipitation solution had a strong effect on the temperature of crystallization of the ACC. Mg is known to inhibit calcium carbonate crystallization and to Ba/Ca ratios as in the solution. This might be explained by the substantially higher precipitation rates achieved in our experiments [START_REF] Gebrehiwet | The Effect of the CO 3 2to Ca 2+ Ion activity ratio on calcite precipitation kinetics and Sr 2+ partitioning[END_REF]. The differential partitioning of Ca, Sr and Ba between the solution and aragonite synthesized by CO 2 diffusion at pH 9 and a [Mg]:[Ca] of 2:1 ([Mg] = 10 mM) was significantly different, as reported in the literature: Sr/Ca and Ba/Ca ratios have been shown to be almost equal in aragonite and its precipitation solution at 25 °C, but lower in aragonite than in the solution when the temperature was increased [START_REF] Dietzel | Co-precipitation of Sr 2+ and Ba 2+ with aragonite by membrane diffusion of CO 2 between 10 and 50 °C[END_REF]. For example, the distribution coefficient of Sr, defined as the ratio between Sr/Ca in the solid and Sr/Ca in the solution, was measured as 1.19 in aragonite at 25 °C. The distribution coefficient of Ba was estimated at 1.5 under the same conditions [START_REF] Dietzel | Co-precipitation of Sr 2+ and Ba 2+ with aragonite by membrane diffusion of CO 2 between 10 and 50 °C[END_REF].

So far, no study has assessed the distribution coefficient of Sr and Ba in amorphous carbonates. In the present study, the amorphous carbonate forming at 20 °C had a smaller Sr/Ca ratio than the solution (i.e., a distribution coefficient between 0.7 and 1) and a higher Ba/Ca ratio than the solution (distribution coefficient between 1.2 and 1.4). To the first order however, these distribution coefficients of Ba and Sr between the solution and an amorphous carbonate phase are not much different from those measured for crystalline aragonite and relatively close to 1.

Implications for intracellular precipitation in G. lithophora

The mechanism of amorphous (Ca, Mg, Sr, Ba)-carbonate precipitation within the cytoplasm of intracellularly calcifying cyanobacteria, including G. lithophora, is not known yet.

Genomic and genetic approaches may provide some crucial keys in the future. However, the particular chemical and crystallographic properties of the carbonate inclusions should also help constraining the conditions under which precipitation occurs. One difficulty in inferring why some cyanobacteria form intracellular carbonates and others do not, is the very limited available information on the chemical composition of the intracellular medium in cyanobacteria. Most of the studies usually assess that Ca 2+ concentration in the cytoplasm is tightly regulated at very low concentrations. For example, [START_REF] Barrán-Berdón | Free Ca 2+ as an early intracellular biomarker of exposure of cyanobacteria to environmental pollution[END_REF] measured an intracellular Ca 2+ concentration of 100 nM in Anabaena sp. PCC 7120 cells (which do not form intracellular carbonates; Benzerara et al. ( 2014)), with a peak at 2.6 μ M during few seconds when the cells were exposed to environmental pollutants. Such low concentrations are not consistent with the conditions used in our experiments but also incompatible with the formation of intracellular amorphous carbonate inclusions. Indeed, consistently with previous studies on ACC and AMC, the present study stresses the need for high supersaturation levels for the formation of amorphous Ca-, Sr-, Ba-and Mg-carbonates.

It can therefore be noted that no valid estimation of intracellular Ca 2+ concentrations is presently available for intracellularly calcifying cyanobacteria. Interestingly, [START_REF] Margolin | Ca 2+ -mediated GTP-dependent dynamic assembly of bacterial cell division protein FtsZ into asters and polymer networks in vitro[END_REF] suggested that relatively high concentrations of free Ca (a few mM) are required at least temporarily in the cytoplasm of Escherichia coli to allow polymerization of the tubulinlike protein FtsZ. They encountered the same paradox as in the case of intracellularly calcifying cyanobacteria: while intracellular free Ca concentrations are low, processes such as ACC precipitation require much higher concentrations. Only precise, time-and spaceresolved measurements of intracellular free Ca concentration will be able to solve that issue in the future. [START_REF] Badger | Photosynthesis and Inorganic Carbon Usage by the Marine Cyanobacterium, Synechococcus sp[END_REF] measured a total inorganic carbon content (mostly corresponding to HCO 3 -) in excess of 30 mM within the cells of Synechococcus sp. Nageli RRIMP N1. This is in the same order of magnitude as inorganic concentrations used in the present study. Finally, pH is another issue which determines which major inorganic carbon species is prevalent. Several studies have measured the intracellular pH of diverse cyanobacterial strains. For example, [START_REF] Belkin | High internal pH conveys ammonia resistance in spirulina platensis[END_REF] measured a pH between 6.8 and 7.9 in Arthrospira platensis. [START_REF] Jiang | Inactivation of Ca 2+ /H + Exchanger in Synechocystis sp. Strain PCC 6803 Promotes Cyanobacterial Calcification by Upregulating CO 2 -Concentrating Mechanisms[END_REF] measured an intracellular pH between 6.9 and 7.5 in a wild type and a mutant strain of Synechocystis sp. PCC 6803. These cyanobacterial strains do not form intracellular carbonates [START_REF] Benzerara | Intracellular Ca-carbonate biomineralization is widespread in cyanobacteria[END_REF]. Similarly to calcium concentration, pH was much higher in our in vitro syntheses than in these studies.

Future studies will have to determine whether intracellularly calcifying cyanobacteria have specifically a higher intracellular pH than noncalcifying cyanobacteria, or alternatively whether pH can be locally much higher than in the rest of the cytoplasm of these cells.

Some studies have shown the positive effect of confinement on the stabilization of ACC by limiting the contact between amorphous carbonates and the surrounding solution [START_REF] Stephens | Amorphous Calcium Carbonate is Stabilized in Confinement[END_REF]. Other studies have shown the stabilization of amorphous carbonates by specific macromolecules, including glycoproteins, polyphosphate or phosphorylated proteins [START_REF] Bentov | Stabilization of amorphous calcium carbonate by phosphate rich organic matrix proteins and by single phosphoamino acids[END_REF], all of which are potentially present in the cytoplasm of G. lithophora. Here we show that the presence of Sr and Ba in addition to Mg might be one additional parameter increasing the stability of poorly crystalline carbonate inclusions in G. lithophora.

The cyanobacterium Candidatus G. lithophora precipitates poorly crystalline carbonate nanoparticles rich in Sr and Ba. All of these features could be reproduced in the present in vitro experiments. For that purpose, highly supersaturated solutions with significant Sr and Ba concentrations were used. Indeed, the fractionation between the solution and amorphous carbonates was always approximately the same for Sr, Ba and Ca, suggesting that the Sr/Ca and Ba/Ca ratios in the carbonates are about the same as in the precipitating solution. Since [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF] measured Sr/Ca and Ba/Ca atomic ratios of 0.14 and 0.39 respectively in the carbonate inclusions of G. lithophora, it can be inferred that the cytoplasm of G. lithophora, or alternatively the solution in putative vesicles in which these inclusions may form, has the same Sr/Ca and Ba/Ca ratios as the inclusions. These ratios are 86 and 1370 times higher than the respective Sr/Ca and Ba/Ca ratios of the extracellular solution in which G. lithophora cells were grown [START_REF] Couradeau | An Early-Branching Microbialite Cyanobacterium Forms Intracellular Carbonates[END_REF]. Therefore, there must be a mechanism inducing concentration of Sr and Ba over Ca between the extracellular and the intracellular medium of G. lithophora.

A few previous studies have shown an enrichment in Sr or Ba over Ca in biological systems.

For example, distribution coefficients of Sr and Ba are different in coccoliths of Emiliania huxleyi compared to inorganically precipitated calcite [START_REF] Langer | Coccolith strontium to calcium ratios in Emiliania huxleyi: The dependence on seawater strontium and calcium concentrations[END_REF][START_REF] Payne | Calcite crystal growth orientation: implications for trace metal uptake into coccoliths[END_REF][START_REF] Langer | Barium partitioning in coccoliths of Emiliania huxleyi[END_REF]. This has been interpreted as the result of the compartmentation of E.

huxleyi cytoplasm and the functioning of specific transporters, which accumulate Sr and Ba in the vesicles where coccoliths are formed.

Similarly, selective sequestration of Sr has also been observed in the desmid green alga

Closterium moniliferum [START_REF] Krejci | Selectivity in biomineralization of barium and strontium[END_REF]. [START_REF] Krejci | Selectivity in biomineralization of barium and strontium[END_REF] suggested that Ca and Sr were transported inward and outward of the cells by the same channels but since the precipitates formed by these cells were sulfates, they favored the hypothesis that concentration of Sr over Ca was the result of the lower solubility of Sr-sulfates compared to Ca-sulfates. In other word, they suggested that differential fractionation of Sr and Ca between the extracellular solution and the solids was due to mineral precipitation and not biological transport processes as in the coccolith scenario. Since no precursor Sr-, Ca-, or Ba-sulfate phase could be detected in G. lithophora, and since the partitioning between the solution and amorphous carbonates is approximately the same for Ca, Sr and Ba, we favor the hypothesis that there are channels and/or transporters which transport Ca, Sr and Ba at different rates in G. lithophora. The regulation of intracellular Ca 2+ concentration results from the balance between Ca 2+ influx via channels and Ca 2+ efflux via active transporters [START_REF] Waditee | Isolation and Functional Characterization of Ca 2+ /H + Antiporters from Cyanobacteria[END_REF]. Several Ca 2+ transporters have been identified and studied (e.g., [START_REF] Waditee | Isolation and Functional Characterization of Ca 2+ /H + Antiporters from Cyanobacteria[END_REF][START_REF] Jiang | Inactivation of Ca 2+ /H + Exchanger in Synechocystis sp. Strain PCC 6803 Promotes Cyanobacterial Calcification by Upregulating CO 2 -Concentrating Mechanisms[END_REF], but their chemical selectivity will need to be characterized in the future.

Sr/Ca and Ba/Ca ratios in carbonates have often been used as proxies to reconstruct paleoenvironment (e.g., [START_REF] Kunioka | Microdistribution of Mg/Ca, Sr/Ca, and Ba/Ca ratios in Pulleniatina obliquiloculata test by using a NanoSIMS: Implication for the vital effect mechanism[END_REF][START_REF] Meibom | Compositional variations at ultra-structure length scales in coral skeleton[END_REF]. The possibility that some biological processes alter these signals is thus important to evaluate.

CONCLUSIONS

We produced in vitro carbonates with the same shape, size, and chemical composition as those observed in G. lithophora. While the poor crystallinity, the shape and the small size may also be impacted by the macromolecules contained within the cells, the present experiments suggest that Sr/Ca and Ba/Ca ratios were the same in the cytoplasm as in amorphous carbonate inclusions, therefore requiring some mechanisms concentrating Sr and Ba over Ca between the extracellular and intracellular medium of the cells. This kind of in vitro approach shows that intracellular inclusions found within bacteria can provide some keys to the chemical composition of the intracellular solution in these bacteria. 104) and ( 110) calcite planes respectively. Fig. 2. Characterization of the solid products of the synthesis with stoichiometric conditions at high carbonate concentration (100 mM) and with NaHCO 3 as a carbonate source. A) EDXS spectrum of the product. Sodium and chloride result from residual salt formed upon sample drying. B) XRD pattern of the sample. "h" stands for halite diffraction peaks; "c" for calcite.

FIGURE CAPTIONS

The diffraction peaks at 23, 29 and 36° are attributed to the (012), ( 104) and (110) calcite planes, respectively.

Fig. 1 .

 1 Fig. 1. Characterization of the solid products formed in the synthesis with stoichiometric

Fig. 3 .

 3 Fig. 3. Characterization of the solid products of the synthesis under stoichiometric conditions

Fig. 4 .

 4 Fig. 4. Characterization of the solid products of the synthesis performed at low carbonate

Fig. 5 .

 5 Fig. 5. Characterization of the solid products of the synthesis performed at low carbonate

Fig. 6 .

 6 Fig. 6. Characterization of the solid products of the synthesis performed at low carbonate

Fig. 7 .

 7 Fig. 7. Characterization of the solid products of the synthesis performed at low carbonate

Table 1 :

 1 List of performed experiments, indicating the concentrations of the different elements before precipitation and the chemical and mineralogical compositions of the resulting precipitates. All of these experiments were carried out at room temperature. The elemental ratios were determined by EDXS and the water quantity was inferred from TGA analyses. The final pH corresponds to the pH measured about 15 min after the precipitation.The saturation indices of the solutions with amorphous calcium carbonate have been calculated based on the solubility product measured by[START_REF] Kellermeier | A Straightforward Treatment of Activity in Aqueous CaCO 3 Solutions and the Consequences for Nucleation Theory[END_REF]

2.1 Carbonate syntheses

  Carbonates were synthesized by direct precipitation in aqueous solutions containing MgCl 2 , CaCl 2 , SrCl 2 , BaCl 2 and Na 2 CO 3 or NaHCO 3 . Solutions of MgCl 2 .6H 2 O (PROLABO , 98%),

	CaCl 2 (PROLABO, 99%), SrCl 2 .6H 2 O (PROLABO, 99%), BaCl 2 .2H 2 O (MERCK, 99%) and
	NaHCO 3 (SIGMA ALDRICH, 99.5%) were prepared separately. The Na 2 CO 3 solution was
	added to the cation solution (i.e., a mixture of MgCl 2 and/or CaCl 2 and/or SrCl 2 and/or BaCl 2

with varying ratios) while stirring. In one experiment, NaHCO 3 was added to the cation solution. The pH of the different experiments are reported in Table

1

. For each experiment, a

Table

	Experiments	[Mg 2+ ]	Initial concentrations (in mM) [Ca 2+ ] [Sr 2+ ] [Ba 2+ ]	[CO 3	2-]	Saturation indices Calcite ACC	Final pH	Mineral composition	Chemical composition
	Stoichiometric conditions at high concentrations (Fig.1)	18.5	53.1	7.7	20.8	100	3.857	3.012	10.63	Amorphous	Mg 0.16 Ca 0.51 Sr 0.08 Ba 0.25 CO 3 •0.86H 2 O
	Stoichiometric conditions at low concentrations (Fig.3)	1.85	5.31	0.77	2.08	10		2.758	1.913	11.02	Amorphous	Mg 0.04 Ca 0.61 Sr 0.06 Ba 0.29 CO 3 •0.84H 2 O
	Higher Mg concentration (Fig.4)	10	5.31	0.77	2.08	10		2.648	1.803	10.64	Amorphous	Mg 0.16 Ca 0.51 Sr 0.05 Ba 0.28 CO 3 •1.03H 2 O
	Without Mg (Fig.5)	0	5.31	0.77	2.08	10		2.783	1.938	10.88	Calcite	Ca 0.59 Sr 0.07 Ba 0.34 CO 3
	Without Sr (Fig.6)	10	5.31	0	2.08	10		2.657	1.812	10.57	Amorphous	Mg 0.16 Ca 0.56 Ba 0.28 CO 3 •0.93H 2 O
	Without Ba (Fig.7)	10	5.31	0.77	0	10		2.669	1.824	10.64	Amorphous	Mg 0.25 Ca 0.67 Sr 0.08 CO 3 •0.88H 2 O
	Without Sr and Ba (Fig.8)	10	5.31	0	0	10		2.678	1.833	10.68	Monohydrocalcite	(Mg 0.26 Ca 0.74 )CO 3 •1.6H 2 O

ACKNOWLEDGMENTS

The authors thank David Montero, Institut des Matériaux de Paris Centre and Région Ile-de-France for SEM-EDXS analyses and Guillaume Morin for help in diffractogram interpretation. Nithavong Cam was supported by French state funds managed by the ANR within the Investissements d'Avenir programme under reference ANR-11-IDEX-0004-02, and more specifically within the framework of the Cluster of Excellence MATISSE. Karim Benzerara has been supported by funding from the European Research Council under the European Community's Seventh Framework Programme (FP7/2007-2013 Grant Agreement no.307110 -ERC CALCYAN). The TEM facility at IMPMC was purchased owing to a support by Region Ile-de-France grant SESAME 2000 E 1435. The SEM facility at IMPMC was purchased owing to a support by Région Ile de France grant SESAME 2006 I-07-593/R.

solutions (i) and collected one month after the precipitation (ii). "m" stands for monohydrocalcite diffraction peaks and "a" for aragonite. Diffraction peaks at 20.5, 29 and 31.7° are attributed to the (111) and (2-11) monohydrocalcite planes for 20.5°, (112) and(2-12) for 29, (031) and(301) for 31


Akihiko Torii

Relja Arandjelović

Josef Sivic

Masatoshi Okutomi

Tomas Pajdla

24/7 place recognition by view synthesis

We address the problem of large-scale visual place recognition for situations where the scene undergoes a major change in appearance, for example, due to illumination (day/night), change of seasons, aging, or structural modifications over time such as buildings built or destroyed. Such situations represent a major challenge for current largescale place recognition methods. This work has the following three principal contributions. First, we demonstrate that matching across large changes in the scene appearance becomes much easier when both the query image and the database image depict the scene from approximately the same viewpoint. Second, based on this observation, we develop a new place recognition approach that combines (i) an efficient synthesis of novel views with (ii) a compact indexable image representation. Third, we introduce a new challenging dataset of 1,125 camera-phone query images of Tokyo that contain major changes in illumination (day, sunset, night) as well as structural changes in the scene. We demonstrate that the proposed approach significantly outperforms other large-scale place recognition techniques on this challenging data.

Introduction

Recent years have seen a tremendous progress [START_REF] Arandjelović | DisLocation: Scalable descriptor distinctiveness for location recognition[END_REF][START_REF] Cao | Graph-Based Discriminative Learning for Location Recognition[END_REF][START_REF] Cao | Minimal Scene Descriptions from Structure from Motion Models[END_REF][START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Cummins | Highly scalable appearanceonly SLAM -FAB-MAP 2.0[END_REF][START_REF] Hays | im2gps: estimating geographic information from a single image[END_REF][START_REF] Knopp | Avoding Confusing Features in Place Recognition[END_REF][START_REF] Li | Worldwide Pose Estimation Using 3D Point Clouds[END_REF][START_REF] Sattler | Improving Image-Based Localization by Active Correspondence Search[END_REF][START_REF] Sattler | Image Retrieval for Image-Based Localization Revisited[END_REF][START_REF] Schindler | City-Scale Location Recognition[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF][START_REF] Zamir | Accurate Image Localization Based on Google Maps Street View[END_REF] in the large-scale visual place recognition problem [START_REF] Li | Probabilistic location recognition using reduced feature set[END_REF][START_REF] Schindler | City-Scale Location Recognition[END_REF]. It is now possible to obtain an accurate camera position of a query photograph within an entire city represented by a dataset of 1M images [START_REF] Arandjelović | DisLocation: Scalable descriptor distinctiveness for location recognition[END_REF][START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF] or a reconstructed 3D point cloud [START_REF] Li | Worldwide Pose Estimation Using 3D Point Clouds[END_REF][START_REF] Sattler | Improving Image-Based Localization by Active Correspondence Search[END_REF]. These representations are built on local invariant features such as SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] so that recognition can proceed across moderate changes in viewpoint, scale or partial occlusion by other objects. Efficiency is achieved by employing inverted file [START_REF] Philbin | Object retrieval with large vocabularies and fast spatial matching[END_REF][START_REF] Sivic | Video Google: A text retrieval approach to object matching in videos[END_REF] or product quantization [START_REF] Jégou | Product quantization for nearest neighbor search[END_REF] indexing techniques. Despite this progress, identifying the same place across major changes in the scene appearance due to illumination (day/night), change of seasons, aging, or structural modifications over time [START_REF] Hauagge | Image matching using local symmetry features[END_REF][START_REF] Matzen | Scene chronology[END_REF], as shown in figure 1, remains a major challenge. Solving this problem would have, however, significant practical implications. Imagine, for example, automatically searching public archives to find all imagery depicting the same place to analyze changes over time for applications in architecture, archeology and urban planning; or visualize the same place in different illuminations, seasons or backward in time.

In this paper, we demonstrate that matching across large changes in scene appearance is easier when both the query image and the database image depict the scene from approximately the same viewpoint. We implement this idea by synthesizing virtual views on a densely sampled grid on the map. This poses the following three major challenges.

First, how can we efficiently synthesize virtual viewpoints for an entire city? Second, how do we deal with the increased database size augmented by the additional synthesized views? Finally, how do we represent the synthetic views in a way that is robust to the large changes in scene appearance?

To address these issues, we, first, develop a view synthesis method that can render virtual views directly from Google street-view panoramas and their associated approximate depth maps, not requiring to reconstruct an accurate 3D model of the scene. While the resulting images are often noisy and contain artifacts, we show that this representation is sufficient for the large-scale place recognition task.

The key advantage of this approach is that the street-view data is available world-wide opening-up the possibility for a truly planet-scale [START_REF] Klingner | Street view motion-from-structure-from-motion[END_REF] place recognition. Secondly, to cope with the large amount of synthesized data -as much as nine times more images than in the original street-view -we use the compact VLAD encoding [START_REF] Arandjelović | All about VLAD[END_REF][START_REF] Jégou | Aggregating local descriptors into a compact image representation[END_REF] of local image descriptors, which is amenable to efficient compression, storage and indexing. Finally, we represent images using densely sampled local gradient based descriptors (SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] in our case) across multiple scales. We found that this representation is more robust to large changes in appearance due to illumination, aging, etc. as it does not rely on repeatable detection of local invariant features, such as the Laplacian of Gaussian [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF]. While local invariant features have been successfully used for almost two decades to concisely represent images for matching across viewpoint and scale [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF] they are often non-repeatable across non-modeled changes in appearance due to, e.g. strong perspective effects or major changes in the scene illumination [START_REF] Aubry | Painting-to-3d model alignment via discriminative visual elements[END_REF][START_REF] Chum | Total recall: Automatic query expansion with a generative feature model for object retrieval[END_REF]. Not relying on the local invariant keypoint detection comes at a price of reduced invariance to geometric transformation. However, we have found this is in fact an advantage, rather than a problem, as the resulting representation is more distinctive and thus copes better with the increased rate of false positive images due to the much larger database augmented with synthetic views.

Related work

Place recognition with local-invariant features. The large-scale place recognition is often formulated as a variation of image retrieval [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF][START_REF] Philbin | Object retrieval with large vocabularies and fast spatial matching[END_REF] where the query photograph is localized by matching it to a large database of geo-tagged images such as Google street-view [START_REF] Cao | Graph-Based Discriminative Learning for Location Recognition[END_REF][START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Cummins | Highly scalable appearanceonly SLAM -FAB-MAP 2.0[END_REF][START_REF] Hays | im2gps: estimating geographic information from a single image[END_REF][START_REF] Knopp | Avoding Confusing Features in Place Recognition[END_REF][START_REF] Sattler | Image Retrieval for Image-Based Localization Revisited[END_REF][START_REF] Schindler | City-Scale Location Recognition[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF][START_REF] Zamir | Accurate Image Localization Based on Google Maps Street View[END_REF]. The 3D structure of the environment can be also reconstructed beforehand and the query is then matched directly to the reconstructed point-cloud [START_REF] Li | Worldwide Pose Estimation Using 3D Point Clouds[END_REF][START_REF] Sattler | Improving Image-Based Localization by Active Correspondence Search[END_REF] rather than individual images. The underlying appearance representation for these methods is based on local invariant features [START_REF] Tuytelaars | Local invariant feature detectors: a survey[END_REF], either aggregated into an image-level index-able representation [START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Cummins | Highly scalable appearanceonly SLAM -FAB-MAP 2.0[END_REF][START_REF] Hays | im2gps: estimating geographic information from a single image[END_REF][START_REF] Knopp | Avoding Confusing Features in Place Recognition[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF][START_REF] Zamir | Accurate Image Localization Based on Google Maps Street View[END_REF], or associated to individual reconstructed 3D points [START_REF] Li | Worldwide Pose Estimation Using 3D Point Clouds[END_REF][START_REF] Sattler | Improving Image-Based Localization by Active Correspondence Search[END_REF]. These methods have shown excellent performance for large-scale matching across moderate changes of scale and viewpoint that are modeled by the local invariant feature detectors. However, matching across non-modeled appearance variations such as major changes in illumination, aging, or season are still a challenge.

We investigate compact representations based on descriptors densely sampled across the image rather than based on local-invariant features. Densely sampled descriptors have been long used for category-level recognition [START_REF] Bosch | Image classification using random forests and ferns[END_REF][START_REF] Dalal | Histograms of Oriented Gradients for Human Detection[END_REF][START_REF] Lazebnik | Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories[END_REF][START_REF] Oliva | Modeling the shape of the scene: a holistic representation of the spatial envelope[END_REF] including category-level localization [START_REF] Hays | im2gps: estimating geographic information from a single image[END_REF], but due to their limited invariance to geometric transformations have been introduced to instance-level recognition only recently [START_REF] Zhao | Oriented pooling for dense and non-dense rotation-invariant features[END_REF]. While we build on this work, we show that combining dense representations with virtual view synthesis can be used for large-scale place recognition across significant changes of scene appearance.

Virtual views for instance-level matching. Related to our work are also methods that generate some form of virtual data for instance-level matching, but typically they focus on extending the range of recognizable viewpoints [START_REF] Irschara | From structure-from-motion point clouds to fast location recognition[END_REF][START_REF] Shan | Accurate geo-registration by ground-to-aerial image matching[END_REF][START_REF] Wu | 3D model matching with viewpoint-invariant patches (VIP)[END_REF] or matching across domains [START_REF] Aubry | Painting-to-3d model alignment via discriminative visual elements[END_REF][START_REF] Sibbing | SIFT-Realistic Rendering[END_REF] and do not consider compact representations for large-scale applications. Irschara et al. [START_REF] Irschara | From structure-from-motion point clouds to fast location recognition[END_REF] generate bag-of-visual-word descriptors extracted from existing views for virtual locations on a map to better model scene visibility. Shan et al. [START_REF] Shan | Accurate geo-registration by ground-to-aerial image matching[END_REF] use 3D structure to synthesize virtual views to match across extreme viewpoint changes for alignment of aerial to groundlevel imagery. Wu et al. [START_REF] Wu | 3D model matching with viewpoint-invariant patches (VIP)[END_REF] locally rectify images based on the underlying 3D structure to extend the viewpoint invariance of local invariant features (SIFT). Their method has been successfully applied for place recognition [START_REF] Chen | City-scale landmark identification on mobile devices[END_REF] but requires either known 3D structure or rectification on the query side. Recently, rendering virtual views has been also explored for cross-domain matching to align paintings to 3D models [START_REF] Aubry | Painting-to-3d model alignment via discriminative visual elements[END_REF] or to match SIFT descriptors between images and laser-scans [START_REF] Sibbing | SIFT-Realistic Rendering[END_REF].

Modelling scene illumination for place recognition. In place recognition, the related work on modeling outdoor illumination has focused on estimating locations and timestamps from observed illumination effects [START_REF] Hauagge | Reasoning about photo collections using models of outdoor illumination[END_REF][START_REF] Jacobs | Geolocating static cameras[END_REF]. In contrast, we focus on recognizing the same scene across changes of illumination. However, if illumination effects could be reliably synthesized [START_REF] Laffont | Transient attributes for high-level understanding and editing of outdoor scenes[END_REF] the resulting imagery could be used to further expand the image database.

Matching local descriptors across large changes in appearance

In this section we investigate the challenges of using local invariant features for image matching across major changes in scene appearance due to day/night illumination and structural changes in the scene. We first illustrate that local invariant features based on the difference of Gaussian (DoG) feature detector are not reliably repeatable in such conditions. Then we show that densely sampled descriptors result in better matches, but suffer from limited invariance to geometric transformations (scale and viewpoint). Finally, we demonstrate that matching can be significantly improved when we match to a virtual view synthesized from approximately the same viewpoint. In this section we illustrate the above points on a matching example shown in figure 2. We verify these findings quantitatively on the place recognition task in section 5.

In all examples in figure 2 we build tentative matches by finding mutually nearest descriptors. The tentative matches are shown in red. We then geometrically verify the matches by repeatedly finding several homographies using RANSAC. The geometrically consistent matches (inliers) are shown in green. We deem all geometrically verified matches as correct (though few incorrect matches may re-main). The quality of matching is measured by the inlier ratio, i.e. the proportion of geometrically consistent matches. The inlier ratio is between 0 and 1 with a perfect score of 1 when all tentative matches are geometrically consistent.

First, we match the upright RootSIFT descriptors [1] sampled at DoG keypoints [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] between a query image and a street-view image depicting the query place (figure 2(a Second, we repeat the same procedure for the synthesized view (figure 2(d)), which captures the query place from approximately the same viewpoint as the query image. The result is shown in figure 2(e). The resulting inlier ratio of only 0.12 indicates that matching the DoG keypoints across large changes in appearance is difficult despite the fact that the two views have the same viewpoint.

Third, we extract RootSIFT descriptors with a width of 40 pixels (in a 640 × 480 image) on a regular densely sam-pled grid with a stride of 2 pixels. The descriptor matching was performed in the same manner as for the descriptors extracted at the sparsely detected keypoints. Matching the densely sampled descriptors across different viewpoints and illuminations already shows an improvement compared to sparse keypoints, with the inlier ratio increasing from 0.05 to 0.31 (figure 2(c)). The fact that the descriptor (SIFT) is identical for both sampling methods suggests that the main problem is non-repeatability of the Difference of Gaussian local invariant features underpinning the sparsely sampled method, rather than the descriptor itself.

Finally, we apply the densely sampled descriptors to the image pair with different illuminations but similar viewpoints (figure 2(d)). The matches are shown in figure 2(f). The inlier ratio further increases to 0.76 clearly demonstrating the benefits of virtual view synthesis for dense descriptor matching.

View synthesis from street-level imagery

In this section we describe our view synthesis method that expands the database of the geo-tagged images with additional viewpoints sampled on a regular grid. To synthesize additional views we use the existing panoramic imagery together with an approximate piece-wise planar depth map associated with each panorama, as illustrated in figure 4. The piece-wise planar depth map provides only a very coarse 3D structure of the scene, which often leads to visible artifacts in the synthesized imagery. However, in section 5 we demonstrate that this quality is sufficient to significantly improve place recognition performance. In addition, this data is essentially available world-wide [15], thus opening up the possibility of planet-scale view synthesis and place recognition [START_REF] Klingner | Street view motion-from-structure-from-motion[END_REF]. The view synthesis proceeds in two steps. We synthesize the candidate virtual camera locations, which is followed by synthesizing individual views. The two steps are discussed next.

We generate candidate camera positions on a regular 5m × 5m grid on the map that covers the original streetview camera positions. We only generate camera positions that are within 20m distance from the original street-view trajectory, where the trajectory is obtained by connecting the neighboring street-view camera positions. We found that going farther than 20m often produces significant artifacts in the synthesized views. We also use the available depth maps to discard camera positions that would lie inside buildings. The camera positions of the synthesized views are illustrated on the map in figure 3.

To synthesize the virtual views at the particular virtual camera position we use the panorama and depth map downloaded from Google maps [15]. Each panorama captures 360 • by 180 • horizontal and vertical viewing angle, respectively, and has the size 13, 312 × 6, 656 pixels, as illustrated in figure 4(a). The depth map is encoded as a set of 3D plane parameters (normal and distance for each plane) and an 512×256 image of indices pointing, for each pixel, to one of the planes, as illustrated in figure 4(c). Using this index we can look-up the corresponding plane for each pixel, which allows us to generate the actual depth map for the panorama, as illustrated in figure 4(b). All views at a particular virtual camera position are synthesized from the panorama and depth map of the closest street-view image. Virtual views are synthesized by standard ray tracing with bilinear interpolation. In detail, for every pixel in the synthesized virtual view, we cast a ray from the center of the virtual camera, intersect it with the planar 3D structure obtained from the depth map of the closest street-view panorama, project the intersection to the street-view panorama, and interpolate the output pixel value from the neighboring pixels. For each virtual camera location we generate 12 perspective images of 1, 280 × 960 pixels (corresponding to 60 degrees of horizontal field of view) with a pitch direction 12 • and the following 12 yaw directions [0 • , 30 • , ..., 360 • ]. This perspective view sampling is similar to e.g. [START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF]. Examples of the synthesized virtual views are shown in figures 1, 8 and 9. While the synthesized views have missing information and artifacts (e.g. incorrectly rendered people or objects), we found this simple rendering is already sufficient to improve place recognition performance. Higher quality synthesis could be potentially obtained by combining information from multiple panoramas. Rendering one virtual view takes about a second, but we expect 1-2 or- ders of magnitude speed-up using a graphics processing unit (GPU). We generate the same set of perspective views for original street-view images and combine the real and virtual views into a single place recognition database. Note that virtual views are only needed for extracting the compact dense VLAD descriptors as described in section 3 and can be discarded afterwards.

Experiments

In this section we describe the newly collected 24/7 Tokyo dataset, give the place recognition performance measures and outline the quantitative and qualitative results of our method compared to several baselines. 24/7 Tokyo dataset. We have collected a new test set of 1, 125 query images captured by Apple-iPhone5s and Sony-Xperia smartphones. We captured images at 125 distinct locations. At each location we captured images at 3 different viewing directions and at 3 different times of day, as illustrated in figure 5. The ground truth GPS coordinates at each location were recorded by manually localizing the position of the observer on the map at the finest zoom level. We estimate that the error of the ground truth location is below 5m.

The dataset is available at [16]. In the following evaluation, we use a subset of 315 query images within the area of about 1, 600m × 1, 600m covered by our geo-tagged database.

Evaluation metric. The query place is deemed correctly recognized if at least one of the top N retrieved database images is within d = 25 meters from the ground truth position of the query. This is a common place recognition metric used e.g. in [START_REF] Chen | City-scale landmark identification on mobile devices[END_REF][START_REF] Sattler | Image Retrieval for Image-Based Localization Revisited[END_REF][START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF]. The percentage of correctly recognized queries (Recall) is then plotted for different values of N .

Implementation details. To compute the Dense VLAD descriptor, we resize each image to have the maximum dimension of 640 pixels. This is beneficial for computational efficiency and limiting the smallest scale of the extracted descriptors. We extract SIFT [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF] descriptors at 4 scales corresponding to region widths of 16, 24, 32 and 40 pixels. The descriptors are extracted on a regular densely sampled grid with a stride of 2 pixels. When using synthesized images, we remove descriptors that overlap with image regions that have no image data (shown in black in the synthesized imagery). We use the SIFT implementation available in Vlfeat [START_REF] Vedaldi | VLFeat: An open and portable library of computer vision algorithms[END_REF] followed by the RootSIFT normalization [START_REF] Arandjelović | Three things everyone should know to improve object retrieval[END_REF], i.e. L1 normalization followed by element-wise square root. The visual vocabulary of 128 visual words (centroids) is built from 25M descriptors randomly sampled from the database images using k-means clustering.

We have kept the original dimension of the SIFT descriptor, unlike [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF]. Each image is then described by an aggregated intra-normalized [START_REF] Arandjelović | All about VLAD[END_REF] VLAD descriptor followed by a PCA compression to 4,096 dimensions, whitening and L2 normalization [START_REF] Jégou | Negative evidences and cooccurrences in image retrieval: the benefit of PCA and whitening[END_REF]. The similarity between a test query and the database images is measured by the normalized dot product, which could be efficiently performed using [START_REF] Jégou | Product quantization for nearest neighbor search[END_REF][START_REF] Muja | Fast approximate nearest neighbors with automatic algorithm configuration[END_REF]. Following [START_REF] Cao | Graph-Based Discriminative Learning for Location Recognition[END_REF], we diversify the returned shortlists by performing spatial non-max suppression on the map, where we associate the score from each virtual view to the closest street-view panorama. Baseline methods. We compare results to the following baselines. First, we evaluate the VLAD descriptor based on the Difference of Gaussian (DoG) local invariant features [START_REF] Lowe | Distinctive image features from scale-invariant keypoints[END_REF][START_REF] Vedaldi | VLFeat: An open and portable library of computer vision algorithms[END_REF] (Sparse VLAD). Here we use the upright RootSIFT descriptors sampled at DoG keypoints, otherwise the descriptor is constructed in the same manner as our densely sampled VLAD. Second, we compare with the standard sparse Fisher vector [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF] (Sparse FV), which was shown to perform well for place recognition [START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF]. The Fisher vector is constructed using the same upright Root-SIFT descriptors as the Sparse VLAD baseline. Following [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF], the extracted SIFT descriptors are reduced to 64 dimensions by PCA. A 256-component Gaussian mixture model is then trained from 25M descriptors randomly sampled from the database images. As in [START_REF] Jégou | Aggregating local image descriptors into compact codes[END_REF], the resulting 256 × 64 dimensional Fisher vector is reduced to 4, 096 dimensions using PCA, followed by whitening and L2 normalization [START_REF] Jégou | Negative evidences and cooccurrences in image retrieval: the benefit of PCA and whitening[END_REF]. Finally, we also compare results to the bag-of-visual-words baseline. We construct the bag-of-visualwords descriptor (Sparse BoVW) using the same upright RootSIFT descriptors as used in the Sparse VLAD baseline. A vocabulary of 200,000 visual words is built by approximate k-means clustering [START_REF] Muja | Fast approximate nearest neighbors with automatic algorithm configuration[END_REF][START_REF] Philbin | Object retrieval with large vocabularies and fast spatial matching[END_REF]. The resulting bagof-visual-word vectors are re-weighted using adaptive assignment [START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF].

Benefits of the dense descriptor and synthesized views.

First, in figure 6 we evaluate the benefits of having (i) dense descriptors (Dense VLAD) and (ii) additional synthesized views (Dense VLAD SYNTH). We compare performance with the standard Fisher vector descriptor based on local invariant features (Sparse FV), which was found to work well for place recognition [START_REF] Torii | Visual Place Recognition with Repetitive Structures[END_REF]. We show results for all queries (figure 6(a)), but to clearly illustrate the differences we also separate the query images to daytime (figure 6 dense descriptor (Dense VLAD) already improves performance compared to the baseline (Sparse FV), it is the combination of the dense descriptor with synthetic virtual views (Dense VLAD SYNTH) which brings significant improvements for queries with difficult illuminations (figure 6(c)), clearly illustrating the importance of both components of our approach.

Comparison to sparse baselines. In figure 7(a), we show a comparison of our method (Dense VLAD SYNTH) to several baselines that use sparsely sampled local invariant features. For VLAD computed from (sparse) DoG keypoints, adding synthetic virtual views (Sparse VLAD SYNTH) helps (compared to Sparse VLAD). In contrast, adding synthetic virtual views to Fisher vector matching (Sparse FV SYNTH) does not improve over the standard FV without virtual views (Sparse FV). Overall, our method significantly improves over all sparse baselines.

Analysis of descriptor dimensionality. In figure 7(b) we investigate how the place recognition performance changes with reducing the dimensionality of the Dense VLAD descriptor from 4, 096 to 2, 048, 1, 024 and 512 dimensions. We observe a drop in performance specially for the lowest dimension. This suggests, that having a sufficiently rich representation is important for matching across large changes in appearance.

How many virtual views? In figure 7(c) we evaluate the required sampling of virtual views. First, we subsample the virtual views spatially from 5 × 5 meter grid (used in our method so far) to 10 × 10 meter grid. The spatial subsampling to 10 × 10 can reduce the number of virtual views by 75% with only a relatively small drop in place recognition performance. Then we subsample the number of yaw directions to only 6 per camera position, one every 60 • (Dense VLAD SYNTH 60deg) compared to 12 yaw directions, one every 30 • used in our method. In this experiment we keep the spatial sampling to 5 × 5 meters. Although the angular subsampling reduces the number synthetic views by only 50% it results in a fairly significant drop in performance, especially at the top 1 position.

Scalability. For the 24/7 Tokyo dataset, our method synthesizes 597, 744 virtual views compared to 75, 984 perspective street-view images in the same area. Hence, our method needs to index about 9 times more images compared to baselines without virtual view synthesis. We believe scaling-up towards place recognition in an entire city can be achieved with standard compression techniques such as Product Quantization (PQ) [START_REF] Jégou | Product quantization for nearest neighbor search[END_REF]. For example, the largest current place recognition benchmark by Chen et al. [START_REF] Chen | City-scale landmark identification on mobile devices[END_REF] that covers a significant portion of the city of San Francisco contains 1M perspective images. We estimate that starting from a database of this size, but generating 9 times more virtual views with our SYNTH method, and compressing the resulting descriptors with PQ, would only require 2.9GB.

Qualitative results. Figures 8 and9 show examples of place recognition results. Notice that query images (left column) include large changes in both viewpoint and illumination compared to the available street-view for the same places (right column). The synthesized views (2nd column) at new positions significantly reduce the variation in viewpoint and thus enable matching across large illumination changes, as discussed in section 3.

Limitations. Figure 10 shows examples of queries which remain very difficult to localize. The typical failure modes are (i) very dark night time images with limited dynamic range, (ii) places with vegetation, which is hard to uniquely describe using the current representation, and (iii) places where view synthesis fails often due to complex underlying 3D structure not captured well by the approximate depth maps available with street-view imagery.

Conclusion

We have described a place recognition approach combining synthesis of new virtual views with a densely sampled but compact image descriptor. The proposed method enables true 24/7 place recognition across major changes in scene illumination throughout the day and night. We have experimentally shown its benefits on a newly collected place recognition dataset -24/7 Tokyo -capturing the same locations in vastly different illuminations. Our work is another example in the recent trend showing benefits of 3D structure for visual recognition. As we build on the widely available Google street-view imagery our work opens-up the possibility of planet-scale 24/7 place recognition.

Figure 1 .

 1 Figure 1. Matching across major changes in scene appearance is easier for similar viewpoints. (a) Query image. (b) The original database image cannot be matched to the query due to a major change in scene appearance combined with the change in the viewpoint. (c) Matching a more similar synthesized view is possible. (d) Illustration of locations of (a-c) on the map. The dots and arrows indicate the camera positions and view directions.

Figure 2 .

 2 Figure 2. Matching across illumination and structural changes in the scene. First row: The same query image is matched to a streetview image depicting the same place from a different viewpoint (a) and to a synthesized virtual view depicting the query place from the same viewpoint (d). Second row: Matching sparsely sampled SIFT descriptors across a major change in illumination is difficult for the same (e) as well as for the different (b) viewpoints. Third row: Densely sampled descriptors can be matched across a large change in illumination (c) and the matching is much easier when the viewpoint is similar (f). In all cases the tentative matches are shown in red and geometrically verified matches are shown in green. Note how the proposed method (f), based on densely sampled descriptors coupled with virtual view synthesis, obtains significantly higher inlier ratio (0.76) on this challenging image pair with major illumination and structural changes in the scene.

  )) from a different viewpoint. The matches are shown in figure 2(b) and result in an inlier ratio of only 0.05, clearly demonstrating the difficulty of matching DoG keypoints across large changes in appearance.

Figure 3 .

 3 Figure 3. Combining street-view imagery with synthetic views. The figure shows camera positions for part of the 24/7-Tokyo dataset. The positions of the original street-view images are shown in red, the positions of synthesized views (5×5m grid) are shown in grey, and the positions of query images are shown in blue. The inset (top right) shows a close-up of one road intersection. The database of geo-tagged images includes 75, 984 views generated from the original 6, 332 street-view panoramas and 597, 744 synthesized views generated at 49, 812 virtual camera positions.

Figure 4 .Figure 5 .

 45 Figure 4. Input data for view synthesis. (a) The street-view panorama. (b) The associated piece-wise planar depth-map. Brightness indicates distance. (c) The individual scene planes are shown in different colors.

Figure 6 .

 6 Figure 6. Evaluation on the 24/7-Tokyo dataset. The fraction of correctly recognized queries (Recall, y-axis) vs. the number of top N retrieved database images (x-axis) for the proposed method (Dense VLAD SYNTH) compared to the baseline methods (Dense VLAD, Sparse FV). The performance is evaluated for all test query images (a), as well as separately for daytime queries (b), and sunset/night queries (c). The benefits of the proposed method (Dense VLAD SYNTH) is most prominent for difficult illuminations (c).

Figure 7 .

 7 Figure 7. Place recognition performance on the 24/7-Tokyo dataset. Each plot shows the fraction of correctly recognized queries (Recall, y-axis) vs. the number of top N retrieved database images (x-axis).

  (b)), and sunset/night queries (figure 6(c)). While having the

Figure 8 .

 8 Figure 8. Example place recognition results for our method (Dense VLAD SYNTH) compared to baseline using only sparsely sampled feature points (Sparse FV). (Left) Query image. (2nd column) The best matching synthesized view by our method (correct). (3rd column) The best matching street-view image by the baseline (Sparse Fisher vectors without synthesized views). (4th column)The original street-view image at the closest position to the query. Note that our method can match difficult queries with challenging illumination conditions.

Figure 9 .Figure 10 .

 910 Figure 9. Example place recognition results with synthesized views (our method) compared to using only the original Google streetview images. (Left) Query image. Note the difficult illumination. (2nd column) The best matching image (correct) by our method (Dense VLAD descriptor with the database expanded by synthesized views). (3rd column) The best matching image (incorrect) by Dense VLAD matching but using only the original street-view images. (4th column)The original street-view database image at the closest position to the query. Our method (2nd column) that uses virtual views with very similar viewpoints to the query can localize queries with difficult (night) illumination, thus enabling true 24/7 localization. This is not possible using the original street-view images (last column), which depict the same places but from quite different viewpoints. Please see additional results on the project webpage[16]

Acknowledgments. This work was partly supported by JSPS KAKENHI Grant Number 24700161, EU FP7-SPACE-2012-312377 PRoViDE, the ERC grant LEAP (no. 336845), ANR project Semapolis (ANR-13-CORD-0003) and the Intelligence Advanced Research Projects Activity (IARPA) via Air Force Research Laboratory, contract FA8650-12-C-7212. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, AFRL, or the U.S. Government.