Valentin Quesneau

Benoît Roubinet

Pierre-Yves Renard

Anthony Romieu
email: anthony.romieu@u-bourgogne.fr.

Identification of novel 7-(diethylamino)coumarins with aggregation-induced emission properties

Keywords:

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Among the various classes of organic-based fluorophores currently used for the construction of small molecule fluorescent chemosensors/chemodosimeters, coumarin derivatives figure prominently because of several valuable features including: [START_REF] Katerinopoulos | For comprehensive reviews[END_REF] convenient synthetic accessibility, (2) good structural flexibility, (3) valuable spectral features (large Stokes shift and high fluorescence quantum yield), ( 4) easy modulation of their fluorescence properties through the implementation of different photophysical processes (e.g., PeT, ICT, FRET, ...) and/or protection-deprotection of an optically tunable amino or hydroxyl group introduced at the C-7 position and (5) low toxicity [START_REF] Katerinopoulos | For comprehensive reviews[END_REF]. Consistently seeking to optimize the fluorogenic response (mainly, intensometric detection mode) arising from selective interaction/reaction of the coumarin-based probe with its supposed target (bio)analyte, a new probe design principle namely the "covalent-assembly" approach has recently emerged [2,[START_REF]For reviews on "covalent-assembly" type probes, see: a) A. Romieu[END_REF]. This cutting-edge strategy is based on in situ formation of a fluorophore from a non-fluorescent compound (also known as caged precursor) and through an effective intramolecular reaction triggered/catalyzed by the species to be detected. The fundamental feature of this approach is that it guarantees, in theory at least, a "turn-on" fluorimetric signal from a zero background, hence an optimal detection sensitivity. In this area, pioneering works of the Swager ( 2003) [4] and Anslyn (2005) groups [2a], on the fluorescent detection of fluoride anions and divalent heavy metal cations respectively, are based on in situ formation of 7-(diethylamino)coumarin (DEAC) derivatives through lactonisation or Heck-type cyclization reactions. Since these early remarkable achievements, more than 50 examples of "covalent-assembly" type probes for the detection of a wide range of analytes including biothiols, enzymes, metal cations and ROS/RNS, have already been published [5]. In all these cases, creation of blue-green emitting 7-dialkylamino/7-hydroxy-(2imino)coumarins or related fluorophores through lactonisation or Pinner cyclization reactions, is the keystone of the sensing mechanism (Fig. 1).

In the specific case of such reaction-based chemodosimeters devised for the selective sensing of fluoride ions whose detection/quantification in drinking or surface water is very important for public health, the analyte-responsive caged precursor is a O-protected 2-(dialkylamino)-4hydroxycinnamonitrile (or cinnamate ester) derivative and the phenol is masked as a tert-butyldimethylsilyl (TBDMS) or tertbutyldiphenylsilyl (TBDPS) ether (Fig. 1). The direct linkage of this triggering unit to the phenol moiety may negatively affect probe properties such as poor stability and slow response time. By analogy with some colorimetric, fluorescent or chemiluminescent probes recently published in the literature [6], a self-immolative spacer can be used to circumvent these possible issues. In this context, we wished to explore the synthesis and fluorogenic reactivity (towards fluoride ions) of next-generation "covalent-assembly" type probes that incorporate a selfimmolative para-hydroxybenzyl alcohol (PHBA) linker [5j, 7]. During the course of this work, we have highlighted an unexpected SEAr reaction leading to regioselective C-3 benzylation of 4-(diethylamino)salicylaldehyde. Since this type of building blocks is frequently used as starting material for the synthesis of coumarins, we next considered the Knoevenagel condensation between this 2-hydroxybenzaldehyde derivative and C-nucleophiles (i.e., malononitrile and benzothiazole-2acetonitrile) to produce novel DEAC-based fluorophores. (Top) "Covalent-assembly" probe design principle applied to fluorogenic detection of (bio)analytes through in situ synthesis of 7dialkylamino/7-hydroxy-2-iminocoumarins or 7-dialkylamino/7hydroxycoumarins; (bottom) examples of "covalent-assembly" type probes for F -detection reported in the literature (BZT = 2-benzothiazolyl, TBDMS = tert-butyldimethylsilyl, TBDPS = tert-butyldiphenylsilyl) [4, 5f, 5i, 5k, 5n]. a Please note: fused julolidine, 4-dimethylaminophenyl and 1,4diethylpiperazine fragments are also frequently used in the design of such "caged" precursors, to red-shift spectral features of in situ formed coumarin.

In this Letter, in the course of our research on self-immolative linker based second generation "covalent-assembly" probes for fluoride ions, we report our findings on the unusual reactivity of 4-(diethylamino)salicylaldehyde towards 4-(tertbutyldimethylsilyloxy)benzyl bromide. This unprecedented reaction led us to consider the preparation and photophysical characterization of novel DEAC-based fluorophores. A comparative study of their spectral features with those of parent DEAC derivatives lacking a C-8 substituent, and conducted in different solvents (aq. buffers, DMSO and EtOH), has put in evidence an expected added value for these coumarin-based fluorophores, namely aggregation-induced emission properties.

Results and discussion

The vast majority of coumarin-based "covalent-assembly" type probes reported in the literature (see Fig. 1 for the general structure), are readily prepared via a well-established two-step process: (1) protection of the phenol group of salicylaldehyde derivative (mostly, 2-OH of 4-(diethylamino)salicylaldehyde or 2,4-dihydroxybenzaldehyde) with the selected trigger-recognition unit and subsequent (2) Knoevenagel condensation with malononitrile or related C-nucleophiles. In cases where a selfimmolative spacer is used to connect salicylaldehyde and recognition moiety, step ( 1) is often a 2-O-alkylation reaction performed with a benzyl bromide derived from PHBA. To the best of our knowledge, this latter strategy was never applied to the design of fluoride-sensitive caged precursors of coumarins and TBDMS or TBDPS moiety is always directly attached to the phenol oxygen atom of salicylaldehyde derivative (Fig. 1) [4, 5f, 5i, 5k, 5n]. In order to fill this gap and to assess the performances of "covalent-assembly" type probes bearing a fluoride-sensitive self-immolative linker, we planned the synthesis of fluorescent chemodosimeters 1and 2 (Fig. 2). This entailed us, first, to revisit the synthesis of alkylating agent namely 4-(tertbutyldimethylsilyloxy)benzyl bromide (4-OTBDMS benzyl

Revisited synthesis of 4-OTBDMS benzyl bromide 3

Published synthetic procedures towards 4-OTBDMS benzyl bromide 3 are based on a four-step reaction sequence (Scheme 1, top) [8]. First, selective protection of the phenol of 4hydroxybenzaldehyde with TBDMS-Cl, followed by NaBH4mediated reduction of the formyl moiety provided 4-OTBDMS benzylic alcohol. Next, in order to provide a good leaving group, this alcohol was esterified with trifluoroacetic anhydride (TFAA), and finally this trifluoroacetate underwent nucleophilic reaction with LiBr. This procedure enables obtaining 4-OTBDMS benzyl bromide 3 with a satisfactory 40% yield, yet requires tedious purification steps. To shorten the overall number of steps, we have explored direct and selective silylation of PHBA (phenol vs. primary alcohol) with TBDMS-Cl in the presence of a base (Scheme 1, bottom). Several organic and inorganic bases (i.e., DBU, DIEA, Cs2CO3 and K2CO3) were tested and DBU appeared to be the best one providing a satisfying 65% isolated yield. Thereafter, the implementation of Appel reaction (i.e., treatment of alcohol with CBr4 and PPh3 in MeCN) enabled the quantitative conversion of 4-OTBDMS benzylic alcohol into the targeted benzyl bromide derivative 3. However, the poor stability of this latter compound over silica gel prevents its chromatographic isolation in a pure form and with a good yield (only 39% was obtained). To overcome this issue, we used liquid-liquid extractions with heptane to recover pure 3 and with an acceptable yield of 55%. Its structure was unambiguously confirmed by NMR analyses and comparison with published spectroscopic data (see Supplementary data and Figs. S1 andS2). If the overall yield is slightly lower than the published procedure (36% vs. 40%), this new procedure competes thanks to a lower number of steps and easier purification processes.

With this alkylating reagent in hand, we next examined its reactivity towards 4-(diethylamino)salicylaldehyde with the aim of rapidly synthesizing self-immolative "covalent-assembly" type probes 1 and 2. By analogy with the self-immolative caged precursors of 7-(dialkylamino)-(2-imino)coumarins already published, especially those bearing either a 4-(pinacolboronate)benzyl [5p] or a 4nitrobenzyl moiety [5am], reactive towards RNS/ROS and nitroreductase (NTR) enzymes respectively, phenol alkylation of 4-(diethylamino)salicylaldehyde was achieved with 1 equiv. of 3, in the presence of K2CO3 (2 equiv.) as base, in MeCN and at room temperature (Scheme 2). Surprisingly, this reaction was found to be not univocal with the formation, in addition to the desired benzyl ether 4, of an unknown product which was isolated by column chromatography over silica gel in a 10% to 25% yield (depending on the reaction attempt and age of benzylation reagent 3). The same mixture of these two compounds was obtained when K2CO3 was replaced by NaH (60% dispersion in mineral oil). The unknown compound is less polar than 4 but has the same molecular mass as revealed by the HPLC-MS analyses (see Figs. S6, S7, S13 and S14). Thus, two hypothetical structures 5 and 6 resulting either from quaternarization of the N,N-diethylanilino moiety (SN reaction) or from Friedel-Crafts type alkylation of the electron-rich benzene ring (SEAr reaction) can be theoretically put forward. Gratifyingly, a comprehensive NMR study based on 1D and 2D experiments (i.e., COSY and NOESY, see Figs. S11 andS12) confirmed both the lack of aromatic proton H-3, the ortho coupling between protons H-5 and H-6 (J = 8.5 Hz) and the spatial proximity between benzylic methylene protons of 4-OTBDMS benzyl moiety and CH2-protons of diethylamino substituent. We may thus conclude that unknown product formed during the O-alkylation process is the Friedel-Crafts adduct 6. Such C-3-alkyl-substituted 4-(diethylamino)salicylaldehyde derivatives have never been described and their formation under mild conditions as these presented here, is somewhat surprising even if the corresponding benzene ring is electronically enriched by phenolic hydroxyl (deprotonated form in the reaction mixture containing K2CO3) and diethylamino donating groups. Indeed, the sole example of Fridel-Crafts type C-2 benzylation of a 1,3diheterosubstitued benzene was reported by Kumar et al. and it was the reaction between resorcinol (1,3-dihydroxybenzene) and benzyl chloride performed under refluxing xylene (140 °C) for 8 h. In this case, a mixture of mono-(C-2), di-(C-2/C-4) and tribenzyl (C-2/C-4/C-6) adducts in the ratio 6:3:1 was obtained (combined yield 82%) [9]. The regioselectivity of Friedel-Crafts type alkylation leading to 6, is in agreement with the Holleman rules since diethylamino and phenol groups are ortho/para directing and formyl is known to be meta directing group [10]. To gain further information about the possible scope of this reaction, we implemented this synthetic procedure with other alkylating agents including 4-nitrobenzyl bromide and 4-(2,4dinitrophenyloxy)benzyl bromide but O-alkylation of 4-(diethylamino)salicylaldehyde was overwhelmingly observed and only traces of Friedel-Crafts product was detected. This result is consistent with the synthesis of "covalent-assembly" type probes presented above and for which this latter undesired reaction was never mentioned. One possible interpretation of this peculiar reactivity is that the formation of benzyl-type carbocation (the reactive intermediate required for SEAr reactions on salicylaldehyde derivative) is favored only in the case 4-OTBDMS benzyl bromide due to the electron-donating ability of its para-substituent (i.e., +M effect of the silyloxy group vs. -M effect of nitro and 2,4-dinitrophenyloxy moieties). Also noteworthy and in agreement with our hypotheses, is that when we assayed the reaction of 4-(diethylamino)salicylaldehyde with 4-OTBDMS benzyl alcohol under Mitsunobu conditions (i.e., treatment with PPh3 and DIAD in Et2O) [11] the sole observed product was O-alkylated product 4.

2-O-Alkylation of 4-(diethylamino)salicylaldehyde with 4-OTBDMS benzyl bromide

To check the reactivity of the salicylaldehyde part of the molecule 6 and to access to novel 7-(diethylamino)coumarins, we have then explored its condensation with malononitrile and benzothiazole-2-acetonitrile.

Synthesis of 8-substituted-7-(diethylamino)coumarins 7 and 8

The synthesis of both coumarins 7 and 8, through Knoevenagel condensation reaction, was achieved under conventional conditions: treatment with C-nucleophile (1 equiv.), in the presence of piperidine (1 equiv.) and anhydrous Na2SO4 (2 equiv.), in EtOH at room temperature (Scheme 3). Purification by column chromatography over silica gel provided the 7-(diethylamino)coumarins 7 and 8 in a satisfying 60% yield even if an additional purification by trituration with MeOH was required to recover 7 in a pure form. It is important to note that the hydrolysis of imine moiety (this two-step Knoevenagelcyclization sequence actually leads to 2-iminocoumarin derivatives) occurred during the purification process. to a literature procedure [12]. As for the Knoevenagel condensation with 2-(2-benzothiazolyl)acetic acid ethyl ester, reaction time was dramatically shortened using microwave irradiation (20 min at 160 °C vs. 6 h under refluxing MeOH [13]), and purification by column chromatography provided 10 with a good 87% yield. For these four compounds, all spectroscopic data, especially IR, NMR and mass spectrometry, were in agreement with the structures assigned (see Supplementary data).

Photophysical characterization of DEAC-based fluorophores

The photophysical properties of synthesized 7-(diethylamino)coumarins 7-10 were evaluated in DMSO (solvent in which these compounds are perfectly soluble and used for the preparation of their stock solutions), in EtOH and in phosphatebuffered saline (PBS, pH 7.5) which mimics physiological conditions. Fluorophores 7-10 were found to be perfectly soluble in PBS only at low concentrations (less than 5 µM). At higher concentrations, light scattering by suspension was clearly observed on UV-vis absorption spectra (Fig. 3C and Figs. S21, S22, S32, S33, S36 and S39). All results are compiled in Table 1 (see Fig. 3 for the absorption/fluorescence spectra of 8 and Supplementary data for the absorption/fluorescence spectra of compounds 7, 9 and 10). In polar organic solvents (DMSO and EtOH), the C-8 substitution of DEAC scaffold caused the broadening and dramatic blue-shift of the absorption band, quite probably because the formation of aggregates especially in the case of derivative 8 which bears the hydrophobic benzothiazolyl C-3 substituent (e.g., ∆λ1/2max values: 89-93 nm for 8 vs. 64-65 nm for its C-8 unsubstituted counterpart 10). The lack of a significant blue-green fluorescence emission upon excitation at 410 nm (ΦF = 1-2%), confirms the preponderance of non-emissive species in solution and a possible radiation less deexcitation pathway caused by rotational motions of 4-OTBDMS benzyl moiety. The spectral signature in aq. buffer (PBS) is somewhat different because 7 and 8 exhibit a wide emission band ranging from 500 to 700 nm (centered at ca. 560 nm) which is consistent with the behavior described by an aggregation-induced emission (AIE) effect [14]. To confirm this property, the fluorescence behavior of 7 and 8 in water/EtOH mixtures was studied. As displayed in Fig. 4, in pure EtOH, 8 shows weak fluorescence centered at 500 nm. When water fraction varies from 10% to 50%, the emission marginally decreases but without alterations in the shape of emission curve. This is probably related to the aggregationcaused quenching (ACQ) effect. When the fraction of water is greater than 50%, the emission maximum peak red shifts to 560 nm along with a fluorescence intensity increase evidently. As water is a poor solvent of 8, the addition of water would induce the formation of aggregates, that hinders the intramolecular rotational motions of 4-OTBDMS benzyl moiety, thus blocking the non-radiative relaxation pathways and boosting the yellow-orange emission (i.e., restriction of intramolecular rotations (RIR) mechanism) [14]. Moreover, the fluorescence quantum yield of 8 in the solid state was measured to be 7% for an emission maximum centered at 534 nm (see Supplementary data for experimental details related to the determination of solid state absolute fluorescence quantum yields and Figs. S41-S44). The same behavior was observed with the less hydrophobic 3cyano derivative 7 but the formation of strongly emissive aggregates was occurred at a higher fraction of water in EtOH (>70%) (Fig. S45). As expected, this AIE effect was found to be negligible in the case of parent C-8 unsubstituted 7-(diethylamino)coumarins 9 and 10 (Fig. 4 and Fig. S45). Indeed, the shape and position of the fluorescence emission spectrum of 3-cyano derivative 9 are similar whatever the solvent used for the spectral measurements. The fluorescence quantum yield is also in the same order of magnitude. For its part, the 3-(2benzothiazolyl) derivative 10 emits intense green light both in DMSO and in EtOH (ΦF = 67% and 75% respectively), but its dissolution in PBS causes a large bathochromic shift in the emission color with an almost complete quenching of its fluorescence (ΦF = 2.5%). These features are consistent with an ACQ effect instead of AIE characteristics displayed by this more hydrophobic derivative. To clarify which rotors of 4-OTBDMS benzyl moiety are responsible for the AIE effect, we have finally synthesized (i.e., aq. TFA deprotection of TBDMS ether of 8) and studied photophysical properties of the 8-(4-hydroxybenzyl) derivative 11. This compound displays the same spectral features and AIE behavior than that for 8 (Fig. 4 and Figs. S39 andS40), even if the fluorescence emission efficiency of aggregates in water is lower (ΦF 2% vs. 7% for 8). These results may indicate that the AIE activity of 7 and 8 is mainly ascribed to rotational motions of the benzyl moiety and not those of silyl group.

Conclusion

In summary, we have reported our findings related to the unprecedented C-3 functionalization of 4-(diethylamino)salicylaldehyde through a Friedel-Crafts alkylation reaction conducted under mild conditions and with electron-rich 4-OTBDMS benzyl bromide 3. This unusual reaction is not applicable to a wide range of para-substituted benzyl bromide derivatives, especially those currently employed for the synthesis of self-immolative fluorescent probes for (bio)sensing purposes. However, the availability of unusual salicylaldehyde derivative 6 has enabled us to prepare two novel 7-(diethylamino)coumarins through conventional Knoevenagel reaction. A comprehensive photophysical study of these fluorophores led both to the discovery of their AIE properties and the identification of 4-OTBDMS benzyl as a novel AIE-active moiety. The conversion of conventional 7-(diethylamino)coumarin dyes to AIE fluorophores is thus realized by a simple transformation (Friedel-Crafts alkylation), avoiding laborious and more complex synthetic operations sometimes associated with the incorporation of popular AIE-active moieties (e.g., tetraphenylethylene, TPE) onto fluorescent scaffolds [14,[START_REF] Wang | For examples of fluorophore functionalization with TPE moiety[END_REF]. We are confident that this strategy and others related could be applied to the introduction of benzyl-type rotors onto fluorophores for which some positions reactive towards electrophiles are clearly identified (e.g., C-4/C-5 and possibly C-2/C-7 positions of xanthene dyes including fluoresceins, pyronins and rhodamines). Lastly, the identification of some 8-substituted 7-hydroxycoumarin derivatives as potent α-amylase inhibitors [9], could maybe lead medicinal chemists to assess the potential of the reported Friedel-Crafts type reaction for facile preparation of coumarin-based antidiabetic agents [18].

Fig. 1 .

 1 Fig. 1.(Top) "Covalent-assembly" probe design principle applied to fluorogenic detection of (bio)analytes through in situ synthesis of 7dialkylamino/7-hydroxy-2-iminocoumarins or 7-dialkylamino/7hydroxycoumarins; (bottom) examples of "covalent-assembly" type probes for F -detection reported in the literature (BZT = 2-benzothiazolyl, TBDMS = tert-butyldimethylsilyl, TBDPS = tert-butyldiphenylsilyl)[4, 5f, 5i, 5k, 5n]. a Please note: fused julolidine, 4-dimethylaminophenyl and 1,4diethylpiperazine fragments are also frequently used in the design of such "caged" precursors, to red-shift spectral features of in situ formed coumarin.

3 ,

 3 because the published procedures are tedious and time-consuming.

Fig. 2 .

 2 Fig.2. Self-immolative "covalent-assembly" type probes for F -detection and based on the use of PHBA linker (BZT = 2-benzothiazolyl, TBDMS = tertbutyldimethylsilyl), targeted in our project.

3 Scheme 1 .

 31 Scheme 1. (Top) Published synthesis of 4-OTBDMS benzyl bromide 3 [8]; (bottom) shortened synthesis of 4-OTBDMS benzyl bromide 3 devised by us (DBU = 1,8-diazabicyclo[5.4.0]undec-7-ene, TBDMS-Cl = tertbutyldimethylsilyl chloride, TFAA = trifluoroacetic anhydride).

Scheme 2 .

 2 Scheme 2. (Top) Published syntheses of NTR-and peroxynitrite-responsive self-immolative "covalent-assembly" type probes based on O-alkylation of 4-(diethylamino)salicylaldehyde with 4-nitrobenzyl bromide or 4-(pinacolboronate)benzyl bromide and subsequent Knoevenagel condensation reaction (TBAOH = tetrabutylammonium hydroxyde) [5p],[5am]; (bottom) alkylation of 4-(diethylamino)salicylaldehyde with 4-OTBDMS benzyl bromide 3 leading to unexpected Friedel-Crafts alkylation product 6.

Scheme 3 .

 3 Scheme 3. Synthesis of 8-substituted 7-(diethylamino)coumarin dyes 7 and 8 (CC (SiO2) = column chromatography over silica gel).In order to assess the effect of 4-OTBDMS benzyl moiety on photophysical properties of theses novel 7-(diethylamino)coumarin derivatives, we have also prepared the parent compounds 9 and 10 lacking this C-8 substituent (Scheme 4). 3-Cyano-7-(diethylamino)coumarin 9 was prepared from 4-(diethylamino)salicylaldehyde and ethyl cyanoacetate, according

Scheme 4 .

 4 Scheme 4. Synthesis of 7-(diethylamino)coumarin dyes 9 and 10 (MW = microwave irradiation).

Fig. 3 .

 3 Fig. 3. Normalized absorption (blue), excitation (Em. 525 or 600 nm, slit 5 nm, green) and emission (Ex. 410 nm, slit 5 nm, red) spectra of coumarin 8 in DMSO (A), EtOH (B) and PBS (C) at 25 °C. Please note: the recorded absorption spectrum of 8 in PBS is due both to light scattering (by aggregates) and absorption.

  piperidine, EtOH, RT, 30 min conditions B: piperidine, EtOH, MW, 160 °C, 20 min

  To the best of our knowledge, few works have already described the facile functionalization of 2H-chromen-2-one scaffold with AIE-active moieties to generate coumarin-based AIE fluorophores with long-wavelength emission. Worth mentioning in this context are 7-(diethylamino)coumarin or 4methylumbelliferone Schiff base derivatives bearing an AIEactive substituent connected to C-3 or C-8 position via an imine linkage[16]. Some of them have been used as fluorescent probes for effective detection of biothiols (cysteine) or heavy metal cations such as Hg(II)[16a, 16b].

Fig. 4 .

 4 Fig. 4. Demonstration of AIE properties of 8-substituted 3-(2benzothiazolyl)-7-(diethylamino)coumarin 8. (A, left) Fluorescence emission (integrated between 425 and 800 nm) of 8, 10 and 11 vs. water fraction in EtOH/water mixtures (concentration = 1.0 µM, Ex. at 410 nm, Em. 425-800 nm, Ex./Em. slit = 5 nm for 8 and 11 and 2 nm for 10); (A, right) schematic representation of intramolecular rotations in coumarins 8 and 11; (B) Photographs of 8 (15 µM) in PB/EtOH (fraction aq. buffer vol%) mixtures taken under 365 nm UV illumination (PB = phosphate buffer, 100 mM, pH 7.6). Please note: for the recording of Em. spectra, water was used instead of PB to avoid interferences causing by partial precipitation of phosphate salts in EtOH as observed in Fig. 4B. Suspension observed in quartz cells (Fig. 4B) was due to phosphate salts precipitation at low fractions of aq. buffer, and to partial precipitation of coumarin 8 at high fractions of aq. buffer.

Table 1

 1 Photophysical properties of DEAC-based fluorophores studied in this work, determined at 25 °C. Determined using coumarin 153 as a standard (ΦF = 38% in EtOH, Ex. at 410 nm) except for 7 in PBS, Ru(bpy)3Cl2 was used in this case (ΦF = 2.8% in water saturated with air, Ex. at 410 nm)[15].

	Cmpd a	Solvent	λmax Abs (nm)	λmax Em (nm)	∆λ1/2max (nm)	ε (M -1 cm -1 )	ΦF (%) b
	7	PBS, pH 7.5	419	561	90	17 700	2
	7	DMSO	411	468	88	16 300	-c
	7	EtOH	405	467	76	16 400	-c
	8	PBS, pH 7.5	423	556	112	20 550	7
	8	DMSO	414	506	93	23 800	1.5
	8	EtOH	405	502	89	22 900	2
	9	PBS, pH 7.5	352, 431	479	-d	12 200, 6 000	-c
	9	PBS + 5% EtOH, pH 7.5	352, 431	479	-d	10 700, 7 100	1.5
	9	DMSO	430	478	49	46 600	3.5
	9	EtOH	423	467	49	46 900	3
	10	PBS, pH 7.5	463, 498	557	-d	17 200, 18 000	2.5
	10	DMSO	468	516	65	56 400	67
	10	EtOH	459	503	64	54 100	75

a Stock solutions (1.0 mg/mL) of fluorophores were prepared in DMSO.

b c

Very low or no fluorescence. d Not determined due to the presence of two maxima.

Acknowledgments

This work is supported by the CNRS, Université de Bourgogne and Conseil Régional de Bourgogne through the "Plan d'Actions Régional pour l'Innovation (PARI) and the "Fonds Européen de Développement Régional (FEDER)" programs. B. R. gratefully acknowledges the "Région-Haute Normandie" for his Ph. D. grant (2012Ph. D. grant ( -2015)). Financial support from Agence Nationale de la Recherche (ANR, AAPG 2018, DetectOP_BChE, ANR-18-CE39-0014), especially for the post-doc fellowship of V. Q. is also greatly acknowledged, as well as Labex SynOrg (ANR-11-LABX-0029). The authors thank the "Plateforme d'Analyse Chimique et de Synthe se Mole ćulaire de l'Universite ́ de Bourgogne" (PACSMUB, http://www.wpcm.fr) for access to spectroscopy instrumentation and Iris Biotech company for the generous gift of some chemical reagents used in this work. The authors also thank Marcel Soustelle (University of Burgundy, ICMUB, UMR CNRS 6302) for elemental analyses, Prof. Ewen Bodio (University of Burgundy, ICMUB, UMR CNRS 6302, OCS team) for access to SAFAS Flx-Xenius XC spectrofluorimeter (equipped with a BaSO4 integrating sphere), Dr. Jean-Franck Bussotti / M. Olivier Chaudon (SAFAS Monaco company) for their precious advice regarding the determination of absolute fluorescence quantum yields, and Drs. Kévin Renault and Ibai E. Valverde (ICMUB, UMR CNRS 6302) for helpful discussions and relevant comments on this manuscript before publication.

Supplementary data

Supplementary data (all synthetic procedures, spectroscopic and photophysical characterizations of fluorophores described in this work) associated with this article can be found, in the online version:


Olivier Gimenez
email: olivier.gimenez@cefe.cnrs.fr

Jean-Dominique Lebreton

Jean-Michel Gaillard

Rémi Choquet

Roger Pradel

Estimating demographic parameters using hidden process dynamic models

Keywords: capture-recapture, hidden Markov models, lifetime reproductive success, multievent models, multistate models, state-space models, state uncertainty

Structured population models are widely used in plant and animal demographic studies to assess population dynamics. In matrix population models, populations are described with discrete classes of individuals (age, life history stage or size). To calibrate these models, longitudinal data are collected at the individual level to estimate demographic parameters.

However, several sources of uncertainty can complicate parameter estimation, such as imperfect detection of individuals inherent to monitoring in the wild and uncertainty in assigning a state to an individual. Here, we show how recent statistical models can help overcome these issues. We focus on hidden process models that run two time series in parallel, one capturing the dynamic of the true states and the other consisting of observations arising from these underlying possibly unknown states. In a first case study, we illustrate hidden Markov models with an example of how to accommodate state uncertainty using Frequentist theory and maximum likelihood estimation. In a second case study, we illustrate state-space models with an example of how to estimate lifetime reproductive success despite imperfect detection, using a Bayesian framework and Markov Chain Monte Carlo simulation. Hidden process models are a promising tool as they allow population biologists to cope with process variation while simultaneously accounting for observation error.

INTRODUCTION

Structured population models [START_REF] Tuljapurkar | Structured-population models in marine, terrestrial, and freshwater systems[END_REF][START_REF] Caswell | Matrix population models: Construction, analysis and interpretation[END_REF]) allow a detailed description of population change over time by accounting for the number of individuals in discrete classes (grouped according to age, stage or state). These models are widely used in plant and animal demographic studies to assess population dynamics as they allow the projection of future population size, estimation of asymptotic population growth rate, age structure, net reproductive rate, generation time and life expectancy among other descriptive statistics. Matrix models have been used to assess population viability (e.g., [START_REF] Ghimire | Demographic variation and population viability in a threatened Himalayan medicinal and aromatic herb (Nardostachys grandiflora): matrix modelling of harvesting effects in two contrasting habitats[END_REF], evaluate the effects of management strategies on life history strategies (e.g., [START_REF] Servanty | Influence of harvesting pressure on demographic tactics: implications for wildlife management[END_REF] and investigate the feedback between demography and evolution (van Tienderen 2000, see also [START_REF] Gamelon | High hunting pressure selects for earlier birth data: Wild boar as a case study[END_REF] for a recent application).

Traditionally, structured population models are calibrated using longitudinal data collected at the individual level. These data are analyzed with statistical models to estimate demographic parameters like survival, growth, dispersal and fecundity, which are then entered into structured population models. However, several sources of uncertainty are associated with parameter estimation that need to be properly accounted for. First, individuals are difficult to monitor in the wild (mobile animals in particular) because they may or may not be observed at a given sampling occasion. This raises the issue of imperfect detection, which, if ignored, can lead to biased estimates and flawed inference about demographic parameters (e.g., [START_REF] Nichols | Capture-recapture models[END_REF], Gimenez et al. 2008). Second, there can be uncertainty related to measurement error, in particular, correctly assigning a state to an individual, even when the individual is observed or captured. This may be the gender of an individual whenever sex is determined through behavioral clues, or epidemiological status if serological tests generate false positives or negatives, or reproductive status whenever the number of offspring is visually determined by counting the number of young accompanying their parents.

To handle the complexity of the dynamics of free-ranging populations and accommodate the various sources of uncertainty associated with parameter estimation, there is a need to integrate structured population models with novel statistical tools. In this paper, we aim to review and demonstrate how recent statistical models can help bring theoretical models (mechanisms) and the analysis of empirical data (patterns) together. We will focus our attention on hidden process models (see Box 1) that have recently received much attention in the literature [START_REF] Buckland | State-space models for the dynamics of wild animal populations[END_REF][START_REF] Mccarthy | Bayesian methods for ecology[END_REF][START_REF] Newman | Hidden process models for animal population dynamics[END_REF]. In hidden process models, two time series are run in parallel: one captures the dynamic of the true (latent) states via a stochastic process model; the other is static and describes the observations that are made from these underlying but possibly unknown states.

In the first section, we review standard multistate capture-recapture models (see Box 1;[START_REF] Lebreton | Modeling individual animal histories with multistate capture-recapture models[END_REF]) that are used to estimate key demographic parameters, which are then fed to structured population models. We pay particular attention to the imperfect detection of individuals in the wild for analyzing individual longitudinal data. In the second and third sections, we reformulate longitudinal multistate models as hidden process models, either as hidden Markov models (HMM; see Box 1, Pradel 2005) or state-space models (SSM; see Box 1, Gimenez et al. 2007). In both sections, case studies are used to illustrate these models. In the second section, we show how hidden-Markov models can be used to deal with uncertainty in the assignment of reproductive status and its importance when assessing life-history trade-offs.

In the third section, we show how state-space models can be used to estimate lifetime reproductive success (sensu [START_REF] Clutton-Brock | Reproductive Success. Studies of Individual Variation in Contrasting Breeding Systems[END_REF] while accounting for a detection probability less than one. Finally, we discuss the limitations of hidden process models as well as several extensions of these models.

OVERVIEW OF MULTISTATE CAPTURE-RECAPTURE MODELS

Capture-recapture (CR; see Box 1) models have become a central tool in population biology for estimating demographic parameters under imperfect detection of individuals [START_REF] Lebreton | Modeling survival and testing biological hypotheses using marked animals: a unified approach with case studies[END_REF]. CR models were originally designed to estimate population size, with demographic parameters such as survival probabilities progressively incorporated in these models, first as nuisance parameters, and eventually as the main focus (for the early history see Lebreton et al. 2009 p. 93 ff). The standard Cormack-Jolly-Seber model (see Box 1;[START_REF] Cormack | Estimates of survival from the sighting of marked animals[END_REF][START_REF] Jolly | Explicit estimates from capture-recapture data with both death and immigration-stochastic model[END_REF][START_REF] Seber | A note on the multiple-recapture census[END_REF]) is based on the embedding of two distinct processes: survival (with survival probabilities denoted as  ) and detection (with recapture probabilities denoted as p) over discrete time occasions. These parameters are indexed appropriately depending on the type of variation of interest (e.g., time, age or sex; [START_REF] Lebreton | Modeling survival and testing biological hypotheses using marked animals: a unified approach with case studies[END_REF].

A notable feature of the original development of CJS models is that the survival and the detection processes were viewed as intimately entangled. Adequately representing how they were imbricated was indeed at the core of early methodological developments to obtain a likelihood that could be used to properly estimate the parameters. For instance, under timedependence, the probability that an individual is never seen again after occasion i, denoted as c i , is obtained through a backward recurrence relationship (Cormack, 1964), accounting for the fact that an individual may die between occasion i and i+1 or survive and not be detected:

c i =1-f i +f i (1-p i+1 )c i+1 with c last occasion =1.
However, when comparing various types of detection, it clearly appeared that the survivalmortality process was a background process distinct from the detection process. For instance for the same time-dependent survival process, one must be able to accommodate several situations illustrated in Figure 1 (see also [START_REF] Lebreton | Competing events, mixtures of information and multistrata recapture models[END_REF]): i) complete detection, as might occur in developed countries for the human population (with administrative declaration of deaths), ii) incomplete detection of live individuals, i.e., live recaptures or resightings, to be analyzed with CJS models or iii) incomplete detection of deaths, as in 'dead recoveries' of banded birds (usually by the general public) to be analyzed with so called dead-recovery models [START_REF] Brownie | Statistical inference from band recovery data -A handbook[END_REF].

[Figure 1 about here]

In the survival process, death or survival between two sampling occasions is treated as strictly conditional on the fact that the individual is alive at the onset of the target interval: the system has no memory, i.e. the survival process is a Markov chain. Hence, CJS models are implicitly made up of a Markov chain on top of which is an imperfect observation process (e.g., [START_REF] Lebreton | Competing events, mixtures of information and multistrata recapture models[END_REF]. In other words, CJS models are hidden process models where the demographic process is represented by a (first-order) Markov chain (see next sections) that is only partially observed due to imperfect detection. Conveniently, the explicit representation of the survival process in CJS models as a two-state Markov chain (alive / dead) readily provides the probability of individual histories, without having to use the complicated c parameters mentioned above (Caswell and Fujiwara 2004, pp. 477-478).

However, representation of the survival process as a Markov chain did not capture much attention for some time for a number of reasons. First, CR models were considered to be specific and exotic statistical tools, in particular because of the historical focus on population size estimation and the associated difficulties in producing an adequate fully stochastic model [START_REF] Jolly | Explicit estimates from capture-recapture data with both death and immigration-stochastic model[END_REF]. The focus on the estimation of demographic flows developed only in the late 80s [START_REF] Burnham | Design and analysis methods for fish survival experiments based on release-recapture[END_REF][START_REF] Clobert | About recent research in models for mark-recapture and mark-resighting data. A reply to C. Brownie[END_REF]. Second, while the succession of events such as survival and death in CR models was clearly Markovian, historically, the statistical point of views on CR models and Markov chains were radically different: when considering n individuals moving over T time steps according to a Markov chain, the development of stochastic process theory for Markov chains was centered on asymptotic results when T tends to infinity, such as ergodicity for instance; on the contrary, CR models asymptotic results, such as the optimality of maximum likelihood estimators for example, are derived for n tending to infinity for a fixed value of T. Third, parameter estimation for Markov chains under partial observation was not fully developed at that time (e.g., [START_REF] Lawless | The information in aggregate data from Markov chains[END_REF] and could not offer solutions to the CR situation.

The initially slow development of the Markov chain approach to analyzing individual histories is illustrated in Figure 2. The first papers explicitly using this point of view were published in the early 1990s, and from there on, parallel growth in human and animal population studies took place at a common rate of ~ 9 % a year.

[Figure 2 about here] Modeling survival as a hidden process has several strong advantages. First, the process model can easily be made more complex or detailed to address new biological questions. Existing CR extensions, such as multisite [START_REF] Arnason | Parameter estimates from mark-recapture experiments on two populations subject to migration and death[END_REF][START_REF] Arnason | The estimation of population size, migration rates and survival in a stratified population[END_REF][START_REF] Hestbeck | Estimates of movement and site fidelity using mark-resight data of wintering canada geese[END_REF]) and recruitment [START_REF] Clobert | The estimation of age-specific breeding probabilities from recaptures or resightings of marked animals. II Longitudinal models[END_REF]) models, entered this common framework; new generalizations could easily be produced, such as multisite recruitment models [START_REF] Lebreton | Estimation by capture-recapture of recruitment and dispersal over several sites[END_REF]) and multistate CR models (reviewed in detail by [START_REF] Lebreton | Modeling individual animal histories with multistate capture-recapture models[END_REF]Williams et al. 2002 pp. 454-468). Second, it linked CR statistical models with parallel developments in population dynamics models and notably matrix population models (Caswell 2001 p. 37). These models are based on a reasonable representation of differences among individuals, through a finite number of so-called 'i-state variables' that summarize relevant information on the past life of an individual to determine its future in terms of, e.g., individual physiology, reproduction or death [START_REF] Diekmann | Mathematical Epidemiology of Infectious Diseases: Model Building, Analysis and Interpretation[END_REF]. The same idea is clearly inherent in multistate CR models. The empirical use of these models requires that individual data should translate into parameter estimates, i.e. estimates of probabilities of the transition from, say, life to death, rosette to flowering plant or non-breeder to breeder. Several papers have addressed estimation and the use of estimates in multistate dynamics models (e.g., [START_REF] Caswell | Beyond survival estimation: mark-recapture, matrix population models, and population dynamics[END_REF].

Third, the observation process can also be made more complex and tuned to specific situations. [START_REF] Dupuis | Bayesian estimation of movement and survival probabilities from capturerecapture data[END_REF] was among the first to distinguish clearly between the two processes in CR models, considering the transitions between states as a Markov chain and imperfect detection as an independent process generating 'missing data'. Soon, however, the detection process was considered to be potentially more complex, e.g., with the development and systematic exploration of the use of unobservable states for modeling temporary emigration [START_REF] Kendall | Using open robust design models to estimate temporary emigration from capture-recapture data[END_REF] and recruitment [START_REF] Pradel | Comparison of different approaches to study the local recruitment of breeders[END_REF]. Specific models for 'mixtures of information' [START_REF] Burnham | A theory for combined analysis of ring recovery and recapture data[END_REF]) considered detection of individuals either as alive or at the time of death. They appear as specific multistate CR models based on the two states alive / just dead [START_REF] Lebreton | Competing events, mixtures of information and multistrata recapture models[END_REF]. The generalization of the observation process currently includes models that account for uncertainty in the assessment of states [START_REF] Kendall | Adjusting multistate capture-recapture models for misclassification bias: Manatee breeding proportions[END_REF]. The observation process then considers a probabilistic relationship between the underlying states (such as breeder vs. non-breeder) and 'events' (such as seen at nest or seen out of a nest). Hence, an individual seen out of a nest may be in either state. Earlier approaches were specific to permanent states (such as sex, [START_REF] Nichols | Estimation of sex-specific survival from capture-recapture data when sex is not always known[END_REF]), but the more general models are multistate CR models that account for state uncertainty, so-called multievent models [START_REF] Pradel | Multievent: an extension of multistate Capture-Recapture Models to uncertain states[END_REF]. The movement towards understanding and unifying CR models as hidden process models (Figure 3) has followed rapidly with the development of specific statistical tools for hidden process models, either hidden-Markov models or state-space models. We review these developments in the following next two sections.

[Figure 3 about here] In the above multistate CR models, it is assumed that the state of an individual is ascertained upon encounter. This poses no difficulty if the state is a geographical location (e.g., wintering site or breeding colony e.g.), but is less obvious when the state is reproductive status (breeder or non-breeder) or a physiological state like diseased vs. healthy. Diagnosis is typically attempted based upon some criterion like the presence of a brood patch or nestbuilding activity (for a breeder) or antibody level (for a potentially diseased individual). The distinction between the criterion and the state leads to models that distinguish the visible layer (the criterion) from the hidden layer (the state). This distinction between what is observed and what is inferred is key to hidden process models. Because the hidden layer is a first order Markov chain, this naturally leads to the introduction of a HMM for analyzing CR data. The key step is to recognize that whether an individual is observed or not may itself be considered as belonging to the 'observable' layer. This conceptual leap led to the definition of multievent CR models (see Box 1; Pradel 2005), which both generalize multistate CR models to accommodate uncertainty in state assignment and firmly establish that CR models are a particular type of HMMs. However, unlike in HMMs, the recorded information in multievent models is named 'events' rather than 'observations', because the record can be a 'nonobservation'.

Review of HMM applications to estimate demographic parameters

HMMs have a tremendous potential with applications in all domains of ecology [START_REF] Pradel | The stakes of capture-recapture models with state uncertainty[END_REF]), but they historically have been seldom used. An obvious application is in epidemiology [START_REF] Conn | Multistate capture-recapture analysis under imperfect state observation: an application to disease models[END_REF]. Another is sex identification in monomorphic or weakly polymorphic species where clues are mainly derived from behavior [START_REF] Nichols | Estimation of sex-specific survival from capture-recapture data when sex is not always known[END_REF][START_REF] Pradel | Estimating population growth rate from capture-recapture data in presence of capture heterogeneity[END_REF].

A less obvious application is investigating the role of memory in movements among a discrete set of sites. When movements between the successive occasions are determined not only by the position occupied at a particular occasion but also by some previous positions, the previous positions must enter in the definition of the state. Then, when an individual is encountered, it may happen that because the previous relevant positions have not been observed, the exact state is unknown. This invalidates the use of multistate CR models but HMMs can be used (Rouan et al. 2009). The most elaborate application to date makes use of information from nest inspection to study the probability of skipping behavior and nest changing in a colony of Cory's shearwater [START_REF] Sanz-Aguilar | Studying the reproductive skipping behavior in long-lived birds by adding nest-inspection to individual-based data[END_REF]). A more methodological example is the incorporation of mark loss in mark-recapture models during a double-mark study [START_REF] Juillet | A capture-recapture model with double-marking, live and dead encounters, and heterogeneity of reporting due to auxiliary mark loss[END_REF]. Another interesting application copes with the issue of keeping track of the number of breeding episodes when detectability is less than one [START_REF] Desprez | Now you see him, now you don't: Experience, not age, is related to reproduction in Kittiwakes[END_REF][START_REF] Pradel | Breeding experience might be a major determinant of breeding probability in long-lived species: the case of the greater flamingo[END_REF].

It is also possible to account for tag loss and different causes of mortality [START_REF] Tavecchia | Modelling mortality causes in longitudinal data in the presence of tag loss: application to raptor poisoning and electrocution[END_REF]. However, by far, the greatest number of applications relates to class heterogeneity whereby a given individual belongs to a particular unknown class for the duration of the study. [START_REF] Cubaynes | Importance of accounting for detection heterogeneity when estimating abundance: The case of French wolves[END_REF] emphasize the importance of explicitly modeling capture heterogeneity to avoid potentially large biases in the estimation of population size (see Ebert et al. 2010 for a review) and [START_REF] Crespin | Is heterogeneity of catchability in capture-recapture studies a mere sampling artifact or a biologically relevant feature of the population?[END_REF] point out that heterogeneity in capture rate may mirror social hierarchy. [START_REF] Véran | Quantifying the impact of longline fisheries on adult survival in the black-footed albatross[END_REF] and Oliver et al. (2011) used capture heterogeneity to correct for unequal sampling over space. Péron et al. (2010a) went further by modeling class heterogeneity in survival, detection and movement simultaneously and assessed its influence on the study of senescence. Péron et al. (2010b) analyzed in further depth the significance of movement class heterogeneity within a population. Given all these examples, we were surprised to not find studies of reproduction that account for uncertainty in the breeding status.

Identification of reproductive status is not always obvious and failing to acknowledge this problem may lead to mistakes or at least to a loss of useful information. This issue is particularly important in studies of life-history trade-offs. Here we develop an example of how such data could be handled.

Case study 1: Dealing with uncertain states

Data and models

Data were collected between 1940 and 1957 by Lance Richdale on Sooty shearwaters

Puffinus griseus (titis hereafter) and were reanalyzed using multistate CR models by [START_REF] Scofield | Titi (Sooty Shearwaters) on Whero Island: An analysis of historic banding data using modern capture-recapture techniques[END_REF] see also [START_REF] Gimenez | Efficient Profile-Likelihood Confidence Intervals for Capture-Recapture Models[END_REF].

Following the way the data were collected, four states were originally considered: ( 1 Non-Breeder state (NB) that includes failed breeders (birds that had bred previously but were currently skipping reproduction or divorced) and pre-breeders (birds that had not yet bred).

Note that because burrows were not checked before hatching, some birds in the category NB might have bred and failed before being checked. We therefore regard those birds in the B state as successful breeders, and those in the NB state as a mixture of non-breeders, pre-breeders and failed breeders.

We start by fitting a model with state-specific survival, detection and transition probabilities between states, with all parameters constant over time. The states were {nonbreeder, breeder, dead} = {NB, B, D}, and the events were {non-detected, detected as not a successful breeder, detected as a successful breeder} = {0, 1, 2}. Upon its first encounter, an individual has a probability π B to be a breeder and the complement probability to be a non-breeder. Then, it moves among the states according to a first-order Markov chain that can be described by the state specific survival probabilities  NB and  B and the transition probabilities conditional on survival ψ NB→B and ψ B→NB . The state process can be represented by the transition matrix with departure states in rows and arrival states in columns:

f NB y NB®NB f NB y NB®B 1-f NB f B y B®NB f B y B®B 1-f B 0 0 1 é ë ê ê ê ê ù û ú ú ú ú .
Equivalently, it can be represented as the product of a survival matrix by a conditional transition matrix:

f NB 0 1-f NB 0 f B 1-f B 0 0 1 é ë ê ê ê ê ù û ú ú ú ú ´y NB®NB y NB®B 0 y B®NB y B®B 0 0 0 1 é ë ê ê ê ê ù û ú ú ú ú
.

The event process conditional on underlying states is described via the detection probabilities p NB and p B . This event can be summarized by a row-stochastic matrix (i.e., probabilities in a row sum up to one) with states in rows and events in columns:

1-p NB p NB 0 1-p B 0 p B 1 0 0 é ë ê ê ê ê ù û ú ú ú ú .
For instance, in row 1 column 2 is the probability that a non-breeder is detected as not a successful breeder p NB .

To demonstrate how uncertain detection of reproductive status could be analyzed, we generated uncertainty in the assignment of Non-Breeder vs. Breeder as follows. For each individual at each detection occasion: a Non-Breeder was assumed to be judged a Non-Breeder with probability 0.2 (and assigned to event 1), and not judged with the complementary probability 0.8 (and assigned to the ambiguous event 3), while a Breeder was assumed to be judged a Breeder with probability 0.7 (and then assigned to event 2), and not judged with the complementary probability 0.3 (and assigned to event 3). This procedure was implemented in R (R Development Core Team 2011; see Appendix A2). To analyze these imperfect data, only the event process changes. In addition to the detection probabilities, we add state-specific probabilities of breeding state assignment δ NB and δ B . With the same presentation as before, the event process becomes:

1-p NB p NB d NB 0 p NB (1-d NB ) 1-p B 0 p B d B p B (1-d B ) 1 0 0 0 é ë ê ê ê ê ù û ú ú ú ú .
Note that the new event 'breeding state not ascertained' (column 4) can arise for non-breeders (first row) as well as breeders (second row), though with potentially different probabilities.

This matrix can also be written as a product of two matrices, highlighting the successive processes of detection and breeding state ascertainment:

1-p NB p NB 0 1-p B 0 p B 1 0 0 é ë ê ê ê ê ù û ú ú ú ú ´1 0 0 0 0 d NB 0 1-d NB 0 0 d B 1-d B é ë ê ê ê ù û ú ú ú .
Note that when the animal is first encountered, the capture process is not modeled because an animal must be encountered at least once to enter the data set. But the reproductive state ascertainment remains valid. Models with or without imperfect assignment of reproductive state were implemented in the freely downloadable program E-SURGE [START_REF] Choquet | Program E-SURGE: a software application for fitting multievent models[END_REF] see Appendix A1 for details).

Results

The confidence intervals for the assignment probabilities include the true value, despite the considerable loss of information due to uncertain state, in particular for the non-breeders which are recognized as such only 20% of the time (Table 1). The estimates of other parameters remain similar to those of the original model. The most notable and logical change is a loss of precision. However, in a real study, there would probably be on the contrary a gain of precision because the data chosen for analysis with multistate models are those deemed most reliable. In contrast, the model with uncertainty would be run on the larger dataset that includes the less reliable data. For this reason, precision should be enhanced by explicitly dealing with breeding state uncertainty.

[Table 1 about here] [START_REF] Gimenez | State-space modelling of data on marked individuals[END_REF][START_REF] Royle | Modeling individual effects in the Cormack-Jolly-Seber model: a statespace formulation[END_REF][START_REF] Schofield | A Unified Capture-Recapture Framework[END_REF] to distinguish the underlying demographic process from the observation process.

Use of SSMs to estimate demographic parameters dataa survey

The SSM formulation provides a great deal of flexibility and has led to substantial progress in modeling key demographic parameters. [START_REF] Link | Modeling association among demographic parameters in open population capture-recapture data[END_REF] showed how to estimate correlations between demographic parameters such as survival and reproductive rates. [START_REF] Servanty | Assessing whether mortality is additive using marked animals: a Bayesian state-space modeling approach[END_REF] used this approach to estimate cause-specific mortalities and test for additivity in mortality. Other applications include comparisons among species [START_REF] Papadatou | Comparing survival among species with imperfect detection using multilevel analysis of mark-recapture data: a case study on bats[END_REF], spatial structure [START_REF] Péron | Nonparametric spatial regression of survival probability: visualization of population sinks in Eurasian Woodcock[END_REF] and the study of migration [START_REF] Calvert | A hierarchical Bayesian approach to multi-state mark-recapture: simulations and applications[END_REF][START_REF] O'hara | Estimation of rates of births, deaths, and immigration from mark-recapture data[END_REF]. Demographic parameters can be estimated for species with complex life cycles using SSMs [START_REF] Clark | Hierarchical bayes for structured, variable populations: from recapture data to life-history prediction[END_REF][START_REF] Buoro | Investigating evolutionary trade-offs in wild populations of Atlantic salmon (Salmo salar): incorporating detection probabilities and individual heterogeneity[END_REF]. Like HMM, important developments included the incorporation of individual heterogeneity via random effects [START_REF] Schofield | Full open population capture-recapture models with individual covariates[END_REF][START_REF] Royle | Modeling individual effects in the Cormack-Jolly-Seber model: a statespace formulation[END_REF][START_REF] Marzolin | Frailty in state-space models: application to actuarial senescence in the dipper[END_REF]; see [START_REF] Gimenez | Incorporating individual heterogeneity in studies on marked animals using numerical integration: capture-recapture mixed models[END_REF] for an implementation of individual random effects in HMM). In addition, [START_REF] Gimenez | Nonparametric estimation of natural selection on a quantitative trait using mark-recapture data[END_REF]2009) used SSM to estimate selection gradients and visualize fitness surfaces based on capture-recapture data, while [START_REF] Papaïx | Combining capture-recapture data and pedigree information to assess heritability of demographic parameters in the wild[END_REF] showed how to estimate components of phenotypic variance that can be attributed to genetic factors, environmental factors and other unknown factors, hence quantifying heritability of demographic parameters. All these applications have used SSM to estimate demographic parameters. However, SSM also allows the estimation of the unknown latent states. To our knowledge, this has rarely been attempted, although it could be useful in reconstructing individuals' fates and population dynamics (Gimenez et al., in prep). We develop an example of this below.

Case study 2: Measuring Lifetime Reproductive Success in the wild

4.2.1. Estimating fitness in the wild can be difficult Lifetime Reproductive Success (LRS), the number of young measured at some stage by an individual during its lifespan is a single-generation measure of the individual's contribution to future generations [START_REF] Clutton-Brock | Reproductive Success. Studies of Individual Variation in Contrasting Breeding Systems[END_REF]Newton, 1989) and thus provides a reliable measure of individual fitness (see [START_REF] Brommer | Single-generation estimates of fitness as proxies for long-term genetic contribution[END_REF] for an empirical demonstration). The estimation of LRS requires information on the number of offspring an individual has produced and successfully raised at some stage (e.g., born, weaned or fledged, first reproducing) at each reproductive attempt. In theory then, exhaustive monitoring of individuals over their whole reproductive life is needed to estimate LRS. In the field, however, estimating LRS is difficult, because individuals are typically observed (or recaptured) only a fraction of their lifetime. In years when a given individual is not observed, whether it gives birth or not and how many offspring it produces at birth or at the end of the parental care period if it does breed remain unknown. In the following section, we show that, by considering two states (breeder vs. nonbreeder) with the associated number of offspring produced in a SSM, reliable estimates of LRS can be obtained while accounting for detection probability less than one. A similar approach was proposed by Rouan et al. (2009b) who used HMM to compute LRS.

Data and models

We illustrate our approach using the reproductive histories of 211 female roe deer that were monitored in the Trois Fontaines forest, North-East France, between 1976 and 2006 (see e.g. [START_REF] Mcloughlin | Lifetime reproductive success and composition of the home range in a large herbivore[END_REF] for further details about this monitoring). We developed a CR model to analyze these live recaptures of female roe deer. In line with recent work by [START_REF] Gimenez | State-space modelling of data on marked individuals[END_REF], [START_REF] Royle | Modeling individual effects in the Cormack-Jolly-Seber model: a statespace formulation[END_REF] and [START_REF] Schofield | A Unified Capture-Recapture Framework[END_REF], we used a state-space formulation of CR models that explicitly separates the demographic process of interest (survival status in a particular reproductive state) from the observations (the data). We considered five states: alive non-breeder (NB), alive and weaning one fawn (B1), alive and weaning two fawns (B2), alive and weaning three fawns (B3) and dead (D). We denoted X i,t the random state vector taking values (1,0,0,0,0), (0,1,0,0,0), (0,0,1,0,0), (0,0,0,1,0), (0,0,0,0,1) if, at time t, individual i is in state NB, B1, B2, B3 or D, respectively. Based on the roe deer life history (see e.g. [START_REF] Gaillard | Family effects on growth and survival of juvenile roe deer[END_REF], we considered several possible observations that were generated from these underlying states: the animal is not seen (coded 0), the animal is seen without any fawn (coded 1), the animal is seen with one fawn (coded 2), the animal is seen with two fawns (coded 3)

and the animal is seen with three fawns (coded 4) in the dataset. We denoted Y i,t the random observation vector taking values (1,0,0,0,0), (0,1,0,0,0), (0,0,1,0,0), (0,0,0,1,0), (0,0,0,0,1) if, at time t, individual i is not seen or seen with zero, one, two or three fawns respectively.

The model has several parameters. We modeled the probability f i,t k that an individual i alive at time t in state k = NB (non-breeder) or B (breeder whatever the number of fawns weaned) survives to time t + 1. Age-dependence in f i,t k was modeled using three classes:

yearling (1 to 2 years), prime-aged (2 to 5 years) and old adults (6 years and older) according to previous survival analyses (see e.g. Gaillard et al. 1993). We also modeled the probability y i,t k®k' that an individual i makes a transition between states k and k' (k, k' = NB, B1, B2 or B3) between occasion t and t + 1. Finally, we considered the probability p i,t k that an individual i is recaptured at time t in state k = NB or B. Note that, although it is possible, we do not consider individual effects on these parameters, so that the index i is dropped hereafter.

The state-space model relies on a combination of two sets of equations: the state equations, which specify the state of individuals at time t + 1 given their state at time t, and the observation equations, which specify the observation of individuals at time t given their state at time t. The state-space formulation of the CR model [START_REF] Gimenez | State-space modelling of data on marked individuals[END_REF]) is given by:

X i,t+1 X i,t ~ multinomial 1, X i,t f t NB y t NB®NB f t NB y t NB®B1 f t NB y t NB®B2 f t NB y t NB®B3 1-f t NB f t B y t B1®NB f t B y t B1®B1 f t B y t B1®B2 f t B y t B1®B3 1-f t B f t B y t B2®NB f t B y t B2®B1 f t B y t B2®B2 f t B y t B2®B3 1-f t B f t B y t B3®NB f t B y t B3®B1 f t B y t B3®B2 f t B y t B3®B3 1-f t B 0 0 0 0 1 é ë ê ê ê ê ê ê ê ê ù û ú ú ú ú ú ú ú ú ae è ç ç ç ç ç ç ç ç ö ø ÷ ÷ ÷ ÷ ÷ ÷ ÷ ÷ and                                        B t B t B t B t B t B t B t B t NB t NB t t i t i t i p p p p p p p p p p X X Y 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 , 1 l multinomia ~, , ,
where the two equations above are the state and the observation equations, respectively. In the state equation, the matrix gathers the conditional probabilities of being in a state at a particular time occasion (columns) given the state at the previous time occasion (rows). Given that an individual is alive in a given state, it can survive and move to another state, survive and remain in the same state or die. To ensure that these probabilities are within the interval [0, 1] and sum to 1, we used a generalized (or multinomial) logit link function for the transition probabilities [START_REF] Choquet | Automatic generation of multistate capture-recapture models[END_REF]. Consider for example the four transition probabilities from state NB to one of the four states NB, B1, B2 and B3. We considered the probability of remaining in the same state as a reference (one minus the sum of the three other transition probabilities) and used the

transformation           NB NB t k NB t   log
for k = B1, B2 or B3 (see Appendix B for technical details). In the observation equation, the matrix gathers the conditional probability of being observed or not at a particular time occasion (columns) given the state at this current occasion (rows).

Given that an individual is alive in state non-breeder or breeder, it can be recaptured or not in state non-breeder or breeder respectively. Given that an individual is dead, it is missed with certainty. We had also to specify the probability of being in a given state when first captured.

Because first reproduction starts at 2 years in roe deer [START_REF] Gaillard | Effects of age and body weight on the proportion of females breeding in a population of roe deer (Capreolus capreolus)[END_REF] and females are marked in their first winter at about 8 months of age (hence as non-breeder, Gaillard et al. 1993), we considered the probability of being in the state non-breeder when first captured to be unity, and null for any other state.

Estimation of LRS using MCMC methods in a Bayesian framework

We used MCMC simulations in a Bayesian context [START_REF] Mccarthy | Bayesian methods for ecology[END_REF][START_REF] King | Bayesian analysis for population ecology[END_REF][START_REF] Kéry | Bayesian population analysis using WinBUGSa hierarchical perspective[END_REF][START_REF] Royle | Hierarchical modelling and inference in ecology[END_REF]Link and Barker 2011) to fit the above SSM.

Prior distributions need to be specified for the parameters of a given model. Bayes' theorem is then used to update the priors using the likelihood to obtain posterior probabilities of all model parameters. The MCMC algorithm then generates values from a Markov chain whose stationary distribution is the joint posterior distributions of all model parameters. We used empirical means and standard deviations to summarize these posterior parameter distributions.

A nice feature of MCMC algorithms is that reproductive states can be treated as parameters to be estimated, just like survival, transition or recapture probabilities (see Newman et al. 2008 and[START_REF] Toni | Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems[END_REF] for alternatives to MCMC). We produced posterior distributions for breeding states of individuals, which were used to calculate LRS. Specifically, for each MCMC iteration, we calculated for each individual the number of fawns it weaned per time step and then computed an average over all (post burn-in) iterations for each of the females. Summarizing across individuals, we obtained the frequency distribution of LRS. We quantified uncertainty around the LRS estimate by computing a 'mean' standard deviation which was obtained as an average, for a particular value of reproductive output, of all the individual LRS standard deviation for individuals having this specific number of offspring (see Appendix B for more details).

We provided vague prior distributions for all model parameters to induce little prior knowledge. Specifically, we chose uniform distributions on [0, 1] for the recapture and survival probabilities, and normal distributions with mean zero and variances 0.01 for the logit of the transition probabilities. We conducted a sensitivity analysis by modifying these priors using beta distributions with informative moments and posterior estimates of parameters were not changed.

Two MCMC chains with over-dispersed initial values were generated using 50,000 iterations with 10,000 burn-in iterations. Convergence was assessed using the Gelman and Rubin statistic, which compares the within-to the between-variability of chains started at different and dispersed initial values [START_REF] Gelman | Inference and monitoring convergence[END_REF]. Mixing was checked visually by inspecting the chains. The simulations were performed using JAGS [START_REF] Plummer | JAGS: A Program for Analysis of Bayesian Graphical Models Using Gibbs Sampling[END_REF]) and the R package rjags was used to call JAGS, export results in R and calculate LRS. The code is available in Appendix B.

Results

Posterior mean estimates for all parameters of the model are provided in Table 2. There was strong heterogeneity in detection probability according to reproductive status, with breeder less prone to recapture than non-breeders. Survival decreased with increasing age, regardless of reproductive status.

[Table 2 about here]

Much of the density of the distribution of LRS falls at zero (Figure 4): there were many individuals that did not successfully wean any fawns over their entire lifetime. We also observed a long tail in the LRS distribution, suggesting that some females had very high reproductive output.

[Figure 4 about here]

DISCUSSION

Multistate CR models are a valuable tool for estimating demographic parameters in the wild when detection probability is less than one. Their extension to hidden process models allows improved estimation of demographic parameters by accounting for variation in the state process (e.g., demographic stochasticity) while dealing with observation error. Using the same data that one would analyze with multistate models, SSM allows the estimation of demographic parameters as well as unknown states with imperfect detection (a feature we used to estimate LRS in section 4.2). By optimizing the use of all available information including those data that could not be used in multistate analyses, HMM allow us to cope with uncertainty in state assignment (section 3.2).

Traditionally, in the CR literature, HMM are fit in a Frequentist framework while SSM are fitted in a Bayesian framework. This does not need to be the case (de Valpine 2009), and examples of Frequentist analyses of SSMs are becoming available [START_REF] Lele | Data cloning: easy maximum likelihood estimation for complex ecological models using Bayesian Markov chain Monte Carlo methods[END_REF][START_REF] De Valpine | Frequentist analysis of hierarchical models for population dynamics and demographic data[END_REF]. We advocate pragmatism when making the decision of one framework over the other [START_REF] Gimenez | Discussion: Towards a Bayesian analysis template?[END_REF]. This decision can be made based on one's programming skills or familiarity with software. However, there is a risk of using a sledgehammer to crack a nut by fitting unnecessarily complex models that come with difficulties [START_REF] Bolker | Learning hierarchical models: advice for the rest of us[END_REF][START_REF] Cressie | Accounting for uncertainty in ecological analysis: the strengths and limitations of hierarchical statistical modelling[END_REF] like non-identifiability for example (Gimenez et al. 2003[START_REF] Gimenez | Weak Identifiability in Models for Mark-Recapture-Recovery Data[END_REF][START_REF] Luo | Parameter identifiability, constraint, and equifinality in data assimilation with ecosystem models[END_REF]Rouan et al. 2009a). We encourage practitioners to think first of the biological question to be addressed, and resort to the Bayesian or Frequentist approach accordingly. For example, in the roe deer example (section 4.2), we aimed at estimating LRS. We opted for SSM fit in a Bayesian framework using MCMC methods because it allowed the simultaneous estimation of unknown reproductive states and demographic parameters while accounting for parameter estimation error (Figure 2). We could have used Frequentist alternatives to compute LRS (as shown by Rouan et al. 2009b).

In the roe deer data analysis (section 4.2), we assumed that uncertainty in assessing the reproductive status was due to the issue of detection only: if a female was seen in the field, its reproductive status corresponded to the observation made. This was a fair assumption since fawns closely follow their mother during the period considered [START_REF] Gaillard | Body mass and individual fitness in female ungulates: bigger is not always better[END_REF]. However, if this assumption were to be violated, one could easily incorporate uncertainty in state assignment in the calculation of LRS by adapting the model used in the titis data analysis (section 3.2).

We reviewed two types of hidden process models, both having a discrete space of states. Continuous hidden process models have never been used to our knowledge but could be useful in fitting an alternative to matrix models known as integral projection models developed by [START_REF] Easterling | Size-specific sensitivity: applying a new structured population model[END_REF] for plant populations. In contrast with matrix models, which require discretizing individuals' states into classes, demographic parameters are modeled as continuous functions of individuals' states in integral projection models (e.g., size in plants or body mass in animals). Continuous hidden process models could be used as a unifying framework to combine the analysis of individual data, the construction of integral projection models and the projection of population fate by considering relevant states as being continuous.

The extension of hidden process models to the analysis of other sources of information than individual data also holds promise. In particular, [START_REF] Besbeas | Integrating markrecapture-recovery and census data to estimate animal abundance and demographic parameters[END_REF] developed an integrated framework to jointly analyze individual data using CR models and count data collected at the population level. So-called integrated population models use hidden process models to build the likelihood of count data (e.g., [START_REF] Schaub | Use of Integrated Modeling to Enhance Estimates of Population Dynamics Obtained from Limited Data[END_REF], while standard product-multinomials are used to form the likelihood of capture-recapture data. The formulation of integrated population models in a unifying framework using HMM or SSM for both count and individual data would certainly benefit from the flexibility of hidden process models.

CAPTURE-RECAPTURE (CR):

A generic term embedding several protocols for monitoring (mainly) plants and animals in the wild using marks to uniquely identify individuals. The analysis of these data allows the estimation of, e.g., survival, dispersal and abundance while explicitly accommodating for detection probability less than one.

CORMACK-JOLLY-SEBER MODEL (CJS

STATE-SPACE MODEL (SSM):

A model that runs two time series in parallel, one captures the dynamic of the true states (latent) and the other consists of observations that are made from these underlying but possibly unknown states.

Appendix A1. Instructions for fitting the HMM to titis data in program E-SURGE

To fit the model without uncertainty in E-SURGE, three states are considered {alive nonbreeder, alive breeder, dead} = {NB, B, D}, and the events are three as well {non-detected, seen non-successful breeder, seen successful breeder} = {0,1,2}.

In GEPAT, declare two steps for the transitions (survival then conditional transitions). To fit the model with uncertainty in E-SURGE, the same three states are required {alive nonbreeder, alive breeder, dead} = {NB, B, D}, but the events are four {non-detected, seen and ascertained as non-breeder, seen and ascertained as breeder, not ascertained} = {0, 1, 2, 3}.

In GEPAT, declare two steps for the transitions (survival then conditional transitions) and two steps for the events (capture then ascertainment). C (i.e. step 1) = firste + nexte.f A (i.e. step 2) = f 0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,2,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,2,3,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0,3,0,0,0,3,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0, [START_REF]-0 # prob[END_REF][START_REF]-0 # prob[END_REF]0,[START_REF]-0 # prob[END_REF]0,[START_REF]-0 # prob[END_REF]2,[START_REF]-0 # prob[END_REF][START_REF]-0 # prob[END_REF][START_REF]-0 # prob[END_REF]2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,2,1,1,0,0,3,2,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,2,1,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0),byrow=T,nrow=50)

} ################################################################## # 2 -BUGS MODEL ################################################################## sink("multistatect.

  ) Breeder (B); (2) Keeping company with another bird in a burrow; (3) Alone in a burrow; (4) On the surface. Because of numerical issues, Scofield et al. (2001) pooled 2-3-4 together in a

Figure 1 :

 1 Figure 1: Three types of registration of a same discrete time survival process. Top: follow-up

Figure 2 :

 2 Figure 2: Number of papers about Markov models (keywords 'Markov' or 'Markovian') in

Figure 3 :

 3 Figure 3: Schematic representation of a hidden process model for an individual i between two

Figure 4 :

 4 Figure 4: Frequency distribution of lifetime reproductive success in roe deer. Error bars are +/-

Figure 1 :Figure 2 :

 12 Figure 1:

Figure 3 :

 3 Figure 3:

Figure 4

 4 Figure 4: 1

  .e. step 1) = f T ( i.e. step 2) = f For Event: C = firste+nexte.f

  ################################################################## # 1 -DATA MANIPULATION ################################################################## # Dataare in matrix 'mydata', one female per row, with coding as follows: # 0 = female not observed; # 1 = female observed with no fawn; # 2 = female observed with 1 fawn; c(e,min(temp[mydata[i,]>=1]))} # compute number of states and remove structural zeros # (typically prior first capture) lgr = NULL for (i in 1:n) { lgr = c(lgr,length(e[i]:matrix(NA,nrow=nrow(mydata),ncol=ncol(mydata))for (i in 1:nrow(age)){ for (j in e[i]:ncol(age)){ if (j == e[i]) age[i,j] <-1 # yearling if ((j > e[i]) & (j <= e[i]+5)) age[i,j] <-2 # prime-age if (j > e[i]+5) age[i,j] <-3 # senescent }

  prob. of non-breeding individuals / by age # phiB survival prob. of breeding individuals / by age # pNB detection prob. of NB individuals # pB detection prob. of B individuals # psiNBB1 transition prob. from NB to B1 # psiB1B2 transition prob. from B1 to B2 # psiB2B3 transition prob. from B2 to B3 # psiB1B3 transition prob. from B1 to B3

4. STATE-SPACE MODELS: ESTIMATING LIFETIME REPRODUCTIVE SUCCESS 4.1. State-space modeling

  Bayesian framework (using Markov Chain Monte Carlo [MCMC] simulation of the joint posterior distribution of parameters; see section 4.2.3) while HMM have been fit in a Frequentist framework using maximum likelihood methods. CR models were reformulated as SSMs by several authors

	4.1.1. SSM for capture-recapture data
	Parallel to the HMM formulation of CR models, there has been increasing use of state-
	space models (SSM) to handle data on marked individuals. State-space models are a

generalization of hidden Markov models in that the states are not necessarily governed by a Markov process. In practice, parameters of SSMs have been estimated in a

  sampling occasions t -1 and t. The first layer is a succession of hidden states (solid circles) or latent states of individual i at time t (X i,t ) depending on its states at time t -1 (X i,t-1 ). The dynamic of the states is driven by transition probabilities, e.g., survival probability if states are alive or dead. The second layer corresponds to observation or not of individual i at time t (Y i,t ) conditional on individual i being in state X i,t . These events are driven by some probabilities, e.g., detection probabilities if observations are being captured or not. The observation process is the visible part of the state process. Note that this representation holds for both hidden-Markov models and state-space models.

TABLE LEGENDS Box 1

 LEGENDS1

	: Glossary (alphabetical order)

Table 1 :

 1 Parameter maximum likelihood estimates (MLE) with standard errors (SE) for the hidden-Markov model fitted to the titis data, with and without uncertainty in state assignment.

	All parameters are probabilities. Note that the probability of remaining non-breeder and the
	probability of making the transition from state breeder to state non-breeder are not displayed
	but can be obtained as a complement.

Table 2 :

 2 Monte Carlo parameter numerical summaries (posterior means with standard deviations) from the state-space model fitted to the roe deer data. All parameters are probabilities. Note that the probability of remaining in the same reproductive state is not displayed but can be obtained as a complement.

  ): A capture-recapture model that is used for open populations to estimate time-dependent survival and detection probabilities. The Cormack-Jolly-Seber model is easily formulated as a hidden process model by considering two states (alive or dead) and two observations or events (detected or not). The dynamic of states is governed by a Markov model parameterized with survival probabilities while the observations are made from these states according to detection probabilities. An extension of the Cormack-Jolly-Seber model that allows estimation of transition between states. Examples of such states are geographical sites, behavioral or physiological conditions like, e.g., breeding vs. non-breeding or healthy vs. diseased.

	HIDDEN PROCESS MODEL: A generic term referring to either a state-space model or a hidden
	Markov model.
	MULTIEVENT MODEL: A particular type of hidden Markov model. Multievent models
	correspond to extensions of multistate models that allow accommodating uncertainty in state
	assignment.
	MULTISTATE MODEL:

HIDDEN MARKOV MODEL (HMM):

A particular case of state-space model in which the states are Markovian, i.e. the next state depends only on the current state and not on the sequence of states that occurred before.

Table 1

 1

	1	:
			Reproductive state	Reproductive state
		parameter	uncertainty	uncertainty
			ignored	modeled
		being a breeder upon first encounter π B	0.704 (0.014)	0.704 (0.028)
		survival of non-breeders  NB	0.801 (0.016)	0.814 (0.016)
		survival of breeders  B	0.849 (0.016)	0.837 (0.019)
		transition non-breeder to breeder ψ NB→B	0.249 (0.021)	0.219 (0.025)
		transition breeder to breeder ψ B→B	0.761 (0.021)	0.774 (0.049)
		detection of non-breeders p NB	0.597 (0.028)	0.565 (0.028)
		detection of breeders p B	0.563 (0.027)	0.598 (0.032)
		non-breeder state assignment δ NB	NA	0.188 (0.014)
		breeder state assignment δ B	NA	0.738 (0.055)

Table 2

 2

	1	:
			posterior mean estimate
		parameter
			(standard deviation)
		transition non-breeder to weaning 1 fawn ψ NB→B1	0.160 (0.033)
		transition non-breeder to weaning 2 fawns ψ NB→B2	0.149 (0.031)
		transition non-breeder to weaning 3 fawns ψ NB→B3	0.208 (0.036)
		transition weaning 1 fawn to non-breeder ψ B1→NB	0.274 (0.067)
		transition weaning 1 to 2 fawns ψ B1→B2	0.163 (0.068)
		transition weaning 1 to 3 fawns ψ B1→B3	0.316 (0.086)
		transition weaning 2 fawns to non-breeder ψ B2→NB	0.229 (0.059)
		transition weaning 2 to 1 fawns ψ B2→B1	0.159 (0.059)
		transition weaning 2 to 3 fawns ψ B2→B3	0.302 (0.082)
		transition weaning 3 fawns to non-breeder ψ B3→NB	0.261 (0.048)
		transition weaning 3 to 1 fawns ψ B3→B1	0.156 (0.045)
		transition weaning 3 to 2 fawns ψ B3→B2	0.224 (0.052)
		detection of breeders p B	0.341 (0.032)
		detection of non-breeders p NB	0.821 (0.110)
		survival of yearling non-breeders  NB (y)	0.941 (0.027)
		survival of prime-age non-breeders  NB (pa)	0.843 (0.031)
		survival of old non-breeders  NB (o)	0.648 (0.057)
		survival of prime-age non-breeders  B (pa)	0.934 (0.019)
		survival of old non-breeders  NB (o)	0.831 (0.030)

ACKNOWLEDGMENTS

The authors warmly thank Mathieu Buoro for his help with Figure 3, Lauriane Rouan for discussion about LRS calculation using capture-recapture data, Paul Scofield and David Fletcher for sharing the titis data, Jean-Michel Marin and Christian Robert for a discussion about the distinction between HMM and SSM. We also thank the participants to the 2009 and 2010 Montpellier workshops on the analysis of capture-recapture data for their feedback on the titis analyses. This research was supported by a grant (reference: ANR-08-JCJC-0028-01) from the 'Jeunes Chercheuses et Jeunes Chercheurs' program of the French ANR to work on capture-recapture mixed models.

Appendix A2. R Code for generating uncertainty in titis data

To artificially generate uncertainty on both the states non-breeder and breeder, we used the R script below to alter the raw capture-recapture data # original data are stored in R matrix titi with individuals in rows and years in columns

# Breeders are ascertained with probability .7 (event = 1), # or not ascertained with probability .

Appendix B. BUGS and R codes to estimate LRS using SSM fitted to deer data

# for each female at each occasion, store its state

alive <-as.matrix(alive) init1 <-list(pB=0. [START_REF]0 # because of the conditioning on first capture, prob[END_REF]phiNBy=0.3,alive=alive) # second list of inits init2 <-list(pB=0.5,phiNBy=0.6,alive=alive) # concatenate list of initial values inits <-list(init1,init2) # specify the parameters to be monitored parameters <-c("phiBsa","phiBa","phiNBy","phiNBsa","phiNBa","pB","pNB","alpha","lrs") # load R package to call JAGS from R library(rjags) # run JAGS start<-as.POSIXlt(Sys.time()) jmodel <-jags.model ("multistatect.bug", mydatax, inits, n.chains = 2,n.adapt = 50000) jsample <-coda.samples(jmodel, parameters, n.iter=10000, thin = 1) end <-as.POSIXlt(Sys.time()) duration = end-start # save results save(jsample,jmodel,duration,file='agedeer30.Rdata') arrows (barx,table(lrsfinal)+res, barx, table(lrsfinal)-res, angle=90, code=3,length = 0.05)