Elizabeth Anne Hillard

Pascal Pigeon

Anne Vessières

Christian Amatore

Gérard Jaouen
email: gerard-jaouen@enscp.fr

The influence of phenolic hydroxy substitution on the electron transfer and anti-cancer properties of compounds based on the 2ferrocenyl-1-phenyl-but-1-ene motif

The ferrocenyl compound 2-ferrocenyl-1,1-bis(4-hydroxyphenyl)-but-1-ene (3), is very cytotoxic against breast cancer cells (IC 50 = 0.44 µM against MDA-MB-231). We now report the synthesis of a new series of para-and meta-substituted mono-and di-ferrocenyl phenols

). This structure-activity relationship study was motivated by our hypothesis that the cytotoxicity of 3 is related to its ability to form a quinone methide structure after two in situ 1-electron oxidations, a process which requires the presence of at least one p-phenol. The mono-ferrocenyl compounds (including those previously reported) are reasonably well recognized by the oestrogen receptors α (RBAs = 0.9-9.6%) and β (RBAs = 0.28-16.3%), although the bulkier di-ferrocenyl compounds show very little affinity. In vitro, the cytotoxic effects of the phenolic complexes are related to the positioning of the hydroxyl group (para-superior to meta-), and to the number of ferrocenyl groups (one superior to two), with IC 50 values against the MDA-MB-231 cell line ranging from 0.44-3.5 µM. On the hormone-dependent breast cancer cell line MCF-7, the observed effect seems to be the result of two components, one cytotoxic (antiproliferative) and one estrogenic (proliferative). Electrochemical studies show that only the compounds with a pphenol engage in proton-coupled intramolecular electron transfer.

Introduction

Although the anti-cancer properties of ferrocene-containing molecules were first studied in the late 1970s, 1 systematic investigations were not carried out until Köpf-Meyer and Neuse established anti-tumour activity for ferricenium salts in 1984. 2 This work led to the proposal that ferrocenyl compounds could be activated in the cell by biooxidation, and that bothferricenium and ferrocene-(in a water soluble form) containing compounds could give rise to cytotoxic effects. 3 To this end ferrocene has been incorporated into water soluble polymers, 4 tethered to a DNA intercalator, 5 phosphino compounds, 6 vitamin B1, 7 and other biomolecules. 8 Diferrocene compounds 9 and ferrocene-bearing transition metal ligands, 10 and a variety of other small ferrocenyl molecules 11 have also been investigated for anti-cancer activity. Cytotoxic pathways involving DNA have been suggested for the activity of ferrocenyl compounds. 12 Our laboratory has been studying the effects on the proliferation of breast cancer cells of ferrocenyl phenols, especially those based on the 1,1-diphenyl-but-1-ene motif. 13,14 Some small organic phenols have been shown to possess oestrogen receptor modulating properties, 15 and, at higher concentrations, cytotoxic properties. 16 We have been trying to enhance the cytotoxicity of these types of compounds by the addition of ferrocene, which we hope will be oxidized to ferricenium within the cell. We routinely test the proliferative/antiproliferative effects of our new compounds on MCF-7 (oestrogen receptor positive) and MDA-MB-231 (oestrogen receptor negative) breast cancer cell lines, and have had varying degrees of success. The most promising compounds to date are shown in Chart 1, all of which exhibit IC 50 values at low micromolar or submicromolar concentrations for the MDA-MB-231 cell line. The MCF-7 cell line is the standard for hormone dependant breast cancers, and the proliferative/antiproliferative effects connected to the estrogenicity/antiestrogenicity of molecules at concentrations of 10 -5 -10 -7 M are primarily mediated by the oestrogen receptor alpha (ERα), a nuclear receptor present in these cells. It is known that the dimethyl amino chain of the active metabolite of the breast cancer drug tamoxifen, and also present in compound 1, interacts with ERα in such a way as to prevent DNA transcription and cell replication. [START_REF] Brzozowski | [END_REF] The other compounds, lacking the amino chain, would be expected to be estrogenic and proliferative on MCF-7 cells. However, compounds 2-4 have shown antiproliferative effects in both the MCF-7 (ER+, hormone dependent) and MDA-MB-231 (ER-, hormone independent) cell lines, which can be attributed only to the innate cytotoxicity of the molecule. It is important to note that 1-4 conform to a particular structural motif, where the ferrocenyl group is located on carbon 2 of the but-1-ene group, the phenol group(s) resides on carbon 1, and a conjugated -system exists between the ferrocenyl and phenol groups. It appears that this motif is directly related to the cytotoxic effects shown by these compounds. In a previous study we discovered that compounds 1-4 exhibit cyclic voltammograms characteristic of an interesting structural rearrangement due to an intramolecular electron transfer from the phenolic donor to the electrochemically generated ferricenium acceptor in basic conditions. 18 We have proposed that the final outcome of this process is the formation of a reactive quinone methide-type (QM) structure after two oneelectron oxidations and the loss of two protons.

Chart 1 Previously reported cytotoxic ferrocenyl phenolic compounds 1, 14 2, 19 3, 13 4. 20 Donor-acceptor assemblies possessing a ferrocene donor have been extensively studied, especially in terms of their non-linear optical properties, 21 and structural rearrangements as a result of proton-or metal ion-coupled intramolecular electron transfer processes. 22 However, the possibility of a ferricenium moiety acting as an acceptor has only recently been explored. 23,24 In particular, a paper by Nishihara and co-workers in Tokyo described the rearrangement of 2-(2-ferrocenylvinyl)hydroquinone to a novel allene quinonoid structure via two one-electron oxidations and intramolecular electron transfer to a ferricenium acceptor. 25 In order to test our hypothesis that QM-generation is related to the observed cancer cell death, and to further study this novel mechanism of formation, we have synthesized a number of new compounds, including those where the hydroxyl group of the phenol is in the metaposition. For example, in compound 6, the ferrocene--system-phenol motif is maintained, but it carries a m-phenol, so that a QM structure is not accessible. Compound 7 is similar to 6, with the addition of a p-phenol group; in this case we expect that the m-phenol will act as a spectator, and the p-phenol might engage in QM formation. We have also synthesized the diferrocenyl p-phenol (8), and m-phenol (9) to evaluate the importance of the steric effect on the biological efficacy of the compounds. We here report the synthetic, electrochemical, and biochemical results for the new compounds 5-9 shown in Chart 2, and compare these results

to those of previously reported compounds 2 and 3 as appropriate.

Chart 2 New compounds studied in this report.

Scheme 1

Results and discussion

Synthesis

The formation of the new compounds 5-9 was generally accomplished by a Friedel-Craft acylation of ferrocene, followed by a McMurry cross-coupling of the ferrocenyl ketone with the appropriate benzophenone. Thus, to synthesize compounds 8 and 9, we first prepared the known ketone 10 26 (yield 62%) and the new ketone 11 (yield 82%) by a Friedel-Craft reaction of the corresponding acyl chlorides with ferrocene, as shown in Scheme 1. The ketones were demethyled with boron tribromide in dichloromethane to give the phenolic compounds 12 and 13 with yields of 74% and 76%, respectively. The yield for the formation of 12 using boron tribromide was higher than the previously reported demethylation of 10 with aluminium trichloride (32%), 26 but the reaction time was longer (unoptimized 12 h vs. 1.25 h). The

McMurry reaction of these ketones with propionyl ferrocene 27 gave the cross-coupled diferrocenyl compounds 8 and 9 with a yield of 39% for each, Scheme 2.

Scheme 2

In the same manner, the McMurry reaction of propionyl ferrocene with dibenzophenone gave compound 5 with a yield of 25%, Scheme 3. This compound was impossible to separate by silica gel column chromatography from the two homo-coupled compounds, due to the similar (low) polarities of these compounds lacking any phenol group. Compound 5 was directly isolated by preparative HPLC of the crude mixture.

Scheme 3

The mono-ferrocenyl compounds 6 and 7 were synthesized from propionyl ferrocene by a

McMurry reaction using the corresponding phenolic ketones (commercially available for 6, and known from the literature for 7 28,29 ) to give a mixture of Z and E isomers in an approximately 1 : 1 ratio.

Compounds 6 and 7 were purified on a silica gel column with dichloromethane or a dichloromethane-acetone solution as the eluent, and then were re-purified via HPLC. Z and E isomers of compounds 6 and 7 were impossible to separate, while isomers of compounds 8 and 9 were partially separated but rapidly (within a few hours) isomerised to give a 1 : 1 mixture of Z and E isomers.

Cyclic voltammetry.

Compounds 5-9 were studied by cyclic voltammetry in methanol and methanol-pyridine solutions. In methanolic solutions, all of the compounds gave rise to the expected reversible ferrocene/ferricenium redox waves, with 5, 6, and 7 giving rise to one wave, and compounds 8 and 9 displaying 2 one-electron waves. The separation of observed redox potentials (E•')

for the two waves generated by compounds 8 and 9 was not significantly different: 162 and 159 mV, respectively. These values are slightly lower than that of trans-Fc(CH=CH)Fc, which has been reported as 170 mV in CH 2 Cl 2 , 30 although the disparity can be accounted for by the decrease in Coulombic repulsion due to the solvent cage. The presence of two oneelectron oxidation waves instead of one two-electron oxidation wave signifies a stabilization of the mixed valence species (Fc II , Fc III ), the extent of which is often expressed as the comproportionation constant, Kc. For 8 and 9, Kc = 550 and 490, respectively, (using the equation E•' = (RT / F)lnKc) 30 and thus these compounds can be considered Robin-Day

Class II mixed-valent complexes, with moderate electron coupling between the oxidized and reduced centres. 31 Oxidation potentials vs. SCE are given in Table 1. In terms of the interaction of the electrochemically generated cations with the added pyridine, the compounds can be divided into two categories. For compounds 5, 6 and 9, which do not possess p-phenols, no significant difference in the electrochemical behaviour was observed upon the addition of pyridine, Fig. 1a,b. Conversely, for those compounds carrying a p-phenol, 7 and 8, the addition of pyridine altered the CV substantially, especially in view of the reversibility of the ferrocene redox couple and the emergence of a new peak slightly higher in potential than the ferrocene oxidation. We have previously observed this electrochemical behaviour for compounds 1-4, and have attributed it to an intramolecular electron transfer from the organic skeleton to the ferricenium moiety coupled with deprotonation of the phenol by pyridine, eventually resulting in a QM structure. 18 While the electrochemical behaviour of compound 7 is similar to that of compounds 1-4, that of 8 is complicated by the presence of a second ferrocenyl group. The voltammetry of 8 in the presence and absence of pyridine shows that the intramolecular electron transfer leading ultimately to the quinone methide formation occurs at the level of the first oxidation wave.

Since the ferricinium group attached to the C1 carbon atom of the central double-bond (see Scheme 4) cannot be conjugated directly to the phenol moiety, one must conclude that the intramolecular phenol oxidation occurs via the C2 ferricinium. Yet since both ferrocene units give rise to a significant electron transfer interaction, one cannot strictly differentiate between the two under electrochemical conditions. Nevertheless, this suggests that the C1 ferrocene moiety is oxidized after the electron transfer to the C2 ferricenium moiety and rearrangement to the quinonoid, as shown in Scheme 4.

Scheme 4

Thus, the electronic environment around the C1 ferrocene is expected to be substantially changed when it is finally oxidized. This is confirmed by the large anodic shift of its oxidation potential in MeOH-py compared to that in MeOH (0.703 vs. 0.504 V, respectively). The influence of nearby quinonoids on the destabilization of the ferricenium cation has been previously observed, and has been attributed to partial electron donation from the ferrocene to the quinonoid moiety. 24 Chart 3 Electron delocalization on a) p-phenol and b) m-phenol.

We asked ourselves why only the compounds possessing a p-phenol engaged in electron transfer, given the ostensible similarities in molecular and electronic structure between these compounds. We have previously shown in these types of compounds that in the initial cation the radical is localized on the ferricenium group, and that there is little electronic delocalization with the phenol group prior to deprotonation. 18 This is supported by the comparison of the ferrocene oxidation potentials in MeOH for 2 (0.397 V), 5 (0.421 V), and 6 (0.403 V), which show little difference, and hence little stabilization of the radical by the phenol group. The answer may be found instead in considering the stability of the neutral radical species which is generated after electron transfer and deprotonation. By drawing the canonical structures contributing to the delocalization of the radical, Chart 3, we can see that the p-phenol is able to delocalize the radical onto the C1,C2-alkene, while the radical in the m-phenol can only be delocalized over the phenol itself. Thus, complexes with a parasubstituted phenol benefit from greater resonance stabilization and are therefore energetically more accessible intermediates than their m-phenol analogues.

Biochemistry

RBA and lipophilicity values. The affinities for the oestrogen receptor of the newly synthesized complexes were measured on the two isoforms of the oestrogen receptor, ERα and ERβ, and are reported as relative binding affinity (RBA) values in Table 2. All the compounds were recognized by both forms of the ER but the RBA values are quite different ranging from high (about 5%) to low values (less than 1%). In the mono-ferrocenyl series the change of the OH substituent from the para-to the meta-position induced only a slight decrease of the RBA value for the alpha form of the estrogen receptor (4.6% versus 3.6% for the monophenols, 9.6% versus 5.4% for the diphenols). On the contrary, the presence of an m-OH substituent dramatically decreased the affinity of the complexes for the beta form of the oestrogen receptor with an RBA ratio ERβ/ERβ of 21 for the monophenols 2 and 6, and 6.8

for the diphenols 3 and 7. Quite surprisingly, the diphenyl complex 5, i.e. the compound with no OH, has a non-zero RBA value of 0.9%. A similar RBA value (0.8%) was previously reported for the corresponding organic molecule, 1,1,2-triphenylbut-1-ene. 32 Finally, the RBA values found for the di-ferrocenyl derivatives are quite low for both forms of the oestrogen receptor. This is probably due to the presence of the two ferrocenyl units which are bulkier than a phenyl substituent. Regarding the log P o/w values, the change of the OH group from the para-to the meta-position plays no role, and, as expected, the complexes with no hydroxyl groups or with two ferrocenyl substituents are more hydrophobic with log P o/w values around 6.4.

Effect of the compounds on the growth of breast cancer cells.

The effect of these complexes at a concentration of 1×10 -6 M was studied on hormoneindependent (MDA-MB-231) and hormone-dependent (MCF-7) breast cancer cells and the results are displayed in Fig. 2. The antiproliferative effect observed on the hormoneindependent breast cancer cells can be attributed only to a cytotoxic effect potentially induced by the ferrocenyl unit. On these cells, complexes 6, 8, and 9 show the lowest antiproliferative effects with IC 50 values between 2.7 and 3.5 µM (Table 2); compounds 2 and 7 are more cytotoxic (IC 50 values around 1 µM); while 3 with an IC 50 of 0.44 µM is the most cytotoxic of the series. These results show that the repositioning of one OH group from the para-to the meta-position significantly lowers the cytotoxicity of the complex (ratio of IC 50 values of 6

versus 2 and 7 versus 3 being respectively 2.4 and 2.3). The presence of a second ferrocenyl unit decreases the cytotoxicity of the complexes (IC 50 of 2.8 µM for 8 and 1.13 µM for 2; 3.5 µM for 9 and 2.7 µM for 6); here also the complex with a m-OH group is less cytotoxic than the one with a p-OH. Finally, as expected, estradiol has no effect on these cells with no ERα.

On the contrary, the effect observed on hormone-dependent breast cancer cells is the result of the estrogenic (proliferative) effect expected for these compounds that all show an affinity for the alpha form of the ER minus its cytotoxic component observed on the hormoneindependent cancer cells. At a concentration of 1 µM and in a medium without phenol red, which is best suited to the expression of the estrogenic component of a molecule, only the most cytotoxic complexes 2 and 3 are able to reverse the strong estrogenic effect shown by estradiol. As expected complexes 5 and 6 which are the less cytotoxic on the MDA-MB-231 cells show a clear proliferative effect.

Experimental

General remarks

The synthesis of all compounds was performed under an argon atmosphere, using standard Schlenk techniques. Anhydrous THF was obtained by distillation from sodiumbenzophenone. Thin layer chromatography was performed on silica gel 60 GF254. Infrared spectra were obtained on an IRFT BOMEM Michelson-100 spectrometer equipped with a DTGS detector as a KBr plate. was added. The mixture was stirred overnight at room temperature. The solution was slowly poured into a mixture of water and ice and extracted with ethyl acetate. The combined organic layers were washed with water, dried over magnesium sulfate and concentrated under reduced pressure. The mixture was chromatographed on a silica gel column with dichloromethane as the eluent to yield the phenol compounds which were recrystallized from an ether-pentane solution.

Formation of compound 12, anisoylferrocene. This compound is described in the literature and was synthesized via the demethylation of 10 using AlCl 3 (yield 32%) in place of BBr 3 , the latter being used in our case. Compound 12 was obtained as a dark orange solid (2.24 g, 74% yield). The characteristics were identical to those described in the literature (mp 186 °C, 186-188 °C, 26 General procedure for formation of compounds 5, 6, 7, 8, 9.

Titanium tetrachloride (3.6 mL, 33 mmol) was added dropwise to a suspension of zinc powder (4 g, 61 mmol) in 80 mL of dry THF at 0 °C. The mixture was heated at reflux for 2 h. A second solution was prepared by dissolving propionyl ferrocene (2.42 g, 10 mmol) and the corresponding ketones (10 mmol) in 50 ml of dry THF. This latter solution was added dropwise to the first solution and then the reflux was continued for 2 h. After cooling to room temperature, the mixture was stirred with water and dichloromethane. The mixture was acidified with diluted hydrochloric acid until the dark colour disappeared and was decanted.

The aqueous layer was extracted with dichloromethane and the combination of organic layers was dried over magnesium sulfate. After concentration under reduced pressure, the crude product was chromatographed on a silica gel column with dichloromethane or a solution of dichloromethane-acetone 95 : 5 as the eluent (5 excepted). For the biological tests, each products were re-purified on semi-preparative HPLC with acetonitrile-water or acetonitrile as the eluent to give pure 5, 6, 7, 8, and 9. The isomers, if any, were either inseparable, or partially separated but remixed in the same flask (because of rapid isomerisation) before evaporation of acetonitrile under reduced pressure. The mixture was extracted with dichloromethane and water, decanted, dried over magnesium sulfate, and concentrated under reduced pressure. The mixture of isomers, if any, was recrystallized in the appropriate solvent.

2-Ferrocenyl-1,1-di-phenyl-but-1-ene, 5.

The reaction was performed with 1.82 g (10 mmol) of commercially available benzophenone. The crude product was directly purified on HPLC with acetonitrile as eluent to yield 5 as an orange solid (0.96 g, 25% yield). The compound was recrystallized from acetonitrile. Mp: 160 °C. 1

2-Ferrocenyl-1-(3-hydroxyphenyl)-1-(4-hydroxyphenyl)-but-1-ene, 7.

The reaction was performed with 2.14 g (10 mmol) of the known 3,4'-dihydroxybenzophenone. 29 The crude product was chromatographed with a solution of CH 2 Cl 2 -acetone 95 : 5 as eluent. The compound was purified on HPLC with acetonitrile-water 70 : 30 as eluent to yield 7 as an orange solid (2.43 g, 58% yield). The mixture of isomers was recrystallized from ethanol.

1,2-di-Ferrocenyl-1-(4-hydroxyphenyl)-but-1-ene, 8.

The reaction was performed with 3.06 g (10 mmol) of the known compound 12. 26 The crude product was chromatographed with CH 2 Cl 2 as eluent. The compound was purified on HPLC with acetonitrile-water 90 : 10 as eluent to yield 8 as an orange solid (1.98 g, 39% yield). The mixture of isomers was recrystallized from an ether-pentane solution. 1 H NMR (300 MHz, CDCl 3 )  1.12 and 1.38 (t, J = 7.4 Hz, 3 H, CH 3 ), 2.32 and 2.81 (q, J = 7.4 Hz, 2 H, CH 2 ), 3.65 and 3.69 (t, J = 1.9 Hz, 2

Biochemical experiments

Materials

Culture conditions

Cells were maintained in monolayer in DMEM with phenol red supplemented with 8-9% fetal calf serum and 2 mM glutamine at 37 °C in a 5% CO 2 air humidified incubator. For proliferation assays, cells were plated in 24-well sterile plates at a density of 1.1 × 10 4 cells for MDA-MB-231 and of 3 × 10 4 cells for MCF-7 in 1 mL of DMEM medium without phenol red, supplemented with 10% decomplemented and hormone-depleted fetal calf serum and 2 mM glutamine and incubated. The following day (D0), 1 ml of the same medium containing the compounds to be tested was added to the plates (final volumes of DMSO: 0.1%; 4 wells for each conditions). After 3 days (D3) the incubation medium was removed and fresh medium containing the compounds was added. After 5 days (D5) the total protein content of the plate was analyzed by methylene blue staining as follows. Cell monolayers were fixed for 1 h in methanol, stained for 1 h with methylene blue (1 mg mL -1 ) in PBS, then washed thoroughly with water. One ml of HCl (0.1 M) was then added and the absorbance of each well was measured at 620 nm with a Biorad spectrophotometer. The results are expressed as the percentage of proteins versus the control.

Conclusions

Our results show that the presence of a p-phenol has a significant influence on both the electrochemistry and on the biological efficacy of these ferrocenyl phenol compounds, supporting our hypothesis that oxidative activation to a QM structure may be a key to their biological activity. Compounds 5 and 6 which do not possess a p-phenol also did not show intramolecular electron transfer and rearrangement to a QM structure in the electrochemical experiments, as expected. Compound 5, lacking any kind of phenolic group, showed no appreciable cytotoxicity, while 6, possessing only a m-phenol, was much less efficacious than its para-substituted analogue 2. Similarly, compound 7, carrying one m-and one p-phenol did not show nearly as strong an activity as that of 3, which possesses two p-phenols. In fact, the biological activity of 7 is much closer to that of 2 (with IC 50 values of 1.13 vs. 1.03 µM, respectively) than that of 3 (0.44 µM).

The diferrocenyl compounds 8 and 9 follow the same trend, with higher biological activity observed for p-phenol 8, than m-phenol 9. As expected, electron transfer from the phenol to the ferricenium moiety is only electrochemically observed for compound 8. However, the activity of these compounds cannot be solely attributed to their electronic and structural rearrangements. There also exists the steric effect of the additional ferrocene group, which has a negative effect on the biological activity of 8 and 9. Whether carrying a p-or m-phenol, the diferrocenyl compounds are less efficacious than their mono-ferrocenyl analogues 2 and 6.

Although each of the compounds carrying a p-phenol behaves similarly from an electrochemical perspective, the steric effect suggests that the QMs are formed, or react, via one or several specific biological pathways. For example, QMs are alkylating agents which can react with O-, N-, or S-nucleophiles, especially glutathione, via a Michael 1,4-addition. 35 The depletion of glutathione can then lead to the alkylation of proteins by an excess of quinone. Alternatively, quinonoids are known to undergo redox cycling in cells resulting in the production of ROS, via such enzymes as NADPH:cytochrome P450 reductase, NADPH:cytochrome b 5 reductase, NADPH:ubiquinone oxidoreductase, and NAD(P)H:quinone oxidoreductase, among others, 36 and iron-containing compounds are known to produce ROS via the Fenton reaction and H 2 O 2 . We are currently investigating the ROS production of these types of compounds, and we hope that future use of specific ROS probes and enzyme-inhibitors will allow us to begin to unravel the metabolic fate of this new class of cytotoxic ferrocenyl phenolic compounds.

Fig. 1

 1 Fig. 1 Cyclic voltammograms of 6 (a), 9 (b), 7 (c), and 8 (d) in 0.1 M Bu 4 NBF 4 /MeOH in absence (solid line) and presence (dashed line) of pyridine in a 1 : 6 volume ratio. Scan rate

Stock solutions ( 1 ×

 1 10 -3 M) of the ferrocenyl complexes 2, 3, and 5-9 to be tested were prepared in DMSO and were kept at 4 °C in the dark; under these conditions they are stable at least two months. Serial dilutions in DMSO were prepared just prior to use. A stock solution (1 × 10 -3 M) of 17β-estradiol was prepared in ethanol. Dulbecco's modified eagle medium (DMEM) was purchased from Gibco BRL, fetal calf serum from Dutscher, Brumath, France, glutamine, estradiol and protamine sulfate were from Sigma. MCF-7 and MDA-MB-231 cells were from the Human Tumor Cell Bank. Sheep uteri weighing approximately 7 g were obtained from the slaughterhouse at Mantes-la-Jolie, France. They were immediately frozen and kept in liquid nitrogen prior to use.Determination of the Relative Binding Affinity (RBA) of the compounds for ERα andERβ. RBA values were measured on ERα from lamb uterine cytosol and on ERβ purchased in solution from Pan Vera (Madison, WI, USA). Sheep uterine cytosol prepared in buffer A (0.05 M Tris-HCL, 0.25 M sucrose, 0.1% β-mercaptoethanol, pH 7.4 at 25 •C) as described previously,14 was used as a source of ERα. For ERβ, 10 µl of the solution containing 3500 pmol ml -1 were added to 16 ml of buffer B (10% glycerol, 50 mM Bis-Tris-Propane pH = 9, 400 mM KCl, 2 mM DTT, 1 mM EDTA, 0.1% BSA) in a silanized flask. Aliquots (200 µl) of ERα in glass tubes or ERβ in polypropylene tubes were incubated for 3 h at 0 °C with [6,7-3 H]-estradiol (2 × 10 -9 M, specific activity 1.62 TBq mmol -1 , NEN Life Science, Boston MA) in the presence of nine concentrations of the ferrocenyl complexes 2, 3, and 5-9 to be tested (between 6 × 10 -7 M and 6 × 10 -9 M for the complexes with RBA values higher than 5% and between 6 × 10 -6 M and 6 × 10 -8 M for the compounds with RBA values lower than 5%) or of 17β-estradiol (between 8 × 10 -8 M and 7.5 × 10 -10 M). At the end of the incubation period, the fractions of [ 3 H]-estradiol bound to the estrogen receptors (Y values) were precipitated by addition of a 200 µl of a cold solution of protamine sulfate (1 mg mL -1 in water). After a 10 min period of incubation at 4 °C, the precipitates were recovered by filtration on 25 mm circle glass microfibre filters (GF/C) using a Millipore 12 well filtration ramp. The filters were rinsed twice with cold phosphate buffer and then transferred in 20 ml plastic vials. After addition of 5 ml of scintillation liquid (BCS Amersham) the radioactivity of each fraction was counted in a Packard tri-carb 2100TR liquid scintillation analyzer. The concentration of unlabeled steroid required to displace 50% of the bound [ 3 H]-estradiol was calculated for 17βestradiol and for each complex by plotting the logit values of Y (logit Y = ln(Y/100 -Y) versus the mass of the competing complex. The RBA was calculated as follows: RBA of a compound = concentration of estradiol required to displace 50% of [ 3 H]-estradiol × 100/concentration of the compound required to displace 50% of [ 3 H]-estradiol. The RBA value of estradiol is by definition equal to 100%. Measurement of octanol/water partition coefficient (log P o/w ) of the compounds. The log P o/w values of the compounds were determined by reverse-phase HPLC on a C-8 column (Nucleosil 5 C8, from Macherey Nagel, France) according to the method previously described by Minick 33 and Pomper. 34 Measurement of the chromatographic capacity factors (kN) for each compounds was performed at various concentrations in the range 85%-60% methanol (containing 0.25% octanol) and an aqueous phase consisting of 0.15% n-decylamine in 0.02 M MOPS (3-morpholinopropanesulfonic acid) buffer pH 7.4 (prepared in 1-octanol-saturated water). These capacity factors (kN) are extrapolated to 100% of the aqueous component given the value of k'w. log P o/w (y) is then obtained by the formula: y = 0.13418 + 0.98452 × log k'w.

Table 1

 1 Formal oxidation and redox potentials for compounds 2, 5-9 vs. SCE. a

		Solvent	E p,o (Fc) E p,o (other)	E•' (Fc/Fc + ) c	Electron
					transfer?
	2 b	MeOH	0.397(2)	0.357(2)	Yes
			0.97(1) d
		MeOH-py	0.423(4) d,e	d
			0.510(3) c
	5 b	MeOH	0.421(3)	0.380(3)	No
		MeOH-py	0.442(3)	0.400(3)
	6	MeOH	0.403(3)	0.365(3)	No
		MeOH-py	0.408(3)	0.375(3)
	7	MeOH	0.418(3)	0.386(3)	Yes
			1.01(1) d
		MeOH-py	0.43(1) d,e	d
			0.528(3) d
	8	MeOH	0.334(3) 0.504(3)	0.300(3) 0.462(3) d	Yes
		MeOH-py	0.34(1) d,e	0.649(3)
			0.444(3) d
			0.703(3)
	9	MeOH	0.344(3) 0.503(3)	0.311(3) 0.470(3)	Yes
		MeOH-py	0.346(3) 0.523(3)	0.313(3) 0.488(3)
	a Scan rate = 0.1 V s -1 . b Values reported from reference 18. c E•'/V is the average of the
	anodic and cathodic peak potentials.

d

Irreversible. e Shoulder.

  1 H and13 C NMR spectra were recorded on a 300 MHz Bruker spectrometer. Mass spectrometry was performed with a Nermag R 10-10C spectrometer. High resolution mass spectrometry (HRMS) was performed on a JEOL MS 700 instrument.Melting points were measured with a Kofler device. Elemental analyses were performed by the microanalysis service of CNRS at Gif sur Yvette. The semi-preparative HPLC separations were performed on a Shimadzu apparatus with a Kromasil C18 column (length of 25 cm, diameter of 2 cm, particles size of 10 µm).Cyclic voltammograms were obtained utilizing an Autolab PG-Stat20 potentiostat, driven by GPES software (General Purpose Electrochemical System, Version 4.8, EcoChemie B.V., Utrecht, the Netherlands), a platinum wire counter electrode, a 500 µM platinum disc working electrode, and an aqueous standard calomel reference electrode. Analyte solutions were 1-2 mM in MeOH with 0.1 M Bu 4 NBF 4 supporting electrolyte. Solvent ratios were 6 : 1 MeOH : py, except for compound 5, where the ratio was 3 : 1 MeOH : py. Unfortunately, the insolubility of these compounds in water prevented the preparation of aqueous samples at suitable concentrations. Solvents were spectrometric grade and used as received. The synthesis is described in reference 26.

	Synthesis and characterization
	Compound 10. 0, 1.0 Hz, 1
	H, H arom ). 13 C NMR (CDCl 3 )  55.5 (OCH 3 ), 70.2 (5 CH Cp), 71.5 (2 CH C 5 H 4 ), 72.6 (2 CH
	C 5 H 4 ), 78.1 (C C 5 H 4 ), 113.1 (CH arom ), 117.6 (CH arom ), 120.6 (CH arom ), 129.2 (CH arom ), 141.1
	(C), 159.5 (C), 198.9 (CO). IR: 3098, 2938, 2836 (CH 3 ), 1638 (CO) cm -1 . MS (CI, NH 3 ) m/z:
	321 [MH] +• .
	Demethylation of compounds 10 and 11. Compound 10 or 11 (3.2 g, 10 mmol), was
	dissolved in dry dichloromethane (60 mL) at 0 °C and boron tribromide (2.84 mL, 30 mmol)

Compound 11. Ferrocene (10.2 g, 54.8 mmol) was dissolved in dry dichloromethane (400 mL). Aluminium trichloride (7.30 g, 54.8 mmol) was added in portions over 15 min. Then, manisoyl chloride (7.79 g, 45.7 mmol) was added slowly over 20 min and the stirring was continued overnight. The solution was slowly poured into a mixture of water and ice and decanted. The aqueous layer was extracted with dichloromethane and the combined organic layers were washed with water, dried over magnesium sulfate and concentrated under reduced pressure. The mixture was chromatographed on a silica gel column with a solution of dichloromethane-petroleum ether 50 : 50 as the eluent. The pure product 11 was obtained as an oil (11.92 g, 82%).

1

H NMR (300 MHz, CDCl 3 )  3.81 (s, 3 H, OCH 3 ), 4.14 (s, 5 H, Cp), 4.51 (t, J = 1.9 Hz, 2 H, C 5 H 4 ), 4.85 (t, J = 1.9 Hz, 2H, C 5 H 4 ), 7.02 (ddd, J = 8.0, 3.6, 1.0 Hz, 1 H, H arom ), 7.30 (t, J = 8.0 Hz, 1 H, H arom ), 7.35 (m, 1 H, H arom ), 7.42 (dt, J = 8.

Acknowledgements

We thank A. Cordaville and M. A. Plamont for technical assistance, and the Agence Nationale de la Recherche for financial support (No. ANR-06-BLAN-0384-01, "FerVect". E.

A. H. thanks the NSF (grant no. 0302042) for financial support.


Frédéric Alexandre
email: frederic.alexandre@inria.fr

Different learning architectures contribute to value prediction in human cognition

Keywords: Value prediction, Bio-inspiration, Competiton, Cognitive architecture

Pavlovian conditioning plays a fundamental role in our cognitive architecture, by its capacity to bind values to stimuli. Due to the multifarious characteristics of this learning mode, many approaches in machine learning have been proposed to implement it in artificial intelligence models. Considering the complementary properties of these models and inspired by biological evidences, we propose not to select the best model but rather to combine them, thereby forming a cognitive architecture. From a functional point of view, we report the good properties and performances of this architecture. From a methodological point of view, this work highlights the interest of defining a cognitive function at the algorithmic level, binding its general properties to its implementation details. It is also proposed that it is a fruitful approach to decipher and organize many information extracted by modern approaches in neuroscience, towards the definition of a global cognitive architecture.

Introduction

Having particularly in mind the brain performing a cognitive function, D. Marr [START_REF] Marr | Vision: A Computational Investigation into the Human Representation and Processing of Visual Information[END_REF] has argued that, to be fully understood, an information processing system must be examined at three different levels of description. The function itself can be globally described at the computational level. The representations and the processes used to implement the function are described at the algorithmic level. The way of computing them are described at the implementational level. This later level, refering to the brain, corresponds to the brain circuitry but Z. Pylyshyn [START_REF] Pylyshyn | Computation and Cognition[END_REF] has proposed a similar tri-level description in a purely cognitive purpose and has simply defined the lower level as the functional architecture.

The decomposition is interesting because it makes it clear that not a unique algorithm can carry out the function and not a unique circuit can implement the algorithm. This decomposition is also interesting in the framework of artificial intelligence because it helps structuring steps to build a model. On one side is the cognitive function to model. On the other side are formalisms available (e.g. neuronal computations). In between are probably the more critical and important steps: defining the structures and the sequences of information processing to bridge both sides (e.g. neuronal architecture and learning rules). The classical approach in artificial intelligence is top-down: from a cognitive function to be modeled, an algorithm and its implementation in a certain formalism are proposed. The approach can be also bottom-up, particularly if the problem is set in computational neuroscience: from experimentally observed neuronal circuits and mechanisms, it is questioned which ones are active to emulate the cognitive function generally associated to the neuronal region.

In this paper, having in mind this decomposition, I extend a series of works performed in my team to model pavlovian conditioning from different points of view, in order to propose a new view of this fundamental learning mode, with an impact on cognitive modeling as well as artificial intelligence and machine learning.

Pavlovian conditioning

Pavlovian conditioning [START_REF] Pavlov | Conditioned Reflexes (V.Anrep, trans.)[END_REF] is an elementary learning process, present in most species, by which an animal learns to predict biologically significant stimuli announcing pain or pleasure (called Unconditioned Stimuli, US, e.g. some food) by some other stimuli (called Conditional Stimuli, CS, e.g. a bell ringing before food delivery) that anticipates its venue.

This learning paradigm has been extensively studied in behavioral neuroscience because it is easy to observe (US anticipation is associated to other socalled pavlovian responses like salivation in the appetitive case and freezing in the aversive case) and not so simple as it might appear: Many experiments have demonstrated that pavlovian learning is not a simple associative process measuring the contingencies between the US and the CS [START_REF] Pelley | The role of associative history in models of associative learning: a selective review and a hybrid model[END_REF]. For example, when an association is learnt and extinguished, it is more rapid to be learnt again. Some authors have proposed that during the process, latent causes are extracted [START_REF] Gershman | Context, learning, and extinction[END_REF]. Others have stressed the role of the context [START_REF] Schmajuk | Stimulus configuration, classical conditioning and the hippocampus[END_REF]. As a results, after a century and thousands of publications, many data have been accumulated, many models have been proposed but pavlovian learning remains not fully understood.

It is also important to underline that pavlovian conditioning has a prominent role in cognitive modeling and in the domain of Artificial General Intelligence, because it gives the capacity to link values to objects and events of the environment and this capacity is important in a cognitive-oriented description of intelligent behavior. For example, in a general description of cognition [START_REF] Anderson | An integrated theory of the mind[END_REF], one central module is concerned with declarative memory, providing facts and knowledge about the world, in which goals can be defined and sensorimotor associations can be elaborated, thus forming other central modules. All these modules heavily rely on the elementary capacity to predict aversive and appetitive values for CS from the innate detection of US. Some cues to elaborate the CS might come from declarative memory and the US correspond to the main behavioral goals for most animals. This capacity is then extended to conditioned reinforcers [START_REF] Belova | Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala[END_REF], corresponding to intermediate goals like, in humans, tools or money, and at the basis of categorization [START_REF] Ashby | A neuropsychological theory of multiple systems in category learning[END_REF] and of judgments [START_REF]Human contingency judgments : Rule based or associative[END_REF] used in strategies and more generally in decision making and planning.

In this paper, we propose to study pavlovian conditioning from two different points of view. On the one hand, from a top-down machine learning point of view, we will wonder how it is possible to associate the CS and the US. In other words, the question is to associate the perception of external stimuli like the CS, also called exteroception, to the internal perception of pain or pleasure, also called interoception, as it is the case with the US. We summarize some more or less classical strategies at the algorithmic level in the next section.

On the other hand in computational neuroscience, we have modeled some neuronal structures and circuits that are reported to play a fundamental role in pavlovian conditioning. This particularly concerns the amygdala, a primitive neuronal structure in the medial temporal lobe and its main afferences. We summarize in the following the main results of our study in a bottom-up way and explain how we integrate these pieces of information towards a more global understanding of pavlovian conditioning, at the algorithmic level and how this can be more generally profitable for cognitive modeling.

Modeling pavlovian conditioning in Machine Learning

In the domain of machine learning, pavlovian conditioning has often been associated to supervised learning. In its simplest aspect, it corresponds, when a stimulus is perceived, to predict the occurrence of the US and the learning can consequently be supervised by the fact that, some milliseconds later, the US is perceived or not. Even if some binary models have been proposed, predicting the magnitude of the US (for example the size of the reward or the intensity of a painful electric shock) is the main purpose of the learning rule. In that aspect, it is interesting to note that one of the main learning rules proposed for pavlovian conditioning by Rescorla and Wagner [START_REF] Rescorla | A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement[END_REF] is similar to the main learning rule setting the bases of supervised learning in neural networks, proposed by Rosenblatt for the Perceptron [START_REF] Rosenblatt | The perceptron: a probabilistic model for information storage and organization in the brain[END_REF]. They both set the focus on a multiplicative term, called error for the perceptron and surprise for pavlovian learning, corresponding to the difference between the predicted and the computed US. This term (null if there is no error, thus ensuring the stability of the rule) also explains that there is no learning when the US is already predicted, including by other CS. Importantly, this has been observed in experimental situations in a famous paradigm called blocking, increasing the interest for this rule.

Nevertheless, for the same kind of structural reasons as the perceptron, the Rescorla-Wagner learning rule has some limitations, particularly related to the complexity of the decision that can be taken in the input space. Concerning the perceptron, it has been shown that its learning rule guarantees the convergence towards a unique minimum value of error, but this results in a good quality of the learnt solution only in case of linear separability in the input space [START_REF] Minsky | Perceptrons: An Introduction to Computational Geometry[END_REF]. Concerning pavlovian conditioning, it has been observed that some mammals can answer to some combination of two variables (x and y) called positive (x AND y) et negative (x XOR y) patterning, differently than the corresponding combined effect of each variable, raising the idea to create configural representation (representing the co-occurrence of x and y), able to provide a different effect. This principle of configural learning proposed in [START_REF] Schmajuk | Stimulus configuration, classical conditioning and the hippocampus[END_REF] resulted in that paper in the same idea of adding hidden units as in the multilayer perceptron, to propose a model of pavlovian conditioning able to perform such complex decision.

The idea of performing error-driven learning in multilayered networks has been made possible in Machine Learning by the back-propagation learning algorithm which allows to make emerge internal representations in the hidden layers. Even if some biologically plausible learning rules have also been proposed [START_REF] O'reilly | Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain[END_REF], the main visible effect of this algorithm is rather today in the huge developments about deep learning [START_REF] Lecun | Deep learning[END_REF], which make the learning of any complex association affordable by a deep network. Nevertheless, as it can be seen in the difficult steps of parameter tuning in these networks, there is no proof of convergence in this case and, instead of looking for the best solution, mastering this technology rather corresponds to obtaining a satisfactory solution in a satisfactory time length.

It is also often forgotten that the only strong theoretical result about layered network is the fact that removing one layer can require to exponentionally increase the size of the previous layer [START_REF] Hastad | On the power of small-depth threshold circuits[END_REF]. To tell it differently, it is possible to approximate any separation with a unique hidden layer and a random fixed connection from the input to the hidden layer (but possibly with a very large hidden layer). This has for example been experimentally studied in [START_REF] Ba | Do Deep Nets Really Need to be Deep?[END_REF], showing that the function learnt by complex deep networks can be efficiently approximated by a simple network with a unique (and large) hidden layer. It is only supposed that the projection to the hidden layer is likely to make emerge the useful internal representations, to be linearly separated for the targeted decision. A strategy when the corresponding knowledge is available, is to directly send the pertinent information to the hidden layer, else the solution is to use random projections and the possibility to create pertinent information becomes very likely when the layer is very large. In summary, the main advantage of deep networks is to have what is called a compact representation (because requiring less neurons), when the representation is elaborated hierarchically, as compared to a flat representation, but this is at the price of a less stable representation.

In a completly different view, it is often neglected in Machine Learning that associative memories can provide a kind of supervised learning. Associative memories are these kinds of recurrent networks, like the Hopfield network [START_REF] Hopfield | Neural networks and physical systems with emergent collective computational abilities[END_REF], that are able to learn "by heart" some prototypes as stable states and, in a recall phase, to consider a new input and stabilize on the closest state stored beforehand. This process is called auto-association, since the network is shown to be able to associate a pattern (the new input) with itself (seen as the closest prototype learnt by heart before). This is generally considered useful in Machine Learning when the new input has been corrupted by noise and when the recall is a way to denoise this input.

This process can easily be extended to hetero-association. In this case, just consider a series of i vectors V i as the concatenation of subvectors X i and Y i and consider that these vectors are learned as prototypes in an associative memory. The principle of this learning process is that a noisy V j will be reconstructed to the closest V i learned before. If, in the vector V i , the noise corresponds to set Y i to zero and let X i unchanged, then the recall will reconstruct the missing part Y i and will consequently act in hetero-association, which can be seen as a kind of supervised learning, even if in this case there is no generalization and the network converges towards a pattern already stored before.

Several other characteristics of this learning process must be underlined. On the one hand, it is interesting because it is very rapid and presenting only one time a prototype is sufficient to memorize it (as opposed to layered perceptrons learning slowly). On the other hand, it can be considered weak because it is subject to interference (close prototypes can create spurious states combining these prototypes) and saturation (learning too many prototypes can lead to a catastrophic forgetting) [START_REF] French | Semi-distributed representations and catastrophic forgetting in connectionist networks[END_REF].

Modeling pavlovian conditioning in Computational Neuroscience

Concerning the implementation of pavlovian conditioning in the brain, the amygdala is a neural structure generally considered to play a central role in this process [START_REF] Ledoux | The amygdala[END_REF]. The structure is composed of a set of nuclei, including the basolateral nucleus for sensory inputs and the central nucleus expressing pavlovian motor responses. Observing direct plastic connections between these sensory and motor nuclei [START_REF] Ciocchi | Encoding of conditioned fear in central amygdala inhibitory circuits[END_REF] is a good argument to propose a shallow layered architecture to model this network. Particularly, the lateral part of the basolateral nucleus has been reported to represent elementary CS, selected in its afferences from the thalamus and primary sensory cortex [START_REF] Ledoux | The amygdala[END_REF], whereas the basal part receives afferences from more integrated regions like associative sensory cortex and hippocampus [START_REF] Goosens | Contextual and Auditory Fear Conditioning are Mediated by the Lateral, Basal, and Central Amygdaloid Nuclei in Rats[END_REF].

A functional analogy with a model like the simple perceptron can be also evoked for the amgdala, since signals corresponding to error of prediction have been observed in the network [START_REF] Li | The conditions that promote fear learning: Prediction error and Pavlovian fear conditioning[END_REF]. Such error signals are known to be carried by neuromodulators, which can act at the learning and functioning levels in the amygdala but also in other cerebral structures. This is the case for acetylcholine, considered to tag errors due to stochasticity in the environment [START_REF] Yu | Uncertainty, Neuromodulation, and Attention[END_REF] and acting to balance the activity between the lateral and basal sensory nuclei [START_REF] Calandreau | Extracellular hippocampal acetylcholine level controls amygdala function and promotes adaptive conditioned emotional response[END_REF], but also to trigger memorization in the hippocampus [START_REF] Hasselmo | The role of acetylcholine in learning and memory[END_REF] and shift selective attention in the higher regions of the sensory cortex [START_REF] Pauli | Attentional control of associative learning-a possible role of the central cholinergic system[END_REF]. To better understand the putative role of neuromodulation on these structures, let us first evoke the participation of the structures to pavlovian conditioning.

The basal nucleus of the amygdala receives inputs from the associative sensory cortex [START_REF] Ledoux | The amygdala[END_REF]. In the visual case for example and in a large region going from the primary visual cortex to the inferotemporal cortex, this path has been described as a layered architecture, extracting more and more abstract cues to create categories for visual recognition [START_REF] Thorpe | Seeking Categories in the Brain[END_REF]. Based on biological observations, it has also been proposed that the release of acetylcholine plays a prominent role for transmitting an error signal, as an attentional signal fed back throughout the architecture and some models have been proposed accordingly, as a biological implementation of a kind of error backpropagation [START_REF] Roelfsema | Attention-gated reinforcement learning of internal representations for classification[END_REF].

The hippocampus is also known to be a major input of the basal nucleus of the amygdala [START_REF] Ledoux | The amygdala[END_REF], generally considered to provide information about the context of the task, that can play an important role in discrimination. This is consistent with the fact that the hippocampus is known to perform arbitrary binding of its inputs, collected in the entorhinal cortex from main sensory regions of the cortex, and to learn these associations through a recurrent network to form a so-called episodic memory [START_REF] Squire | Declarative and nondeclarative memory: multiple brain systems supporting learning and memory[END_REF]. Recently, we have also proposed that the hippocampus might in fact associate interoception and exteroception in its inputs, making it a perfect candidate for learning the vector CS-US in heteroassociation [START_REF] Kassab | Integration of exteroceptive and interoceptive information within the hippocampus: a computational study[END_REF] and recalling US when the CS is presented alone. In a more computational study [START_REF] Kassab | A Modular Network Architecture Resolving Memory Interference through Inhibition[END_REF], we have also shown that this organization is in fact beneficial to resist to interference.

The complementary roles in learning of the cortex and the hippocampus have already been theoretized [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF]. Whereas the cortex is able to learn slowly cues important for discrimination and generalization, the hippocampus has a different learning mode, allowing to learn quickly, by heart, some specific configuration to be remembered. In the case of the amygdala, this should correspond either to extract cues in the cortex to learn some features of the CS or, when this results in an error of prediction, to learn by heart the global sensory configuration in the hippocampus and to store it as a special case to be remembered. It is consequently interesting to remind now that acetylcholine release triggers storing in the hippocampus.

With the aim of integrating all these elements in a unique picture, we begin to report preliminary work we did to model pavlovian conditioning in the amygdala [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF]. Our main goal was to evaluate which of the many standard rules proposed for pavlovian conditioning [START_REF] Pelley | The role of associative history in models of associative learning: a selective review and a hybrid model[END_REF] was the most faithful, as compared to biological data. What we proposed, in short, is that several populations with different behavior, recently reported in the basolateral nucleus [START_REF] Herry | Switching on and off fear by distinct neuronal circuits[END_REF], receive different kinds of cues, each kind reported to be important in some classic rule of pavlovian learning. In addition, we proposed that, consistent with the fact that several populations with different excitatory and inhibitory roles have also been recently described in the central nucleus of the amygdala [START_REF] Ciocchi | Encoding of conditioned fear in central amygdala inhibitory circuits[END_REF], the behavior actually expressed by the amygdala is in fact the result of the competition between different lines of elementary sensorimotor association in the amygdala. In conclusion, contrarily to rules from experimental psychology, becoming more and more complex as they want to model pavlovian conditioning, taking more and more facts into account, we propose in [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF] that the same complexity of behavior can be expressed by the simple competition between different lines of association, underlying the role of neuromodulation to favor some lines over others, depending on the context or the recent history of behavior [START_REF] Alexandre | Feature, Configuration, History : a bioinspired framework for information representation in neural networks[END_REF].

One step further, what we propose in this paper is not to forget that cues arriving in the sensory input of the amygdala are themselves learnt in other regions. This is consequently a wider view, proposing to model pavlovian conditioning in a network of neuronal structures and to consider that critical events in the network can trigger important learning or functioning steps in different locations throughout the network. In this view, as it was already proposed in [START_REF] Carrere | A pavlovian model of the amygdala and its influence within the medial temporal lobe[END_REF], the amygdala seen as a simple perceptron decides which US to predict from a variety of inputs received in the basolateral nucleus. These inputs can correspond to simple elementary CS coming from the thalamus or to more elaborated hints or configurations learnt in other cortical areas or in the hippocampus. Possibly, a release of acetylcholine will give a stronger weight to complex inputs, meaning that the environment is too noisy to rely on simple cues.

This mode of learning is incremental in the sense that, at the beginning, it can be supposed that hardly no complex cues have been extracted in the hippocampus or in the highest levels of the cortex. Errors of prediction will result in neuromodulation triggering learning in the hippocampus of the specific cases poorly considered before. Later on, if the same case occurs again, it will trigger recall in the hippocampus that will reconstruct the corresponding US and send it back to the amygdala. Concerning learning in the layered architecture of the cortex and in addition to bio-plausible mechanisms proposed in [START_REF] Roelfsema | Attention-gated reinforcement learning of internal representations for classification[END_REF] and mentioned above, the phenomenon of consolidation must be also evoked. In this process, also reported in neuroscience and described in [START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF], the hippocampus can act as a supervisor to the cortex and send it some examples previously learned by heart in episodic memory, in order to transfer it to semantic memory in the cortex. As far as we know, no stable learning rules have been proposed so far to implement this important learning process, but this will be very important to consider in the future because it might allow to remove examples in the hippocampus and prevent it from saturation.

Discussion

The capacity for an intelligent agent to associate a sensory state (taken in the widest sense, including more or less abstract multimodal and contextual information) to a value (particularly representing its possible contribution to achieving the current goal of the agent) is very desirable to define in a cognitive architecture. It can be paralleled, in natural cognition, with pavlovian conditioning linking a state (more or less abstract stimulus, context) to the unconditioned stimulus it predicts, annoucing a pain or a pleasure. In the living, this learning has been shown to contribute both to declarative and implicit memories [START_REF] Squire | Declarative and nondeclarative memory: multiple brain systems supporting learning and memory[END_REF].

In this paper, we have summarized pavlovian conditioning as the capacity to associate a vector representing the sensory state with a vector representing the anticipated internal state and we have evoked three classical strategies in machine learning to learn this kind of association, each with its respective strengths and weaknesses. Shallow layered networks ensure a quick processing and optimal learning but only for simple direct associations. In contrast, deep layered networks can learn complex associations, which is particularly useful to elaborate internal representations, concepts and abstractions, but at the price of a very long learning that can fall in local minima and must choose pertinent configuations. Of completly different nature is the elaboration of associative memories in recurrent networks. Prototypes can be learned by heart, possibly in one shot but are sensitive to interference and globally an associative memory can be subject to saturation.

From this global analysis, one strategy is to look for the best compromise; another one is to try to benefit from the strengths of all. In the previous section, we have argued to show that the brain exploits this latter strategy. Whereas pavlovian conditioning can be summarized as an implicit sensorimotor learning taking place in the amygdala and associating simple cues to pavlovian responses related to the predicted US, this cannot explain all the observed pavlovian associations and other inputs to the amygdala carrying other kind of information must be also considered. This is particularly the case for inputs coming from the hippocampus and from high cortical areas and learned and processed in these structures with different criteria. What we have also made clear here is that these structures are also dependent on the processing made in the amygdala and particularly on neuromodulation released from its errors of prediction.

This system of combined learning occuring in different sites in parallel but also for common reasons can be interpreted from a phylogenetic point of view. In primitive animals, only simple associations can be learnt [START_REF] Laberge | Evolution of the Amygdala: New Insights from Studies in Amphibians[END_REF]. With evolution, experience from recent episodes and association of multimodal information can be integrated to the pavlovian association.

It is of course also very tempting to extrapolate this interpretation and the related observations to the domains of cognitive modeling and artificial intelligence. Proposing models based on the cooperation of memories is not new in these domains [START_REF] Alexandre | Biological Inspiration for Multiple Memories Implementation and Cooperation[END_REF][START_REF] Mcclelland | Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory[END_REF] and relates to the design of ensemble or multi-method approaches. This must be more and more seriously considered, since new technologies in experimental neuroscience allow to observe that most evolved cognitive functions are in fact elaborated from an interplay between neuronal structures [START_REF] Paz | Measuring correlations and interactions among four simultaneously recorded brain regions during learning[END_REF]. In the present case, this gives an elegant way to introduce specific cases to an associative process with no loss of generalisation. Using hidden layers to elaborate configural representations is not mandatory, as it has been reminded that a simple layer could perform the same task (at the price of a larger hidden layer), but it is interesting for using internal representations for other purposes. This hierarchy of hidden layers provides a level of abstraction (not obtained by a shallow network) that could be exploited for other cognitive function and particularly for declarative memory and for executive functions.

We have also mentioned up to which point neuromodulation plays an important role in associating the processing and learning phases in the neuronal structures, and can be seen as a rare ground of coordination between the otherwise parallel processes. In the future, we plan to better understand their functions [START_REF] Alexandre | Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures[END_REF] and we have already proposed some mechanisms to decipher some temporal aspects of pavlovian conditioning [START_REF] Kaushik | A biologically inspired neuronal model of reward prediction error computation[END_REF] and to go beyond, towards operant conditioning and associated high level cognitive functions [START_REF] Carrere | Modeling the sensory roles of noradrenaline in action selection[END_REF].

To conclude on a more methodological point of view and coming back to Marr's levels of description [START_REF] Marr | Vision: A Computational Investigation into the Human Representation and Processing of Visual Information[END_REF], we have presented some top-down approaches in Machine Learning and bottom-up approaches in computational neuroscience, forming the state of the art on the current understanding of pavlovian conditioning. We demonstrate here that synthetizing these analyses by the algorithmic level is a unique way to propose new interpretations and to revisit cognitive modeling on this topic.