Yann Hardivillier
email: yann.hardivillier@univ-lemans.fr

Françoise Denis

Marie-Véronique Demattei

Paco Bustamante

Marc Laulier

Richard Cosson

François Rabelais

E A 3868

Metal influence on metallothionein synthesis in the hydrothermal vent mussel Bathymodiolus thermophilus

Keywords: Bathymodiolus, Bivalvia, Gene expression, DNA, Hydrothermal, Metabolism, Metal, Metallothionein, Protein, RNA

The present study reports on the metallothionein expression in the hydrothermal vent mussel Bathymodiolus thermophilus. Metallothioneins (MT) are proteins involved in intracellular metal regulation and conserved throughout the animal kingdom. The hydrothermal vent environment presents peculiarities (high levels of sulfides and metals, low pH, anoxia) that may have driven associated species to develop original evolutionary ways to face these extreme living conditions. Mussels were exposed to different metal solutions at the atmospheric pressure. The MT mRNA levels and MT contents were measured in gills and mantles of each exposed mussel. The intracellular metal distribution was estimated in fractions obtained after the centrifugation of tissue homogenates. A few of the tested metals (Ag, Cu, Cd, Hg and Zn) were able to significantly induce MT mRNA levels. Silver was the only one that produced a significant increase of the MT protein level in both mantle and gills. The gills always presented higher MT protein levels than the mantle did, while their MT mRNA levels were similar. Our data show that MT mRNA and MT protein levels do not follow a clear relationship in the gills and mantle of B. thermophilus and we assume that a post-transcriptional control occurs in these mussels.

Introduction

Hydrothermal vents were discovered by the end of the seventies [START_REF] Lonsdale | Clustering of suspension-feeding macrobenthos near abyssal hydrothermal vents at oceanic spreading centers[END_REF]. The magma source and surrounding sea water interactions produce hot fluids that are dispersed in the hydrothermal environment. These fluids are characterized by a low pH, high hydrogen sulfide and metal concentrations as well as anoxia [START_REF] Sarradin | Chemical environment of the hydrothermal mussel communities in the Lucky Strike and Menez Gwen vent fields, Mid Atlantic Ridge[END_REF]Geret et al., 2002). High concentrations of cadmium (Cd) and copper (Cu) may impact biological processes at the endemic fauna level. The reported ratio between Cu concentrations in surface water and hydrothermal environment is about one thousand [START_REF] Douville | The rai w fl (36°14′N MAR): h influence of ultramafic rocks and phase separation on trace metal content in Mid-Atlantic Ridge hydrothermal fluids[END_REF]. Bathymodiolus thermophilus is a common bivalve (Mytilidae) of the Pacific Ocean hydrothermal vent ecosystems [START_REF] Desbruyeres | Variations in deep-sea hydrothermal vent communities on the Mid-Atlantic Ridge near the Azores plateau[END_REF]. These mussels may have developed a high tolerance or some adaptive strategies to live in these peculiar environments [START_REF] Sarradin | Chemical environment of the hydrothermal mussel communities in the Lucky Strike and Menez Gwen vent fields, Mid Atlantic Ridge[END_REF]. Metallothioneins (MTs) are small, cysteine rich, metal storage proteins conserved throughout the animal kingdom and involved in intracellular metal regulation. Their participation in metal detoxication is well established (Geret et al., 1998;[START_REF] Cajaraville | The use of biomarkers to assess the impact of pollution in coastal environments of the Iberian Peninsula: a practical approach[END_REF]. Beside these roles, MTs protect cells against damage induced by alkylating agents, oxygen radicals and ionizing radiations [START_REF] Cherian | Role of metallothioneins in carcinogenesis[END_REF]. Discovered over four decades ago [START_REF] Margoshe | A cadmium protein from equine kidney[END_REF], they are characterized by selectively metal binding affinities which allow the use of MTs as molecular probes (biomarkers) in environmental biomonitoring [START_REF] Hennig | Metal-binding proteins as metal pollution indicators[END_REF]. Not all biological functions of MTs are clearly resolved but the existence of different isoforms may suggest that they have distinct roles. Two major isoforms have been identified in Mytilidae: MT-10 and MT-20 (Mackay et al., 1993;[START_REF] Hardivillier | Do organisms living around hydrothermal vent sites contain specific metallothioneins? The case of the genus Bathymodiolus (Bivalvia, Mytilidae)[END_REF]. The MT-10 isoform appears to be involved in the regulation of free intracellular metal content in Mytilus edulis and is constitutively expressed at a basal level, whereas MT-20 production seems to be specifically induced by Cd [START_REF] Dallinger | Metallothionein research in terrestrial invertebrates: synopsis and perspectives[END_REF][START_REF] Lemoine | Potential use of the levels of the mRNA of a specific metallothionein isoform (MT-20) in mussel (Mytilus edulis) as a biomarker of cadmium contamination[END_REF]. Many studies describe MTs in coastal organisms (e.g. [START_REF] Amiard-Triquet | Metallothioneins in Arctic bivalves[END_REF][START_REF] Barsyte | Cloning and characterization of metallothionein cDNAs in the mussel Mytilus edulis L. digestive gland[END_REF][START_REF] Engelken | cDNA cloning and cadmium-induced expression of metallothionein mRNA in the zebra mussel Dreissena polymorpha[END_REF][START_REF] Cotou | Metallothionein levels in the bivalves Callista chione and Venus verrucosa from two Mediterranean sites[END_REF][START_REF] Serafim | Variation of metallothionein and metal concentrations in the digestive gland of the clam Ruditapes decussatus: sex and seasonal effects[END_REF][START_REF] Moraga | Genetic responses to metal contamination in two clams: Ruditapes decussatus and Ruditapes philippinarum[END_REF][START_REF] Mouneyrac | Partitioning of accumulated trace metals in the talitrid amphipod crustacean Orchestia gammarellus: a cautionary tale on the use of metallothionein-like proteins as biomarkers[END_REF][START_REF] Berthet | Accumulation and soluble binding of cadmium, copper, and zinc in the polychaete Hediste diversicolor from coastal sites with different trace metal bioavailabilities[END_REF] but so far very few studies concern hydrothermal vent organisms (i.e. Cosson and Vivier, 1997;[START_REF] Denis | Characterization and expression of a Bathymodiolus sp. metallothionein gene[END_REF][START_REF] Leignel | Small metallothionein genes in coastal and hydrothermal mussels. Small metallothionein MT-10 genes in coastal and hydrothermal mussels[END_REF][START_REF] Hardivillier | Do organisms living around hydrothermal vent sites contain specific metallothioneins? The case of the genus Bathymodiolus (Bivalvia, Mytilidae)[END_REF]. The aim of our investigation was to study the metallothionein gene expression in B. thermophilus exposed to metals at the atmospheric pressure. A comparison with the reported MT expression in M. edulis [START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF] will permit us to show putative specificities of hydrothermal vent mussels. The study of MT synthesis regulation could help for understanding how hydrothermal mussels adapted to the presence of high concentrations of metals in their environment.

Material and methods  2.1 Experimental exposure of bivalves

Vent mussels (B. thermophilus) were collected during the HOPE 99 hydrothermal cruise (Chief Scientist: F. Lallier) on the EPR at the site named BioVent (N 09°46′-W 104°21′-2516 m depth) w h h m m l "N l " pl y f m the oceanographic research vessel " A l " (IFR M R). Groups of 4 individuals were transferred to 10 L aquaria containing surface sea water enriched with various metals (Cd in the form of CdNO 3 , Cu in the form of CuNO 3 , Hg in the form of HgNO 3 , Zn in the form of ZnNO 3 , each in H 2 SO 4 and Ag in the form of AgNO 3 in HNO 3 ). The molluscs were exposed to mono-metallic solutions (Merck) (Ag,Cd,Cu,Hg and Zn respectively 20;200;40;20 and 1000 μg L -1 ) for 44 h at atmospheric pressure. Controls were maintained in the same conditions in clean seawater. No mortality was recorded during the experiment. At the end of the exposure period, gills and mantle were excised and immediately frozen in liquid nitrogen for further analysis. Specifically in order to compare the DNA sequences of the Pacific mussel with an other related species, Bathymodiolus azoricus were collected at the site named Lucky S k (N 37°13′ W 32°19′ -1700 m depth) in the Azores region (Mid-Atlantic Ridge) during the ATOS cruise (Chief Scientist: P.M. Sarradin-RV " A l "-ROV "V c 3000"-IFREMER) by 2001. As for Pacific mussels, excised tissues were immediately frozen in liquid nitrogen when taken onboard and stored until required.



Metal analysis

Tissue samples were homogenized in 5 volumes of buffer (100 mM Tris, pH 8.1, 10 mM βmercaptoethanol). Homogenates were centrifuged for 30 min at 30000 g at 4 °C. An aliquot of ch p w k p -80 °C l h determination of the total protein content. A second aliquot was submitted to heat-denaturation (95 °C, 15 min) and centrifuged (10000 g, 15 min) in order to separate the thermostable metallothioneins from thermolabile proteins. Newly obtained p w f z -80 °C l h f c f MTs. The content of tested metals (Ag, Cd, Cu, Hg, Zn) in control and corresponding exposed mussel tissues was determined by atomic absorption spectrophotometry (AAS) after acidic digestion with nitric acid in the supernatants and pellets of the first centrifugation Complementary analyses of essential metals (Cu, Zn) were also performed in both fractions resulting from the centrifugation of homogenized non-essential (Ag, Cd, Hg) exposed mussel tissues. Blanks and reference materials (NRCC) were taken through the procedure in the same way as the samples. Levels of metals are given relatively to the wet weight of tissues.

 2.3 MT mRNA analysis

The total RNA was isolated from each sample according to the classic acid-guanidium thiocyanate-phenol-chloroform extraction method [START_REF] Chomczynski | Single-step method of RNA isolation by acid guanidinium thiocyanate-phenol-chloroform extraction[END_REF]. Briefly, tissues were ground to a fine powder using a mortar and a pestle in liquid nitrogen and then homogenized in an acid guanidium thiocyanate buffer. They were mixed thoroughly with 3 M sodium acetate and extracted with phenol/chloroform/iso-amyl alcohol (Genaxys). After centrifugation at 10 000 g for 20 min, the aqueous phase was precipitated with 2-propanol at -20 °C during 30 min. The RNA was finally pelleted by centrifugation and washed twice in 70% ethanol before being dissolved in diethyl pyrocarbonate (DEPC)-treated water, and stored at -80 °C. y g y f h mpl w c f m y gel electrophoresis and ethidium bromide staining. MT gene expression was analysed by dot blot hybridizations with MT-10 or MT-20 mRNA probes [START_REF] Hardivillier | Do organisms living around hydrothermal vent sites contain specific metallothioneins? The case of the genus Bathymodiolus (Bivalvia, Mytilidae)[END_REF]. Actin-1, 28S, and histone-4 templates were used to generate 32P labelled probes employed for standardisation. Ten micrograms of each RNA mpl w m x w h 20 μ 100% f m m 7 μ 37% f m l hy 2 μ 20× SSC h 68 °C f 15 min and immediately cooled on ice. RNA samples were transferred to positively charged nylon blotting membranes (Amersham) using a dot-blot apparatus. RNA was immobilized on the membranes by heating at 80 °C during 2 h. Specificity of hybridization conditions was optimized using blots of cDNA prepared from each analysed gene. Prehybridization was done at 42 °C for 2 h in 50% formamide, 1% SDS, 1 M NaCl and 10 μg μ -1 lm p m DNA. l ll g reactions were performed at 37 °C for 10 min in a mixture containing reaction buffer, random decamers, deionized water, CT TT T α32 -dATP and Klenow fragment in a final volume of 50 μ cc ing to the Manufacturer's instructions (Fermentas). The reaction was stopped by adding EDTA. Unincorporated nucleotides were removed using Sephadex G-50 columns. Each radioactive probe was synthesized at the same time from a single lot of 32P to allow their level comparisons because of their close adenine content. Hybridization with the labelled probe was done overnight (12-16 h) at 42 °C in the buffer mentioned above. All hybridizations were carried out under the same conditions. Membranes were washed sequentially in 2×, 1× and 0.5× SSC solutions (30 min) at 42 °C. Blots were then exposed to an Instant Imager (PACKARD) for quantification of gene expressions.

Between each hybridization, membranes were dehybridized by incubating 30 min in boiling 1% SDS. Radioactivity counting was performed as a control. The quantification of the metallothionein gene expression has to be standardised by comparing its level with those of other genes that are considered as stable and non-interfering with the metal detoxification processes. The chosen ones were actin-1, histone-4 and 28S genes.

Complementary DNA (cDNA) was synthesized using pp x m ly 5 μg f l RNA. Oligo(dT)20 primers were h p f l l m f 12 μ w h D C treated water. This mixture was denatured at 70 °C for 5 min and put on ice to allow the primers to anneal to the template. Samples were reverse transcribed for 90 min at 42 °C with Moloney Murine Leukemia Virus Reverse Transcriptase (MMLV RT®, Promega), RNAse inhibitor (RNA ® m g ) NT c l m f 25 μ . Th c w stopped incubating 5 min at 68 °C. The resulting reverse transcribed products were used for PCR amplification performed with a set of specific oligonucleotide primers for actin-1, histone-4 and 28S gene (Fig. 1). Polymerase chain reactions were performed with Taq polymerase® (Promega), buffer, primers, dNTPs and nuclease-free water in a final volume of 25 μ . Th thermal cycling program on a minicycler thermocycler® (Eppendorf) was set as follows: 35 cycles of 1 min at 94 °C (denaturing phase), 1 min at 57 °C for actin-1, 53 °C for histone-4 and 57 °C for 28S (annealing phase), and 1 min at 72 °C (elongation phase) followed by a single cycle at 72 °C for 10 min. PCR products were separated by gel electrophoresis and ethidium bromide staining. Bands of the expected size were excised under UV light and purified using the WIZARD SV gel and PCR clean up system (Promega) according to the Manufacturer's instructions. Purified products were ligated into the pGEM-T easy vector system I (Promega) with T4 DNA ligase. Escherichia coli JM109 high efficiency competent cells (Promega) were transformed with 10 μL of the ligation mixture and plated on ampicillin plates with Xgal and IPTG. Individual colonies were screened by PCR using the primers mentioned above. Insert-containing plasmids were purified using the Wizard plus Minipreps DNA purification System (Promega) according to the manufacturer's instructions. They were sequenced on a fl c m c f m h 5′ 3′ w h h ABI RISM Dy T m Cycle Sequencing Ready Reaction Kit (Perkin Elmer, USA). Nucleic sequences of B. azoricus and B. thermophilus were manually inspected and vector sequences were edited out. Forward and reverse sequences were aligned after editing with a multiple sequence alignment tool ClustalW (version 1.83) [START_REF] Thompson | CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, positions-specific gap penalties and weight matrix choice[END_REF]. Expasy translate tool software was used to translate the nucleic acid sequence in all potential reading frames. Characterization searching was performed, using the BLAST program [START_REF] Altschul | Basic local alignment search tool[END_REF], using the nucleic acid and predicted amino acid sequence to compare the sequences to others deposited in multiple databases.

 2.5 Protein analysis

Total protein levels of the crude supernatant were measured by the Lowry method [START_REF] Lowry | Protein measurement with the Folin phenol reagent[END_REF]. The protein levels were calculated using BSA (bovine serum albumin) as reference material and expressed as mg of protein (BSA equivalent) per g of wet mass of tissue (mg.g -1 w.wt.).BSA equivalent) per g of tissue wet weight (mg.g -1 w.wt.). The levels of MT in heat-denatured supernatants were estimated by differential pulse polarography (DPP) [START_REF] Olafson | An electrochemical approach to quantification and characterization of metallothioneins[END_REF]Sim, 1979, improved by Thompson and[START_REF] Thompson | An improved electrochemical method for the quantification of metallothioneins in marine organisms[END_REF]. A calibration curve was obtained by the standard addition method, using rabbit liver metallothionein (SIGMA, M-7641) as reference material. All the reagents used did not give a polarographic signal during the dosage of MT [START_REF] Erk | Advantages and disavantages of voltametric method in studying cadmium-metallothionein interactions[END_REF]. Levels of MT are given relative to the wet mass of tissue (mg g -1 w.wt.).

 2.6 Statistics

The data represent means ± S.D. Differences between treatment groups were evaluated by Kruskal-Wallis nonparametric tests followed by Mann-Whitney tests when a significant ff c w c ( ≤0.05). All c were performed using Sigma Stat 2.0 software.

Results  3.1 Expression standard isolation (Actin-1, Histone-4, 28S gene)

Total RNAwas extracted from both species (B. thermophilus and B. azoricus) and was reverse transcribed. Fragments of each standard were isolated from B. azoricus and B.

thermophilus.

An actin-1 cDNA fragment of approximately 300 bp was amplified by PCR from both species, cloned in the pGEM-T easy vector system I and sequenced. Inserts presented an actin-1 fragment of 275 bp long and nearly identical for both species except for two silent mutations at positions 129 and 231 (99% homology). A manual verification of the chromatograms corroborated the sequences. These two cDNA fragments also present strong nucleotide homology with other bivalve actin-1 sequences (Fig. 2): 95% with Mytilus galloprovincialis (accession no. AF157491), 91% with Crassostrea gigas (accession no. AB071191), 90% with Placopecten magellanicus (accession no. U55046) and 85% with Crassostrea virginica (accession no. X75894). The B. azoricus and the B. thermophilus Actin-1 sequences were deposited in EMBL under accession nos. AJ786402 and AJ786403.

A similar strategy was used to isolate a fragment of the histone-4 gene in both mussel species (Fig. 3). Fragments of 251 bp were cloned from both species and found nearly identical except for five mutation sites at positions 38, 45, 75, 165 and 217 (98% homology). These mutations are all silent except for that at position 38. Both histone gene fragments were translated in silico and the proteins showed a substitution of the lysine at position 13 by a glutamine residue. The histone-4 cDNA fragments show 84% homology with histone-4 in M. edulis (accession no. AJ492925) which confirms the evolutionary relationship between the hydrothermal vent mussels and M. edulis. The B. azoricus and the B. thermophilus Histone-4 sequences were deposited in EMBL under accession no. AJ786405 and no. AJ786404. The sequences of the histone-4 fragments that were isolated in both Bathymodiolus species present more variations with the histone-4 of M. edulis than the actin-1 fragments of the blue mussel.

A 274 bp fragment of the 28S gene was amplified in both hydrothermal vent mussel species by PCR. The two sequences are completely identical underlining the highly related nature of the two species (Fig. 4). These fragments present strong nucleotide homology with other bivalve 28S genes: 98% with M. edulis (accession no. AF120587) and M. galloprovincialis (accession no. AB103129), and 96% with Modiolus nipponicus (accession no. AB103125). The B. azoricus and the B. thermophilus 28S sequences were deposited in EMBL under accession nos. AJ786406 and AJ786407.



Metal distribution

The exposures of B. thermophilus for 44 h result in an increase of total levels of Ag, Cd and Zn in the gills and the mantle relatively to the levels measured in both organs of control mussels (Table 1). Conversely, it seems that no uptake of Cu and Hg resulted from the exposure of mussels to these metals in the time course of the experiment.

The subcellular elemental distribution of Cu and Zn in the gills and the mantle of B. thermophilus is given in Fig. 5 for control and exposed mussels.

The exposure to Ag resulted in a significant increase of Zn level in the soluble fraction of the gills, while Cu levels increased in both soluble and insoluble fractions of the same tissue. The exposure to Cd resulted in significant increases of Cu and Zn levels in the soluble fractions of the gills, and of Cu in the insoluble fraction of the mantle while Zn increased in the soluble fraction of the latter organ. The exposure to Cu resulted in increases of Cu in both fractions of the gills associated with an increase of Zn in the soluble fraction. In the same time, Cu decreased significantly in the insoluble fraction of the mantle. The exposure to Zn resulted in an increase of Cu levels in the soluble fraction of the gills and a decrease in the insoluble fraction of the mantle, while Zn increased in both fractions in the gills but only in the soluble fraction in the mantle.

 3.3 Levels of MT mRNA

Normalised levels of MT-10 and MT-20 mRNAs were detectable in gills and in the mantle of untreated and treated B. thermophilus all over the exposure experiments (Fig. 6).

The expression of both isoforms appeared to be stable and was not significantly induced in a metal-dependent manner. Statistical analysis shows that there was no significant difference (P>0.05) between MT mRNA levels of untreated and treated mussels. Standard deviations were low and express the little variation between individuals within each tested condition. This supports the hypothesis of the absence of induction of the metallothionein gene. Gills and mantle presented the same behaviour. No significant differences were observed between the MT mRNA levels within these two organs (P>0.05).

The MT-20 mRNA levels were always detected higher than those of the MT-10 mRNA. The MT mRNA expression was standardised by the signal of an actin-1, a histone-1 and a 28S probe. Each standard level remained constant all over the mussel treatments. The standard deviations of the 28S were greater than those obtained with the two other standards. This underlines the superior inter-individual variability of the transcriptional activity of the 28S gene.



Levels of MT proteins

MT contents in both studied organs of control or exposed B. thermophilus are presented in Fig. 7. MT levels were always significantly (P<0.05) higher in the gills than in the mantle.

The statistical analysis showed that silver was the only metal able to produce a significant increase of the MT level between exposed and control mussels after 44 h of experiment. This increase was associated to low standard deviations. It was detected in the gills and in the mantle corresponding to respective enhancements of 14% and 66%. The mantle did not show any significant increase following the exposure to other metals. Cu, Hg and Zn produced a significant decrease of MT l l ( p c ly -37% -44% -46%) c mp ly to the level of MT in the gills of control mussels. Even if no mortality occurred, experimental conditions, particularly pressure variations, may have induced cellular perturbations that would have been able to produce these decreases. Kruskal-Wallis test showed no significant differences between the levels of MT in both organs of Cd-exposed and control mussels.

 4. Discussion

Our study reports on variations of MT protein and MT mRNA syntheses in the gills and mantle of the hydrothermal vent mussel B. thermophilus following a short exposure to monometallic solutions. This was performed to trigger MT gene induction by the uptake of tested metals and especially by the modification of their subcellular distribution taking into account that MTs are metal-induced soluble metalloproteins. Overall, gills have a major importance in the bioaccumulation of metals in Mytilidae [START_REF] Roméo | Biomonitoring of trace metals in the Black Sea (Romania) using mussels Mytilus galloprovincialis[END_REF]. In control, the gills of B. thermophilus exhibited higher metal levels than that observed for the mantle, confirming the observations reported by [START_REF] Rousse | Bioaccumulation of metal within the hydrothermal mytilidae Bathymodiolus sp. from the Mid-Atlantic Ridge[END_REF] on B. azoricus from the Mid-Atlantic Ridge. This higher content results from their large interface with the hydrothermal environment and their involvement in respiration and nutrition [START_REF] Cosson-Mannevy | Putative neoplastic disorder in mussels (Mytilus edulis) from southern Vancouver Island waters, British Columbia[END_REF][START_REF] Langston | Metal handling strategies in molluscs[END_REF][START_REF] Marigomez | Cellular and subcellular distribution of metals in molluscs[END_REF]. The mantle is responsible for the synthesis of the shell of the bivalve molluscs and has a lesser importance in the uptake and storage of metals. Beside a short duration (44 h) the exposure to Ag, Cd and Zn resulted in an increase of the total content of respective metals in the gills and mantle of exposed mussels. The lack of observed uptake of Cu and Hg after the exposure may be related to a poorly available chemical form of these metals in the experimental solutions. Another factor likely to interfere with a normal behaviour of the mussels in the presence of metals is the stress due to the collection on the bottom of the ocean by 2500 m depth and the fact that the experiment was performed at the atmospheric pressure. The subcellular distribution of Cu and Zn varies with the considered element and tissue. Copper was more abundant in the insoluble fraction of both gills and mantle, while Zn was mainly found in the soluble fraction of the gills and equally distributed in the mantle. This pattern is frequently observed in bivalves [START_REF] Duquesne | Metal accumulation in the clam Tridacna crocea under natural and experimental conditions[END_REF][START_REF] Rousse | Bioaccumulation of metal within the hydrothermal mytilidae Bathymodiolus sp. from the Mid-Atlantic Ridge[END_REF][START_REF] Canesi | Heavy metals and glutathione metabolism in mussel tissues[END_REF]Geret, 2000;[START_REF] Geffard | Do seasonal changes affect metallothionein induction by metals in mussels, Mytilus edulis?[END_REF]. In the marine invertebrate, Nereis diversicolor, Cu is more bound to the insoluble fraction when individuals are collected from Cu contaminated areas than from non-contaminated areas [START_REF] Berthet | Accumulation and soluble binding of cadmium, copper, and zinc in the polychaete Hediste diversicolor from coastal sites with different trace metal bioavailabilities[END_REF]. Hydrothermal vent environment is known to have elevated concentrations of Cu [START_REF] Douville | The rai w fl (36°14′N MAR): h influence of ultramafic rocks and phase separation on trace metal content in Mid-Atlantic Ridge hydrothermal fluids[END_REF]Kadar et al., 2006). The high level of Cu found in the insoluble fraction of both organs of the hydrothermal mussels is convenient with the pattern observed in N. diversicolor. Copper is an essential element required for the biological function of numerous proteins including hemocyanins which are efficient dioxygen transporting molecules in Mollusca. The high level of Cu in the insoluble fraction may be attributed to degraded forms of these molecules that would have been transferred into lysosomes, the organelle that is able to sequester metals under insoluble form [START_REF] Viarengo | Biochemical characterization of a copper-thionein involved in Cu accumulation in the lysosomes of the digestive gland of mussels exposed to the metal[END_REF]. The larger proportion of zinc in the soluble fraction has to be related to its involvement in the structure of numerous thermostable (e.g. MTs) and thermolabile proteins [START_REF] Bustamante | Interspecific and geographical variations of trace element concentrations in Pectinidae from European waters[END_REF]. When the exposure to a single metal was followed by an increase of the total level of this metal and/or a modification of the distribution of essential metals (e.g. an increase of zinc in the soluble fraction) we could expect to observe a correlated activation of the MT gene transcription and an increase of MT protein level in the concerned tissue. Metallothioneins are proteins involved in stress phenomena. Their induction by metals was firstly reported by [START_REF] Piscator | On cadmium in normal human kidneys together with a report on the isolation of metallothionein from levers of cadmium-exposed rabbits[END_REF] and until today, studies of MT gene expression corroborated this postulate whatever the studied species [START_REF] Olsson | Functional comparison of the metal-regulated transcriptional control regions of metallothionein genes from cadmium-sensitive and tolerant fish species[END_REF][START_REF] Lam | Metal toxicity and metallothionein gene expression studies in common carp and tilapia[END_REF][START_REF] Morris | Identification and characterisation of metallothioneins from environmental indicator species as potential biomonitors[END_REF][START_REF] Pedersen | Purification and characterization of a calcium-induced metallothionein from the shore crab Carcinus maenas[END_REF][START_REF] Raspor | Assessment of metal exposure of marine edible mussels by means of biomarker[END_REF][START_REF] Sturzenbaum | The identification, cloning and characterization of earthworm metallothionein[END_REF][START_REF] Hensbergen | Metallothionein-bound cadmium in the gut of the insect Orchesella cincta (Collembola) in relation to dietary cadmium exposure[END_REF]. In M. edulis, nine isoforms were isolated and classified into two pools of protein, the MT-10 and the MT-20 (Mackay et al., 1993). It was suggested that the presence of such a number of isoforms may reflect different metal binding affinities and specificities of the promoters of the different genes. An analysis of MT gene expression was reported in M. edulis by [START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF]. They showed that blue mussel differentially expresses MT and suggested that the two MT pools were involved in different mechanisms implying that they may have different functions. The first pool including the MT-10 isoforms was constitutively synthesized and may be involved in the regulation of essential metals such as Cu and Zn. The other pool including the MT-20 isoform was induced by Cd and could be implicated in detoxifying processes. Results presented in this work do not fit with this previous report on littoral mussels. Contrary to other studies on MT gene expression in mytilids [START_REF] George | Heavy metal detoxication in the mussel Mytilus edulis composition of Cdcontaining kidney granules (tertiary lysosomes)[END_REF][START_REF] Viarengo | Role of metallothioneins in Cu and Cd accumulation and elimination in the g gill and digestive gland cells of Mytilus galloprovincialis lam[END_REF][START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF], there is no apparent induction of transcription and translation of the MT genes by each of the tested metals in hydrothermal mussels under our experimental conditions. Results of exposed mussels were nearly identical with controls. Whatever exposure resulted in the same pattern of MT gene expression: the responses of MT-10 and MT-20 genes remain constant. The high basal MT-10 and MT-20 mRNA levels in unexposed mussels may indicate pre-existing MT mRNAs of both isoforms [START_REF] Roesijadi | Characterization of mercury-binding proteins from the gills of marine mussels exposed to mercury[END_REF][START_REF] Viarengo | Role of metallothioneins in Cu and Cd accumulation and elimination in the g gill and digestive gland cells of Mytilus galloprovincialis lam[END_REF]. In our work, the pattern of the MT-20 gene expression is in contrast with that of the littoral blue mussel which does not show any MT-20 transcripts in non-exposed mussels [START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF]. The response of the Bathymodiolus MT-20 gene was always higher than that of the MT-10 gene. This contrasts with the behaviour of the MT genes reported in M. edulis which presents a constitutively synthesized MT-10 isoform whereas the MT-20 is induced by Cd [START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF]. The comparison of MT gene expression in hydrothermal and littoral mussels underlines that two species belonging to the same family (Mytilidae) may present differences in relation with the disparities between their environments. No difference of MT mRNA levels was denoted between the gills and the mantle of B. thermophilus (Fig. 6). In M. edulis, the gills present a higher MT gene expression than the mantle and the digestive gland tissues [START_REF] Lemoine | Metallothionein Isoforms in Mytilus edulis (Mollusca, Bivalvia): Complementary DNA Characterization and Quantification of Expression in Different Organs after Exposure to Cadmium, Zinc, and Copper[END_REF]. Similar results were obtained by [START_REF] Denis | Characterization and expression of a Bathymodiolus sp. metallothionein gene[END_REF] for hydrothermal mussels. This former work reported on a non-standardised expression of MT gene in Bathymodiolus sp. In our work, the response of MT genes in B. thermophilus does not seem to be organ specific underlying the major importance of the standardisation of the hybridization experiments by an outer gene probe considered as noninterfering in the metal metabolism. The nature of the standard appears to be of great importance. Even if the transcriptional profiles of the three tested genes presented stable patterns, the sizes of the standard deviations were different. Both actin-1 and histone-4 signals presented tiny standard deviations while the 28S profile showed more elevated ones. This underlines the fact that the inter-individual variability is greater for the expression of the 28S gene than for the answer of the two other genes. The 28S gene expression could vary in exposed mussels owing to an involvement in protein synthesis. The hydrothermal ecosystem exhibits naturally high metal concentrations. Fluids which are emitted from hydrothermal vents are fully charged with polymetallic sulfurs that are dispersed into the water surrounding the molluscs [START_REF] Sarradin | Chemical environment of the hydrothermal mussel communities in the Lucky Strike and Menez Gwen vent fields, Mid Atlantic Ridge[END_REF]. It implies that hydrothermal mussels may tolerate higher metal concentrations than the littoral blue mussel. Gills shows higher levels of metals than the mantle which plays a less preponderant role in metal uptake [START_REF] Mouneyrac | Effects of natural factors: salinity and body weight on cadmium, copper, zinc and Metallothionein Like Protein levels in resident populations of oysters (Crassostrea gigas) from a polluted estuary[END_REF] and MT synthesis is more important in gills, while there is no difference in the levels of MT mRNA between gills and mantle. These results suggest that different posttranscriptional controls of the MT synthesis may occur in these organs. By analysing metal distribution, MT mRNA and MT protein in target tissues of untreated and treated animals, we hoped to provide further insight into MT inducibility. In this regard, bivalves, particularly mussels accumulate trace elements at often high levels, depending on the metal and the tissue [START_REF] Langston | Metal handling strategies in molluscs[END_REF]. Clear variations in metal distribution and in MT expression could have been identified but our study showed complex profiles of metal distribution and MT expression. A lack of correlation between the metal concentrations, the inducibility of MT mRNA and MT proteins was denoted. Although regulation of MT synthesis seems to occur at the transcriptional level [START_REF] Haq | Signaling events for metallothionein induction[END_REF], experimental evidence suggests that MT levels are regulated by the rate of protein degradation [START_REF] Palmiter | Constitutive Expression of Metallothionein-III (MT-III), but Not MT-I, Inhibits Growth When Cells Become Zinc Deficient[END_REF][START_REF] Kershaw | Degradation and metal composition of hepatic isometallothioneins in rats[END_REF][START_REF] Mckim | In vitro degradation of apo-, zinc-, and cadmium-metallothionein by cathepsins B, C, and D[END_REF]. The possibility that MT genes may not have such an important role to play in B. thermophilus should as well be hypothesized. Bivalves are known to accumulate metals and Bathymodiolus was reported to have a high metal content [START_REF] Rousse | Bioaccumulation of metal within the hydrothermal mytilidae Bathymodiolus sp. from the Mid-Atlantic Ridge[END_REF] but the low levels of metals bound to MT raised the question about their involvement in the metal detoxication processes [START_REF] Fiala-Médioni | Bioaccumulation and detoxification of heavy metals in Bathymodiolus azoricus from Azores hydrothermal vents on the Mid Atlantic ridge[END_REF]. Bathymodiolus sp. houses symbiotic bacteria in vacuoles within the apical end of specialized gill cells called bacteriocytes [START_REF] Fiala-Medioni | Evidence of in situ uptake and incorporation of bicarbonate and amino acids by a hydrothermal vent mussel[END_REF]. The symbiotic bacteria could play a fundamental role in the metal detoxication of metal. Marine åproteobacteria, for example, common bacteria of the hydrothermal vent sites, tolerate high concentration of Cd [START_REF] Jeanthon | Susceptibility to heavy metals and characterization of heterotrophic bacteria isolated from two hydrothermal vent polychaete annelids, Alvinella pompejana and Alvinella caudata[END_REF][START_REF] Cottrell | Diversity of dissimilatory bisulfite reductase genes of bacteria bssociated with the deep-sea hydrothermal vent polychaete annelid Alvinella pompejana[END_REF]. The absorption of metal ions by these bacteria by phenomena of ingestion, and the excretion of these metals under particulate forms unavailable to biota could decrease the toxicity of metals for the hydrothermal communities. The eventuality that symbiotic bacteria may interact with mussels to detoxify the organism cannot therefore be excluded. The comparison of MT levels observed in hydrothermal and in littoral mussels emphasises great disparities. The amounts measured in M. edulis and in C. gigas (Geret, 2000) show that gills and mantle of these animals are characterized by smaller MT levels than that of hydrothermal mussels (Table 2). This could demonstrate that MT may have a major role in hydrothermal mussels. Their synthesis would not be as important if they were not fundamental for the cellular integrity. The monometallic solutions used to expose the hydrothermal vent mussels were higher than that naturally occurring in their environment (Table 3). The exposure concentrations for Cd, Cu and Hg were very high when compared to the natural conditions while the lowest ratio was for Zn as it is considered as the primary inducer of MT synthesis (Roesijadi, 1996). Nevertheless, hydrothermal mussels live in a diluted fluid and do not face concentrations as high as those used for our exposures. When B. thermophilus were sampled and brought from the bottom of the ocean up to the surface, they were submitted to a huge pressure difference (250-300 atmospheres). MT gene expression might be influenced by such a pressure shock. This factor could be a new investigation field in the comprehension of the biological roles of MTs. B. azoricus mussels from Menez-Gwen vent site (850 m depth, Middle Atlantic Ridge) are able to survive in aquaria for a long period after being brought at the atmospheric pressure [START_REF] Kadar | Experimentally induced endosymbiont loss and re-acquirement in the hydrothermal vent bivalve Bathymodiolus azoricus[END_REF]Hourdez, personal communication). Experimentations performed with these mussels may allow to investigate/discriminate if MT gene expression can return to a basal level close to that of M. edulis or be induced by a metallic stimulus. The study of these particular mussels would also allow to observe long term behavioural observations. Our short exposures and the use of high concentrations of metals combined with the pressure stress were likely to preclude sufficient metal uptake and subcellular distribution disturbations important enough to allow significant increases of MT protein levels.

The characterization of the regulation sequences in the promotor region of the Bathymodiolus MT genes and the study of the ability of symbionts to sequester large quantities of metals may provide answers to the questions mentioned above. AB071191) and Placopecten magellanicus (accession no. U55046) using ClustalW 1.83 [START_REF] Thompson | CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, positions-specific gap penalties and weight matrix choice[END_REF] and edited using the multiple sequence alignment editor Bioedit [START_REF] Hall | BioEdit: a user-friendly biological sequence alignment editor and analysis program for Windows 95/98/NT[END_REF]. [START_REF] Thompson | CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, positions-specific gap penalties and weight matrix choice[END_REF] and edited using the multiple sequence alignment editor Bioedit [START_REF] Hall | BioEdit: a user-friendly biological sequence alignment editor and analysis program for Windows 95/98/NT[END_REF] [START_REF] Géret | Sy hè m ll h ch z x l (l'h î l m l ) réponse à une contamination métallique par la voie directe et par la voie trophique[END_REF] and in hydrothermal modioles (controls).

Species Gills Mantle

Mytilus edulis 202 386

Crassostrea gigas 115 123

Bathymodiolus thermophilus 240 625

Figure 1 .Figure 2 .

 12 Figure 1. N cl c (5' -3') f h CR p m . A: Primers used for Actin-1 amplification; B: Primers used for Histone-4 amplification; C: Primers used for 28S amplification

Figure 4 .

 4 Figure 4. The 28S nucleotide sequences of Bathymodiolus azoricus (accession no. AJ786406) and Bathymodiolus thermophilus (accession no. AJ786407) were aligned with 28S sequences of Modiolus nipponicus (accession no. AB103125), Mytilus edulis (accession no. AF120587), and Mytilus galloprovincialis (accession no. AB103129), using ClustalW 1.83[START_REF] Thompson | CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, positions-specific gap penalties and weight matrix choice[END_REF] and edited using the multiple sequence alignment editor Bioedit[START_REF] Hall | BioEdit: a user-friendly biological sequence alignment editor and analysis program for Windows 95/98/NT[END_REF]

Figure 5 .Figure 6 .Figure 7 .

 567 Figure 5. Subcellular distribution of metals (Cu, Zn) in soluble and pellet fractions obtained after the centrifugation of gills ( : insoluble fraction, : soluble fraction) and mantle (■: insoluble fraction and □: soluble fraction) homogenates (mean ± S.D., n=10, μg g -1 w.w.). Asterisks indicate significant differences ( ≤0.05) w c l (C) exposed mussels (Ag, Cd, Cu, Hg, Zn) as determined by Kruskal-Wallis tests.

Table 1 .

 1 Total levels of metals (µg.g -1 w.w., n=4 ± S.D.) in mussel tissues of control specimens and those exposed for 44 h to monometallic solutions of Ag, Cd, Cu, Hg and Zn.

		Ag	Cd	Cu	Hg	Zn
	Control	3.90	0.49	35.81	0.34	15.30
	Gills
	Exposed	4.58	1.21	41.68	0.30	25.53
	Control	3.71	0.07	9.38	0.48	7.61
	Mantle
	Exposed	4.76	0.23	7.91	0.31	14.33

Table 2 .

 2 Average levels of metallothioneins (µg.g -1 w.w.) in littoral molluscs

Table 3 .

 3 Metal concentrations used in the exposures of B. thermophilus and natural metal concentrations occurring at hydrothermal vent fields.

	Metal	Exposure concentrations (µg.L -1 )	Vent field concentration (µg.L -1 )	Reference
	Ag	20	0.46 (Menez Gwen)	Douville et al., 2002
	Cd	200	0.25 (Menez Gwen)	Douville et al., 2002
	Cu	40	0.12 (Menez Gwen)	Douville et al., 2002
	Hg	20	0.005 (Kagoshima Bay)	Ando et al., 2002
	Zn	1000	131 (Menez Gwen)	Douville et al., 2002

Table 4 .

 4 Metal LD50 in close species from Bathymodiolus thermophilus.

	Metal	LD50 (mg.L -1 )	Species	Reference
	Ag	14	Mytilus edulis	Martin et al., 1981
	Cd	1.53	Perna viridis	Yap et al., 2004
	Cu	0.25	Perna viridis	Yap et al., 2004
	Hg	0.25	Mytilopsis sallei	Devi, 1996
	Zn	3.20	Perna viridis	Yap et al., 2004

Acknowledgments

This work was supported by a grant of the Ministère de l'Éducation Nationale, de la Recherche et des Technologies, by the EU programme VENTOX: EVK3-CT-1999-00003, (coordinator: D. Dixon) and by the MDF. Our thanks go to Pr F. Lallier and Dr P. M. Sarradin, chief scientists of the HOPE and ATOS cruises for providing access to the research material. The authors gratefully acknowledge Monika Ghosh for the English corrections.

 [START_REF] Thompson | CLUSTAL W: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, positions-specific gap penalties and weight matrix choice[END_REF]and edited using the multiple sequence alignment editor Bioedit [START_REF] Hall | BioEdit: a user-friendly biological sequence alignment editor and analysis program for Windows 95/98/NT[END_REF]) B: The proteic histone-4 protein sequences of B. azoricus and B. thermophilus were translated in silico and aligned with histone-4 sequences of M. edulis (accession no. AF506076) using the same tools.


Konstantin Avrachenkov
email: k.avrachenkov@sophia.inria.fr

Oussama Habachi
email: oussama.habachi@univ-avignon.fr

Alexey Piunovskiy

Yi Zhang

Théorie du contrôle optimal impulsif à horizon infini avec application au contrôle de congestion dans Internet

Keywords: Contrôle optimale impulsif, Horizon de temps infini, Critères actualisé et moyenne sur le temps, Alpha-équité, Contrôle de congestion dans Internet Optimal impulsive control, Infinite time horizon, Average and discounted criteria, Alpha-fairness, Internet Congestion Control

Dans ce papier, nous développons une approche basée sur les équations de Bellman pour les problèmes de contrôle optimal impulsif à horizon infini. Les deux critères actualisé et moyenne sur le temps sont pris en considération. Nous établissons des conditions naturelles et très générales en vertu desquelles un triplet canonique de contrôle produit une politique de rétroaction optimale. Ensuite, nous appliquons nos résultats généraux pour le contrôle de congestion dans Internet offrant un cadre idéal pour la conception des algorithmes de gestion active de files d'attente. En particulier, nos résultats théoriques généraux suggèrent un système de gestion active de files d'attente à seuil qui prend en compte les principaux paramètres du protocole de contrôle de transmission.

Introduction

Recently, there has been a steady increase in the demand for QoS (Quality of Services) and fairness among the increasing number of IP (Internet Protocol) flows. With respect to QoS, a plethora of research focuses on smoothing the throughput of AIMD (Additive Increase Multiplicative Decrease)-based congestion control for the Transmission Control Protocol (TCP), which is prevailingly employed in today's transport layer communication. These approaches adopt various congestion window updating policies to determine how to adapt the congestion window size to the network environment. Besides, there have been proposals of new high speed congestion control algorithms that can efficiently utilize the available bandwidth for large volume data transfers, see [START_REF] Floyd | High speed TCP for large congestion windows[END_REF][START_REF] Kelly | Scalable TCP : Improving performance in high-speed wide area networks[END_REF][START_REF] Leith | H-TCP : TCP for high-speed and long-distance networks[END_REF][START_REF] Wei | FAST TCP : motivation, architecture, algorithms, performance[END_REF]. Although TCP gives efficient solutions to end-to-end error control and congestion control, the problem of fairness among flows is far from being solved. See for example, [START_REF] Altman | Fairness in MIMD congestion control algorithms[END_REF][START_REF] Möller | Inter-protocol fairness between TCP New Reno and TCP Westwood[END_REF][START_REF] Li | Experimental evaluation of tcp protocols for high-speed networks[END_REF] for the discussions of the unfairness among various TCP versions.

The fairness can be improved by the Active Queue Management (AQM) through the participation of links or routers in the congestion control. The first AQM scheme, the Random Early Drop (RED), is introduced in [START_REF] Floyd | Random early detection gateways for congestion avoidance[END_REF], and allows to drop packets before the buffer overflows. The RED was followed by a plethora of AQM schemes ; a survey of the most recent AQM schemes can be found in [START_REF] Adams | Active queue management : A survey[END_REF]. However, the improvement in fairness provided by AQMs is, on the one hand, still not satisfactory ; and, on the other hand, at the core of the present paper.

Since most of the currently operating TCP versions exhibit a saw-tooth like behavior, it appears that the setting of impulsive control is very well suited for the Internet congestion control. Furthermore, since the end users expect permanent availability of the Internet, it looks natural to consider the infinite time horizon setting. With the best of our efforts, we could not find any available results about infinite time horizon optimal impulsive control problems. Thus, a general theory for infinite time horizon optimal impulsive control needs to be developed. We note that the results available in [START_REF] Miller | Impulsive control in continuous and discrete-continuous systems[END_REF] and references therein about finite time horizon optimal impulsive control problems cannot directly be applied to the infinite horizon with non-decreasing energy of impulses. In [START_REF] Miller | Impulsive control in continuous and discrete-continuous systems[END_REF] the impulsive control is described with the help of Stieltjes integral with respect to bounded variation function. Clearly, the bounded variation function cannot represent an infinite number of impulses with non-decreasing energy.

Therefore, in the first part of the paper, we develop Bellman equation based approach for infinite time horizon optimal impulsive control problems. We consider both discounted and time average criteria. We establish very general and at the same time natural conditions under which a canonical control triplet produces an optimal feedback policy.

Then, in the second part of the paper we apply the developed general results to the Internet congestion control. The network performance is measured by the long-run average α-fairness and the discounted α-fairness, see [START_REF] Mo | Fair end-to-end window-based congestion control[END_REF], which can be specified to the total throughput, the proportional fairness and the max-min fairness maximization with the particular values of the tuning parameter α. The model in the present paper is different from the existing literature on the network utility maximization see e.g., [START_REF] Kunniyur | End-to-end congestion control schemes : utility functions, random losses and ecn marks[END_REF][START_REF] Kelly | Rate control for communication networks : shadow prices, proportional fairness and stability[END_REF][START_REF] Low | Optimization flow control-i : basic algorithm and convergence[END_REF], in at least two important aspects : (a) we take into account the fine, saw-tooth like, dynamics of congestion control algorithms, and we suggest the use of per-flow control and describe its form. Indeed, not long ago a per-flow congestion control was considered infeasible. However, with the introduction of modern very high speed routers, the per-flow control becomes realistic, see [START_REF] Noirie | Semantic networking : Flow-based, traffic-aware, and self-managed networking[END_REF]. (b) By solving rigorously the impulsive control problems, we propose a novel AQM scheme that takes into account not only the traffic transiting through bottleneck links but also end-to-end congestion control algorithms implemented at the edges of the network. More specifically, our scheme asserts that a congestion notification (packet drop or explicit congestion notification) should be sent out whenever the current sending rate is over a threshold, whose closed-form expression is computed.

The remainder of this paper proceeds as follows. In the next section we give preliminary results regarding general average and discounted impulsive optimal control problems. In Section 3, we describe the mathematical models for the congestion control, and solve the underlying optimal impulsive control problems based on the results obtained in Section 3. Section 4 concludes the paper.

Preliminary result

In this section, we establish the verification theorems for a general infinite horizon impulsive control problem under the long-run average criterion and the discounted criterion, which are then used to solve the concerned Internet congestion control problems in the next section.

Description of the controlled process

Let us consider the following dynamical system in X ⊆ IR n (with X being a nonempty measurable subset of R n , and some initial condition x(0) = x 0 ∈ X) governed by

dx = f (x, u)dt, (1)
where u ∈ U is the gradual control, with U being an arbitrary nonempty Borel space. Suppose another nonempty Borel space V is given, and, at any time moment T , if he decides so, the decision maker can apply an impulsive control v ∈ V leading to the following new state :

x(T ) = j(x(T -), v), (2)
where j is a measurable mapping from R n × V to X. Thus, we have the next definition of a policy.

Definition 1 A policy π is defined by a U -valued measurable mapping u(t) and a sequence of impulses

{T i , v i } ∞ i=1 with v i ∈ V and • • • ≥ T i+1 ≥ T i ≥ 0, which satisfies T 0 := 0 and lim i→∞ T i = ∞. A policy π is called a feedback one if one can write 1 u(t) = u f (x(t)), T L i = inf{t > T i-1 : x(t) ∈ L}, v i = v f,L (x(T - i ))
, where u f is a U -valued measurable mapping on R n , and L ⊂ X is a specified (measurable) subset of X. A feedback policy is completely characterized and thus denoted by the triplet (u f , L, v f,L ).

We are interested in the (admissible) policies π under which the following hold (with any initial state). (a) T 0 ≤ T 1 < T 2 < . . .. 2 (b) The controlled process x(t) described by ( 1) and ( 2) is well defined : for any initial state x(0) = x 0 , there is a unique piecewise differentiable function x π (t) with x π (0) = x 0 , satisfying (1) for all t, wherever the derivative exists ; satisfying (2) for all T = T i , i = 1, 2, . . . ; and satisfying that x π (t) is continuous at each t = T i . (c) Within a finite interval, there are no more than finitely many impulsive controls. The controlled process under such a policy π is denoted by x π (t).

Optimal impulsive control problem and Bellman equation

Let c(x, u) be the reward rate if the controlled process is at the state x and the gradual control u is applied, and C(x, v) be the reward earned from applying the impulsive control v.

1. Here the superscript f stands for "feedback". 2. In the case when two (or more) impulses v i and v i+1 are applied simultaneously, that is T i+1 = T i , we formulate this as a single impulse v with the effect j(x, v) := j(j(x, v i ), v i+1 ), and include v into the set V .

Inria

Under the policy π and initial state x 0 , the average reward is defined by

J(x 0 , π) = lim inf T →∞ 1 T    T 0 c(x π (t), u(t))dt + N (T ) i=1 C(x π (T - i ), v i )    , (3)
where and below N (T ) := sup {n > 0, T n ≤ T }, and x(T - 0 ) := x 0 ; and the discounted reward (with the discount factor ρ > 0) is given by

J ρ (x 0 , π) = lim inf T →∞ J T ρ (x 0 , π),
where

J T ρ (x 0 , π) = T 0 e -ρt c(x π (t), u(t))dt + i=1,2,...Ti∈[0,T ] e -ρTi C(x π (T i -0), v i ). (4)
We only consider the class of (admissible) policies π such that the right side of (3) (resp., (4)) is well defined under the average (resp., discounted) criterion, i.e., all the limits and integrals are finite, which is automatically the case, e.g., when C and c are bounded functions. The optimal control problem under the average criterion reads

J(x 0 , π) → max π , (5)
and the one under the discounted criterion reads

J ρ (x 0 , π) → max π . (6)
A policy π * is called (average) optimal (resp., (discounted) optimal) if J(x 0 , π * ) = sup π J(x 0 , π) (resp., J ρ (x 0 , π * ) = sup π J ρ (x 0 , π)) for each x 0 ∈ X. Below we consider both problems (3) and [START_REF] Möller | Inter-protocol fairness between TCP New Reno and TCP Westwood[END_REF], and provide the corresponding verification theorems for an optimal feedback policy, see Theorems 1 and 2.

For the average problem (3), we consider the following condition.

Condition 1 There are a continuous function h(x) on X and a constant g ∈ R such that the following hold.

(i) The gradient ∂h ∂x exists everywhere apart from a subset D ⊂ X, whereas under every policy π and for each initial state x 0 , h(x π (t)) is absolutely continuous on [T i , T i+1 ), i = 0, 1, . . . ; and {t ∈ [0, ∞) : x π (t) ∈ D} is a null set with respect to the Lebesgue measure. (ii) For all x ∈ X \ D,

max sup u∈U c(x, u) -g + ∂h ∂x , f (x, u) , sup v∈V [C(x, v) + h(j(x, v)) -h(x)] = 0, (7)
and for all x ∈ D,

sup v∈V [C(x, v) + h(j(x, v)) -h(x)] ≤ 0.
(iii) There are a measurable subset L * ⊂ X and a feedback policy

π * = (u f * , L * , v f,L * ) such that for all x ∈ X \ (D ∪ L * ), c(x, u f * (x)) -g + ∂h ∂x , f (x, u f * (x) = 0 and for all x ∈ L * , C(x, v f,L * ) + h(j(x, v f,L * (x)
))h(x) = 0, and j(x, v f,L * (x)) / ∈ L * . (iv) For any policy π and each initial state

x 0 ∈ X, lim sup T →∞ h(x π (T )) T ≥ 0, whereas lim sup T →∞ h(x π * (T )) T = 0.
Equation [START_REF] Li | Experimental evaluation of tcp protocols for high-speed networks[END_REF] is the Bellman equation for problem [START_REF] Leith | H-TCP : TCP for high-speed and long-distance networks[END_REF]. (g, π * , h) from Condition 1 is called a canonical triplet, and the policy π * is called a canonical policy. The next result asserts that any canonical policy is optimal for problem (5).

Theorem 1 For the average problem [START_REF] Altman | Fairness in MIMD congestion control algorithms[END_REF], any feedback policy π * satisfying Condition 1 is optimal, and g in Condition 1 is the value function, i.e., g = sup π J(x 0 , π) for each x 0 ∈ X.

Proof 1 For each arbitrarily fixed T > 0, initial state x 0 ∈ X and policy π, it holds that

h(x π (T )) = h(x 0 ) + T 0 ∂h ∂x (x π (t)), f (x π (t), u(t)) dt + i:Ti∈[0,T ] h(j(x π (T - i ), v i )) -h(x π (T - i )) . (8)
Therefore,

T 0 c(x π (t), u(t))dt + N (T ) i=1 C(x π (T - i ), v i ) + h(x π (T )) = h(x 0 ) + T 0 c(x π (t), u(t)) + ∂h ∂x (x π (t)), f (x π (t), u(t)) dt + i:Ti∈[0,T ] h(j(x π (T - i ), v i )) -h(x π (T - i )) + C(x π (T - i ), v i ) ≤ h(x 0 ) + T 0 gdt,
where the last inequality is because of ( 7) and the definition of g and h as in Condition 1. It

follows that 1 T T 0 c(x π (t), u(t))dt + N (T ) i=1 C(x π (T - i ), v i ) + h(x π (T )) T ≤ h(x0) T + g, and conse- quently, J(x 0 , π) + lim sup T →∞ h(x π (T )) T ≤ g. Since lim sup T →∞ h(x π (T )) T
≥ 0 for each π, we obtain J(x 0 , π) ≤ g for each policy π. For the feedback policy π * from Condition 1, since

lim sup T →∞ h(x π * (T )) T
= 0, and we have J(x 0 , π * ) = g. The statement is proved.

For the discounted problem (6), we formulate the following condition.

Condition 2 There is a continuous function W (x) on X such that the following hold.

(i) Gradient ∂W ∂x exists everywhere apart from a subset D ⊂ X ⊂ IR n ; for any policy π and for any initial state x 0 , the function W (x π (t)) is absolutely continuous on all intervals [T i-1 , T i ), i = 1, 2, . . . ; and the Lebesgue measure of the set {t ∈ [0, ∞) : x π (t) ∈ D} equals zero. (ii) The following Bellman equation

max sup u∈U c(x, u) -ρW (x) + ∂W ∂x , f (x, u) , sup v∈V [C(x, v) + W (j(x, v)) -W (x)] = 0. ( 9
)
is satisfied for all x ∈ X \ D and sup v∈V [C(x, v) + W (j(x, v)) -W (x)] ≤ 0 for all x ∈ D.
(iii) There are a measurable subset L * ⊂ X and a feedback policy

π * = (u f * (x), L * , v f,L * ) such that c(x, u f * (x)) -ρW (x) + ∂W ∂x , f (x, u f * (x) = 0 for all x ∈ X \ (D ∪ L * ) and C(x, v f,L * (x)) + W (j(x, v f,L * (x))) -W (x) = 0 for all x ∈ L * ; moreover, g(x, v f,L * (x)) ∈ X \ L * .
(iv) For any initial state x 0 ∈ X, lim sup T →∞ e -ρT W (x π (T )) ≥ 0 for any policy π and lim sup T →∞ e -ρT W (x π * (T )) = 0.

Theorem 2 For the discounted problem (6), any feedback policy π * satisfying Condition 2 is optimal, and sup

π J ρ (x 0 , π) = W (x 0 ) = J ρ (x 0 , π * ) for each x 0 ∈ X. Inria Proof 2
The proof proceeds along the same line of reasoning as in that of Theorem 1 ; instead of ( 8), one should now make use of the representation

0 = W (x 0 ) + T 0 e -ρt ∂W (x π (t)) ∂x , f (x π (t), u(t)) -ρW (x π (t)) dt + i=1,2,...Ti∈[0,T ] e -ρTi {W (g(x π (T i -0), v i ) -W (x π (T i -0))} -e -ρT W (x π (T )).
3 Applications of the optimal impulsive control theory to the Internet congestion control

In this section, we firstly informally describe the impulsive control problem for the Internet congestion control, which will then be later formalized in the framework of the previous section. Let us consider n TCP connections operating in an Internet Protocol (IP) network of L links defined by a routing matrix A, whose element a lk is equal to one if connection k goes through link l, or zero otherwise. 3 Denote by x k (t) the sending rate of connection k at time t. We also denote by P (k) the set of links corresponding to the path of connection k. In this section, the column vector notation x(t) := (x 1 (t), . . . , x n (t)) T is in use.

The data sources are allowed to use different TCP versions, or if they use the same TCP, the TCP parameters (round-trip time, the increase-decrease factors) can be different. Therefore, we suppose that the sending rate of connection k evolves according to the following equation

d dt x k (t) = a k x γ k (t), (10)
in the absence of congestion notification, and the TCP reduces the sending rate abruptly if a congestion notification is sent to the source k, i.e., when a congestion notification is sent to the source k at time moment T i,k with T 0,k := 0 and T i+1,k ≥ T i,k , its sending rate is reduced as follows

x k (T i,k ) = b k x k (T - i,k ) < x k (T - i,k ). (11)
Here and below, a k , b k and γ k are constants, which cover at least two important versions of the TCP end-to-end congestion control ; if γ k = 0 we retrieve the AIMD congestion control mechanism (see [START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF]), and if γ k = 1 we retrieve the Multiplicative Increase Multiplicative Decrease (MIMD) congestion control mechanism (see [START_REF] Kelly | Scalable TCP : Improving performance in high-speed wide area networks[END_REF][START_REF] Zhang | Convergence of trajectories and optimal buffer sizing for MIMD congestion control[END_REF]). Also note that [START_REF] Miller | Impulsive control in continuous and discrete-continuous systems[END_REF] and [START_REF] Mo | Fair end-to-end window-based congestion control[END_REF] correspond to a hybrid model description that represents well the saw-tooth behaviour of many TCP variants, see [START_REF] Hespanha | Hybrid modeling of TCP congestion control[END_REF][START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF][START_REF] Zhang | Convergence of trajectories and optimal buffer sizing for MIMD congestion control[END_REF].

When T i+2,k > T i+1,k = T i,k > T i-1,k
, multiple (indeed, two in this case) congestion notifications are being sent out simultaneously at T i+1,k = T i,k ; as explained in the previous section, we will understand such multiple reductions on the sending rate as a single "big" impulsive control. In this section we write T i := (T i,1 , . . . , T i,n ) for the ith time moments of the impulsive control for each of the n connections, and assume that the decision of reducing the sending rate of connection k is independent upon the other connections. Since there is no gradual control, we tentatively call the sequence of T 1 , T 2 , . . . a policy for the congestion control problem, which will be formalized below.

We will consider two performance measures of the system ; namely the time average α-fairness function

J(x 0 ) = lim inf T →∞ 1 1 -α n k=1 1 T T 0 x 1-α k (t)dt,
and the discounted α-fairness function

J(x 0 ) = lim inf T →∞ 1 1 -α n k=1 T 0 e -ρT x 1-α k (t)dt,
to be maximized over the consecutive moments of sending congestion notifications T i , i = 1, 2, . . . . In the meanwhile, due to the limited capacities of the links, the expression lim inf T →∞

1 T T 0 Ax(t)dt (resp., lim inf T →∞
T 0 e -ρt Ax(t)dt) under the average (resp., discounted) criterion should not be too big. Therefore, after introducing the weight coefficients λ 1 , . . . , λ L ≥ 0, we consider the following objective functions to be maximized :

L(x 1 , . . . , x n ) = n k=1 lim inf T →∞ 1 T T 0 x 1-α k (t) 1 -α dt - L l=1 λ l k:l∈P (k) lim inf T →∞ 1 T T 0 x k (t)dt (12)
in the average case, and

Lρ (x 1 , . . . , x n ) = n k=1 lim inf T →∞ T 0 e -ρt x 1-α k (t) 1 -α dt - L l=1 λ l k:l∈P (k) lim inf T →∞ T 0 e -ρt x k (t)dt (13)
in the discounted case, where we recall that P (k) indicates the set of links corresponding to connection k. We can interpret the second terms in ( 12) and ( 13) as "soft" capacity constraints.

Below we obtain the optimal policy for the problems

L(x 1 , . . . , x n ) → max T1,T2,... . (14)
and

Lρ (x 1 , . . . , x n ) → max T1,T2,... , (15)
respectively.

Solving the average optimal impulsive control problem for the Internet congestion control

We first consider in this subsection the average problem [START_REF] Low | Optimization flow control-i : basic algorithm and convergence[END_REF]. Concentrated on policies satisfying [START_REF] Low | Optimization flow control-i : basic algorithm and convergence[END_REF] it is sufficient to consider the case of n = 1. Indeed, one can legitimately rewrite the function [START_REF] Kunniyur | End-to-end congestion control schemes : utility functions, random losses and ecn marks[END_REF] as

lim inf T →∞ 1 1 -α n k=1 1 T T 0 x 1-α k (t)dt = lim T →∞ 1 1 -α n k=1 1 T T 0 x 1-α k (t)dt < ∞ and lim inf T →∞ 1 T T 0 x k (t)dt < ∞ for each k = 1, . . . , n, for problem
L(x 1 , . . . , x n ) = n k=1 lim inf T →∞ 1 T T 0 x 1-α k (t) 1 -α -λ k x k (t) dt,

Inria

where λ k = l∈P (k) λ l , which allows us to decouple different sources. Thus, we will focus on the case of n = 1, and solve the following optimal control problem

J(x 0 ) = lim inf T →∞ 1 T T 0 x 1-α (t) 1 -α -λx(t) dt → max T1,T2,... , (16)
where x(t) is subject to [START_REF] Miller | Impulsive control in continuous and discrete-continuous systems[END_REF], [START_REF] Mo | Fair end-to-end window-based congestion control[END_REF] and the impulsive controls T 1 , T 2 , . . . with the initial condition x(0) = x 0 . Here and below the index k = 1 has been omitted for convenience.

In the remaining part of this subsection, using the verification theorem (see Theorem 1), we rigorously obtain the optimal policy and value to problem [START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF] in closed-forms.

Let us start with formulating the congestion control problem [START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF] in the framework given in the previous section, which also applies to the next subsection. Indeed, one can take the following system parameters ;

X = (0, ∞), j(x, v) = b v x, C(x, v) = 0 with v ∈ V = {1, 2, . . .}, f (x, u) = ax γ , and c(x, u) = x 1-α
1-αλx with u ∈ U, which is a singleton, i.e., there is no gradual control, so that in what follows, we omit u ∈ U everywhere. For practical reasons, it is reasonable to focus only on policies π, under which there is some constant T π ≥ 0 such that for each t > T π , x π (t) belongs to a π-dependent but t-independent compact subset of X. 4 Theorem 3 Suppose λ > 0, γ ∈ [0, 1], α > 0, α = 1, 2αγ = 0, a ∈ (0, ∞), and b ∈ (0, 1). Let us consider the average congestion control problem [START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF]. Then the optimal policy is given by

π * = (L * , v L * ) with L * = [x, ∞), and v L * (x) = k if x ∈ [ x b k-1 , x b k ) ⊆ L * for k = 1, 2, . . . , where x = (2 -γ)(1 -b 2-α-γ ) (2 -α -γ)(1 -b 2-γ )λ 1 α > 0. (17)
When γ < 1, the value function is given by

J(x 0 , π * ) = g := xλ α 1 -α (1 -γ)(1 -b 2-γ ) (2 -γ)(1 -b 1-γ ) ; (18)
and when γ = 1,

J(x 0 , π * ) = g := xλ α 1 -α b -1 ln (b) . ( 19
)
Proof 3 Suppose γ < 1. By Theorem 1, it suffices to show that Condition 1 is satisfied by the policy π * = (L * , v L *

), the constant g given by ( 18) and the function

h(x) = h 0 (x), if x ∈ (0, x), h k (x) = h 0 (b k x), if x ∈ [x/b k-1 , x/b k ), (20)
where and x is given by [START_REF] Zhang | Convergence of trajectories and optimal buffer sizing for MIMD congestion control[END_REF]. For reference and to improve the readability, we write down the Bellman equation [START_REF] Li | Experimental evaluation of tcp protocols for high-speed networks[END_REF] for problem [START_REF] Avrachenkov | Convergence of trajectories and optimal buffer sizing for AIMD congestion control[END_REF] as follows ;

h 0 (x) = 1 a -x 2-α-γ (1-α)(2-α-γ) + λ x 2-γ 2-γ + g x 1-γ 1-γ ,
max x 1-α 1 -α -λx -g + ∂h ∂x (x)ax γ , sup m=1,2,... {h(b m x) -h(x)} = 0. ( 21
)
Since parts (i,iv) of Condition 1 are trivially satisfied, we only verify its parts (ii,iii) as follows.

Consider firstly x ∈ (0, x) = X\L * . Then, we obtain from direct calculations that

x 1-α 1-α -λx - g + ∂h(x) ∂x ax γ = x 1-α 1-α -λx -g + ∂h0(x) ∂x ax γ = 0. Let us show that sup m=1,2,... {h(b m x) -h(x)} =
4. This requirement can be withdrawn in the next subsection dealing with the discounted problem.

RR n°8403

sup m=1,2,... {h 0 (b m x)h 0 (x)} ≤ 0 for x ∈ (0, x) as follows. Define ∆ 1 (x) := h 0 (bx)h 0 (x) for each x ∈ (0, x). Then one can show that ∆ 1 (x) < 0 for each b ∈ (0, 1). Indeed, direct calculations give

∆ 1 (x) = x 1-γ a - (b 2-α-γ -1)x 1-α (1 -α)(2 -α -γ) + λ (b 2-γ -1)x 2 -γ + g (b 1-γ -1) 1 -γ ,
so that for the strict negativity of ∆ 1 (x), it is equivalent to showing it for the following expression

∆1 (x) := - (b 2-α-γ -1)x 1-α (1 -α)(2 -α -γ) + λ (b 2-γ -1)x 2 -γ + g (b 1-γ -1) 1 -γ ,
whose first order and second order derivatives (with respect to x) are given by

∆′ 1 (x) = - (b 2-α-γ -1)x -α 2 -α -γ + λ (b 2-γ -1) 2 -γ and ∆′′ 1 (x) = α(b 2-α-γ -1)x -α-1 2 -α -γ .
Under the conditions of the parameters, ∆′′ 1 (x) < 0 for each x ∈ (0, x), and thus the function ∆1 (x) is concave on (0, x) achieving its unique maximum at the stationary point given by x =

x = (2-γ)(1-b 2-α-γ ) (2-α-γ)(1-b 2-γ )λ 1 α > 0.
Note that ∆1 (x) = 0 and lim x↓0 ∆1 (x) ≤ 0. It follows from the above observations and the standard analysis of derivatives that ∆1 (x) < 0 and thus ∆ 1 (x) < 0 for each x ∈ (0, x). Since ∂x ∂b ≤ 0 for each b ∈ (0, 1) as can be easily verified, one can replace b with b m (m = 2, 3, . . . ) in the above argument to obtain that h 0 (b m x)h 0 (x) < 0 for each x ∈ (0, x), and thus sup m=1,2,...

{h 0 (b m x) -h 0 (x)} ≤ 0 (22)
for x ∈ (0, x), as desired. Hence, it follows that Condition 1(ii,iii) is satisfied on (0, x). Next, we show by induction that Condition 1(ii,iii) is satisfied on

[ x b k-1 , x b k ), k = 1,
m x) -h(x) = h 0 (b m x) -h 0 (bx) ≤ 0, which follows from that bx ∈ (0, x), b m x = b m-1 (bx) ∈ (0, x) for each x ∈ [x, x b )
, and ( 22). Furthermore, one can show that

∆ 2 (x) := x 1-α 1 -α -λx -g + ∂h(x) ∂x ax γ ≤ 0 (25)

Inria

Infinite Horizon Optimal Impulsive Control Theory with Application to Internet Congestion Control11

for each x ∈ [x, x/b), which follows from the following observations. Since h

(x) = h 1 (x) = h 0 (bx), we see ∆ 2 (x) = -(b 2-α-γ -1)x 1-α 1-α +λ(b 2-γ -1)x+g(b 1-γ -1) for each x ∈ [x, x b
), and in particular,

∆ 2 (x) = 0, (26)
as can be easily verified. The derivative of the function ∆ 2 (x) with respect to x is given by

∆ ′ 2 (x) = -(b 2-α-γ -1)x -α + λ(b 2-γ -1). If 2 -γ -α < 0, then ∆ ′ 2 (x) < 0, which together with (26) shows ∆ 2 (x) ≤ 0 on [x, x b ). If 2 -γ -α > 0, then ∆ ′′ 2 (x) = α(b 2-α-γ -1)
x -α-1 < 0 and thus, the function ∆ 2 (x) is concave with the maximum attained at the stationary point

x = 1-b 2-α-γ (1-b 2-γ )λ 1 α . Since 1-b 2-α-γ (1-b 2-γ )λ 1 α ≤ x, (26) implies ∆ 2 (x) ≤ 0 on [x, x b
), as desired. By the way, for the later reference, the above observations actually show that

G(x) := - (b 2-α-γ -1)x 1-α 1 -α + λ(b 2-γ -1)x + g(b 1-γ -1) ≤ 0 (27)
for all x ≥ x. Thus, combining (23), (24), and (25) shows that Condition 1(ii,iii) is satisfied on

[x, x b ). Assume that for each x ∈ [ x b k-1 , x b k
) and each k = 1, 2, . . . , M, relations ( 23) and (25) hold, together with h(b k x)h(x) = 0 (the corresponding version of (24)).

(28)

Now we consider the case of k = M +1, i.e., when

x ∈ [ x b M , x b M +1 ). For each x ∈ [ x b M , x b M +1 ), when m = 1, 2, . . . , M, it holds that b m x ∈ [ x b M -m , x b M +1-m ), and thus h(b m x) -h(x) = h 0 (b M+1 x) - h 0 (b M+1 (x)) = 0; when m = M + 1, M + 2, . . . , b m x ∈ (0, x) = L * , and thus h(b m x) -h(x) = h 0 (b m x) -h 0 (b M+1 x) = 0 if m = M + 1, and h(b m x) -h(x) = h 0 (b m-(M+1) (b M+1 x)) - h 0 (b M+1 x) ≤ 0 if m > M + 1, by (22). Thus, we see (23) holds for x ∈ [ x b M , x b M +1
). Note that in the above we have also incidentally verified the validity of (28) for the case of k = M + 1.

Below we verify (25) for the case of k = M + 1, which would complete the proof by induction. To this end, we first present some preliminary observations that hold for each k = 1, 2, . . . . For each k = 1, 2, . . . , since h

(x) = h k (x) = h 0 (b k x) for each x ∈ [ x b k-1 , x b k ), we have ∆ 2 (x) := - (b k(2-α-γ) -1)x 1-α (1 -α) + λx(b k(2-γ) -1) + g(b k(1-γ) -1).
For the convenience of later reference, let us introduce the notation

∆k (x) := - (b k(2-α-γ) -1)x 1-α 1 -α + λ(b k(2-γ) -1)x + g(b k(1-γ) -1) = b k(1-γ) (- b k(1-α) x 1-α 1 -α + λb k x + g) -(- x 1-α 1 -α + λx + g) for each x > 0. Therefore, for x ∈ [ x b k-2 , x b k-1 ), we have ∆ 2 (x) = ∆k-1 (x) = b (k-1)(1-γ) (- b (k-1)(1-α) x 1-α 1 -α + λb k-1 x + g) -(- x 1-α 1 -α + λx + g).
Let us define

F (x) := - x 1-α 1 -α + λx + g RR n°8403
for each x > 0. We then have from the direct calculations that

∆k-1 ( x b k-2 ) = b (k-1)(1-γ) F (bx) -F ( x b k-2 ) ( 29
)
for each k = 1, 2, . . . . Focusing on F ( x b k-2 ), we have b 1-γ F ( x b k-2 ) = - x1-α 1 -α b 1-γ b (k-2)(1-α) + λx b 1-γ b k-2 + gb 1-γ = - x1-α 1 -α b 2-α-γ b (k-1)(1-α) + λx b 2-γ b k-1 + gb 1-γ = - ( x b (k-1) ) 1-α 1 -α b 2-α-γ + λ( x b k-1 )b 2-γ + gb 1-γ .
Recall that in the above, we have proved that G(x) ≤ 0 for x ≥ x, see (27). Thus, we have

G( x b k-1 ) ≤ 0, i.e., - b 2-α-γ ( x b k-1 ) 1-α 1-α + λb 2-γ ( x b k-1 ) + gb 1-γ ≤ - ( x b k-1 ) 1-α 1-α + λ( x b k-1 ) + g. Conse- quently, b 1-γ F ( x b k-2 ) ≤ - ( x b k-1 ) 1-α 1 -α + λ( x b k-1 ) + g = F ( x b k-1 ).
Now we verify (25) for the particular case of k = M + 1. By the inductive supposition, (25

) holds for x ∈ [ x b M -1 , x b M ), we thus have ∆ 2 ( x b M -1 ) ≤ 0, and
0 ≥ ∆M ( x b M-1 ) = b M(1-γ) F (bx) -F ( x b M-1 ) ≥ b (M)(1-γ) F (bx) - 1 b 1-γ F ( x b M ).
Therefore, we obtain that b (M+1)(1-γ) F (bx) -F ( x b M ) ≤ 0, and by (29),

∆ 2 ( x b M ) ≤ 0. ( 30
)
Furthermore, the derivative of the function ∆M+1 (x) with respect to x is given by ∆′

M+1 (x) = -(b (M+1)(2-α-γ) -1)x -α + λ(b (M+1)(2-γ) -1). If 2 -γ -α < 0, then ∆′ M+1 (x) < 0. Thus, by (30), we obtain that ∆ 2 (x) = ∆M+1 (x) ≤ 0 for x ∈ [ x b M , x b M +1 ). If 2 -γ -α > 0, then ∆′′ M+1 (x) = α(b (M+1)(2-α-γ) -1)
x -α-1 < 0, and in turn, the function ∆M+1 (x) is concave with the maximum attained at the stationary point (2-γ) . From this we see

x = 1-b (M +1)(2-α-γ) (1-b (M +1)(2-γ) )λ 1 α . Moreover, we have M m=0 b m(2-α-γ) M m=0 b m(2-γ) ≤ 1 b M α , which follows from the fact that for each m = 0, 1, . . . , M, m(2 -α -γ) + M α ≥ m(2 -γ), so that b m(2-α-γ) b Mα ≤ b m
(1 -b 2-α-γ ) M m=0 b m(2-α-γ) (1 -b 2-γ ) M m=0 b m(2-γ) ≤ 1 b Mα 1 -b 2-α-γ 1 -b 2-γ ⇒ 1 -b (M+1)(2-α-γ) (1 -b (M+1)(2-γ) )λ ≤ 1 b Mα (2 -γ)(1 -b 2-α-γ ) (2 -α -γ)(1 -b 2-γ )λ ⇔ 1 -b (M+1)(2-α-γ) (1 -b (M+1)(2-γ) )λ ( 1 α ) ≤ x b M .
Finally, it follows from the last line of the previous inequalities, the concavity of the function ∆M+1 and (30

) that ∆ 2 (x) ≤ 0 for x ∈ [ x b M , x b M +1
), which verifies (25), and thus completes the proof.

The case of γ = 1 can be similarly treated.

Inria

Solving the discounted optimal impulsive control problem for the Internet congestion control

The discounted problem turns out more difficult to deal with, and we suppose the sending rate increases additively, i.e., dx k (t) dt = a k > 0, and decreases multiplicatively, i.e., j(x, v) = bx with b ∈ (0, 1) when a congestion notification is sent, see [START_REF] Miller | Impulsive control in continuous and discrete-continuous systems[END_REF] and [START_REF] Mo | Fair end-to-end window-based congestion control[END_REF]. Furthermore, we assume α ∈ (1, 2).

Similarly to the average case, upon rewriting the objective function in problem [START_REF] Noirie | Semantic networking : Flow-based, traffic-aware, and self-managed networking[END_REF] as

Lρ (x 1 , . . . , x n ) = n k=1 lim inf T →∞ T 0 e -ρt x 1-α k (t) 1-α -λ k x k (t) dt, where λ k = l∈P (k) λ l
, it becomes clear that there is no loss of generality to focus on the case of n = 1 ;

Jρ (x 0 ) = lim inf T →∞ T 0 e -ρt x 1-α (t) 1 -α -λx(t) dt → max T1,T2,... , (31)
Now the Bellman equation ( 9) has the form

max x 1-α 1 -α -λx -ρW (x) + a dW dx , sup i≥1 [W (b i x) -W (x)] = 0. (32)
The linear differential equation

x 1-α 1 -α -λx -ρ W (x) + a d W dx = 0 (33)
can be integrated :

W (x) = e ρ a (x-1) λ ρ + λa ρ 2 + 1 ρ(α -1) + w1 - x 1-α ρ(α -1) - λ ρ x - aλ ρ 2 - e ρ a x ρ x 1 e -ρ a u u -α du. ( 34
)
Here w1 = W (1) is a fixed parameter. Suppose for a moment that no impulses are allowed, so that x(t) = x 0 + at. We omit the π index because here is a single control policy. We have a family of functions W (x) depending on the initial value w 1 , but only one of them represents the criterion lim inf

T →∞ T 0 e -ρt (x(t)) 1-α 1 -α -λx(t) dt = W * (x 0 ).
In this situation, for the function W , all the parts of Condition 2 are obviously satisfied (D = ∅, T 1 = ∞, L * = ∅) except for (iv). Since W * < 0, the case lim sup T →∞ e -ρT W * (x(T )) > 0 is excluded and we need to find such an initial value w * 1 that lim

T →∞ e -ρT W (x(T )) = 0, where x(T ) = x 0 + aT, x 0 > 0. (35)
Equation ( 35) is equivalent to the following :

lim T →∞ e ρ a x0 e -ρ a λ ρ + λa ρ 2 + 1 ρ(α -1) + w 1 - 1 ρ x0+aT 1 e -ρ a u u -α du = 0.
Therefore,

w * 1 = e ρ a ρ ρ a α-1 Γ 1 -α, ρ a - 1 ρ(α -1) - λ(ρ + a) ρ 2 , (36)
and W * (x 0 ) is given by (34) at w 1 = w * 1 . Here Γ(y, z) = ∞ z e -u u y-1 du is the incomplete gamma function [19, 3.381-3]. By the way, W * is the maximal non-negative solution to the differential equation (33).

For the discounted impulsive control problem [START_REF] Noirie | Semantic networking : Flow-based, traffic-aware, and self-managed networking[END_REF], the solution is given in the following statement.

Theorem 4 (a) Equation

H(x) := e ρx(1-b) a -1 (1 -b)λa ρ -(1 -b)e ρx a
x bx e -ρu a u -α du (37)

-e ρx(1-b) a -b x 1-α (1 -b 1-α ) α -1 + λx(1 -b) = 0
has a single positive solution x.

(b) Let

w 1 = e ρ a ρ x 1 e -ρu a u -α du - λ(ρ + a) ρ 2 - 1 ρ(α -1) (38)
- x1-α (1 -b 1-α ) ρ(α -1) + (1 -b)λx ρ + e ρbx a ρ x bx e -ρ a u u -α du e ρbx-ρ a -e ρx-ρ a
and, for 0 < x < x, put W (x) = W (x), where W is given by formula (34) under w1 = w 1 . For the intervals x, x b , x b , x b 2 , . . . the function W is defined recursively : W (x) := W (bx). Then the function W satisfies items (i, ii, iii) of Condition 2. (c) The function W (x 0 ) = sup π J ρ (x 0 , π) = J ρ (x 0 , π * ) is the Bellman function, where the (feedback) optimal policy π * is given by

L * = [x, ∞), v f,L * (x) = i if x ∈ x b i-1 , x b i .
Some comments and remarks are in position, before we give the proof of this theorem. For b = 0.5, ρ = 1, α = 1.3, λ = 2, a = 0.2 the graph of function W is presented on Fig. 1. Here x = 0.7901 and w 1 = -4.9301. The dashed line represents the graph of function

z(x) = - 1 ρ x 1-α α -1 + λx . (39)
When W (x) = z(x), we have d

W dx = 0 ; if W (x) > z(x) ( W (x) < z(x)) function W increases (decreases). The dotted line represents the graph of function v(x) = a(x -α -λ) ρ 2 - 1 ρ x 1-α α -1 + λx .
If W (x) = v(x) then from (33) we have

a 2 d 2 W dx 2 = a 2 aρ d W dx + λa -ax -α = a 2 ρ 2 W (x) + ρλx + ρx 1-α α -1 + λa -ax -α = 0, Inria
W (x) = W (bx) and d W (x) dx x = d W (bx) dx x . (40)
As is shown in the proof of Theorem 4, these two equations are satisfied if and only if x solves equation (37).

Let us calculate the limit of x when ρ approaches zero. One can easily show that, for any

x > 0, lim ρ→0 H(x) = 0 and lim ρ→0 H(x) ρ = x 2 (1 -b) a λ(b 2 -1) 2 + x -α (1 -b 2-α ) 2 -α . Let x0 = 2(1 -b 2-α ) λ(1 -b 2 )(2 -α) 1/α , (41)
i.e.

lim ρ→0 H(x) ρ    > 0, if x < x0 , < 0, if x > x0 , = 0, if x = x0 . Function H(x)
ρ is continuous wrt ρ. Therefore, for any small enough ε > 0, ∃δ > 0 : ∀ρ ∈ (0, δ) H(x 0ε) ρ > 0 and H(x 0 + ε) ρ < 0 meaning that xρ , the solution to (37) at ρ ∈ (0, δ), satisfies xρ ∈ (x 0ε, x0 + ε). This means lim ρ→0+ xρ = x0 . Note that (41) is the optimal threshold if we consider the long-run average reward with the same reward rate c(x).

Proof of Theorem 4. Thus, the only possibility to satisfy (40) is the case when the increasing part III of W (x) touches the increasing part I of function W (bx). The inflection line v(x) is located between the increasing and decreasing branches of the function z(x), so that the part III of W (x) is convex and the part I of W (bx) is concave, meaning that no more than one point x can satsify the equations (40).

Using formula (34), the equations (40) can be rewritten as follows : After we multiply these equations by factors 1be (b-1) ρx a and a ρ 1e (b-1) ρx a correspondingly and subtract the equations, the variable w1 is cancelled and we obtain equation

0 = W (x) -W (bx) = e
0 = 1 -be (b-1) ρx a b 1-α x 1-α ρ(α -1) + e bρx a ρ bx x e -ρu a u -α du - x 1-α ρ(α -1) -(1 -b) λx ρ - a ρ 1 -e (b-1)ρx a be bρx a a bx x e -ρu a u -α du -(1 -b) λ ρ
which is equivalent to H(x) = 0. Equation (38) follows directly from the first of equations (42) : if we know the value of x (equal x), we can compute the value of w1 = w 1 .

Inria

Infinite Horizon Optimal Impulsive Control Theory with Application to Internet Congestion Control17

To prove the solvability of the equation (37) we compute the following limits :

lim x→∞ H(x) ≤ -lim x→∞ e ρx(1-b) a • λx(1 -b) = -∞; lim x→0 H(x) = lim x→0 (b -1) x 1-α (1 -b 1-α ) α -1 + x bx e -ρu a u -α du ,
and the positive expression in the square brackets does not exceed

x 1-α (1 -b 1-α ) α -1 + x bx 1 - ρu a + 1 2 ρu a 2 u -α du = x 1-α (1 -b 1-α ) α -1 + u 1-α 1 -α - ρu 2-α a(2 -α) + ρ 2 u 3-α 2a 2 (3 -α) x bx = ρ 2 u 3-α 2a 2 (3 -α) - ρu 2-α a(2 -α) x bx → 0 as x → 0, so that lim x→0 H(x) = 0.
Finally,

dH dx = - ρ(1 -b) a e ρx(1-b) a x 1-α (1 -b 1-α ) α -1 + λx(1 -b) + b -e ρx(1-b) a λ(1 -b) -x -α (1 -b 1-α ) + λ(1 -b) 2 e ρx(1-b) a - ρ a (1 -b)e ρx a x bx e -ρu a u -α du -(1 -b)e ρx a e -ρx a x -α -be -ρbx a (bx) -α = - ρ(1 -b)(1 -b 1-α ) a(α -1) x 1-α -b -1 - ρx(1 -b) a (1 -b 1-α )x -α -λ(1 -b) +λ(1 -b) 2 - ρ a (1 -b) x 1-α (1 -b 1-α ) 1 -α -(1 -b) 1 + ρx a x -α 1 - ρx a -b 1 - ρbx a (bx) -α + O(x),
where lim x→0 O(1) = 0 ; so

lim x→0 dH dx = lim x→0 ρ(1 -b)(1 -b 2-α ) a x 1-α = +∞
meaning that the continuous function H(x) increases from zero when x ≈ 0 and becomes negative for big values of x. Therefore, equation (37) has a single positive solution x. (b) Item (i) of Condition 2 is obviously satisfied (D = ∅).

For Item (ii), we consider the following three cases. (α) Let 0 < x ≤ x. The differential equation (33) holds for function W on the interval 0 < x < x. For these values of x,

for any i ≥ 1, W (b i x) < W (x). ( 43
)
To prove this, note that function W (bx) = W (bx) is increasing (Fig, 1), so that W For x ∈ (x, x 1 ) the function W (x) is concave ; hence

a dW dx x < a dW dx x = a d W dx x = ρ W (x) - x1-α 1 -α + λx = ρ[ W (x) -z(x)].
(See formula (39).) Since W (x) increases starting from W (x) = W (x) and z(x) decreases, we have dW dx The Bellman equation ( 32) is satisfied for all x > 0. Item (iii) of condition 2 is also obviously satisfied :

x < ρ[W (x) -z(x)] = ρW (x) - x 1-α 1 -α -λx ,
L * = [x, ∞); v f,L * (x) = v i if x ∈ x b i-1 , x b i .
(c) Note that item (iv) of Condition 2 is not satisfied. Indeed, there is an admissible control such that, on any time interval (T -1, T ], x π (t) is so close to zero that e -ρT W (x π (t)) < -1. (Remember that lim x→0 W (x) = -∞.)

Let us fix an arbitrary x 0 > 0 and modify the reward rate :

ĉ(x) = c(x), if x ≥ min{x 0 , bx} := x; c(x), if x < x.

Note that ĉ ≥ c. The function W (x) given by (34) will change only for x < x ≤ x 0 and remains increasing in its part I, meaning that this modified function Ŵ satisfies all items (i)-(iii) of Condition 2 : the proof is identical to the one presented above. But now Condition 2 (iv) is also satisfied because the function Ŵ is bounded. Therefore, according to Theorem 2, sup π Ĵ(x 0 , π) = Ŵ (x 0 ) = Ĵ(x 0 , π * ), where Ĵ corresponds to the reward rate ĉ. But sup π J ρ (x 0 , π) ≤ sup π Ĵ(x 0 , π) = Ŵ (x 0 ) = W (x 0 ), and for the feedback policy π * , which is independent of x 0 , we have W (x 0 ) = Ŵ (x 0 ) = Ĵ(x 0 , π * ) = J ρ (x 0 , π * ).

The last equality holds because, under the feedback policy π * , starting from x 0 , the trajectory x π * (t) satisfies x π * (t) ≥ x for all t ≥ 0, and in this region ĉ = c.

Remark 1

The above two theorems assert that if the sending rate is smaller than x, then do not send any congestion notification, while if the sending rate is greater or equal to x, then send (multiple, if needed) congestion notifications until the sending rate is reduced to some level below x with x given by [START_REF] Zhang | Convergence of trajectories and optimal buffer sizing for MIMD congestion control[END_REF] under the average criterion and by Theorem 4(a) under the discounted criterion. This defines our proposed threshold-based AQM scheme.

Conclusion

To sum up, in this paper, we studied optimal impulsive control problems on infinite time interval with both discounted and time average criteria. We have established Bellman equations and provided conditions for the verification of canonical triplet. Our general results are then applied to construct a novel AQM scheme, which takes into account not only the traffic transiting through the bottleneck links but also the congestion control algorithms operating at the edges of the network. We are currently working on practical aspects of the proposed scheme and its validation. Preliminary results indicate that the new scheme improves fairness significantly with respect to alternative solutions like the RED algorithm.

  2, . . . . Let us consider the case of k = 1, i.e., the interval [x, x b ). By the definition of the function h(x), we have sup m=1,2,... {h(b m x)h(x)} = 0 (23) for x ∈ [x, x/b). Indeed, by the definition of h(x), we have h(bx)h(x) = 0 (24) for x ∈ [x, x/b), whereas for each m = 2, 3, . . . and x ∈ [x, x b ), it holds that h(b

Figure 1 -

 1 Figure 1 -Graph of the Bellman function W (x) (bright line with the star point markers).

Proof 4

 4 (a) Firstly, let us prove that no more than one positive number x can satisfy equations (40). If x satisfies (40) then function W canot have only one increasing branch above function v because two increasing functions W (x) and W (bx) cannot have common points. The increasing part I of function W (x) cannot intersect with W (bx). The strictly decreasing part II of function W (x) cannot intersect with the parts II and III of function W (bx). Possible common points with the part I of W (bx) are of no interest because here d W (x) dx < 0 and d W (bx) dx ≥ 0. The increasing part III of function W (x) can intersect with the parts I and II of function W (bx), but again the latter case is of no interest because here d W (x) dx ≥ 0 and d W (bx) dx < 0.

1 e

 1 -ρu a u -α du -x 1-α (1b 1-α ) ρ(α -1) a u -α du.

  (bx) > W (b 2 x) > . . . . Part III of the function W (x) is convex and function W (bx) touching smoothly W (x) at point x, is concave, so that W (bx) = W (bx) < W (x) here. The same inequality holds for smaller values of x where W (x) decreases (part II) and W (bx) = W (bx) increases. Part I of the function W (x) is obviously bigger than W (bx), too. Thus the Bellman equation (32) is satisfied on the interval 0 < x < x and also on the interval (0, x].(β) Consider x ∈ (x, x/b] and denote x 1 and x 2 the points of the analytical maximum and minimum of the function W (x) = W (bx). (See Fig.1.)

  and the Bellman equation (32) is satisfied because here W (x) = W (bx) = W (bx) and W (b i+1 x) < W (bx) for all i ≥ 1 because of (43).For x ∈ [x 1 , x 2 ] we have W (x) > z(x) and a dW dx ≤ 0 : remember, W (x) = W (bx) and the latter function is of type II forx ∈ [x 1 , x 2 ]. Therefore, again a dW dx ρ W (x) -1 ρ x 1-α 1α λx < 0and the Bellman equation (32) is satisfied. For x ∈ (x 2 , x/b], we have W increases here and b ∈ (0, 1). Next,ρ[W (x)z(x)] = ρ[ W (bx)z(x)] > ρ[ W (bx)z(bx)]because the function z(x) decreases. Therefore,a dW dx ρ[W (x)z(x)] < d W dx bx ρ[ W (bx)z(bx)] = 0 :bx ≤ x, and, for these values, equation (33) holds. We see that the Bellman equation (32) is satisfied.(γ) Supposea dW dx ρ[W (x)z(x)] < 0 for x ∈ x b i-1 , x b i , for some natural i ≥ 1. Then, for x ∈ x b i , x b i+1 , we have a dW dx ρ[W (x)z(x)] = ba dW dx bx ρ[W (bx)z(x)].If dW dx bx < 0 then the last expression is negative. Otherwise, ρ[W (x)z(x)] < a dW dx bx ρ[W (bx)z(bx)] < 0 Inria by the induction supposition.

RR n°8403

Without loss of generality, we assume that each link is occupied by some connection, and each connection is routed through some link.RR n°8403

route des Lucioles -BP 93 06902 Sophia Antipolis Cedex Publisher Inria Domaine de Voluceau -Rocquencourt BP 105 -78153 Le Chesnay Cedex inria.fr ISSN 0249-6399

Acknowledgement

This work is partially funded by INRIA Alcatel-Lucent Joint Lab, ADR "Semantic Networking".