Aurélien Cocq

Bastien Léger

Sébastien Noël

Hervé Bricout

Florence Djedaïni-Pilard

Sébastien Tilloy

Eric Monflier

Anionic Amphiphilic Cyclodextrins bearing Oleic Grafts for the Stabilization of Ruthenium Nanoparticles Efficient in Aqueous Catalytic Hydrogenation

Oleic succinyl β-cyclodextrin was proved to be efficient for the stabilization of ruthenium nanoparticles (NPs) in aqueous medium. These NPs were characterized by FTIR spectroscopy and transition electron microscopy (TEM). The catalytic activity of these NPs was evaluated in the aqueous hydrogenation of petrosourced and biosourced unsaturated compounds such as benzene and furfural derivatives. The catalytic system can be easily recycled and reused up to nine runs without any loss of activity and selectivity, demonstrating its robustness.

Transition metal nanoparticles (NPs) are widely used in the field of catalysis. [1][2][3] This kind of catalytic system provides high active surface area and can be dispersed in water by using capping agents. [4] Among the described water-soluble capping agents, cyclodextrins (CDs) are efficient biosourced molecules. CDs are water-soluble chemical receptors consisting of six (α-CD), seven (β-CD) or eight (γ-CD) 1,4-linked α-D-glucopyranose units. [5] CDs are able not only to include organic substrates into their hydrophobic cavity but CDs can also interact with the metallic nanoparticles surfaces. They have been employed under various forms and environments such as free form (native or modified CD), inclusion complex form with appropriate guests molecules (ionic surfactants, phosphanes), or in combination with polymer matrices (physically or chemically incorporated). [6] In this context, easy accessible oleic succinyl-cyclodextrins were considered as eco-compatible capping agents. [7] Indeed, CDs and oleic acid derivatives are industrially produced from renewable resources. In addition, this kind of structure contains different functional moieties such as carboxylate function, oleic succinyl chain and CD ring which could favor both electrostatic and steric stabilizations of NPs in water. The presence of the hydrophobic cavity of the CD could also promote the meeting between hydrophobic substrate and NPs. In this communication, we report that oleic succinyl-β-cyclodextrin (OS--CD) (Scheme 1) was efficient to stabilize ruthenium nanoparticles. These NPs were characterized by FTIR spectroscopy and transition electron microscopy (TEM). The catalytic activity of these NPs was evaluated in the aqueous hydrogenation of petrosourced (alkyl olefins and benzene derivatives) and biosourced unsaturated compounds (furfural derivatives, methyl oleate and citronellal).

The recyclability of the catalytic system was also evaluated in the case of furfuryl alcohol as substrate.

The synthesis of OS-β-CD was carried out in two steps according to a previously described method. [7] Briefly, the first step consists in the alkenylation of maleic anhydride by oleic acid to produce an oleic succinic anhydride (OSA). The second step involves the grafting of the OSA on β-CD followed by a treatment with an aqueous sodium hydroxide solution to give the carboxylate functions. The degree of substitution is equal to 1.6, i.e. 1.6 hydroxyl group on 21 was functionalized by the oleic-succinyl group. In Scheme 1, in order to simplify this representation, only one kind of graft was presented, among the two possible, knowing that each hydroxyl groups of the β-CD can react with one or the other carbonyl group of the succinic anhydride. The watersolubility and the critical aggregation concentration of OS-β-CD were equal to 286 g.L -1 and 170 mg.L -1 , respectively (at 20°C). For the synthesis of ruthenium nanoparticules stabilized by OS-β-CD (Ru_OS-β-CD NPs), the ruthenium salt Ru(NO)(NO3)3 (1 equiv.) and OS-β-CD (3 equiv.) were solubilized in water and stirred for 30 min at room temperature before addition of sodium borohydride (10 equiv.) as a reducing agent. Note that the 3 equivalents of OS-β-CD corresponds to almost 5 equivalents of oleic graft and therefore to 10 equivalents of carboxylate functions with respect to ruthenium. The color of the aqueous solution changed from orange to dark brown once sodium borohydride was added, illustrating the reduction of Ru(III) and the formation of Ru NPs. The resulting colloidal suspension was kept under stirring 18 h to permit the total transformation of NaBH4 excess into sodium metaborate (NaBO2) by hydrolysis. An interaction of NaBO2 with the metal nanoparticles is possible. The reduction of the metal precursor has been studied by ATR-FTIR spectroscopy. Fig. 1 plots the ATR-FTIR spectra of lyophilized samples corresponding to ruthenium nitrosyl nitrate (Fig. 1a), ruthenium nitrosyl nitrate after the addition of sodium borohydride (Fig. 1b), OS-β-CD (Fig. 1c) and Ru_OS-β-CD NPs (Fig. 1d).

[a] Dr. A. The spectrum of Ru(NO)(NO3)3 showed absorption bands at 1270 and 1840 cm -1 corresponding respectively to the nitrate (NO3) and nitrosyl (NO) stretching bands (Fig. 1a). After addition of NaBH4 to a Ru(NO)(NO3)3 aqueous solution and stirring at room temperature during 18h, the bands at 1270 and 1840 cm -1 , related to NO3 -and NO moieties, disappear in the corresponding spectrum (Fig. 1b). This modification can be explained by the simultaneous reduction of Ru(III) and NO3 -, followed by NO leaching. [8] However, it should be noticed that in this case, no stable NPs were formed since a sedimentation of black ruthenium in water was observed. In Figure 1c, OS-β-CD IR spectrum reveals bands coming from the vibrational modes of the glucosyl units carbon-oxygene single bonds [930-1150 cm -1 ]. Stretching bands characteristic of carboxylate anionic groups [1550 cm -1 ] and of carbonyl group of the ester moiety [1740 cm -1 ] are also clearly observed, together with a very weak signal due to the carbon-carbon double bond of the graft [around 1680 cm -1 ]. At higher wavenumbers, stretching bands of C-H bonds [2800-3020 cm -1 ] and of the hydroxyl groups [3300 cm -1 ] are also present. The spectrum of Ru_OS-β-CD NPs also shows the disappearance of nitrate and nitrosyl bands due to the reduction of the metallic precursor into NPs (Compare Fig. 1d to Fig. 1a). Moreover, no modification of the spectral features of the OS--CD was observed, proving that its structure was not modified during the synthesis of the NPs. Ru_OS-β-CD NPs were also characterized by transmission electron microscopy (TEM). The TEM image revealed no aggregation of metal nanoparticles. Ru_OS-β-CD NPs were homogeneously dispersed with a narrow size distribution centered at 2.56 ± 0.60 nm with 80% of the NPs having a diameter between 2 and 3 nm (Fig. 2a and2b). The zeta potential of Ru_OS-β-CD NPs was measured and equal to -41 mV. This value was in good agreement with zeta potential values measured for NPs stabilized in the presence of carboxylate functions (between -40 mV and -60 mV). [9] This value is also in correlation with good stability of the colloidal suspension in the aqueous phase. As a control, Ru_β-CD NPs have been synthesized from Ru(NO)(NO3)3 and native β-CD to study the influence of the grafting on the cyclodextrin backbone on the dispersion and the size of Ru NPs. The TEM image of Ru_β-CD NPs (Fig. 2c and2d) clearly shows that metal nanoparticles are organized into nonordered superstructures with a mean diameter of 3.97 ± 0.86 nm. This kind of organization has also been reported in the literature for Ru NPs stabilized with methylated cyclodextrins. [10] Moreover, the size distribution is less narrow in the case of Ru_β-CD NPs than in the case of Ru_OS-β-CD NPs and two different populations can be distinguished, one centered on 3.5 nm and another one centered on 6 nm. This TEM study proves the beneficial effect of the grafting of the oleic unit on β-CD on the size and the dispersion of Ru NPs in the aqueous phase.

The catalytic activity of the Ru_OS-β-CD NPs was first evaluated in the aqueous hydrogenation of styrene as model substrate for control experiments. The catalytic experiments were performed at a temperature of 30°C, under a hydrogen pressure of 10 bar and with a styrene/ruthenium molar ratio equal to 100. Some control experiments were done under the same reaction conditions always considering the same amount of ruthenium salt and NaBH4 as described for the Ru_OS-β-CD NPs synthesis. More precisely, Ru colloidal suspensions stabilized either by oleic acid sodium salt, -CD alone or by an oleic acid sodium salt/-CD physical mixture have been synthesized and tested.

Table 1. Hydrogenation of styrene in the presence of Ru NPs [a] Entry Stabilizer Time (min)

Conv. (%) [b] NP Stability

Before catalysis

After catalysis

1 (-) 360 <1 No No 2
Oleic acid sodium salt [c] 360 <1 No No 3 β-CD [d] 60 100 [g] Yes No 4 Oleic acid sodium salt + β-CD [e] 60 <1 No No 5 OS-β-CD [f] 60 100 [g] Yes Yes [a] Reaction conditions: Ru (1×10 -5 mol), Styrene (1×10 -3 mol), hydrogen pressure (10 bar), temperature (30 °C), stirring rate (750 rpm), 12 mL water. [b] Conversion of styrene determined by GC analysis with external standard (decane), [c] Oleic acid sodium salt (5×10 -5 mol). [d] -CD (3×10 -5 mol). [e] Prepared by mixing -CD (3×10 -5 mol) and oleic acid sodium salt (5×10 -5 mol). [f] OS--CD (3×10 -5 mol). [g] yield of 100% in ethylcyclohexane.

Without stabilizer or in the presence of oleic acid sodium salt, the conversion of styrene was non-significant (Table 1; entries 1 and 2). This could be explained by a lack of stability before the catalytic test. In the case of β-CD alone as protective agent, the styrene was completely converted into ethylcyclohexane within 1h (Table 1, entry 3). However, the Ru NPs were unstable and precipitated during the reaction. The experiment performed in the presence of a physical mixture of oleic acid sodium salt and β-CD gave no conversion and the catalytic system became unstable during the hydrogenation (Table 1, entry 4). The experiment performed in the presence of OS-β-CD as stabilizer led to a total conversion of styrene into ethylcyclohexane within 1h and with a colloidal suspension perfectly stable at the end of the reaction (Table 1,entry 5). Thanks to a kinetic monitoring, it should be noticed that styrene was completely converted into ethylcyclohexane within two successive reaction steps (see Supporting Information). It means hydrogenation of styrene into ethylbenzene followed by ethylbenzene into ethylcyclohexane. This last experiment undoubtedly showed that the catalytic system based on Ru_OS-β-CD NPs was efficient and stable. As the oleic arm of OS-β-CD contains a carbon-carbon double bond, the stability of OS-β-CD has been evaluated during the catalytic experiment. It is important to underline that after 18h at 30°C under 10 bar of hydrogen pressure, the carbon-carbon double bond of the graft was not reduced as evidenced by NMR experiments (see Supporting Information).

Table 2. Hydrogenation of various substrates in presence of Ru_OS-β-CD NPs [a] Entry Substrate Products Time (min) [b] 1 [a] Reaction conditions: Ru (1×10 -5 mol), OS-β-CD (3×10 -5 mol), substrate (1×10 - 3 mol), hydrogen pressure (10 bar), temperature (30 °C), stirring rate (750 rpm), 12 mL water; [b] Time to get the final product with a yield of 100% determined by GC analysis with external standard (decane, except dodecane for 1-decene).

To extend this catalytic study, the hydrogenation of other substrates was investigated. All the catalytic results are summarized in table 2 and the final products correspond to the fully saturated substrates. In the case of alkenes with terminal double bonds, the reaction time significantly increased with the alkyl chain length (Table 2, entries 1-3). This tendency was already described in the case of the aqueous hydrogenation of terminal olefins catalyzed by metallic NPs assisted by CDs. [11] In the presence of Ru_OS-β-CD NPs, anisole, styrene, allylbenzene, benzaldehyde and benzyl alcohol were completely converted into their fully saturated equivalent after 80, 60, 150, 1140 and 780 minutes, respectively (Table 2, entries 4-8). For styrene, allylbenzene and benzaldehyde, the reduction took place in two consecutive steps. The C=C (vinyl or allyl) or C=O bond were firstly reduced followed by the reduction of the aromatic ring. As an example, the figure 3 depicted the reaction profile of the hydrogenation of benzaldehyde. This profile illustrates the sequential nature of the reaction as benzaldehyde was firstly hydrogenated into benzyl alcohol (99% of yield into 170 minutes), which is subsequently hydrogenated into hydroxymethylcyclohexane (100% of yield into 1140 minutes). In the case of benzaldehyde and benzyl alcohol, the catalytic activities were very low compared to other aromatic substrates (Table 2, compare entries 4-6 to 7-8). The reaction profiles relative to styrene and allylbenzene were presented in the Supporting Information. The efficiency of these NPs was also evaluated on substrates issued from biomass. The catalytic system based on Ru_OS-β-CD NPs was inefficient for the reduction of methyl oleate (Table 2,entry 9). This result is in accordance with the fact that the carbon-carbon double bond of OS-β-CD oleic arm was not reduced under these reaction conditions. The reduction of (±) citronellal, a monoterpenoid present in the citronella oil, was also considered. Citronellal can be reduced into citronellol, 3,7dimethyloctan-1-al and 3,7-dimethyloctan-1-ol (the fully saturated product). Contrary to the previous examples, the different steps of reduction took place simultaneously and maximum yields of citronellol and 3,7-dimethyloctan-1-al were reached after 90 minutes (25% and 7%, respectively). Then, the both percentages decreased and 3,7-dimethyloctan-1-ol was mainly produced (Fig. 4). The hydrogenation of furfural and furfuryl alcohol (produced from plant materials rich in pentosans) was also evaluated. The catalytic activity was better in the case of furfuryl alcohol than furfural (Table 2, entries 11 and 12). This behaviour has already been described and seems logical since in the case of furfural, the carbonyl double bond has to be also reduced. From a kinetic monitoring point of view, the hydrogenation of furfural into tetrahydrofurfuryl alcohol takes place in two consecutive steps (furfural into furfuryl alcohol and then furfuryl alcohol into tetrahydrofurfuryl alcohol; see Supporting Information). This reaction pathway has also been previously reported in the literature in the case of Ru NPs stabilized by an anionic cyclodextrin polymer. The recyclability of the catalytic system was also investigated by reusing the aqueous catalytic phase for successive furfuryl alcohol hydrogenation runs (Fig. 5). In this case, the furfuryl alcohol /ruthenium molar ratio is equal to 50 and the hydrogen pressure to 40 bar in order to reduce the reaction time. After the first run (30 min), the product of the reaction was extracted with chloroform. Although the OS-β-CD concentration was 25 times higher than its critical aggregation concentration, no formation of emulsion was observed. This point is essential to facilitate an easy separation between the aqueous and the organic phases.

After decantation and chloroform removal, the aqueous suspension was reloaded with furfuryl alcohol and dihydrogen in the autoclave and reused under the same experimental conditions. The Ru_OS-β-CD NPs can be recycled and reused for at least 9 consecutive runs with similar yields (between 85-100 %) without any loss of the stability of the colloidal suspension.

The stability of Ru_OS-β-CD NPs was confirmed by TEM experiments performed after the ninth run. The TEM image revealed that NPs were still homogeneously dispersed with a slightly larger size distribution centered at 3.43 ± 0.62 nm (compared to 2.56 ± 0.60 nm before catalysis experiment) and with 75% of the NPs with size ranging from 2 to 3.5 nm (see Supporting Information). Additionally, other experiments were performed to check the absence of ruthenium in the extracted organic phase. To this end, the organic phase extracted from the first and the ninth runs were reused at 30°C under 40 bar of H2 with fresh furfuryl alcohol. After 3 hours, the analysis of organic phase by GC indicated no conversion of the fresh furfuryl alcohol in the both cases. This absence of conversion showed that no leaching of active ruthenium nanoparticles in the organic phase occurred during the extraction step. These results are very impressive since up to now only a CD/polymer combination could allow such a good stability and good recyclability of a nanoheterogeneous catalytic system. [6c,11] Indeed, in the case of Ru NPs stabilized by a free form CD (methylated β-CD), a destabilization of the catalytic system, was observed after the fifth recycling run. [10b] In conclusion, we showed that oleic succinyl-β-cyclodextrin was efficient for the synthesis of catalytically active ruthenium nanoparticles. The TEM image revealed that Ru_OS-β-CD NPs were homogeneously dispersed with a narrow size distribution centered on 2.56 nm. These NPs were efficient for the hydrogenation of olefins, aromatic and biosourced substrates. In the case of furfuryl alcohol, the catalytic system can be easily recycled and reused for several consecutive runs demonstrating the robustness of this catalytic system. These preliminary results are very promising since this is the first free form CD able to efficiently stabilize Ru NPs during recycling experiments. So, this easily synthesized oleic succinyl-β-cyclodextrin could find other applications by varying the nature of the transition metal or considering another catalytic reaction.

Experimental Section

Chemicals and reagents

Native -cyclodextrin, abbreviated as -CD, was kindly supplied by Roquette Frères (Lestrem, France). All chemicals were purchased from Aldrich Chemicals and Acros Organics and were used without further purification. Ruthenium nitrosyl nitrate (1.5 wt% of Ru aqueous solution) was supplied by Strem Chemicals.

Characterization techniques

Transmission Electron Microscopy (TEM) was performed on a Tecnai microscope (200 kV). The sample into powder was deposited onto the carbon coated copper grid. All the TEM images are gathered in Supporting Information. Metal particle size distributions have been determined from the measurement of 199 particles for Ru_OS-β-CD NPs, 174 particles for Ru_OS-β-CD NPs after the ninth catalytic run and 207 particles for Ru_β-CD NPs. The nanoparticles were found in arbitrarily chosen area of the images using the program ImageJ software. Zeta potential measurements were carried out in water suspension using a Malvern Zetasizer Nano ZS at a controlled temperature (25±0.1°C). The -potential corresponds to the potential difference between the dispersion medium and the electrical double layer of the fluid attached to the dispersed particles. The measurements are based on a Laser Doppler electrophoretic mobility of the heterogeneous catalyst via the Helmholtz-Smoluchowski equation  = (η/ε)μe, where μe is defined as the ratio between the velocity of the heterogeneous catalyst and the magnitude under the applied electric field, η is the viscosity of the suspending liquid and ε stands for the dielectric conductivity of water. The heterogeneous catalysts were analyzed without any previous treatment. All the samples were analyzed in triplicate using the DTS Software from Malvern Instruments to acquire the phase plot and the zeta potential distribution. Fourier transform infrared spectroscopy (FTIR) experiments were carried out in the 4000-400 cm -1 region with a spectral resolution of 2 cm -1 using a Shimadzu IR Prestige-21 spectrometer equipped with a PIKE MIRacle diamond crystal. The samples were always prepared as follow : inorganic salts, organic compounds were dissolved in HPLC water. To get a final volume for each sample of 12 mL, the required volume of water is added. The solutions or dispersions were kept under vigorous stirring during 18 hours and then lyophilized. The dried samples were finally analyzed by FTIR spectroscopy.

Synthesis of Ru_OS--CD NPs

In a typical experiment, the colloidal suspension was prepared as follows at ambient temperature. 50.5 mg of the OS-β-CD (30 µmol, 3 molar equivalents with respect to ruthenium) were dissolved in 5 mL of HPLC water. Besides this solution, 66.8 mg (10 μmol) of a 1.5 wt% ruthenium nitrosyl nitrate solution (10 µmol of Ru) were diluted in 3 mL of water which was added to the previous solution. The mixture was kept under constant stirring at room temperature for 30 minutes before the addition of 4 mL of 0.025 M sodium borohydride solution (0.1 mmol, 10 molar equivalents with respect to ruthenium). The color of the reaction medium changed upon addition of NaBH4 from orange to dark brown due to the reduction of Ru(III). The resulting colloidal suspension was kept under stirring 18 h before their utilization for the catalytic test to permit the total transformation of NaBH4 excess into NaBO2 by hydrolysis. The final suspension is visually stable for months and no sedimentation is observed.

Catalytic test

All hydrogenation experiments have been performed using a stainless steel autoclave Parker-Autoclave Engineers containing a glass vessel which was charged with 12 mL of standard ruthenium colloidal suspension and the desired substrate amount.

Hydrogen was fed to the system at constant pressure up to the desired pressure. The mixture was heated up to 30 °C with a thermostated oil bath and stirred at 750 rpm. Conversions and selectivities were determined by GC analysis using the calibration method with an external standard.

Recyclability study

The recycling experiments have been performed using furfuryl alcohol as substrate. At the end of the reaction, all organic compounds were extracted using chloroform. After decantation and chloroform removal, the aqueous phase containing the catalyst was reused for another run by reloading with furfuryl alcohol and fed with hydrogen.

Scheme 1 .

 1 Scheme 1. Schematic representation of OS-β-CD.

Figure 1 .

 1 Figure 1. ATR-FTIR spectra of (a) Ru(NO)(NO3)3; (b) Ru(NO)(NO3)3 + NaBH4; (c) OS-β-CD and (d) Ru_OS-β-CD NPs.

Figure 2 .

 2 Figure 2. a) TEM image at a magnification of × 200 000 and b) size distribution of Ru_OS-β-CD NPs ; c) TEM image at a magnification of × 200 000 and d) size distribution of Ru_β-CD NPs.

Figure 3 .

 3 Figure 3. Reaction profile for the hydrogenation of benzaldehyde using Ru_OS--CD NPs. Reaction conditions: Ru (1×10 -5 mol), OS-β-CD (3×10 -5 mol), substrate (1×10 -3 mol), hydrogen pressure (10 bar), temperature (30 °C), stirring rate (750 rpm), 12 mL water.

Figure 4 .

 4 Figure 4. Reaction profile for the hydrogenation of citronellal using Ru_OS--CD NPs. Reaction conditions: Ru (1×10 -5 mol), OS-β-CD (3×10 -5 mol), substrate (1×10 -3 mol), hydrogen pressure (10 bar), temperature (30 °C), stirring rate (750 rpm), 12 mL water.

  [6b]

Figure 5 .

 5 Figure 5. Reusability of the catalytic system Ru_OS-β-CD NPs in the hydrogenation of furfuryl alcohol. Reaction conditions: Ru (1×10 -5 mol), OS-β-CD (3×10 -5 mol), furfuryl alcohol (0.5 ×10 -3 mol), hydrogen pressure (40 bar), temperature (30 °C), stirring rate (750 rpm), 12 mL water, 30 min. Yield in tetrahydrofurfuryl alcohol determined by GC analysis with external standard (decane).

Acknowledgements

This work was performed, in partnership with the SAS PIVERT, within the frame of the French Institute for the Energy Transition (Institut pour la Transition Energétique (ITE) P.I.V.E.R.T.) (http://www.institut-pivert.com) selected as an Investment for the Future (Investissements d'Avenir). This work was supported, as part of the Investments for the Future, by the French Government under the reference ANR-001-01. The TEM facility in Lille (France) is supported by the Conseil Regional des Hauts de France, and the European Regional Development Fund (ERDF). We are grateful to Dr. Ahmed Addad for his contribution in setting up the TEM measurements. Jérémy Ternel, Nicolas Kania, and Dominique Prévost are acknowledged for NMR analysis and technical assistance. PIVERT is acknowledged for the financial support of AC thesis.

Keywords: • Cyclodextrins • Hydrogenation • Metal nanoparticles • Oleic acid • Supramolecular catalysis


Xiangtian Bai

Li Guo

Hongyuan Huo

Jiangshui Zhang

Yi Zhang

Zhao-Liang Li

Rse-Net:Road-shape enhanced neural network for Road extraction in high resolution remote sensing image

Keywords: road extraction, semantic segmentation, deep learning, Convolutional Neural Network, multi-scale

Automatically extracting roads from remote sensing images is increasingly important for autonomous driving and geographic information systems. However, due to factors such as ground objects blocking road boundaries and some roads being narrow and long in highresolution remote sensing images, it is difficult for the current mainstream pixel-level extraction model to guarantee the continuity of roads and the smoothness of boundaries. To solve this problem, this paper designs a neural network Rse-net based on semantic segmentation, using a two-stream semantic segmentation network algorithm to make the network pay more attention to the boundary information of roads and narrow roads. This structure uses shape stream to process road boundary information (roads' shape stream), and processes it in parallel with classic stream. Use the Gated Shape CNN to connect the two streams of roads' shape stream and classic stream, and use the classical stream to optimize the boundary information in the shape stream. At the same time, a multi-scale convolutional attention mechanism is used between the decoder and the encoder to expand the receptive field through large-core attention, and obtain more semantic information without causing too much calculation. Finally, the effectiveness of the proposed network is verified by comparative experiments.

Introduction

With the continuous development of earth observation technology and computer technology, how to extract accurate targets from large-scale, high-resolution remote sensing images has become an important issue. Roads are an important feature in remote sensing images. How to correctly and quickly extract the road is of great significance to the fields of geographic information system, automatic driving and navigation, disaster rescue, road detection and map update.

In recent years, deep learning has been widely used in medicine, image processing, computer vision and other fields. The method based on deep learning can obtain features from a large number of samples, and the use of deep learning to extract targets has higher efficiency than traditional machine learning methods [START_REF] Liu | Main road extraction from zy-3 grayscale imagery based on directional mathematical morphology and vgi prior knowledge in urban areas[END_REF]Song and Civco 2004). In the field of remote sensing, deep learning is mainly embodied in semantic segmentation and extraction of feature categories, target detection and identification of special features, and change detection and analysis of surface change characteristics. [START_REF] Quan | Building Extraction From Remote Sensing Images With DoG as Prior Constraint[END_REF] uses Dog edge information as prior information, and uses an end-to-end network to refine the constraints on the feature map, thereby improving the accuracy of extracting buildings from aerial and satellite remote sensing images. Based on the MD-LinkNeSt network learning algorithm, Gu et al. (2021) used a MD cascade (Cascade Mode) to improve the details of the feature map, and used UAV remote sensing images to extract roads in open-pit mining areas, thereby assisting mining areas to solve intelligent scheduling and unmanned driving. [START_REF] Zang | Landuse mapping for high-spatial resolution remote sensing image via deep learning: A review[END_REF] conducted a comparative study on supervised learning, unsupervised learning, and pixel or object-based learning, and carried out land use mapping research based on the above machine learning based on high-resolution remote sensing images. [START_REF] Chen | Building extraction from remote sensing images with sparse token transformers[END_REF] designed an efficient two-channel transformer structure that can learn long-term dependencies in both space and channel dimensions in labels, and achieved state-ofthe-art accuracy on its benchmark building extraction dataset.

For road information extraction, the semantic segmentation task in deep learning is generally used for road extraction. According to the annotation type and deep learning model, the algorithm can be divided into three categories: unsupervised learning, weakly supervised learning and fully supervised learning.

Unsupervised learning generally utilizes automatic semantic annotation for training. [START_REF] Abdelfattah | PLGAN: Generative Adversarial Networks for Power-Line Segmentation in Aerial Images[END_REF] used an adversarial network to distinguish remote sensing images, and introduced a semantic decoder with embedded vectors as input to produce semantic images, thus realizing unsupervised learning.

Weakly supervised learning uses a small amount of labeled data for training, which can maximize the value of data. [START_REF] Chen | GA-Net: A geometry prior assisted neural network for road extraction[END_REF] designed an efficient dual-channel converter structure, which can learn the long-term dependencies in the space and channel dimensions of the label, thereby improving the recognition accuracy of buildings.

The above two methods cannot guarantee the accuracy of road extraction. In contrast, fully supervised learning is the current mainstream method for road information extraction, which can guarantee the accuracy and efficiency of extraction.

Many algorithms have been developed based on fully supervised learning theory. Lu et al. (2021) proposed the GAMS-NET model for the problem of discontinuous road detection, which obtains the dependence of spatial information from the two dimensions of space and channel, and applies multi-scale residual learning to extract multi-scale features. Li et al. (2021) proposed a CRAE-Net model using a combined spatial attention module and channel attention module. The model can be used to optimize and fuse multi-scale features, so that it retains detailed spatial information in high-resolution remote sensing images, and ensures the extraction of narrow roads and smooth boundaries based on threshold segmentation. [START_REF] Qu | MSPNet: Multi-Scale Strip Pooling Network for Road Extraction from Remote Sensing Images[END_REF] At present, for road extraction, the information extraction of main roads and branch lines based on remote sensing images has achieved good results, but when there are occluders, it is easy to detect road gaps and discontinuities [START_REF] Liu | Survey of Road Extraction Methods in Remote Sensing Images Based on Deep Learning[END_REF]. There are still the following main problems in extracting roads from remote sensing images [START_REF] Lu | GAMSNet: Globally aware road detection network with multi-scale residual learning[END_REF]Li et al. 2021):

(1) Narrow roads are not easy to extract, resulting in disconnected and discontinuous roads. Road extraction needs to fully consider the context semantic information to ensure the continuity of road extraction. Narrow roads have fewer pixels, and the network will lose part of the global semantic information after multiple downsampling, and the connectivity of the road cannot be guaranteed. However, if the feature information extracted by the shallow convolutional layer is used for fusion, the original noise in the high-resolution data will be introduced, which will interfere with the accuracy of narrow road feature extraction in Figure 1.

(2) The occlusion of ground objects leads to imprecise and incomplete extraction of road information. When high-resolution images are acquired, it is inevitable that there will be occlusion by ground objects. For example, the vegetation, public facilities or vehicles on both sides of the road cover part of the road surface, and there are deviations in feature extraction, which will affect the outline of the road, making the road outline area appear jagged in Figure 1. (1) The Rse-net model consists of a decoder-encoder structure. The network can transfer features of different scales and expand the receptive field through skip connections and multi-scale convolutional attention modules [START_REF] Guo | Segnext: Rethinking convolutional attention design for semantic segmentation[END_REF], and obtain more spatial semantic information, enabling it to extract narrow roads. This model is also an end-to-end model, which is applicable to multiple data sources to ensure its flexibility.

(2) We use a two-stream mode to extract road features including the traditional stream (classical stream) and the road boundary stream (roads' shape stream). The dual-stream structure can use the traditional stream and extracted multi-scale features to simplify and optimize the road boundary stream, and perform feature transfer through a gated convolutional layer [START_REF] Takikawa | Gated-scnn: Gated shape cnns for semantic segmentation[END_REF]. Therefore, the road's shape stream pays more attention to the contour information, and finally integrates the contour information and the characteristic information of the classic stream to optimize the road contour.

Finally, experiments are performed on two public datasets including Deep Globe Road dataset and CHN6-CUG dataset. Compared with other algorithms, it has a certain improvement and solves the above two problems to a certain extent. The IOU indicators on the two data sets are 69.27% and 70.63% respectively, and the ablation experiment is performed to verify the role of each module.

Methodology

The Rse-net model is a convolutional neural network model using a two-stream architecture. The modified model mainly refers to the D-LinkNet network designed by [START_REF] Zhou | Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[END_REF]. By expanding the receptive field, the D-Linknet network can properly solve problems such as narrow roads and complexity by integrating multiscale features through the central part while retaining certain contextual semantic information. However, D-Linknet still has problems such as identification errors and the inability to guarantee road connectivity.

The Rse-net model first uses a lightweight Resnet (HE et al. 2016) for preliminary feature extraction in the decoder part. For the characteristics of narrow roads, large spans, and straight shapes, we introduce a multi-scale convolutional attention module (MSCAB). This module is used to arouse the attention between features, increase the weight of the road area, make the model pay more attention to the road area, and suppress the feature extraction of other areas. Use the SPPCSPC (used in the middle of YOLO v7) (Wang et al. 2022) module as an intermediate layer for high-level semantic information extraction. At the same time, we also designed a two-stream architecture, and the road's shape stream can process shape contour information separately. The classical stream is the previous decoder-encoder structure. Dual streams can be processed in parallel. The initial input of the stream structure adopts the features processed by the canny operator in the first layer of feature layer in the classic stream structure. At the same time, we will use a gated convolutional layer (Gated Shape CNN) [START_REF] Takikawa | Gated-scnn: Gated shape cnns for semantic segmentation[END_REF]to connect the two stream structures. This structure can use the high-level semantic information in the classic stream structure to process the lowlevel semantic information of the road's shape stream, which can effectively remove redundant noise, help the road's shape stream to only process contour-related information, and suppress the rest of the feature information. We will adopt a stream structure with a relatively simple structure to deal with the characteristics of the road's shape stream, and prevent the model from having too many parameters while producing clearer road profile information. Finally, the classical stream and the road's shape stream are fused to obtain the final result map. the blue branch is the direction of the classical stream, and the purple branch is the direction of the road's shape stream.

Classical Stream

Considering that the road has a large span, multi-connectivity, occlusion and partial narrowness, the network requires a larger sensing to extract the global information of road. We consider a D-LinkNet network consisting of an encoder-decoder structure, which is robust and fast. We choose ResNet34 as its backbone neural network, which has the advantage of being lightweight and has been pre-trained on ImageNet.

ResNet34 uses a Residual structure (residual structure). Considering that when the network layer depth reaches a certain level, network degradation, gradient disappearance, and gradient explosion will occur, Batch Normalization is used to speed up training. This structure has been proved to have good robustness through a large number of experiments. The resulting features with high-level semantic information are then passed into the central part, which employs the SPPCSPC (Spatial Pyramid Pooling Cross Stage Partial Conv)module. The SPPCSPC module expands the receptive field as much as possible and deepens the network level without changing the feature resolution. This method helps to obtain multi-scale context information, obtain a larger range of continuous roads, and improve the continuity of roads. The processed features are then passed into the decoder part. In the decoder part, we argue that local spatial information is equally important as global spatial information. Using the mechanism of the UNet network, the data obtained by the central layer is continuously up-sampled.

Then the upsampled features of the same scale and the features passed through the MSCAB module are fused. The low-level semantic features extracted by the shallow neural network have more global information, which can improve the connectivity and integrity of the road. However, the noise of high-resolution remote sensing images is introduced, which will cause detailed damage to the road, and the boundary will be affected. The high-level semantic features extracted by the high-level neural network have more local semantic information and protect the details of the road, but lack of global information and cannot guarantee the connectivity of the road. Although the fused features are consistent in size, there will be certain semantic differences between the two after multiple feature extractions. Therefore, simply fusing low-level semantic features with high-level semantic features will lead to information bias. Therefore, we will use the multi-scale convolution attention module to process the shallow features, assign weights to the road area, make the shallow features pay more attention to the road, and then fuse with the deep features, and gradually restore the resolution through upsampling. In the process of upsampling, the multi-scale convolution module is used to process the shallow features extracted by the decoder, and then fused with the upsampled features to obtain features with the same size and resolution as the input image. Finally, the features of Classical Stream and Road's Shape Stream are fused, and the final result is obtained through convolution and Sigmoid activation function. After the image is input into the network, the Resnet network and the Canny operator are used to preprocess and extract the boundary respectively. Input the preprocessed features into the Classical stream, and input the features processed by the Canny operator into the Road's shape stream. Continuous downsampling in the Classical Stream extracts feature pyramids with dimensions of 1/4, 1/8, 1/16, and 1/32, respectively. In each downsampling process, extract a feature and use the MSCAB module for attention processing, and use the skip connection to input to the decoder part. The feature pyramid is input to the decoder through the SPPCSPC module in Dblock, and is fused with the features processed by the attention module, and then the resolution is restored through continuous upsampling. In the process of upsampling, a feature is extracted and processed by the canny operator, and then input into the Road's shape stream. The specific processing method will be introduced in detail in Section 2.2.

Figure 3. Detailed presentation of the improved model. The blue background is the encoder, the red part is the decoder, and the yellow part is the roads' shape stream.

Roads' Shape Stream

The current mainstream road extraction model is to put information such as shape, size, texture and color together for processing. However, this information sometimes interferes with each other, reducing the accuracy of road extraction. Here we refer to a gated convolutional neural network designed by Takikawa (2019) for semantic segmentation.

Therefore, we propose a road's shape stream architecture based on convolutional neural network, which can extract road features in parallel with classical stream. The road's shape stream is a model that only focuses on road contour features. The road's shape stream can use the features extracted by the classical stream to assist in adjusting the features extracted by the road's shape stream, so that the road's shape stream only focuses on the edge of the road. When classical stream processes features, the obtained features are relatively more advanced and have richer semantics. Some of the more advanced features are introduced into the road's shape stream through the gate structure, so that it pays more attention to the boundary information of the road's shape stream.

Since the features of the road's shape stream are relatively low-level, it can also effectively deal with the noise of the features in the classical stream.

The gate structure is composed of Gated Convolutional Layer (GCL), which filters out the features of other parts, so that the road's shape stream only focuses on and processes contour and shape-related information. The principle is that the gated convolutional layer deactivates the low-level semantic features in the road's shape stream and the part that is not related to the high-level semantic features of the classical stream. The low-level semantic features in these road's shape stream are generally color and texture information. After stopping these activations, the road's shape stream will pay more attention to the shape features of the road, so as to achieve the function of extracting the outline and shape of the road. At the same time, the road's shape stream should adopt a very shallow structure and maintain the same resolution as the original image for processing. Finally, the features extracted from the classical stream and the features extracted from the road's shape stream are fused to obtain the road extraction image.

The feature output and image gradient extracted by the decoder in the classic stream are used as the input of the road's shape stream, thereby generating contour information as the output. In order to avoid the burden of parameter calculation on the entire model, the road's shape stream mainly uses residual modules and gated convolutional layers alternately. Ground-truth (GT) is extracted from road labels as binary shape truth. A supervised binary cross-entropy loss function is used on the output boundary to supervise the road's shape stream to ensure that the road's shape stream only focuses on the road's boundary feature information. We use the canny operator to extract the extracted road boundaries from high-resolution remote sensing images.

Then, the road boundary image is up-sampled by bilinear interpolation to become an image with the same size and resolution as the original high-resolution image, and a residual module is used for feature extraction. Then enter the gated convolution layer together with the features extracted by the first layer decoder of the classic stream. After passing through the gated convolution layer, a residual module is used for feature extraction, and then enters the gated convolution layer together with the features extracted by the second layer decoder. A total of three gated convolutional layers are performed according to this mode, and finally the shape information is obtained through a sigmoid activation function, which is fused with the features obtained by the classic stream to obtain the final result.

(1)

Let there be a total of n pixels in a feature map, let 𝑡 ∈ 0,1, … , 𝑛, where 𝑠 𝑡 and 𝑟 𝑡 are the corresponding traditional stream and road boundary stream processed by the gated convolution layer, after connecting together, use a 1×1 convolutional layer and a sigmoid function to process it as a feature attention map.

(2)

The feature attention map obtained by formula (2) is dot-multiplied with the feature distribution map passed in the classic stream, and the feature distribution in the classic stream is added, and weighted using channel weights after transposition. 𝑠̂ represents the feature output of the gated convolution layer and will be used as the feature input of the next layer. 𝜔 𝑡 is the channel representation of the kernel (channelwise weighting with kernel), s is the feature input after 1×1 convolution processing from the classic stream, and (i, j) indicates that each pixel needs to be traversed.

It can be seen that the gated convolution layer can be regarded as a combination of attention mechanism and residual network.

Since road extraction is a binary classification task, we use a binary cross-entropy loss function (BCE) to supervise the network. The loss function of the network can be expressed as:

(3)

The former is used to supervise the segmentation results in the classic stream to obtain the loss between features and real labels, and the latter supervises the segmentation results in the road's shape stream to obtain the contour results and the boundary loss calculated by the canny operator. Finally, combine the two to get the final loss function.

Multi-scale Convolutional Attention Module

In the decoder part of the classic stream, the features that have been continuously

upsampled and the features processed by the multi-scale convolution attention module (MSCAB module) are fused to avoid the problem of feature redundancy caused by direct rough fusion of the two. The MSCAB module uses simple mathematical multiplication to wake up its attention. For the spatial information encoding of roads, the convolutional attention module is simpler and more effective than the traditional attention mechanism, and can also reduce the number of parameters.

Currently, the convolutional network has a strong inductive bias, so it has better This module has the following characteristics: 1) acquire multi-scale features, and

2) focus on local information. Use 1*1 convolution to process the features of each channel, and use the output as the weight in the attention mechanism to weight the road part, so that the model pays more attention to the feature of the road.

Here are the formulas we use for our setup:

𝑀𝑆𝐶𝐴𝐵 = 𝐶𝑜𝑛𝑣 1×1 (∑ 𝐵𝑟𝑎𝑛𝑐ℎ 𝑖 4 𝑖=1 (𝐷𝑊 𝐶𝑜𝑛𝑣(𝐹) )) (4)
𝑂𝑢𝑡 𝐹 = 𝑀𝑆𝐶𝐴𝐵 ⊗ 𝐹 (5)

For Equation (4), Branchi is the channel number of feature entry, i∈{1,2,3,4}, DW (DepthWise convolution) is a deep convolution. The realization principle is: firstly, a 5×5 convolution operation is performed on the input features, and the new features obtained after convolution are divided into channel operations using 1×1 standard convolution. Compared with the traditional convolution operation, DW not only reduces the number of parameters, but also improves the feature extraction effect.

Because it performs feature extraction for each channel, and the size of the convolution kernel used by each channel is different, the information it obtains is also diverse. At the same time, since the road extraction requires a larger receptive field to obtain more global information and ensure the connectivity of the road, we use strip convolution to simulate the large kernel in the convolution. In general, the larger the convolution kernel and the larger the receptive field, the more and richer the global feature information obtained by the model, which is conducive to accurate judgment of objects.

However, large kernel convolution will lead to an increase in the amount of calculation, and the calculation performance will also decrease. In the previous VGG and Inception networks, two 3×3 convolution kernels were used to replace a 5×5 convolution, which increased the network depth and reduced parameters while having the same receptive field. For the case of valid receptive field (VRF), [START_REF] Peng | Large kernel matters--improve semantic segmentation by global convolutional network[END_REF] used strip convolution such as 1× k+ k× 1 convolution form instead of k×k large convolution kernel to reduce Calculate costs and parameter quantities. Considering that the road is mostly a strip structure, strip convolution is also conducive to extracting strip features.

The sizes of the convolution kernels we designed in each branch are: 3, 7, 11, 21, and the image is filled to ensure that the image size does not change, considering that the convolution kernel is striped, when the convolution kernel is 1×k, the image padding is designed to be 0×n; When the convolution kernel is k×1, the image padding is designed to be n×0. The padding size corresponds to the size of the convolution kernel, the convolution kernel size is 3, 7, 11, 21, and the padding size is 1, 3, 5, 10. Finally, multiscale feature fusion is performed, using the features obtained from different receptive fields, and finally fused and channel mixed through a 1×1 convolution to achieve crosschannel interaction and information integration.

Methodology Innovation

Rse-net applies dual-stream semantic segmentation to the input image and effectively enhances the integrity of road boundary extraction. Even if there is the problem of vehicle or building shadow masking, the accuracy of road extraction can be improved by the algorithm. As the roads in the images have narrow and large span features, we use MSCAB block to convey the contextual semantic information and use strip convolution to process the information, which effectively ensures the connectivity of the roads.

Experimental results and discussion

Datasets

In terms of data sets, we used three typical data sets for experiments. The first one is the DeepGlobe Road dataset, which is the dataset used in the CVPR2018: DeepGlobe Road Extraction Challenge competition and the dataset used by Dlinknet to achieve SOTA performance. Another dataset is the CHN6-CUG dataset proposed by Zhu et al. (2019), a pixel-level VHR satellite image annotated by humans. This task has broad implications for disaster preparedness, the environment, infrastructure development, and epidemic prevention.

Implementation details

For this model, we implemented it using the Pytorch deep learning framework. Using the Adam optimizer, the maximum learning rate is 0.001, and the minimum learning

Evaluation Methods

In this paper, since road extraction is a binary classification task, we use the following metrics to evaluate the model: Miou, Iou, Pa, Precision, Recall. Iou is the intersection and union ratio, that is, the ratio of the intersection and union of the real label of the road and the predicted value. Miou is the average intersection ratio, that is, the average value of the background and road intersection ratio. Pa is the accuracy rate, that is, the proportion of the number of pixels with the correct prediction category to the total number of pixels. Precision is the accuracy rate, that is, the correct proportion of the model's road prediction. Recall is the recall rate, that is, how many roads in the image can be found by the model. The F1 index is the harmonic mean of Precision and Recall.Their calculation formula is as follows:

(6) (7) (8) (9) (10)
where, TP is the correct prediction, a real example, the model predicts a positive example, but it is actually a positive example. FP is a prediction error, a false positive example, the model predicts a positive example, but it is actually a negative example.

FN is a prediction error, a false negative example, the model predicts a negative example, but it is actually a positive example. TN is the prediction is correct, true negative example, the model predicts a negative example, but it is actually a negative example. Since the test set consists of multiple images, we will calculate the accuracy of all images in the test set file, and finally average the evaluation results of all images.

In addition, in order to calculate the difference between the predicted road graphs and the ground truth, we will evaluate the predicted road graphs using the Average Path Length Similarity (APLS) [START_REF] Van Etten | SpaceNet: A remote sensingdataset and challenge series[END_REF], which effectively measures the difference in optimal path length between node pairs of the predicted road graphs and the ground truth, with 1 being the best result and 0 being the worst outcome.

(11)

Where N represents the number of unique paths, represents the length of the road, a represents the source node of the label, b represents the destination node, and represents the node of the predicted road map. TOPO is a comparison of random locations within a certain distance to outward paths, by calculating the percentage of nodes that cannot be matched and the true location of the ground is missed. TOPO compares the outward paths within a certain distance from a random location and counts the percentages of unmatched predicted points and missed ground truth location.

Road Classification Results and Comparative Analysis Based on CHN6-CUG Dataset

In this section, we use Rse-net and Unet model under different conditions, DeepLab V3+ model, Dlinknet34 model and Segformer [START_REF] Xie | SegFormer: Simple and efficient design for semantic segmentation with transformers[END_REF] As can be seen in Table 1, compared with the traditional semantic segmentation method, the improved model we proposed has a certain improvement in classification accuracy. Blue is False negative, and red is False positive.

Figure 6 shows the prediction results of the SpaceNet dataset in different methods.

The image of the SpaceNet dataset have low clarity, and the cross-domain problem that each city has its own differential style, it is on the low side both in terms of the value of the metrics and the presentation of the images. Compared with traditional methods, our method has less noise and ensures the extraction of road edge information while taking into account the road connectivity. As can be seen from the first row of images, our method can effectively solve the problem of vehicles parked on the road causing the road surface to be obscured, because the MSCAB block can obtain more spatial semantic information. In the second row of images, some of the roads are obscured by the shadows of the buildings, but our method is effective in predicting true road. In the third row of images, although the surface colours are essentially the same, our method can obtain the information of road boundaries a through two-stream architecture,

effectively improving the extraction.

Ablation Experiment

This section will conduct Ablation Experiment on the proposed two modules (multi- It can be seen from the comparison of evaluation indicators that the multi-scale convolution attention module has a large improvement in accuracy, and the road shape stream structure pays more attention to road outline information and lacks attention to road connectivity and integrity, resulting in an insignificant improvement in accuracy.

Below we mainly compare the U-shaped model parameters composed of decoders and encoders.

Conclusions

Due to the rapid changes in the shape of roads at present, although the accuracy of road data obtained by using traditional map update and measurement methods is relatively high, it is expensive and time-consuming, and some remote areas do not update roads for a long time. In contrast, using remote sensing images to obtain roads is economical and effective. At the same time, with the help of the powerful function of deep learning, road features can be obtained accurately and efficiently, so as to extract roads. The (1). With the advancement of transfer learning and supervised learning, the deep learning method has greatly improved the accuracy and speed of road extraction, which is more effective and economical than conventional methods.

(2). Compared with the model composed of decoder and encoder, the transformer model with the same training strategy has more advantages, higher accuracy, and requires a simpler environment.

(3). For the characteristics of roads such as occlusion, narrowness, large span, and

  proposed an endto-end multi-scale model MSPNet with a symmetric encoder-decoder network for the task of road extraction. The model can preserve spatial details, optimize road stability, and handle large-scale images. Gao et al. (2019) added an expansion module and a message module to enhance the connectivity of the road edge on the basis of the commonly used decoder-encoder for semantic segmentation. It can not only expand the convolution to increase the receptive field, but also obtain the spatial relationship between pixels. Li et al. (2021) proposed a cascaded attention density network, which embeds two attention modules, global attention and core attention, into the density network. The core attention module is used to extract road occlusion areas, and the global attention module is used to extract global context information, thereby improving the connectivity and stability of the road network. Wang et al. (2022) developed an enhanced road extraction model DDU-Net for the road extraction problem. It utilizes the skip connection between the decoder and the encoder to extract and fuse multi-scale features to solve the road extraction problem of different sizes. At the same time, without losing detailed information, the dilated convolution is used to expand the receptive field, and the dual-decoder structure is used to restore more details. Abdollahi et al. (2021) proposed an automatic deep-learning based road vectorization network. The network contains two Unet networks, which can segment and vectorize roads at the same time. The first network has powerful representation capabilities and can obtain coherent roads even in complex road networks. The second network is connected to the first network and vectorized using all the previously generated feature maps to finally generate the road map. Aiming at the current problems of roads, Tan et al. (2020) proposed a scale-sensitive coded decoder to solve the contradiction between the spatial resolution and semantic information of features of different scales. The decoder contains a scale fusion module and a scale sensitive module. For different scale features have different semantic features and texture features, use the scale fusion module to perform multi-scale fusion, and set a weight tensor to reduce the specificity of noisecontaining features, thereby improving the extraction effect of useful features. Abdollahi et al. (2020) et al. conducted a comprehensive review of the current commonly used remote sensing road extraction applications, which mainly studied generative adversarial networks, deconvolutional neural networks, fully convolutional neural networks, and patch-based convolutional neural networks. The existing problems of the current network learning model are pointed out: the accuracy is low when multiple roads are mixed, the road extraction by deep learning will be affected by architecture, hyperparameters, etc., it is difficult to solve problems such as occlusion, and there is a lack of effective large data sets.

Figure 1 .

 1 Figure 1. The problem of ground object occlusion in the existing road extraction method leads to jagged road boundaries and the inability to extract narrow roads. (Based on CHN6-CUG Dataset) In response to the above problems, we propose a two-stream (including roads' shape stream and classical stream) structure model Rse-net. The roads' shape stream can effectively restore the outline of the road and avoid jagged edges of the extracted road. The multi-scale convolutional attention module in the classical stream can extract narrow roads to ensure the connectivity of the roads, thereby improving the accuracy of road network information extraction. Our contributions are summarized as follows.

Figure 2 .

 2 Figure 2. Flow chart of road extraction. The network is an end-toend network, divided into classical stream and road's shape stream, the blue branch is the direction of the classical stream, and the purple branch is the direction of the road's shape stream.

  generalization ability and faster convergence speed. The attention mechanism reduces the dependence on external information and can capture the relationship between data and features. A large number of experiments have proved that the application of spatial attention module and channel attention module can help improve the performance of neural networks.[START_REF] Guo | Segnext: Rethinking convolutional attention design for semantic segmentation[END_REF] proposed a SegNext model, and proved through experiments that convolutional attention is in some respects a more effective way to process contextual semantic information than the traditional Transformer's selfattention mechanism. Multi-scale convolutional attention module has the following advantages: First of all, the connection between the decoder and the encoder to extract multi-scale features can be established, which helps to improve the accuracy of road extraction. Second, Multi-scale convolutional attention module allows the model to prioritize road segmentation by prioritizing road regions within high-resolution remote sensing imagery. Finally, the computational complexity of MSCAB is lower compared to some channel or spatial attention modules, compared with other data sources, highresolution remote sensing images have the characteristics of complex ground objects and high noise, the modules that need to be added cannot increase the burden on the model, and prevent over-fitting phenomenon caused by too long training time or insufficient number of samples.The multi-scale convolutional attention module also utilizes the large-kernel attention mechanism[START_REF] Li | Ntire 2022 challenge on efficient super-resolution: Methods and results[END_REF]) (Large-kernel Attention (LKA)), which can capture long-distance dependencies and ensure the connectivity of the road. At the same time, the complexity of LKA is low, and it is easy to process high-resolution remote sensing images. This mechanism can achieve good results in image classification, but for roads in high-resolution imagery, more attention needs to be paid to the aggregation of multi-scale features. Therefore, on this basis, we re-constructed a new multi-scale convolutional attention module MSCAB for the characteristics of road extraction.

Figure 4 .

 4 Figure 4. Detailed diagram of GCL Gated Convolutional Layer). Process feature from Roads' stream and contour stream separately.

The

  DeepGlobe Road dataset is a pixel-level labeled dataset that contains 6226 training images, 1243 validation images, and 1101 test images. All the image sizes are 1024*1024, and the images cover parts of Thailand, India, and Indonesia. The image scenes mainly include cities, countryside, wilderness, woodland, and tropical rainforest. The CHN6-CUG data set mainly comes from roads in some cities in China, including Beijing Chaoyang District, Shanghai Yangpu District, Wuhan City Center, Shenzhen Nanshan District, Hong Kong Shatian District, and Macau, six cities with different urbanization levels. The main road features include urban roads, forest roads, country roads, factory and mine roads, intersections and overpasses. The image size is 512*512, 4511 images are used for training, 903 images are used for verification, and the test data is extracted from the training data according to 9:1. The image pixel resolution of dataset DeepGlobe is 1300*1300. Due to the large amount of data, we selected two cities: Shanghai and Paris, which have 1508 images, and use the centerline of road to train the network.

  rate is 0.00001 to optimize the model, and use NVIDIA GPU3080 and 12G video memory for training. The batch size is set to 4 when processing the 1024*1024 resolution image of the DeepGlobe Road dataset, and the batch size is set to 16 when processing the 512*512 resolution image of the CHN6-CUG dataset and SpaceNet dataset because the image pixel resolution of dataset DeepGlobe is 1300*1300, it's too large to our model.. Considering that the size of the three datasets is inconsistent, a resampling part is added to the model to ensure the resolution of the image as much as possible when it enters the model training.

  The best results have been achieved on the CHN6-CUG dataset. Compared with the classic Unet network and using vgg16 as the backbone neural network, the Miou index has increased by 7.44%, and the Iou index has increased by 14.31%. Compared with Unet which added SE_Block channel attention, the Iou indicator has been improved by 12.29%. Compared with adding Simam's non-parametric attention module Unet, it has improved by 6.8%. Compared with the Dlinknet model that obtained the CVPR road SOTA performance in 2018, Miou has increased by 1.58%, and Iou has increased by 1.24%. At the same time, Pa, Precision and Recall indicators have also been partially improved. Generally speaking, the RSENET model we proposed has higher accuracy and better performance, and compared with other attention modules, our multi-scale convolution attention module is also effective. At the same time, it can also be seen that the current mainstream transformer model is not as effective as the convolutional neural network constructed by the traditional decoder-encoder without adjusting the learning strategy and partial structure.

Figure 5 .Figure 6 .

 56 Figure 5. Comparative analysis of road information extraction results based on CHN6-CUG dataset. (a) is a sample image, (b) is a label image, and (c-e) are roads extracted using different algorithms. Blue is False negative and red is False positive. The blue part is the multi-extraction part, and the red part is the missing extraction part. We also fused the extracted road map with high-resolution remote sensing images. It can be seen that the road model extracted by the RSENET network is better than other networks. And after the multi-scale convolutional attention module, the connectivity of the road is improved. The road is also rarely broken, which benefits from the large receptive field of the multi-scale convolutional attention module and the fusion function of multi-scale feature extraction. The large receptive field helps to improve the connectivity of the extracted roads, and avoids road fragmentation, suppresses noise information, reduces multi-extracted parts, and can well fuse and transmit topological

  scale convolutional attention module and contour flow structure) to prove its performance. We still use the CHN6-CUG dataset for experiments, and maintain the original training strategy and epoch, and use evaluation indicators such as Miou and Iou to evaluate the module. We will use dlinknet34 as the baseline for experiments, and we will also use the SE_Block channel attention module and the Siamam parameterless attention module for Ablation Experiment.

  GFLOPS represents the amount of calculation, which refers to the time complexity of the model, that is, the length of the model execution time, and params represents the amount of parameters, that is, the space complexity, and is also the size of the video memory occupied by the model. It can be seen from the comparison that our proposed multi-scale convolutional attention module does not increase much in terms of calculation and parameter volume compared with the original model Dlinknet. The module is relatively lightweight, and the large-core strip convolution layer used effectively avoids the disadvantages of using a single large-core convolution layer with too many parameters, and at the same time effectively expands the receptive field and integrates multi-scale spatial information. The proposed road shape stream structure uses a new network branch and performs the canny operator to calculate the edge information of the image, resulting in an increase in the calculation amount of the model compared with the original model. However, the extraction effect for the edge contour is still effective, reducing the jagged phenomenon of the extracted road, and its parameter amount is not much increased compared with the original model. However, compared with the traditional Unet model using VGG and Resnet50 as the backbone neural network, the RSENET model we proposed has certain advantages in terms of parameters and calculations, whether or not the attention mechanism is used.

  development of the current neural network model has brought new opportunities for road extraction. We design a new Res-net model, After our experimental comparison on the two data sets, using the same training strategy, this research has obtained the following results.

  strong connectivity, the model used must have the characteristics of multi-scale fusion, large receptive field, and deep enough network. We design several modules to solve the above problems, and demonstrate the effectiveness of our method through experiments. The multi-scale convolutional attention module is used for larger receptive fields and multi-scale fusion features, which can solve problems such as narrow roads and disconnected roads. Road shape stream can make the model pay more attention to the edge information, and make the model get more features through multiscale information extraction and fusion, effectively reducing the road jaggedness and imperfect extraction, and refining the road boundary. The effectiveness of these two modules is proved by ablation experiments. Miou reached 84.57% on DeepGlobe Road dataset and 83.68% on CHN6-CUG dataset. Based on the road information extracted by the RSENET model, compared with other models, it can be seen that the RSE model is effective.(4). There are still problems. First, although most of the road edge information is solved in this paper, some difficult-to-distinguish areas still affect the accuracy of road extraction. The noise information in these places inhibits the transmission of effective information, making the model unable to distinguish. The second is the lack of effective road datasets for training models. Although the two datasets used above have a lot of content and excellent quality, there are still problems such as insufficient coverage, insufficient variety of ground features, and insufficient regional considerations. Finally, the portability and portability of the model are relatively poor, and the deployment of deep learning still requires high-power GPUs to accelerate training. It is difficult to adapt a single model to various road data, and we still need to design a more general method to extract roads.

Table 1 .

 1 model to conduct experiments and comparisons. As shown in Table 1, the backbone uses different backbone neural networks, and the Block uses different modules such as SE_Block's channel attention and simam's non-parameter attention. To ensure fairness, all models use pre-trained weights to prevent the features extracted by the encoder from being too scattered, and to speed up convergence. After using the pre-trained weights, the epoch has converged in 50-70 passes, and we are going to use 100 epochs to control the variables and use the same training strategy. Comparison results based on the CHN6-CUG dataset. The unit is (%).

	Methods	Backbone	Block	Miou	Iou	Pa
	Unet	Vgg16	FALSE	76.12	54.86	65.54	70.85
	Unet	Resnet50	FALSE	79.53	61.41	76.46	76.09
	AttUnet	Vgg16	SE	76.72	55.88	65.90	71.70
	AttUnet	Resnet50	SE	79.70	61.43	74.34	76.29
	AttUnet	Resnet50	Simam	80.08	62.37	75.12	76.82
	D-Linknet34	Resnet34	FALSE	82.10	67.93	79.88	81.59
	D-Linknet34	Resnet34	SE	83.25	68.38	82.9	81.22
	D-Linknet34	Resnet34	Simam	83.44	68.77	84.59	81.49
	Deeplab V3+	Xception	FALSE	69.79	42.80	49.81	59.93
	Deeplab V3+	Mobilenet	FALSE	75.58	53.76	62.19	69.92
	Segformer	B0	FALSE	71.60	46.29	55.11	62.54
	Rse-net	Resnet34	MSCAB	83.68	69.27	83.18	81.83
			_Gate

Table 3 .

 3 Ablation Experiment using D-Linknet as the baseline. Unit is(%).

	Methods	Backbone	Block	Miou	Iou	Pa	Precision Recall
	dlinknet34	resnet34	FALSE	82.94	67.63	79.36	82.01	79.29
	dlinknet34	resnet34	MSCAB 83.80	69.37	81.82	82.11	81.61
	dlinknet34	resnet34	GATE	83.18	68.66	81.13	83.36	81.19
	Rse-net	resnet34	MSCAB	84.57	70.63	82.91	82.43	82.86
			_GATE

Table 4

 4 Comparison of calculation and parameter quantities.

	Methods	Backbone	Block	Total	Total
				GFLOPS(G)	params(M)
	Unet	vgg	False	451.672	24.891

information of roads. For the use of the road shape stream, the jagged edge problem of the road has also been partially solved. Even if there is partial occlusion, the effective information transmitted by the road shape stream can be used to extract and preserve the road shape, ensuring the effective transmission of edge information. In the red box in the second row, the road shape stream can also extract the unlabeled cement floor. It can be seen from the image that the unlabeled part in the red frame also has the characteristics of the road, so we think that this part can also belong to the road part, and our model can also extract this part. Therefore, the model we propose has a strong ability to extract features, and can effectively extract road features. It can be seen from the third and fifth row of images that the RSENET model is very effective for narrow road extraction, ensuring the connectivity and topology of the road. In addition, there are still some areas where roads cannot be extracted. The possible reason is that trees are covered by trees and shadows, and noise is added. However, the RSENET model uses multi-scale contextual semantic information and fuses these semantic information to infer that the area is a road, which improves the recognition rate.

Road Classification Results and Comparative Analysis Based on

DeepGlobe Road dataset

In this section, we will use the scheme consistent with Section 3.4 to conduct comparative experiments on the DeepGlobe Road dataset. To ensure the consistency of the experimental environment, the image resampling part is added to the image preprocessing to preserve the semantic information of the original size image as much as possible and avoid data defects caused by compressed images. we propose has an obvious effect on the extraction of small roads. This is due to the introduction of a multi-scale convolutional attention module, so that the model can obtain more receptive fields to infer small roads and ensure the structure and connectivity of the roads. The addition of the road shape stream can also allow the model to focus on the outline information of the road, generate a rough road boundary, and integrate it with the traditional stream, and can extract road information that is not easy to judge.

Road Classification Results and Comparative Analysis Based on SpaceNet dataset

In this section, we will use the scheme consistent with Section 3.4 and Section 3.5 to conduct comparative experiments on the SpaceNet dataset. As can be seen in Table 3, compared with the traditional semantic segmentation method, the RSENET model has a better road information extraction effect, and achieved the best results on the SpaceNet dataset. Compared with the Dlinknet model