Sughosh Deshpande

Abdallah Bouzid

Pierre Lagarrigue

Yann Landon

Anna Carla Araujo
email: araujo@insa-toulouse.fr

Data maps for material identification in helical milling by spindle power monitoring

Hole making on stacked aerospace materials is a major operation during aircraft assembly which poses significant challenges during manufacturing because of different material machinability. Strategies involving smart machining including adapting proper cutting conditions using real time monitoring can lead to significant improvements. This paper is a continuation of our research that uses data map methodology characterized by different specific force regions for workpiece material identification. In this article, spindle power is monitored utilizing CNC machine internal sensor during helical milling of Aluminium and Titanium alloys in a aerospace stack for estimating cutting coefficients. Our previous research addressed the material detection in circular milling of Aluminium and Titanium alloys independently using a force dynamometer. The result shows the applicability of data map technique consisting of axial force coefficients for material identification in helical milling and highlights the significance of stack sequence.

Introduction

Data acquisition, monitoring and feedback loop for corrective actions plays a vital role for performance optimization in smart machining strategies implemented in industries. The hardware and techniques required for data monitoring should be feasible enough for easy implementation on the shop floor without expensive investments and disturbance to current processes [START_REF] Axinte | Assessment of the effectiveness of a spindle power signal for tool condition monitoring in machining processes[END_REF]. Data acquisition of state of the art machine tool can add significant value and are also easy to acquire without any additional accessories or hardware setup [START_REF] Lauro | Monitoring and processing signal applied in machining processes -a review[END_REF]. The internal sensors in the machine tool capture a variety of data and can be useful for the operator to improve productivity and tool life during machining.

Aerospace stack hole making is one such application where lot of research is being carried out in order to enhance the process productivity and hole quality [START_REF] Zhaoju | Investigation on performance characteristics and metallographic transformation on drilling aluminium/titanium sandwich[END_REF][START_REF] Vijayan | Comprehensive Review on Drilling of Multimaterials Stacks[END_REF]. Each of the material layer in a stack has different machinability and properties. Using a single tool, one shot machining, for both materials is challenging. This could lead to poor hole quality and lower production rates. The methodology of adapting cutting parameters after identification of different materials in real time can increase productivity. This can be achieved by real time force and power monitoring, data acquisition and implementing a feedback loop [START_REF] Araujo | Smart drilling for Aerospace Industry: state of art in research and education[END_REF]. Pardo et al [START_REF] Pardo | Assessment of decision-making algorithms for adaptive drilling of aerospace stacks[END_REF] discusses decision making algorithms in order to identify process incidences in stack drilling. Monitoring of signals from AE sensors for tool position identification in Carbon Fibre Reinforced Plastics (CFRP)/Aluminium stacks is shown by Neugebaur et al [START_REF] Neugebauer | Acoustic emission as a tool for identifying drill position in fiber-reinforced plastic and aluminum stacks[END_REF]. Fang et al [START_REF] Fang | A Force Sensorless Method for CFRP/Ti Stack Interface Detection during Robotic Orbital Drilling Operations[END_REF] developed force sensor less method to detect stack interface in robotic orbital drilling operations which can be helpful to adapt proper cutting conditions. 1 Helical milling (also called orbital drilling) is a hole making technique, especially advantageous, for machining of hard and exotic materials resulting in better hole quality and tool life compared to axial drilling [START_REF] Iyer | Helical milling: An enabling technology for hard machining precision holes in AISI D2 tool steel[END_REF]. The exit burs produced in helical milling is considerably less compared to axial drilling [START_REF] Ge | Investigating hole making performance of Al 2024-T3/Ti-6Al-4V alloy stacks: A comparative study of conventional drilling, peck drilling and helical milling[END_REF]. Sun et al [START_REF] Sun | Hole-making processes and their impacts on the microstructure and fatigue response of aircraft alloys[END_REF] showed higher fatigue life and surface finish for holes produced in helical milling in comparison with axial drilling of Aluminium and Titanium alloys. Better chip evacuation, less cutting forces result in better tool life and hole quality in helical milling. Implementation of helical milling for producing holes in aerospace stacks is also widely researched topic and considerable work is being done in this area [START_REF] Wang | Experimental research in CFRP/Ti stack through different helical milling strategies[END_REF][START_REF] Denkena | Helical milling of CFRP-titanium layer compounds[END_REF].

The state of the art shows monitoring the gradient of thrust forces or cutting torque to identify the material and tool position in a stack and can be challenging especially at the interface layers. The data maps of cutting force coefficients for individual materials can have significant advantages for real time material identification especially at the material interface to clearly distinguish the materials based on specific data points [START_REF] Deshpande | Specific force map for smart machining applications with rotating tools[END_REF].

This paper presents the applicability of data maps for the identification of work piece materials during helical milling of aerospace stack made of Aluminium and Titanium alloys. Experiments are done to verify the possibility of using CNC machine internal sensor to monitor spindle power for estimating cutting coefficients. The cutting force data is also monitored by a force dynamometer in order to compare with the spindle power signal especially at material transition for gradient change. In the following sections, the cutting force model, specific force coefficients and cutting power in helical milling are presented, followed by experimental set-up, results and conclusions.

Helical Milling

In helical milling, the tool describes a helical trajectory as shown in Fig. 1(a). The combination of tool movement in XY plane and Z plane simultaneously results in gradual removal of material for every orbital rotation of the tool (Eq.1).

⃗ f z = ⃗ f xy (xy plane)+ ⃗ f a (axial) (1)
P = fa 2πRtt fxy = f a π(D -D t ) f xy (2) α = arctan P π(D-Dt) = arctan f a f xy ( 3
)
N orb = fa Z N P (4)
The depth of material removed per orbital rotation is designated as pitch P (mm/rev) and it is a function of the feed and the radius of the trajectory R tt , calculated based on the tool diameter D t and the machined diameter D (Eq.2). The resulting ramp angle is indicated as α (Eq. 3), a function of the axial feed and the circular feed [START_REF] Wang | Prediction of cutting forces in helical milling process[END_REF].

Figure 1(b) shows tool trajectory with reference to fixed Cartesian coordinate (X m , Y m ) and rotating tool coordinate (X t ,Y t ) systems. The tool rotates around its own axis at N (rpm) spindle speed (or θ2 rad/s) and around hole center O at N orb rpm (or θ1 rad/s) to machine the hole. The offset distance between tool center C and hole center O is designated as R tt [START_REF] Araujo | Analytical and experimental investigations on thread milling forces in titanium alloy[END_REF]. [17] and the chip thickness h p is described by Martellotti equation [START_REF] Martellotti | An analysis of the milling process[END_REF].

Chip geometry

b p (θ 2 ) = P - P π arccos R tt -R t cos(θ 2 ) √ R 2 tt + R 2 t -2R t R tt cos(θ 2 ) (5)
h p (θ 2 ) = f z sin(θ 2 ) (6) b b = D t 2 (7
)

h b = f z P π(D -D t ) (8)
In the bottom edge, the chip length b b can be approximated to tool radius (Eq. 7) and the chip thickness h b is described by Eq. 8. The bottom uncut chip thickness can be assumed constant and independent of cutting velocity. Hence, the specific force coefficients can be identified for the bottom edge and peripheral edge of the tool based on the local uncut chip load.

Cutting forces and coefficients

The mechanistic force models predict the cutting forces based on uncut chip load at the cutting edge. The cutting force predictions has become more and more closer to the experimental values by including various factors like tool geometry, operation type, tool vibrations etc in the force models [START_REF] Altintas | Generalized Modeling of Mechanics and Dynamics of Milling Cutters[END_REF]. The machining force ⃗ F m is calculated considering the small finite elements of the cutting flute in the cutting edge referential frame decomposed: tangential (dF t ), radial (dF r ) and axial (dF z ) components for the peripheral flute j is given by:

⃗ dF m =   dF t dF r dF a   . ( 9
)
Figure 2(b) illustrates the cutting forces exerted by the peripheral and bottom edges of the tool on the work piece during helical milling. The tangential, radial and axial force components are depicted at peripheral (F tp ,F rp ,F ap ) and bottom cutting edges (F tb , F rb , F ab ) respectively. However, the tangential (F tb ) and radial (F rb ) force components at the bottom edge is significantly small and can be neglected. The force components can be expressed as a function of uncut chip cross section and flute rotation angle θ 2 as:

F tp,j = K tp .h p (θ 2 ) j b p (θ 2 ) j ( 10
)
F rp,j = K rp .h p (θ 2 ) j b p (θ 2 ) j ( 11
)
F ap,j = K ap .h p (θ 2 ) j b p (θ 2 ) j ( 12
)
F ab,j = K ab .h b R t ( 13
)
where K tp ,K rp ,K ap are the specific force coefficients for tangential, radial and axial force components obtained at peripheral cutting edge j and K ab is the specific force coefficient from axial force component obtained at bottom cutting edge j. Hence the total force due to peripheral and bottom edges considering Z flutes in a tool can be expressed as below:

F t = ∑ Z j=1 K tp .h p (θ 2 ) j b p (θ 2 ) j ( 14
)
F r = ∑ Z j=1 K rp .h p (θ 2 ) j b p (θ 2 ) j ( 15
)
F a = ∑ Z j=1 K ap .h p (θ 2 ) j b p (θ 2 ) j + Z.K ab .h b R t (16)
However, during experiments, the cutting forces recorded are in fixed Cartesian coordinate system (F xm , F ym , F zm ) and can be transferred to one of the reference cutting edges. The reference flute position (θ 2 ) being known, the angular position of remaining flutes for a particular tool can be calculated:

(θ 2 ) j = (θ 2 ) + (j -1)λ (17)
where λ=2π/Z is the tool pitch angle The axial components (F ap +F ab ) can be approximated to cutting force (F zm ) obtained in Z direction during experiments. The rotation matrix for transformation of forces is given by Eq.18:

  F r (θ 2 ) j -F t (θ 2 ) j F a   =   cos(-θ 2 ) j -sin(-θ 2 ) j 0 sin(-θ 2 ) j cos(-θ 2 ) j 0 0 0 1   .   F X F Y F Z   (18)

Cutting Power

In case of end milling , depending upon the number of flutes, the cutting power is the summation of power consumed by the flutes (j = 1...Z) in active contact with the work piece at any instant.

P ower cut = Z ∑ j=1 [P ower p (j) + P ower b (j)] (19)
where P ower p and P ower b denotes cutting power because of peripheral and bottom cutting edges respectively which are in active contact with the work piece during helical milling. Depending upon the cutter pitch angle, the number of peripheral flutes simultaneously in contact with the work piece can be determined considering peripheral angle of immersion between ϕ= 0 to 180 o for helical milling.

The bottom edges of all the flutes are always in contact with the work piece during helical milling. Tangential cutting force F tp at the peripheral cutting edge is the major component of cutting force in magnitude and can be considered for the calculation of cutting power P ower p at peripheral edge. Similarly, axial cutting force F ab is the major cutting force at the bottom edge and can be used to express cutting power P ower b at bottom edge of the tool. The cutting power can be expressed approximately as a function of cutting force and cutting velocity V c . The cutting velocity at a point on the bottom edge varies along the edge based on its distance from the tool center. Hence the cutting power is expressed as a function of average cutting velocity V c for the bottom cutting edges as shown below:

P ower cut = Z ∑ j=1 1 60 { g(θ 2 ) • (F tp ) j • V c + (F ab ) j • V c } ( 20
)
where g(θ 2 )=0 or 1 is a contact function based on position of the peripheral flute in or out of immersion angle zone. Tangential F tp and axial F ab cutting forces can be further expressed as a function of specific force coefficients (K tp ,K ab ) and uncut chip load (h p , b p , h b , b b ) as follows:

P ower cut = Z ∑ j=1 1 60 { g(θ 2 ) • [h p (θ 2 ) • b p (θ 2 )] j • K tp • V c + [h b • b b ] j • K ab • V c } (21)
Machine spindle power can be monitored during machining and the total spindle power monitored has components of cutting and non cutting power because of spindle inertia and power because of tool feed motion expressed as: P ower total = P ower idle + P ower cut + P ower f eed (

The idle power component for a particular machine spindle can be determined to get the actual cutting power. The power because of tool feed motion is assumed to be relatively less in helical milling. During experiments, power total can be recorded at different pre defined cutting conditions. The modelled cutting power (Eq. 21) can be re-written to determine values of K tp and K ab as a function of cutting power and chip dimensions.

Materials and Methods

This section describes about the helical milling experiments conducted on Aluminium (2024A) and Titanium (Ti6Al4V) alloy stacks for a range of cutting conditions and setup as described in the following subsections. The aim of these experiments is to monitor spindle power for material identification during hole making of stacked aerospace materials by utilizing CNC machine internal sensors.

Experimental setup

Helical milling experiments were carried out on a CNC milling center DMU85-DMG mono block machine under wet condition. A carbide end mill from Fraisa (code: P45217391) with 4 flutes and 8 mm diameter was used for all the helical milling experiments (Fig. 3(b)). The stack is made of Aluminium and Titanium alloy with 4 mm thickness of each layer bearing dimensions 35 mm X 100 mm. The layers are stacked together by bolt/nut assembly using a fixture as shown in the Fig. 3(a). The workpiece is fixed on a 9257B Kistler dynamometer, connected to a 5070 Kistler amplifier. Analogical data of force is converted to digital using a 9201 National Instruments acquisition module with 1000 Hz acquisition rate. For power acquisition, Sinucom NC trace software tool which can monitor spindle power at a rate of 4 ms is utilised. Along with spindle power, machine data like X, Y, Z position, spindle angular position is also recorded.

Design of experiments

Table 1 summarizes the cutting conditions adopted for helical milling experiments and the corresponding stack sequence. The stack sequence is varied during experiments and represented as Al→Ti for Aluminium Titanium sequence and Ti→Al for Titanium Aluminium sequence. Cutting velocity V c is 40 m/min (for 8mm diameter tool) in all the experiments considering machining of both material layers by the same tool and recommendations by the tool supplier. Different feed per tooth in axial (f a ) and peripheral (f xy ) is defined considering the resulting chip thickness and the corresponding cutting forces predicted by algorithms developed in Python. Hence, corresponding feed per tooth (f z ) at tool center point for helical trajectory is calculated along with pitch (P ) and orbital rotation speed (N orb ) from Eq.1 to Eq. 4. The hole diameter produced is 11mm since it is one of the most popular diameters produced in aerospace industry.

Experimental results

The following subsections present the cutting power, cutting forces monitored during helical milling of stacks and estimated specific force coefficients data map. The specific coefficients are estimated from the cutting power signal in order to verify the possibility of utilising spindle power for material identification.

Cutting power

Figure 4 (a) to (d) shows cutting power as a function of hole depth at various feed per tooth and stack sequences. The material layers can be approximately identified by the distinct steps observed in the power signal at the stack interface approximately at 4 mm where tool transition occurs from one layer to other. The power signal shows disturbances while the tool is transiting from Titanium to Aluminium layer and is further observed during machining of Aluminium as shown in Fig. 4 (d). This can be mostly attributed to sticky chips of Titanium sticking to the cutting edge which is re-machined while machining Aluminium. This phenomenon was also reaffirmed in the cutting force signals for Ti→Al sequences in some cases. It is important to note that these signal disturbances were not consistent for all Ti→Al stack sequence experiments but was observed frequently.

Cutting forces

Figure 5 (a) and (b) shows the cutting forces (F t , F r and F a ) obtained at the cutting flute after force transformation using rotation matrix for different stack sequences. However, in this article, the cutting force signals are shown only for one of the cutting conditions since the main purpose was to just compare it with the cutting power signal especially at the stack interface.

Cutting force coefficients

Cutting force coefficients K tp (at tool periphery) and K ab (at tool bottom edge) are estimated from cutting power signals as described in subsection 2.3. The coefficients are identified by taking maximum power values at every quarter orbital rotation of the tool and corresponding chip dimensions at that instant. K ab and K tp are represented as a function of hole depth for different stack sequences as shown in the Fig. 6 and7. The data points of K ab observed in Fig. 6 (a) shows distinct density of points at two regions indicating two different material layers. Density Based Spatial Clustering Algorithm (DBSCAN) is adopted in order to distinguish the data sets based on the density while removing some of the noise.

Conclusion

The objective of this work is to check the applicability of data maps for material identification during helical milling of aerospace stacks. This is achieved by monitoring machine spindle power from the CNC machine internal sensor to estimate specific coefficients. This technique is advantageous as it does not require any external expensive sensors or any modification in the setup in order to place the sensor and is easy for shop floor implementation.

The data map of axial specific force coefficient K ab makes it possible to identify the material layer in real time distinguishing two separate regions. However, it is difficult to identify two different material layers by observing data map of tangential cutting force coefficient (K tp ) as the data points are unevenly distributed . The stack sequence plays a crucial role in terms of generated cutting forces and required cutting power for machining. Especially, the movement of tool from Titanium to Aluminium layer is accompanied by disturbances in monitored signal of cutting force and power

Figure 2 (

 2 Figure2(a) shows chip generation during helical milling. There are two zones of chip generation: at the peripheral edge, that it is represented by the index 'p' in this article, and the bottom edge of the tool, represented by 'b' index. The chip thickness (h p ) at the peripheral side of the tool is similar to conventional milling process which is a function of tool revolution angle (θ 2 ) and feed per tooth f z . The chip width b p is a function of θ 2 and the axial position of the tool in the orbital rotation

Figure 1 :

 1 Figure 1: Tool trajectory in helical milling.

Figure 2 :

 2 Figure 2: Chip generation and cutting forces during helical milling.

Figure 3 :

 3 Figure 3: Experimental details.

  Ti→Al: fa=0.003 mm/th, fz=0.059 mm/th

Figure 4 :

 4 Figure 4: Cutting power with hole depth at different cutting conditions.

  Figure 6 (b) shows the data map of K ab after implementing clustering model (DBSCAN) indicating data points in different symbols. Symbol (□) indicates Titanium layer

Figure 5 :

 5 Figure 5: Cutting force in rotating tool system as a function of hole depth at f a =0.002 mm/th, f z =0.044 mm/th.

Figure 6 :

 6 Figure 6: Data map of cutting force coefficients for Al→Ti stack at f a =0.002 mm/th, f z =0.044 mm/th

Figure 7 (

 7 a) shows the data map of K ab for T i → Al stack sequence and it again shows distinct region of data points.Figure 7 (b) shows data map after implementation of clustering algorithm indicating two colours for two material layers. The data map of K tp is shown in Fig. 7 (c) and the indications are similar to data map of K tp for Al → T i stack sequence. It is difficult to distinguish different material layers.

Table 1 :

 1 Cutting conditions Stack f

	Ti-→ Al	0.002	0.044		30
	Ti-→Al Al-→Ti Al-→Ti	0.003 0.002 0.003	0.059 0.044 0.059	40	40 30 40	0.5
	Ti-→Al Ti-→Al	0.001 0.002	0.044 0.059		30 40	0.3

a (mm/th) f z (mm/th) V c (m/min) N orb (rev/min) P (mm/rev)

The map of cutting force coefficients can be grouped by clustering algorithms and can be used to train machine learning models in the future for smart machining applications including adapting optimised cutting parameter suitable for a particular layer.


Fuzhi Wu

Jiasong Wu

Youyong Kong

Chunfeng Yang

Guanyu Yang

Huazhong Shu

Guy Carrault

Lotfi Senhadji

Convolutional Modulation Theory: A bridge between Convolutional Neural Networks and Signal Modulation Theory

Keywords: Convolutional neural network, signal modulation theory, energy spectrum distribution, classification, segmentation

Although there have been a lot of researches on convolutional neural networks (CNNs), still what happens in this black box remains a mystery. In this paper, we establish the connection between CNNs and signal modulation. From a signal modulation point of view, the forward-propagation process of CNNs can be explained as a process of modulating the input signals to the vicinity of a special energy spectrum distribution, and the backpropagation process is searching for the appropriate distribution which is better for classification or other tasks.

Several experiments have been carried out to verify the modulated explanation of CNNs. Furthermore, we verify that modulating the signal to the appropriate energy spectrum distribution in advance can effectively improve the classification and segmentation accuracy.

Introduction

In recent years, deep learning [START_REF] Hinton | A fast learning algorithm for deep belief nets[END_REF]- [START_REF] Schmidhuber | Deep learning in neural networks: An overview[END_REF] and especially convolutional neural networks (CNNs) [START_REF] Lecun | Backpropagation applied to handwritten zip code recognition[END_REF], [START_REF] Lecun | Gradient-based learning applied to document recognition[END_REF] have been widely used in many research fields and industrial applications. The great success of CNNs is impressive.

However, why CNN can work well is a long way from clear explanation, which has become a bottleneck restricting the development of CNNs and also their applications to areas where interpretable artificial intelligence technology is necessary as, for example, in medical domain. Therefore, researchers have proposed many interpretation frameworks, which can be roughly divided into two classes: model-specific interpretability and model-agnostic interpretability [START_REF] Adadi | Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI)[END_REF].

Model-specific interpretability methods can only interpret specific model types. For example, Hershey et al. [START_REF] Hershey | Deep unfolding: Model-based inspiration of novel deep architectures[END_REF] proposed a framework for deriving novel deep network architectures from model-based inference algorithms by unfolding the steps of the algorithm and untying the model parameters across iterations. Wu et al. [START_REF] Wu | PCANet: An energy perspective[END_REF] presented a new way to visualize, explain and understand every step of principal components analysis network (PCANet) from an energy perspective. The drawback of this practice is that these interpretation methods are not sufficiently generic, that is, if we want to use the particular type of interpretation, we have to choose the specific models and cannot use other models. Therefore, model-agnostic interpretation methods have become the focus of research in recent years. These model-free methods broadly fall into five technique types: [START_REF] Hinton | A fast learning algorithm for deep belief nets[END_REF] Visualization approaches try to visualize the representations to explore the pattern hidden inside a neural unit. For example, Matthew and Fergus [START_REF] Zeiler | Visualizing and understanding convolutional networks[C]//European conference on computer vision[END_REF] proposed deconvolution network (DeConvNet) method in which the network computations were backtracked to identify which image patches are responsible for certain neural activations. Simonyan et al. [START_REF] Simonyan | Deep inside convolutional networks: Visualising image classification models and saliency maps[END_REF] demonstrated that the visualization results of image classification model by using convolutional networks (ConvNets) could be obtained by numerical optimization of input image. ( 2) Knowledge extraction approaches try to extract, in a comprehensible form, the knowledge acquired by a network during training and encoded as an internal representation. For example, Tan et al. [START_REF] Tan | Distill-and-compare: Auditing black-box models using transparent model distillation[END_REF] investigated how to use model distillation to extract complex models into transparent models. Che et al. [START_REF] Che | Distilling knowledge from deep networks with applications to healthcare domain[END_REF] introduced a knowledge extraction method called interpretable mimic learning to learn interpretable phenotypic features, so as to make reliable predictions while imitating the performance of deep learning models. Xu et al. [START_REF] Xu | Interpreting deep classifier by visual distillation of dark knowledge[END_REF] introduced DarkSight, a visualization method used to interpret the predictions of black box classifiers on datasets inspired by the concept of dark knowledge. (3) Influence methods focus on changing the input or internal components to estimate the importance or the relevance of a feature and to better understand the network. For example, Koh and Liang [START_REF] Koh | Understanding black-box predictions via influence functions[END_REF] used influence functions to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points that most responsible for a given prediction. Bach et al. [START_REF] Bach | On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation[END_REF] proposed the layer-wise relevance propagation algorithm to compute the relevance between decision and classifier. (4) Example-based explanation approaches try to understand the behavior of machine learning by studying particular instances of dataset. For example, Kim et al. [START_REF] Kim | Examples are not enough, learn to criticize! criticism for interpretability[C]//Advances in neural information processing systems[END_REF] developed the maximum mean discrepancy critic (MMD-critic) which efficiently learns prototypes and criticism, designed to aid human interpretability. (5) Theoretical connection. An effective method is to establish the relationship between deep learning and some well-developed theories, and then use these theories to explain the neural networks and also guide the construction of neural networks. Some well-developed theories include: (a) Renormalization Theory.

Mehta and Schwab [START_REF] Mehta | An exact mapping between the variational renormalization group and deep learning[END_REF] explained deep neural networks (DNNs) as a renormalization group like procedure to extract relevant features from structured data. (b) Probabilistic Theory. Patel et al. [START_REF] Patel | A probabilistic theory of deep learning[END_REF] developed a new probabilistic framework for deep learning based on a Bayesian generative probabilistic model. (c) Information Theory. Tishby and Zaslavsky [START_REF] Tishby | Deep learning and the information bottleneck principle[END_REF] analyzed CNNs by using the theoretical framework of the information bottleneck principle. Then, Steeg and Galstyan [START_REF] Ver Steeg | The information sieve[C]//International Conference on Machine Learning[END_REF] further introduced a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. (d) Numerical Differential Equations.

Lu et al. [START_REF] Lu | Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations[END_REF] bridged deep architectures and numerical differential equations. (e) Group Theory. Paul and Venkatasubramanian [START_REF] Paul | Why does Deep Learning work? -A perspective from Group Theory[END_REF] showed the intrinsic relations between group theory and deep networks, and explained why unsupervised deep learning works.

In signal modulation domain, many works have been conducted to optimize the traditional signal modulation system through deep learning, for example, classification of signal modulation type [START_REF] Wang | Modulation format recognition and OSNR estimation using CNN-based deep learning[END_REF]- [START_REF] Yongshi | CNN-based modulation classification in the complicated communication channel[END_REF], optimization of transmitter and receiver [START_REF] Karanov | End-to-end deep learning of optical fiber communications[END_REF], [START_REF] Schmitz | A deep learning wireless transceiver with fully learned modulation and synchronization[END_REF], quantization of L-values for gray-coded modulation [START_REF] Arvinte | Deep learning-based quantization of L-values for Gray-coded modulation[END_REF] and spatial modulation multiple-input multiple-output (SM-MIMO) transmit antenna selection [START_REF] Yang | Adaptive spatial modulation MIMO based on machine learning[END_REF]. Some works establish links between the communication system and the autoencoder, interpreting the communication as an autoencoder [START_REF] Shea | An introduction to machine learning communications systems[END_REF]- [START_REF] Dörner | Deep learning based communication over the air[J][END_REF].

Unlike them, we try a completely opposite problem. In this paper, we aim to provide a new model-agnostic interpretation method for CNNs by leveraging the well-studied signal modulation theory, which leads to a clear and profound understanding of CNNs, together with new insights. Specifically, we try to bridge the deep learning and signal modulation by studying the spectrum distribution of features in CNNs. We conclude that in the forwardpropagation of CNNs, what happens in the black box is explained as the "generalized shifting" of energy spectrum and experiments show that different types of energy spectra will be modulated to be near a similar spectrum distribution. We conclude that the back-propagation process can be thought of as a searching process for an optimal energy spectrum distribution that is most conducive to related tasks by stochastic gradient descent methods.

Experiments show that different networks will modulate features in the similar direction finally and better classified features will be closer to the optimal spectrum. Finally, the applications of our theory on one-dimensional and twodimensional public datasets reveal that our theory is helpful for the design of CNNs. The contributions of the paper are as follows:

1) We propose a new interpretation framework for CNNs by using the signal modulation theory for the first time and therefore bridging the deep learning domain and signal modulation domain.

2) In the forward-propagation of CNNs, every operation is explained by the signal modulation theory and what happens in the black box is explained as the "generalized shifting" of energy spectrum. For the backpropagation process, it can be thought of as a searching process for an optimal energy spectrum distribution that is most conducive to classification or other tasks. Several validation experiments corroborate our theory.

3) We found the relation between feature spectrum distribution and task effects and several experiments reveal that our theory is helpful for the design of CNNs.

The paper is organized as follows. Signal modulation theory is reviewed in Section 2. Section 3 explains the basic operators and propagation process of CNNs. Some mysteries of CNNs are discussed in Section 4. In Section 5, two experiments are given to verify the modulation explanation of CNNs. (2)

The Signal Modulation Theory

𝑚[𝑛] = 𝑥[𝑛] • 𝑐[𝑛] = 𝑥[𝑛] • cos 2𝜋𝑓 𝑐 𝑛.
If the Fourier transforms of and are denoted as and respectively, then we obtain:

𝑥[𝑛] 𝑚[𝑛] 𝑋(𝑓) 𝑀(𝑓) . (3) 𝐸𝑀(𝑓) = |𝑀(𝑓)| 2 = 1 4 |𝑋(𝑓 + 𝑓 𝑐 ) + 𝑋(𝑓 -𝑓 𝑐 )| 2
One of the most important applications of signal modulation is to achieve energy spectrum shift, that is, the energy spectrum of the modulating signal to be transmitted is shifted to a frequency band near a carrier signal, so that the process of sending or handling modulated signal will be more convenient than the original modulating signal. is moved to both sides of carrier frequency . 𝑓 𝑐

As shown in

Multi-carrier modulation and multiple-input multiple-output (MIMO) modulation

Besides single-carrier modulation shown in the above section, multi-carrier modulation is also widely used to further improve the transmission performance. As shown in Fig.

𝑚 𝑘 [𝑛] = 𝑥 𝑘 [𝑛]cos (2𝜋𝑓 𝑘 𝑛),𝑘 = 1,2,…,𝐾 𝑀 . Then, the modulated signal is given:

Serial/ Parallel

𝑥[𝑛] 𝑥 1 [n] 𝑥 2 [n] 𝑥 𝐾 𝑀 [𝑛] ∑ 𝑚 𝐾 𝑀 [n] 𝑠[n] cos 2𝜋𝑓 1 𝑛 cos 2𝜋𝑓 2 𝑛 cos 2𝜋𝑓 𝐾 𝑀 𝑛 𝑚 2 [n] 𝑚 1 [n]
. (5) 𝑠[𝑛] = ∑ 𝐾 𝑀 𝑘 = 1 𝑚 𝑘 [𝑛] = ∑ 𝐾 𝑀 𝑘 = 1 𝑥 𝑘 [𝑛]cos (2𝜋𝑓 𝑘 𝑛)
Multi-carrier modulation can be implemented in many ways. A common technical approach is frequency division multiplexing (FDM), that is, the total frequency width is greater than the sum of the frequencies of each sub-channel, while ensuring that the frequency bands of the signals transmitted in each sub-channel do not interfere with each other. That is to say: Eq. (3), the energy spectrum of Eq. ( 5) is given by ( 7)

𝐸𝑆(𝑓) = | ∑ +∞ 𝑛 = -∞ ∑ 𝐾 𝑀 𝑘 = 1 𝑥 𝑘 [𝑛]cos (2𝜋𝑓 𝑘 𝑛)𝑒 -𝑗2𝜋𝑓𝑛 | 2 = | ∑ 𝐾 𝑀 𝑘 = 1 𝑀 𝑘 (𝑓)| 2 = ∑ 𝐾 𝑀 𝑘 = 1 𝐸𝑀 𝑘 (𝑓) = ∑ 𝐾 𝑀 𝑘 = 1 1 4 |𝑋 𝑘 (𝑓 -𝑓 𝑘 ) + 𝑋 𝑘 (𝑓 + 𝑓 𝑘 )| 2 ,
where are the spectrum of modulating signals ;

is the energy spectrum of

𝑋 𝑘 (𝑓),𝑘 = 1,2,…,𝐾 𝑀 [ ] k
x n 𝐸𝑀 𝑘 (𝑓) . From Eq. ( 7), we can see that multi-carrier modulation system realizes simultaneously energy spectrum [ ] k m n shift of multiple signals, which is shown in Fig. 3.

𝑓 𝑓 𝑓 𝑓 𝑓 𝑓 𝑓 𝐸𝑋1(𝑓) 𝐸𝑋2(𝑓) 𝐸𝑋𝐾 𝑀 (𝑓) -𝑓 1 𝑓 1 -𝑓 2 -𝑓𝐾 𝑀 -𝑓𝐾 𝑀 𝑓𝐾 𝑀 𝑓 2 𝑓2 𝑓𝐾 𝑀 𝑓1 -𝑓1 -𝑓2 𝐸𝑀1(𝑓) 𝐸𝑀 2 (𝑓) 𝐸𝑀 𝐾𝑀 (𝑓) 𝐸𝑆(𝑓) … … … … Fig. 3.
The energy spectrum shift of multi-carrier modulation system.

In the actual signal transmission application, we usually combine multi-carrier technology with multiple-input multiple-output (MIMO) technology, which is a typical diversity technology used for anti-fading in communication systems. The basic principle of diversity technology is to send multiple copies carrying the same information through multiple channels. Each copy undergoes a multi-carrier modulation system on each channel. The principle block diagram of the multi-carrier MIMO system is shown in Fig. 4.

After employing spatial diversity, the input signal is divided into I M parts on the antenna array at the transmitting end and can be expressed as:

, ( 8
) 1 2 [ ] [ ] [ ] [ ] M M T I I n x n x n x n       x  
where the superscript T denotes transpose, I M represents the number of antennas at the transmitting end. Then in each channel, is transmitted through the antenna after its corresponding multi-carrier 𝑥 𝑝 [𝑛],𝑝 = 1,2,...,𝐼 𝑀 modulation. The channel response matrix can be expressed as , whose element represents the

M M O I   H  ℎ 𝑞,𝑝
response coefficient from the -th transmitting antenna to the -th receiving antenna. Note that denotes real 𝑝 𝑞  number domain. Then, the receiving vector after modulation of multi-carrier MIMO system can be expressed as:

, ( 9
) 1 2 [ ] [ ] [ ] [ ] M M T O O n r n r n r n        r Hs  
where O M represents the number of antennas at the receiving end and (10)

1 2 [ ] [ ] [ ] M M T I I s n s n s n       s  
is the modulated signal vector.

Eq. ( 9) can be further refined into: 3) and ( 7), the energy spectrum of Eq. ( 11) is given by

(11) 𝑟 𝑞 [𝑛] = ∑ 𝐼 𝑀 𝑝 = 1 ℎ 𝑞,𝑝 𝑠 𝑝 [𝑛] = ∑ 𝐼 𝑀 𝑝 = 1 ℎ 𝑞,𝑝[ ∑ 𝐾 𝑀 𝑘 = 1 𝑥 𝑝,𝑘 [𝑛]cos (2𝜋𝑓 𝑝,𝑘 𝑛) ] ,𝑞 = 1,2

The connection between CNNs and Modulation Theory

CNNs are hierarchical models whose inputs are raw data, such as RGB image, audio signal, and so on. Then CNNs stack various layers composed of a series of operations, such as convolution operation, pooling operation, and non-linear activation function, the purpose of which is to extract high-level semantic information from the input layer and abstract it layer by layer. This process of feature extraction is called "forward-propagation". Finally, the last layer of the CNNs formalizes its target task (classification, segmentation, regression, etc.) into an objective function. More generally speaking, the construction of CNNs is like a process of building blocks, using the Conv-ReLU-Pooling operator as the "basic unit" in turn to "build" on the original data and "stack" layer by layer, as shown in Fig. 5. This "basic unit" will be the focus of our research. In this section, we will bridge CNNs and modulation theory and explain the correspondence in these two frames, whose main points are briefly summarized in Table 1

Back-propagation

The back-propagation process is searching for the appropriate energy spectrum distribution which is best for classification, through gradient descent.

Energy spectrum shift

Through different modulation methods, the energy spectrum of the original signal is moved to a higher carrier frequency, which can improve the anti-interference and anti-fading ability of the system.

Building block

Convolution Activation function (ReLU)

Pooling

Conv-ReLU-Pooling

Multi-carrier MIMO amplitude modulation

Single-carrier amplitude modulation

Multi-carrier amplitude modulation

Continuous amplitude modulation

Convolution

The convolutional layer is the core layer of the CNNs, which can realize the feature extraction of the input data. The convolutional layer contains multiple convolution kernels. When the convolutional kernel works, it will regularly scan the input data, and perform matrix element multiplication and summation on the input data, thereby enhancing some features of the input data. Note that the commutativity of convolution is not very important in the practical application of CNNs, most neural network libraries regard cross-correlation function as convolution:

(13) 𝑦 𝑞 [𝑛] = ∑ 𝐼 𝐶 𝑝 = 1 𝑤 𝑞,𝑝 [𝑛]⨀𝑥 𝑝 [𝑛] = ∑ 𝐼 𝐶 𝑝 = 1 ∑ 𝐾 𝐶 𝑘 = 1 𝑤 𝑞,𝑝 [𝑘]𝑥 𝑝 [𝑛 + 𝑘],𝑞 = 1,2,…,𝑂 𝐶 ,
where is the number of input channels, is the number of output channels, denotes the -th channel

𝐼 𝐶 𝑂 𝐶 𝑥 𝑝 [𝑛] 𝑝
of the input feature map and is the -th channel of the output feature of convolution layer, 𝑦 𝑞 [𝑛] 𝑞

is the convolutional (or correlation) kernel corresponding to the , , , , [1]
[2] C T K q p q p q p q p C w w w K      w = [ ]  
-th input channel and the -th output channel, and is the convolution (or cross-correlation) operation. Let 𝑝 𝑞 ⊙ then the single-channel convolution (or correlation) can be obtained: 1,

C C I O   (14) 𝑦[𝑛] = (𝑥⨀𝑤)[𝑛] = ∑ 𝐾 𝐶 𝑘 = 1 𝑤[𝑘]𝑥[𝑛 + 𝑘],
where we discard the subscripts p and q for simplicity.

Connection between convolution and modulation

Let us first establish the relationship between the single-channel convolution as reported above and the multicarrier modulation. Comparing Eq. ( 14) with Eq. ( 5), if we replace by , by , [ ]

k x n [ ] x n k  cos (2𝜋𝑓 𝑘 𝑛) [ ] w k
in Eq. ( 5), and also set K C =K M , then we can obtain Eq. ( 14). Therefore, the single-channel convolution operation can be achieved by constructing a special multi-carrier modulation which is shown in Fig. 6. For example, if the size of the convolution kernel is K C =3, the specific calculation process of convolution is shown in the left part of Fig. 6. Correspondingly, the right part is a multi-carrier modulation system with three channels. The multiplier represents scalar multiplication of vectors, the adder represents the sum of the corresponding positions of the three vectors, and then we can get the same calculation result as the convolution. Then, we establish the relationship between the multi-channel convolution and the multi-carrier MIMO modulation, whose correspondence of parameters is shown in Table 2. Furthermore, comparing Eq. ( 13) with Eq. ( 11), if we replace by , by , and also set K C =K M , and

𝑥[1] 𝑥[2] 𝑥[3] 𝑥[4] … … 𝑥[𝑁 -1] 𝑥[𝑁] 𝑤[1] 𝑤[2] 𝑤[3] 𝑦[1] = 𝑥[1]𝑤[1] + 𝑥[2]𝑤[2] + 𝑥[3]𝑤[3] 𝑦[2] = 𝑥[2]𝑤[1] + 𝑥[3]𝑤[2] + 𝑥[4]𝑤[3] 𝑦[3] = 𝑥[3]𝑤[1] + 𝑥[4]𝑤[2] + 𝑥[5]𝑤[3] … … 𝑦[𝑁 -2] = 𝑥[𝑁 -2]𝑤[1] + 𝑥[𝑁 -1]𝑤[2] + 𝑥[𝑁]𝑤[3] Serial/ Parallel 𝑥[𝑛] 𝑥[3] 𝑥(1) ... 𝑥(𝑁 ) 𝑤[3] 𝑤[1] 𝑤[2] 𝑤[3] 𝑤[1] 𝑤[2] 𝑤[3] 𝑥[1] 𝑥[2] ... 𝑥[𝑁 -2] 𝑤[2] 𝑤[1] 𝑥 2 𝑤[2] 𝑥 3 𝑤[2] … 𝑥 𝑁𝑴 -1 𝑤[2] 𝑥 3 𝑤[3] 𝑥 4 𝑤[3] … 𝑥 𝑁𝑴 𝑤[3] 𝑥 1 𝑤[1] + 𝑥[2]𝑤[2] + 𝑥[3]𝑤[3] 𝑥 2 𝑤[1] + 𝑥[3]𝑤[2] + 𝑥[4]𝑤[3] … 𝑥 𝑁 𝑴 -𝟐 𝑤[1] +𝑥 𝑁 𝑴 -1 𝑤[2] +𝑥 𝑁 𝑴 𝑤[3] s[1] 𝑠[2] … 𝑠[𝑁 𝑴 -2] 𝑥[2] 𝑥[3] ... 𝑥[𝑁𝑴 -1] 𝑥 1 𝑤[1] 𝑥 2 𝑤[1] … 𝑥 𝑁𝑴 -2 𝑤[1] 𝑥[𝑛] 𝑥[n] 𝑥[n] 𝑚[𝑛] 𝑚[𝑛] 𝑚[𝑛] 𝑠[𝑛]
I C =I M in Eq. , [ ] p k x n [ ] p x n k  ℎ 𝑞,𝑝 cos (2𝜋𝑓 𝑝,𝑘 𝑛) , [ ] q p
w k [START_REF] Lecun | Backpropagation applied to handwritten zip code recognition[END_REF], then we can obtain Eq. ( 13). Therefore, the multi-channel convolution operation can be achieved by constructing a special multi-carrier MIMO modulation which is shown in Fig. 7.

For the commonly used two-dimensional image data in CNNs, we can construct a similar system, which will not be repeated here. At this point in the article, we can regard the convolution operation of CNNs as a special multi-carrier MIMO modulation system.

Difference between convolution and modulation

Although multi-channel convolution can be transformed into a multi-carrier MIMO modulation structure, there are still some differences between them. In signal modulation, in order to achieve a long-distance transmission of signal, we need to move the energy spectrum of low frequency signal to a position which is near the carrier energy

𝑠 [n] 𝑠 [n] 𝑠 [n] 𝑟 [n] 𝑟 [n] 𝑟 [n] ℎ , ℎ , ℎ , ℎ , ℎ , ℎ , ℎ , ℎ , ℎ , 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 1 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 2 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 𝐼 𝒉𝟏,𝟏 𝒉𝟏,𝟐 … 𝒉𝟏, 𝒉𝟐,𝟏 𝒉𝟐,𝟐 … 𝒉𝟐, … … … … 𝒉 ,𝟏 𝒉 ,𝟐 … 𝒉 , 𝑖𝑛𝑝𝑢𝑡 𝑥 [𝑛 ] 𝑥 [𝑛] 𝑥 [n] 𝒙 𝟏 … … ∑ 𝒚𝟏,𝟏[𝟏] 𝒚𝟏,𝟏[𝟐] … 𝒚𝟏,𝟏[𝑵𝑪 -𝟐] 𝒙𝟏[𝟏] 𝒙𝟏[𝟐] … 𝒙𝟏[𝑵𝑪] 𝒙𝟐[𝟏] 𝒙𝟐[𝟐] … 𝒙𝟐[𝑵𝑪] 𝒘𝟏,𝟐[𝟏] 𝒘𝟏,𝟐[𝟐] 𝒘𝟏,𝟐[𝟑] 𝑥(𝑡) 𝑤,(𝑡) 𝒙 [𝟏] 𝒙 [𝟐] … 𝒙 [𝑵𝑪] 𝑥 (𝑡) 𝒘𝟏, [𝟏] 𝒘𝟏, [𝟐] 𝒘𝟏, [𝟑]
𝑤, (𝑡) We analyze the signal energy spectrum shift of multi-channel convolution in a pretrained CNN for 1-D audio classification. Note that the CNN structure we used is Network-audio-1 which is shown in Fig. A1 in Appendix.

𝒙 𝟐 𝒙 𝑰𝑪 𝒚𝟏,𝟐[𝟏] 𝒚𝟏,𝟐[𝟐] … 𝒚𝟏,𝟐[𝑵𝑪 -𝟐] 𝒚𝟏, [𝟏] 𝒚𝟏, [𝟐] … 𝒚𝟏, [𝑵𝑪 -𝟐] 𝒚𝟏 [𝟏] 𝒚𝟏 [𝟐] … 𝒚𝟏 [𝑵𝑪 -𝟐] … 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 1 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 2 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 𝐼 𝒘 ,𝟏 [𝟏] 𝒘 ,𝟏 [𝟐] 𝒘 ,𝟏 [𝟑] ⊙ ⊙ ⊙ 𝑥[𝑛] 𝑤 , [𝑛] 𝒙 𝟏 … ∑ 𝒚 ,𝟏 [𝟏] 𝒚 ,𝟏 𝟐 … 𝒚 ,𝟏 [𝑵𝑪 -𝟐] 𝒙𝟏[𝟏] 𝒙𝟏[𝟐] … 𝒙𝟏[𝑵𝑪] 𝒙𝟐[𝟏] 𝒙𝟐[𝟐] … 𝒙𝟐[𝑵𝑪] 𝒘 ,𝟐 [𝟏] 𝒘 ,𝟐 [𝟐] 𝒘 ,𝟐 [𝟑] 𝑥[𝑛] 𝑤 , [𝑛] 𝒙 [𝟏] 𝒙 [𝟐] … 𝒙 [𝑵𝑪] 𝑥 [𝑛] 𝒘 , [𝟏] 𝒘 , [𝟐] 𝒘 , [𝟑] 𝑤 , [𝑛] 𝒙 𝟐 𝒙 𝑰𝑪 𝒚 ,𝟐 [𝟏] 𝒚 ,𝟐 [𝟐] … 𝒚 ,𝟐 [𝑵𝑪 -𝟐] 𝒚 , [𝟏] 𝒚 , [𝟐] … 𝒚 , [𝑵𝑪 -𝟐] 𝒚 [𝟏] 𝒚 [𝟐] … 𝒚 [𝑵𝑪 -𝟐] 𝒚𝟏 [𝟏] 𝒚𝟏 [𝟐] … 𝒚𝟏 [𝑵𝑴 -𝟐] 𝒚 [𝟏] 𝒚[𝟐] … 𝒚 [𝑵𝑪 -𝟐] 𝒚 [𝟏] 𝒚 [𝟐] … 𝒚 [𝑵𝑴 -𝟐]
We simply choose the second convolutional layer Conv2 in Network-audio-1 with multi-channel input and multichannel output for analysis instead of Conv1 with single-channel input and multi-channel output. The multi-channel convolution is defined in Eq. ( 13). The energy spectra of input and output signals are obtained by:

(15) 𝐸 𝑥 𝑝 (𝑓) = | ∑ +∞ 𝑛 = -∞ 𝑥 𝑝 [𝑛]𝑒 -𝑗2𝜋𝑓𝑛 | 2 ,𝑝 = 1,2,…,𝐼 𝐶 , (16) 𝐸 𝑦 𝑞 (𝑓) = | ∑ +∞ 𝑛 = -∞ 𝑦 𝑞 [𝑛]𝑒 -𝑗2𝜋𝑓𝑛 | 2 ,𝑞 = 1,2,…,𝑂 𝐶 ,
In order to analyze the general law, we randomly select 1000 audio segments in urbansound8K [START_REF] Salamon | A Dataset and Taxonomy for Urban Sound Research[END_REF], which are then fed into the pretrained CNN for audio classification. The average energy spectra of the input feature map and the output feature map of Conv2 layer are respectively given by ( 17)

𝜓 𝑥 (𝑓) = 1 𝑆 𝐶 1 𝐼 𝐶 ∑ 𝑆 𝐶 𝑖 = 1 ∑ 𝐼 𝐶 𝑝 = 1 𝐸 𝑥 𝑖,𝑝 (𝑓), ( 18
) 𝜓 𝑦 (𝑓) = 1 𝑆 𝐶 1 O𝐶 ∑ 𝑆 𝐶 𝑖 = 1 ∑ O𝐶 𝑞 = 1 𝐸 𝑦 𝑖,𝑞 (𝑓),
where , , and are explained in Table 2, and

𝑆 𝐶 𝐼 𝐶 𝑂 𝐶 (19) 𝐸 𝑥 𝑖,𝑝 (𝑓) = | ∑ +∞ 𝑛 = -∞ 𝑥 𝑖,𝑝 [𝑛]𝑒 -𝑗2𝜋𝑓𝑛 | 2 , (20) 𝐸 𝑦 𝑖,𝑞 (𝑓) = | ∑ +∞ 𝑛 = -∞ 𝑦 𝑖,𝑞 [𝑛]𝑒 -𝑗2𝜋𝑓𝑛 | 2 ,
where means the -th input channel corresponding to the i-th audio sample and means the -th convolution can be transformed into a form of signal modulation, the specific carrier will be more complicated at this time. Therefore, the shift of energy spectrum of the convolutional layer is no longer a simple movement, but a process of energy spectrum shifting and reshaping (reallocation). We called the process of moving the energy spectrum to the vicinity of a specific distribution generalized shifting. From Fig. 9 we can find that the energy spectra increase in the frequency , and correspondingly the energy spectra of other parts decrease. After the Conv2 layer, the [START_REF] Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Springenberg | Striving for simplicity: The all convolutional net[END_REF] f   energy spectra are mainly moving to the low frequency. The concentrated area of energy spectra is narrowing during convolution. The energy spectrum generalized shifts of other convolutional layers are shown in Fig. 10.

Activation function

Activation function is another important component of CNNs, and many different kinds of activation functions have been proposed, such as sigmoid, tanh, ReLU, Leaky ReLU [START_REF] Xu | Empirical evaluation of rectified activations in convolutional network[END_REF], ELU [START_REF] Rönnberg | The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances[END_REF], parametric ReLU [START_REF] Trottier | Parametric exponential linear unit for deep convolutional neural networks[END_REF], maxout [START_REF] Goodfellow | Maxout networks[C]//International conference on machine learning[END_REF],

and sine [START_REF] Parascandolo | Taming the waves: sine as activation function in deep neural networks[END_REF], among them ReLU is probably the most popular activation function used in CNNs and the definition of ReLU is: ,

[ ] (0, [ ]), 1, 2,...,

q q C a n max y n q O  
where is the -th channel of the output feature of convolution layer defined in Eq. ( 13). Eq. ( 21) can also be 𝑦 𝑞 [𝑛] 𝑞 expressed as:

(22) [ ] [ ] ( [ ]), 1, 2,..., q q q C a n y n H y n q O  
and is the Heaviside function [START_REF] Bracewell R N | The Fourier transform and its applications[END_REF] given by: 𝐻(𝑡)

(23) ( ) 1 if t>0 and ( ) 0 if t<0 H t H t  
Comparing Eq. ( 22) with Eq. ( 2), we can see that ReLU can be constructed by a single carrier DSB-SC modulation, in which is the modulating signal and is the carrier signal.

[ ]

q y n 𝐻(𝑡)
The Fourier transform of Eq. ( 22) can be expressed as follows:

(24) 𝐴 𝑞 (𝑓) = ∑ +∞ 𝑛 = -∞ 𝑎 𝑞 [𝑛]𝑒 -𝑗2𝜋𝑓𝑛 ,𝑞 = 1,2,…,𝑂 𝐶 .
Note that as the support of convolution is finite, the is well defined.

[ ]

q y n 𝐴 𝑞 (𝑓)
We also check the energy spectrum generalized shift in a pretrained 1-D CNN of Network-audio-1 whose structure is shown in Fig. A1 in Appendix. We simply choose the second ReLU layer ReLU2 for the comparison with Conv2. The energy spectrum before the ReLU2 is shown in Eq. ( 16), and the energy spectrum after the layer ReLU2 is given by:

(25) 𝐸 𝑎 𝑞 (𝑓) = |𝐴 𝑞 (𝑓)| 2 ,𝑞 = 1,2,…,𝑂 𝐶 .
Similarly, we randomly select 1000 audio segments in urbansound8K dataset [START_REF] Salamon | A Dataset and Taxonomy for Urban Sound Research[END_REF], and send them to the 1-D Network-audio-1, and obtain the input and output feature maps of ReLU2 layer. Then, the average energy spectrum of each channel and each audio segment before the ReLU2 is shown in Eq. ( 18) and the average energy spectrum of each channel and each audio segment after the ReLU2 layer is calculated as follows:

(26) 𝜓 𝑎 (𝑓) = 1 𝑆 𝐶 1 𝑂 𝐶 ∑ 𝑆 𝐶 𝑖 = 1 ∑ 𝑂 𝐶 𝑝 = 1 𝐸 𝑎 𝑖,𝑝 (𝑓),
where is the energy spectrum of the -th channel of output feature map corresponding to the -th sample after E 𝑎 𝑖,𝑝 (𝑓) 𝑞 𝑖 the ReLU2 layer. and are explained in Table 2. The plots of in Eq. ( 18) and in Eq. ( 26) using 𝑂 𝐶 𝑆𝑐 𝜓 𝑦 (𝑓) 𝜓 𝑎 (𝑓)

ReLU are shown in Fig. 11. We can find that the energy spectra decrease in the frequency , and [START_REF] Springenberg | Striving for simplicity: The all convolutional net[END_REF][START_REF] Springenberg | Striving for simplicity: The all convolutional net[END_REF] f   correspondingly the energy spectra of other parts increase. After the ReLU2 layer, the energy spectra are moving to the high frequency which is opposite to the direction of spectrum generalized shift of conv2 layer. The spectrum generalized shifts of other ReLU layers are shown in Fig. 12. Although each layer here is performing ReLU operations, the moving direction of each ReLU layer is slightly different, and we will make a more specific introduction in Section 4.

For other activation functions, we can always use Taylor's formula to convert the activation function into the following form: which can also be constructed by a single-carrier DSB-SC modulation. We will not

[ ] [ ] [ ] a n y n n  
give examples one by one and we verified the phenomenon of energy spectrum generalized shifts of some activation functions like sigmoid, tanh, sine, ELU on the Network-audio-1 whose activation functions are changed from ReLU to others. The plots of and using other activation functions instead of ReLU are shown in Fig. 13, 𝜓 𝑦 (𝑓)

𝜓 𝑎 (𝑓)

from which we can also observe the phenomenon of energy spectrum generalized shift.

Therefore, in the perspective of signal modulation, the combination of convolution and activation function is not just a superposition of two independent linear and nonlinear operators, but a continuous process of modulation in which energy spectrum of input will be shifted to the appropriate sidebands from two opposite directions. FFT+shift F(0 ) F(1 ) F(2 ) F(3 ) F( 4) F( 5) F( 6) F( 7)

Besides

F(2 ) F(3 ) F(5 ) F(4 )
Spectrum pool (a)The computational process of a typical spatial average pooling (b)The computational process of special spectrum pooling Fig. 14. The main pooling methods include spatial pooling and spectrum pooling.

Spatial pooling

We try to understand a specific pooling layer, like average pooling, whose main operation is to calculate the average of the matrices in a specific range which can be easily understood as a convolution. We map the pooling function to a convolution function (step size is decided according to the resolution required by the network), as

1 C K
shown in Fig. 15. Therefore, this type of pooling can completely be replaced by the convolution function with specific structure as follows:

, ( 27
) 1 ( [ ]) [ ] 1 C K C ap x n x n K  
where is the size of convolutional kernel decided by the parameters of pooling layer. Therefore, the process of 𝐾 𝐶 average pooling of each channel can be understood as a process of single-channel convolution. For max pooling layer, Springenberg [START_REF] Springenberg | Striving for simplicity: The all convolutional net[END_REF] found that max pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy. Therefore, we can regard spatial pooling as a special convolution and explain the pooling by the same way that we used to interpret the convolution, that is, we can regard spatial pooling as a way of modulation with one special carrier signal.

Spectral pooling

In spectral pooling, the image is truncated into a suitable size in the frequency domain. The network achieves the purpose of information compression by cropping coefficients of the low frequency of transformed feature maps.

The spectral pooling can be stated as:

(28) 𝑆𝑃(𝑥[𝑛]) = 𝑋(𝑓) ⋅ 𝐻(𝑓),
where is the frequency representation of input feature , namely, is firstly mapped to frequency

𝑋(𝑓) 𝑥[𝑛] 𝑥[𝑛]

domain by Fast Fourier transform (FFT) and then the low frequency part is shifted to the center using fftshift.

is the frequency representation of output feature. is a modified Heaviside function as follows

𝑆𝑃(𝑥[𝑛]) 𝐻(𝑓) (29)
𝐻(𝑓) = { 1, | 𝑓 -𝑁 𝑃 2 | ≤ 𝑑 2 , 0, | 𝑓 -𝑁 𝑃 2 | ≥ 𝑑 2 ,
where is the size of input images, d is the size of output features. 𝑁 𝑃 According to convolution theorem, Eq. ( 28) can be restated as:

, ( 30
) ( [ ]) [ ] [ ] sp x n x n h n  
where is the result of the inverse Fourier transform of . From Eq. ( 29), we can the spectral [ ] n

𝐻(𝑓)

pooling as a special convolution with one specific convolution kernel, that is, spectral pooling is also a special modulation according to Section 3.1.

Through the previous description, we can consider convolution, activation, and pooling as three different modulation methods. Therefore, Conv-ReLU-Pooling, the basic operators of CNN, can be regarded as a combination of three different modulation, playing the role of the basic unit of signal modulation. Then the process of stacking Conv-ReLU-Pooling operators is actually a process of continuous signal modulation.

Answers to some questions in CNNs

Given the above analysis about building connection in basic units (Conv, ReLU and Pooling) of CNNs and signal modulation, we can answer the following questions from a modulation point of view:

 Since the operator of feature extraction layer of CNNs can be interpreted as a series of signal modulation, why

CNNs need modulation?

In the signal modulation theory, we have mentioned that high frequency signals are more suitable for propagation in channels than low frequency signals, so the original low frequency modulating signal x(t) needs performing modulation and shifting the energy spectrum to the position of high frequency carrier signal c(t). Similarly, in the signal classification problem, if the original input signals are not suitable for classification, then, we also need to move the energy spectra of original signals to a specific frequency band and this specific energy spectrum distribution can achieve the optimal classification goal. Therefore, CNNs need to perform modulation and generalized shifting the original signal along the direction of this specific energy spectrum distribution to obtain a good classification performance. Notice that we will verify that there exists this optimal energy spectrum distribution, and different CNNs will modulate different signals to the similar energy spectrum distribution in Section 5.

 Why each convolution in CNN is followed by ReLU function?

In contrast to the explanation by Glorot et al. [START_REF] Glorot | Deep sparse rectifier neural networks[END_REF], we try to explain the question from a modulation point of view.

Through the experiments in Section 3.2, we notice that the direction of energy spectrum generalized shift of the activation layer is usually opposite to that of the convolution layer. In order to explore the more specific relationship between two energy spectrum generalized shifts, we introduce the energy spectrum difference of a layer of CNNs to represent the role of the layer in the energy spectrum generalized shift of the network. For example, the energy spectrum differences of convolution layer and activation layer are respectively given by: (31) 𝐶𝐷𝐼𝐹𝐹(𝑓) = 𝜓 𝑦 (𝑓) -𝜓 𝑥 (𝑓), (32) 𝐴𝐷𝐼𝐹𝐹(𝑓) = 𝜓 𝑎 (𝑓) -𝜓 𝑦 (𝑓), where , and are explained in Eqs. ( 17), ( 18) and ( 26), respectively. Fig. 16 shows the 𝜓 𝑥 𝜓 𝑦 (𝑓)

𝜓 𝑎 (𝑓) 𝐶𝐷𝐼𝐹𝐹 and of all of the convolution layers and ReLU layers of Network-audio-1. We can find that these (𝑓) 𝐴𝐷𝐼𝐹𝐹(𝑓) two opposite directions of spectrum generalized shifts are common in the network.

Furthermore, in order to analyze the role of the two totally different kinds of structures, we no longer limit to exploring the differences of each single layer and try to accumulate the energy spectrum differences of all the convolution and activation layers respectively. The accumulation of energy spectrum differences of the convolution layer and the activation layer are respectively given by: In addition, since the batch normalization [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF] is increasingly used to optimize convolution recently, we further do an experiment to verify the role of activation functions by replacing convolution with convolution_bn (convolution+batch normalization). From Fig. 18 we can find that the reverse spectrum regulation principle of activation functions (ReLU, sigmoid, sine, tanh) is also applicable.

(33) 𝐴𝐶𝐷𝐼𝐹𝐹(𝑓) = ∑ 𝑁 𝐶𝐿 𝑛 = 1 𝐶𝐷𝐼𝐹𝐹 𝑛 (𝑓), (34) 𝐴𝐴𝐷𝐼𝐹𝐹(𝑓) = ∑ 𝑁 𝐴𝐿 𝑛 = 1 𝐴𝐷𝐼𝐹𝐹 𝑛 (𝑓),

 Can another commonly used operator deconvolution be explained by signal modulation?

Up-sampling by deconvolution [START_REF] Zeiler | IEEE Computer Society Conference on computer vision and pattern recognition[END_REF] is a popular method for implementing semantic segmentation in deep learning. While the operation is called deconvolution, it is in fact a special kind of convolution. Deconvolution first enlarges the size of the input image to a certain ratio by interpolating , then rotates the convolution kernel, and 0 finally performs convolution. Dosovitskiy et al. [START_REF] Dosovitskiy | Learning to generate chairs with convolutional neural networks[END_REF] found that Unpooling+Convolution has similar effect as deconvolution. Time domain interpolation (unpooling) does not have much effect on the resolution of the image energy spectrum, but only adds some small extra high frequency components. The main energy spectrum changes in deconvolution are still happen in the convolution process. So, we can still regard deconvolution (a special convolution) as a special modulation.

 Why need back-propagation in CNNs?

In Section 3, we connect the basic units (Conv, ReLU, Pooling) of CNNs with signal modulation. We think that the forward-propagation process of CNN is a continuous modulation process, and each operator of Conv, ReLU and

Verification and Application Experiments

In this section, we give several experiments to verify the proposed modulation explanation theory, whose application examples are also shown. The experiments are implemented using PyTorch and Matlab on a PC machine, which sets up Ubuntu 16.04 operating system and has an Intel(R) Core(TM) i7-4790K CPU with speed of 4.00 GHz *8 and 32 GB RAM, and has one NVIDIA GTX1080-Ti GPU. Some of the models in this section are provided by torchvision in PyTorch, and some hyperparameters of models have been modified to meet the video memory requirements.

Verification experiment

In the proposed modulation theory, CNNs are continuous energy spectrum modulation process. The verification experiments are divided into three parts:

(1) In the forward-propagation process, what the network does is moving the energy spectrum distribution of input signals. Similar to the process of signal modulation, that the carrier signal modulates different modulating signals to the vicinity of energy spectrum of the carrier signal, the CNNs (a continuous spectrum modulation process) modulate different input signals to the similar energy spectrum distribution. Experiment will be conducted to verify whether input signals with different energy spectrum distributions will be moved to the similar distribution.

(2) During the back-propagation process, the CNNs continuously adjust the Conv-ReLU-Pooling (carrier signal parameters) to make the energy spectrum of the modulated signal change in a specific direction. In the second part, we will verify that the back-propagation process of networks of different structures (composed of basic Conv-ReLU-Pooling structure) essentially moves the energy spectrum along different routes toward the similar direction of energy spectrum distribution which is beneficial for classification.

(3) Since CNNs actually act on each individual image, we will explore how different the output spectra are if two different images are taken from one category and find the association between optimal spectrum distribution and individual classification effects.

Forward-propagation

In this part, the network we choose is a 152-layer residual network (Since we study a generalized modulation process with forward propagation, here we test on an untrained network with random initialization). Ten different categories of ImageNet [START_REF] Deng | ImageNet: A large-scale hierarchical image database[END_REF] are randomly selected and the specific categories are shown in the Table 4.

Then we explore the average energy spectrum distributions of the input images and the output features of CNNs, as shown in Fig. 20. The average energy spectra of input images and output features of each category are given:

(37) 1 1 ( , ) ( , ), S N input input i i S AE E N        (38) 1 1 ( , ) ( , )
,

S N feature feature i i S AE E N       
where is the number of samples of each category, and 𝑁 𝑠

,

I I I I I I u v N N C C j N N input input i i p i p p p u v I I E E input u v e C C                           (40) 1 2 1 2 2 2 , , 1 1 1 1 1 1 ( , ) ( , ) ( , )
,

F F F F F F u v N N C C j N N feature feature i i q i q q q u v F F E E feature u v e C C                          
where and are the number of channels of input images and output features. is the element

I C F C , ( , ) i p input u v of , which is the channel of the sample of input images. is the 1 2 , I I N N i p   input  𝑝𝑡ℎ 𝑖𝑡ℎ , ( , ) i q feature u v
element of , which is the channel of the sample of output features. For each

1 2 , F F N N i q   feature  𝑞𝑡ℎ 𝑖𝑡ℎ
category, we calculate Eqs. ( 39) and [START_REF] O'shea | An introduction to deep learning for the physical layer[END_REF] after averaging 1300 samples in ImageNet with Natural Image Statistical Toolbox [START_REF] Bainbridge | A toolbox and sample object perception data for equalization of natural images[END_REF]. The curves in the image represent 60%, 80%, and 90% of the image energy spectrum for each section from the center to the periphery. Algorithm 1 provides the pseudo-code for how we get the average energy spectrum for each section. Notice that the 80% and 90% curves of the modulated image are so close that they may be indistinguishable but the curves of input image are not so. From Table 4, we can find that CNN normalizes different categories of images with different initial energy spectrum distributions to the similar energy spectrum distribution during the process of forward-propagation. One of the most famous scale invariances of natural image statistics [START_REF] Ruderman | The statistics of natural images[END_REF], [START_REF] Srivastava | On Advances in Statistical Modeling of Natural Images[J[END_REF] is 1/ρ law, which states that the amplitude of the averaged Fourier spectrum of the ensemble of natural images obeys a distribution [START_REF] Hou | Saliency Detection: A Spectral Residual Approach[END_REF]:

𝐴𝐹(𝜌) (41) ( ) 1/ , AF   
where is the radius from the spectrum center. Therefore, the average energy spectrum obeys the 𝜌 ( ) So, if we want to better understand CNNs with modulation theory, we need to relate the spectrum of a specific image to the target. We will then study the connection between the optimal spectrum distribution and the effect of the task.

AE  distribution: , (42)
The network we choose is the 152-layer residual network (Since we study correlation between the classification results and spectrum, here we test on a trained network). We choose the specific category of "Tench" in ImageNet.

In order to compare the association between spectrum and classification results, we select 20 images with the best classification results and 20 images with the worst classification results. Notice that we consider the images with a high probability of Top-1 being "Tench" in the classification results as good classifications and vice versa as bad classifications.

We take the same study approach as in Section 5.1.1, but here we no longer do a study of the average spectrum of a category, but specifically compare each individual image (Eq. ( 40)). We compare the optimal spectrum distribution, the average spectrum distribution of the Top-20, and the spectrum distribution of the individual image in the Top-20. We compare the optimal spectrum distribution, the average spectrum distribution of the Bottom-20, and the spectrum distribution of the individual image in the Bottom-20. Table 5 shows the results. Top-20 indicates the 20 best-performing images, and Bottom-20 indicates the 20 worst-performing images. We can find that for each individual image, good classified images will be closer to this optimal spectrum distribution, and the less well classified ones will be further away.

At last, combined with our validation experiments, we give our final understanding of the CNNs as a tool to finish modulating. The CNNs are trying to do a spectrum normalization process to modulate different types of images to a similar spectrum distribution nearby, and by gradient descent methods the backpropagation process searches for an optimal spectrum distribution which is beneficial to the set task and better performed cases will be more inclined to be closer to. We have given our understanding of deep learning as a black box, how can we use modulation theory to improve deep learning? Since images with spectrum closer to the optimal spectral distribution tend to perform better for the target task, having an energy spectrum generalized shift process in advance become a natural fit in cases where we want to exploit the spectrum for the set tasks. We give some application experiments on 1-D audio classification task and 2-D image segmentation in the next section.

Application experiment

In this section, we will show the application examples of the proposed modulation explanation theory. Since we have explained that the CNNs are essentially a continuous signal modulation process, what the back-propagation of the CNNs really does is to find an energy spectrum distribution which is most conducive to classification or other tasks, and input signals with different energy spectrum distributions will be moved to this optimal spectrum distribution in the forward-propagation. Naturally, we consider whether this energy spectrum distribution can be used in some practical applications to guide the network design and to obtain an improved result in some tasks, for example, classification, segmentation, etc.

Here the energy spectrum generalized shift, that is, energy spectrum difference of CNN is given: fitting of Eq. ( 43). Finally, we try and see if can be used to improve the performance of CNNs. In the 𝐷𝐼𝐹𝐹(𝑓)

following two experiments, we shift the input spectrum to a better spectrum distribution in advance by determined carrier signals. We called the process post-shift. We will verify whether post-shift signals will get better results compared to raw data.

Input

CNN Features

Classifier

Energy spectrum of output feature Energy spectrum of input

Energy spectrum difference

Train Fig. 22. Energy spectrum difference of CNN can be obtained from the energy spectrum of input and output feature of a trained CNN.

Audio classification

In this experiment, we try to move the energy spectrum of original audios to the vicinity of the energy spectrum distribution which is conducive to classification in advance, and then to see if the energy spectrum post-shift process can obtain an improved result in audio classification as shown in Fig. 23. We first take two datasets Urbansound8k [START_REF] Salamon | A Dataset and Taxonomy for Urban Sound Research[END_REF] and Dcase2016 [START_REF] Mesaros | TUT database for acoustic scene classification and sound event detection[END_REF] as Data-audio-A and Data-audio-B respectively. Data-audio-A is used to get the energy spectrum difference in advance and Data-audio-B is the part we use to validate the results. Then we create two simple CNNs: Network-audio-2 and Network-audio-3 whose detailed convolution and pooling hyperparameters are shown in Fig. A2 in Appendix. The stochastic gradient descent algorithm and cross-entropy loss are adopted in both CNNs, and each CNN is trained with 200 epochs. As shown in Fig. 24, experiment is carried according to the following three steps and Algorithm 2 shows the pseudo-code:

(1) Data preprocessing. We firstly extract six features from these audios, that is, Short Time Fourier Transform (STFT), Mel Frequency Cepstrum Coefficient (MFCC), Chroma, Mel Spectrogram, Spectral Contrast and Tonnetz feature, and then combine these features as input.

(

Conclusion and discussion

In this paper, we explained CNNs from a new perspective based on signal modulation theory. In our explained framework, each basic operator in CNNs can be seen as the modulation of the signals. The combination of convolution and ReLU layer is thought of as a special delta modulation. The stack of basic module of Conv-ReLU-Pooling in CNNs is essentially a continuous modulation of the signal energy spectrum. What happens in forward-propagation is regarded as moving the energy spectrum of different input signals to the certain energy spectrum distribution. For the back-propagation process, it can be thought of as a searching process for an optimal energy spectrum distribution that is most conducive to classification or other tasks and the optimal carrier signals will be found by gradient descent during the process. Our experiments prove that the CNNs modulate the original signals with different energy spectrum distributions to the similar spectrum distribution to achieve better classification goal. CNNs with different structures will move the energy spectrum distribution along different paths but towards the similar end and images that are shifted closer to the optimal spectrum distribution will tend to perform better. Furthermore, shifting the input signal to the vicinity near the optimal energy spectrum distribution in advance can significantly improve the performance of classification and the shifted images can guide the semantic segmentation more effectively.

In this work, we explain what happens in CNNs by studying the spectrum distribution of features in the spatial domain.

However, in most classical CNNs, the domain of the features is continuously down-sampled and the dimensionality channel domain of the features or the width of CNN is constantly increasing. Attempts to better understand CNNs in the channel domain can be of great interest to the study of interpretability in deep learning. The next phase of our research will focus on the channel domain in more depth. We see this will be an important direction for further understanding of CNNs and needs further study.

Appendix: Network detail

We list all the networks we used throughout this article in detail. The structure of Network-audio-1 is shown in Fig. A1.

Fig. 1 ,Fig. 1 .

 11 Fig. 1. The process of double sideband suppressed carrier (DSB-SC). and denote spectrum and energy spectrum of modulated signal , 𝑋(𝑓) 𝐸𝑋(𝑓) 𝑥[𝑛] respectively; and denote spectrum and energy spectrum of modulated signal , respectively. The energy spectrum of modulating signal 𝑀(𝑓) 𝐸𝑀(𝑓) 𝑚[𝑛]

Fig. 2 .

 2 Fig. 2. General block diagram of a multi-carrier modulation. K M is the number of carrier signals.

( 6 )

 6 𝑀 𝑝 (𝑓) ⋅ 𝑀 𝑞 (𝑓) = 0,𝑝 ≠ 𝑞, where and are the spectrum of modulated signals and respectively. So similar to 𝑀 𝑝 (𝑓) 𝑀 𝑞 (𝑓)

1 |𝑋Fig. 4 .

 14 Fig.4. Signal model of multi-carrier MIMO modulation system.

Fig. 6 .

 6 Fig. 6. The relationship between the single-channel convolution and the multi-carrier modulation. The left part is the convolution operation process in the practical application of CNNs, and the right part is the corresponding multi-carrier amplitude modulation model. and represent the length of input 𝑁 𝐶 𝑁 𝑀 signal of convolution and modulation, respectively.

  spectrum. Generally speaking, these carrier signals are fixed and have been determined in advance. For example, we usually use the carrier signals shown in Fig. 8 in OFDM-MIMO which is one of the most widely used multicarrier MIMO modulation methods. In contrast to the fixed carrier signals, we have no idea about what kind of energy spectrum distribution is beneficial to classification or segmentation in CNNs and also what kind of carrier signals can improve the classification accuracy of input signals. So, the "carrier parameters" in CNNs are usually randomly initialized, and then the optimal carrier signals are found by stochastic gradient descent method.

Fig. 7 .Fig. 8 .

 78 Fig. 7. The relationship between the multi-channel convolution and the multi-carrier MIMO modulation. The left part of figure shows a convolution schematic process. The input dimension is , (𝐼 𝐶 , 𝑁 𝐶 ) the convolutional kernel size is , and the stride is . The right part of figure shows the corresponding MIMO modulation system. The number of input antennas is , the number of output (𝑂 𝐶 , 𝐼 𝐶 , 3) 1 𝐼 𝑀 antennas is , and the number of carrier signals per channel is 3. 𝑂 𝑀

  output channel corresponding to the i-th audio sample. The plots of and are shown in Fig.9. Although 𝜓 𝑥 (𝑓)𝜓 𝑦 (𝑓)

Fig. 9 .Fig. 10 .

 910 Fig. 9. The average energy spectra of x i,p [n] and y i,q [n]. and are shown in red and blue, respectively. 𝜓 𝑥 (𝑓) 𝜓 𝑦 (𝑓)

  Conv2 and ReLU2 of Network-audio-1 for 1-D audio classification, the results of 2-D image data are shown in the supplement document.

Fig. 11 .

 11 Fig. 11. The average energy spectra of y q [n] and a q [n] using ReLU function. ψ y (f) and ψ a (f) are shown in red and blue, respectively.

Fig. 12 .Fig. 13 .

 1213 Fig. 12. The energy spectrum generalized shift of all of the ReLU layers of the Network-audio-1.

Fig. 15 .

 15 Fig. 15. The process of calculating with instead of average pooling. 1 C K input 

  where and are the number of convolution layers and activation layers of the network respectively. 𝑁 𝐶𝐿 𝑁 𝐴𝐿 𝐶𝐷𝐼𝐹𝐹 𝑛 and mean the n-th and the n-th , respectively. The plots of and (𝑓)𝐴𝐷𝐼𝐹𝐹 𝑛 (𝑓) 𝐶𝐷𝐼𝐹𝐹(𝑓) 𝐴𝐷𝐼𝐹𝐹(𝑓) 𝐴𝐶𝐷𝐼𝐹𝐹(𝑓)are shown in Fig.17, from which we can clearly observe that the directions of the two operators 𝐴𝐴𝐷𝐼𝐹𝐹(𝑓) (convolution and activation) are opposite and the amplitudes are close. So, in a modulation point of view, the convolution+ReLU can be seen as a delta modulation process. In other words, the energy spectrum generalized shift process of CNNs with only convolution is given: (35) F𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) = 𝐼𝑁𝑃𝑈𝑇(𝑓) +𝐴𝐶𝐷𝐼𝐹𝐹(𝑓), where and are the energy spectrum of input and output features. The energy spectrum 𝐼𝑁𝑃𝑈𝑇(𝑓) F𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) generalized shift process of CNNs with convolution+ReLU is given: (36) 𝐹𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) = 𝐼𝑁𝑃𝑈𝑇(𝑓) +𝐴𝐶𝐷𝐼𝐹𝐹(𝑓) +𝐴𝐴𝐷𝐼𝐹𝐹(𝑓) = 𝐼𝑁𝑃𝑈𝑇(𝑓) + (1 -𝜆)𝐴𝐶𝐷𝐼𝐹𝐹(𝑓) = 𝐼𝑁𝑃𝑈𝑇(𝑓) + Δ𝐴𝐶𝐷𝐼𝐹𝐹(𝑓), where is the correlation coefficient decided by the specific convolution layer. Therefore, what CNNs with  convolution+ReLU need to learn is the delta shift . Convlolution+ReLU is a more refined energy Δ𝐴𝐶𝐷𝐼𝐹𝐹(𝑓) spectrum generalized shift than the generalized shift of single convolution. That is why CNNs with convolution+ReLU can stack more layers than CNNs only with convolution.

Fig. 16 .Fig. 17 .Fig. 18 .

 161718 Fig. 16. The and of all of the convolution layers and ReLU layers of Network-audio-1. 𝐶𝐷𝐼𝐹𝐹(𝑓) 𝐴𝐷𝐼𝐹𝐹(𝑓)

  Pooling will modulate the input signals. Then the modulated signals will be sent to the classifier of CNNs to get the classification results. Although both signal modulation and CNNs are doing the work of modulating the energy spectrum of input signals, the biggest difference between CNNs and conventional signal modulation is the existence of back-propagation process in CNNs. In signal modulation, we know that low frequency signals are not conducive to transmit, and high frequency signals are more suitable for transmission, so what we need is just a fixed high frequency carrier which can modulate low frequency original signal into high frequency modulated signal. However, in CNNs, although it is also a continuous modulation process, we only know that the original signal cannot be easily classified, and we have no idea about what kind of energy spectrum distribution of signal is easy to classify. So, CNNs need a back-propagation process to search for a suitable energy spectrum distribution. In back-propagation, the process of adjusting parameters of convolution kernels is also the process of adjusting the energy spectrum distribution of carrier signals as shown in Fig. 19.

Fig. 19 .

 19 Fig. 19. The process of back-propagation is also the process of adjusting the energy spectrum of carrier signals.

Fig. 20 .Table 4 .

 204 Fig. 20. The energy spectrum distributions of the input images and the output features of CNNs. The lower half is the forward-propagation of CNNs. The input image is regarded as the modulating signal, and the output feature as the modulated signal. In the upper part the energy spectrum distributions of the modulating signal and the modulated signal are compared.

2 (

 2 which means that the average energy spectrum of the natural images, after averaging over orientations, lies approximately on a straight line on a log-log scale. Therefore, the specific relationship of and   log ( ) AE  of five different CNNs can be obtained by linear regression, that is, . In this experiment, 200 epochs are conducted in the process of back-propagation. For each epoch, are ( , )   calculated and the plots are shown in Fig. 21. We can find that although the paths are not exactly the same, the backpropagation processes of five different CNNs are similar: the networks move the image energy spectrum towards the similar end, which is beneficial to classification. Notice that the starting points of five different CNNs are totally different mainly due to the different modulation results caused by different CNNs with totally different structures.

Fig. 21 .

 21 Fig. 21. The energy spectrum modulation process of five different CNNs (AlexNet_bn, VGG19_bn, Resnet18, Resnet152, and Resnext50). The abscissa, ordinate and vertical coordinates in the figure represent , , and , respectively. 

Table 5 .

 5 Comparison between spectrum distributions of individual image in one certain class. The second column are 20 images with the best classification results. The fourth column are 20 images with the worst classification results. Top-20 indicates the 20 best-performing images, and Bottom-20 indicates the 20 worst-performing images.

( 43 )

 43 𝐷𝐼𝐹𝐹(𝑓) = 𝐹𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) -𝐼𝑁𝑃𝑈𝑇(𝑓), where and are the energy spectrum of input and output features. As shown in Fig.22, 𝐼𝑁𝑃𝑈𝑇(𝑓) 𝐹𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) we firstly train a CNN to obtain the and . Then we can calculate the by linear 𝐹𝐸𝐴𝑇𝑈𝑅𝐸(𝑓) 𝐼𝑁𝑃𝑈𝑇(𝑓) 𝐷𝐼𝐹𝐹(𝑓)

Fig. 23 .

 23 Fig. 23. Original audios and post-shift audios are sent to the same network to compare the classification results.

)

  Fig.25, we can find that the classification accuracy is effectively improved by shifting the energy spectrum of signal to the vicinity of a certain excellent energy spectrum distribution in advance.

Fig. 24 .

 24 Fig. 24. The flow chart of 1-D audio classification experiment. An excellent energy spectrum distribution is obtained through a CNN, then the energy spectrum difference is used to shift the energy spectrum of signal to this good distribution in advance, and finally the classification results of the post-shift signal and the original signal are compared.

Fig. 25 .( 1 )( 2 )Fig. 26 .

 251226 Fig. 25. Comparison results of the classification accuracy of the original signal and post-shift signal. The blue one is the accuracy curve of post-shift audio in Data-audio-B, and the red one is that of the original audio.

Fig. A1 . 4 Fig. A2 .

 A14A2 Fig. A1. The structure of Network-audio-1. Note that this CNN is modified appropriately from VGG-11 to process 1-D audio data. The structures of Network-audio-2 and network-audio-3 are shown in Fig. A2.

Fig. A3 .

 A3 Fig. A3. The structures of Network-image-4 and Network-image-5 used in Section 5.2.2. (a) Network-image-4 is used to obtain the optimal energy spectrum difference; (b) The obtained optimal energy spectrum difference is used to optimize the Network-iamge-5.

  2.1 Energy spectrum shift of single-carrier amplitude modulation for the discrete-time signalSignal modulation is an important concept in communication theory. The information carried in the low frequency modulating signal (or baseband signal) will be greatly attenuated and distorted when this signal is 𝑥[𝑛] sent on the transmission channel directly. Therefore, it is necessary to modulate into a high frequency signal 𝑥[𝑛]

	named modulated signal	𝑚[𝑛]	that is more suitable for sending on the transmission channel. The modulated signal
	𝑚[𝑛]	is simply obtained by multiplying the modulating signal	𝑥[𝑛]	with a high frequency carrier signal	. 𝑐[𝑛]
	Through the signal modulation method, we can use the carrier signal	𝑐[𝑛]	to modulate the signal	𝑥[𝑛]	so that we
	can get a modulated signal	𝑚[𝑛]	whose characteristic parameters, such as amplitude, frequency and phase, carry
	the information of modulating signal	. Sinusoidal amplitude modulation is one of the most widely used analog 𝑥[𝑛]
	modulation methods in transmission of signals:
							𝑐[𝑛] = cos 2𝜋𝑓 𝑐 𝑛,	(1)
	where	𝑓 𝑐	is the frequency of	. Then, the modulated signal 𝑐[𝑛]	𝑚[𝑛]	is given:

  ,…,𝑂 𝑀 ,

	where	, [ ] p k x n	corresponds to the -th modulating signal of the -th copy in the -th transmit channel, 𝑘 𝑝 𝑝	cos (2𝜋
	𝑓 𝑝,𝑘 𝑛)	corresponds to the -th carrier signal of the -th transmit channel. 𝑘 𝑝
	Similar to Eqs. (

Table 1 .

 1 . The connection between CNNs and modulation theory

		Input			Conv-ReLU-Pooling Forward-propagation …… Conv-ReLU-Pooling	Target task	Classification Segmentation Regression
															……
								Fig. 5. The forward-propagation process of CNNs and Modulation Theory
												CNNs	Modulation Theory
	Objective	Minimize the cost function. If we train with mean square error, we need to minimize	Minimize the interference of the signal in the
		( ; ) M g D		1 C S  1 S i C 	( ( ) i g x		i y	2 ) ,			channel and the fading of the modulating signals.
		where	D		1 {( , ), ( , ),..., ( , 1 2 2 C S x y x y x y	S	C	)}	,	y	i	is the true label of sample	x	i	,
		g	is the classifier,	S	C	is the number of samples. By minimizing the loss function,
		the appropriate parameters (filters) are obtained, that is, pretrained model, through
		which excellent classification or recognition effect can be obtained.
	Solve	Forward-propagation
		Stack the Conv-ReLU-Pooling in different ways, modulate the input data to a
		specific energy spectrum distribution (Features = 𝑓(𝑿;𝜗)	, where	𝑿 = {𝑥 1 ,𝑥 2 ,…,𝑥 𝐼𝑆 }
		is the sample set and	𝜗	represents the parameters of filters), then send the result to
		the classifier.

Table 2 .

 2 Multi-channel convolution and its corresponding multi-carrier MIMO systemThe number of convolution kernel parameters 𝑂 𝐶 × 𝐼 𝐶 × 𝐾 𝐶The number of subcarrier parameters O M ×I M ×K M

				Multi-channel convolution	Multi-carrier MIMO modulation system
	The number of data samples S C	The number of modulating signals S M
	The number of input feature map I C	The number of transmit antennas I M
	The number of output feature map O C	The number of receive antennas O M
	The size of input feature map N C	The length of modulating signal N M
	The output feature map	) 𝑂 𝐶 × (𝑁 𝐶 -𝐾 𝐶 +1	The modulated signal	) 𝑂 𝑀 × (𝑁 𝑀 -𝐾 𝑀 +1
	𝑜𝑢𝑡𝑝𝑢𝑡 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 1
	𝑐ℎ𝑎𝑛𝑛𝑒𝑙 1	𝑥[𝑡]	⊙	𝒘𝟏,𝟏[𝟏] 𝒘𝟏,𝟏[𝟐] 𝒘𝟏,𝟏[𝟑] 𝑤,[𝑡]
	𝑐ℎ𝑎𝑛𝑛𝑒𝑙 2		⊙
		…			multi-carrier
	𝑐ℎ𝑎𝑛𝑛𝑒𝑙 𝐼		⊙		modulation
					multi-carrier
					𝑖𝑛𝑝𝑢𝑡	modulation
		…			…
	𝑜𝑢𝑡𝑝𝑢𝑡 𝑐ℎ𝑎𝑛𝑛𝑒𝑙 𝑂			multi-carrier
					modulation

Table 6 .

 6 Comparison of Mean accuracy and Mean IU verification accuracy of the original image signal and modulated image signal in Data-image-B.

		Mean accuracy	Mean IU
	Original images	0.4737	0.3621
	Post-shift images	0.5169	0.4017

ACKNOWLEDGEMENT

This work was supported in part by the National Key Research and Development Program of China (No. 2021ZD0113202), and in part by the National Natural Science Foundation of China under Grants 61876037, 62171125, 31800825, 61871117, 61871124, 61773117, 61872079, and in part by the Pre-Research Foundation of 50912040302, and in part by INSERM under the Grant the calls IAL and IRP. The authors would like to thank the Big Data Computing Center of Southeast University for providing the facility support on the numerical calculations in this paper.